<div dir="ltr">Hi,<div><br></div><div>Just to follow up, I can explain the logic behind this:</div><div><br></div><div>It's not merely a matter of convenience (or history) for picking it at runtime, but simply that some algorithms will have different efficiencies depending on the vector length.</div><div><br></div><div>For instance, with a SIMD vector length of 4, it might work well to organize interactions into 4x4 tiles. Some of those interactions might fall outside the cutoff, but since the cluster is relatively small the wasted fraction is limited.</div><div>If the SIMD vector length is suddenly 16 instead, it would be trivial to just use exactly the same code, which would then mean the tiles are 16x16 interactions. However, with such gigantic clusters we would now waste a *much* larger fraction of the calculations.</div><div><br></div><div>Instead, we adapt to the SIMD with and use a different algorithm where we e.g. pack each 4x1 row into 25% of a longer SIMD register so we use the throughput of wider SIMD without wasting calculations due to more zero-interactions, but this can require us to use e.g. a different neighborlist generation setup.</div><div><br></div><div>Presently we detect this at compile time. It's theoretically possible to do at runtime instead, but a bit of work. However, there will always be algorithms where knowing the SIMD width in the hardware will enable you to write a more efficient implementation rather than keeping it opaque.</div><div><br></div><div>Cheers,</div><div><br></div><div>Erik</div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Oct 27, 2020 at 1:32 PM <<a href="mailto:gilles@rist.or.jp">gilles@rist.or.jp</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex">Guido,<br>
<br>
Strictly speaking, SVE gives developers and compilers the opportunity to <br>
build/generate vector length agnostic (VLA) binaries/libraries<br>
whose performance almost as good as their fixed vector length <br>
counterpart. Such VLA programs can run (correctly) on any arm+sve <br>
processors,<br>
regardless the vector length that is available at runtime.<br>
<br>
Gromacs is highly optimized and assumes the vector length is known at <br>
cmake time, and this is assumed in several places of the code base.<br>
So the most straightforward and efficient way of supporting SVE is to <br>
fix the SVE vector length at cmake time.<br>
<br>
FWIW, I made a proof-of-concept of non bonded VLA kernels, and <br>
performance is not even close to current (fixed vector length)<br>
non bonded kernels.<br>
<br>
Cheers,<br>
<br>
Gilles<br>
<br>
----- Original Message -----<br>
> Hi Gilles,<br>
> <br>
> Thanks a lot, I tested again and it is working with ArmIE for a length <br>
of 512. I will close the Issue in GitLab.<br>
> <br>
> SVE allows to set the vector length at run time, is there a particular <br>
reason for doing this at compile time?<br>
> <br>
> Cheers, Guido. <br>
> <br>
> -----Original Message-----<br>
> From: <a href="mailto:gromacs.org_gmx-developers-bounces@maillist.sys.kth.se" target="_blank">gromacs.org_gmx-developers-bounces@maillist.sys.kth.se</a> [mailto:<br>
<a href="mailto:gromacs.org_gmx-developers-bounces@maillist.sys.kth.se" target="_blank">gromacs.org_gmx-developers-bounces@maillist.sys.kth.se</a>] On Behalf Of <br>
Gilles Gouaillardet<br>
> Sent: Tuesday, October 27, 2020 10:05 AM<br>
> To: <a href="mailto:gmx-developers@gromacs.org" target="_blank">gmx-developers@gromacs.org</a><br>
> Subject: Re: [gmx-developers] Segfault in 2021-beta1 enabling SVE and <br>
ArmIE emulator<br>
> <br>
> Guido,<br>
> <br>
> <br>
> The SVE vector length must be fixed at cmake time, and the default <br>
value is 512.<br>
> <br>
> In your case, you have to pass<br>
> <br>
> -DGMX_SIMD_ARM_SVE_LENGTH=256<br>
> <br>
> to your cmake command line.<br>
> <br>
> <br>
> Cheers,<br>
> <br>
> <br>
> Gilles<br>
> <br>
> On 10/27/2020 5:58 PM, Guido Giuntoli wrote:<br>
> ><br>
> > Hi,<br>
> ><br>
> > I have run the nonbonded-benchmark using the ArmIE emulator.<br>
> ><br>
> > I compiled gmx_d (DOUBLE FP) with GCC 10.2 for SVE by using:<br>
> ><br>
> > *CC=gcc-10 CXX=g++-10 \*<br>
> ><br>
> > *cmake .. \*<br>
> ><br>
> > * -DCMAKE_BUILD_TYPE=Release \*<br>
> ><br>
> > *-DCMAKE_C_FLAGS="-march=armv8-a+sve" \*<br>
> ><br>
> > *-DCMAKE_CXX_FLAGS="-march=armv8-a+sve" \*<br>
> ><br>
> > * -DGMX_MPI=off \*<br>
> ><br>
> > * -DGMX_GPU=off \*<br>
> ><br>
> > * -DGMX_USE_OPENCL=off \*<br>
> ><br>
> > * -DGMX_BUILD_MDRUN_ONLY=off \*<br>
> ><br>
> > * -DGMX_DOUBLE=on \*<br>
> ><br>
> > * -DGMX_SIMD=ARM_SVE \*<br>
> ><br>
> > * -DBUILD_SHARED_LIBS=off \*<br>
> ><br>
> > * -DGMX_FFT_LIBRARY=fftw3 \*<br>
> ><br>
> > * -DGMX_INSTALL_NBLIB_API=OFF \*<br>
> ><br>
> > * -DGMXAPI=OFF \*<br>
> ><br>
> > * | tee cmake.log*<br>
> ><br>
> > I ran the benchmark with:<br>
> ><br>
> > *armie -msve-vector-bits=256 \*<br>
> ><br>
> > * gmx_d nonbonded-benchmark \*<br>
> ><br>
> > * -size 1 \*<br>
> ><br>
> > * -nt 1 \*<br>
> ><br>
> > * -simd 4xm \*<br>
> ><br>
> > * -coulomb ewald \*<br>
> ><br>
> > * -notable \*<br>
> ><br>
> > * -combrule geometric \*<br>
> ><br>
> > * -nohalflj \*<br>
> ><br>
> > * -noenergy \*<br>
> ><br>
> > * -noall \*<br>
> ><br>
> > * -cutoff 1 \*<br>
> ><br>
> > * -iter 5 \*<br>
> ><br>
> > * -warmup 0 \*<br>
> ><br>
> > * -nocycles*<br>
> ><br>
> > I get:<br>
> ><br>
> > *armie's SVESigHandler caught signal SIGSEGV from the process being <br>
> > emulated, passing it to the default signal handler.*<br>
> ><br>
> > *Guest process terminated by signal: 15152 Segmentation fault /home/.<br>
> > /codes/gromacs/gromacs-2021-beta1/build-sve/bin/gmx_d*<br>
> ><br>
> > According to the last meeting, I am eager to know if this branch has <br>
> > passed the regression tests using SVE. BTW, are you using SVE-<br>
enabled <br>
> > hardware or emulation for your development/tests?<br>
> ><br>
> > *Best regards | Mit freundlichen Grüßen*<br>
> ><br>
> > **<br>
> ><br>
> > *Guido Giuntoli***<br>
> ><br>
> > **<br>
> ><br>
> > HUAWEI TECHNOLOGIES Duesseldorf GmbH<br>
> > Hansaallee 205, 40549 Dusseldorf, Germany, *<a href="http://www.huawei.com" rel="noreferrer" target="_blank">www.huawei.com</a>* <br>
> > <<a href="http://www.huawei.com/" rel="noreferrer" target="_blank">http://www.huawei.com/</a>> Registered Office: Düsseldorf, Register <br>
Court <br>
> > Düsseldorf, HRB 56063, Managing Director: Li Peng, Li Jian, Shi <br>
> > Yanli**<br>
> ><br>
> > Sitz der Gesellschaft: Düsseldorf, Amtsgericht Düsseldorf, HRB 56063,<br>
> > Geschäftsführer: Li Peng, Li Jian, Shi Yanli<br>
> ><br>
> > *-------------------------------------------------------------------<br>
--<br>
> > --------------------------*<br>
> ><br>
> > *This e-mail and its attachments contain confidential information <br>
from <br>
> > HUAWEI, which is intended only for the person or entity whose <br>
address <br>
> > is listed above. Any use of the information contained herein in any <br>
> > way (including, but not limited to, total or partial disclosure, <br>
> > reproduction, or dissemination) by persons other than the intended<br>
> > recipient(s) is prohibited. If you receive this e-mail in error, <br>
> > please notify the sender by phone or email immediately and delete <br>
> > it!***<br>
> ><br>
> ><br>
> --<br>
> Gromacs Developers mailing list<br>
> <br>
> * Please search the archive at <a href="http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List" rel="noreferrer" target="_blank">http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List</a><br>
before posting!<br>
> <br>
> * Can't post? Read <a href="http://www.gromacs.org/Support/Mailing_Lists" rel="noreferrer" target="_blank">http://www.gromacs.org/Support/Mailing_Lists</a><br>
> <br>
> * For (un)subscribe requests visit<br>
> <a href="https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers" rel="noreferrer" target="_blank">https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers</a><br>
or send a mail to <a href="mailto:gmx-developers-request@gromacs.org" target="_blank">gmx-developers-request@gromacs.org</a>.<br>
> -- <br>
> Gromacs Developers mailing list<br>
> <br>
> * Please search the archive at <a href="http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List" rel="noreferrer" target="_blank">http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List</a><br>
before posting!<br>
> <br>
> * Can't post? Read <a href="http://www.gromacs.org/Support/Mailing_Lists" rel="noreferrer" target="_blank">http://www.gromacs.org/Support/Mailing_Lists</a><br>
> <br>
> * For (un)subscribe requests visit<br>
> <a href="https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers" rel="noreferrer" target="_blank">https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers</a><br>
or send a mail to <a href="mailto:gmx-developers-request@gromacs.org" target="_blank">gmx-developers-request@gromacs.org</a>.<br>
> <br>
-- <br>
Gromacs Developers mailing list<br>
<br>
* Please search the archive at <a href="http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List" rel="noreferrer" target="_blank">http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List</a> before posting!<br>
<br>
* Can't post? Read <a href="http://www.gromacs.org/Support/Mailing_Lists" rel="noreferrer" target="_blank">http://www.gromacs.org/Support/Mailing_Lists</a><br>
<br>
* For (un)subscribe requests visit<br>
<a href="https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers" rel="noreferrer" target="_blank">https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers</a> or send a mail to <a href="mailto:gmx-developers-request@gromacs.org" target="_blank">gmx-developers-request@gromacs.org</a>.<br>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div>Erik Lindahl <<a href="mailto:erik.lindahl@dbb.su.se" target="_blank">erik.lindahl@dbb.su.se</a>></div><div>Professor of Biophysics, Dept. Biochemistry & Biophysics, Stockholm University</div><div>Science for Life Laboratory, Box 1031, 17121 Solna, Sweden</div></div></div></div></div></div></div></div></div>