[gmx-users] GPU acceleration

Szilárd Páll pall.szilard at gmail.com
Tue Jun 9 23:43:54 CEST 2015 

No, it's not. Is this 5.0 or master branch?
--
Szilárd


On Fri, Jun 5, 2015 at 11:48 PM, Alex <nedomacho at gmail.com> wrote:
> Thanks for the info. I tried with GMX_EMULATE_GPU=1 preceding the mdrun
> line, crashed after 100 steps. Is that the expected behavior?
>
> On Fri, Jun 5, 2015 at 2:46 PM, Szilárd Páll <pall.szilard at gmail.com> wrote:
>
>> On Fri, Jun 5, 2015 at 10:39 PM, Alex <nedomacho at gmail.com> wrote:
>> > In that case, I am a bit lost, as I was sure GMXRC needed modification.
>> > :)). Since I never ran anything on GPU and have an mdrun line that looks
>> > like
>> >
>> > mdrun -nt 4 -s <tpr> -c <final structure>
>>
>> gmx mdrun -ntomp 4
>>
>> No need to do domain decomposition. Even with CPU runs it'll be faster
>> if you don't.
>>
>> > what should be added to trigger the behavior you're suggesting? I
>> > understand that it will likely blow up the system, but for the heck of
>> > it...
>>
>> those environment variables I mentioned. Plus timer resetting, e.g.
>> with -resethway if you want reliable timings.
>>
>> > I am now looking at the stats and see that (NxN QSTab + LJ) and (NxN
>> QSTab
>> > [F]) take up about 86% of all flops in my simulation. Which of those get
>> > offloaded?
>>
>> All of that will be offloaded. However, those are theoretical (useful)
>> flops, the timing table on the bottom is more relevant. Set the
>> GMX_CYCLE_SUBCOUNTER=ON breakdown of the "Force".
>>
>> However, if you just follow the previous emulate+skip non-bondeds
>> advise, you'll get a run that does the offload except that it skips
>> calculating - which is useful because you have no GPU to calculate
>> with.
>>
>> > Thank you,
>> >
>> > Alex
>> >
>> >
>> > On Fri, Jun 5, 2015 at 2:26 PM, Szilárd Páll <pall.szilard at gmail.com>
>> wrote:
>> >
>> >> Hi Alex,
>> >>
>> >> Why do you think GMXRC needs to be changed? You simply need to set
>> >> those env. vars to trigger the GPU code-path - and you will likely
>> >> have to freeze the system, otherwise it will probably not run more
>> >> than a few steps without seeing an explosion. Based on this you'll
>> >> know how fast you can run with GPU offload, but you can also
>> >> guesstimate how fast of a GPU you need (based on the PME+Bonded force
>> >> calculation time + publicly available CUDA kernel performance).
>> >>
>> >> Cheers,
>> >> --
>> >> Szilárd
>> >>
>> >>
>> >> On Fri, Jun 5, 2015 at 10:13 PM, Alex <nedomacho at gmail.com> wrote:
>> >> > Hi Szilárd,
>> >> >
>> >> > Well, yes, it is an old CPU, but I am actually quite content with the
>> >> > ns/day performance it churns out on all four cores. I'm looking at
>> >> >
>> >>
>> http://manual.gromacs.org/documentation/5.1-beta1/user-guide/environment-variables.html
>> >> > and it is my understanding that GMXRC needs to be changed to include
>> >> those
>> >> > flags, but no recompiling is required. Is that correct?
>> >> >
>> >> > Thanks,
>> >> >
>> >> > Alex
>> >> >
>> >> >
>> >> > On Fri, Jun 5, 2015 at 1:52 PM, Szilárd Páll <pall.szilard at gmail.com>
>> >> wrote:
>> >> >
>> >> >> Hi Alex,
>> >> >>
>> >> >> That's a quite old CPU so it's hard to tell how fast is it, you'll
>> >> >> have to try. There are ways to "guesstimate" how fast does your GPU
>> >> >> have to be to keep up with your CPU cores (hint search for
>> >> >> GMX_EMULATE_GPU / GMX_NO_NONBONDED on the list).
>> >> >>
>> >> >> In any case, unless your CPU is so slow that it balances well with
>> the
>> >> >> 750Ti, I would strongly recommend that you get a GTX 960 which will
>> >> >> cost just a bit more, but it will be 1.6-1.8x faster at computing
>> >> >> non-bonded interactions.
>> >> >>
>> >> >>  On Fri, Jun 5, 2015 at 9:08 PM, Alex <nedomacho at gmail.com> wrote:
>> >> >> > Hi all,
>> >> >> >
>> >> >> > We're about to start a bit of a toy project to test GPU-accelerated
>> >> GMX.
>> >> >> > What I have here is my old workstation (Dell Precision T3500:
>> >> Quad-core
>> >> >> >  Xeon W3530 @ 2.8 GHz, 12 gigs of RAM) running Linux Mint 17.1. The
>> >> plan
>> >> >> is
>> >> >> > to get something like GeForce GTX 750 (
>> >> >> >
>> >> >>
>> >>
>> http://www.amazon.com/GeForce-750Ti-GDDR5-Graphics-GTX750TI-OC-2GD5/dp/B00IB9P1KG/ref=sr_1_3?ie=UTF8&qid=1433529993&sr=8-3&keywords=Nvidia+GTX750
>> >> >> > )
>> >> >> > and maybe a new PSU.
>> >> >> >
>> >> >> > Questions:
>> >> >> >
>> >> >> > 1. With a card like this, how much acceleration (very
>> approximately)
>> >> can
>> >> >> we
>> >> >> > expect in a "standard" simulation (protein+lipid+water, PME
>> >> >> electrostatics)?
>> >> >>
>> >> >> In PME simulations you should see about 3x speedup, more if you are
>> >> >> lucky and possibly a little less if you are not so lucky with the
>> >> >> computational workload. What FF are you using?
>> >> >>
>> >> >> > 2. Should we put any extra considerations into cooling? We expect
>> >> >> week-long
>> >> >> > runs, and I am not sure these cards have this type of hardcore
>> gaming
>> >> in
>> >> >> > mind. :)
>> >> >>
>> >> >> I don't think so, but it depends how stuffed or dusty is that Dell
>> >> >> workstation. These Maxwell generation (96x) GPUs run quite cool.
>> >> >>
>> >> >> > Any additional comments are highly appreciated. Please keep in mind
>> >> that
>> >> >> > this is a toy project to utilize an old workstation.
>> >> >>
>> >> >> There is a GPU fan speed issue with desktop NVIDIA cards, the fan
>> >> >> rotation is limited to 60% and cards can overheat. You can force the
>> >> >> fan speed higher using nvidia-settings (you'll need an X server
>> >> >> running, though). For more details you can check slide 28 here:
>> >> >> https://goo.gl/7DnSri
>> >> >>
>> >> >> Cheers,
>> >> >> --
>> >> >> Szilárd
>> >> >>
>> >> >> > Thank you,
>> >> >> >
>> >> >> > Alex
>> >> >> > --
>> >> >> > Gromacs Users mailing list
>> >> >> >
>> >> >> > * Please search the archive at
>> >> >> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before
>> >> >> posting!
>> >> >> >
>> >> >> > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>> >> >> >
>> >> >> > * For (un)subscribe requests visit
>> >> >> > https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users
>> or
>> >> >> send a mail to gmx-users-request at gromacs.org.
>> >> >> --
>> >> >> Gromacs Users mailing list
>> >> >>
>> >> >> * Please search the archive at
>> >> >> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before
>> >> >> posting!
>> >> >>
>> >> >> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>> >> >>
>> >> >> * For (un)subscribe requests visit
>> >> >> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users
>> or
>> >> >> send a mail to gmx-users-request at gromacs.org.
>> >> > --
>> >> > Gromacs Users mailing list
>> >> >
>> >> > * Please search the archive at
>> >> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before
>> >> posting!
>> >> >
>> >> > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>> >> >
>> >> > * For (un)subscribe requests visit
>> >> > https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or
>> >> send a mail to gmx-users-request at gromacs.org.
>> >> --
>> >> Gromacs Users mailing list
>> >>
>> >> * Please search the archive at
>> >> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before
>> >> posting!
>> >>
>> >> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>> >>
>> >> * For (un)subscribe requests visit
>> >> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or
>> >> send a mail to gmx-users-request at gromacs.org.
>> >>
>> > --
>> > Gromacs Users mailing list
>> >
>> > * Please search the archive at
>> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before
>> posting!
>> >
>> > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>> >
>> > * For (un)subscribe requests visit
>> > https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or
>> send a mail to gmx-users-request at gromacs.org.
>> --
>> Gromacs Users mailing list
>>
>> * Please search the archive at
>> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before
>> posting!
>>
>> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>>
>> * For (un)subscribe requests visit
>> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or
>> send a mail to gmx-users-request at gromacs.org.
>>
> --
> Gromacs Users mailing list
>
> * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before posting!
>
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
> * For (un)subscribe requests visit
> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or send a mail to gmx-users-request at gromacs.org.

More information about the gromacs.org_gmx-users mailing list