[gmx-users] GPU slower than I7

Szilárd Páll szilard.pall at cbr.su.se
Fri Oct 22 14:03:44 CEST 2010


Hi Renato,

First of all, what you're seeing is pretty normal, especially that you
have a CPU that is crossing the border of insane :) Why is it normal?
The PME algorithms are just simply not very well not well suited for
current GPU architectures. With an ill-suited algorithm you won't be
able to see the speedups you can often see in other application areas
- -even more so that you're comparing to Gromacs on a i7 980X. For
more info + benchmarks see the Gromacs-GPU page:
http://www.gromacs.org/gpu

However, there is one strange thing you also pointed out. The fact
that the "NODE" and "Real" time in your mdrun-gpu timing summary is
not the same, but has 3x deviation is _very_ unusual. I've ran
mdrun-gpu on quite a wide variety of hardware but I've never seen
those two counter deviate. It might be an artifact from the cycle
counters used internally that behave in an unusual way on your CPU.

One other thing I should point out is that you would be better off
using the standard mdrun which in 4.5 by default has thread-support
and therefore will run on a single cpu/node without MPI!

Cheers,
--
Szilárd



On Thu, Oct 21, 2010 at 9:18 PM, Renato Freitas <renatoffs at gmail.com> wrote:
> Hi gromacs users,
>
> I have installed the lastest version of gromacs (4.5.1) in an i7 980X
> (6 cores or 12 with HT on; 3.3 GHz) with 12GB of RAM and compiled its
> mpi version. Also I compiled the GPU-accelerated
> version of gromacs. Then I did a  2 ns simulation using a small system
> (11042 atoms)  to compare the performance of mdrun-gpu vs mdrun_mpi.
> The results that I got are bellow:
>
> ############################################
> My *.mdp is:
>
> constraints         =  all-bonds
> integrator          =  md
> dt                  =  0.002    ; ps !
> nsteps              =  1000000  ; total 2000 ps.
> nstlist             =  10
> ns_type             =  grid
> coulombtype    = PME
> rvdw                = 0.9
> rlist               = 0.9
> rcoulomb            = 0.9
> fourierspacing      = 0.10
> pme_order           = 4
> ewald_rtol          = 1e-5
> vdwtype             =  cut-off
> pbc                 =  xyz
> epsilon_rf    =  0
> comm_mode           =  linear
> nstxout             =  1000
> nstvout             =  0
> nstfout             =  0
> nstxtcout           =  1000
> nstlog              =  1000
> nstenergy           =  1000
> ; Berendsen temperature coupling is on in four groups
> tcoupl              = berendsen
> tc-grps             = system
> tau-t               = 0.1
> ref-t               = 298
> ; Pressure coupling is on
> Pcoupl = berendsen
> pcoupltype = isotropic
> tau_p = 0.5
> compressibility = 4.5e-5
> ref_p = 1.0
> ; Generate velocites is on at 298 K.
> gen_vel = no
>
> ########################
> RUNNING GROMACS ON GPU
>
> mdrun-gpu -s topol.tpr -v > & out &
>
> Here is a part of the md.log:
>
> Started mdrun on node 0 Wed Oct 20 09:52:09 2010
> .
> .
> .
>     R E A L   C Y C L E   A N D   T I M E   A C C O U N T I N G
>
>  Computing:     Nodes   Number          G-Cycles        Seconds     %
> ------------------------------------------------------------------------------------------------------
>  Write traj.    1               1021                    106.075 31.7            0.2
>  Rest                   1               64125.577               19178.6 99.8
> ------------------------------------------------------------------------------------------------------
>  Total          1               64231.652               19210.3 100.0
> ------------------------------------------------------------------------------------------------------
>
>                        NODE (s)                Real (s)                (%)
>       Time:    6381.840                19210.349               33.2
>                       1h46:21
>                        (Mnbf/s)   (MFlops)     (ns/day)        (hour/ns)
> Performance:    0.000   0.001   27.077  0.886
>
> Finished mdrun on node 0 Wed Oct 20 15:12:19 2010
>
> ########################
> RUNNING GROMACS ON MPI
>
> mpirun -np 6 mdrun_mpi -s topol.tpr -npme 3 -v > & out &
>
> Here is a part of the md.log:
>
> Started mdrun on node 0 Wed Oct 20 18:30:52 2010
>
>     R E A L   C Y C L E   A N D   T I M E   A C C O U N T I N G
>
>  Computing:             Nodes   Number  G-Cycles    Seconds             %
> --------------------------------------------------------------------------------------------------------------
>  Domain decomp. 3              100001     1452.166      434.7             0.6
>  DD comm. load          3              10001        0.745          0.2
>       0.0
>  Send X to PME         3              1000001    249.003       74.5
>          0.1
>  Comm. coord.           3              1000001   637.329        190.8
>          0.3
>  Neighbor search        3              100001     8738.669      2616.0
>         3.5
>  Force                       3              1000001   99210.202
> 29699.2        39.2
>  Wait + Comm. F       3              1000001   3361.591       1006.3         1.3
>  PME mesh               3              1000001   66189.554     19814.2
>       26.2
>  Wait + Comm. X/F    3              60294.513 8049.5          23.8
>  Wait + Recv. PME F 3              1000001    801.897        240.1           0.3
>  Write traj.                 3              1015         33.464
>  10.0             0.0
>  Update                     3              1000001    3295.820
> 986.6          1.3
>  Constraints              3              1000001     6317.568
> 1891.2          2.5
>  Comm. energies       3              100002      70.784          21.2
>           0.0
>  Rest                        3                              2314.844
>    693.0           0.9
> --------------------------------------------------------------------------------------------------------------
>  Total                        6              252968.148    75727.5
>                 100.0
> --------------------------------------------------------------------------------------------------------------
> --------------------------------------------------------------------------------------------------------------
>  PME redist. X/F        3              2000002    1945.551      582.4
>          0.8
>  PME spread/gather   3              2000002    37219.607    11141.9        14.7
>  PME 3D-FFT            3              2000002    21453.362     6422.2
>        8.5
>  PME solve               3              1000001     5551.056
> 1661.7           2.2
> --------------------------------------------------------------------------------------------------------------
>
> Parallel run - timing based on wallclock.
>
>                        NODE (s)         Real (s)                    (%)
>       Time:    12621.257       12621.257           100.0
>                       3h30:21
>                        (Mnbf/s)           (GFlops)                (ns/day)              (hour/ns)
> Performance:    388.633            28.773          13.691         1.753
> Finished mdrun on node 0 Wed Oct 20 22:01:14 2010
>
> ######################################
> Comparing the performance values for the two simulations I saw that in
> "numeric terms" the simulation using the GPU gave (for example) ~27
> ns/day, while when I used  mpi this value is aproximatelly half (13.7
> ns/day).
> However, when I compared the time that each simulation
> started/finished, the simulation using mpi tooks 211 minutes while the
> gpu simulation tooked 320 minutes to finish.
>
> My questions are:
>
> 1. Why in the performace values I got better results with the GPU?
>
> 2. Why the simulation running on GPU was 109 min. slower than on 6
> cores, since my video card is a GTX 480 with 480 gpu cores? I was
> expecting that the GPU would accelerate greatly the simulations.
>
>
> Does anyone have some idea?
>
> Thanks,
>
> Renato
> --
> gmx-users mailing list    gmx-users at gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-request at gromacs.org.
> Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>



More information about the gromacs.org_gmx-users mailing list