Apologies for reviving such an old thread. For clarifications, interlagos and bulldozer both have a modular architecture, as mentioned earlier. Each bulldozer module has 2 integer cores and one floating point unit shared between the two cores. So, although you have 64 cores (counting integer cores) reported by the os, the number of floating point units is still 32. Moreover, each FP unit can process two threads when it is possible, but since gromacs is so compute intensive I am guessing it is saturated by just one. Hence you are not observing a scale-up by moving from 32 to 64 threads.<br>
<br>Regards,<br>Manu Vajpai<br>IIT Kanpur<br><br><div class="gmail_quote">On Fri, Mar 16, 2012 at 4:24 PM, Szilárd Páll <span dir="ltr"><<a href="mailto:szilard.pall@cbr.su.se" target="_blank">szilard.pall@cbr.su.se</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi Sara,<br>
<br>
The bad performance you are seeing is most probably caused by the<br>
combination of the new AMD "Interlagos" CPUs, compiler, operating<br>
system and it is very likely the the old Gromacs version also<br>
contributes.<br>
<br>
In practice these new CPUs don't perform as well as expected, but that<br>
is partly due to compilers and operating systems not having full<br>
support for the new architecture. However, based on the quite<br>
extensive benchmarking I've done, the with such a large system should<br>
be considerably better than what your numbers show.<br>
<br>
This is what you should try:<br>
- compile Gromacs with gcc 4.6 using the "-march=bdver1" optimization flag;<br>
- have at least 3.0 or preferably newer Linux kernel;<br>
- if you're not required to use 4.0.x, use 4.5.<br>
<br>
Note that you have to be careful with drawing conclusions from<br>
benchmarking on small number of cores with large systems; you will get<br>
artifacts from caching effects.<br>
<br>
<br>
And now a bit of fairly technical explanation, for more details ask Google ;)<br>
<br>
The machine you are using has AMD Interlagos CPUs based on the<br>
Bulldozer micro-architecture. This is a new architecture, a departure<br>
from previous AMD processors and in fact quite different from most<br>
current CPUs. "Bulldozer cores" are not the traditional physical<br>
cores. In fact the hardware unit is the "module" which consists of two<br>
"half cores" (at least when it comes to floating point units). and<br>
enable a special type of multithreading called "clustered<br>
multithreading". This is slightly similar to the Intel cores with<br>
Hyper-Threading.<br>
<br>
<br>
Cheers,<br>
--<br>
Szilárd<br>
<div class="HOEnZb"><div class="h5"><br>
<br>
<br>
On Mon, Feb 20, 2012 at 5:12 PM, Sara Campos <<a href="mailto:srrcampos@gmail.com">srrcampos@gmail.com</a>> wrote:<br>
> Dear GROMACS users<br>
><br>
> My group has had access to a quad processor, 64 core machine (4 x Opteron<br>
> 6274 @ 2.2 GHz with 16 cores)<br>
> and I made some performance tests, using the following specifications:<br>
><br>
> System size: 299787 atoms<br>
> Number of MD steps: 1500<br>
> Electrostatics treatment: PME<br>
> Gromacs version: 4.0.4<br>
> MPI: LAM<br>
> Command ran: mpirun -ssi rpi tcp C mdrun_mpi ...<br>
><br>
> #CPUS Time (s) Steps/s<br>
> 64 195.000 7.69<br>
> 32 192.000 7.81<br>
> 16 275.000 5.45<br>
> 8 381.000 3.94<br>
> 4 751.000 2.00<br>
> 2 1001.000 1.50<br>
> 1 2352.000 0.64<br>
><br>
> The scaling is not good. But the weirdest is the 64 processors performing<br>
> the same as 32. I see the plots from Dr. Hess on the GROMACS 4 paper on JCTC<br>
> and I do not understand why this is happening. Can anyone help?<br>
><br>
> Thanks in advance,<br>
> Sara<br>
><br>
</div></div><div class="HOEnZb"><div class="h5">> --<br>
> gmx-users mailing list <a href="mailto:gmx-users@gromacs.org">gmx-users@gromacs.org</a><br>
> <a href="http://lists.gromacs.org/mailman/listinfo/gmx-users" target="_blank">http://lists.gromacs.org/mailman/listinfo/gmx-users</a><br>
> Please search the archive at<br>
> <a href="http://www.gromacs.org/Support/Mailing_Lists/Search" target="_blank">http://www.gromacs.org/Support/Mailing_Lists/Search</a> before posting!<br>
> Please don't post (un)subscribe requests to the list. Use the<br>
> www interface or send it to <a href="mailto:gmx-users-request@gromacs.org">gmx-users-request@gromacs.org</a>.<br>
> Can't post? Read <a href="http://www.gromacs.org/Support/Mailing_Lists" target="_blank">http://www.gromacs.org/Support/Mailing_Lists</a><br>
--<br>
gmx-users mailing list <a href="mailto:gmx-users@gromacs.org">gmx-users@gromacs.org</a><br>
<a href="http://lists.gromacs.org/mailman/listinfo/gmx-users" target="_blank">http://lists.gromacs.org/mailman/listinfo/gmx-users</a><br>
Please search the archive at <a href="http://www.gromacs.org/Support/Mailing_Lists/Search" target="_blank">http://www.gromacs.org/Support/Mailing_Lists/Search</a> before posting!<br>
Please don't post (un)subscribe requests to the list. Use the<br>
www interface or send it to <a href="mailto:gmx-users-request@gromacs.org">gmx-users-request@gromacs.org</a>.<br>
Can't post? Read <a href="http://www.gromacs.org/Support/Mailing_Lists" target="_blank">http://www.gromacs.org/Support/Mailing_Lists</a><br>
</div></div></blockquote></div><br>