<br><br><div class="gmail_quote">On Thu, Apr 28, 2011 at 10:05 AM, Bruno Monnet <span dir="ltr"><<a href="mailto:bruno.monnet@hp.com">bruno.monnet@hp.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<div bgcolor="#ffffff" text="#000000">
<font face="Helvetica, Arial, sans-serif">Hi,<br>
</font><br>
I'm not really a Gromacs user, but I'm currently benchmarking
Gromacs 4.5.4 on a large cluster. It seems that my communication
(PME) is really high and gromacs keeps complaining for more PME
nodes :<br>
<br>
<blockquote><tt> Average load imbalance: 113.6 %</tt><br>
<tt> Part of the total run time spent waiting due to load
imbalance: 3.3 %</tt><br>
<tt> Steps where the load balancing was limited by -rdd, -rcon
and/or -dds: X 9 % Y 9 % Z 9 %</tt><br>
<tt> Average PME mesh/force load: 3.288</tt><br>
<tt> Part of the total run time spent waiting due to PP/PME
imbalance: 32.6 %</tt><br>
<br>
<tt>NOTE: 32.6 % performance was lost because the PME nodes</tt><br>
<tt> had more work to do than the PP nodes.</tt><br>
<tt> You might want to increase the number of PME nodes</tt><br>
<tt> or increase the cut-off and the grid spacing.</tt><br>
</blockquote>
<br>
I can't modify the original dataset as I only have the TPR file. I
switched from dlb yes -> dlb auto since it seems to have trouble
with more than 6000 / 8000 cores.<br></div></blockquote><div>You can set the number of PME nodes with -npme. All other nodes are used for the particle-particle computations (PP).There is also a tool called g_tune_pme which optimizes it automatically for you.</div>
<div><br></div><div>On that many cores you might see a significant speed-up by using the prerelease version of GROMACS 4.6. You can obtain that from git using the branch "threading". It uses threading for the PME nodes. The number of threads used by the PME nodes is set with the environment variable GMX_PME_NTHREADS. The total number of cores should be equal to the number of PP nodes + GMX_PME_NTHREADS * number of PP nodes. Let me know if you try it - I would be interested in feedback.</div>
<div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;"><div bgcolor="#ffffff" text="#000000">
I tried to add " -gcom " parameter. This speedup the computation.
This parameter is not really explained in the Gromacs documentation.
Could you give me some advice on how I could use it ?<br></div></blockquote><div>It defines how often e.g. the total energy is computed which is important for pressure and temperature coupling. As long as you don't set it to higher than 10 it shouldn't affect the accuracy significantly. But it does effect it minimally thus you should document that in your results.</div>
<div><br></div><div>Roland</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;"><div bgcolor="#ffffff" text="#000000">
<br>
Best regards,<br>
Bruno Monnet<br>
<br>
<div>
</div>
</div>
<br>--<br>
gmx-users mailing list <a href="mailto:gmx-users@gromacs.org">gmx-users@gromacs.org</a><br>
<a href="http://lists.gromacs.org/mailman/listinfo/gmx-users" target="_blank">http://lists.gromacs.org/mailman/listinfo/gmx-users</a><br>
Please search the archive at <a href="http://www.gromacs.org/Support/Mailing_Lists/Search" target="_blank">http://www.gromacs.org/Support/Mailing_Lists/Search</a> before posting!<br>
Please don't post (un)subscribe requests to the list. Use the<br>
www interface or send it to <a href="mailto:gmx-users-request@gromacs.org">gmx-users-request@gromacs.org</a>.<br>
Can't post? Read <a href="http://www.gromacs.org/Support/Mailing_Lists" target="_blank">http://www.gromacs.org/Support/Mailing_Lists</a><br></blockquote></div><br><br clear="all"><br>-- <br>ORNL/UT Center for Molecular Biophysics <a href="http://cmb.ornl.gov">cmb.ornl.gov</a><br>
865-241-1537, ORNL PO BOX 2008 MS6309<br>