Hi Carsten,<br>I have also tried scaling gromacs for a number of nodes ....but was not able to optimize it beyond 20 processor..on 20 nodes i.e. 1 processor per node..<br>I am not getting the point of optimizing PME for the number of nodes, is it like we can change the parameters for PME for MDS or using some other coulomb type for the same, please explain.<br>
and suggest me the way to do it.<br><br>With Thanks,<br>Vivek<br><br><div class="gmail_quote">2008/11/10 Carsten Kutzner <span dir="ltr"><<a href="mailto:ckutzne@gwdg.de">ckutzne@gwdg.de</a>></span><br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div style="">Hi, <div><br></div><div>most likely the Ethernet is the problem here. I compiled some numbers for the DPPC </div><div>benchmark in the paper "Speeding up parallel GROMACS on high-latency networks", </div>
<div><a href="http://www3.interscience.wiley.com/journal/114205207/abstract?CRETRY=1&SRETRY=0" target="_blank">http://www3.interscience.wiley.com/journal/114205207/abstract?CRETRY=1&SRETRY=0</a></div><div>which are for version 3.3, but PME will behave similarly. If you did not already use </div>
<div>separate PME nodes, this is worth a try, since on Ethernet the performance will drastically</div><div>depend on the number of nodes involved in the FFT. I also have a tool which finds the</div><div>optimal PME settings for a given number of nodes, by varying the number of PME nodes</div>
<div>and the fourier grid settings. I can send it to you if you want.</div><div><br></div><div>Carsten</div><div><br></div><div><br><div><div><div></div><div class="Wj3C7c"><div>On Nov 9, 2008, at 10:30 PM, Yawar JQ wrote:</div>
<br></div></div><blockquote type="cite"><div><div></div><div class="Wj3C7c"><div><font face="arial,helvetica,sans-serif">I was wondering if anyone could comment on these benchmark results for the d.dppc benchmark?</font></div>
<div><font face="arial,helvetica,sans-serif"></font> </div> <div> <table style="width: 179pt; border-collapse: collapse;" border="0" cellpadding="0" cellspacing="0" width="238"> <colgroup> <col style="width: 35pt;" width="47"> <col style="width: 77pt;" width="102"> <col style="width: 67pt;" width="89"> </colgroup><tbody> <tr style="height: 15pt;" height="20">
<td style="border: medium none rgb(212, 208, 200); background-color: transparent; width: 35pt; height: 15pt;" height="20" width="47"><font face="Calibri">Nodes</font></td> <td style="border: medium none rgb(212, 208, 200); background-color: transparent; width: 77pt;" width="102">
<font face="Calibri">Cutoff (ns/day)</font></td> <td style="border: medium none rgb(212, 208, 200); background-color: transparent; width: 67pt;" width="89"><font face="Calibri">PME (ns/day)</font></td></tr> <tr style="height: 15pt;" height="20">
<td style="border: medium none rgb(212, 208, 200); background-color: transparent; height: 15pt;" height="20"><font face="Calibri">4</font></td> <td style="border: medium none rgb(212, 208, 200); background-color: transparent;">
<font face="Calibri">1.331</font></td> <td style="border: medium none rgb(212, 208, 200); background-color: transparent;"><font face="Calibri">0.797</font></td></tr> <tr style="height: 15pt;" height="20"> <td style="border: medium none rgb(212, 208, 200); background-color: transparent; height: 15pt;" height="20">
<font face="Calibri">8</font></td> <td style="border: medium none rgb(212, 208, 200); background-color: transparent;"><font face="Calibri">2.564</font></td> <td style="border: medium none rgb(212, 208, 200); background-color: transparent;">
<font face="Calibri">1.497</font></td></tr> <tr style="height: 15pt;" height="20"> <td style="border: medium none rgb(212, 208, 200); background-color: transparent; height: 15pt;" height="20"><font face="Calibri">16</font></td>
<td style="border: medium none rgb(212, 208, 200); background-color: transparent;"><font face="Calibri">4.5</font></td> <td style="border: medium none rgb(212, 208, 200); background-color: transparent;"><font face="Calibri">1.92</font></td>
</tr> <tr style="height: 15pt;" height="20"> <td style="border: medium none rgb(212, 208, 200); background-color: transparent; height: 15pt;" height="20"><font face="Calibri">32</font></td> <td style="border: medium none rgb(212, 208, 200); background-color: transparent;">
<font face="Calibri">8.308</font></td> <td style="border: medium none rgb(212, 208, 200); background-color: transparent;"><font face="Calibri">0.575</font></td></tr> <tr style="height: 15pt;" height="20"> <td style="border: medium none rgb(212, 208, 200); background-color: transparent; height: 15pt;" height="20">
<font face="Calibri">64</font></td> <td style="border: medium none rgb(212, 208, 200); background-color: transparent;"><font face="Calibri">13.5</font></td> <td style="border: medium none rgb(212, 208, 200); background-color: transparent;">
<font face="Calibri">0.275</font></td></tr> <tr style="height: 15pt;" height="20"> <td style="border: medium none rgb(212, 208, 200); background-color: transparent; height: 15pt;" height="20"><font face="Calibri">128</font></td>
<td style="border: medium none rgb(212, 208, 200); background-color: transparent;"><font face="Calibri">20.093</font></td> <td style="border: medium none rgb(212, 208, 200); background-color: transparent;"><font face="Calibri">-</font></td>
</tr> <tr style="height: 15pt;" height="20"> <td style="border: medium none rgb(212, 208, 200); background-color: transparent; height: 15pt;" height="20"><font face="Calibri">192</font></td> <td style="border: medium none rgb(212, 208, 200); background-color: transparent;">
<font face="Calibri">21.6</font></td> <td style="border: medium none rgb(212, 208, 200); background-color: transparent;"><font face="Calibri">-</font></td></tr></tbody></table></div> <div> </div> <div><font face="arial,helvetica,sans-serif">It seems to scale relatively well up to 32-64 nodes without PME. This seems slightly better than the benchmark results for Gromacs 3 on <a href="http://www.gromacs.org/" target="_blank">www.gromacs.org</a>. </font></div>
<div><font face="arial,helvetica,sans-serif"></font> </div> <div><font face="arial,helvetica,sans-serif">Can someone comment on the magnitude of the performance hit and lack of scaling with PME is worrying me. </font></div>
<div><font face="arial,helvetica,sans-serif"></font> </div> <div><font face="arial,helvetica,sans-serif">For the PME runs, I set rlist,rvdw,rouloumb=1.2 and the rest set to the defaults. I can try it with some other settings, larger spacing for the grid, but I'm not sure how much more that would help. Is there a more standardized system I should use for testing PME scaling?</font></div>
<div><font face="arial,helvetica,sans-serif"></font> </div> <div><font face="arial,helvetica,sans-serif">This is with GNU compilers and parallelization with OpenMPI 1.2. I'm not sure what we're using for the FFTW The compute nodes are Dell m600 blades w/ 16GB of RAM and dual quad core Intel Xeon 3GHz processors. I believe it's all ethernet interconnects.</font></div>
<div><font face="arial,helvetica,sans-serif"></font> </div> <div><font face="arial,helvetica,sans-serif">Thanks,</font></div> <div><font face="arial,helvetica,sans-serif">YQ</font></div></div></div> _______________________________________________<br>
gmx-users mailing list <a href="mailto:gmx-users@gromacs.org" target="_blank">gmx-users@gromacs.org</a><br><a href="http://www.gromacs.org/mailman/listinfo/gmx-users" target="_blank">http://www.gromacs.org/mailman/listinfo/gmx-users</a><br>
Please search the archive at <a href="http://www.gromacs.org/search" target="_blank">http://www.gromacs.org/search</a> before posting!<br>Please don't post (un)subscribe requests to the list. Use the <br>www interface or send it to <a href="mailto:gmx-users-request@gromacs.org" target="_blank">gmx-users-request@gromacs.org</a>.<br>
Can't post? Read <a href="http://www.gromacs.org/mailing_lists/users.php" target="_blank">http://www.gromacs.org/mailing_lists/users.php</a></blockquote></div><br></div></div><br>_______________________________________________<br>
gmx-users mailing list <a href="mailto:gmx-users@gromacs.org">gmx-users@gromacs.org</a><br>
<a href="http://www.gromacs.org/mailman/listinfo/gmx-users" target="_blank">http://www.gromacs.org/mailman/listinfo/gmx-users</a><br>
Please search the archive at <a href="http://www.gromacs.org/search" target="_blank">http://www.gromacs.org/search</a> before posting!<br>
Please don't post (un)subscribe requests to the list. Use the<br>
www interface or send it to <a href="mailto:gmx-users-request@gromacs.org">gmx-users-request@gromacs.org</a>.<br>
Can't post? Read <a href="http://www.gromacs.org/mailing_lists/users.php" target="_blank">http://www.gromacs.org/mailing_lists/users.php</a><br></blockquote></div><br>