Hi<div><br></div><div>Try with -dd 2 x 2 x 2 option with 8 nodes and check the performance. I too got the same problems, i could overcome with symmetrical -dd option.</div><div><br></div><div>Srinivas.<br><br><div class="gmail_quote">
On Fri, May 7, 2010 at 8:57 AM, irene farabella <span dir="ltr"><<a href="mailto:i.farabella@mail.cryst.bbk.ac.uk">i.farabella@mail.cryst.bbk.ac.uk</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
Hello!<br>
<br>
I am new to Gromacs and especially at parallel runs. I have some problem<br>
running my system using domain decomposition. I apologize but it will<br>
be a long mail...<br>
<br>
My system is made up by a membrane protein embedded is a mixed lipid<br>
bilayer (POPE/POPG).<br>
I tried to run it on 8 nodes but the simulation crashed due to "A charge<br>
group moved too far between two domain decomposition steps" and<br>
high % of load imbalance between nodes. I then tested the same run on a<br>
single node and it worked. I tried then different amount of nodes,<br>
changing DD (using mdrun -dd option) and it seems that more node I am<br>
using the less the performance in terms of ns/day although the load<br>
imbalance % is highly variable. During this test I found that the<br>
optimal nodes for my system is running it on 6 node with a DD 3:2:1 (vol<br>
0.80 imb F 1% )<br>
<br>
---- from the log file:<br>
<br>
Initializing Domain Decomposition on 6 nodes<br>
Dynamic load balancing: auto<br>
Will sort the charge groups at every domain (re)decomposition<br>
Initial maximum inter charge-group distances:<br>
two-body bonded interactions: 0.703 nm, LJ-14, atoms 21579 21587<br>
multi-body bonded interactions: 2.038 nm, Proper Dih., atoms 20434 20405<br>
Minimum cell size due to bonded interactions: 2.241 nm<br>
Maximum distance for 5 constraints, at 120 deg. angles, all-trans: 0.820 nm<br>
Estimated maximum distance required for P-LINCS: 0.820 nm<br>
Domain decomposition grid 3 x 2 x 1, separate PME nodes 0<br>
Domain decomposition nodeid 0, coordinates 0 0 0<br>
<br>
Table routines are used for coulomb: TRUE<br>
Table routines are used for vdw: FALSE<br>
Will do PME sum in reciprocal space.<br>
...<br>
<br>
Making 2D domain decomposition grid 3 x 2 x 1, home cell index 0 0 0<br>
<br>
Center of mass motion removal mode is Linear<br>
We have the following groups for center of mass motion removal:<br>
0: Protein_POPE_POPG<br>
1: SOL_NA+_CL-<br>
There are: 82711 Atoms<br>
Charge group distribution at step 2000000: 5512 5533 5668 5694 5535 5626<br>
Grid: 7 x 10 x 14 cells<br>
Initial temperature: 309.243 K<br>
--------<br>
<br>
Using this setting I finally managed to equilibrate my system by going trough a<br>
series of restrained runs.<br>
<br>
Surprisingly after 6,5 ns of non-restrained run (Step 3289500) the simulation<br>
crashes with :<br>
"Fatal error:<br>
A charge group moved too far between two domain decomposition steps<br>
This usually means that your system is not well equilibrated ".<br>
<br>
It seems strange that it crashes only at 3289500 steps of a non-restrained run.<br>
I am now running a short run starting from a short while before the<br>
crash step using a single processor and, as suspected, it is going<br>
smoothly. My guess is that something is going wrong with the domain<br>
decomposition of a such non-homogeneous system, considering that there<br>
are also charged lipids that complicate it but I have no idea how to<br>
solve/improve it.<br>
<br>
I am using gromacs-4.0.5.<br>
<br>
Any suggestion are welcome.<br>
<br>
--<br>
Irene Farabella<br>
Wellcome Trust PhD student<br>
<br>
Institute of Structural and Molecular Biology<br>
Department of of Biological Sciences<br>
Birkbeck<br>
University of London<br>
Malet Street<br>
London WC1E 7HX<br>
Telephone +44 (0)20 7631 6815<br>
<font color="#888888">--<br>
gmx-users mailing list <a href="mailto:gmx-users@gromacs.org">gmx-users@gromacs.org</a><br>
<a href="http://lists.gromacs.org/mailman/listinfo/gmx-users" target="_blank">http://lists.gromacs.org/mailman/listinfo/gmx-users</a><br>
Please search the archive at <a href="http://www.gromacs.org/search" target="_blank">http://www.gromacs.org/search</a> before posting!<br>
Please don't post (un)subscribe requests to the list. Use the<br>
www interface or send it to <a href="mailto:gmx-users-request@gromacs.org">gmx-users-request@gromacs.org</a>.<br>
Can't post? Read <a href="http://www.gromacs.org/mailing_lists/users.php" target="_blank">http://www.gromacs.org/mailing_lists/users.php</a><br>
</font></blockquote></div><br><br clear="all"><br>-- <br>*********************************************<br>J. Srinivasa Rao <br>Post-doctoral Research Associate<br>C/o Prof. Luis R Cruz Cruz<br>Computational Biophysics Group<br>
Department of Physics<br>Drexel University<br>3141 Chestnut St<br>Philadelphia, PA 19104, USA.<br>Ph: Off: 215-895-1989<br> Mob: 704-706-4191<br>Web:<a href="http://jsrao.web.officelive.com/default.aspx">http://jsrao.web.officelive.com/default.aspx</a><br>
**********************************************<br><br><br>
</div>