<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
<title></title>
</head>
<body bgcolor="#ffffff" text="#000000">
Hi Nikos,<br>
<br>
I experienced the same king of things with a core i7 on one node and a
corequad on an second node (Gromacs 4.0.3).<br>
Running on 8 threads (i7) or 4 cores in a single node is 30% faster
than 8 or 12 "cores" on 2 nodes. I noticed that my gigabit switch is
not limiting bandwidth using openmpi with rsh (~30 Gbps/70 max). <br>
Running on a single node cpu is 100% used by user (mdruns) while using
2 nodes each cpu is only 50% used by user, the 50% remaining being used
by system. The top command shows 4 mdrun jobs using 100% CPU.<br>
I guess system usage is for network transferts ... Using ssh, system
usage is quite the same and bandwidth is doubled.<br>
<br>
Any ideas about that system activity and how to reduce it ?<br>
<br>
Thanks<br>
<br>
<br>
<br>
Berk Hess a écrit :
<blockquote cite="mid:BLU134-W6673B3166F9FA2ECD56798EB50@phx.gbl"
type="cite">
<style>
.hmmessage P
{
margin:0px;
padding:0px
}
body.hmmessage
{
font-size: 10pt;
font-family:Verdana
}
</style>Hi,<br>
<br>
Oops, I meant 72000, which is only a factor of 10.<br>
I guess it might be faster one two nodes then, but probably not 2 times.<br>
If you use PME you can also experiment with putting all the PME nodes<br>
on one machine and the non-PME nodes on the other,<br>
probably with mdrun -ddorder pp_pme<br>
<br>
Gromacs supports near to maxint atoms.<br>
The question is much more what kind of system size you are
scientifically interested in.<br>
<br>
Ethernet will never scale very well for small numbers of atoms per core.<br>
Infiniband will scale very well.<br>
<br>
Berk<br>
<br>
<br>
<hr id="stopSpelling">Date: Wed, 18 Feb 2009 12:56:16 -0800<br>
From: <a class="moz-txt-link-abbreviated" href="mailto:lastexile7gr@yahoo.de">lastexile7gr@yahoo.de</a><br>
Subject: RE: [gmx-users] gromacs-4.0.2, parallel performance in two
quad core xeon machines<br>
To: <a class="moz-txt-link-abbreviated" href="mailto:gmx-users@gromacs.org">gmx-users@gromacs.org</a><br>
<br>
<table border="0" cellpadding="0" cellspacing="0">
<tbody>
<tr>
<td
style="font-family: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; font-size: inherit; line-height: inherit; font-size-adjust: inherit; font-stretch: inherit;"
valign="top">Hello,<br>
<br>
thank you for your answer. I just wondering though. How am I supposed
to have a system with more than 99999 atoms, while the gro file has a
fixed format giving up to 5 digits in the number of atoms? <br>
<br>
<br>
What else should I change in order to succeed better performance from
my hardware if I can succeed having a much bigger system? You say so
that ethernet has reached its limits.. <br>
<br>
I was concidering using a supercomputing center in Europe and as far as
I know they are using nodes which are using the Cell 9 core processors
technology in each node. How someone there can accomplish a better
performance using gromacs 4 using more nodes? Which might be the limit
there in such machines. <br>
<br>
Thank you once again,<br>
Nikos<br>
<br>
--- Berk Hess <i><a class="moz-txt-link-rfc2396E" href="mailto:gmx3@hotmail.com"><gmx3@hotmail.com></a></i> schrieb am <b>Mi,
18.2.2009:<br>
</b>
<blockquote style="margin-left: 5px; padding-left: 5px;"><b>Von:
Berk Hess <a class="moz-txt-link-rfc2396E" href="mailto:gmx3@hotmail.com"><gmx3@hotmail.com></a><br>
Betreff: RE: [gmx-users] gromacs-4.0.2, parallel performance in two
quad core xeon machines<br>
An: <a class="moz-txt-link-abbreviated" href="mailto:lastexile7gr@yahoo.de">lastexile7gr@yahoo.de</a><br>
Datum: Mittwoch, 18. Februar 2009, 19:16<br>
<br>
</b>
<div id="EC_yiv278737063">
<style>
.ExternalClass #EC_yiv278737063 .EC_hmmessage P
{padding:0px;}
.ExternalClass #EC_yiv278737063
{font-size:10pt;font-family:Verdana;}
</style><b> Hi,<br>
<br>
You can not scale a system of just 7200 atoms<br>
to 16 cores which are connected by ethernet.<br>
400 atoms per core is already the scaling limit of Gromacs<br>
on current hardware with the fastest available network.<br>
<br>
On ethernet a system 100 times as large might scale well to two nodes.<br>
<br>
Berk<br>
<br>
<br>
</b>
<hr id="EC_stopSpelling"><b>Date: Wed, 18 Feb 2009 09:40:28
-0800<br>
From: <a class="moz-txt-link-abbreviated" href="mailto:lastexile7gr@yahoo.de">lastexile7gr@yahoo.de</a><br>
To: <a class="moz-txt-link-abbreviated" href="mailto:gmx-users@gromacs.org">gmx-users@gromacs.org</a><br>
Subject: [gmx-users] gromacs-4.0.2, parallel performance in two quad
core xeon machines <br>
<br>
</b>
<table border="0" cellpadding="0" cellspacing="0">
<tbody>
<tr>
<td
style="font-family: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; font-size: inherit; line-height: inherit;"
valign="top">Hello,<br>
<br>
we have built a cluster with nodes that are comprised by the following:
dual core Intel(R) Xeon(R) CPU E3110 @ 3.00GHz. The memory of each node
has 16Gb of memory. The switch that we use is a dell power connect
model. Each node has a Gigabyte ethernet card..<br>
<br>
I tested the performance for a system of 7200 atoms in 4cores of one
node, in 8 cores of one node and in 16 cores of two nodes. In one node
the performance is getting better.<br>
The problem I get is that moving from one node to two, the performance
decreases dramatically (almost two days for a run that finishes in less
than 3 hours!).<br>
<br>
I have compiled gromacs with --enable-mpi option. I also have read
previous archives from Mr Kurtzner, yet from what I saw is that they
are focused on errors in gromacs 4 or on problems that previous
versions of gromacs had. I get no errors, just low performance.<br>
<br>
Is there any option that I must enable in order to succeed better
performance in more than one nodes? Or do you think according to your
experience that the switch we use might be the problem? Or maybe should
we have to activate anything from the nodes?<br>
<br>
Thank you in advance,<br>
Nikos<br>
<br>
</td>
</tr>
</tbody>
</table>
<b><br>
<br>
</b>
<hr><b>Express yourself instantly with MSN Messenger! <a
moz-do-not-send="true" rel="nofollow"
href="http://clk.atdmt.com/AVE/go/onm00200471ave/direct/01/">MSN
Messenger</a> </b></div>
</blockquote>
</td>
</tr>
</tbody>
</table>
<br>
<br>
<hr>Express yourself instantly with MSN Messenger! <a
moz-do-not-send="true"
href="http://clk.atdmt.com/AVE/go/onm00200471ave/direct/01/"
target="_new">MSN Messenger</a>
<pre wrap="">
<hr size="4" width="90%">
_______________________________________________
gmx-users mailing list <a class="moz-txt-link-abbreviated" href="mailto:gmx-users@gromacs.org">gmx-users@gromacs.org</a>
<a class="moz-txt-link-freetext" href="http://www.gromacs.org/mailman/listinfo/gmx-users">http://www.gromacs.org/mailman/listinfo/gmx-users</a>
Please search the archive at <a class="moz-txt-link-freetext" href="http://www.gromacs.org/search">http://www.gromacs.org/search</a> before posting!
Please don't post (un)subscribe requests to the list. Use the
www interface or send it to <a class="moz-txt-link-abbreviated" href="mailto:gmx-users-request@gromacs.org">gmx-users-request@gromacs.org</a>.
Can't post? Read <a class="moz-txt-link-freetext" href="http://www.gromacs.org/mailing_lists/users.php">http://www.gromacs.org/mailing_lists/users.php</a></pre>
</blockquote>
<br>
</body>
</html>