<html><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">Dear mark, and Roland:<div><br></div><div>Thanks for all the suggestions.</div><div>I was confused gromacs4 paper with another one by Erik Lindahl. (<a href="http://www.ncbi.nlm.nih.gov/pubmed/19229308">http://www.ncbi.nlm.nih.gov/pubmed/19229308</a>) Since there they studied the same system (Kv1.2 voltage-gated ion channel), and they mentioned about the same speed as the benckmark, on a cray-xt4 machine. Here is also a link for the machine I'm working on: http://www.nersc.gov/nusers/systems/franklin/</div><div><br></div><div>Bin. </div><div><br></div><div><br><div><div><div>On Mar 17, 2010, at 1:39 AM, Roland Schulz wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><br><br><div class="gmail_quote">On Wed, Mar 17, 2010 at 3:55 AM, Mark Abraham <span dir="ltr"><<a href="mailto:Mark.Abraham@anu.edu.au">Mark.Abraham@anu.edu.au</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;"> <div class="im">On 17/03/2010 6:19 PM, BIN ZHANG wrote:<br> <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> Dear Mark:<br> <br> Thanks for your reply and sorry for my less info.<br> </blockquote> <br></div> I'm confused about your statement "I was trying to build gromacs 4.0 on a cray-xt4 machine, the same one as the benchmark in the gromacs4 paper" because I can't see any mention of results on such a machine in J. Chem. Theory Comput. 2008, 4, 435-447<div class="im"> <br> <br> <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> Here is the configuration line I used:<br> module load fftw/<a href="http://2.1.5.1" target="_blank">2.1.5.1</a><br> module load libxml2<br> CXX=CC CC=cc F77=ftn MPICC=cc LIBS="$XML2_LIBDIR -L/usr/X11/lib64"<br> ./configure --enable-mpi --enable-double --enable-fortran<br> --prefix=$HOME/gmx/para --disable-nice --with-fft=fftw2<br> --program-suffix=_mpi<br> </blockquote> <br></div> The use of double precision and the absence of hardware-optimized inner loops (--enable-fortran, since AFAIK there's no alternative on XT4) will slow things down compared to an Intel machine.</blockquote><div><br></div> <div>No. These are standard AMD CPUs and thus SSE2 works find.</div><div><br></div><div>I use </div><div><div>module swap PrgEnv-pgi PrgEnv-gnu</div><div>module swap gcc gcc/4.2.4</div></div><div>./configure '--enable-mpi' '--without-x' 'CC=cc' 'CPP=cpp' 'CXXCPP=cpp' 'CXX=CC' '--without-xml' 'CPPFLAGS=-I/sw/xt5/fftw/3.1.2/sles10.1_gnu4.2.4/include' 'LDFLAGS=-L/sw/xt5/fftw/3.1.2/sles10.1_gnu4.2.4/lib -lfftw3f' 'CFLAGS=-O3 -fomit-frame-pointer -finline-functions -Wall -Wno-unused -msse2 -funroll-all-loops -std=gnu99' </div> <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;"><div class="im"><br> <br> <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> Also, I searched the maillist and added ddorder in the mdrun line, but<br> that doesn't seem to help much.<br> aprun -n 128 $gmxdir/mdrun_mpi -deffnm npt_02 -g npt_02.log -ddorder<br> cartesian -npme 32<br> </blockquote> <br></div> You will need to experiment with npme and ddorder to find what's best. Also consider g_tune_pme in the git codebase.<br></blockquote><div><br></div><div>If you want to scale to larger number of CPUs you probably need 2D PME. It is available in the 2dpme git branch. (That it is in a branch means it is at the moment considered experimental - even more than mast that is)</div> <div><br></div><div>Roland</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;"><font color="#888888"> <br> Mark</font><div><div></div><div class="h5"><br> <br> <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> The system is a membrane protein and I'm using berger lipid/OPLS-AA<br> force filed.<br> <br> Thanks<br> Bin<br> <br> <br> On Mar 17, 2010, at 12:02 AM, Mark Abraham wrote:<br> <br> <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> On 17/03/2010 5:43 PM, BIN ZHANG wrote:<br> <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> Dear all:<br> <br> I was trying to build gromacs 4.0 on a cray-xt4 machine, the same one as<br> the benchmark in the gromacs4 paper (). However, my timing for a similar<br> system, ~100,000 atoms, is almost 3 times slower than in the paper. For<br> example, I only got ~25 ns/day with 128 cpus. So is there any special<br> flags, or tricks I can use during the configuration for detailed tuning?<br> </blockquote> <br> Probably. You've made it hard to help you when you haven't provided<br> (at least) your configure command, compiler and version, mdrun command<br> line and simulation system composition.<br> <br> Mark<br> --<br> gmx-users mailing list <a href="mailto:gmx-users@gromacs.org" target="_blank">gmx-users@gromacs.org</a><br> <a href="http://lists.gromacs.org/mailman/listinfo/gmx-users" target="_blank">http://lists.gromacs.org/mailman/listinfo/gmx-users</a><br> Please search the archive at <a href="http://www.gromacs.org/search" target="_blank">http://www.gromacs.org/search</a> before<br> posting!<br> Please don't post (un)subscribe requests to the list. Use the www<br> interface or send it to <a href="mailto:gmx-users-request@gromacs.org" target="_blank">gmx-users-request@gromacs.org</a>.<br> Can't post? Read <a href="http://www.gromacs.org/mailing_lists/users.php" target="_blank">http://www.gromacs.org/mailing_lists/users.php</a><br> </blockquote> <br> </blockquote> -- <br> gmx-users mailing list <a href="mailto:gmx-users@gromacs.org" target="_blank">gmx-users@gromacs.org</a><br> <a href="http://lists.gromacs.org/mailman/listinfo/gmx-users" target="_blank">http://lists.gromacs.org/mailman/listinfo/gmx-users</a><br> Please search the archive at <a href="http://www.gromacs.org/search" target="_blank">http://www.gromacs.org/search</a> before posting!<br> Please don't post (un)subscribe requests to the list. Use the www interface or send it to <a href="mailto:gmx-users-request@gromacs.org" target="_blank">gmx-users-request@gromacs.org</a>.<br> Can't post? Read <a href="http://www.gromacs.org/mailing_lists/users.php" target="_blank">http://www.gromacs.org/mailing_lists/users.php</a><br> </div></div></blockquote></div><br><br clear="all"><br>-- <br>ORNL/UT Center for Molecular Biophysics <a href="http://cmb.ornl.gov">cmb.ornl.gov</a><br>865-241-1537, ORNL PO BOX 2008 MS6309<br> -- <br>gmx-users mailing list <a href="mailto:gmx-users@gromacs.org">gmx-users@gromacs.org</a><br><a href="http://lists.gromacs.org/mailman/listinfo/gmx-users">http://lists.gromacs.org/mailman/listinfo/gmx-users</a><br>Please search the archive at http://www.gromacs.org/search before posting!<br>Please don't post (un)subscribe requests to the list. Use the <br>www interface or send it to gmx-users-request@gromacs.org.<br>Can't post? Read http://www.gromacs.org/mailing_lists/users.php</blockquote></div><br></div></div></body></html>