Hi,<div><br></div><div>we would like to implement parallel output later this year.</div><div><br></div><div>We would suggest to do the following. Any suggestions are very welcome.</div><div><br></div><div>The idea is to write all files at the time when the checkpoint is written (e.g. every 15min). The reason is, that while the IO bandwidth is not so much a problem even with multi-million atom systems, the latency of the IO system can reduce the parallel efficiency significantly. Thus the idea to store frame in memory until the checkpoint is written and thus making writes much less frequent.</div>
<div>We don't think this will have any (mayor) disadvantages, since in case the simulation crashes we don't loose anything because we would need to redo the simulation from the last checkpoint anyhow. Also the memory usage would be very low because it would be mainly useful for high parallelization (thus small number of atoms per node). </div>
<div>This would also allow to do parallel write very easily. Without writing several frames it is not efficient to do parallel writes because of the sorting and the low parallel IO bandwidth resulting from small chunk size (splitting one frame over several nodes would make each chunk small). If N frames are stored than it is easy to sort and write those N frames on N nodes also get the higher IO bandwidth of parallel writes possible for large chunks.</div>
<div><br></div><div>Martin, may I ask why you are interested in parallel output?</div><div><br></div><div>BTW: Regarding parallel read of XTC for analysis tools. I suggest we add an XTC meta-file to solve the problem of parallel read for XTC. To be able to read frames in parallel we need to know the starting positions of the frame. Using the bisect search for XTC in parallel will probably give poor performance on most parallel IO systems (small random access IO pattern - is what parallel IO systems don't like at all). Using TRR instead for parallel analysis is also not such a good idea because even with parallel IO several analysis will be IO bound and thus we could benefit from the XTC compression. Thus an XTC file with a meta-file containing the starting positions should give the best performance. A separate meta-file instead of adding the positions to the header has the advantage that we don't change the current format and thus don't break compatibility with 3rd party softare. Having a separate meta-file has the disadvantage of the required bookkeeping to make sure that the XTC file and the metafile are up-to date to each other, but I think this shouldn't be to difficult to solve. And if a meta-file is missing or not up-to date it is possible to generate it on the fly.</div>
<div><br></div><div>Roland<br><br><div class="gmail_quote">On Tue, Jul 6, 2010 at 4:58 PM, Sander Pronk <span dir="ltr"><<a href="mailto:pronk@cbr.su.se" target="_blank">pronk@cbr.su.se</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">That's true; right now only the MPI rank 0 process writes trajectories (and with -multisim multiple processes get to be rank 0 for their communicator).<br>
Under normal circumstances, the time spent writing out trajectories should be negligible compared to the rest of the MD algorithm. For the version beyond the soon-to-be released 4.5, we're looking into parallel I/O for the processing tools, but nothing has been decided yet.<br>
<div><div></div><div><br>
On Jul 6, 2010, at 22:44 , Martin M. Ossowski wrote:<br>
<br>
> Hello,<br>
><br>
> Can anyone tell me whether one or more MPI processes are involved in writing trajectories to the disk. My feeling is that one process, say rank 0, does all the output file writing but I would like to confirm this.<br>
><br>
> -Thank you,<br>
> -Martin.<br>
><br>
> --<br>
> gmx-developers mailing list<br>
> <a href="mailto:gmx-developers@gromacs.org" target="_blank">gmx-developers@gromacs.org</a><br>
> <a href="http://lists.gromacs.org/mailman/listinfo/gmx-developers" target="_blank">http://lists.gromacs.org/mailman/listinfo/gmx-developers</a><br>
> Please don't post (un)subscribe requests to the list. Use thewww interface or send it to <a href="mailto:gmx-developers-request@gromacs.org" target="_blank">gmx-developers-request@gromacs.org</a>.<br>
<br>
--<br>
gmx-developers mailing list<br>
<a href="mailto:gmx-developers@gromacs.org" target="_blank">gmx-developers@gromacs.org</a><br>
<a href="http://lists.gromacs.org/mailman/listinfo/gmx-developers" target="_blank">http://lists.gromacs.org/mailman/listinfo/gmx-developers</a><br>
Please don't post (un)subscribe requests to the list. Use the<br>
www interface or send it to <a href="mailto:gmx-developers-request@gromacs.org" target="_blank">gmx-developers-request@gromacs.org</a>.<br>
</div></div></blockquote></div><br><br clear="all"><br>-- <br>ORNL/UT Center for Molecular Biophysics <a href="http://cmb.ornl.gov" target="_blank">cmb.ornl.gov</a><br>865-241-1537, ORNL PO BOX 2008 MS6309<br>
</div>