<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">Just out of curiosity: how long does MPI_File_sync take?<div><br></div><div>Sander<br><div><br></div><div><br><div><div>On Oct 13, 2010, at 10:07 , Roland Schulz wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><br><br><div class="gmail_quote">On Wed, Oct 13, 2010 at 3:35 AM, Erik Lindahl <span dir="ltr"><<a href="mailto:lindahl@cbr.su.se" target="_blank">lindahl@cbr.su.se</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word">Hi,<br><div><div><div>On Oct 13, 2010, at 9:23 AM, Roland Schulz wrote:</div><blockquote type="cite"><div class="gmail_quote"><div><font color="#000000"><br></font></div><div>This is not what we are doing at the moment. At the moment (flush after frame, sync after checkpoint) it is possible that the trajectory is broken. But the check-pointing append feature guarantees that it automatically fixes it. I like the approach of fast writing + automatic fix in the worst case better than having to guarantee that it is always correct from the beginning. Also it would be extremely difficult to guarantee it for all cases (e.g. for the case of a crash during writing of a frame). </div>
</div></blockquote><div><br></div></div><div>Yes, but that's a huge difference: Presently you might get broken frames if your simulation crashes. If you are on a file system that never flushes to disk with fflush() you won't get frames on the frontend, but at least they aren't broken.</div>
</div></div></blockquote><div><br></div><div>I think, it is also currently possible (but unlikely) that the trajectory appears broken. While a frame is written it is possible (I'm pretty sure I encountered that before). But I see the point that we at least want to make it as unlikely (most of the time it is not currently writing) as possible without affecting the performance.</div>
<div><br></div><div>This might actually not be a problem with MPI-IO because we buffer the whole frame in memory and then have one MPI_File_write call for the whole frame (or more precise a MPI_File_write_ordered for a couple of frame). Thus because we always write a whole frame in one go it should not be an issue. We'll test to make sure.</div>
<div>If it is still an issue we can buffer more frames to not cause a performance problem with MPI_File_sync after each write.</div><div><br></div><div>Independent of my original question and the CollectiveIO work, we might want to make sure that we guarantee to fsync every 15min, even when we don't checkpoint or only checkpoint infrequent. This might be a fix we want to add to the release branch.</div>
<div><br></div><div>Roland</div><div><br></div></div>
-- <br>gmx-developers mailing list<br><a href="mailto:gmx-developers@gromacs.org">gmx-developers@gromacs.org</a><br>http://lists.gromacs.org/mailman/listinfo/gmx-developers<br>Please don't post (un)subscribe requests to the list. Use the <br>www interface or send it to gmx-developers-request@gromacs.org.</blockquote></div><br></div></div></body></html>