<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
<title></title>
</head>
<body bgcolor="#ffffff" text="#000000">
That probably depends on if you run on 10 or 100000 nodes.<br>
<br>
I just discussed the flushing with Erik.<br>
I forgot the motivation for this, but it was to have only whole frames
disks.<br>
If you don't flush, you'll often have partial frames.<br>
So the options are: or flush every frame or buffer and then flush or
fsync.<br>
For the mpi i/o this is no issue, since we buffer internally, we can
simply<br>
on fsync on write, which should happen at least when checkpointing.<br>
<br>
I guess the only remaining question is if flushing could be slow under<br>
circumstances where we would not want to use the mpi buffered i/o?<br>
<br>
Berk<br>
<br>
On 10/13/2010 11:18 AM, Sander Pronk wrote:
<blockquote cite="mid:6178ADCA-27E5-458E-9B2A-F28BF5F53627@cbr.su.se"
type="cite">Just out of curiosity: how long does MPI_File_sync take?
<div><br>
</div>
<div>Sander<br>
<div><br>
</div>
<div><br>
<div>
<div>On Oct 13, 2010, at 10:07 , Roland Schulz wrote:</div>
<br class="Apple-interchange-newline">
<blockquote type="cite"><br>
<br>
<div class="gmail_quote">On Wed, Oct 13, 2010 at 3:35 AM, Erik
Lindahl <span dir="ltr"><<a moz-do-not-send="true"
href="mailto:lindahl@cbr.su.se" target="_blank">lindahl@cbr.su.se</a>></span>
wrote:<br>
<blockquote class="gmail_quote"
style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div style="word-wrap: break-word;">Hi,<br>
<div>
<div>
<div>On Oct 13, 2010, at 9:23 AM, Roland Schulz wrote:</div>
<blockquote type="cite">
<div class="gmail_quote">
<div><font color="#000000"><br>
</font></div>
<div>This is not what we are doing at the moment. At the moment
(flush after frame, sync after checkpoint) it is possible that
the trajectory is broken. But the check-pointing append
feature guarantees that it automatically fixes it. I like the approach
of fast writing + automatic fix in the worst case better than having
to guarantee that it is always correct from the beginning. Also it
would be extremely difficult to guarantee it for all cases (e.g. for
the case of a crash during writing of a frame). </div>
</div>
</blockquote>
<div><br>
</div>
</div>
<div>Yes, but that's a huge difference: Presently you might get
broken frames if your simulation crashes. If you are on a file system
that never flushes to disk with fflush() you won't get frames on the
frontend, but at least they aren't broken.</div>
</div>
</div>
</blockquote>
<div><br>
</div>
<div>I think, it is also currently possible (but unlikely) that the
trajectory appears broken. While a frame is written it is possible (I'm
pretty sure I encountered that before). But I see the point that we at
least want to make it as unlikely (most of the time it is not currently
writing) as possible without affecting the performance.</div>
<div><br>
</div>
<div>This might actually not be a problem with MPI-IO because we
buffer the whole frame in memory and then have one MPI_File_write call
for the whole frame (or more precise a MPI_File_write_ordered for a
couple of frame). Thus because we always write a whole frame in one go
it should not be an issue. We'll test to make sure.</div>
<div>If it is still an issue we can buffer more frames to not cause
a performance problem with MPI_File_sync after each write.</div>
<div><br>
</div>
<div>Independent of my original question and the CollectiveIO work,
we might want to make sure that we guarantee to fsync every 15min, even
when we don't checkpoint or only checkpoint infrequent. This might be a
fix we want to add to the release branch.</div>
<div><br>
</div>
<div>Roland</div>
<div><br>
</div>
</div>
-- <br>
gmx-developers mailing list<br>
<a moz-do-not-send="true" href="mailto:gmx-developers@gromacs.org">gmx-developers@gromacs.org</a><br>
<a class="moz-txt-link-freetext" href="http://lists.gromacs.org/mailman/listinfo/gmx-developers">http://lists.gromacs.org/mailman/listinfo/gmx-developers</a><br>
Please don't post (un)subscribe requests to the list. Use the <br>
www interface or send it to <a class="moz-txt-link-abbreviated" href="mailto:gmx-developers-request@gromacs.org">gmx-developers-request@gromacs.org</a>.</blockquote>
</div>
<br>
</div>
</div>
</blockquote>
<br>
</body>
</html>