<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div><blockquote type="cite"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0.8ex; border-left-width: 1px; border-left-color: rgb(204, 204, 204); border-left-style: solid; padding-left: 1ex; position: static; z-index: auto; "><div style="word-wrap:break-word"><div><div><div class="im"><div><br></div></div>The difference is that an IO thread would virtually never run though; it would instantly block waiting for the filesystem, and in the mean time the real threads would get control back?</div>
</div></div></blockquote><div>Yes but you can only have one IO thread per file (otherwise the synchronization becomes quite difficult). Thus if the overhead is larger than the time between writes than your are still waiting. The time for MPI_File_sync can be * extremely* long (compared to fflush). </div>
<div><br></div></div></blockquote></div><div><br></div><div>There is already quite a bit of code dealing with files and threads. We should be add a single syncing thread without too much effort. Is there any danger of running out of space for threads? (they each have a stack, etc).</div><div><br></div><div>BTW MPI_File_sync is not a collective call, right?</div><div><br></div><div><br></div><div><blockquote type="cite"><div>Priority 1A is that we should never write "broken" trajectory frames to disk - that has caused huge amounts of grief in the past, and can be really confusing to users.</div><div><br></div><div>I think that basically leaves two long-term options:</div><div><br></div><div>1) Make sure that each frame is properly flushed/synced</div><div>2) Buffer IO and wait until the next checkpoint time before you write the frames to disk.</div><div><br></div><div>If we go with #2, there are two additional (minor?) issues: First, we need to check if checkpointing is disabled or only done every 5-10h, and in that case anyway sync frames ever ~15 minutes. Second, there could be a number of systems where we run out of memory if we buffer things. Then we need to designate a buffer amount and flush files when this is full.</div></blockquote></div><div><div><br></div></div><div>With a broken frame do you mean a frame that has been partially written up to an EOF, or do you mean a frame that is corrupted for some other reason? The first case sounds like something that we should be able to deal with..</div><div><br></div><div><br></div><div><br></div><div><br></div></body></html>