<br><br><div class="gmail_quote">On Tue, Apr 17, 2012 at 2:44 AM, Erik Lindahl <span dir="ltr"><<a href="mailto:lindahl@cbr.su.se">lindahl@cbr.su.se</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="im"><br>
On Apr 16, 2012, at 10:08 PM, Roland Schulz wrote:<br>
><br>
> Obviously, by requiring that an external library works everywhere, we decide that no external library can ever be used (because it is impossible to test against all platforms/compilers) :-). Also I don't think it makes any sense to choose portability over every other important metric (maintainability, features, ...), no matter how small the portability difference and how huge the other advantages. Additionally. I think it should be up to the developers who want a certain portability to put in the effort to get it (as it is with other features).<br>
<br>
</div>Unilke other features (such as a better RNG or faster parallel IO), portability has to be a feature of every single line of code in Gromacs. It only takes 30 seconds for anybody to destroy it, while it can be months of time to fix it if it is necessary to reimplement code. You can't "put in" portability separately from the rest of the code.<br>
</blockquote><div>E.g. for parallelization the issue is very similar as it is for portability. Supporting domain decomposition makes it more difficult for everyone and everyone has to make sure that they don't brake it. And it is only included because it essential to Gromacs and used by almost everyone.</div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="im">
> I think we should have a list of supported platforms/compilers. As a developer it should be my duty to make sure that any code changes work on any of those (ideally Jenkins should do it for me) and I should be responsible to fix any bugs on any supported platform in my code. On the other hand, bugs on any non supported platform should be the responsibility of the person who wants it.<br>
<br>
</div>We already have this list: An ISO C (soon C++) compiler. If your code does not compile on a standards-compliant compiler, the code is buggy.<br></blockquote><div>This is only an idea but not a practical rule. It is impossible to write any OS depending code (e.g. file operations) with that requirement. E.g. futil.c only works if the OS is either POSIX or Windows. And also this rule is unclear about how it is implemented/tested. Is it enough to show that a strict inheritance to the standard is followed (e.g. gcc -pendantic) or does it require to test all possible compilers (impossible). </div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="im"><br>
> If we would require all developers to support all platforms, we would shift the responsibility to support esoteric platforms to the developers of new features, even if the majority has no interest in the platform. This is not how we handle new features and I don't see any reason why we should do that for portability.<br>
<br>
</div>Again, the difference between a feature and portability is that you can live without a new feature if it is possible to turn off, while non-portable code means Gromacs won't run at all. Only requiring an ISO compiler is a core value we have had for almost 20 years, and that decision won't be changed easily. '<br>
<br>
Historically, the portability to new platforms has been a tremendous advantage for Gromacs.<br></blockquote><div>As far as I know, at least for the last few years portability to exotic platforms (e.g. PS3, K) has been important to only a very small part of users/developers. </div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="im">
> I think we shouldn't require better portability to external code than we require for own own.<br>
<br>
</div>Indeed - but the whole point is that Gromacs code that isn't portable when used with an ISO C/C++ compiler should result in a redmine issue and be fixed. When this has been urgent (e.g. for a new supercomputer or compiler), we have virtually always managed to fix it within 24h.<br>
<br>
We don't require this of external code or libraries for optional features, of course, but if a library is going to be used for a core Gromacs feature (like FFTs, or the XDR functionality) we have made sure there is a built-in version where we can take responsibility for the portability ourselves.<br>
</blockquote><div><br></div><div>But I think that "fancy" IO is also an optional feature. I agree that it is a very important feature and it has many disadvantages if the same format is not used everywhere. But it is also non-essential. And at that point it should become a matter of cost-benefit and not a matter of principal. I.e. how many people benefit from features made possible by HDF5 (e.g. because limited developer time wouldn't allow them without HDF5) versus how much of a pain is it to the few people how have to live with XTC (and conversion). And one very important factor in that cost-benefit analysis is the ratio of users. </div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="im">><br>
><br>
> > I think the first step is to be concrete about exactly what useful features we will get automatically with HDF5 that will be very difficult or impossible to implement in our > own format?<br>
> It is one option to do that research first. We have started it but are not finished. Another option is, we first agree on the requirements any potential external library has to fulfill. I think we currently don't have any guidelines for this.<br>
<br>
</div>The route we have taken this far is that you can use pretty much any external library, as long as there is Gromacs built-in functionality (potentially much slower) to make sure Gromacs always works. For instance, we have FFTPACK built-in although we prefer to use FFTW externally.<br>
<div class="im"><br>
> Well even if all of TNG would only be available with HDF5, I don't see why we would need to have it 100% portable. The XTC compression is not orders of magnitudes worse.<br>
<br>
</div>1) Because we want to avoid producing XTC for new trajectories. We want a small library that other programs (VMD, PyMol, Babel, etc.) can include to be able to read/write these trajectory formats. Case in point: I believe there are at least two orders of magnitude more users that want Gromacs file formats they can directly feed into other programs compared to the number of users for which it is important with more efficient parallel IO.<br>
</blockquote><div>Parallel IO is not something which would benefit from HDF5. The advantages I see (as I said a preliminary list because we are in the middle of looking into it):</div><div>- User can use HDF5 tools to look at files</div>
<div>- User can write easy scripts (simple binding to scripting languages) to analyze data</div><div>- No large extra code base for container format in Gromacs (maintainability) </div><div>- Less developer time (we can focus on MD specific code and not how to implement a container format)</div>
<div>- Easy to implement convenience features (like adding structure and topology to trajectory; combining trajectory, energies, and extra output (e.g. pull) and thus having (optionally) only one output file; having build in history; making files user extendable)</div>
<div>- Trajectory easy to index and seek (e.g. for parallel analysis)</div><div><br></div><div>Obviously one could either keep the own container format very simple, and thus would miss most convince features, tools, and language bindings, or one could make it very complex, and thus require a lot of developer time and have the maintainability issues. But without an external library one cannot have both.</div>
<div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
2) Because it would introduce internal incompatibilities in Gromacs. If the default file format isn't available on all platforms we would be back in the situation before we had XTC available everywhere, and Gromacs compiled on host B would not always be able to read the files produced on host A with the very same Gromacs version.<br>
</blockquote><div>But what is the problem if all non exotic systems would support the new file format. Then one can still guarantee that the analysis can be done in the new format. And the only thing, the (very few users) of exotic platforms have to do, is to convert the XTC format to the new format (post production) to avoid any problems.</div>
<div><br></div><div>Roland</div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
<br>
<br>
<br>
<br>
<br>
</blockquote></div><br><br clear="all"><div><br></div>-- <br>ORNL/UT Center for Molecular Biophysics <a href="http://cmb.ornl.gov">cmb.ornl.gov</a><br>865-241-1537, ORNL PO BOX 2008 MS6309<br>