<html><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">Hi,<br><div><html>On Apr 1, 2008, at 1:26 PM, Mathias PUETZ wrote:</html><blockquote type="cite"><br><font size="2" face="sans-serif">Espresso file format, like most other ascii formats has a serious problem,</font> <br><font size="2" face="sans-serif">if you worry about parallel IO: They are hardly parallelizable.</font> <br><font size="2" face="sans-serif">Even though IO may not be a problem today, given time, it will,</font> <br><font size="2" face="sans-serif">as simulated systems and number of compute nodes get larger</font> <br><font size="2" face="sans-serif">and serial CPU power for ASCII formatting on rank 0 no longer scales.</font> <br><font size="2" face="sans-serif">I would seriously recommend to consider John's suggestion and go for HDF5,</font> <br><font size="2" face="sans-serif">which parallelizes well and offers high flexibility.</font> <br><font size="2" face="sans-serif">For those who want the simplicity of Fortran array IO, I would rather spend</font> <br><font size="2" face="sans-serif">a bit of extra effort to develop a comfortable reader tool, that can extract ascii</font> <br><font size="2" face="sans-serif">readable data for those who don't want the complexity of having to deal</font> <br><font size="2" face="sans-serif">with HDF5 directly (although HDF5 comes with it's own flexible ascii readers</font> <br><font size="2" face="sans-serif">which might be sufficientfor most tasks).</font> </blockquote><div><br></div><div>We actually considered NetCDF a long time ago, but at that time we decided against it since HDF5 was coming, but was too new/unstable then :-)</div><div><br></div><div>I think a lot of people (including me...) like to be able to do "simple" coordinate manipulation through scripts that just grep/awk for atom names, but I like Mathias suggestion of having a separate tool to translate back/forth instead, and keep the "core" format HDF5.</div><div><br></div><div>The only thing that worries me (just a little bit :-) is that it would make us entirely dependent on a big external library. I know that HDF5 is _very_ portable, but at least in theory we could end up in a situation where Gromacs doesn't work on some obscure platform e.g. because there's a compiler bug affecting HDF5. </div><div><br></div><div>Of course, that might be a reasonable compromise, but since I ended up doing my own implementation of the Unix external data representation (XDR) when we first ported Gromacs to windows I've toyed around with the idea of having some minimal built-in HDF5-generating code as a backup...</div><div><br></div><div>Mathias/John, do you or anybody else have any experience from using HDF5 for development? Have there been different library versions that you need to install, or do packages usually include their own copy of the library?</div><div><br></div><div>Cheers,</div><div><br></div><div>Erik</div><div><br></div></div><div><blockquote type="cite"><br> <br><tt><font size="2">> Message: 5<br> > Date: Mon, 31 Mar 2008 13:42:41 -0700<br> > From: "John Chodera" <<a href="mailto:jchodera@gmail.com">jchodera@gmail.com</a>><br> > Subject: Re: [gmx-developers] Shall we ditch gro and g96 files?<br> > To: "Discussion list for GROMACS development"<br> > <<a href="mailto:gmx-developers@gromacs.org">gmx-developers@gromacs.org</a>><br> > Message-ID:<br> > <<a href="mailto:14cc10610803311342i7f9ed758r8ed8fe95569573da@mail.gmail.com">14cc10610803311342i7f9ed758r8ed8fe95569573da@mail.gmail.com</a>><br> > Content-Type: text/plain; charset=ISO-8859-1<br> > <br> > Gentlemen,<br> > <br> > I know I don't chime in very often here, but I wanted to take this<br> > opportunity to say that I very much support the idea of replacing the<br> > limited-precision text-based formats like .gro, .pdb, and .g96 with<br> > more flexible, portable, full-precision file formats.<br> > <br> > Berk's suggestions of Espresso sounds very reasonable, but I would<br> > encourage you to instead look at netCDF and HDF5:<br> > <br> > netCDF:<br> > <a href="http://www.unidata.ucar.edu/software/netcdf/">http://www.unidata.ucar.edu/software/netcdf/</a><br> > <br> > HDF5:<br> > <a href="http://hdf.ncsa.uiuc.edu/HDF5/">http://hdf.ncsa.uiuc.edu/HDF5/</a><br> > <br> > Both of these formats provide easy-to-use libraries with APIs that<br> > support nearly every language you could want to use (including C,<br> > Fortran, and Python). They provide platform-independent, extensible<br> > formats for storing numerical information. Both provide attribute<br> > support, and HDF5 even allows hierarchical organization of objects,<br> > making it very much like XML but with support for multidimensional<br> > arrays of the same precision as used internally in gromacs. The<br> > libraries are robust, efficient, and well-supported.<br> > <br> > AMBER, for example, has already moved to netCDF for their trajectory<br> > format, though (unfortunately) not yet for their coordinate/restart<br> > files.<br> > <br> > <a href="http://amber.scripps.edu/netcdf/nctraj.html">http://amber.scripps.edu/netcdf/nctraj.html</a><br> > <br> > Cheers,<br> > <br> > John<br> > <br> > --<br> > Dr. John D. Chodera <<a href="mailto:jchodera@gmail.com">jchodera@gmail.com</a>> | Mobile : 415.867.7384<br> > Postdoctoral researcher, Pande lab | Lab phone : 650.723.1097<br> > Department of Chemistry, Stanford University | Lab fax : 650.724.4021<br> > <a href="http://www.dillgroup.ucsf.edu/~jchodera">http://www.dillgroup.ucsf.edu/~jchodera</a><br> > <br> > On 29/03/2008, David van der Spoel <<a href="mailto:spoel@xray.bmc.uu.se">spoel@xray.bmc.uu.se</a>> wrote:<br> > > Hi,<br> > ><br> > > as you are aware all coordinate files have their drawbacks.<br> > > - gro & pdb have limited space for coordinates which is problematic for<br> > > simulating large systems<br> > > - pdb has no velocities<br> > > - gro & g96 can not store information on the element (i.e. can not<br> > > distinguish between Calpha and Calcium or Hgamma and Mercury, pdb can do<br> > > this)<br> > > - gro stores non-rectanular boxes in an awkward manner<br> > ><br> > > I would therefore propose to make better coordinate file format that has<br> > > - coordinates<br> > > - velocities<br> > > - box as three edges and three angles (as in pdb file)<br> > > - atom name (and number)<br> > > - residue name and number<br> > > - element type (we could also introduce special elements for united<br> > > atoms or course grained particles, but they should not overlap with real<br> > > elements)<br> > > - variable format (no fixed column widths)<br> > ><br> > > In order to encourage the use of such a more flexible file format I<br> > > would then propose that we remove the facility for writing gro and g96<br> > > files.<br> > ><br> > > Please let me know what you think.<br> > ><br> > > --<br> > > David van der Spoel, Ph.D.<br> > > Molec. Biophys. group, Dept. of Cell & Molec. Biol., Uppsala University.<br> > > Box 596, 75124 Uppsala, Sweden. Phone: +46184714205. Fax: +4618511755.<br> > > spoel@xray.bmc.uu.se spoel@gromacs.org <a href="http://folding.bmc.uu.se">http://folding.bmc.uu.se</a><br> > > _______________________________________________<br> > > gmx-developers mailing list<br> > > <a href="mailto:gmx-developers@gromacs.org">gmx-developers@gromacs.org</a><br> > > <a href="http://www.gromacs.org/mailman/listinfo/gmx-developers">http://www.gromacs.org/mailman/listinfo/gmx-developers</a><br> > > Please don't post (un)subscribe requests to the list. Use the<br> > > www interface or send it to <a href="mailto:gmx-developers-request@gromacs.org">gmx-developers-request@gromacs.org</a>.<br> > ><br> > <br> > <br> > ------------------------------<br> > <br> > _______________________________________________<br> > gmx-developers mailing list<br> > <a href="mailto:gmx-developers@gromacs.org">gmx-developers@gromacs.org</a><br> > <a href="http://www.gromacs.org/mailman/listinfo/gmx-developers">http://www.gromacs.org/mailman/listinfo/gmx-developers</a><br> > <br> > <br> > End of gmx-developers Digest, Vol 48, Issue 1<br> > *********************************************<br> </font></tt> <br><font size="2" face="sans-serif"><br> Viele Grüsse / Best regards,<br> Dr. Mathias Pütz<br> <br> IT Specialist for Application Performance<br> <br> Deep Computing - Strategic Growth Business<br> IBM Systems & Technology Group<br> <br> e-mail: <a href="mailto:mpuetz@de.ibm.com">mpuetz@de.ibm.com</a><br> mobile: + 49-(0)160-7120602<br> fax: + 49-(0)6131-84-6660<br> <br> Anschrift:<br> IBM Deutschland GmbH<br> Department B513<br> Hechtsheimer Str. 2 / Building 12<br> 55131 Mainz<br> Germany<br> <br> IBM Deutschland GmbH<br> Vorsitzender des Aufsichtsrats: Hans Ulrich Maerki<br> Geschäftsführung: Martin Jetter (Vorsitzender), Christian Diedrich, Christoph Grandpierre, Matthias Hartmann, Thomas Fell, Michael Diemer<br> Sitz der Gesellschaft: Stuttgart<br> Registergericht: Amtsgericht Stuttgart, HRB 14562 WEEE-Reg.-Nr. DE 99369940</font> <br> <br>_______________________________________________<br>gmx-developers mailing list<br><a href="mailto:gmx-developers@gromacs.org">gmx-developers@gromacs.org</a><br>http://www.gromacs.org/mailman/listinfo/gmx-developers<br>Please don't post (un)subscribe requests to the list. Use the <br>www interface or send it to gmx-developers-request@gromacs.org.</blockquote></div><br><div apple-content-edited="true"> <span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div>------------</div><div>Erik Lindahl <<a href="mailto:lindahl@cbr.su.se">lindahl@cbr.su.se</a>> Backup: <<a href="mailto:erik.lindahl@gmail.com">erik.lindahl@gmail.com</a>></div><div>Assistant Professor, Computational Structural Biology</div><div>Center for Biomembrane Research, Dept. Biochemistry & Biophysics</div><div>Stockholm University, SE-106 91 Stockholm, Sweden</div><div>Tel: +46(0)8164675 Mobile: +46(0)704218767 Fax: mail a PDF instead</div><div><br></div><div><br></div></div></span><br class="Apple-interchange-newline"> </div><br></body></html>