Chris,<div><br></div><div>one quick look at the code suggest that the reason is that the memory isn't freed correctly. read_first_frame is allocating the memory for one frame but in most cases it isn't freed. Thus you only see this error with many small files. </div>
<div><br></div><div>Since it is open source the best way to get it fixed, is to quickly do it yourself ;-) </div><div><br></div><div>Roland</div><div><br><div class="gmail_quote">On Wed, Oct 13, 2010 at 1:40 AM, <span dir="ltr"><<a href="mailto:chris.neale@utoronto.ca" target="_blank">chris.neale@utoronto.ca</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Thank you for the insight Mark.<br><br>
To get it on the record, this is fairly annoying for trjcat (not trjconv). If anyone else has run into this, then please do post because a memory efficient cat mechanism will only be coded if there is sufficient interest.<br>
<br>
But again, thanks for your good answer Mark, probably you are correct.<br>
<br>
Chris.<br>
<br>
----- Original Message -----<br>
From: <a href="mailto:chris.neale@utoronto.ca" target="_blank">chris.neale@utoronto.ca</a><br>
Date: Wednesday, October 13, 2010 10:55<br>
Subject: [gmx-users] why does trjcat take so much memory?<br>
To: <a href="mailto:gmx-users@gromacs.org" target="_blank">gmx-users@gromacs.org</a><br>
<br>
[Hide Quoted Text]<br>
I have 28,000 .xtc files, each having a single frame and each<br>
150K. If I run du -hs on the directory containing these .xtcs, I<br>
get 4.4 GB. Nevertheless, when I run trjcat -f *.xtc -o<br>
../tot.xtc , my memory consumption goes over 11 GB and then I<br>
run out of available memory.<br>
<br>
Sure, I could find a larger memory system, but why does it take<br>
=3x more memory than the individual files took in disk space?<br>
I'd guess that the increased space reflects trjconv converting the compressed .xtc format back to normal precision, and then back to compressed. To cope with all its possibilities, trjconv probably needs to either keep the loaded frames in memory, or use a two-pass algorithm. It does the former.<br>
<br>
Mark<br><font color="#888888">
<br>
<br>
-- <br>
gmx-users mailing list <a href="mailto:gmx-users@gromacs.org" target="_blank">gmx-users@gromacs.org</a><br>
<a href="http://lists.gromacs.org/mailman/listinfo/gmx-users" target="_blank">http://lists.gromacs.org/mailman/listinfo/gmx-users</a><br>
Please search the archive at <a href="http://www.gromacs.org/Support/Mailing_Lists/Search" target="_blank">http://www.gromacs.org/Support/Mailing_Lists/Search</a> before posting!<br>
Please don't post (un)subscribe requests to the list. Use thewww interface or send it to <a href="mailto:gmx-users-request@gromacs.org" target="_blank">gmx-users-request@gromacs.org</a>.<br>
Can't post? Read <a href="http://www.gromacs.org/Support/Mailing_Lists" target="_blank">http://www.gromacs.org/Support/Mailing_Lists</a><br>
</font></blockquote></div><br><br clear="all"><br>-- <br>ORNL/UT Center for Molecular Biophysics <a href="http://cmb.ornl.gov" target="_blank">cmb.ornl.gov</a><br>865-241-1537, ORNL PO BOX 2008 MS6309<br>
</div>