On Mon, Dec 3, 2012 at 6:58 PM, Roland Schulz <span dir="ltr"><<a href="mailto:roland@utk.edu" target="_blank">roland@utk.edu</a>></span> wrote:<br><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
Hi,<div><br></div><div>I think both the error message when <span style="font-family:arial,sans-serif;font-size:12.800000190734863px">OMP_NUM_THREADS is set (especially for group kernels) and the error message if </span><font face="arial, sans-serif">rdtscp isn't supported (this is disabled by </font><span style="font-family:arial,sans-serif;font-size:12.800000190734863px">GMX_DISTRIBUTABLE_BUILD</span><span style="font-family:arial,sans-serif;font-size:12.800000190734863px"> ) </span><span style="font-family:arial,sans-serif">should be improved (anything is better than illegal instruction). So I would open a bug so we can track that. </span></div>
</blockquote><div><br></div>As I said in my previous mail, the former should not be a concern (but do let me know if you don't see the message on the stderr), the latter is IMO way too specific. Users will get illegal instruction errors relatively often as probably many are used to being able to run binaries across different machines, something that will often not work with 4.6. In general, if you want to be able to issue a warning before the user gets the illegal instruction, you need to compile the cpuid code without optimization and call it as the very first thing to check the capability of the CPU against what the binary was compiled with. Even this might not work with e.g. -march, but at least it will be useful in most cases and not only when rdtscp is not supported.<br>
<br>Cheers,<br>--<br>Szilárd<div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div><span style="font-family:arial,sans-serif"><br></span></div><div><span style="font-family:arial,sans-serif">Roland</span></div><div class="gmail_extra"><br><br><div class="gmail_quote"><div class="im">On Mon, Dec 3, 2012 at 10:50 AM, Shirts, Michael (mrs5pt) <span dir="ltr"><<a href="mailto:mrs5pt@eservices.virginia.edu" target="_blank">mrs5pt@eservices.virginia.edu</a>></span> wrote:<br>
</div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div class="im">So in the end, setting GMX_DISTRIBUTABLE_BUILD seems to have been the<br>
solution. I only compiled with SSE2, so I don't think it was a<br>
GMX_CPU_ACCELERATION mismatch.<br>
<br>
To summarize, I cleaned everything out, configured with<br>
<br>
cmake ../gromacs -DGMX_DISTRIBUTABLE_BUILD=Yes -DGMX_GPU=No -DGMX_DOUBLE=Yes<br>
-DGMX_CPU_ACCELERATION=SSE2 -DGMX_INSTALL_PRE<br>
<br>
And ran with:<br>
<br>
export OMP_NUM_THREADS=1<br>
$BINPATH/mdrun_d -ntmpi 8 -deffnm $DATAPATH/eth.vrescale.50<br>
<br>
And everything seems to work OK now. Possibly I had some issues from not<br>
having a clean configure? Apologies for not being able to generate a more<br>
reproducible error log.<br>
<br>
Thanks all for suggestions and comments.<br>
</div><div><br><div class="im">
Best,<br>
~~~~~~~~~~~~<br>
Michael Shirts<br>
Assistant Professor<br>
Department of Chemical Engineering<br>
University of Virginia<br>
<a href="mailto:michael.shirts@virginia.edu" target="_blank">michael.shirts@virginia.edu</a><br>
<a href="tel:%28434%29-243-1821" value="+14342431821" target="_blank">(434)-243-1821</a><br>
<br>
<br>
</div></div>> From: Roland Schulz <<a href="mailto:roland@utk.edu" target="_blank">roland@utk.edu</a>><br>
> Date: Sun, 2 Dec 2012 23:12:31 -0500<div><div class="h5"><br>
<div><div>> To: "<a href="mailto:michael.shirts@virginia.edu" target="_blank">michael.shirts@virginia.edu</a>" <<a href="mailto:michael.shirts@virginia.edu" target="_blank">michael.shirts@virginia.edu</a>>, Discussion<br>
> list for GROMACS development <<a href="mailto:gmx-developers@gromacs.org" target="_blank">gmx-developers@gromacs.org</a>><br>
> Subject: Re: [gmx-developers] Gromacs trying to use OpenMP instead of<br>
> thread-mpi despite trying to convince it?<br>
><br>
> On Sun, Dec 2, 2012 at 12:58 PM, Shirts, Michael (mrs5pt) <<br>
> <a href="mailto:mrs5pt@eservices.virginia.edu" target="_blank">mrs5pt@eservices.virginia.edu</a>> wrote:<br>
><br>
>> So, more progress, but no simulations running yet.<br>
>><br>
>> mdrun -nt 8 -ntmpi 8 gives the same error as before (I actually tried that<br>
>> before, and forgot to include it in my error report)<br>
>><br>
>> mdrun -ntmpi 8 -ntomp 1 gives the error<br>
>> Fatal error:<br>
>> OMP_NUM_THREADS (8) and the number of threads requested on the command line<br>
>> (1) have different values<br>
>> For more information and tips for troubleshooting, please check the GROMACS<br>
>> website at <a href="http://www.gromacs.org/Documentation/Errors" target="_blank">http://www.gromacs.org/Documentation/Errors</a><br>
>><br>
><br>
> We probably should print a notice that OMP_NUM_THREADS is set. Otherwise<br>
> this is really confusing if OMP_NUM_THREADS isn't set by the user but by<br>
> the system.<br>
><br>
><br>
>> Fatal error:<br>
>> OMP_NUM_THREADS is invalid: '0'<br>
>><br>
><br>
> This is also for ntomp. Also there you wand to use 1 not 0 to disable<br>
> OpenMP (1 because it is total number of threads and thus 1 means serial).<br>
><br>
>><br>
>> /var/spool/PBS/mom_priv/jobs/<a href="http://2053253.lc5.itc.virginia.edu.SC" target="_blank">2053253.lc5.itc.virginia.edu.SC</a>: line 22:<br>
>> 12201<br>
>> Illegal instruction /h3/n1/shirtsgroup/gromac<br>
>> s_46/install/bin/mdrun_d -ntomp 1 -ntmpi -8 -deffnm<br>
>> /bigtmp/mrs5pt/eth.vrescale.50<br>
>><br>
><br>
> I suppose the CPU on the compute node is different from the build host. You<br>
> need to change GMX_CPU_ACCELERATION to the one correct for the compute<br>
> node. It could also help to set GMX_DISTRIBUTABLE_BUILD (both are cmake<br>
> options).<br>
><br>
> Roland<br>
><br>
><br>
>><br>
>> Best,<br>
>> ~~~~~~~~~~~~<br>
>> Michael Shirts<br>
>> Assistant Professor<br>
>> Department of Chemical Engineering<br>
>> University of Virginia<br>
>> <a href="mailto:michael.shirts@virginia.edu" target="_blank">michael.shirts@virginia.edu</a><br>
>> <a href="tel:%28434%29-243-1821" value="+14342431821" target="_blank">(434)-243-1821</a><br>
>><br>
>><br>
>>> From: Berk Hess <<a href="mailto:hess@kth.se" target="_blank">hess@kth.se</a>><br>
>>> Date: Sun, 2 Dec 2012 09:34:11 +0100<br>
>>> To: "<a href="mailto:michael.shirts@virginia.edu" target="_blank">michael.shirts@virginia.edu</a>" <<a href="mailto:michael.shirts@virginia.edu" target="_blank">michael.shirts@virginia.edu</a>>,<br>
>> Discussion<br>
>>> list for GROMACS development <<a href="mailto:gmx-developers@gromacs.org" target="_blank">gmx-developers@gromacs.org</a>><br>
>>> Subject: Re: [gmx-developers] Gromacs trying to use OpenMP instead of<br>
>>> thread-mpi despite trying to convince it?<br>
>>><br>
>>> Hi,<br>
>>><br>
>>> You queuing system probably doesn't set OMP_NUM_THREADS then<br>
>>> and I assume this machine has at least 16 (HT) cores.<br>
>>> mdrun -ntmpi 8 -ntomp 1<br>
>>> will do what you want, or:<br>
>>> mdrun -nt 8 -ntmpi 8<br>
>>><br>
>>> Cheers,<br>
>>><br>
>>> Berk<br>
>>><br>
>>> On 12/02/2012 08:28 AM, Shirts, Michael (mrs5pt) wrote:<br>
>>>> Quick question:<br>
>>>><br>
>>>> Compiling the most recent code in release-4-6, I compile without OpenMP<br>
>>>> (because using group rather than verlet cutoffs), and using any of the<br>
>>>> below:<br>
>>>><br>
>>>> mdrun_d -ntmpi 8 -deffnm ethrun<br>
>>>> or<br>
>>>> mdrun_d -nt 8 -deffnm ethrun<br>
>>>> or<br>
>>>> mdrun_d -deffnm ethrun<br>
>>>> or<br>
>>>> mdrun_d -ntomp 0 -deffnm ethrun<br>
>>>> or<br>
>>>> mdrun_d -ntomp 0 -ntmpi8 -deffnm ethrun<br>
>>>><br>
>>>> I get:<br>
>>>> Fatal error:<br>
>>>> OpenMP threads are requested, but Gromacs was compiled without OpenMP<br>
>>>> support<br>
>>>> For more information and tips for troubleshooting, please check the<br>
>> GROMACS<br>
>>>> website at <a href="http://www.gromacs.org/Documentation/Errors" target="_blank">http://www.gromacs.org/Documentation/Errors</a><br>
>>>><br>
>>>> Even though I'm presumably requesting thread-mpi. Worked fine with -nt<br>
>>>> previously (before the new -nt options introduced a few months back).<br>
>>>><br>
>>>> Any suggestions or something I'm doing wrong? Perhaps gromacs is<br>
>>>> interpreting the cluster environment as requesting OpenMP somehow?<br>
>> FWIW, the<br>
>>>> PBS script request line is "#PBS -l select=1:mpiprocs=8:ncpus=8".<br>
>>>><br>
>>>> Apologies if I missed this answers somewhere out there already.<br>
>>>><br>
>>>> Thanks,<br>
>>>> ~~~~~~~~~~~~<br>
>>>> Michael Shirts<br>
>>>> Assistant Professor<br>
>>>> Department of Chemical Engineering<br>
>>>> University of Virginia<br>
>>>> <a href="mailto:michael.shirts@virginia.edu" target="_blank">michael.shirts@virginia.edu</a><br>
>>>> <a href="tel:%28434%29-243-1821" value="+14342431821" target="_blank">(434)-243-1821</a><br>
>>>><br>
>>><br>
>><br>
>> --<br>
>> gmx-developers mailing list<br>
>> <a href="mailto:gmx-developers@gromacs.org" target="_blank">gmx-developers@gromacs.org</a><br>
>> <a href="http://lists.gromacs.org/mailman/listinfo/gmx-developers" target="_blank">http://lists.gromacs.org/mailman/listinfo/gmx-developers</a><br>
>> Please don't post (un)subscribe requests to the list. Use the<br>
>> www interface or send it to <a href="mailto:gmx-developers-request@gromacs.org" target="_blank">gmx-developers-request@gromacs.org</a>.<br>
>><br>
>><br>
>><br>
>><br>
>><br>
><br>
><br>
> --<br>
> ORNL/UT Center for Molecular Biophysics <a href="http://cmb.ornl.gov" target="_blank">cmb.ornl.gov</a><br>
> <a href="tel:865-241-1537" value="+18652411537" target="_blank">865-241-1537</a>, ORNL PO BOX 2008 MS6309<br>
<br>
<br>
<br>
<br>
<br>
</div></div></div></div></blockquote></div><div><div class="h5"><br><br clear="all"><div><br></div>-- <br>ORNL/UT Center for Molecular Biophysics <a href="http://cmb.ornl.gov" target="_blank">cmb.ornl.gov</a><br>865-241-1537, ORNL PO BOX 2008 MS6309<br>
</div></div></div>
<br>--<br>
gmx-developers mailing list<br>
<a href="mailto:gmx-developers@gromacs.org">gmx-developers@gromacs.org</a><br>
<a href="http://lists.gromacs.org/mailman/listinfo/gmx-developers" target="_blank">http://lists.gromacs.org/mailman/listinfo/gmx-developers</a><br>
Please don't post (un)subscribe requests to the list. Use the<br>
www interface or send it to <a href="mailto:gmx-developers-request@gromacs.org">gmx-developers-request@gromacs.org</a>.<br></blockquote></div><br></div>