<div class="gmail_quote">On Wed, Jul 11, 2012 at 6:51 PM, Christoph Junghans <span dir="ltr"><<a href="mailto:junghans@votca.org" target="_blank">junghans@votca.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
2012/7/11 Roland Schulz <<a href="mailto:roland@utk.edu">roland@utk.edu</a>>:<br>
<div><div class="h5">> On Wed, Jul 11, 2012 at 7:19 AM, Alexey Shvetsov<br>
> <<a href="mailto:alexxy@omrb.pnpi.spb.ru">alexxy@omrb.pnpi.spb.ru</a>> wrote:<br>
>> Roland Schulz писал 2012-07-11 10:47:<br>
>>> On Wed, Jul 11, 2012 at 2:33 AM, Alexey Shvetsov<br>
>>> <<a href="mailto:alexxy@omrb.pnpi.spb.ru">alexxy@omrb.pnpi.spb.ru</a>> wrote:<br>
>>>> Hi!<br>
>>>><br>
>>>> mvpich2/mvapich (as well as its derviations like platform<br>
>>>> mpi,pc-mpi,intel-mpi) also will behave differently. So user can get<br>
>>>> cryptic message about launching mpd, in case of launching mdrun -np<br>
>>>> directly<br>
>>> Not quite. mpich2 requires for MPI_Comm_spawn to work that the<br>
>>> application is run with mpirun/mpiexec. See<br>
>>><br>
>>> <a href="http://lists.mcs.anl.gov/pipermail/mpich-discuss/2012-June/012638.html" target="_blank">http://lists.mcs.anl.gov/pipermail/mpich-discuss/2012-June/012638.html</a><br>
>>> for the details. We would need to detect that and don't try to spawn<br>
>>> in that case (and run in serial with a warning).<br>
>>> Thus mpich2 would require: mpirun mdrun -np x. Of course that isn't<br>
>>> more convenient than mpirun -np x mdrun. The only advantage would be<br>
>>> that as with tmpi "mpirun mdrun" would automatically use as many<br>
>>> cores<br>
>>> as are available and are useful for the system, whereas without spawn<br>
>>> the user needs to decides the number of cores and we can't have any<br>
>>> automatic mechanism helping the user.<br>
>>><br>
>>> Roland<br>
>><br>
>> Ok. But how this will work with batch systems that automaticaly send<br>
>> number of processes to mpiexec, effective launch command will be<br>
>><br>
>> $ mpiexec mdrun_mpi $mdrunargs<br>
><br>
> The idea was to only do any spawn if mdrun_mpi is started in serial<br>
> (mpiexec -n 1). It was only meant to make mdrun_mpi behave the same as<br>
> tmpi mdrun for a single node. On clusters with batch system nothing<br>
> would have changed over the current situation.<br>
</div></div>1.) I agree with Roland that keeping the -nt option would be<br>
misleading, even if -nt gets a new meaning - "number of tasks", where<br>
tasks can be threads or mpi tasks.<br>
Also from my experience, half of the users are not aware of the -nt<br>
option either, they just start mdrun without any special setting,<br>
which means "guess" and we should keep that.<br>
So making -nt obsolete is not bad, I think.<br></blockquote><div><br></div><div>Without thinking about the "I just want to type mdrun and nothing else" type of users it would make sense to require a user to use -ntmpi and/or -ntomp. However, I have the feeling that users could be quite overwhelmed with the technical nature of the different parallelization schemes. That's why the convenience-option -nt could be good to keep.</div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">2.) One class of system that has not been discussed yet, are the one<br>
with different number of OMP threads per node (like the Intel MIC),<br>
that should also be possible by explicitly defining OMP_NUM_THREADS on<br>
each node.<br></blockquote><div><br></div><div>That you should work both automatically as well as manually already now as both the detection and run configuration setup happens on a per-process basis. You won't be able to have different # of OpenMP threads per process with thread-MPI, though.</div>
<div><br></div><div>I'm tweaking the detection and # of threads code right now, so your help/feedback would be useful. Do you still have access to such a machine? If not, emulating this scenario would be possible with running on machines that have different core count.</div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
3.) With all these thread options and combination, we will need<br>
something like g_tune_mdrun, which I guess could be an extension of<br>
g_tune_pme.<br></blockquote><div><br></div><div>On the ling run it would, but I'd rather see resources channeled into a more flexible multi-level & task-parallelization setup in which we can dynamically tune the number of threads per task and per process as well as the number of processes. However, this is getting off-topic, let's get back to it after 4.6.</div>
<div><br></div><div>Cheers,</div><div>--</div><div>Sz.</div><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
Christoph<br>
<div class="HOEnZb"><div class="h5"><br>
<br>
<br>
><br>
> Roland<br>
><br>
>><br>
>><br>
>><br>
>>><br>
>>>><br>
>>>> Roland Schulz писал 2012-07-11 05:09:<br>
>>>>> On Tue, Jul 10, 2012 at 8:09 PM, Szilárd Páll<br>
>>>>> <<a href="mailto:szilard.pall@cbr.su.se">szilard.pall@cbr.su.se</a>> wrote:<br>
>>>>>> On Tue, Jul 10, 2012 at 11:15 PM, Berk Hess <<a href="mailto:hess@kth.se">hess@kth.se</a>> wrote:<br>
>>>>>>> Hi,<br>
>>>>>>><br>
>>>>>>> We are working on the final part of the 4.6 release, which is<br>
>>>>>>> making the MPI<br>
>>>>>>> and OpenMP thread setup automated, fully checked and user<br>
>>>>>>> friendly.<br>
>>>>>>> We have to decide on the naming of the options.<br>
>>>>>>> Roland has an implementation of mpi spawn ready. This would allow<br>
>>>>>>> to do<br>
>>>>>>> mdrun -np #processes instead of using mpirun (at least with<br>
>>>>>>> openmpi).<br>
>>>>>><br>
>>>>>> Would this feature add anything but the convenience of being able<br>
>>>>>> to<br>
>>>>>> run without mpirun on a single node? Without MPI spawning working<br>
>>>>>> reliably in most cases (or with the ability to detect with a high<br>
>>>>>> certainty when it does not), enabling an -np mdrun option would<br>
>>>>>> just<br>
>>>>>> lead to confusion when mdrun exits with cryptic MPI error due to<br>
>>>>>> not<br>
>>>>>> being able to spawn.<br>
>>>>> The idea was to make mdrun behave the same whether it is compiled<br>
>>>>> with<br>
>>>>> real MPI or tMPI. Thus also only support a single node. But MPICH<br>
>>>>> is<br>
>>>>> behaving quite stupid and they also don't seem to care. And only<br>
>>>>> supporting it for OpenMPI is probably also more confusing then<br>
>>>>> helpful<br>
>>>>> (then tmpi+OpenMPI would behave the same but MPICH/MVAPICH would<br>
>>>>> behave different). So you are probably right that it is better to<br>
>>>>> not<br>
>>>>> add spawn at all.<br>
>>>>><br>
>>>>>> Therefore, I'd be OK with a new *hidden* -np option that only<br>
>>>>>> works<br>
>>>>>> in<br>
>>>>>> single-node case, but not with a non-hidden one advertised in the<br>
>>>>>> documentation/wiki.<br>
>>>>> As a hidden option it would only help for testing. But I don't<br>
>>>>> think<br>
>>>>> it is worth adding it for just that.<br>
>>>>><br>
>>>>> Roland<br>
>>>><br>
>>>> --<br>
>>>> Best Regards,<br>
>>>> Alexey 'Alexxy' Shvetsov<br>
>>>> Petersburg Nuclear Physics Institute, NRC Kurchatov Institute,<br>
>>>> Gatchina, Russia<br>
>>>> Department of Molecular and Radiation Biophysics<br>
>>>> Gentoo Team Ru<br>
>>>> Gentoo Linux Dev<br>
>>>> mailto:<a href="mailto:alexxyum@gmail.com">alexxyum@gmail.com</a><br>
>>>> mailto:<a href="mailto:alexxy@gentoo.org">alexxy@gentoo.org</a><br>
>>>> mailto:<a href="mailto:alexxy@omrb.pnpi.spb.ru">alexxy@omrb.pnpi.spb.ru</a><br>
>>>> --<br>
>>>> gmx-developers mailing list<br>
>>>> <a href="mailto:gmx-developers@gromacs.org">gmx-developers@gromacs.org</a><br>
>>>> <a href="http://lists.gromacs.org/mailman/listinfo/gmx-developers" target="_blank">http://lists.gromacs.org/mailman/listinfo/gmx-developers</a><br>
>>>> Please don't post (un)subscribe requests to the list. Use the<br>
>>>> www interface or send it to <a href="mailto:gmx-developers-request@gromacs.org">gmx-developers-request@gromacs.org</a>.<br>
>>>><br>
>>>><br>
>>>><br>
>>><br>
>>><br>
>>><br>
>>> --<br>
>>> ORNL/UT Center for Molecular Biophysics <a href="http://cmb.ornl.gov" target="_blank">cmb.ornl.gov</a><br>
>>> 865-241-1537, ORNL PO BOX 2008 MS6309<br>
>><br>
>> --<br>
>> Best Regards,<br>
>> Alexey 'Alexxy' Shvetsov<br>
>> Petersburg Nuclear Physics Institute, NRC Kurchatov Institute,<br>
>> Gatchina, Russia<br>
>> Department of Molecular and Radiation Biophysics<br>
>> Gentoo Team Ru<br>
>> Gentoo Linux Dev<br>
>> mailto:<a href="mailto:alexxyum@gmail.com">alexxyum@gmail.com</a><br>
>> mailto:<a href="mailto:alexxy@gentoo.org">alexxy@gentoo.org</a><br>
>> mailto:<a href="mailto:alexxy@omrb.pnpi.spb.ru">alexxy@omrb.pnpi.spb.ru</a><br>
>> --<br>
>> gmx-developers mailing list<br>
>> <a href="mailto:gmx-developers@gromacs.org">gmx-developers@gromacs.org</a><br>
>> <a href="http://lists.gromacs.org/mailman/listinfo/gmx-developers" target="_blank">http://lists.gromacs.org/mailman/listinfo/gmx-developers</a><br>
>> Please don't post (un)subscribe requests to the list. Use the<br>
>> www interface or send it to <a href="mailto:gmx-developers-request@gromacs.org">gmx-developers-request@gromacs.org</a>.<br>
>><br>
>><br>
>><br>
><br>
><br>
><br>
> --<br>
> ORNL/UT Center for Molecular Biophysics <a href="http://cmb.ornl.gov" target="_blank">cmb.ornl.gov</a><br>
> 865-241-1537, ORNL PO BOX 2008 MS6309<br>
> --<br>
> gmx-developers mailing list<br>
> <a href="mailto:gmx-developers@gromacs.org">gmx-developers@gromacs.org</a><br>
> <a href="http://lists.gromacs.org/mailman/listinfo/gmx-developers" target="_blank">http://lists.gromacs.org/mailman/listinfo/gmx-developers</a><br>
> Please don't post (un)subscribe requests to the list. Use the<br>
> www interface or send it to <a href="mailto:gmx-developers-request@gromacs.org">gmx-developers-request@gromacs.org</a>.<br>
<br>
<br>
<br>
</div></div><span class="HOEnZb"><font color="#888888">--<br>
Christoph Junghans<br>
Web: <a href="http://www.compphys.de" target="_blank">http://www.compphys.de</a><br>
</font></span><div class="HOEnZb"><div class="h5">--<br>
gmx-developers mailing list<br>
<a href="mailto:gmx-developers@gromacs.org">gmx-developers@gromacs.org</a><br>
<a href="http://lists.gromacs.org/mailman/listinfo/gmx-developers" target="_blank">http://lists.gromacs.org/mailman/listinfo/gmx-developers</a><br>
Please don't post (un)subscribe requests to the list. Use the<br>
www interface or send it to <a href="mailto:gmx-developers-request@gromacs.org">gmx-developers-request@gromacs.org</a>.<br>
</div></div></blockquote></div><br>