<div dir="ltr"><br><div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, Aug 7, 2014 at 9:50 PM, Theodore Si <span dir="ltr"><<a href="mailto:sjyzhxw@gmail.com" target="_blank">sjyzhxw@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
Hi Mark,<br>
<br>
Thanks for your attention. I have some questions for you.</div></blockquote><div><br></div><div>That's nice, but you haven't answered my question, or responded to my earlier suggestions for simplifying how you're trying to use VT to find where the problem is. We're not black boxes with infinite time, here ;-)</div>
<div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div bgcolor="#FFFFFF" text="#000000"><div class="">
1. Profilers that just need to compile with standard profiling flags
are fine - just configure with -DCMAKE_BUILD_TYPE=Profile and go for
it. <br></div>
It seems that this is convenient. What profilers can be used in this
way? <font color="#ff0000">What tools do you guys use to profile?</font></div></blockquote><div><br></div><div>I don't know. We use none routinely, for the aforementioned reasons. They're hard to get to build, run without perturbation, and analyze the results. Some of that is intrinsic to the problem space. There's a project making slow progress to integrate some support for more routine use of Extrae.</div>
<div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div bgcolor="#FFFFFF" text="#000000"><div class="">
2. If you need to pass things to the wrapper compiler, make yourself
a script to wrap the wrapper compiler and give that to CMake. <br></div>
I think it's the case when I compile GMX with Vampir Trace. We need
to use the command <font color="#33ff33">vtcc -vt:cc mpicc
sourcefile.c -o a.out</font> to instrument MPI code. Should
"-vt:cc mpicc" be passed to vtcc, which is set to CMAKE_C_COMPILER,
as argument? I don't know how to "make yourself a script to wrap the
wrapper compiler and give that to CMake". How to give a script to
CMake exactly? I have not used CMake before, please give me some
instructions.<br></div></blockquote><div><br></div><div>A compiler is a command you run that expects a bunch of flags and input files to follow. So you just make a shell script that conforms to that interface, so that CMake can be given a script and it will follow the rules it expects compilers to follow. So something like</div>
<div><br></div><div>#!/bin/sh</div><div>vtcc -vt:cc mpicc $@<br></div><div><br></div><div>in myvtcc.sh could be passed to cmake with</div><div><br></div><div>cmake .. -DCMAKE_C_COMPILER=/path/to/myvtcc.sh</div><div><br></div>
<div>You'd have to do exactly the same kind of thing to hack a Makefile or a command line - you need to replace all calls to the actual compiler with "vtcc -vt:cc mpicc" which is what this shell script is doing in a way that makes it easy for CMake not to have to be robust against spaces and quoting.</div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div bgcolor="#FFFFFF" text="#000000">
When I use Vampir Trace, some weird errors occur:<br>
<br>
1. In some source files, some multiple-line comments are processed
by the compiler. If I change them to one-line comments, it works
fine. I have no idea why that happens.<br></div></blockquote><div><br></div><div>No idea, but someone's not conforming to a language spec. Given that a wide range of compilers are happy with those input files, I'm guessing VT's doing some parsing that isn't implemented quite well enough. You can avoid those particular problems by configuring gromacs without TNG support, e.g. cmake -DGMX_USE_TNG=off</div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div bgcolor="#FFFFFF" text="#000000">
2. If I use the following configuration:<br>
<span style="font-family:Calibri" lang="zh-CN">cmake ..</span><span style="font-family:Calibri" lang="en-US"> </span><span style="font-family:Calibri" lang="zh-CN">-DCMAKE_INSTALL_PREFIX=$MYPRG</span><span style="font-family:SimSun" lang="en-US">/</span><span style="font-family:Calibri" lang="zh-CN">gmx</span><span style="font-family:Calibri" lang="en-US"> </span><span style="font-family:Calibri" lang="zh-CN">-</span><span style="font-family:Calibri" lang="en-US">DCU</span><span style="font-family:Calibri" lang="zh-CN">DA_TOOLKIT_ROOT_DIR=/usr/local/cuda</span><span style="font-family:Calibri" lang="en-US"> -D</span><span style="font-family:Calibri" lang="zh-CN">CUDA_CUDA_LIBRARY</span><span style="font-family:Calibri" lang="en-US">=/usr/lib64/libcuda.so </span><span style="font-family:Calibri" lang="zh-CN">-DCMAKE_C_COMPILER=vtcc
-DCMAKE_CXX_COMPILER</span><span style="font-family:Calibri" lang="en-US">=</span><span style="font-family:Calibri" lang="zh-CN">vtc</span><span style="font-family:Calibri" lang="en-US">xx -D</span><span style="font-family:Calibri" lang="zh-CN">CUDA_NVCC_EXECUTABLE</span><span style="font-family:Calibri" lang="en-US">=vtnvcc </span><span style="font-family:Calibri" lang="zh-CN">-DGMX_MPI=ON</span><span style="font-family:Calibri" lang="en-US"> </span><span style="font-family:Calibri" lang="zh-CN">-DGMX_BUILD_OWN_FFTW=ON</span><br>
The compilation can be done. And when I run grompp to generate tpr,
it's OK and Vampir Trace will generate trace files. When I run mdrun
without parameters, it also generate trace files. However, when I
actually start mdrun with a tpr file, it will give a <font color="#ff0000">segmentation fault</font>.</div></blockquote><div><br></div><div>I made suggestions here earlier.</div><div><br></div><div>Mark</div><div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div bgcolor="#FFFFFF" text="#000000"><div><div class="h5">
<br>
<br>
<div>On 8/7/2014 10:56 PM, Mark Abraham
wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Hi,
<div><br>
</div>
<div>First, what are you hoping to learn? Many questions have a
known answer, or are known not to have a clear answer ;-)</div>
<div><br>
</div>
<div>It's a compound problem. Profilers that just need to
compile with standard profiling flags are fine - just
configure with -DCMAKE_BUILD_TYPE=Profile and go for it. Those
that need to influence the compilation or linker line are more
problematic. Just passing the (full path to) the wrapper
compiler should work fine. If you need to pass things to the
wrapper compiler, make yourself a script to wrap the wrapper
compiler and give that to CMake. Whether the things the tool's
wrapper compiler does clashes with things GROMACS is doing
varies a lot, and you will need to get involved in the details
to see what the origin of any problems are. That's not
anybody's fault, per se, but the writer of a wrapper compiler
typically hopes the end user is not managing details
themselves, but to get high performance you often have to
manage details, and some of those details show up on the
compiler command lines generated by the GROMACS build system.
And naturally you'll be interested in MPI+OpenMP+CUDA, each of
which compounds the problem with further wrapper compilers or
command-line stuff.</div>
<div><br>
</div>
<div>Once it's working, you have the problem of whether you can
get useful data. Instrumenting every function call, or
compromising function inlining is guaranteed to be useless
because that overhead kills things. The main interesting case
to profile is where the MD step iteration time is a few
milliseconds, and you can't introduce thousands of increments
of tens of nanoseconds and get sensible profiles. So either
you have to restrict the instrumentation to high-level
functions (which is painful; the output at the end of the
GROMACS log file is a coarse version of this averaged over
many steps and execution contexts), or use a sampling-based
approach.</div>
<div><br>
</div>
<div>Then you need to start collecting data after the run-time
performance tuning that mdrun does has already stabilized - at
least a few hundred MD steps. Longer if the MD load is
imbalanced, which is also the main interesting case to
consider for code modifications.</div>
<div><br>
</div>
<div>Mark</div>
</div>
<div class="gmail_extra"><br>
<br>
<div class="gmail_quote">On Wed, Aug 6, 2014 at 10:58 AM,
Theodore Si <span dir="ltr"><<a href="mailto:sjyzhxw@gmail.com" target="_blank">sjyzhxw@gmail.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<p dir="ltr">what kind of tools work best with gromacs 5.0?</p>
<br>
--<br>
Gromacs Developers mailing list<br>
<br>
* Please search the archive at <a href="http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List" target="_blank">http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List</a>
before posting!<br>
<br>
* Can't post? Read <a href="http://www.gromacs.org/Support/Mailing_Lists" target="_blank">http://www.gromacs.org/Support/Mailing_Lists</a><br>
<br>
* For (un)subscribe requests visit<br>
<a href="https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers" target="_blank">https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers</a>
or send a mail to <a href="mailto:gmx-developers-request@gromacs.org" target="_blank">gmx-developers-request@gromacs.org</a>.<br>
</blockquote>
</div>
<br>
</div>
<br>
<fieldset></fieldset>
<br>
</blockquote>
<br>
</div></div></div>
<br>--<br>
Gromacs Developers mailing list<br>
<br>
* Please search the archive at <a href="http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List" target="_blank">http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List</a> before posting!<br>
<br>
* Can't post? Read <a href="http://www.gromacs.org/Support/Mailing_Lists" target="_blank">http://www.gromacs.org/Support/Mailing_Lists</a><br>
<br>
* For (un)subscribe requests visit<br>
<a href="https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers" target="_blank">https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers</a> or send a mail to <a href="mailto:gmx-developers-request@gromacs.org">gmx-developers-request@gromacs.org</a>.<br>
</blockquote></div><br></div></div>