<html><head><style>body{font-family:Helvetica,Arial;font-size:13px}</style></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;">Hi Carlo,</div><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;"><br></div><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;">I finally had some time to double-check, and my installation indeed passes all regression tests on a Mac Pro with dual AMD Firepro D300 using OS X 10.10.4. Based on the other comments today, this appears to be caused by OS X 10.9 on your machine. Unfortunately OpenCL code is compiled at runtime, and if the error occurs regardless of gcc or clang, it is probably either in the system header files or driver, which means it won’t even help to compile statically on a machine with OS X 10.10.</div><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;"><br></div><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;">Unless anybody has any alternative idea, it sounds as if we’ll have to consider a runtime check to disable OpenCL for AMD on versions prior to 10.10.4.</div><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;"><br></div><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;"><br></div><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;">Cheers,</div><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;"><br></div><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;">Erik</div><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;"><br></div><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;"><br></div><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;"><br></div><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;"><br></div> <div class="airmail_ext_on" style="color:black"><br>From: <span style="color:black">Carlo Camilloni</span> <a href="mailto:carlo.camilloni@gmail.com"><carlo.camilloni@gmail.com></a><br>Reply: <span style="color:black">Carlo Camilloni</span> <a href="mailto:carlo.camilloni@gmail.com"><carlo.camilloni@gmail.com>></a><br>Date: <span style="color:black">20 Jul 2015 at 19:50:26</span><br>To: <span style="color:black">Mark Abraham</span> <a href="mailto:mark.j.abraham@gmail.com"><mark.j.abraham@gmail.com>></a><br>Cc: <span style="color:black">gmx-developers@gromacs.org</span> <a href="mailto:gmx-developers@gromacs.org"><gmx-developers@gromacs.org>></a>, <span style="color:black">gromacs.org_gmx-developers@maillist.sys.kth.se</span> <a href="mailto:gromacs.org_gmx-developers@maillist.sys.kth.se"><gromacs.org_gmx-developers@maillist.sys.kth.se>></a>, <span style="color:black">Erik Lindahl</span> <a href="mailto:erik.lindahl@gmail.com"><erik.lindahl@gmail.com>></a><br>Subject: <span style="color:black"> Re: [gmx-developers] gromacs 5.1rc1 OpenCL problem with Parrinello-Rahman <br></span></div><br> <blockquote type="cite" class="clean_bq"><span><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div></div><div>
<title></title>
Hi again,
<div class=""><br class=""></div>
<div class="">I don’t know if you had the opportunity to reproduce
this bug, in the meantime I have compiled it with gcc 4.8, openMP
and sse4.1,</div>
<div class="">again cpu results are consistently right while OpenCL
results are consistently wrong, so it looks like it is NOT a
problem of Clang 6</div>
<div class="">but it could be either a problem of the gpu driver or
maybe OpenCL itself, I don’t know..</div>
<div class=""><br class=""></div>
<div class="">this is the new log</div>
<div class=""><br class=""></div>
<div class="">
<div style="margin: 0px; font-family: Menlo;" class="">GROMACS
version: VERSION 5.1-rc1</div>
<div style="margin: 0px; font-family: Menlo;" class="">
Precision: single</div>
<div style="margin: 0px; font-family: Menlo;" class="">Memory
model: 64 bit</div>
<div style="margin: 0px; font-family: Menlo;" class="">MPI
library: thread_mpi</div>
<div style="margin: 0px; font-family: Menlo;" class="">OpenMP
support: enabled (GMX_OPENMP_MAX_THREADS = 32)</div>
<div style="margin: 0px; font-family: Menlo;" class="">GPU
support: enabled</div>
<div style="margin: 0px; font-family: Menlo;" class="">OpenCL
support: enabled</div>
<div style="margin: 0px; font-family: Menlo;" class="">invsqrt
routine: gmx_software_invsqrt(x)</div>
<div style="margin: 0px; font-family: Menlo;" class="">SIMD
instructions: SSE4.1</div>
<div style="margin: 0px; font-family: Menlo;" class="">FFT
library: fftw-3.3.4-sse2-avx</div>
<div style="margin: 0px; font-family: Menlo;" class="">RDTSCP
usage: enabled</div>
<div style="margin: 0px; font-family: Menlo;" class="">C++11
compilation: disabled</div>
<div style="margin: 0px; font-family: Menlo;" class="">TNG
support: enabled</div>
<div style="margin: 0px; font-family: Menlo;" class="">Tracing
support: disabled</div>
<div style="margin: 0px; font-family: Menlo;" class="">Built on:
Mon 20 Jul 2015 18:21:43
BST</div>
<div style="margin: 0px; font-family: Menlo;" class="">Built by:
carlo [CMAKE]</div>
<div style="margin: 0px; font-family: Menlo;" class="">Build
OS/arch: Darwin 13.4.0 x86_64</div>
<div style="margin: 0px; font-family: Menlo;" class="">Build CPU
vendor: GenuineIntel</div>
<div style="margin: 0px; font-family: Menlo;" class="">Build CPU
brand: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz</div>
<div style="margin: 0px; font-family: Menlo;" class="">Build CPU
family: 6 Model: 62 Stepping: 4</div>
<div style="margin: 0px; font-family: Menlo;" class="">Build CPU
features: aes apic avx clfsh cmov cx8 cx16 f16c htt lahf_lm mmx msr
nonstop_tsc pcid pclmuldq pdcm pdpe1gb popcnt pse rdrnd rdtscp sse2
sse3 sse4.1 sse4.2 ssse3 tdt x2apic</div>
<div style="margin: 0px; font-family: Menlo;" class="">C compiler:
/opt/local/bin/gcc GNU 4.8.5</div>
<div style="margin: 0px; font-family: Menlo;" class="">C compiler
flags: -msse4.1 -Wextra
-Wno-missing-field-initializers -Wno-sign-compare -Wpointer-arith
-Wall -Wno-unused -Wunused-value -Wunused-parameter -O3
-DNDEBUG -funroll-all-loops -fexcess-precision=fast
-Wno-array-bounds</div>
<div style="margin: 0px; font-family: Menlo;" class="">C++
compiler: /opt/local/bin/g++ GNU 4.8.5</div>
<div style="margin: 0px; font-family: Menlo;" class="">C++ compiler
flags: -msse4.1 -Wextra
-Wno-missing-field-initializers -Wpointer-arith -Wall
-Wno-unused-function -O3 -DNDEBUG -funroll-all-loops
-fexcess-precision=fast -Wno-array-bounds</div>
<div style="margin: 0px; font-family: Menlo;" class="">Boost
version: 1.58.0 (external)</div>
<div style="margin: 0px; font-family: Menlo;" class="">OpenCL
include dir: /System/Library/Frameworks/OpenCL.framework</div>
<div style="margin: 0px; font-family: Menlo;" class="">OpenCL
library:
/System/Library/Frameworks/OPENCL.framework</div>
<div style="margin: 0px; font-family: Menlo;" class="">OpenCL
version: 1.2</div>
</div>
<div style="margin: 0px; font-family: Menlo;" class=""><br class=""></div>
<div style="margin: 0px; font-family: Menlo;" class=""><br class=""></div>
<div class="">Best,</div>
<div class="">Carlo</div>
<div class=""><br class=""></div>
<div class=""><br class=""></div>
<div class=""><br class="">
<div>
<blockquote type="cite" class="">
<div class="">On 17 Jul 2015, at 13:54, Carlo Camilloni
<<a href="mailto:carlo.camilloni@gmail.com" class="">carlo.camilloni@gmail.com</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">Hi,
<div class=""><br class=""></div>
<div class="">Unfortunately on the Mac Pro I am running os x 10.9.5
and the newer clang is not yet available so I cannot test it</div>
<div class="">with a different version. Erik if you have time to
tell me what to change to get gromacs compiled with gcc I can give
it a try</div>
<div class=""><br class=""></div>
<div class="">Carlo</div>
<div class=""><br class=""></div>
<div class=""><br class=""></div>
<div class="">
<div class="">
<blockquote type="cite" class="">
<div class="">On 16 Jul 2015, at 20:28, Mark Abraham <<a href="mailto:mark.j.abraham@gmail.com" class="">mark.j.abraham@gmail.com</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div dir="ltr" class="">Hi,
<div class=""><br class=""></div>
<div class="">Philip Fowler made a recent positive report on
OpenCL+AMD+Mac and he was using /usr/bin/cc Clang
6.1.0.6020053, so there may be an issue with Clang 6.0 (whatever
real clang version that is...)</div>
<div class=""><br class=""></div>
<div class="">Mark</div>
</div>
<br class="">
<div class="gmail_quote">
<div dir="ltr" class="">On Thu, Jul 16, 2015 at 7:20 PM Erik
Lindahl <<a href="mailto:erik.lindahl@gmail.com" class="">erik.lindahl@gmail.com</a>> wrote:<br class=""></div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word" class="">
<div style="font-family: Helvetica, Arial; font-size: 13px; margin: 0px;" class="">Hi,</div>
<div style="font-family: Helvetica, Arial; font-size: 13px; margin: 0px;" class=""><br class=""></div>
<div style="font-family: Helvetica, Arial; font-size: 13px; margin: 0px;" class="">The one obvious difference I can see is that you have
clang 6.0.0, while at least my Macbook pro
says 6.1.0. </div>
<div style="font-family: Helvetica, Arial; font-size: 13px; margin: 0px;" class=""><br class=""></div>
<div style="font-family: Helvetica, Arial; font-size: 13px; margin: 0px;" class="">Could you check if there is any update available for the
developer tools, and if so does that help?</div>
<div style="font-family: Helvetica, Arial; font-size: 13px; margin: 0px;" class=""><br class=""></div>
<div style="font-family: Helvetica, Arial; font-size: 13px; margin: 0px;" class="">It is actually much better to use gcc since you will then
also use all CPU cores. However, that requires patching a header
file in /usr/include; I’ll dig up what I did there - we need to put
it in the documentation anyway.</div>
<div style="font-family: Helvetica, Arial; font-size: 13px; margin: 0px;" class=""><br class=""></div>
<div style="font-family: Helvetica, Arial; font-size: 13px; margin: 0px;" class="">Cheers,</div>
<div style="font-family: Helvetica, Arial; font-size: 13px; margin: 0px;" class=""><br class=""></div>
<div style="font-family: Helvetica, Arial; font-size: 13px; margin: 0px;" class="">Erik</div>
<div style="" class=""></div>
</div>
<div style="word-wrap:break-word" class="">
<div style="" class=""><br class="">
From: <span style="" class="">Carlo Camilloni</span> <a href="mailto:carlo.camilloni@gmail.com" target="_blank" class=""><carlo.camilloni@gmail.com></a><br class=""></div>
</div>
<div style="word-wrap:break-word" class="">
<div style="" class="">Reply: <span style="" class="">Carlo
Camilloni</span> <a href="mailto:carlo.camilloni@gmail.com" target="_blank" class=""><carlo.camilloni@gmail.com>></a><br class="">
Date: <span style="" class="">16 Jul 2015 at
19:07:13</span><br class="">
To: <span style="" class="">Erik Lindahl</span> <a href="mailto:erik.lindahl@gmail.com" target="_blank" class=""><erik.lindahl@gmail.com>></a><br class="">
Cc: <span style="" class=""><a href="mailto:gromacs.org_gmx-developers@maillist.sys.kth.se" target="_blank" class="">gromacs.org_gmx-developers@maillist.sys.kth.se</a></span>
<a href="mailto:gromacs.org_gmx-developers@maillist.sys.kth.se" target="_blank" class=""><gromacs.org_gmx-developers@maillist.sys.kth.se>></a>,
<span style="" class=""><a href="mailto:gmx-developers@gromacs.org" target="_blank" class="">gmx-developers@gromacs.org</a></span>
<a href="mailto:gmx-developers@gromacs.org" target="_blank" class=""><gmx-developers@gromacs.org>></a></div>
</div>
<div style="word-wrap:break-word" class="">
<div style="" class=""><br class="">
Subject: <span style="" class="">Re: [gmx-developers] gromacs
5.1rc1 OpenCL problem with Parrinello-Rahman<br class=""></span></div>
</div>
<div style="word-wrap:break-word" class=""><br class="">
<blockquote type="cite" class="">
<div class=""></div>
<span class="">Hi Erik,</span>
<div class=""><span class=""><br class=""></span></div>
<div class=""><span class="">here it is:</span></div>
<div class=""><span class=""><br class=""></span></div>
<div class=""><span class=""><br class=""></span></div>
<div style="margin:0px;font-size:12px;font-family:Menlo" class="">
<span class="">GROMACS version: VERSION
5.1-rc1</span></div>
<div style="margin:0px;font-size:12px;font-family:Menlo" class="">
<span class="">Precision:
single</span></div>
<div style="margin:0px;font-size:12px;font-family:Menlo" class="">
<span class="">Memory model: 64
bit</span></div>
<div style="margin:0px;font-size:12px;font-family:Menlo" class="">
<span class="">MPI library:
none</span></div>
<div style="margin:0px;font-size:12px;font-family:Menlo" class="">
<span class="">OpenMP support: disabled</span></div>
<div style="margin:0px;font-size:12px;font-family:Menlo" class="">
<span class="">GPU support:
enabled</span></div>
<div style="margin:0px;font-size:12px;font-family:Menlo" class="">
<span class="">OpenCL support: enabled</span></div>
<div style="margin:0px;font-size:12px;font-family:Menlo" class="">
<span class="">invsqrt routine:
gmx_software_invsqrt(x)</span></div>
<div style="margin:0px;font-size:12px;font-family:Menlo" class="">
<span class="">SIMD instructions: AVX_256</span></div>
<div style="margin:0px;font-size:12px;font-family:Menlo" class="">
<span class="">FFT library:
fftw-3.3.4-sse2-avx</span></div>
<div style="margin:0px;font-size:12px;font-family:Menlo" class="">
<span class="">RDTSCP usage:
enabled</span></div>
<div style="margin:0px;font-size:12px;font-family:Menlo" class="">
<span class="">C++11 compilation: disabled</span></div>
<div style="margin:0px;font-size:12px;font-family:Menlo" class="">
<span class="">TNG support:
enabled</span></div>
<div style="margin:0px;font-size:12px;font-family:Menlo" class="">
<span class="">Tracing support: disabled</span></div>
<div style="margin:0px;font-size:12px;font-family:Menlo" class="">
<span class="">Built on: Thu 16
Jul 2015 09:34:14 BST</span></div>
<div style="margin:0px;font-size:12px;font-family:Menlo" class="">
<span class="">Built by: carlo@
[CMAKE]</span></div>
<div style="margin:0px;font-size:12px;font-family:Menlo" class="">
<span class="">Build OS/arch: Darwin 13.4.0
x86_64</span></div>
<div style="margin:0px;font-size:12px;font-family:Menlo" class="">
<span class="">Build CPU vendor: GenuineIntel</span></div>
<div style="margin:0px;font-size:12px;font-family:Menlo" class="">
<span class="">Build CPU brand: Intel(R) Xeon(R) CPU
E5-1650 v2 @ 3.50GHz</span></div>
<div style="margin:0px;font-size:12px;font-family:Menlo" class="">
<span class="">Build CPU family: 6 Model: 62
Stepping: 4</span></div>
<div style="margin:0px;font-size:12px;font-family:Menlo" class="">
<span class="">Build CPU features: aes apic avx clfsh cmov cx8 cx16
f16c htt lahf_lm mmx msr nonstop_tsc pcid pclmuldq pdcm pdpe1gb
popcnt pse rdrnd rdtscp sse2 sse3 sse4.1 sse4.2 ssse3 tdt
x2apic</span></div>
<div style="margin:0px;font-size:12px;font-family:Menlo" class="">
<span class="">C compiler:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc
Clang 6.0.0.6000057</span></div>
<div style="margin:0px;font-size:12px;font-family:Menlo" class="">
<span class="">C compiler flags: -mavx
-Wall -Wno-unused -Wunused-value -Wunused-parameter
-Wno-unknown-pragmas -O3 -DNDEBUG</span></div>
<div style="margin:0px;font-size:12px;font-family:Menlo" class="">
<span class="">C++ compiler:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++
Clang 6.0.0.6000057</span></div>
<div style="margin:0px;font-size:12px;font-family:Menlo" class="">
<span class="">C++ compiler flags: -mavx -Wextra
-Wno-missing-field-initializers -Wpointer-arith -Wall
-Wno-unused-function -Wno-unknown-pragmas -O3
-DNDEBUG</span></div>
<div style="margin:0px;font-size:12px;font-family:Menlo" class="">
<span class="">Boost version: 1.58.0
(external)</span></div>
<div style="margin:0px;font-size:12px;font-family:Menlo" class="">
<span class="">OpenCL include dir:
/System/Library/Frameworks/OpenCL.framework</span></div>
<div style="margin:0px;font-size:12px;font-family:Menlo" class="">
<span class="">OpenCL library:
/System/Library/Frameworks/OPENCL.framework</span></div>
<div style="margin:0px;font-size:12px;font-family:Menlo" class="">
<span class="">OpenCL version: 1.2</span></div>
<div style="margin:0px;font-size:12px;font-family:Menlo;min-height:14px" class=""><span class=""><br class=""></span></div>
<div style="margin:0px;font-size:12px;font-family:Menlo;min-height:14px" class=""><span class=""><br class=""></span></div>
<div style="margin:0px;font-size:12px;font-family:Menlo" class="">
<span class="">Running on 1 node with total 12 logical cores, 2
compatible GPUs</span></div>
<div style="margin:0px;font-size:12px;font-family:Menlo" class="">
<span class="">Hardware detected:</span></div>
<div style="margin:0px;font-size:12px;font-family:Menlo" class="">
<span class=""> CPU info:</span></div>
<div style="margin:0px;font-size:12px;font-family:Menlo" class="">
<span class=""> Vendor: GenuineIntel</span></div>
<div style="margin:0px;font-size:12px;font-family:Menlo" class="">
<span class=""> Brand: Intel(R) Xeon(R) CPU
E5-1650 v2 @ 3.50GHz</span></div>
<div style="margin:0px;font-size:12px;font-family:Menlo" class="">
<span class=""> Family: 6 model: 62
stepping: 4</span></div>
<div style="margin:0px;font-size:12px;font-family:Menlo" class="">
<span class=""> CPU features: aes apic avx clfsh cmov
cx8 cx16 f16c htt lahf_lm mmx msr nonstop_tsc pcid pclmuldq pdcm
pdpe1gb popcnt pse rdrnd rdtscp sse2 sse3 sse4.1 sse4.2 ssse3 tdt
x2apic</span></div>
<div style="margin:0px;font-size:12px;font-family:Menlo" class="">
<span class=""> SIMD instructions most likely to fit
this hardware: AVX_256</span></div>
<div style="margin:0px;font-size:12px;font-family:Menlo" class="">
<span class=""> SIMD instructions selected at GROMACS
compile time: AVX_256</span></div>
<div style="margin:0px;font-size:12px;font-family:Menlo" class="">
<span class=""> GPU info:</span></div>
<div style="margin:0px;font-size:12px;font-family:Menlo" class="">
<span class=""> Number of GPUs detected:
2</span></div>
<div style="margin:0px;font-size:12px;font-family:Menlo" class="">
<span class=""> #0: name: AMD Radeon HD - FirePro D500
Compute Engine, vendor: AMD, device version: OpenCL 1.2 , stat:
compatible</span></div>
<div style="margin:0px;font-size:12px;font-family:Menlo" class="">
<span class=""> #1: name: AMD Radeon HD - FirePro D500
Compute Engine, vendor: AMD, device version: OpenCL 1.2 , stat:
compatible</span></div>
<div style="margin:0px;font-size:12px;font-family:Menlo" class="">
<span class=""><br class=""></span></div>
<div style="margin:0px;font-size:12px;font-family:Menlo" class="">
<span class="">Carlo</span></div>
<div style="margin:0px;font-size:12px;font-family:Menlo" class="">
<span class=""><br class=""></span></div>
<div style="margin:0px;font-size:12px;font-family:Menlo" class="">
<span class=""><br class=""></span></div>
<div style="margin:0px;font-size:12px;font-family:Menlo" class="">
<span class=""><br class=""></span></div>
<div class=""><span class="">On 16 Jul 2015, at 18:59, Erik Lindahl
<<a href="mailto:erik.lindahl@gmail.com" target="_blank" class="">erik.lindahl@gmail.com</a>> wrote:</span></div>
<span class=""><br class=""></span>
<div style="font-family:Helvetica,Arial;font-size:13px;margin:0px" class=""><span class="">Hi Carlo,</span></div>
<div style="font-family:Helvetica,Arial;font-size:13px;margin:0px" class=""><span class=""><br class=""></span></div>
<div style="font-family:Helvetica,Arial;font-size:13px;margin:0px" class=""><span class="">IIRC, I managed to get it working on a Mac
Pro with AMD Firepro D300 GPUs. Please post the contents of the top
of your logfile (where it says everything about the compilers &
config), and mention what hardware you tried it on - then we’ll see
if we can reproduce it.</span></div>
<div style="font-family:Helvetica,Arial;font-size:13px;margin:0px" class=""><span class=""><br class=""></span></div>
<div style="font-family:Helvetica,Arial;font-size:13px;margin:0px" class=""><span class="">Cheers,</span></div>
<div style="font-family:Helvetica,Arial;font-size:13px;margin:0px" class=""><span class=""><br class=""></span></div>
<div style="font-family:Helvetica,Arial;font-size:13px;margin:0px" class=""><span class="">Erik</span></div>
<span class=""><br class="">
From: Carlo Camilloni <a href="mailto:carlo.camilloni@gmail.com" target="_blank" class=""><carlo.camilloni@gmail.com></a><br class="">
Reply: <span class=""><a href="mailto:gmx-developers@gromacs.org" target="_blank" class="">gmx-developers@gromacs.org</a></span> <a href="mailto:gmx-developers@gromacs.org" target="_blank" class=""><gmx-developers@gromacs.org>></a><br class="">
Date: 16 Jul 2015 at 18:52:21<br class="">
To: <span class=""><a href="mailto:gromacs.org_gmx-developers@maillist.sys.kth.se" target="_blank" class="">gromacs.org_gmx-developers@maillist.sys.kth.se</a></span>
<a href="mailto:gromacs.org_gmx-developers@maillist.sys.kth.se" target="_blank" class=""><gromacs.org_gmx-developers@maillist.sys.kth.se>></a><br class="">
Subject: Re: [gmx-developers] gromacs 5.1rc1 OpenCL problem
with Parrinello-Rahman<br class="">
<br class=""></span>
<div class=""></div>
Hi,<br class="">
<br class="">
I tested the OpenCL kernel on my macbook (nvidia gpu) and here it
produces the correct forces,<br class="">
so it could be a problem related to amd+osx, or maybe to some
specific compiler/os version<br class="">
<br class="">
Carlo<br class="">
<br class="">
<br class="">
> On 15 Jul 2015, at 17:42, Carlo Camilloni <<a href="mailto:carlo.camilloni@gmail.com" target="_blank" class="">carlo.camilloni@gmail.com</a>> wrote:<br class="">
><br class="">
> Hi,<br class="">
><br class="">
> these are the tests that fail:<br class="">
><br class="">
> FAILED. Check checkpot.out (12 errors), checkforce.out (3516
errors) file(s) in dd121 for dd121<br class="">
> FAILED. Check checkpot.out (10 errors), checkforce.out (4027
errors) file(s) in nbnxn-energy-groups for
nbnxn-energy-groups<br class="">
> FAILED. Check checkpot.out (26 errors), checkforce.out (2998
errors) file(s) in nbnxn-free-energy for
nbnxn-free-energy<br class="">
> FAILED. Check checkpot.out (26 errors), checkforce.out (2998
errors) file(s) in nbnxn-free-energy-vv for
nbnxn-free-energy-vv<br class="">
> FAILED. Check checkpot.out (11 errors), checkforce.out (4039
errors) file(s) in nbnxn-ljpme-geometric for
nbnxn-ljpme-geometric<br class="">
> FAILED. Check checkpot.out (14 errors), checkforce.out (52
errors) file(s) in nbnxn-ljpme-LB for nbnxn-ljpme-LB<br class="">
> FAILED. Check checkpot.out (14 errors), checkforce.out (52
errors) file(s) in nbnxn-ljpme-LB-geometric for
nbnxn-ljpme-LB-geometric<br class="">
> FAILED. Check checkpot.out (10 errors), checkforce.out (4029
errors) file(s) in nbnxn-vdw-force-switch for
nbnxn-vdw-force-switch<br class="">
> FAILED. Check checkpot.out (10 errors), checkforce.out (4032
errors) file(s) in nbnxn-vdw-potential-switch for
nbnxn-vdw-potential-switch<br class="">
> FAILED. Check checkpot.out (4 errors), checkforce.out (250
errors) file(s) in nbnxn-vdw-potential-switch-argon for
nbnxn-vdw-potential-switch-argon<br class="">
> FAILED. Check checkpot.out (10 errors), checkforce.out (4027
errors) file(s) in nbnxn_pme for nbnxn_pme<br class="">
> FAILED. Check checkpot.out (10 errors), checkforce.out (4027
errors) file(s) in nbnxn_pme_order5 for nbnxn_pme_order5<br class="">
> FAILED. Check checkpot.out (10 errors), checkforce.out (4027
errors) file(s) in nbnxn_pme_order6 for nbnxn_pme_order6<br class="">
> FAILED. Check checkpot.out (9 errors), checkforce.out (4028
errors) file(s) in nbnxn_rf for nbnxn_rf<br class="">
> FAILED. Check checkpot.out (2 errors), checkforce.out (4
errors) file(s) in nbnxn_rzero for nbnxn_rzero<br class="">
> FAILED. Check mdrun.out, md.log file(s) in nbnxn_vsite for
nbnxn_vsite<br class="">
> FAILED. Check checkpot.out (13 errors), checkforce.out (15512
errors) file(s) in octahedron for octahedron<br class="">
> FAILED. Check mdrun.out, md.log file(s) in position-restraints
for position-restraints<br class="">
> FAILED. Check mdrun.out, md.log file(s) in pull_constraint for
pull_constraint<br class="">
> FAILED. Check checkpot.out (10 errors), checkforce.out (4021
errors) file(s) in pull_cylinder for pull_cylinder<br class="">
> FAILED. Check checkpot.out (11 errors), checkforce.out (39054
errors) file(s) in swap_x for swap_x<br class="">
> FAILED. Check checkpot.out (11 errors), checkforce.out (39053
errors) file(s) in swap_y for swap_y<br class="">
> FAILED. Check checkpot.out (12 errors), checkforce.out (39054
errors) file(s) in swap_z for swap_z<br class="">
> 23 out of 60 complex tests FAILED<br class="">
> FAILED. Check mdrun.out, md.log file(s) in expanded for
expanded<br class="">
> FAILED. Check mdrun.out, md.log file(s) in transformAtoB for
transformAtoB<br class="">
> 2 out of 10 freeenergy tests FAILED<br class="">
><br class="">
><br class="">
> Carlo<br class="">
><br class="">
><br class="">
>><br class="">
>><br class="">
>> Message: 4<br class="">
>> Date: Wed, 15 Jul 2015 15:35:13 +0000<br class="">
>> From: Mark Abraham <<a href="mailto:mark.j.abraham@gmail.com" target="_blank" class="">mark.j.abraham@gmail.com</a>><br class="">
>> To: <a href="mailto:gmx-developers@gromacs.org" target="_blank" class="">gmx-developers@gromacs.org</a>,<br class="">
>> <a href="mailto:gromacs.org_gmx-developers@maillist.sys.kth.se" target="_blank" class="">gromacs.org_gmx-developers@maillist.sys.kth.se</a><br class="">
>> Subject: Re: [gmx-developers] gromacs 5.1rc1 OpenCL
problem with<br class="">
>> Parrinello-Rahman<br class="">
>> Message-ID:<br class="">
>> <<a href="mailto:CAMNuMATveVRRyBBwn312xrY+w3M7deC2Hs3A7PZnaeugkw+VVA@mail.gmail.com" target="_blank" class="">CAMNuMATveVRRyBBwn312xrY+w3M7deC2Hs3A7PZnaeugkw+VVA@mail.gmail.com</a>><br class="">
>> Content-Type: text/plain; charset="utf-8"<br class="">
>><br class="">
>> Hi,<br class="">
>><br class="">
>> Thanks. If a difference of that magnitude can be seen,
then it should also<br class="">
>> show up when running the regressiontests (e.g.
cmake<br class="">
>> -DREGRESSIONTEST_DOWNLOAD=on and then make check) as a
failure<br class="">
>> of complex/nbnxn-ljpme-LB (which is the only P-R test that
can run on the<br class="">
>> GPU). If other tests fail, then the problem is actually
more widespread.<br class="">
>><br class="">
>> It may be that there is some issue with some part of the
Mac+clang+OpenCL<br class="">
>> stack - we didn't target it during development, and at the
last minute when<br class="">
>> Erik was unexpectedly able to get it to compile. I don't
know if he got<br class="">
>> tests to pass. Erik?<br class="">
>><br class="">
>> Mark<br class="">
>><br class="">
>> On Wed, Jul 15, 2015 at 5:22 PM Carlo Camilloni
<<a href="mailto:carlo.camilloni@gmail.com" target="_blank" class="">carlo.camilloni@gmail.com</a>><br class="">
>> wrote:<br class="">
>><br class="">
>>><br class="">
>>> Dear Mark and Szilard,<br class="">
>>><br class="">
>>> thanks for your answer. I filed a bug in redmine but
in the meantime I was<br class="">
>>> running more tests and I am a bit scared by what I
found:<br class="">
>>><br class="">
>>> what I have done is the following I have performed a
single step run with<br class="">
>>> gmx51-rc1 compiled with cuda, again clang and so
on<br class="">
>>> and compared the forces on the first step with -nb cpu
or not (I am using<br class="">
>>> -pforce 1), the forces are identical:<br class="">
>>><br class="">
>>> ie.:<br class="">
>>><br class="">
>>> cuda-gpu<br class="">
>>><br class="">
>>> step 0 atom 1 x 3.940 5.612 2.226 force
1.90839e+03<br class="">
>>> step 0 atom 2 x 3.852 5.659 2.211 force
4.24845e+02<br class="">
>>> step 0 atom 3 x 3.979 5.665 2.303 force
6.89472e+02<br class="">
>>> step 0 atom 4 x 3.992 5.610 2.139 force
7.42053e+02<br class="">
>>><br class="">
>>><br class="">
>>> cpu:<br class="">
>>><br class="">
>>> step 0 atom 1 x 3.940 5.612 2.226 force
1.90839e+03<br class="">
>>> step 0 atom 2 x 3.852 5.659 2.211 force
4.24845e+02<br class="">
>>> step 0 atom 3 x 3.979 5.665 2.303 force
6.89472e+02<br class="">
>>> step 0 atom 4 x 3.992 5.610 2.139 force
7.42053e+02<br class="">
>>><br class="">
>>> if I do the same test on the version compiled with
OpenCL<br class="">
>>><br class="">
>>> cpu:<br class="">
>>><br class="">
>>> (the former are done on my macbook pro avx2_256 this
latter on a MacPro<br class="">
>>> avx_256, this should<br class="">
>>> explain the small differences in the forces)<br class="">
>>><br class="">
>>> step 0 atom 1 x 3.940 5.612 2.226 force
1.90838e+03<br class="">
>>> step 0 atom 2 x 3.852 5.659 2.211 force
4.24848e+02<br class="">
>>> step 0 atom 3 x 3.979 5.665 2.303 force
6.89470e+02<br class="">
>>> step 0 atom 4 x 3.992 5.610 2.139 force
7.42043e+02<br class="">
>>><br class="">
>>> opencl-gpu:<br class="">
>>> step 0 atom 1 x 3.940 5.612 2.226 force
1.48597e+03<br class="">
>>> step 0 atom 2 x 3.852 5.659 2.211 force
6.26942e+02<br class="">
>>> step 0 atom 3 x 3.979 5.665 2.303 force
8.44032e+02<br class="">
>>> step 0 atom 4 x 3.992 5.610 2.139 force
7.92786e+02<br class="">
>>><br class="">
>>> I am afraid there is something wrong in OpenCL
kernels.<br class="">
>>><br class="">
>>> I am using the topol-nvt-nogen.tpr I have uploaded on
redmine.<br class="">
>>><br class="">
>>> Best,<br class="">
>>> Carlo<br class="">
>>><br class="">
>>><br class="">
>>><br class="">
>>> --<br class="">
>>> Gromacs Developers mailing list<br class="">
>>><br class="">
>>> * Please search the archive at<br class="">
>>> <a href="http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List" target="_blank" class="">http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List</a>
before<br class="">
>>> posting!<br class="">
>>><br class="">
>>> * Can't post? Read <a href="http://www.gromacs.org/Support/Mailing_Lists" target="_blank" class="">http://www.gromacs.org/Support/Mailing_Lists</a><br class="">
>>><br class="">
>>> * For (un)subscribe requests visit<br class="">
>>> <a href="https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers" target="_blank" class="">https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers</a><br class="">
>>> or send a mail to <a href="mailto:gmx-developers-request@gromacs.org" target="_blank" class="">gmx-developers-request@gromacs.org</a>.<br class="">
>>><br class="">
>> -------------- next part --------------<br class="">
>> An HTML attachment was scrubbed...<br class="">
>> URL: <<a href="http://maillist.sys.kth.se/pipermail/gromacs.org_gmx-developers/attachments/20150715/fb1d4126/attachment.html" target="_blank" class="">http://maillist.sys.kth.se/pipermail/gromacs.org_gmx-developers/attachments/20150715/fb1d4126/attachment.html</a>><br class="">
>><br class="">
>> ------------------------------<br class="">
>><br class="">
>> --<br class="">
>> Gromacs Developers mailing list<br class="">
>><br class="">
>> * Please search the archive at <a href="http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List" target="_blank" class="">http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List</a>
before posting!<br class="">
>><br class="">
>> * Can't post? Read <a href="http://www.gromacs.org/Support/Mailing_Lists" target="_blank" class="">http://www.gromacs.org/Support/Mailing_Lists</a><br class="">
>><br class="">
>> * For (un)subscribe requests visit<br class="">
>> <a href="https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers" target="_blank" class="">https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers</a>
or send a mail to <a href="mailto:gmx-developers-request@gromacs.org" target="_blank" class="">gmx-developers-request@gromacs.org</a>.<br class="">
>><br class="">
>> End of <a href="http://gromacs.org/" target="_blank" class="">gromacs.org</a>_gmx-developers Digest, Vol 135, Issue
17<br class="">
>>
***********************************************************<br class="">
><br class="">
<br class="">
--<br class="">
Gromacs Developers mailing list<br class="">
<br class="">
* Please search the archive at <a href="http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List" target="_blank" class="">http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List</a>
before posting!<br class="">
<br class="">
* Can't post? Read <a href="http://www.gromacs.org/Support/Mailing_Lists" target="_blank" class="">http://www.gromacs.org/Support/Mailing_Lists</a><br class="">
<br class="">
* For (un)subscribe requests visit<br class="">
<a href="https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers" target="_blank" class="">https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers</a>
or send a mail to <a href="mailto:gmx-developers-request@gromacs.org" target="_blank" class="">gmx-developers-request@gromacs.org</a>.<br class="">
<div class="">
<div style="font-family:helvetica,arial;font-size:13px" class="">
-- <br class="">
<span style="font-family:'helvetica Neue',helvetica" class="">Erik
Lindahl <</span><a href="mailto:erik.lindahl@gmail.com" style="font-family:'helvetica Neue',helvetica" target="_blank" class="">erik.lindahl@gmail.com</a><span style="font-family:'helvetica Neue',helvetica" class="">> </span><br style="font-family:'helvetica Neue',helvetica" class="">
<span style="font-family:'helvetica Neue',helvetica" class="">Professor of Biophysics, Dept. Biochemistry & Biophysics,
Stockholm </span><span style="font-family:'helvetica Neue',helvetica" class="">University </span><br style="font-family:'helvetica Neue',helvetica" class="">
<span style="font-family:'helvetica Neue',helvetica" class="">Professor of Theoretical biophysics, Dept. Theoretical Physics,
Royal Inst. </span><span style="font-family:'helvetica Neue',helvetica" class="">Technology </span><br style="font-family:'helvetica Neue',helvetica" class="">
<span style="font-family:'helvetica Neue',helvetica" class="">Science for Life Laboratory, Box 1031, 17121 Solna,
Sweden</span></div>
</div>
<br class=""></blockquote>
<div class="">
<div style="font-family:helvetica,arial;font-size:13px" class="">
-- <br class="">
<span style="font-family:'helvetica Neue',helvetica" class="">Erik
Lindahl <</span><a href="mailto:erik.lindahl@gmail.com" style="font-family:'helvetica Neue',helvetica" target="_blank" class="">erik.lindahl@gmail.com</a><span style="font-family:'helvetica Neue',helvetica" class="">> </span><br style="font-family:'helvetica Neue',helvetica" class="">
<span style="font-family:'helvetica Neue',helvetica" class="">Professor of Biophysics, Dept. Biochemistry & Biophysics,
Stockholm </span><span style="font-family:'helvetica Neue',helvetica" class="">University </span><br style="font-family:'helvetica Neue',helvetica" class="">
<span style="font-family:'helvetica Neue',helvetica" class="">Professor of Theoretical biophysics, Dept. Theoretical Physics,
Royal Inst. </span><span style="font-family:'helvetica Neue',helvetica" class="">Technology </span><br style="font-family:'helvetica Neue',helvetica" class="">
<span style="font-family:'helvetica Neue',helvetica" class="">Science for Life Laboratory, Box 1031, 17121 Solna,
Sweden</span></div>
</div>
</div>
--<br class="">
Gromacs Developers mailing list<br class="">
<br class="">
* Please search the archive at <a href="http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List" rel="noreferrer" target="_blank" class="">http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List</a>
before posting!<br class="">
<br class="">
* Can't post? Read <a href="http://www.gromacs.org/Support/Mailing_Lists" rel="noreferrer" target="_blank" class="">http://www.gromacs.org/Support/Mailing_Lists</a><br class="">
<br class="">
* For (un)subscribe requests visit<br class="">
<a href="https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers" rel="noreferrer" target="_blank" class="">https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers</a>
or send a mail to <a href="mailto:gmx-developers-request@gromacs.org" target="_blank" class="">gmx-developers-request@gromacs.org</a>.</blockquote>
</div>
</div>
</blockquote>
</div>
<br class=""></div>
</div>
</div>
</blockquote>
</div>
<br class=""></div>
</div></div></span></blockquote> <div id="bloop_sign_1437430292630532864" class="bloop_sign"><div style="font-family:helvetica,arial;font-size:13px">-- <br><span style="font-family: 'helvetica Neue', helvetica;">Erik Lindahl <</span><a href="mailto:erik.lindahl@gmail.com" style="font-family: 'helvetica Neue', helvetica;">erik.lindahl@gmail.com</a><span style="font-family: 'helvetica Neue', helvetica;">> </span><br style="font-family: 'helvetica Neue', helvetica;"><span style="font-family: 'helvetica Neue', helvetica;">Professor of Biophysics, Dept. Biochemistry & Biophysics, Stockholm </span><span style="font-family: 'helvetica Neue', helvetica;">University </span><br style="font-family: 'helvetica Neue', helvetica;"><span style="font-family: 'helvetica Neue', helvetica;">Professor of Theoretical biophysics, Dept. Theoretical Physics, Royal Inst. </span><span style="font-family: 'helvetica Neue', helvetica;">Technology </span><br style="font-family: 'helvetica Neue', helvetica;"><span style="font-family: 'helvetica Neue', helvetica;">Science for Life Laboratory, Box 1031, 17121 Solna, Sweden</span></div></div></body></html>