<div dir="ltr"><div>Hi,</div><div><br></div><div>github is just a mirror, and it looks like something stopped pushing the tags to it about a year ago. <a href="https://gitlab.com/gromacs/gromacs">https://gitlab.com/gromacs/gromacs</a> is the (brand new!) primary, so use that.</div><div><br></div><div>mark<br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, 25 Mar 2020 at 10:41, jan <<a href="mailto:rtm443x@googlemail.com">rtm443x@googlemail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hi,<br>
there is no tag I can find that has '2020' in it.<br>
I saw the v20xx.y pattern, with 'v2019.1' being the most recent<br>
looking, but otherwise no. This is the code cloned from github BTW,<br>
just in case there's another git repo.<br>
Could I ask that someone just check and see if I've missed it.<br>
<br>
Regarding the WSL, thanks I'll look into that!<br>
<br>
cheers<br>
<br>
jan<br>
<br>
<br>
<br>
On 24/03/2020, Szilárd Páll <<a href="mailto:pall.szilard@gmail.com" target="_blank">pall.szilard@gmail.com</a>> wrote:<br>
> Hi,<br>
><br>
> On Tue, Mar 24, 2020 at 11:24 PM jan <<a href="mailto:rtm443x@googlemail.com" target="_blank">rtm443x@googlemail.com</a>> wrote:<br>
><br>
>> Hi all,<br>
>> I got the git repo from <<a href="https://github.com/gromacs/gromacs" rel="noreferrer" target="_blank">https://github.com/gromacs/gromacs</a>>, used<br>
>> HEAD and started compiling.<br>
>><br>
>> I guess I should use the right tag, bit I could not see a tag that<br>
>> looked like the release the tarball says it is, which is 2020.1 so I<br>
>> just compiled it (with cygwin) even though it's had commits more<br>
>> recently.<br>
>><br>
><br>
> Release maintenance branches are named "release-VERSION", so the<br>
> release-2020 branch is a good stable place to start from.<br>
> Alternatively, releases are tagged with tags prefixed by "v", so v2020.1<br>
> for the 2020.1 release.<br>
><br>
><br>
>> Anyway I hit this:<br>
>><br>
>><br>
>> /cygdrive/c/Users/jan/Desktop/gromacs_git/gromacs/src/external/thread_mpi/src/tmpi_init.cpp:476:42:<br>
>> error: ‘strdup’ was not declared in this scope; did you mean ‘strcmp’?<br>
>> 476 | threads[i].argv[j] = strdup( (*argv)[j] );<br>
>><br>
>> which comes from<br>
>><br>
>> #if !(defined( _WIN32 ) || defined( _WIN64 ) )<br>
>> threads[i].argv[j] = strdup( (*argv)[j] ); // <--- this line<br>
>> #else<br>
>> threads[i].argv[j] = _strdup( (*argv)[j] );<br>
>> #endif<br>
>><br>
>> so it looks like it didn't pick up I'm running in windows.<br>
>> I'm not so familiar with make (or these build tools generally), but I<br>
>> poked around in the makefile (generated by cmake I assume) but could<br>
>> not see where to add the symbol for _WIN64<br>
>><br>
><br>
> AFAICT the _WIN64 macro should be defined only if code uses 64-bit Windows<br>
> API, see<br>
> <a href="https://www.cygwin.com/faq.html#faq.programming.64bitporting-cygwin64" rel="noreferrer" target="_blank">https://www.cygwin.com/faq.html#faq.programming.64bitporting-cygwin64</a><br>
><br>
> As a side-note, there are a few reports/tutorials on compiling GROMACS on<br>
> cygwin I know of, but may be a bit dated:<br>
> <a href="http://tmacchant3.starfree.jp/gromacs/win/notes_cygwin.html" rel="noreferrer" target="_blank">http://tmacchant3.starfree.jp/gromacs/win/notes_cygwin.html</a><br>
> <a href="http://cdlc.cau.ac.kr/Gromacs/966" rel="noreferrer" target="_blank">http://cdlc.cau.ac.kr/Gromacs/966</a><br>
><br>
> Since I understand it has been compiled and tested under MSVC2017<br>
>> I'll try that tomorrow.<br>
>><br>
><br>
> Native MSVC should be easier (IIRC fftw might require a separate download),<br>
> but a possibly even easier option is to just use WSL. After installing the<br>
> Linux subsystem and built toolchain, you should be able to just follow the<br>
> regular user guide installation procedure.<br>
><br>
> Cheers,<br>
> --<br>
> Szilárd<br>
><br>
><br>
>> cheers<br>
>><br>
>> jan<br>
>><br>
>><br>
>><br>
>> On 24/03/2020, Paul Bauer <<a href="mailto:paul.bauer.q@gmail.com" target="_blank">paul.bauer.q@gmail.com</a>> wrote:<br>
>> > Hello,<br>
>> ><br>
>> > I agree with Mark that the webinars should be a good start to have an<br>
>> idea<br>
>> > a out the code.<br>
>> ><br>
>> > Concerning the error you are getting, this shouldn't happen if you work<br>
>> and<br>
>> > build from a git repository. But it is still something I think should<br>
>> > be<br>
>> > fixed (especially because I have been the one pushing for it against<br>
>> Mark's<br>
>> > objections)<br>
>> ><br>
>> > Cheers<br>
>> ><br>
>> > Paul<br>
>> ><br>
>> > On Tue, 24 Mar 2020, 19:50 Mark Abraham, <<a href="mailto:mark.j.abraham@gmail.com" target="_blank">mark.j.abraham@gmail.com</a>><br>
>> wrote:<br>
>> ><br>
>> >> Hi gmx developers!<br>
>> >><br>
>> >><br>
>> >> On Tue, 24 Mar 2020 at 15:29, jan <<a href="mailto:rtm443x@googlemail.com" target="_blank">rtm443x@googlemail.com</a>> wrote:<br>
>> >><br>
>> >>> Hi all,<br>
>> >>> given what's said below I need to be clear about where I am.<br>
>> >>> I'm a back-end dev specialising somewhat in SQL/RDBMSs + data<br>
>> >>> management (but not data analysis of any note), with plenty of<br>
>> >>> experience of other languages etc. however I have never done any<br>
>> >>> x86/x64 as such, I normally use high-level languages, I have no<br>
>> >>> experience at all with GPUs and (as already mentioned) my C/C++ is<br>
>> >>> neolithic, and I've no experience with the modern c/c++ build tools.<br>
>> >>> My linux is not great. And the nearest I can get to your kind of<br>
>> >>> mathematics is a background in classical logic ie. not very close at<br>
>> >>> all.<br>
>> >>><br>
>> >>> None of these bother me; all are fixable, but it will take time. I'll<br>
>> >>> need a minimal amount of guidance to get gowing - "read this" is fine<br>
>> >>> but I'll need no handholding, and you don't have time for that<br>
>> >>> anyway.<br>
>> >>><br>
>> >><br>
>> >> Great - you're coming at the questions from a totally different<br>
>> >> perspective, which is healthy for everyone, but that's going to give<br>
>> >> you<br>
>> >> a<br>
>> >> steep learning curve. There's some useful recorded webinars from<br>
>> BioExcel<br>
>> >> given by particularly Paul, Szilard, Carsten, and I over recent years<br>
>> >> that<br>
>> >> are a good starting point for understanding how the code operates at<br>
>> >> run<br>
>> >> time, but you should look for something else for "intro to molecular<br>
>> >> dynamics for non-scientists." There's a bunch of material online - has<br>
>> >> anybody got suggestions?<br>
>> >><br>
>> >> First things first, I need to compile this stuff up.<br>
>> >>> I'm using windows + cygwin - is this the best environment or do you<br>
>> >>> recommend working entirely in Linux?<br>
>> >>><br>
>> >><br>
>> >> We test in CI on Windows + MSVC, so that works fine, but there has<br>
>> >> been<br>
>> >> no<br>
>> >> love on cygwin for about a decade. It's almost certainly fixable, but<br>
>> >> never<br>
>> >> been a priority. So go native one way or the other :-)<br>
>> >><br>
>> >> I've followed the instructions to get it working on cygwin and got<br>
>> >>> stuck. Compilation fails with<br>
>> >>><br>
>> >>> C:\Program Files\Python38\python.exe: can't open file<br>
>> >>><br>
>> >>><br>
>> '/cygdrive/c/Users/jan/Desktop/gromacs/gromacs-2020.1/admin/createFileHash.py':<br>
>> >>> [Errno 2] No such file or directory<br>
>> >>> CMake Error at cmake/gmxGenerateVersionInfoRelease.cmake:115 (file):<br>
>> >>> file failed to open for reading (No such file or directory):<br>
>> >>><br>
>> >>> /cygdrive/c/Users/jan/Desktop/gromacs/gromacs-2020.1/computed_checksum<br>
>> >>><br>
>> >>> computed_checksum indeed doesn't exist. What now?<br>
>> >>><br>
>> >><br>
>> >> Sigh, that's broken implementation of a new feature that I never<br>
>> >> thought<br>
>> >> was worth its cost. Don't know how to fix it.<br>
>> >><br>
>> >> The above is going by<br>
>> >>> <<br>
>> <a href="http://manual.gromacs.org/documentation/current/install-guide/index.html" rel="noreferrer" target="_blank">http://manual.gromacs.org/documentation/current/install-guide/index.html</a><br>
>> >>> ><br>
>> >>><br>
>> >>> Other problem is that untarring the tarball works, however on windows<br>
>> >>> it's natural to use something like 7-zip. This doesn't work and I<br>
>> >>> lost<br>
>> >>> a couple of hours to that (certainly not the fault of the<br>
>> >>> instructions<br>
>> >>> but FYI anyway).<br>
>> >>><br>
>> >>> Comments below too<br>
>> >>><br>
>> >>><br>
>> >>> On 24/03/2020, Mark Abraham <<a href="mailto:mark.j.abraham@gmail.com" target="_blank">mark.j.abraham@gmail.com</a>> wrote:<br>
>> >>> > Hi,<br>
>> >>> ><br>
>> >>> > Jan, the biggest bang-for-buck optimizations relevant to<br>
>> >>> > Folding@Home<br>
>> >>> are<br>
>> >>> > to<br>
>> >>> ><br>
>> >>> > a) offer to build them an OpenCL-enabled GROMACS "core" (ie the<br>
>> >>> > version<br>
>> >>> of<br>
>> >>> > GROMACS that they run, when they run GROMACS). Currently they seem<br>
>> >>> > to<br>
>> >>> run<br>
>> >>> > all GPU jobs with OpenCL and OpenMM, which is nice but leaves a lot<br>
>> of<br>
>> >>> > throughput on the table. The GROMACS OpenCL port is mature and<br>
>> stable,<br>
>> >>> runs<br>
>> >>> > on AMD/NVIDIA/Intel current GPUs, and should present no more<br>
>> >>> > driver/user<br>
>> >>> > problems than their OpenMM one. Their concept of a GPU slot is a<br>
>> >>> > single<br>
>> >>> GPU<br>
>> >>> > accompanied by a single CPU thread/, whereas the GROMACS OpenCL<br>
>> >>> > port<br>
>> >>> would<br>
>> >>> > prefer multiple dedicated cores. That's still better than leaving<br>
>> GPUs<br>
>> >>> > empty if there's not enough OpenMM jobs in the queue, though the<br>
>> >>> > actual<br>
>> >>> > performance will be woeful compared to GROMACS when you give it a<br>
>> >>> healthy<br>
>> >>> > chunk of CPU cores also. Could even be better than OpenMM's GPU<br>
>> >>> > core,<br>
>> >>> > depending how modern that one is, too ;-) The GROMACS CUDA port is<br>
>> >>> better<br>
>> >>> > still (and in 2020 can do a decent job even with only a single CPU<br>
>> >>> core),<br>
>> >>> > but they have made a philosophical choice to use OpenCL only.<br>
>> >>><br>
>> >>> That has to come later when I get up to speed, but carefully noted,<br>
>> >>> thanks.<br>
>> >>><br>
>> >>> > b) update the GROMACS CPU core in F@H because the one used in F@H<br>
>> >>> > is<br>
>> >>> > several years behind and losing the benefit of the hard<br>
>> >>> > optimization<br>
>> >>> work<br>
>> >>> > that we've done in the meantime.<br>
>> >>><br>
>> >>> Why would f@h not do this already??<br>
>> >>><br>
>> >><br>
>> >> Limited resources and priorities for it. It's a science-driven<br>
>> >> project,<br>
>> >> and if the people prepared to do the work want to use not-GROMACS for<br>
>> >> their<br>
>> >> science then that is that...<br>
>> >><br>
>> >> But again, noted.<br>
>> >>><br>
>> >>> > c) demonstrate that they can maintainably and usefully offer more<br>
>> than<br>
>> >>> two<br>
>> >>> > x86 builds of that GROMACS CPU core (GROMACS has lots of SIMD<br>
>> >>> specialized<br>
>> >>> > flavours, but F@H only offers SSE4.1 and basic AVX from those<br>
>> >>> > flavours,<br>
>> >>><br>
>> >>> Yes, thought this might be the case. Definitely worth it for newer<br>
>> >>> chips.<br>
>> >>> However, please note that SIMD performance for later chips do not<br>
>> >>> always mix well with non-SIMD code and can overall *cost* performance<br>
>> >>> <<br>
>> <a href="https://blog.cloudflare.com/on-the-dangers-of-intels-frequency-scaling/" rel="noreferrer" target="_blank">https://blog.cloudflare.com/on-the-dangers-of-intels-frequency-scaling/</a>><br>
>> >>><br>
>> >><br>
>> >> Yes thanks, most of us know ;-) Just updating to add AVX2 would give a<br>
>> >> clear win.<br>
>> >><br>
>> >><br>
>> >>> > which leaves a lot of performance on the table on recent x86 CPUs.<br>
>> >>> > We<br>
>> >>> > already have all the logic needed to work out which pre-built<br>
>> >>> > GROMACS<br>
>> >>> > to<br>
>> >>> > download and run, because we use it in containerized GROMACS builds<br>
>> >>> also.)<br>
>> >>> ><br>
>> >>> > Unfortunately they've never open-sourced any of that, so finding<br>
>> >>> > out<br>
>> >>> where<br>
>> >>> > to start is the first challenge. But that way you'll have a lot<br>
>> >>> > more<br>
>> >>> impact<br>
>> >>> > sooner than you will from profiling GROMACS runs after 30 years of<br>
>> >>> > optimization. ;-)<br>
>> >>><br>
>> >>> I dunno yet. Model tuning is beyond me obviously, but I've seen some<br>
>> >>> stuff in the code that I question WRT optimality. However if it's<br>
>> >>> cold<br>
>> >>> code or all memory bound then I'll be fixing the wrong thing. Time to<br>
>> >>> profile, but need to get it to compile first.<br>
>> >>><br>
>> >><br>
>> >> Memory? What's that? :-D GROMACS memory usage is typically measured in<br>
>> >> megabytes, with sophisticated data-parallelism to keep the working set<br>
>> >> for<br>
>> >> each core down around cache sizes. Obviously you can scale up the<br>
>> problem<br>
>> >> to get out of cache, but the problem sizes that suit interesting<br>
>> >> science<br>
>> >> are comparable with the amount of L3 cache you get on a socket these<br>
>> >> days.<br>
>> >><br>
>> >> There's a big pile of code in the repo that warrants exhaustive<br>
>> >> optimization, and a lot that is used by only a handful of people,<br>
>> >> which<br>
>> >> generally doesn't. It's hard to make a valuable impact in either kind<br>
>> >> of<br>
>> >> place, for different reasons.<br>
>> >><br>
>> >> Mark<br>
>> >><br>
>> >> Happy to take this offline and reduce mailing list clutter.<br>
>> >>><br>
>> >>> cheers<br>
>> >>><br>
>> >>> jan<br>
>> >>><br>
>> >>> ><br>
>> >>> > Mark<br>
>> >>> ><br>
>> >>> > On Mon, 23 Mar 2020 at 14:59, jan <<a href="mailto:rtm443x@googlemail.com" target="_blank">rtm443x@googlemail.com</a>> wrote:<br>
>> >>> ><br>
>> >>> >> Hi,<br>
>> >>> >> I'm a general back-end dev. Given the situation, and folding@home<br>
>> >>> >> using gromacs, I thought I'd poke through the code. I noticed<br>
>> >>> >> something unexpected, and was advised to email it here. in<br>
>> edsam.cpp,<br>
>> >>> >> this:<br>
>> >>> >><br>
>> >>> >><br>
>> >>> >> void do_linacc(rvec* xcoll, t_edpar* edi)<br>
>> >>> >> {<br>
>> >>> >> /* loop over linacc vectors */<br>
>> >>> >> for (int i = 0; i < edi->vecs.linacc.neig; i++)<br>
>> >>> >> {<br>
>> >>> >> /* calculate the projection */<br>
>> >>> >> real proj = projectx(*edi, xcoll,<br>
>> >>> >> edi->vecs.linacc.vec[i]);<br>
>> >>> >><br>
>> >>> >><br>
>> >>> >> /* calculate the correction */<br>
>> >>> >> real preFactor = 0.0;<br>
>> >>> >> if (edi->vecs.linacc.stpsz[i] > 0.0)<br>
>> >>> >> {<br>
>> >>> >> if ((proj - edi->vecs.linacc.refproj[i]) < 0.0)<br>
>> >>> >> {<br>
>> >>> >> preFactor = edi->vecs.linacc.refproj[i] - proj;<br>
>> >>> >> }<br>
>> >>> >> }<br>
>> >>> >> if (edi->vecs.linacc.stpsz[i] < 0.0)<br>
>> >>> >> {<br>
>> >>> >> if ((proj - edi->vecs.linacc.refproj[i]) > 0.0)<br>
>> >>> >> {<br>
>> >>> >> preFactor = edi->vecs.linacc.refproj[i] - proj;<br>
>> >>> >> }<br>
>> >>> >> }<br>
>> >>> >> [...]<br>
>> >>> >><br>
>> >>> >><br>
>> >>> >> In both cases it reaches the same code<br>
>> >>> >><br>
>> >>> >> preFactor = edi->vecs.linacc.refproj[i] - proj<br>
>> >>> >><br>
>> >>> >> That surprised me a bit, is it deliberate? If so it may be the<br>
>> >>> >> code<br>
>> >>> >> can be simplified anyway.<br>
>> >>> >><br>
>> >>> >> That aside, if you're looking for performance I might be able to<br>
>> >>> >> help.<br>
>> >>> >> I don't know the high level stuff *at this point* and my C++ is so<br>
>> >>> >> rusty it creaks, but I can brush that up, do profiling and<br>
>> >>> >> whatnot.<br>
>> >>> >> I'm pretty experience, just not in this area. Speeding things up<br>
>> >>> >> is<br>
>> >>> >> something I've got a track record of (though I usually have a good<br>
>> >>> >> feel for the problem domain first, which I don't here)<br>
>> >>> >><br>
>> >>> >> Would it be of some value for me to try getting more speed? If so,<br>
>> >>> >> first thing I'd need is to get this running under cygwin, which<br>
>> >>> >> I'm<br>
>> >>> >> struggling with.<br>
>> >>> >><br>
>> >>> >> regards<br>
>> >>> >><br>
>> >>> >> jan<br>
>> >>> >> --<br>
>> >>> >> Gromacs Developers mailing list<br>
>> >>> >><br>
>> >>> >> * Please search the archive at<br>
>> >>> >> <a href="http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List" rel="noreferrer" target="_blank">http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List</a><br>
>> >>> before<br>
>> >>> >> posting!<br>
>> >>> >><br>
>> >>> >> * Can't post? Read <a href="http://www.gromacs.org/Support/Mailing_Lists" rel="noreferrer" target="_blank">http://www.gromacs.org/Support/Mailing_Lists</a><br>
>> >>> >><br>
>> >>> >> * For (un)subscribe requests visit<br>
>> >>> >><br>
>> >>><br>
>> <a href="https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers" rel="noreferrer" target="_blank">https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers</a><br>
>> >>> >> or send a mail to <a href="mailto:gmx-developers-request@gromacs.org" target="_blank">gmx-developers-request@gromacs.org</a>.<br>
>> >>> >><br>
>> >>> ><br>
>> >>> --<br>
>> >>> Gromacs Developers mailing list<br>
>> >>><br>
>> >>> * Please search the archive at<br>
>> >>> <a href="http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List" rel="noreferrer" target="_blank">http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List</a><br>
>> before<br>
>> >>> posting!<br>
>> >>><br>
>> >>> * Can't post? Read <a href="http://www.gromacs.org/Support/Mailing_Lists" rel="noreferrer" target="_blank">http://www.gromacs.org/Support/Mailing_Lists</a><br>
>> >>><br>
>> >>> * For (un)subscribe requests visit<br>
>> >>><br>
>> <a href="https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers" rel="noreferrer" target="_blank">https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers</a><br>
>> >>> or send a mail to <a href="mailto:gmx-developers-request@gromacs.org" target="_blank">gmx-developers-request@gromacs.org</a>.<br>
>> >>><br>
>> >> --<br>
>> >> Gromacs Developers mailing list<br>
>> >><br>
>> >> * Please search the archive at<br>
>> >> <a href="http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List" rel="noreferrer" target="_blank">http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List</a><br>
>> >> before<br>
>> >> posting!<br>
>> >><br>
>> >> * Can't post? Read <a href="http://www.gromacs.org/Support/Mailing_Lists" rel="noreferrer" target="_blank">http://www.gromacs.org/Support/Mailing_Lists</a><br>
>> >><br>
>> >> * For (un)subscribe requests visit<br>
>> >> <a href="https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers" rel="noreferrer" target="_blank">https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers</a><br>
>> >> or send a mail to <a href="mailto:gmx-developers-request@gromacs.org" target="_blank">gmx-developers-request@gromacs.org</a>.<br>
>> ><br>
>> --<br>
>> Gromacs Developers mailing list<br>
>><br>
>> * Please search the archive at<br>
>> <a href="http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List" rel="noreferrer" target="_blank">http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List</a> before<br>
>> posting!<br>
>><br>
>> * Can't post? Read <a href="http://www.gromacs.org/Support/Mailing_Lists" rel="noreferrer" target="_blank">http://www.gromacs.org/Support/Mailing_Lists</a><br>
>><br>
>> * For (un)subscribe requests visit<br>
>> <a href="https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers" rel="noreferrer" target="_blank">https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers</a><br>
>> or send a mail to <a href="mailto:gmx-developers-request@gromacs.org" target="_blank">gmx-developers-request@gromacs.org</a>.<br>
><br>
-- <br>
Gromacs Developers mailing list<br>
<br>
* Please search the archive at <a href="http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List" rel="noreferrer" target="_blank">http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List</a> before posting!<br>
<br>
* Can't post? Read <a href="http://www.gromacs.org/Support/Mailing_Lists" rel="noreferrer" target="_blank">http://www.gromacs.org/Support/Mailing_Lists</a><br>
<br>
* For (un)subscribe requests visit<br>
<a href="https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers" rel="noreferrer" target="_blank">https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers</a> or send a mail to <a href="mailto:gmx-developers-request@gromacs.org" target="_blank">gmx-developers-request@gromacs.org</a>.</blockquote></div>