<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<div class="moz-cite-prefix">On a second note, I just checked our CI
settings and the most time we keep artifacts is for 1 week, so
this can't be the issue. It is more likely that pipeline artifacts
are being stored indefinitely, and that our docker images take up
a lot of space.</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">But I'm open to suggestions here.<br>
<br>
Cheers</div>
<div class="moz-cite-prefix"><br>
Paul<br>
</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">On 30/09/2022 09:22, Paul bauer wrote:<br>
</div>
<blockquote type="cite"
cite="mid:d4759a18-9360-1bf9-2cc5-ab2965b1ccb9@gmail.com">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<div class="moz-cite-prefix">Yes, I concur that we should reduce
the time the artifacts are available for most jobs. I'll upload
a change for that now.<br>
<br>
Cheers<br>
<br>
Paul<br>
</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">On 29/09/2022 18:33, Erik Lindahl
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:cd2bc591-d8a2-488d-a5cc-6b24c0f53b2c@Spark">
<meta http-equiv="content-type" content="text/html;
charset=UTF-8">
<title></title>
<div name="messageBodySection">
<div dir="auto">I suggest we automatically expire everything
in 30 days, just like we now do for cache.<br>
<br>
I guess it's always trivial to rerun a pipeline if somebody
wants the artifacts?<br>
<br>
Cheers,<br>
<br>
Erik</div>
</div>
<div name="messageSignatureSection"><br>
--
<div dir="auto">Erik Lindahl, Professor of Biophysics</div>
<div dir="auto">Science for Life Laboratory, Stockholm
University & KTH</div>
</div>
<div name="messageReplySection">On 29 Sep 2022 at 17:30 +0100,
Eric Irrgang <a class="moz-txt-link-rfc2396E"
href="mailto:ericirrgang@gmail.com" moz-do-not-send="true"><ericirrgang@gmail.com></a>,
wrote:<br>
<blockquote type="cite" style="border-left-color: grey;
border-left-width: thin; border-left-style: solid; margin:
5px 5px;padding-left: 10px;">Hi Devs.<br>
<br>
According to <a class="moz-txt-link-freetext"
href="https://gitlab.com/gromacs/gromacs/-/usage_quotas"
moz-do-not-send="true">https://gitlab.com/gromacs/gromacs/-/usage_quotas</a>
we are almost at 5TB of retained job artifacts, and growing.<br>
<br>
Do we have any idea of how big that can get before it
becomes a problem?<br>
<br>
I notice that we have selected the option to override the
`expire_in` job artifact parameter for successful pipelines
and to instead retain artifacts for the most recent commit
on each ref indefinitely.<br>
<a class="moz-txt-link-freetext"
href="https://docs.gitlab.com/ee/ci/pipelines/job_artifacts.html#keep-artifacts-from-most-recent-successful-jobs"
moz-do-not-send="true">https://docs.gitlab.com/ee/ci/pipelines/job_artifacts.html#keep-artifacts-from-most-recent-successful-jobs</a><br>
<br>
Is this necessary or appropriate? We have a _lot_ of
branches, and it is unclear how/whether artifacts are
trimmed for refs for branches that have been removed.<br>
<br>
If the existing value of `expire_in` is too short, maybe we
could set it to a longer, but finite time instead of
"forever".<br>
<br>
If we want to further reduce our artifacts volume, we could
set `expire_in` to something less.<br>
<br>
We could also consider generating less artifact volume by
merging some jobs or stages. The config jobs, for instance,
take only marginally longer to run than the overhead they
generate, so we would lose very little CPU/pod time by
merging them with the build jobs.<br>
<br>
Best,<br>
Eric<br>
--<br>
Gromacs Developers mailing list<br>
<br>
* Please search the archive at <a
class="moz-txt-link-freetext"
href="https://www.gromacs.org/gmx-devel.html"
moz-do-not-send="true">https://www.gromacs.org/gmx-devel.html</a>
before posting!<br>
<br>
* Can't post? Read <a class="moz-txt-link-freetext"
href="https://www.gromacs.org/gmx-devel.html"
moz-do-not-send="true">https://www.gromacs.org/gmx-devel.html</a><br>
<br>
* For (un)subscribe requests visit<br>
<a class="moz-txt-link-freetext"
href="https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers"
moz-do-not-send="true">https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers</a>
or send a mail to <a class="moz-txt-link-abbreviated
moz-txt-link-freetext"
href="mailto:gmx-developers-request@gromacs.org"
moz-do-not-send="true">gmx-developers-request@gromacs.org</a>.<br>
</blockquote>
</div>
<br>
<fieldset class="moz-mime-attachment-header"></fieldset>
</blockquote>
<p><br>
</p>
<pre class="moz-signature" cols="72">--
Paul Bauer, PhD
GROMACS Development Manager
KTH Stockholm, SciLifeLab
0046737308594</pre>
</blockquote>
<p><br>
</p>
<pre class="moz-signature" cols="72">--
Paul Bauer, PhD
GROMACS Development Manager
KTH Stockholm, SciLifeLab
0046737308594</pre>
</body>
</html>