Re: Treating tests as special case

[Björn Höfling]

On Thu, 05 Apr 2018 12:14:53 +0200
Ricardo Wurmus <address@hidden> wrote:

> Björn Höfling <address@hidden> writes:
> 
> > And you mentioned different environment conditions like machine and
> > kernel. We still have "only" 70-90% reproducibility.  
> 
> Where does that number come from?  In my tests for a non-trivial set
> of bioinfo pipelines I got to 97.7% reproducibility (or 95.2% if you
> include very minor problems) for 355 direct inputs.
> 
> I rebuilt on three different machines.

I have no own numbers but checked Ludivic's blog post from October 2017:

https://www.gnu.org/software/guix/blog/2017/reproducible-builds-a-status-update/

"We’re somewhere between 78% and 91%—not as good as Debian yet, [..]".

So if your numbers are valid for the whole repository, that is good
news and would mean we are now better than Debian [1], and that would
be worth a new blog post.

Björn

[1] https://isdebianreproducibleyet.com/

pgpbxNPxZXDbi.pgp
Description: OpenPGP digital signature