[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[bug#70169] [PATCH 0/7] Reproducible `make dist' tarball in defiance of
From: |
Janneke Nieuwenhuizen |
Subject: |
[bug#70169] [PATCH 0/7] Reproducible `make dist' tarball in defiance of Autotools and Gettext |
Date: |
Wed, 03 Apr 2024 23:04:14 +0200 |
User-agent: |
Gnus/5.13 (Gnus v5.13) |
Ludovic Courtès writes:
Hello,
> Janneke Nieuwenhuizen <janneke@gnu.org> skribis:
>
>> The recent XZ-utils
>> <https://www.openwall.com/lists/oss-security/2024/03/29/4>
>> debacle inspired me to resurrect and finish my patch set for creating a
>> reproducible source tarball for Guix, i.e., finally have `make dist' be
>> reproducible (when run from Git). I've been using a version of these patches
>> in simpler projects for some years now and stole one from Timothy Samplet's
>> Gash project.
>
> Yay, kudos to you and Timothy!
\o/
>> Autotools and Gettext still make it harder than necessary to do reproducible
>> (responsible?) computing, which is especially sad given the fact that the
>> Reproducible Builds project recently had their 10th birthday
>> <https://reproducible-builds.org/_lfs/presentations/2023-05-27-R-B-the-first-10-years/#/>.
>>
>> Gettext tooling embeds timestamps found in the file-system, fails to respect
>> SOURCE_DATE_EPOCH, and lacks options like `--pot-creation-date' so that we
>> have to resort to SED to fixup. The caching of all sorts of information, in
>> separate build stages, also doesn't help.
>
> Sadness indeed. Hopefully things will improve in the coming weeks, now
> that there’s an impetus.
Yes, that would be nice. With more people joining the effort, it could
be fixed brilliantly, in no time :)
>> To create a reproducible source tarball, having a reproducible build
>> environment is a prerequitite, so this would have to be recorded too.
>> Using this patch set, I created a tarball doing something like
>>
>> guix pull --commit=1dbe492b993a7629df3b35146ce0272b52913776
>> guix shell
>> bootstrap && ./configure --localstatedir=/var --sysconfdir=/etc && make dist
>> guix hash guix-1.3.0.57425-80a228.tar.gz
>> 0mk59ay5k2dxmjni9fx4i8qyfhvlgxbhqzsjpg2pbw381nskkxbj
>
> I applied the whole series on top of
> df64d48e6f9f648044aa5279c045b8d6f7bee604 (the ‘base-commit’ at the
> bottom of your message). Thus I got the same content as you but with a
> different commit ID.
Yeah..., that's why I pushed `wip-tarball'. We even look at the
committer's timestamp (not author, as that could be quite old).
> “make dist” gave me guix-1.3.0.57425-9f4a4a.tar.gz. The name indeed
> corresponds to the tip of my tree:
[..]
> But as a result, I get a different hash, and since the directory in the
> tarball has a different name, ‘diffoscope’ isn’t very helpful.
>
> There’s at least one relevant difference in the gzip metadata:
>
> --- guix-1.3.0.57425-9f4a4a.tar.gz
> +++ /tmp/guix-1.3.0.57425-80a228.tar.gz
> ├── filetype from file(1)
> │ @@ -1 +1 @@
> │ -gzip compressed data, from Unix, original size modulo 2^32 208138240 gzip
> compressed data, rese
> rved method, ASCII, extra field, encrypted, from FAT filesystem (MS-DOS,
> OS/2, NT)
> │ +gzip compressed data, from Unix, original size modulo 2^32 222504960 gzip
> compressed data, rese
> rved method, ASCII, has CRC, was "", has comment, encrypted, from FAT
> filesystem (MS-DOS, OS/2, NT
>
>
> (Your tarball has a CRC and comment, mine doesn’t.)
I believe this is a red herring. I saw this all day whenever one file
had a difference...
> Maybe we’ll have to iterate once you’ve pushed a first version, so we
> can truly build the same thing. Or we should push the branch somewhere
I boldly pushed `origin/wip-tarball', you may try that :)
Be sure to git diff between your tip and origin/wip-tarball to ascertain
I didn't place easter eggs (just kidding).
> (or use the one from <https://data.qa.guix.gnu.org/> once it’s been
> created).
Thanks a lot for trying!
Greetings,
Janneke