qemu-devel
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Why we should avoid new submodules if possible


From: Warner Losh
Subject: Re: Why we should avoid new submodules if possible
Date: Wed, 28 Sep 2022 07:15:53 -0600



On Wed, Sep 28, 2022 at 7:09 AM Daniel P. Berrangé <berrange@redhat.com> wrote:
On Wed, Sep 28, 2022 at 05:53:17AM -0400, Michael S. Tsirkin wrote:
> On Wed, Sep 28, 2022 at 10:37:14AM +0100, Daniel P. Berrangé wrote:
> > On Wed, Sep 28, 2022 at 05:26:42AM -0400, Michael S. Tsirkin wrote:
> > > On Tue, Jun 28, 2022 at 12:21:39PM +0200, Thomas Huth wrote:
> > > > On 28/06/2022 12.03, Michael S. Tsirkin wrote:
> > > > [...]
> > > > > For biosbits if we are going this route then I feel a submodule is much
> > > > > better.  It records which version exactly each qemu version wants.
> > > >
> > > > As far as I know, you can also specify the version when using pip, can't
> > > > you? So that's not really an advantage here.
> > > >
> > > > On the contrary, submodules have a couple of disadvantages that I really
> > > > dislike:
> > > >
> > > > - submodules do not get updated automatically when doing a "git checkout",
> > > > we have to update them via a script instead. This causes e.g. trouble if you
> > > > rsync your source tree to a machine that has no access to the internet and
> > > > you forgot to update the submodule before the sync
> > > >
> > > > - the content of submodules is not added to the tarballs that get created on
> > > > the git forges automatically. There were lots of requests from users in the
> > > > past that tried to download a tarball from github and then wondered why they
> > > > couldn't compile QEMU.
> > > >
> > > > - we include the submodule content in our release tarballs, so people get
> > > > the impression that hte submodule content is part of the QEMU sources. This
> > > > has two disadvantages:
> > > >  * We already got bug reports for the code in the submodule,
> > > >    where people did not understand that they should report that
> > > >    rather to the original project instead (i.e. you ship it - you
> > > >    own it)
> > > >  * People get the impression that QEMU is a huge monster
> > > >    application if they count the number of code lines, run
> > > >    their code scanner tools on the tarball contents, etc.
> > > >    Remember "nemu", for example, where one of the main complaints
> > > >    was that QEMU has too many lines of code?
> > > >
> > > > - If programs includes code via submodules, this gets a higher
> > > >   burder for distro maintainers, since they have to patch each
> > > >   and every package when there is a bug, instead of being able to
> > > >   fix it in one central place.
> > > >
> > > > So in my opinion we should avoid new submodules if there is an alternative.
> > > >
> > > >  Thomas
> > >
> > > So looking at the latest proposals downloading files from CI,
> > > checksumming them etc etc. No auto checkout, not added automatically
> > > either, right?
> > >
> > > This seems to be the only difference:
> > > - we include the submodule content in our release tarballs
> >
> > That's just one of the issues with submodules. Working with them in general
> > is not a pleasant experiance.
>
> This is what I asked about at the maintainers summit.
> I'd like to map the issues and see if there's anything
> we can do to solve them. In particular we will likely
> keep using submodules where we historically did
> so it's time well spent.
>
> I agree generally but the big question is what to replace these with.
> Below I assume the replacement is a script such as avocado or pytest
> with its own hashing, calling wget internally etc etc.
>
>
> > Thomas pointed out some of the issues, such
> > as 'git checkout' ignoring submodules, requiring extra steps to sync them.
>
>
> Not different from a home grown SCM as part of test script, right?

We're not building a home grown SCM as part of a test script, so
this answer is irrelevant.

> > There's also the perenial problem that developers frequently send
> > patches that mistakenly include submodule changes,
>
> OK, so the thing to do would be to look for ways to exclude submodule changes
> from git commits.

If someone wants to make git suck less with submodules great, but needs
someone to actually do the work.

A big part of the problem is knowing which of the following commands I have to
do to undo the uncommitted changes, the committed changes, the staged changes,
etc:

git submodule update --init --recursive
git submodule update --init
git submodule foreach --recursive git reset --hard
git submodule foreach --recursive git clean -xfd
git submodule foreach --recursive git clean -xfg

(all of these are in my history, I honestly don't know the difference between the last two).
And each 'oops' takes time away from upstreaming bsd-user I don't really have that
much of. I've wasted hours on this over the past year between all the different ways
it can screw up.

To be fair, it is a relatively small fraction of the time, but as you can tell from the
animation in my email it inspires much passion.

Warner
 
> > I'd really like to see us doing more to eliminate as much use of submodules
> > as is possible over time.p
>
> Or try to fix the problems, right?

Again needs someone to actually make it happen.

Meanwhile  QEMU already has an integrated test harness in the form
of Avocado that does everything needed. If Avocado had just been
used for this biosbits test in the first place, the test would
likely have already been merged to QEMU, instead of us having this
never ending debate on how to re-invent an alternative to what
already avocado does.

With regards,
Daniel
--
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|



reply via email to

[Prev in Thread] Current Thread [Next in Thread]