Re: Why we should avoid new submodules if possible

[Daniel P . Berrangé]

On Wed, Sep 28, 2022 at 05:26:42AM -0400, Michael S. Tsirkin wrote:
> On Tue, Jun 28, 2022 at 12:21:39PM +0200, Thomas Huth wrote:
> > On 28/06/2022 12.03, Michael S. Tsirkin wrote:
> > [...]
> > > For biosbits if we are going this route then I feel a submodule is much
> > > better.  It records which version exactly each qemu version wants.
> > 
> > As far as I know, you can also specify the version when using pip, can't
> > you? So that's not really an advantage here.
> > 
> > On the contrary, submodules have a couple of disadvantages that I really
> > dislike:
> > 
> > - submodules do not get updated automatically when doing a "git checkout",
> > we have to update them via a script instead. This causes e.g. trouble if you
> > rsync your source tree to a machine that has no access to the internet and
> > you forgot to update the submodule before the sync
> > 
> > - the content of submodules is not added to the tarballs that get created on
> > the git forges automatically. There were lots of requests from users in the
> > past that tried to download a tarball from github and then wondered why they
> > couldn't compile QEMU.
> > 
> > - we include the submodule content in our release tarballs, so people get
> > the impression that hte submodule content is part of the QEMU sources. This
> > has two disadvantages:
> >  * We already got bug reports for the code in the submodule,
> >    where people did not understand that they should report that
> >    rather to the original project instead (i.e. you ship it - you
> >    own it)
> >  * People get the impression that QEMU is a huge monster
> >    application if they count the number of code lines, run
> >    their code scanner tools on the tarball contents, etc.
> >    Remember "nemu", for example, where one of the main complaints
> >    was that QEMU has too many lines of code?
> > 
> > - If programs includes code via submodules, this gets a higher
> >   burder for distro maintainers, since they have to patch each
> >   and every package when there is a bug, instead of being able to
> >   fix it in one central place.
> > 
> > So in my opinion we should avoid new submodules if there is an alternative.
> > 
> >  Thomas
> 
> So looking at the latest proposals downloading files from CI,
> checksumming them etc etc. No auto checkout, not added automatically
> either, right?
> 
> This seems to be the only difference:
> - we include the submodule content in our release tarballs

That's just one of the issues with submodules. Working with them in general
is not a pleasant experiance. Thomas pointed out some of the issues, such
as 'git checkout' ignoring submodules, requiring extra steps to sync them.
The flipside on tarballs is that the auto-generated tarballs from gitlab
do not include submodules, so the tests will be missing the files they
wanted. There's also the perenial problem that developers frequently send
patches that mistakenly include submodule changes, which is related to the
way that 'git checkout' doesn't sync submodule state when switching branches.
I'd really like to see us doing more to eliminate as much use of submodules
as is possible over time. 

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|