[PATCH v2 00/10] meson: replace submodules with wrap files

[Paolo Bonzini]

This series replaces git submodules for bundled libraries with .wrap
files that can be used directly by meson for subprojects.  These have
several advantages, either immediate or potential:

* option parsing and downloading is delegated to meson

* the commit is stored in a text file instead of a magic entry in the
  git tree object, and can be a branch name or a version number as well

* now that QEMU's configure script knows how to install missing Python
  dependencies, we could stop shipping external dependencies that are
  only used as a fallback, and download them on demand if the build
  platform lacks them.  For example dtc could be downloaded at build
  time, controlled by --enable-download, even when building from
  a tarball.  This is _not_ done in this patch series, but Marc-André
  has tried it before[1].

* we could also add .wrap files for other dependencies that are missing
  on the GCC compile farm machines, or for people who build on Windows
  and might enjoy getting the mandatory dependencies (pixman, zlib,
  glib, possibly SDL?) via wraps.  In theory meson already supports
  "meson wrap update-db" to automatically use wraps for anything required
  but missing, but one would need to test that it actually works; see
  for example https://github.com/mesonbuild/meson/issues/11821.

dtc and keycodemapdb both support meson, and previously reviewed
patches already run their build system via subproject(), so the
wraps are automatically taken into account.  Two other submodules,
berkeley-softfloat-3 and berkeley-testfloat-3, are used to test QEMU and
can be changed to use wraps; however this requires a few more changes
to extract the corresponding parts of tests/fp/meson.build.  It's mostly
code movement rather than new code.

The remaining submodules consist of tests/lcitool/libvirt-ci and the
firmware in roms/.  The former is only used in very specific cases,
while the latter is mostly used only as a pointer used to create the QEMU
tarball.  Unfortunately, git-submodule.sh is still needed for roms/SLOF,
parts of which are used in the QEMU build process for pc-bios/s390-ccw.
For now, the git-submodule.sh rules are moved from the main Makefile to
pc-bios/s390-ccw/ and, to limit the number of configure options,
--with-git-submodules is merged with --enable-download.


Patch 1 removes the --with-git= option for consistency, since git cannot
be overridden for "meson subprojects download".  There is still a GIT
environment variable, which I could look at supporting in meson as well.

Patch 2 renames the recently introduced --enable-pypi command line
option to --enable-download, and makes it control meson's -Dwrapmode
option as well.

Patches 3-4 are small improvements to git-submodule.sh, which make
sense when the submodule updates are split across multiple Makefiles.
At the end of the series actually the updates move entirely to
pc-bios/s390-ccw/Makefile, but these improvements are useful anyway in
my opinion.

Patch 5 replaces submodule update with meson's subproject download for
existing subprojects (including libfdt and keycodemapdb).

Patches 6-7 move roms/SLOF handling to pc-bios/s390-ccw.  While in
the future it is possible that SLOF's networking code will be copied
to the s390 ROMs, for now I am going for a smaller change.

Patch 8 converts berkeley-{soft,test}float-3 to subprojects too,
contextually moving their build rules out of tests/fp/meson.build.

Patches 9-10 are residual cleanups.

Paolo

[1] 
https://patchew.org/QEMU/20230302131848.1527460-1-marcandre.lureau@redhat.com/20230302131848.1527460-5-marcandre.lureau@redhat.com/



Paolo Bonzini (10):
  configure: remove --with-git= option
  configure: rename --enable-pypi to --enable-download, control
    subprojects too
  git-submodule: allow partial update of .git-submodule-status
  build: log submodule update from git-submodule.sh
  meson: subprojects: replace submodules with wrap files
  configure: move SLOF submodule handling to pc-bios/s390-ccw
  pc-bios/s390-ccw: always build network bootloader
  meson: subprojects: replace berkeley-{soft,test}float-3 with wraps
  build: remove git submodule handling from main makefile
  configure: remove --with-git-submodules=

 .gitignore                                    |   2 -
 .gitlab-ci.d/buildtest-template.yml           |   4 +-
 .gitmodules                                   |  15 -
 Makefile                                      |  12 -
 configure                                     | 169 ++----
 meson.build                                   |  17 +-
 pc-bios/s390-ccw/Makefile                     |  16 +-
 scripts/archive-source.sh                     |  27 +-
 .../ci/org.centos/stream/8/x86_64/configure   |   3 -
 scripts/git-submodule.sh                      |  69 ++-
 scripts/make-release                          |   5 +
 subprojects/.gitignore                        |   8 +
 subprojects/berkeley-softfloat-3.wrap         |   5 +
 subprojects/berkeley-testfloat-3.wrap         |   5 +
 subprojects/dtc                               |   1 -
 subprojects/dtc.wrap                          |   4 +
 subprojects/keycodemapdb                      |   1 -
 subprojects/keycodemapdb.wrap                 |   4 +
 subprojects/libvfio-user                      |   1 -
 subprojects/libvfio-user.wrap                 |   4 +
 .../berkeley-softfloat-3/meson.build          | 339 +++++++++++
 .../berkeley-softfloat-3/meson_options.txt    |   1 +
 .../berkeley-testfloat-3/meson.build          | 220 +++++++
 .../berkeley-testfloat-3/meson_options.txt    |   1 +
 tests/fp/meson.build                          | 541 +-----------------
 25 files changed, 716 insertions(+), 758 deletions(-)
 create mode 100644 subprojects/.gitignore
 create mode 100644 subprojects/berkeley-softfloat-3.wrap
 create mode 100644 subprojects/berkeley-testfloat-3.wrap
 delete mode 160000 subprojects/dtc
 create mode 100644 subprojects/dtc.wrap
 delete mode 160000 subprojects/keycodemapdb
 create mode 100644 subprojects/keycodemapdb.wrap
 delete mode 160000 subprojects/libvfio-user
 create mode 100644 subprojects/libvfio-user.wrap
 create mode 100644 subprojects/packagefiles/berkeley-softfloat-3/meson.build
 create mode 100644 
subprojects/packagefiles/berkeley-softfloat-3/meson_options.txt
 create mode 100644 subprojects/packagefiles/berkeley-testfloat-3/meson.build
 create mode 100644 
subprojects/packagefiles/berkeley-testfloat-3/meson_options.txt

-- 
2.40.1