[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[PULL 11/21] mkvenv: always pass locally-installed packages to pip
From: |
Paolo Bonzini |
Subject: |
[PULL 11/21] mkvenv: always pass locally-installed packages to pip |
Date: |
Tue, 6 Jun 2023 16:31:06 +0200 |
Let pip decide whether a new version should be installed or the current
one is okay. This ensures that the virtual environment is updated
(either upgraded or downgraded) whenever a new version of a package is
requested.
The hardest part here is figuring out if a package is installed in
the venv (which also has to be done twice to account for the presence
of either setuptools in Python <3.8, or importlib in Python >=3.8).
Suggested-by: Peter Maydell <peter.maydell@linaro.org>
Cc: John Snow <jsnow@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
python/scripts/mkvenv.py | 76 ++++++++++++++++++++++++++++++++++++++--
1 file changed, 74 insertions(+), 2 deletions(-)
diff --git a/python/scripts/mkvenv.py b/python/scripts/mkvenv.py
index 3a9aef46a51..a47f1eaf5d6 100644
--- a/python/scripts/mkvenv.py
+++ b/python/scripts/mkvenv.py
@@ -553,6 +553,74 @@ def pkgname_from_depspec(dep_spec: str) -> str:
return match.group(0)
+def _get_path_importlib(package: str) -> Optional[str]:
+ # pylint: disable=import-outside-toplevel
+ # pylint: disable=no-name-in-module
+ # pylint: disable=import-error
+ try:
+ # First preference: Python 3.8+ stdlib
+ from importlib.metadata import ( # type: ignore
+ PackageNotFoundError,
+ distribution,
+ )
+ except ImportError as exc:
+ logger.debug("%s", str(exc))
+ # Second preference: Commonly available PyPI backport
+ from importlib_metadata import ( # type: ignore
+ PackageNotFoundError,
+ distribution,
+ )
+
+ try:
+ return str(distribution(package).locate_file("."))
+ except PackageNotFoundError:
+ return None
+
+
+def _get_path_pkg_resources(package: str) -> Optional[str]:
+ # pylint: disable=import-outside-toplevel
+ # Bundled with setuptools; has a good chance of being available.
+ import pkg_resources
+
+ try:
+ return str(pkg_resources.get_distribution(package).location)
+ except pkg_resources.DistributionNotFound:
+ return None
+
+
+def _get_path(package: str) -> Optional[str]:
+ try:
+ return _get_path_importlib(package)
+ except ImportError as exc:
+ logger.debug("%s", str(exc))
+
+ try:
+ return _get_path_pkg_resources(package)
+ except ImportError as exc:
+ logger.debug("%s", str(exc))
+ raise Ouch(
+ "Neither importlib.metadata nor pkg_resources found. "
+ "Use Python 3.8+, or install importlib-metadata or setuptools."
+ ) from exc
+
+
+def _path_is_prefix(prefix: Optional[str], path: str) -> bool:
+ try:
+ return (
+ prefix is not None and os.path.commonpath([prefix, path]) == prefix
+ )
+ except ValueError:
+ return False
+
+
+def _is_system_package(package: str) -> bool:
+ path = _get_path(package)
+ return path is not None and not (
+ _path_is_prefix(sysconfig.get_path("purelib"), path)
+ or _path_is_prefix(sysconfig.get_path("platlib"), path)
+ )
+
+
def _get_version_importlib(package: str) -> Optional[str]:
# pylint: disable=import-outside-toplevel
# pylint: disable=no-name-in-module
@@ -741,8 +809,12 @@ def _do_ensure(
for spec in dep_specs:
matcher = distlib.version.LegacyMatcher(spec)
ver = _get_version(matcher.name)
- if ver is None or not matcher.match(
- distlib.version.LegacyVersion(ver)
+ if (
+ ver is None
+ # Always pass installed package to pip, so that they can be
+ # updated if the requested version changes
+ or not _is_system_package(matcher.name)
+ or not matcher.match(distlib.version.LegacyVersion(ver))
):
absent.append(spec)
else:
--
2.40.1
- [PULL 02/21] atomics: eliminate mb_read/mb_set, (continued)
- [PULL 02/21] atomics: eliminate mb_read/mb_set, Paolo Bonzini, 2023/06/06
- [PULL 04/21] meson.build: Group the UI entries in a separate summary section, Paolo Bonzini, 2023/06/06
- [PULL 05/21] meson.build: Group the network backend entries in a separate summary section, Paolo Bonzini, 2023/06/06
- [PULL 06/21] meson.build: Group the audio backend entries in a separate summary section, Paolo Bonzini, 2023/06/06
- [PULL 08/21] scsi/qemu-pr-helper: Drop support for 'old' libmultipath API, Paolo Bonzini, 2023/06/06
- [PULL 07/21] meson.build: Use -Wno-undef only for SDL2 versions that need it, Paolo Bonzini, 2023/06/06
- [PULL 09/21] Revert "tests/requirements.txt: bump up avocado-framework version to 101.0", Paolo Bonzini, 2023/06/06
- [PULL 10/21] tests: Use separate virtual environment for avocado, Paolo Bonzini, 2023/06/06
- [PULL 12/21] configure: remove --with-git= option, Paolo Bonzini, 2023/06/06
- [PULL 13/21] configure: rename --enable-pypi to --enable-download, control subprojects too, Paolo Bonzini, 2023/06/06
- [PULL 11/21] mkvenv: always pass locally-installed packages to pip,
Paolo Bonzini <=
- [PULL 14/21] git-submodule: allow partial update of .git-submodule-status, Paolo Bonzini, 2023/06/06
- [PULL 15/21] build: log submodule update from git-submodule.sh, Paolo Bonzini, 2023/06/06
- [PULL 16/21] meson: subprojects: replace submodules with wrap files, Paolo Bonzini, 2023/06/06
- [PULL 17/21] configure: move SLOF submodule handling to pc-bios/s390-ccw, Paolo Bonzini, 2023/06/06
- [PULL 18/21] pc-bios/s390-ccw: always build network bootloader, Paolo Bonzini, 2023/06/06
- [PULL 21/21] configure: remove --with-git-submodules=, Paolo Bonzini, 2023/06/06
- [PULL 20/21] build: remove git submodule handling from main makefile, Paolo Bonzini, 2023/06/06
- [PULL 19/21] meson: subprojects: replace berkeley-{soft, test}float-3 with wraps, Paolo Bonzini, 2023/06/06
- Re: [PULL 00/21] (Mostly) build system patches for 2023-06-06, Richard Henderson, 2023/06/06