Re: 'check-avocado' fails after c03f57fd5b ("Revert "tests: Use separate ...")

[Daniel Henrique Barboza]

Hi,

I managed to work around it. I'll post here the debugs for future reference.

- Here's all the steps after a clean qemu clone:

[danielhb@grind build]$ make check-avocado
  AVOCADO tests/avocado
/home/danielhb/work/test/qemu/build/pyvenv/bin/python3: No module named 
avocado.__main__; 'avocado' is a package and cannot be directly executed
make: *** [/home/danielhb/work/test/qemu/tests/Makefile.include:139: 
check-avocado] Error 1
 
[danielhb@grind build]$ V=1 make check-avocado
/home/danielhb/work/test/qemu/build/pyvenv/bin/python3 -B -m avocado --show=app 
run --job-results-dir=/home/danielhb/work/test/qemu/build/tests/results  
--filter-by-tags-include-empty --filter-by-tags-include-empty-key -t 
arch:riscv64 --failfast tests/avocado
/home/danielhb/work/test/qemu/build/pyvenv/bin/python3: No module named 
avocado.__main__; 'avocado' is a package and cannot be directly executed
make: *** [/home/danielhb/work/test/qemu/tests/Makefile.include:139: 
check-avocado] Error 1

[danielhb@grind build]$ cat 
/home/danielhb/work/test/qemu/build/pyvenv/bin/avocado
#!/home/danielhb/work/test/qemu/build/pyvenv/bin/python3
# -*- coding: utf-8 -*-
import re
import sys
from avocado.core.main import main
if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
    sys.exit(main())
[danielhb@grind build]$

[danielhb@grind build]$ ./pyvenv/bin/python3
Python 3.11.4 (main, Jun  7 2023, 00:00:00) [GCC 13.1.1 20230511 (Red Hat 
13.1.1-2)] on linux
Type "help", "copyright", "credits" or "license" for more information.
> > > from importlib.metadata import distribution
> > > avocado = distribution('avocado-framework')
> > > next((x for x in avocado.entry_points if x.name == 'avocado'))
EntryPoint(name='avocado', value='avocado.core.main:main', 
group='console_scripts')
> > >

[danielhb@grind build]$ ./pyvenv/bin/avocado --help
Traceback (most recent call last):
  File "/home/danielhb/work/test/qemu/build/./pyvenv/bin/avocado", line 5, in 
<module>
    from avocado.core.main import main
ModuleNotFoundError: No module named 'avocado.core.main'
[danielhb@grind build]$


- I got suspicious after the above command failure, and noticed that 'avocado' 
didn't work
even outside of the QEMU tree:


[danielhb@grind ~]$ avocado --help
Traceback (most recent call last):
  File "/usr/bin/avocado", line 33, in <module>
    sys.exit(load_entry_point('avocado-framework==92.0', 'console_scripts', 
'avocado')())
             
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/bin/avocado", line 25, in importlib_load_entry_point
    return next(matches).load()
           ^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/importlib/metadata/__init__.py", line 202, in load
    module = import_module(match.group('module'))
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1140, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'avocado.core.main'


- Turns out that I had 2 avocado versions installed: one from F38 and other 
from pip.
If I remove the 'pip' version I got a different error:

 (01/13) tests/avocado/empty_cpu_model.py:EmptyCPUModel.test: STARTED
 (06/13) tests/avocado/tuxrun_baselines.py:TuxRunBaselineTest.test_riscv64: 
ERROR: Test.__init__() got an unexpected keyword argument 'run.results_dir' 
(0.04 s)
 (...)
 (01/13) tests/avocado/empty_cpu_model.py:EmptyCPUModel.test: ERROR: 
Test.__init__() got an unexpected keyword argument 'run.results_dir' (0.04 s)
 (...)

- Which seems to be related to a known bug according to:

https://avocado-framework.readthedocs.io/en/101.0/releases/100_1.html


In the end I don't need 'avocado' outside of testing QEMU, so my solution was to
remove all avocado packages from the system and let QEMU install whatever it is
needed inside pyvenv. 'check-avocado' now works in 'master'. I am still unsure
why this particular patch triggered all this problem here, but I don't believe
this is worth pursuing unless other people starts to see problems. For now we
can leave it as is IMO.



Thanks for the help!


Daniel


On 9/5/23 07:49, Paolo Bonzini wrote:
> On Tue, Sep 5, 2023 at 12:39 PM Paolo Bonzini <pbonzini@redhat.com> wrote:
> > > /home/danielhb/work/test/qemu/build/pyvenv/bin/python3 -B 
> > > python/scripts/mkvenv.py ensuregroup --online 
> > > /home/danielhb/work/test/qemu/pythondeps.toml avocado
> > > mkvenv: checking for avocado-framework(>=88.1, <93.0)
> > > mkvenv: checking for pycdlib>=1.11.0
> > >     AVOCADO tests/avocado
> > > /home/danielhb/work/test/qemu/build/pyvenv/bin/python3: No module named 
> > > avocado.__main__; 'avocado' is a package and cannot be directly executed
> > > make: *** [/home/danielhb/work/test/qemu/tests/Makefile.include:139: 
> > > check-avocado] Error 1
> >
> > Can you run it with "V=1" and also "cat
> > /home/danielhb/work/test/qemu/build/pyvenv/bin/avocado" please?
>
> Also:
>
> 1) run the following under the pyvenv/bin/python3 REPL:
>
> from importlib.metadata import distribution
> avocado = distribution('avocado-framework')
> next((x for x in avocado.entry_points if x.name == 'avocado'))
>
> FWIW here with a similar system I get
>
> EntryPoint(name='avocado', value='avocado.core.main:main',
> group='console_scripts')
>
> 2) try running "pyvenv/bin/avocado --help" and see if it works
>
> Paolo