Re: [PATCH 16/16] tests/acceptance: Test the MIPSsim machine

[Philippe Mathieu-Daudé]

On 9/30/20 11:49 AM, Alex Bennée wrote:
> 
> Daniel P. Berrangé <berrange@redhat.com> writes:
> 
>> On Tue, Sep 29, 2020 at 10:38:30AM +0100, Alex Bennée wrote:
>>>
>>> Philippe Mathieu-Daudé <f4bug@amsat.org> writes:
>>>
>>>> +Alex/Daniel
>>>>
>>>> On 9/28/20 10:33 PM, Willian Rampazzo wrote:
>>>>> On Mon, Sep 28, 2020 at 2:31 PM Philippe Mathieu-Daudé <f4bug@amsat.org> 
>>>>> wrote:
>>>>>>
>>>>>> Add a test for the mipssim machine, based on the recommended
>>>>>> test setup from Thomas Huth:
>>>>>> https://www.mail-archive.com/qemu-devel@nongnu.org/msg606846.html
>>>>>>
>>>>>> The test is quick and can be run as:
>>>>>>
>>>>>>   $ avocado --show=console run -t machine:mipssim tests/acceptance/
>>>>>>    (1/1) 
>>>>>> tests/acceptance/machine_mips_mipssim.py:MipsSimMachine.test_mipssim_linux_console:
>>>>>>   console: Linux version 3.6.11 (root@711bb8ba16a7) (gcc version 4.8.3 
>>>>>> (Buildroot 2014.11) ) #2 Sun Sep 27 13:39:35 UTC 2020
>>>>>>   console: Setting default memory size 0x02000000
>>>>>>   console: bootconsole [early0] enabled
>>>>>>   console: CPU revision is: 00019300 (MIPS 24Kc)
>>>>>>   console: FPU revision is: 00739300
>>>>>>   ...
>>>>>>   console: CPU frequency 12.00 MHz
>>>>>>   console: Calibrating delay loop... 950.27 BogoMIPS (lpj=4751360)
>>>>>>   ...
>>>>>>   console: MIPSNet Ethernet driver. Version: 2007-11-17. (c)2005 MIPS 
>>>>>> Technologies, Inc.
>>>>>>   ...
>>>>>>   console: Welcome to Buildroot
>>>>>>   console: buildroot login: root
>>>>>>   console: # root
>>>>>>   console: -sh: root: not found
>>>>>>   console: # ping -c 3 10.0.2.2
>>>>>>   console: PING 10.0.2.2 (10.0.2.2): 56 data bytes
>>>>>>   console: 64 bytes from 10.0.2.2: seq=0 ttl=255 time=48.231 ms
>>>>>>   console: 64 bytes from 10.0.2.2: seq=1 ttl=255 time=9.407 ms
>>>>>>   console: 64 bytes from 10.0.2.2: seq=2 ttl=255 time=2.298 ms
>>>>>>   console: --- 10.0.2.2 ping statistics ---
>>>>>>   console: 3 packets transmitted, 3 packets received, 0% packet loss
>>>>>>   PASS (7.99 s)
>>>>>>
>>>>>> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
>>>>>> ---
>>>>>> Cc: Thomas Huth <huth@tuxfamily.org>
>>>>>> ---
>>>>>>  MAINTAINERS                              |  1 +
>>>>>>  tests/acceptance/machine_mips_mipssim.py | 56 ++++++++++++++++++++++++
>>>>>>  2 files changed, 57 insertions(+)
>>>>>>  create mode 100644 tests/acceptance/machine_mips_mipssim.py
>>>>>>
>>>>>> diff --git a/MAINTAINERS b/MAINTAINERS
>>>>>> index 5eed1e692b4..17d8a012b0e 100644
>>>>>> --- a/MAINTAINERS
>>>>>> +++ b/MAINTAINERS
>>>>>> @@ -240,6 +240,7 @@ F: include/hw/misc/mips_*
>>>>>>  F: include/hw/timer/mips_gictimer.h
>>>>>>  F: tests/acceptance/linux_ssh_mips_malta.py
>>>>>>  F: tests/acceptance/machine_mips_malta.py
>>>>>> +F: tests/acceptance/machine_mips_mipssim.py
>>>>>>  F: tests/tcg/mips/
>>>>>>  K: ^Subject:.*(?i)mips
>>>>>>
>>>>>> diff --git a/tests/acceptance/machine_mips_mipssim.py 
>>>>>> b/tests/acceptance/machine_mips_mipssim.py
>>>>>> new file mode 100644
>>>>>> index 00000000000..b2749917b08
>>>>>> --- /dev/null
>>>>>> +++ b/tests/acceptance/machine_mips_mipssim.py
>>>>>> @@ -0,0 +1,56 @@
>>>>>> +# Functional tests for the MIPS simulator (MIPSsim machine)
>>>>>> +#
>>>>>> +# Copyright (c) 2020 Philippe Mathieu-Daudé <f4bug@amsat.org>
>>>>>> +#
>>>>>> +# This work is licensed under the terms of the GNU GPL, version 2 or 
>>>>>> later.
>>>>>> +# See the COPYING file in the top-level directory.
>>>>>> +#
>>>>>> +# SPDX-License-Identifier: GPL-2.0-or-later
>>>>>> +
>>>>>> +import os
>>>>>> +import logging
>>>>>> +import time
>>>>>> +
>>>>>> +from avocado import skipUnless
>>>>>> +from avocado_qemu import Test
>>>>>> +from avocado_qemu import exec_command_and_wait_for_pattern
>>>>>> +from avocado_qemu import interrupt_interactive_console_until_pattern
>>>>>> +from avocado_qemu import wait_for_console_pattern
>>>>>> +
>>>>>> +class MipsSimMachine(Test):
>>>>>> +
>>>>>> +    timeout = 30
>>>>>> +    KERNEL_COMMON_COMMAND_LINE = 'printk.time=0 '
>>>>>> +
>>>>>> +    @skipUnless(os.getenv('AVOCADO_ALLOW_UNTRUSTED_CODE'), 'untrusted 
>>>>>> code')
>>>>>> +    def test_mipssim_linux_console(self):
>>>>>> +        """
>>>>>> +        Boots the Linux kernel and checks that the console is 
>>>>>> operational
>>>>>> +        :avocado: tags=arch:mipsel
>>>>>> +        :avocado: tags=machine:mipssim
>>>>>> +        :avocado: tags=device:mipsnet
>>>>>> +        """
>>>>>> +        kernel_url = ('https://github.com/philmd/qemu-testing-blob/raw/'
>>>>>> +                      '32ea5764e1de8fffa0d59366c44822cd06d7c8e0/'
>>>>>> +                      'mips/mipssim/mipsel/vmlinux')
>>>>>
>>>>> So, are you willing to maintain some images on your GitHub to avoid
>>>>> the image changes when they are not found?
>>>>
>>>> No, I am not willing to do that.
>>>>
>>>> But I see it pointless to have everyone (including CI) to spend 1h
>>>> building this image, when the sources and build recipe is available,
>>>> making the built image reproducible.
>>>
>>> I agree we don't want to build from scratch each time. However as we
>>> move from relying on third parties (which outside of the major distros
>>> has proved very flaky) we are getting to the point of managing our GPL
>>> liabilities.
>>
>> I'm wondering what the 1 hour time to build the images is
>> spent on, and what does the image need to actually do ?
> 
> Buildroot will also build the toolchain and libc of chouce before
> building it's rootfs. To be honest the rootfs part of the build is
> fairly minimal if it's mostly busybox.

Exactly, half of the time is spent building binutils and gcc.

> 
>> Is it anything more than boot the kernel successfully ?
>>
>> I've not tried a timed build, but surely a minimal / tiny
>> Linx kernel build is much less than an hour ? Combine with
>> a tiny C program that simply prints "OK" as /bin/init and
>> powers off the VM. Or we need more advanced then a minimal
>> busybox build.
> 
> Starting from a buildroot is useful for development certainly because
> it's a lot easier to add stuff to it if you need to.
> 
>> I think it'd be great if we could self-build a minimal
>> kernel+init for all platforms we target, using distro
>> provided cross-compilers.

We can't because MIPSsim has been dropped from the Linux kernel v3.18,
and trying to build a such old kernel produce a *lot* of errors with
a current (cross) compiler. See:
https://lists.nongnu.org/archive/html/qemu-devel/2018-04/msg04071.html

What provides buildroot is old enough toolchain to be able to build
old code.

> 
> We've been down this round before haven't we? Wasn't that what the JeOS
> builds where all about?
> 
> Anyway things have moved on from the bad old days of crosstool and the
> distros are better at packaging sane cross compilers that can build
> kernels and basic user space. However leaving everyone to use their own
> distro compiler is going to lead to repeatability problems - or maybe we
> just treat it like we do with the existing TCG tests?
> 
>>
>>
>> Regards,
>> Daniel
> 
>