Re: [Qemu-devel] [PATCH v2] tests/boot_linux_console: add a test for riscv64 + virt

[Chih-Min Chao]

On Thu, Jul 18, 2019 at 11:34 PM Alistair Francis <address@hidden>
wrote:

> On Thu, Jul 18, 2019 at 8:00 AM Chih-Min Chao <address@hidden>
> wrote:
> >
> >
> >
> > On Thu, Jul 18, 2019 at 6:47 AM Alistair Francis <address@hidden>
> wrote:
> >>
> >> On Tue, Jul 16, 2019 at 10:21 PM Chih-Min Chao <address@hidden>
> wrote:
> >> >
> >> >
> >> >
> >> > On Wed, Jul 17, 2019 at 6:59 AM Alistair Francis <
> address@hidden> wrote:
> >> >>
> >> >> On Tue, Jul 16, 2019 at 2:50 PM Philippe Mathieu-Daudé
> >> >> <address@hidden> wrote:
> >> >> >
> >> >> > On 7/16/19 10:43 PM, Alistair Francis wrote:
> >> >> > > On Tue, Jul 16, 2019 at 6:56 AM Chih-Min Chao <
> address@hidden> wrote:
> >> >> > >>
> >> >> > >>
> >> >> > >> On Tue, Jul 16, 2019 at 12:34 AM Alistair Francis <
> address@hidden> wrote:
> >> >> > >>>
> >> >> > >>> On Mon, Jul 15, 2019 at 6:02 AM Philippe Mathieu-Daudé
> >> >> > >>> <address@hidden> wrote:
> >> >> > >>>>
> >> >> > >>>> On 7/15/19 1:09 PM, Chih-Min Chao wrote:
> >> >> > >>>>> On Mon, Jul 15, 2019 at 5:15 PM Philippe Mathieu-Daudé
> >> >> > >>>>> <address@hidden <mailto:address@hidden>> wrote:
> >> >> > >>>>>
> >> >> > >>>>>     On 7/15/19 11:08 AM, Chih-Min Chao wrote:
> >> >> > >>>>>     > Similar to the mips + malta test, it boots a Linux
> kernel on a virt
> >> >> > >>>>>     > board and verify the serial is working.  Also, it
> relies on the serial
> >> >> > >>>>>     > device set by the machine itself.
> >> >> > >>>>>     >
> >> >> > >>>>>     > If riscv64 is a target being built, "make
> check-acceptance" will
> >> >> > >>>>>     > automatically include this test by the use of the
> "arch:riscv64" tags.
> >> >> > >>>>>     >
> >> >> > >>>>>     > Alternatively, this test can be run using:
> >> >> > >>>>>     >
> >> >> > >>>>>     >   $ avocado run -t arch:riscv64 tests/acceptance
> >> >> > >>>>>     >
> >> >> > >>>>>     > packages
> >> >> > >>>>>     >   debian official
> >> >> > >>>>>     >     binutils-riscv64-linux-gnu_2.32-8
> >> >> > >>>>>     >     opensbi_0.4-1_all
> >> >> > >>>>>     >     linux-image-4.19.0-5-riscv64 4.19.37-4
> >> >> > >>>>>     >   third-party
> >> >> > >>>>>     >
> >> >> > >>>>>
> https://github.com/groeck/linux-build-test/rootfs/riscv64/rootfs.cpio.gz
> >> >> > >>>>>     >     (the repo is also used in mips target acceptance)
> >> >> > >>>>>     >
> >> >> > >>>>>     > Signed-off-by: Chih-Min Chao <address@hidden
> >> >> > >>>>>     <mailto:address@hidden>>
> >> >> > >>>>>     > ---
> >> >> > >>>>>     >  .travis.yml                            |  2 +-
> >> >> > >>>>>     >  tests/acceptance/boot_linux_console.py | 66
> >> >> > >>>>>     ++++++++++++++++++++++++++++++++++
> >> >> > >>>>>     >  2 files changed, 67 insertions(+), 1 deletion(-)
> >> >> > >>>>>     >
> >> >> > >>>>>     > diff --git a/.travis.yml b/.travis.yml
> >> >> > >>>>>     > index 5d3d6ee..21fcead 100644
> >> >> > >>>>>     > --- a/.travis.yml
> >> >> > >>>>>     > +++ b/.travis.yml
> >> >> > >>>>>     > @@ -232,7 +232,7 @@ matrix:
> >> >> > >>>>>     >
> >> >> > >>>>>     >      # Acceptance (Functional) tests
> >> >> > >>>>>     >      - env:
> >> >> > >>>>>     > -        - CONFIG="--python=/usr/bin/python3
> >> >> > >>>>>
>  
> --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,aarch64-softmmu,arm-softmmu,s390x-softmmu,alpha-softmmu"
> >> >> > >>>>>     > +        - CONFIG="--python=/usr/bin/python3
> >> >> > >>>>>
>  
> --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,aarch64-softmmu,arm-softmmu,s390x-softmmu,alpha-softmmu,riscv64-softmmu"
> >> >> > >>>>>     >          - TEST_CMD="make check-acceptance"
> >> >> > >>>>>     >        after_failure:
> >> >> > >>>>>     >          - cat tests/results/latest/job.log
> >> >> > >>>>>     > diff --git a/tests/acceptance/boot_linux_console.py
> >> >> > >>>>>     b/tests/acceptance/boot_linux_console.py
> >> >> > >>>>>     > index 3215950..0f638bc 100644
> >> >> > >>>>>     > --- a/tests/acceptance/boot_linux_console.py
> >> >> > >>>>>     > +++ b/tests/acceptance/boot_linux_console.py
> >> >> > >>>>>     > @@ -354,3 +354,69 @@ class BootLinuxConsole(Test):
> >> >> > >>>>>     >          self.vm.launch()
> >> >> > >>>>>     >          console_pattern = 'Kernel command line: %s' %
> >> >> > >>>>>     kernel_command_line
> >> >> > >>>>>     >          self.wait_for_console_pattern(console_pattern)
> >> >> > >>>>>     > +
> >> >> > >>>>>     > +    def test_riscv64_virt(self):
> >> >> > >>>>>     > +        """
> >> >> > >>>>>     > +        :avocado: tags=arch:riscv64
> >> >> > >>>>>     > +        :avocado: tags=machine:virt
> >> >> > >>>>>     > +        """
> >> >> > >>>>>     > +        deb_url = ('
> https://snapshot.debian.org/archive/debian/'
> >> >> > >>>>>     > +
>  '20190424T171759Z/pool/main/b/binutils/'
> >> >> > >>>>>     > +
> >> >> > >>>>>      'binutils-riscv64-linux-gnu_2.32-8_amd64.deb')
> >> >> > >>>>>     > +        deb_hash =
> ('7fe376fd4452696c03acd508d6d613ca553ea15e')
> >> >> > >>>>>     > +        deb_path = self.fetch_asset(deb_url,
> asset_hash=deb_hash)
> >> >> > >>>>>     > +        objcopy_path =
> '/usr/bin/riscv64-linux-gnu-objcopy'
> >> >> > >>>>>     > +        objcopy_path =
> self.extract_from_deb(deb_path, objcopy_path)
> >> >> > >>>>>     > +        libbfd_path =
> >> >> > >>>>>     '/usr/lib/x86_64-linux-gnu/libbfd-2.32-riscv64.so
> >> >> > >>>>>     <http://libbfd-2.32-riscv64.so>'
> >> >> > >>>>>     > +        libbfd_path = self.extract_from_deb(deb_path,
> libbfd_path)
> >> >> > >>>>>     > +        process.run('ls -al %s' % (objcopy_path))
> >> >> > >>>>>     > +
> >> >> > >>>>>     > +        deb_url = ('
> https://snapshot.debian.org/archive/debian/'
> >> >> > >>>>>     > +
>  '20190708T032337Z/pool/main/o/opensbi/'
> >> >> > >>>>>     > +                   'opensbi_0.4-1_all.deb')
> >> >> > >>>>>     > +        deb_hash =
> ('2319dcd702958291d323acf5649fd98a11d90112')
> >> >> > >>>>>     > +        deb_path = self.fetch_asset(deb_url,
> asset_hash=deb_hash)
> >> >> > >>>>>     > +        opensbi_path =
> ('/usr/lib/riscv64-linux-gnu/opensbi/'
> >> >> > >>>>>     > +                        'qemu/virt/fw_jump.elf')
> >> >> > >>>>>     > +        opensbi_path =
> self.extract_from_deb(deb_path, opensbi_path)
> >> >> > >>>>>     > +
> >> >> > >>>>>     > +        deb_url =
> >> >> > >>>>>     ('https://snapshot.debian.org/archive/debian-ports/'
> >> >> > >>>>>     > +
>  '20190620T095935Z/pool-riscv64/main/l/linux/'
> >> >> > >>>>>     > +
> >> >> > >>>>>      'linux-image-4.19.0-5-riscv64_4.19.37-4_riscv64.deb')
> >> >> > >>>>>     > +        deb_hash =
> ('bf5b5680c41d92134d22caef4fbd277c5217e1f0')
> >> >> > >>>>>     > +        deb_path = self.fetch_asset(deb_url,
> asset_hash=deb_hash)
> >> >> > >>>>>     > +        kernel_path = '/boot/vmlinux-4.19.0-5-riscv64'
> >> >> > >>>>>     > +        kernel_path = self.extract_from_deb(deb_path,
> kernel_path)
> >> >> > >>>>>     > +        kimage_path = self.workdir + "/Image"
> >> >> > >>>>>     > +        env = os.environ
> >> >> > >>>>>     > +        env['LD_LIBRARY_PATH'] = ('%s:' %
> >> >> > >>>>>     (os.path.dirname(libbfd_path)) +
> >> >> > >>>>>     > +
>  env.get('LD_LIBRARY_PATH', ''))
> >> >> > >>>>>     > +        process.run(('%s -O binary -O binary -R'
> >> >> > >>>>>     > +                     '.note -R .note.gnu.build-id -R
> .comment -S
> >> >> > >>>>>     %s %s') %
> >> >> > >>>>>     > +                     (objcopy_path, kernel_path,
> kimage_path))
> >> >> > >>>>>
> >> >> > >>>>>     Please explain why you need to do that...
> >> >> > >>>>>
> >> >> > >>>>>     Also note these tests are run on other host architecture
> than x86, at
> >> >> > >>>>>     least ppc64/s390x (so this won't work there).
> >> >> > >>>>>
> >> >> > >>>>> it is because riscv64 vmlinux doesn't have physical loading
> address
> >> >> > >>>>> information and
> >> >> > >>>>> depends on certain stage bootloader to move kernel raw data
> to specific
> >> >> > >>>>> physical address (0x8020_0000)
> >> >> > >>>>>
> >> >> > >>>>> The vmlinux program headers are
> >> >> > >>>>>   Type             Offset       VirtAddr
> PhysAddr
> >> >> > >>>>>     FileSiz  MemSiz   Flg Align
> >> >> > >>>>>   LOAD           0x001000 0xffffffe000000000
> 0x0000000000000000 0x0303a6
> >> >> > >>>>> 0x0303a6 R E 0x1000
> >> >> > >>>>>   LOAD           0x032000 0xffffffe000031000
> 0x0000000000031000 0x828f0c
> >> >> > >>>>> 0x828f0c RWE 0x1000
> >> >> > >>>>>   NOTE           0x85aec0 0xffffffe000859ec0
> 0x0000000000859ec0 0x00004c
> >> >> > >>>>> 0x00004c R   0x4
> >> >> > >>>>>
> >> >> > >>>>> For legacy bootloader, bbl, It can handle payload in ELF
> format.
> >> >> > >>>>> However, for the newer OpenSBI, it can only handle raw data
> and
> >> >> > >>>>> debian doesn't provide linux image in raw data format.
> >> >> > >>>
> >> >> > >>> This doesn't sound right. You should just be able to say
> -kernel
> >> >> > >>> <anything> and it should work. It has worked for me with
> vmlinux and
> >> >> > >>> Image files when using master (not the 4.0 release but
> master/4.1).
> >> >> > >>> Although all of my testing was with the 5.1 kernel, so maybe
> there is
> >> >> > >>> a difference there?
> >> >> > >>>
> >> >> > >>> What isn't working for you? Can you include the errors and
> output of -d in_asm?
> >> >> > >>>
> >> >> > >>>
> >> >> > >>> Alistair
> >> >> > >>
> >> >> > >>
> >> >> > >> Hi Alistair,
> >> >> > >>    I have come across error before starting target simulation.
> What I means is to execute
> >> >> > >>             qemu-system-riscv64 -M virt -m 256M -nographic
> -bios ./opensbi/build/platform/qemu/virt/firmware/fw_jump.elf -kernel
> vmlinux
> >> >> > >>    then get the error message
> >> >> > >>             rom: requested regions overlap (rom mrom.reset.
> free=0x000000000001eb7c, addr=0x0000000000001000)
> >> >> > >>             qemu-system-riscv64: rom check and register reset
> failed
> >> >> > >
> >> >> > > Hmmm.... I don't remember seeing this problem in my testing, but
> it
> >> >> > > does seem to make sense.
> >> >> > >
> >> >> > > We have two options here:
> >> >> > > 1. Use the Image file from Linux 5.0+ (my preferred option as
> 4.19 is
> >> >> > > getting old)
> >> >
> >> >  I choose 4.19 version rather than 5.0+  because 4.19 is stable
> branch maintained by Greg.
> >> >  But it is ok to use 5.0 trunk, the latest version provided by debian.
> >> >
> >> >>
> >> >> > > 2. Use the device loader (documented here:
> >> >> > >
> https://github.com/riscv/opensbi/blob/master/docs/platform/qemu_virt.md)
> >> >
> >> >
> >> > The document describes two user cases
> >> >   1. fw_payload.elf
> >> >       a. include the Image (raw data file, not elf)  into data section
> >> >       b. -kernel load the each part to the correct address
> >> >
> >> >      the program header of  fw_payload.elf are
> >> >      Type           Offset   VirtAddr           PhysAddr
>  FileSiz  MemSiz   Flg Align
> >> >     LOAD           0x0000b0 0x0000000080000000 0x0000000080000000
> 0x00a008 0x00b0b8 RWE 0x10               <-  fw_payload  main program
> >> >     LOAD           0x00a0b8 0x0000000080200000 0x0000000080200000
> 0x8dbe20 0x8dbe20 R E 0x1                    <-  arch/riscv/boot/Image   or
> u-boot.bin
> >> >
> >> >   2. fw_jump.elf
> >> >       a. use  "-kernel <raw image> or -device  loader,file=<raw
> Image>,addr=0x80200000" to load correct address
> >> >       b. fw_jump.elf just jump to the entry address 0x80200000
> >> >
> >> >
> >> >  The issues are
> >> >    a. debian linux-image package only provides vmlinux.
> >>
> >> This can be fixed by upgrading the kernel version. Fedora should have
> >> a 5.2 kernel that you can use instead of Debian.
> >>
> >> >        so I use objcopy in acceptance test
> >> >    b. riscv64 vmlinux doesn't have correct PhysAddr and -kernel can't
> move data to expected address
> >> >        explicit load the Image to specific
> >> >            -kernel raw_image   (if file is not elf or uImage,
> riscv_kerner_loader treat it is as raw image and place it at 0x8020_0000)
> >> >            -device loader,file=<raw_image>,0x80200000  (do the same
> thing)
> >>
> >> Does the device loader not work?
> >>
> >> Alistair
> >
> >
> >   The device loader works for "Image" case (raw data)
> >   If the data is raw image,  for riscv_load_kernel implementation,
>  "-kernel <raw_image>" and  -device loader,file=<raw_image>,0x80200000  are
> the same.
> >   They finally call "load_image_targphys_as"
> >   The only difference is that target address is hardcoded or feed from
> command line option.
> >
> >   I don't prefe one than the other.  Both of them are Ok to me.  I could
> refine that if one of them is more preferable.
>
> I'm a little confused with what you mean, so if you have something
> that will work just send a new version and we can discuss it there.
>
> PS: I'm still in favour of just using a newer kernel and a Image file.
>
> Alistair
>
>  ok, I will use the latest kernel available in Debian package and refine
the command in next version.

chihmin.chao


> >
> >
> >  chihmin.chao
> >>
> >>
> >> >
> >> >   ELF should work only if it has correct loading information.
> >> >
> >> >   chihmin.chao
> >> >>
> >> >> >
> >> >> > Nice doc, this example is exactly what I had in mind:
> >> >> >
> >> >> >   qemu-system-riscv64 -M virt -m 256M -nographic \
> >> >> >         -kernel build/platform/qemu/virt/firmware/fw_jump.elf \
> >> >> >         -device
> >> >> >
> loader,file=<linux_build_directory>/arch/riscv/boot/Image,addr=0x80200000 \
> >> >> >         -drive file=<path_to_linux_rootfs>,format=raw,id=hd0 \
> >> >> >         -device virtio-blk-device,drive=hd0 \
> >> >> >         -append "root=/dev/vda rw console=ttyS0"
> >> >> >
> >> >> > If the Image is an ELF, I assume we don't need to specify the
> address,
> >> >> > because the device loader knows how to parse this format. Again,
> >> >> > Alistair is the expert ;)
> >> >>
> >> >> Yep, an elf should just work.
> >> >>
> >> >  describe the issue above
> >> >
> >> >>
> >> >> >
> >> >> > If for some reason it only works with Linux 5.0+, let's use these!
> >> >>
> >> >> Only 5.0+ builds an elf.
> >> >>
> >> >> Alistair
> >> >>
> >> >> >
> >> >> > Regards,
> >> >> >
> >> >> > Phil.
>