Re: [qemu-s390x] linux-user: s390x issue on Fedora 30 (dynamic library loader?)

[David Hildenbrand]

On 19.08.19 12:32, Laurent Vivier wrote:
> Le 17/08/2019 à 18:51, David Hildenbrand a écrit :
>> On 17.08.19 18:22, Laurent Vivier wrote:
>>> Le 17/08/2019 à 18:14, David Hildenbrand a écrit :
>>>> On 17.08.19 17:59, David Hildenbrand wrote:
>>>>> Hi everybody,
>>>>>
>>>>> I was just trying to run qemu-s390x (linux-user) with a very simple
>>>>> binary (gzip + lib/ld64.so.1, compiled under Fedora 27). This used to
>>>>> work just fine a while ago (especially when I was working on vector
>>>>> instructions using QEMU v3.1). However, now I can't get past a SEGFAULT
>>>>> in the dynamic library loader (I assume it is trying to locate glibc). I
>>>>> tried a couple of other binaries that definitely used to work (from
>>>>> Fedora 30).
>>>>>
>>>>> I checked QEMU v4.1, v4.0 and v3.1. All are broken for me. Which is
>>>>> weird - because it used to work :/
>>>>>
>>>>> I remember that I was running Fedora 29 the last time I had it running,
>>>>> so my gut feeling is that this is related to some other system library
>>>>> (but which?). I am running on an up-to-date Fedora 30 x86-64 now.
>>>>>
>>>>> Any ideas? Has this been reported already? (not sure if this is a Fedora
>>>>> 30 issue)
>>>>>
>>>>> LANG=C ~/git/qemu/s390x-linux-user/qemu-s390x -d in_asm -L . gzip --help
>>>>>
>>>>> ----------------
>>>>> IN: _dl_load_cache_lookup
>>>>> 0x00000040008854c2:  larl       %r1,0x4000895030
>>>>> 0x00000040008854c8:  lg %r8,264(%r11)
>>>>> 0x00000040008854ce:  mvghi      0(%r1),-1
>>>>> 0x00000040008854d4:  la %r3,0(%r3,%r8)
>>>>> 0x00000040008854d8:  l  %r7,12(%r8)
>>>>> 0x00000040008854dc:  llgfr      %r2,%r7
>>>>> 0x00000040008854e0:  sllg       %r1,%r2,1
>>>>> 0x00000040008854e6:  agr        %r1,%r2
>>>>> 0x00000040008854ea:  sllg       %r1,%r1,2
>>>>> 0x00000040008854f0:  la %r6,16(%r1,%r8)
>>>>> 0x00000040008854f4:  sgr        %r3,%r6
>>>>> 0x00000040008854f8:  stg        %r3,256(%r11)
>>>>> 0x00000040008854fe:  ahi        %r7,-1
>>>>> 0x0000004000885502:  jl 0x40008850f0
>>>>>
>>>>> ----------------
>>>>> IN: _dl_load_cache_lookup
>>>>> 0x0000004000885506:  srak       %r10,%r7,1
>>>>> 0x000000400088550c:  lgfr       %r2,%r10
>>>>> 0x0000004000885510:  sllg       %r1,%r2,1
>>>>> 0x0000004000885516:  agr        %r1,%r2
>>>>> 0x000000400088551a:  sllg       %r1,%r1,2
>>>>> 0x0000004000885520:  l  %r1,20(%r1,%r8)
>>>>> 0x0000004000885524:  clrjhe     %r1,%r3,0x40008850f0
>>>>>
>>>>> Segmentation fault (Speicherabzug geschrieben)
>>>>>
>>>>>
>>>>> Core was generated by
>>>>> `/home/dhildenb/git/qemu/s390x-linux-user/qemu-s390x -d in_asm -L . gzip
>>>>> --help'.
>>>>> Program terminated with signal SIGSEGV, Segmentation fault.
>>>>> #0  0x00007fdc5d7c3232 in sigsuspend () from /lib64/libc.so.6
>>>>> [Current thread is 1 (Thread 0x7fdc5d7127c0 (LWP 31072))]
>>>>> Missing separate debuginfos, use: dnf debuginfo-install
>>>>> glib2-2.60.6-1.fc30.x86_64 glibc-2.29-15.fc30.x86_64
>>>>> libgcc-9.1.1-1.fc30.x86_64 libstdc++-9.1.1-1.fc30.x86_64
>>>>> pcre-8.43-2.fc30.x86_64 zlib-1.2.11-16.fc30.x86_64
>>>>> (gdb) bt
>>>>> #0  0x00007fdc5d7c3232 in sigsuspend () from /lib64/libc.so.6
>>>>> #1  0x000055f826135a9c in dump_core_and_abort
>>>>> (target_sig=target_sig@entry=11)
>>>>>     at /home/dhildenb/git/qemu/linux-user/signal.c:613
>>>>> #2  0x000055f826135e37 in handle_pending_signal
>>>>> (cpu_env=cpu_env@entry=0x55f8292cec48, sig=sig@entry=11,
>>>>>     k=k@entry=0x55f8292d7df0) at
>>>>> /home/dhildenb/git/qemu/linux-user/signal.c:877
>>>>> #3  0x000055f826136edd in process_pending_signals
>>>>> (cpu_env=cpu_env@entry=0x55f8292cec48)
>>>>>     at /home/dhildenb/git/qemu/linux-user/signal.c:953
>>>>> #4  0x000055f82613a13a in cpu_loop (env=0x55f8292cec48) at
>>>>> /home/dhildenb/git/qemu/linux-user/s390x/cpu_loop.c:150
>>>>> #5  0x000055f8260ce2ba in main (argc=<optimized out>,
>>>>> argv=0x7fff587a69d8, envp=<optimized out>)
>>>>>     at /home/dhildenb/git/qemu/linux-user/main.c:819
>>>>>
>>>>> Thanks!
>>>>>
>>>>
>>>> CCing QEMU-devel + use my proper dev mail address (I need more coffee :))
>>>>
>>>
>>> I generally test qemu-s390x before my PR. Last time, I have tested with
>>> Fedora 30 x86_64 but my target are always debian.
>>
>> What puzzles me is that it used to work just fine about 3-4 months ago.
>> I still have the binaries/libs lying around that I used back then (when
>> debugging a vector instruction-related issue). Whatever binary/QEMU
>> version I try now, it keeps segfaulting.
>>
>> Via qemu-system-s390x, inside a Fedora guest, the binaries work
>> perfectly fine. So I really suspect that this has to do with my host system.
>>
>>>
>>> So I guess the problem is with the target.
>>>
>>> I will have a look on Monday.
>>
> 
> I'm not able to reproduce it. In a chroot or using "-L" with only
> /bin/gzip, /lib64/ld64.so.1 and /lib/libc.so.6 in the directory.
> 
> My host is Fedora 30 (updated today) and the target directory is Fedora
> 30 too (from
> https://download-ib01.fedoraproject.org/pub/fedora-secondary/releases/30/Container/s390x/images/Fedora-Container-Minimal-Base-30-1.2.s390x.tar.xz)
> 
> Do you have more details?

Thanks for having a look!

I just tried with the Fedora 30 target files they seem to work just fine
indeed (chroot and -L after manually copying the two lib files).

Now, I took the same files from RHEL8 (everything is built with
z13/vector instruction support enforced under RHEL8). (I copied them
from a running guest using scp) - rhel-guest-image-8.0-1854.s390x.qcow2.


Trying to run them results in the reported issue.

~/git/qemu/s390x-linux-user/qemu-s390x  -L . gzip
Segmentation fault (Speicherabzug geschrieben)

I can spot one main difference:

/lib64/libc-2.28.so: ELF 64-bit MSB shared object, IBM S/390, version 1
(GNU/Linux), dynamically linked, interpreter /lib/ld64.so.1,
BuildID[sha1]=f2ed86061df0bad28270244424e58eb95f60c00c, for GNU/Linux
3.2.0, not stripped, too many notes (256)

/lib64/ld-2.28.so: ELF 64-bit MSB shared object, IBM S/390, version 1
(SYSV), dynamically linked,
BuildID[sha1]=6a58fd6ea86ec36455655ff718509ee4320cefc2, not stripped,
too many notes (256)

The libraries are not completely stripped. But even when stripping, I
get the same issue when trying to load them.

The binaries/libraries work perfectly fine within qemu-system-s390x.

-- 

Thanks,

David / dhildenb