[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Qemu-block] [Qemu-devel] Migration sometimes fails with IDE and Qem
From: |
Peter Lieven |
Subject: |
Re: [Qemu-block] [Qemu-devel] Migration sometimes fails with IDE and Qemu 2.2.1 |
Date: |
Tue, 07 Apr 2015 21:02:34 +0200 |
User-agent: |
Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.4.0 |
Am 07.04.2015 um 20:56 schrieb John Snow:
>
>
> On 04/07/2015 02:44 PM, Peter Lieven wrote:
>> Am 07.04.2015 um 17:29 schrieb Dr. David Alan Gilbert:
>>> * Peter Lieven (address@hidden) wrote:
>>>> Hi David,
>>>>
>>>> Am 07.04.2015 um 10:43 schrieb Dr. David Alan Gilbert:
>>>>>>>> Any particular workload or reproducer?
>>>>>>> Workload is almost zero. I try to figure out if there is a way to
>>>>>>> trigger it.
>>>>>>>
>>>>>>> Maybe playing a role: Machine type is -M pc1.2 and we set -kvmclock as
>>>>>>> CPU flag since kvmclock seemed to be quite buggy in 2.6.16...
>>>>>>>
>>>>>>> Exact cmdline is:
>>>>>>> /usr/bin/qemu-2.2.1 -enable-kvm -M pc-1.2 -nodefaults -netdev
>>>>>>> type=tap,id=guest2,script=no,downscript=no,ifname=tap2 -device
>>>>>>> e1000,netdev=guest2,mac=52:54:00:ff:00:65 -drive
>>>>>>> format=raw,file=iscsi://172.21.200.53/iqn.2001-05.com.equallogic:4-52aed6-88a7e99a4-d9e00040fdc509a3-XXX-hd0/0,if=ide,cache=writeback,aio=native
>>>>>>> -serial null -parallel null -m 1024 -smp
>>>>>>> 2,sockets=1,cores=2,threads=1 -monitor tcp:0:4003,server,nowait -vnc
>>>>>>> :3 -qmp tcp:0:3003,server,nowait -name 'XXX' -boot
>>>>>>> order=c,once=dc,menu=off -drive
>>>>>>> index=2,media=cdrom,if=ide,cache=unsafe,aio=native,readonly=on -k de
>>>>>>> -incoming tcp:0:5003 -pidfile /var/run/qemu/vm-146.pid -mem-path
>>>>>>> /hugepages -mem-prealloc -rtc base=utc -usb -usbdevice tablet
>>>>>>> -no-hpet -vga cirrus -cpu qemu64,-kvmclock
>>>>>>>
>>>>>>> Exact kernel is:
>>>>>>> 2.6.16.46-0.12-smp (i think this is SLES10 or sth.)
>>>>>>>
>>>>>>> The machine does not hang. It seems just I/O is hanging. So you can
>>>>>>> type at the console or ping the system, but no longer login.
>>>>>>>
>>>>>>> Thank you,
>>>>>>> Peter
>>>>>> Interesting observation: Migrating the vServer again seems to fix to
>>>>>> problem (at least in one case I could test just now).
>>>>>>
>>>>>> 2.6.8-24-smp is also affected.
>>>>> How often does it fail - you say 'sometimes' - is it a 1/10 or a 1/1000 ?
>>>> Its more often than 1/10 I would say.
>>> OK, that's not too bad - it's the 1/1000 that are really nasty to find.
>>> In your setup, how easy would it be for you to try :
>>> with either 2.1 or current head?
>>> with a newer machine-type?
>>> without the cdrom?
>>
>> Its all possible. I can clone the system and try everything on my test
>> systems. I hope
>> it reproduces there.
>>
>> Has the cdrom the power of taking down the bus?
>>
>> Peter
>>
>
> I don't know if CDROM could stall the entire bus, but I suspect the reason
> for asking is this: dgilbert and I had tracked down a problem previously
> where during migration, outstanding requests being handled by the ATAPI code
> can get lost during migration if, for instance, the user has only prepared
> the command (via bmdma) but has not yet written to the register to activate
> the command yet.
That sounds like it could be related.
>
> So if something like this happens:
>
> - User writes to the ATA registers to program a command
> - Migration occurs
> - User writes to the BMDMA register to initiate the command
>
> We can lose some of the state and data of the request. David had checked in a
> workaround for at least ATAPI that simply coaxes the guest OS into trying the
> command again to unstick it.
Do you have the commit for me?
>
> I think we determined last time that we couldn't fix this problem without
> changing the migration format, so we opted not to do it for 2.3. We had also
> only noticed it with ATAPI drives, not HDDs, so a proper fix got kicked down
> the road since we thought the workaround was sufficient.
Maybe normally we use virtio nowadays and maybe the new kernel implementation
(libata /dev/sdX) can't get locked? What I do not understand is how a second
migration can unlock from this state?
>
> IIRC our success rate with reproducing it was something on the order of 1/50,
> too.
>
> If you can reproduce it without a CDROM but using the BMDMA interface, that's
> a good data point. If you can't reproduce it using the ISA interface, that's
> a phenomenal data point and implicates BMDMA pretty heavily.
To be 100% sure we are talking about the same? How would I use the ISA and how
would I use the BMDMA interface?
Thanks,
Peter
- Re: [Qemu-block] Migration sometimes fails with IDE and Qemu 2.2.1, (continued)
- Re: [Qemu-block] Migration sometimes fails with IDE and Qemu 2.2.1, John Snow, 2015/04/06
- Re: [Qemu-block] Migration sometimes fails with IDE and Qemu 2.2.1, Peter Lieven, 2015/04/06
- Re: [Qemu-block] Migration sometimes fails with IDE and Qemu 2.2.1, Peter Lieven, 2015/04/06
- Re: [Qemu-block] [Qemu-devel] Migration sometimes fails with IDE and Qemu 2.2.1, Dr. David Alan Gilbert, 2015/04/07
- Re: [Qemu-block] [Qemu-devel] Migration sometimes fails with IDE and Qemu 2.2.1, Peter Lieven, 2015/04/07
- Re: [Qemu-block] [Qemu-devel] Migration sometimes fails with IDE and Qemu 2.2.1, Paolo Bonzini, 2015/04/07
- Re: [Qemu-block] [Qemu-devel] Migration sometimes fails with IDE and Qemu 2.2.1, Peter Lieven, 2015/04/07
- Re: [Qemu-block] [Qemu-devel] Migration sometimes fails with IDE and Qemu 2.2.1, Dr. David Alan Gilbert, 2015/04/07
- Re: [Qemu-block] [Qemu-devel] Migration sometimes fails with IDE and Qemu 2.2.1, Peter Lieven, 2015/04/07
- Re: [Qemu-block] [Qemu-devel] Migration sometimes fails with IDE and Qemu 2.2.1, John Snow, 2015/04/07
- Re: [Qemu-block] [Qemu-devel] Migration sometimes fails with IDE and Qemu 2.2.1,
Peter Lieven <=
- Re: [Qemu-block] [Qemu-devel] Migration sometimes fails with IDE and Qemu 2.2.1, John Snow, 2015/04/07
- Re: [Qemu-block] [Qemu-devel] Migration sometimes fails with IDE and Qemu 2.2.1, Peter Lieven, 2015/04/09
- Re: [Qemu-block] [Qemu-devel] Migration sometimes fails with IDE and Qemu 2.2.1, Peter Lieven, 2015/04/09
- Re: [Qemu-block] [Qemu-devel] Migration sometimes fails with IDE and Qemu 2.2.1, Paolo Bonzini, 2015/04/09
- Re: [Qemu-block] [Qemu-devel] Migration sometimes fails with IDE and Qemu 2.2.1, Dr. David Alan Gilbert, 2015/04/07
- Re: [Qemu-block] [Qemu-devel] Migration sometimes fails with IDE and Qemu 2.2.1, Peter Lieven, 2015/04/07
- Re: [Qemu-block] [Qemu-devel] Migration sometimes fails with IDE and Qemu 2.2.1, Peter Lieven, 2015/04/09
- Re: [Qemu-block] [Qemu-devel] Migration sometimes fails with IDE and Qemu 2.2.1, Peter Lieven, 2015/04/09
- Re: [Qemu-block] [Qemu-devel] Migration sometimes fails with IDE and Qemu 2.2.1, Dr. David Alan Gilbert, 2015/04/09
- Re: [Qemu-block] [Qemu-devel] Migration sometimes fails with IDE and Qemu 2.2.1, Peter Lieven, 2015/04/09