[Qemu-devel] [Bug 1832422] [NEW] SSE CMP ops with 8bit immediate throw sigill with oversized byte

[address@hidden]

Public bug reported:

The SSE comparison ops that use an 8bit immediate as a comparison type
selector throws a sigill when the immediate is oversized.

Test op that I found this on is here `66 0f c2 c0 d1          cmppd  
xmm0,xmm0,0xd1`
According to the x86-64 documentation only bits [2:0] are used for these ops 
(and [4:0] for the AVX variant)
Currently qemu just checks if the value is >=8 and will throw a sigill in that 
case. It instead needs to mask.

I have a small patch that fixes the issue for the SSE variant.

** Affects: qemu
     Importance: Undecided
         Status: New

** Patch added: "Fixes the bug"
   
https://bugs.launchpad.net/bugs/1832422/+attachment/5270247/+files/compare_ops.diff

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1832422

Title:
  SSE CMP ops with 8bit immediate throw sigill with oversized byte

Status in QEMU:
  New

Bug description:
  The SSE comparison ops that use an 8bit immediate as a comparison type
  selector throws a sigill when the immediate is oversized.

  Test op that I found this on is here `66 0f c2 c0 d1          cmppd  
xmm0,xmm0,0xd1`
  According to the x86-64 documentation only bits [2:0] are used for these ops 
(and [4:0] for the AVX variant)
  Currently qemu just checks if the value is >=8 and will throw a sigill in 
that case. It instead needs to mask.

  I have a small patch that fixes the issue for the SSE variant.

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1832422/+subscriptions