When validating the parameters of VBE ioport writes the X co-ordinate
is silently rounded down to a multiple of 8. For valid resolutions
(such as 1366x768) which are not divisible by 8 this causes
miscalculations because the display surface has shorter lines than
expected. For example, a VNC client receives updates with a
"staircase" effect.
Reduce the rounding to a multiple of two.
Signed-off-by: Simon Rowe <simon.rowe@nutanix.com>
---
hw/display/vga.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/hw/display/vga.c b/hw/display/vga.c
index 892fedc8dc..ea659e2812 100644
--- a/hw/display/vga.c
+++ b/hw/display/vga.c
@@ -581,14 +581,14 @@ static void vbe_fixup_regs(VGACommonState *s)
}
/* check width */
- r[VBE_DISPI_INDEX_XRES] &= ~7u;
+ r[VBE_DISPI_INDEX_XRES] &= ~1u;
if (r[VBE_DISPI_INDEX_XRES] == 0) {
r[VBE_DISPI_INDEX_XRES] = 8;
}
if (r[VBE_DISPI_INDEX_XRES] > VBE_DISPI_MAX_XRES) {
r[VBE_DISPI_INDEX_XRES] = VBE_DISPI_MAX_XRES;
}
- r[VBE_DISPI_INDEX_VIRT_WIDTH] &= ~7u;
+ r[VBE_DISPI_INDEX_VIRT_WIDTH] &= ~1u;
if (r[VBE_DISPI_INDEX_VIRT_WIDTH] > VBE_DISPI_MAX_XRES) {
r[VBE_DISPI_INDEX_VIRT_WIDTH] = VBE_DISPI_MAX_XRES;
}
--
2.22.3