[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [PATCH 1/4] sm501: Fix bounds checks
From: |
Peter Maydell |
Subject: |
Re: [PATCH 1/4] sm501: Fix bounds checks |
Date: |
Mon, 15 Jun 2020 17:53:54 +0100 |
On Mon, 15 Jun 2020 at 17:40, BALATON Zoltan <balaton@eik.bme.hu> wrote:
>
> On Mon, 15 Jun 2020, Peter Maydell wrote:
> > The calculations for sb/se/db/de all have a term which
> > multiplies by (width + pitch), which makes me suspect
> > they also need a similar fix ?
>
> Maybe. I'll have to check again. Actually is there a simpler way to check
> if two rectangles overlap when they are given with base, x, y, w, h, pitch
> where base is the first byte of the screen, pitch is length of one line
> and x,y is coordinates of top left corner and w,h is dimensions of the
> rect. Now that I think about it we also need to take into accounf the
> bytes per pixel value (1 << format) because base is given in bytes while
> others are in pixels so these formulae likely need some fixes. Pixman has
> some functions for these but those assume common base so to use those we
> would need to bring the two rectangles to common base which I could not
> find out how to do. Probably this is really simple for someone who already
> did a lot of these before.
I think the thing that makes it particularly awkward is that
the source and dest can have different pitches. That means it's
not a simple "do two rectangles overlap" test because the dest
area might not be a rectangle at all when looked at from the
POV of the source.
Do guests usually set src and dst pitch identical? If so it
might be worth having a more accurate rectangle-overlap test
for the common case and a looser check for the hard-to-handle
case.
I might have a think about this and draw some diagrams tomorrow :-)
thanks
-- PMM
- Re: [PATCH 2/4] sm501: Drop unneded variable, (continued)
Re: [PATCH 0/4] More sm501 fixes and optimisations, BALATON Zoltan, 2020/06/12