Re: Re[2]: [Gnash] Raw video/audio

[Craig Kelley]

Hi Udo,

On Sat, Mar 22, 2008 at 3:53 AM, Udo Giacomozzi <address@hidden> wrote:
> Hello Craig,
>
>  Friday, March 21, 2008, 6:13:37 PM, you wrote:
>  CK> It is an X11 extension that uses hardware acceleration to scale planar
>  CK> data without using the general-purpose CPU.  Most video cards support
>  CK> this extension under XFree86/xorg.  The Windows analog would be in
>  CK> DirectX somewhere.  With it, you can take a "small" window and blow it
>  CK> up to any size for very little cost.  It was added to Adobe's latest
>  CK> Linux standalone and browser plugins (9.0.115?).  Here is more
>  CK> information about it:
>
>  CK> http://en.wikipedia.org/wiki/Xvideo
>
>  Since it is about scaling video it is more interesting to use it for
>  embedded video. AFAIK the video stream we get is in YUV and needs to
>  be converted to RGB. So XVideo could scale/translate the video and
>  bring it to RGB color space. The rest (vector rendering) should remain
>  unaffected by XVideo to keep the quality. For video playback
>  (especially full screen playback) this could mean a significant
>  performance gain.

I had considered this, but found some problems that I couldn't resolve
simply.  XVideo can only scale a rectangular window (x, y, width,
height).  This would preclude having a layer of video with a layer
vector graphics above it if the vector graphics couldn't share the
same color space as the video graphics.  Namely, the vector graphics
need to be put in the final buffer in YUV format.

XVideo does support masks in the form of color keys, but this would
require fairly tight integration into the vector rendering engine.
Maybe this is feasible?

Observing the behavior of Adobe's implementation, it seems that they
opted to send all the information in YUV format to the XVideo X11
extension.  You can put scripted vector graphics on top of an FLV, and
it will still use XVideo to scale the whole thing up to "fullscreen
mode".

>  So, instead of scaling the final frame, we should integrate XVideo in
>  the rendering process.
>
>  It's still interesting to use XVideo to scale the final frame since
>  Adobe lately introduced a new mode where the rendered vector is
>  scaled up for full screen playback:
>  http://www.adobe.com/devnet/flashplayer/articles/full_screen_mode.html

Yes; it is compelling to have this ability.  I took a 500x450 SWF
animation (no video on the stage at all) and dumped it from gnash with
my experimental code, used mencoder to wrap it up in an MPEG-4/AC3 AVI
file, and then used mplayer to blow it up to 1600x1200.  It looked
VERY nice, and used next to zero CPU time to play back.

FFMPEG has an rgb2yuv routine in swscale.  I may play around with that
to see if the translation is too expensive.  It has a lot of inline
x86 MMX and SSE assembly in it, and I'm not sure how amiable the gnash
developers are to such things.  It's a double-negative for things like
H.264, which have been moved from YUV to RGB in the renderer already
-- and now would be moved *back* via resampling... :-/

 -Craig

-- 
http://inconnu.islug.org/~ink finger address@hidden for PGP block