[Gnash] Status of Configurable GStreamer Pipeline

[Sean McNamara]

Hi All,

Those of you hanging out on FreeNode in #gnash know of my recent work
on making Gnash's GStreamer pipeline configurable. There are many
approaches to this, and I wanted to choose one that is both flexible
and not dependent on other libraries that Gnash doesn't currently
depend on.

My status as of this morning is as follows, peeling apart the work in layers:

1. Configuration level: I modified rc.cpp to accept a parameter to
~/.gnashrc called "GSTAudioSink", a string, which is your one-element
pipeline that you want to use. This element must begin by taking data
of type audio/x-raw-* as input, and outputting in a "sink" element.
Note that you CAN use "bin" type elements here. For example,
gconfaudiosink constructs an arbitrarily complex pipeline starting
with audio/x-raw-* and ending with a sink, simply by referencing it --
it depends on what you have in GConf. For example, it is possible to
change the pitch of all of your Gnash movies by going to
/system/gstreamer/0.10/default/audiosink and changing it to something
like

audioconvert ! pitch pitch=0.9 tempo=1.0 ! audioconvert !
audioresample ! your_output_sink

An example of a valid ~/.gnashrc entry for this functionality would be

set GSTAudioSink pulsesink

However, the current code is not smart enough to take something like

set GSTAudioSink one_element ! two_element ! three_element ! pulsesink

because this is a pipeline and I would have to create each of these
elements and link them together. I am working on that gradually, but
for now, try to keep it to a single element (even if that element
eventually expands into a more complex sub-pipeline).

2. Functionality layer: Works as intended, except for the lack of
explicit sub-pipeline definition as mentioned above. Numerically, at
least 70% done.

3. Code layer: The code is a mess -- partly because I am new to C++
and partly because this was a quick hack. I tried to use good style
and spacing consistent with the rest of the source (as consistent as
possible on a project with many maintainers), but it's lacking in
comments. Logging, however, is rather verbose and thorough. If the new
functionality can't get a pipeline, you'll definitely know it. I also
log non-fatal messages when I get a valid element.

4. Security layer: Not tested for security. I guess someone could
probably buffer overflow you or do something evil like "set
GSTAudioSink" (with no parameter). Although, any valid std::string
that can be converted to a c_str() should be able to be passed to
gst_element_factory_make(), which just returns NULL whenever the
parameter is not a valid element. And yes, I check for NULL return
every time I call this method.

5. Compatibility layer: If all attempts to get a valid pipeline
through ~/.gnashrc and gconfaudiosink fail, it will fall back to the
(previously hard-coded) "autoaudiosink". Additionally, an unmodified
gstreamer install will have "autoaudiosink" in GConf, so if you have
no customization on your GConf/Gstreamer settings, you'll just get
what is currently implemented. So, worst case, there's a little bit
more overhead each time a new pipeline is initialized. And that brings
us to...

6. Performance layer: Not optimized for performance. The whole method
of playing sound in Gnash needs to be revised, but this is a larger
job. Ideally, we would be able to construct a single sound pipeline
per flash movie, mix various simultaneous sounds together, and play
dead air (zero-volume signal) in between sounds. I'm not sure if
gstreamer can handle the mixing for us, or not, but that's another
topic for later. Anyway, for now, there is a slight additional
overhead each time a sound is played in a flash movie. For
long-running sounds like YouTube videos, this is negligible; for flash
games that constantly start new sound effects, this might pose a
performance bottleneck on older systems. Like I said, this would be
resolved at a higher level of design than that which I am currently
addressing.

7. Documentation layer: The manual (docbook) has not been updated yet
by me. I need to add in documentation of the GSTAudioSink parameter.

8. Testing layer: I have not written any tests yet. I am unfamiliar
with ALL of the testing frameworks currently used by Gnash, and would
appreciate any help or guidance in getting some unit tests written.
Since I would like to get this committed sooner rather than later, I
would prefer to write what the developers would largely consider
"minimally acceptable" to test this functionality, see that it passes,
and submit the patch for review.

Speaking of patch, what is the preferred format and venue for
submitting patches to Gnash?

Thanks so much,

Sean McNamara