Re: [fluid-dev] Timing revisited

[Josh Green]

On Fri, 2009-04-10 at 15:09 +0200, David Henningsson wrote:
> What first was a quick fix that took an hour or two to write, expanded
> to fix ticket #15, and should also be expanded to fix the sequencer. And
> now I'm about to suggest that timing should be a separate
> component/subsystem.
> 
> First some history (from someone who has only been around the project a
> month!). Fluidsynth's was initially designed for live playing only but
> it's real strength - and where it is most used - is as a rendering
> library, you put midi & sf2 in, you get audio out. The synth component
> is working as a rendering library, and with my patches we have a MIDI
> player and a file renderer component that also works in the rendering
> library context.
> 
> As a rendering library, there is almost no use for anything else than
> sample timers. But for live, real-time playing usage, there are needs
> for synchronization with other things, wall clock, midi clock, and
> perhaps other timing sources. Now the existing wall clock timer
> implementation (or its interaction with other components) has at least
> two serious flaws, 1) it has concurrency problems and 2) it works bad
> with large audio period sizes.
> 
> To solve the concurrency issue, we must not input MIDI to the synth
> during fluid_synth_one_block, only between. A callback from the synth to
> the "timing component" before or after fluid_synth_one_block serves two
> purposes, 1) it says that now it's ok to input some more MIDI, 2) it
> says that the sample timer has advanced. The timing component can then
> call registered players/sequencers to tell them to input MIDI.
> If this is a rendering library, nothing else is needed. But for
> real-time playing, the timing component must resynchronize with the wall
> clock (or something else). This can be done rather easily. If the wall
> clock has a triggered event every second, we correct the elapsed time
> every second, but we use the sample timers inbetween for millisecond
> precision.
> 
> Assuming you agree that this is a good idea, should such a timing
> component be scheduled for 1.1.0 or should it be in the 2.0 branch?
> 
> // David
> 

Hello David,

I personally think that the timing issues should be sorted out in the
1.x branch.

As for using a callback for resolving the concurrency issues.  That
sounds to me like you may still end up with race conditions, it would
just make them less likely to happen.  What about using some sort of
message queue to pass the MIDI events to the synth?  I imagine it is
probably good to try and avoid locks if at all possible in the synthesis
thread, but perhaps some lock-less mechanism can be used (circular
buffer for example) to pass the events.  Does this make sense?  glib has
portable atomic integers which could be used for this task.

I'm not sure how to resolve the issue with driver buffer sizes and MIDI
timing precision in the real time case.  It seems like we may just have
to be content with the MIDI events being processed at the audio driver's
buffer size for now.  If the user can get their audio buffer size low
enough though, then the problem goes away, which is what someone would
want anyways for the realtime scenario.  For the non realtime case, the
sequencer API could be used, though it sounds like it needs some work.

I'm still not as informed as I'd like to be with the actual code and
issues involved, so please let me know if there are faults in my
thinking.

        Josh