[fluid-dev] We need a new, better sequencer!

[Tom M.]

Current situation: I have a program[1], which reads standard MIDI files, 
processes it's events and finally adds all MIDI events to fluid_sequencer.

The current sequencer sucks:

1. It is slow.
2. A meaningful ordering for events with the same tick has not been considered.
3. Its implementation codebase is *huge and complicated*, implementing its own 
heap, maintaining five internal queues, etc.
4. Events that have been processed are deleted and gone.
5. The sequencer supports the system timer as alternative time source.
6. Time Scaling that only allows to make things worse.

1. Why is it slow?

My program supports MIDI track loops. It unrolls them by adding all MIDI events 
within the loop over and over again to the sequencer. Thus, the sequencer has 
to cope with many events, most of them will be dispatched in the far future. 
For such a scenario, there is a while() loop [2] where it tries to determine 
where to insert a new event, which turns out to be the bottleneck.

2. A meaningful ordering?

As per MIDI standard, events are ordered by tick count. If two events share the 
same tick count, the order is undefined. In such a case almost all MIDI players 
out there fall back to the order the events appear in the "stream" (e.g. file). 
IMO, this doesn't make sense. Let me remind you of an issue [3], where a 
progChange and a NoteOn happen at the same tick. Question: Which instrument 
will the NoteOn trigger? The new program or the old program?

To overcome this problem I'm suggesting to implement an ordering function which 
makes sure that NoteOn events are always last (within the same tick count). On 
the other hand FLUID_SEQ_UNREGISTERING events are always first within the same 
tick count. This is because if an unregistering happens at a given tick, it 
doesn't make any sense to keep posting events, for which the client won't have 
time to process them, because the client will be unregistered at the same tick 
anyway. For all other events the order is retained.

3. Simpler implementation?

I just had a quick try using C++ std::priority_queue. The results are very 
promising in terms of performance. Whereas the UI thread of my program 
previously was stuck in [2] for seconds, with the C++ implementation I'm unable 
to notice any delay.

4. Processed events are gone

Why does it matter? Because originally, fluid_sequencer_t along with its 
fluid_event_t were designed to replace fluid_player_t and fluid_midi_event_t. 
Meanwhile, the fluid_player supports seeking, thus events must not be deleted 
after they have been processed.
So, if we ever want to have a realistic chance to remove fluid_midi_event_t and 
replace it by fluid_event_t and its sequencer, this must be considered.

5. The system timer

I see no reason to retain support for the system timer. The only time source 
for dispatching events at the right time should be the sample timer. Whether 
this actually happens in real-time or not, depends on whoever calls the 
rendering functions of the synth, e.g. audio driver vs. fast-file-renderer.

6. Time Scale

Also, I see no reason to keep the time scaling. The default scale is 1 tick per 
milisecond. There is no way to get a higher resolution, you can only make the 
resolution worse. Why would you want to do this? (Perhaps in case of the system 
timer, to avoid that it fires a callback every milisecond. But this is not a 
reason anymore, see 5.)


To address those issues I see two ways to go:

1. Next to the current sequencer implementation, add a new C++ implementation 
for the sequencer and decide at cmake time whether to use the old pure C or the 
new C++ implementation.
2. Replace the current implementation of the sequencer with some heavy glib 
usage.

Any opinions or comments highly welcome.


Tom


[1] https://github.com/derselbst/ANMP
[2] 
https://github.com/FluidSynth/fluidsynth/blob/08848864d246bd0371b12e493e4e927baff77d08/src/midi/fluid_seq.c#L1030-L1042
[3] https://lists.nongnu.org/archive/html/fluid-dev/2017-05/msg00004.html