Re: [fluid-dev] We need a new, better sequencer!

[Antoine Schmitt]

Hello Marcus and Tom and all,

as the original implementor of the fluidsynth sequencer, and using my 
engineer's glasses ("make it simple"), here are my opinions and thoughts on 
these matters :

- C++ would a big dependency to add to fluidsynth.. I leave it to the community 
to decide whether or not to add it
- if the performance (speed) of adding a large number of events at once can be 
improved using a C++ implementation, it surely can also be improved using an 
optimization of the current C implementation.
- about re-ordering events happening on the same tick : it is always good to 
make the client responsible for its actions and not make decisions on its 
behalf. The "first added, first served" implementation seems the most 
straightforward to me, especially if most MIDI players do it also.
- the sequencer is also often used for real time usage (this was my usage), 
where events are added every pattern (1 second or so) regarding the upcoming 
pattern to be rendered. This means that the System Timer feature is actually 
used by clients of fluidsynth. If it is removed from fluidsynth, provisions 
should be made for clients to implement their own timer and callback to the 
sequencer rendering function.
- I have no opinion regarding the Time Scale. I guess that this can be used to 
optimize speed if time scale is known to be lower than the standard one. But 
maybe this is useless..

Hope this helps
Antoine



> Le 10 déc. 2019 à 11:03, Tom M. via fluid-dev <address@hidden> a écrit :
> 
> Thanks for your thoughts, Marcus.
> 
>>> 2. A meaningful ordering?
>>> To overcome this problem I'm suggesting to implement an ordering function 
>>> which makes sure that NoteOn events are always last (within the same tick 
>>> count).
> 
>> As you say: almost all MIDI players fall back to the event order, so why 
>> shoudl Fluidsynth be different?
> 
> I'm looking at it with my software engineer glasses: Whenever something has 
> undefined behaviour, one can think about how to tweak and optimize things.
> 
>> Why is "Note-On last" the better order? How do you know that that special 
>> (and seemingly Fluidsynth specific) ordering is what the author of the MIDI 
>> file wanted?
> 
> When something happens at the same tick, I don't claim to know anything. My 
> proposed solution is simply the best guess I can come up with in such a 
> situation. Two examples:
> 
> a) NoteOn and ProgChange happen in that order at the same tick. Would it make 
> sense to turn on the note using the old instrument, although we are just 
> about to receive a new program at the same tick? I don't think so. The 
> NoteOn, although first in the stream, should use the new program.
> 
> b) NoteOn at tick 0, after e.g. 100 ticks: NoteOn and NoteOff. All three note 
> events happen in the given order on the same channel and trigger the same 
> key. How would fluidsynth interpret them currently? The first note is 
> triggered, it sounds for a while until the second NoteOn arrives. That second 
> NoteOn kills overlapping notes, i.e. it kills the first note. But oops, just 
> after that second note was turned on, it will be turned off right again, 
> because we have a NoteOff at the same tick, which apparently was meant to 
> turn off the first note though.
> 
> For case b) I have MIDI files that do such things. Ofc you are right Marcus, 
> technically the MIDI converter program is to blame. I cannot really change 
> the converter though. I would alternatively need to handle those cases in my 
> program. I was just thinking of a possibly acceptable way for fluidsynth to 
> handle such "corner-cases".
> 
> 
>>> 3. Simpler implementation?
>> What trips me up though: if your program was really stuck in that loop for 
>> seconds, I'm wondering how many events you were actually adding to the 
>> sequencer... Millions?
> 
> I'm takling about ten-thousands of events. Here are some measurements:
> 
> virtual void MidiWrapper::open(): Added 23976 events to the seq, took 12 ms
> void FluidsynthWrapper::Render(float*, frame_t): Rendering took 897 ms
> 
> virtual void MidiWrapper::open(): Added 28395 events to the seq, took 17 ms
> void FluidsynthWrapper::Render(float*, frame_t): Rendering took 1441 ms
> 
> virtual void MidiWrapper::open(): Added 84961 events to the seq, took 48 ms
> void FluidsynthWrapper::Render(float*, frame_t): Rendering took 19709 ms
> 
> The problem is the very first rendering call (in my case: 
> fluid_synth_process), when the sample-timers are called back until they 
> ultimately reach the while loop I mentioned. You can see there seems to be an 
> exponential increase. ATM, I just tested 4 different MIDI files. The first 3 
> files have a playduration of 4-6 minutes, the last one has 12 minutes 
> playback. Ofc, this heavily relies on the timing order of the events, so 
> making a generalized benchmark is not so straight forward to do.
> 
> Same test, but with std::priority_queue :
> 
> virtual void MidiWrapper::open(): Added 23976 events to the seq, took 9 ms
> void FluidsynthWrapper::Render(float*, frame_t): Rendering took 4 ms
> 
> virtual void MidiWrapper::open(): Added 28395 events to the seq, took 16 ms
> Warning: void FluidsynthWrapper::Render(float*, frame_t): Rendering took 3 ms
> 
> MidiWrapper::open(): Added 84961 events to the seq, took 39 ms
> Warning: void FluidsynthWrapper::Render(float*, frame_t): Rendering took 4 ms
> 
> 
>>> 5. The system timer
>> isn't there a global timing event that MIDI devices can send to each other 
>> to synchronise playback position and playback speed?
> 
> You seem to refer to "System Real Time Messages". But I don't know how this 
> could be integrated into fluidsynth.
> 
>>> 6. Time Scale
>> Similar comment as above and again, just thinking out loud: wouldn't it be a 
>> great feature to allow slower or faster MIDI playback?
> 
> Indeed, supporting tempo changes would be a great and necessary feature for 
> the seq, if we ever want to use it in the midi player. Ok, so better improve 
> the time scaling rather than remove it.
> 
> 
>> adding another implementation in C++ is not cleanup IMO... it adds more 
>> technological debt because we now have to keep two different implementations 
>> in sync. 
> 
> Having two implementations would be temporarily. First add an "experimental" 
> C++ impl., then deprecate the current C implementation while using the C++ 
> impl. by default and finally remove the deprecated.
> 
>> But I do wonder: what can C++ do that C can't? And especially: why can C++ 
>> do it faster?
> 
> Simply that I'm more familiar how to do those things in C++ rather than with 
> glib. And I'm not sure what Carlo, you, and the other embedded system guys 
> said, if we fortify the glib dependency. I'm not sure how they think about a 
> dependency to the C++ std lib either...
> 
> 
> Tom
> 
> 
> 
> 
> _______________________________________________
> fluid-dev mailing list
> address@hidden
> https://lists.nongnu.org/mailman/listinfo/fluid-dev