Re: [fluid-dev] New development

[Josh Green]

On Mon, 2009-01-26 at 12:06 +0100, Bernat Arlandis i Mañó wrote:
> Still, keep in mind that it's not my main goal splitting into separate libs
> but having good modularization to the point this would be really easy and
> practical.
> 

Sounds good.

> > libInstPatch *does* handle 24 bit audio and floating point as well.  Its
> > got its own audio managing for converting formats internally and what
> > not.  Swami and libInstPatch support 24 bit audio in SoundFont files.
> > FluidSynth does not.  That is what I would like to change.
> > 
> I'll need help there with libInstPatch integration. I don't know exactly
> what libInstPatch can do and what not, but using it seems a good idea.
> 

There is still a little work that I would like to do with libInstPatch
to bring it up to the task.  The sample caching code is what I have been
working on.  Swami loads instruments on demand, rather than a whole
SoundFont all at once.  I haven't yet implemented code for managing this
cache as it grows and freeing unused samples and what not, but its in
the works.

> > No that isn't quite right.  The SoundFont loader API is used for
> > synthesis in FluidSynth (not loading SoundFont files themselves).
> > libInstPatch and Swami do their own instrument management, but when they
> > want to synthesize those instruments, the SoundFont loader API is used.
> > This API abstracts the synthesis into a set of voices which can be
> > created by the application.  The voices have a list of SoundFont based
> > parameters, modulators and sample data.  In this way though, FluidSynth
> > can be used to synthesize any format, well at least within the confines
> > of SoundFont parameters.  Its a flexible API, but I think it could use
> > some cleanup and expansion of its capabilities (different audio formats
> > for example, like 24 bit).
> > 
> That's really interesting, this is what I like the least from FS.
> Theoretically this would help supporting every sound font format but it
> becomes a very hard thing to do mainly because when trying you'll be
> implementing a synthesis engine inside every font loader. There's another
> solution that would work best. Later on this.
> 

Actually, it seems to work pretty well abstracting instruments into
voices.  I think the way it is modeled is OK, its just that the API
needs some cleanup.  I'm not sure what you mean by "implementing a
synthesis engine inside every font loader".  If you mean, loading the
instruments into memory, calculating the parameters and creating the
appropriate voices when a note-on is pressed, that is what libInstPatch
and the FluidSynth Swami plugin does.

> > I'm starting to think having libInstPatch be a dependency could be a
> > good move.  libInstPatch is itself a glib/gobject based library.  It has
> > some additional dependencies, but most of them are optional (the Python
> > binding for example).  The components that would be of the most interest
> > would be the instrument loading and synthesis cache objects.  The cache
> > allows for the "rendering" of instrument objects into a list of
> > potential voices.  When a MIDI note-on event occurs, these voices can be
> > selected in a lock-free fashion, the cache is generated at MIDI program
> > selection time.  It seems like FluidSynth should be able to take
> > advantage of this code, whether it be used in something like Swami or
> > standalone.
> > 
> I really think all the sound font loader stuff should go there, after
> having moved the synthesis related parts to the synth component.
> 

I could probably get a libInstPatch enabled FluidSynth running pretty
quickly, since a lot of the code is already written.

> > I think the next question is.  When should we branch?  It probably makes
> > the most sense to release 1.0.9 and then branch off 2.x.  At some point
> > 2.x would become the head and we would make a 1.x branch.
> > 
> These should be totally independent, don't think of them as related in any
> way. We can branch when it's needed, and 1.0.9 can be released whenever you
> want. If you can wait a few days I'll kick off the new branch with a
> proposal and I'll also throw a couple fixes in, they started playing with
> the code but have become a bit more serious.
> 
> Usually, in most projects, new development goes to the trunk and stable
> releases are branches. However, since people here is used to having a
> stable trunk we can start the experimental 2.x in a branch.
> 

Sounds good.

> > Some decisions should be made about what remains to put into 1.0.9.
> > 
> > What of the following should be added?
> > - PortAudio driver (it exists, does it just need to be improved?)
> > - Jack MIDI driver
> > - ASIO driver
> > 
> That's another discussion, we should think about two different and
> independent development branches. Personally, I'm not interested, but
> someone might want to do them for 1.x and we could be merge them later in
> the 2.x branch.
> 

I'll see if I feel inspired to work on any of those in the coming week.
If not, then its time for a 1.0.9 release.

Cheers!
        Josh