[Traverso-devel] JIT Audio File Conversion

[ben levitt]

Hi devs,

I've been thinking about how to handle working with audio files other
than those that can be de/encoded by libsndfile, and have the same
samplerate as the audio driver.

I think that we should allow working natively with other formats,
decoding/resampling them as needed, Just In Time.

To do this, I'm thinking that ReadSource and MonoReader should not use
libsndfile directly.  (Same with WriteSource?)  And should instead
keep one of a new set of AudioFile classes that would handle reading
in and decoding/resampling the audio. Something like:

+ AudioFile
 - AudioFileLibSndFile
 - AudioFileMp3
 - AudioFileOgg
 - AudioFileAac
 - AudioFileFlac
 - AudioFileResample

All AudioFile classes would know how to:
|- open some kinds of audio files, for either read or write
|- set write parameters (specific to each AudioFile class)
|- read N frames of audio from it
|- write N frames to it
|- seek to frame X
|- close

AudioFile would additionally have a static method that could take a
filename, and return a new instance of the correct AudioFileXX class.
(Purely by extension?  Looking at the file's header?)

AudioFileResample would exist in order to change the sample rate of an
audio file to match the audio driver's sample rate.  It could take
another AudioFile class instead of a file, so any audio file could be
resampled after being decoded.  It could have super-efficient
passthrough when the sample rates are the same, so other AudioFiles
could always get wrapped by one of these...

For the formats like mp3 where position in the encoded file does not
necessarily map linearly to position in the decoded file, it looks
like we can fairly quickly build a mapping, when we import the file,
of uncompressed audio position to each encoded "frame" (chunk of
audio)'s position, so that we could quickly seek to an exact
uncompressed frame.

Being able to work directly on compressed files, instead of having to
decompress them first will be great!  And of course, I'd want to set
up an option to bypass all of this and just convert on import to a wav
file of the samplerate of the audio driver for people who notice a
slowdown by working with compressed audio natively, like anyone
working on a large, complex project.

Thoughts?

Ben