On Fri, Mar 5, 2021 at 1:16 PM Pip Cet <pipcet@gmail.com> wrote:
I say let's go with stdio.
Maybe setbuffer(3) could help us here? I could run some benchmarks for
that if the idea isn't out of the question.
Actually, calling setbuffer() with a large buffer will make glibc
reread the entire file on every fseek(), rendering the dump so slow I
gave up and interrupted it.
However, there's open_memstream: that would have the added advantage
of actually making glibc write out the entire file in one go, so it
seems to shave a few extra milliseconds off the build.
(However however, glibc's memstreams are somewhat broken. I'll file a
bug report if there isn't one already...)
Still, that way we could use stdio today, leave the buffering to
glibc, and hopefully be able to switch trivially to a "open this file
but keep it in memory" combined memstream/FILE* one day.