[Monotone-devel] Re: nvm.sqlite3.binary adapted to rosters (and finished)

[Bruce Stephens]

Christof Petig <address@hidden> writes:

[...]

> $ cat MT/revision
> c60d6232e87c2f388eca77b32f77e328c967c139
>
> ...  61265920 monotone_BLOB.db
> ...  87126016 monotone026.db
> ... 167739392 monotone025.db
>
> So rosters shrunk the database by 50% and BLOBs shrink by another 30% !

Wow!

> IMHO BLOBs are as readable as base64 encoded text (containing several
> newlines) and gzip compressed values end at the first NUL byte (so they
> don't display).
>
> I checked the diff to mainline again and did not find any strange
> arrangements (unlike the time I announced cvssync to be ready for
> review), so perhaps we might consider switching to BLOBs and rosters at
> once in 0.26final.

That would be good; it's one of the things that's obviously wrong in
the current implementation.

> I wrote this patch to be minimally invasive so removing base64 coding
> from internal data structures (e.g. cert) could be a nice project for
> the future.

I think the idea is to allow arbitrary names and values (even though
nothing really does that yet, that I can see), so I'd guess there'd be
some annoying parsing problems.  Maybe not, though.  

Maybe certs would be basic_io if Graydon and Nathaniel started from
scratch, which would save the base64 encoding, at the expense of
quoting some characters.

> _Perhaps_ using BLOBs for IDs (hex->BLOB) might give a win, too. But
> that's a different story because this clearly hurts command line
> interaction with the database.

That feels fairly minor to me.  Might be worth trying some part of
that and doing timings (and maybe guesstimating size difference), but
I'd guess it's not worth it.

[...]