[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Monotone-devel] nvm.sqlite3.binary adapted to rosters (and finished
From: |
Conrad Steenberg |
Subject: |
Re: [Monotone-devel] nvm.sqlite3.binary adapted to rosters (and finished) |
Date: |
Fri, 13 Jan 2006 14:11:07 -0800 |
Hi Christof
Just out of curiosity, do you notice speed improvements as well as the
obvious size reduction?
Thanks!
Conrad
On Fri, 2006-01-13 at 19:37 +0100, Christof Petig wrote:
> Hi,
>
> I finished the first part (base64->BLOB) of migrating the database
> layout to use BLOBs and am very pleased with the results:
>
> ## ------------- ##
> ## Test results. ##
> ## ------------- ##
>
> All 298 tests behaved as expected.
> PASS: ./testsuite
> ==================
> All 2 tests passed
> ==================
>
> $ cat MT/revision
> c60d6232e87c2f388eca77b32f77e328c967c139
>
> ... 61265920 monotone_BLOB.db
> ... 87126016 monotone026.db
> ... 167739392 monotone025.db
>
> So rosters shrunk the database by 50% and BLOBs shrink by another 30% !
>
> IMHO BLOBs are as readable as base64 encoded text (containing several
> newlines) and gzip compressed values end at the first NUL byte (so they
> don't display).
>
> I checked the diff to mainline again and did not find any strange
> arrangements (unlike the time I announced cvssync to be ready for
> review), so perhaps we might consider switching to BLOBs and rosters at
> once in 0.26final.
>
> I wrote this patch to be minimally invasive so removing base64 coding
> from internal data structures (e.g. cert) could be a nice project for
> the future.
>
> _Perhaps_ using BLOBs for IDs (hex->BLOB) might give a win, too. But
> that's a different story because this clearly hurts command line
> interaction with the database.
>
> Christof
>
> Caveats:
>
> sqlite> select cast('ABC' as BLOB)='ABC';
> 0
>
> so we have to remember which columns are stored as BLOB and pass
> appropriate data types to the query (that's what queryarg is for). And
> we also have to cast revision_certs.value to TEXT before comparing it to
> a string.
> Proper use of column affinities (actually specifying a column type)
> should help a bit.
> _______________________________________________
> Monotone-devel mailing list
> address@hidden
> http://lists.nongnu.org/mailman/listinfo/monotone-devel
--
Conrad Steenberg <address@hidden>
California Institute of Technology
smime.p7s
Description: S/MIME cryptographic signature