Re: [Monotone-devel] Re: mtn cvs_import's "almost certainly a bug"

[Eric Anderson]

Jonathan S. Shapiro writes:
 > On Fri, 2006-08-04 at 13:01 -0700, Eric Anderson wrote:
 > 
 > > You're probably running into the problem of monotone keeping the
 > > entire commit in memory.  eddb7e59361efeb8d9300ba0ddd7483272565097
 > > from net.venge.monotone.experiment.performance fixes this.
 > 
 > Damn but this all sounds very familiar... OpenCM used to do this too.
 > Once of the *really* nice things about using sha1 is that partial
 > results can be stored as long as you don't store anything that depends
 > on them.

I suspect that something different is going on here; there was an
optimization introduced a while ago that batched together updates
because batched updates are more efficient that individual ones.  To
limit memory usage, the netsync portion of the code did periodic
database commits.  For the commit operation (and probably cvs_import),
that wasn't put in place.  All I did was make monotone push out the
data to the database (which writes it to the log as an uncommitted
transaction), thereby limiting memory usage.
        -Eric