Re: HOWTO: benchmarking monotone (was Re: [Monotone-devel] "memory exhausted" error for 'mtn list status' command)

[Eric Anderson]

Nathaniel Smith writes:
 > Weirdly, the memtime data doesn't really reflect this.  For "before"
 > I get:
 > 
 >   ls_unknown-avg-resident-MiB,4.35963439941,4.33722305298,4.40716743469
 >   ls_unknown-avg-size-MiB,21.4379463196,21.345328331,21.6831884384
 >   ls_unknown-max-resident-MiB,4.703125,4.69921875,4.70703125
 >   ls_unknown-max-size-MiB,22.34375,22.34765625,22.34765625
 >   ls_unknown-num-samples,78,65,75
 >   ls_unknown-system-time,0.051,0.037,0.044
 >   ls_unknown-user-time,0.478,0.470,0.477
 >   ls_unknown-wall-time,0.549,0.519,0.529
 > 
 > And for "after":
 > 
 >   ls_unknown-avg-resident-MiB,3.36142158508,3.18865299225,3.06260299683
 >   ls_unknown-avg-size-MiB,19.809679985,19.9870185852,16.5047159195
 >   ls_unknown-max-resident-MiB,4.69140625,4.69140625,4.69140625
 >   ls_unknown-max-size-MiB,21.83203125,21.83203125,21.83203125
 >   ls_unknown-num-samples,299,208,93
 >   ls_unknown-system-time,0.047,0.048,0.036
 >   ls_unknown-user-time,0.458,0.477,0.461
 >   ls_unknown-wall-time,2.092,1.467,0.743
 > 
 > If anything, this would suggest that memory usage had _increased_?
 > That doesn't make much sense to me, and massif seems like the more
 > trustworthy party here, with its fine-grained deterministic approach.
 > The numbers also seem weirdly large -- maybe we're measuing how much
 > of the binary has gotten swapped into memory, for instance?  Eric, any
 > thoughts?

This looks perfectly correct assuming that "after" is the run that
includes your fix.  avg resident size has decreased from 4.35 to 3.1
MiB, avg size has decreased from 21.4 MiB to 19.8 MiB. Similar on the
maximum values.  The only odd thing is that all three "after" runs
took way more wall clock time, but similar system and user time.
        -Eric