Re: [Monotone-devel] oprofile data for mtn 0.37.

[Jack Lloyd]

On Thu, Mar 13, 2008 at 12:07:57PM -0400, Zack Weinberg wrote:
> On Thu, Mar 13, 2008 at 11:08 AM, Jack Lloyd <address@hidden> wrote:
> >  Pipe actually keeps each message around forever (all storage is
> >  reclaimed once a) all output has been read out b) the message in
> >  question is not currently being processed). You can read out a out of
> >  order / interleaved with reading others / whatever.
> 
> Oh, I see now.
> 
> Is *all* storage reclaimed, or is there some array indexed by message
> number that's going to keep growing forever if I recycle pipes?

It should be everything (ie, zero increased memory usage over time, as
long as you read the entire contents of each message out) - the code
handling this is in out_buf.cpp if you would like to review.

Message numbers are currently stored as 32-bit integers, and there
will probably be failures on overflow (in particular at 2^32-2 ==
Pipe::LAST_MESSAGE) -- how likely would a process-persistent Pipe
eventually hit this limit in (say) long-running a monontone server?

You have: 2^32 / (300 / s)
You want: day
        * 165.7009

The message number can (and probably should) be changed to a long
long, though also the underlying issue that Pipe should detect/signal
this condition needs fixing.

WRT leaks, attached is a simple testcase, which I have only run up to
about a million messages (slow machine), but there was no perceptible
increase in memory usage as the # of messages run through the Pipe
increases. I will attempt running this out to 2^32 messages to make
sure though (and confirm/deny if there are failures at overflow).

-Jack

pipe_test.cpp
Description: Text document