Re: [Monotone-devel] Problem with monotone 0.29

[Nathaniel Smith]

On Thu, Aug 24, 2006 at 09:13:12PM +0200, Detlef Vollmann wrote:
> Nathaniel Smith wrote:
> > On Thu, Aug 24, 2006 at 11:16:09AM +0200, Detlef Vollmann wrote:
> > > So I did:
> > > $ mtn --db=/source/monotone/snapshot/oe-060823.mtn checkout 
> > > --branch=org.openembedded.dev 
> > > --revision=c9f0e213d8e0fdc01e39c1d5ebd4f3e5de4db6b1
> > >
> > > and this is the command that is still running (after checking out
> > > about 140MB) ...
> > 
> > Odd; I just ran exactly this command with the static linux/x86 0.29
> > binary we distribute, and the checkout completed fine in not much time
> > at all.  ~20 seconds CPU time, somewhat more than that real.
> I didn't find a real static binary, but only one that requires
> glibc 2.3, while my machine only has 2.2.3...

Right... it isn't possible to create real static binaries with glibc
:-(.

> > What system are you on?
> I think that's the interesting point:
> My local copy of the database sits on a server with a really old
> installation (originally redhat 6.3 only only partly updated).
> The working copy sits on my workstation (not really up-to-date,
> but at least glibc 2.3.2 and gcc 3.2.2).
> The workstation accesses the server's files using NFS, but I
> had problems using monotone with a database over NFS.
> So I used the workstation to build a fully static binary for
> the server.

This seems suspicious to me -- binaries compiled with glibc 2.3, even
with the -static switch, still require glibc 2.3 be available to
fully work.  It's _possible_ this isn't your problem -- the weird
not-quite-static issues only show up in certain cases.  But it's
something to watch out for.

(You might also try
  http://venge.net/monotone/downloads/mtn-0.29-linux-x86.bz2
anyway... I'm not sure the binary currently there _is_ compiled with
glibc 2.3, that might be a lie as well :-).  It definitely is
statically linked to libstdc++, though, so that shouldn't be an
issue.)

> On the server I mount the target directory
> of the workstation over NFS and do the checkout.
> Possibly that's the problem.

That's also possible.

> I just ran a new pull and then the checkout again, and monotone
> went into the endless loop after only writing about 8MB to
> the target directory.
> So I started the checkout again, and this time it succeeded!
> So it's not a deterministic failure.

Oh, goodie.

> > Is it possible your build is broken somehow?
> Actually, I don't think so, but you never know...
> 
> > Can you determine where monotone is freezing up?  Is it using CPU?
> As much as it can get (nearly 100% in top).
> 
> >  If
> > you attach to it with gdb, can you get a backtrace (by doing:
> >   $ gdb `which mtn` `pidof mtn`
> >   (gdb) backtrace
> > )?
> As I stripped the executable, the output is not very interesting:
> Attaching to program: /usr/local/bin/mtn, process 2612
> 0x08468605 in ?? ()
> (gdb) bt
> #0  0x08468605 in ?? ()
> #1  0x08467889 in ?? ()
> #2  0x0830d9a8 in ?? ()
> #3  0x0830da7e in ?? ()
> #4  0x08383fb3 in ?? ()
> #5  0x08384cfc in ?? ()
> #6  0x0830f02f in ?? ()
> #7  0x0814402b in ?? ()
> #8  0x08145870 in ?? ()
> #9  0x0814a7e2 in ?? ()
> #10 0x080f3387 in ?? ()
> #11 0x080a612e in ?? ()
> #12 0x0829f6af in ?? ()
> #13 0x082a3b7c in ?? ()
> #14 0x0843fbc9 in ?? ()

Uh... yeah.  A non-stripped build would be helpful; I'm not sure how
else we're going to figure out what's going on here :-).

> >  Does strace say anything interesting?
> As the output would be huge (it first writes lots of files/directories
> before running off), I didn't try this.

You can use strace's -P option to attach it to an already running
process.  I.e., you can wait for it to hang, and only then start
strace, with something like 'strace -P <pid of hung mtn>'.  Might be
useful, might not, depending on what it's actually doing in there...

-- Nathaniel

-- 
Sentience can be such a burden.