[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Cvs-dev] [PATCH, one-liner] memory leak bug (buffer.c:buf_free_data
From: |
Sergei Trofimovich |
Subject: |
Re: [Cvs-dev] [PATCH, one-liner] memory leak bug (buffer.c:buf_free_data) in all CVS versions |
Date: |
Mon, 14 Jan 2013 20:33:11 +0300 |
On Mon, 14 Jan 2013 18:09:13 +1100
"Arthur Barrett" <address@hidden> wrote:
> Sergei,
>
> Thanks for the report.
>
> But there are some problems.
>
> 1. you didn't attach a patch.
My bad. Reattached.
Backup link if I have broken something again:
http://sources.gentoo.org/cgi-bin/viewvc.cgi/gentoo-x86/dev-vcs/cvs/files/cvs-1.12.12-fix-massive-leak.patch?revision=1.1
> 2. CVS 1.11.23 buffer.c does not have buf_free_data
Seems to be introduced around 2004:
> Author: dprice <dprice>
> Date: Thu Sep 2 16:24:56 2004 +0000
>
> * server.c (do_cvs_command): Pass new args to fd_buffer_initialize().
> ...
> 3. CVS 1.12 buffer.c does not have buf_free_data
The version I discovered was a cvs-1.12.12 (newer ones
up to CVS cvs tree still contain the bug as far as I see.
Looking at rsync://cvs.savannah.gnu.org/sources/cvs/)
> 4. CVS 2.x/CVSNT doesn't have buf_free_data
It's not about this software, right?
http://ftp.gnu.org/non-gnu/cvs/source/
> 5. cvsps is nothing to do with CVS (though the functionality is already
> in CVSNT for 5 years or more)
cvsps is a tool to export cvs repo as atomic changesets eatable
by 'git fast-import' for example.
It connects to CVS server (or runs it locally for :local: CVSROOT)
and runs roughly the following commands:
<connect to host>
cvs rlog proj # parse revisions
cvs co -r 1.1 file
cvs co -r 1.2 file
<disconnect from host>
> If you can submit the missing information (patch to CVS 1.12.13 or CVS
> 1.11.23) that may help.
>
> Regards,
>
>
> Arthur Barrett
> Product Manager, CVSNT
>
>
> > -----Original Message-----
> > From:
> > address@hidden
> > [mailto:address@hidden
> > rg] On Behalf Of Sergei Trofimovich
> > Sent: 12 January 2013 08:47
> > To: address@hidden
> > Subject: [Cvs-dev] [PATCH, one-liner] memory leak bug
> > (buffer.c:buf_free_data) in all CVS versions
> >
> >
> > Hello ladies and gentlemen!
> >
> > In my recent exploration on tools
> > reading entire cvs history I've
> > noticed horrible leak.
> >
> > It allows to suck out all RAM out
> > of your cvs server just by issuing
> > any (say, cvs checkout) simple command
> > many times.
> >
> > A description of an attached patch with
> > some numbers:
> >
> > buf_free_data must free data independently
> > of send or reseived bytes over network.
> >
> > Moreover, when buffer is usually freed
> > buffer _is_ empty, but has one clean mapped page.
> >
> > I've observed massive 'cvs server' leaks
> > when importing large gentoo-x86 repo with 'cvsps'.
> > Leak ate all my 32GBs of RAM and killed process.
> > (Leaked around 3 pages per client request).
> >
> > valgrind found the leak easily:
> >
> > $ valgrind \
> > cvsps \
> > --root :local:$HOME/portage/gentoo-x86.rsync \
> > --fast-export \
> > gentoo-x86/dev-vcs/git-annex 2>l |
> > git fast-import
> >
> > ==13504== 1,248 bytes in 52 blocks are still
> > reachable in loss record 41 of 47
> > ==13504== at 0x4C2C19B: malloc (vg_replace_malloc.c:270)
> > ==13504== by 0x48A556: xnmalloc_inline (xmalloc.c:40)
> > ==13504== by 0x48A5B5: xmalloc (xmalloc.c:56)
> > ==13504== by 0x4855F5: new_memnode (pagealign_alloc.c:91)
> > ==13504== by 0x48571B: pagealign_alloc
> > (pagealign_alloc.c:151)
> > ==13504== by 0x485739: pagealign_xalloc
> > (pagealign_alloc.c:182)
> > ==13504== by 0x408DD7: get_buffer_data (buffer.c:98)
> > ==13504== by 0x409C0C: buf_input_data (buffer.c:738)
> > ==13504== by 0x45BB63: do_cvs_command (server.c:3847)
> > ==13504== by 0x45D39E: serve_co (server.c:4809)
> > ==13504== by 0x45F845: server (server.c:6438)
> > ==13504== by 0x438784: main (main.c:1066)
> >
> > And now it takes constant space (less, than 18MB)
> > for 'cvs server' process to convert all gentoo-x86
> > by serving more, than 5 000 000 client requests.
> >
> > Signed-off-by: Sergei Trofimovich <address@hidden>
> >
> > Thanks!
> >
> > --
> >
> > Sergei
> >
>
> _______________________________________________
> Cvs-dev mailing list
> address@hidden
> https://lists.nongnu.org/mailman/listinfo/cvs-dev
>
--
Sergei
cvs-1.12.12-fix-massive-leak.patch
Description: Text Data
signature.asc
Description: PGP signature