[Monotone-devel] excessive memory usage during commit

[jack-monotone]

Hello all, 

I've noticed mtn uses a lot of memory while committing changes to a ~11MiB
binary file. I ran the commit with:

    valgrind --tool=massif mtn commit -meh

I've attached the massif log file. 

Here's the salient bit at peak heap usage:

| | ->97.83% (1,187,110,745B) 0x60A1EBF: std::string::reserve(unsigned long) 
(basic_string.tcc:485)
| | | ->28.57% (346,709,952B) 0x5855B3: 
piecewise_applicator::finish(std::string&) (xdelta.cc:541)
| | | | ->28.57% (346,709,952B) 0x558605: database_impl::get_version(id const&, 
data&, std::string const&, std::string const&) (database.cc:2002)
| | | |   ->28.57% (346,709,952B) 0x5590F1: database::get_file_version(file<id> 
const&, file<data>&) (database.cc:2305)
| | | |     ->14.29% (173,354,976B) 0x4DA82F: 
commands::cmd_commit::exec(app_state&, std::vector<utf8, std::allocator<utf8> > 
const&, std::vector<arg_type, std::allocator<arg_type> > const&) const 
(cmd_ws_commit.cc:1276)
| | | |     | ->14.29% (173,354,976B) 0x46E3E3: commands::process(app_state&, 
std::vector<utf8, std::allocator<utf8> > const&, std::vector<arg_type, 
std::allocator<arg_type> > const&) (cmd.cc:118)
| | | |     |   ->14.29% (173,354,976B) 0x6D6AD0: cpp_main(int, char**) 
(monotone.cc:285)
| | | |     |     ->14.29% (173,354,976B) 0x6D7385: main (main.cc:148)
| | | |     |
| | | |     ->14.29% (173,354,976B) 0x559D22: 
database::put_file_version(file<id> const&, file<id> const&, file<delta> 
const&) (database.cc:2360)
| | | |       ->14.29% (173,354,976B) 0x4DAAC1: 
commands::cmd_commit::exec(app_state&, std::vector<utf8, std::allocator<utf8> > 
const&, std::vector<arg_type, std::allocator<arg_type> > const&) const 
(cmd_ws_commit.cc:1288)
| | | |         ->14.29% (173,354,976B) 0x46E3E3: commands::process(app_state&, 
std::vector<utf8, std::allocator<utf8> > const&, std::vector<arg_type, 
std::allocator<arg_type> > const&) (cmd.cc:118)
| | | |           ->14.29% (173,354,976B) 0x6D6AD0: cpp_main(int, char**) 
(monotone.cc:285)
| | | |             ->14.29% (173,354,976B) 0x6D7385: main (main.cc:148)
| | | |
| | | ->27.31% (331,349,984B) 0x60A2272: std::string::append(std::string 
const&, unsigned long, unsigned long) (basic_string.tcc:348)
| | | | ->27.31% (331,349,984B) 0x5834A9: 
apply_delta(boost::shared_ptr<delta_applicator>, std::string const&) 
(xdelta.cc:433)
| | | |   ->27.31% (331,349,984B) 0x583ABF: apply_delta(std::string const&, 
std::string const&, std::string&) (xdelta.cc:449)
| | | |     ->27.31% (331,349,984B) 0x583C50: patch(data const&, delta const&, 
data&) (xdelta.cc:473)
| | | |       ->27.31% (331,349,984B) 0x559D67: 
database::put_file_version(file<id> const&, file<id> const&, file<delta> 
const&) (database.cc:2363)
| | | |       | ->27.31% (331,349,984B) 0x4DAAC1: 
commands::cmd_commit::exec(app_state&, std::vector<utf8, std::allocator<utf8> > 
const&, std::vector<arg_type, std::allocator<arg_type> > const&) const 
(cmd_ws_commit.cc:1288)
| | | |       |   ->27.31% (331,349,984B) 0x46E3E3: 
commands::process(app_state&, std::vector<utf8, std::allocator<utf8> > const&, 
std::vector<arg_type, std::allocator<arg_type> > const&) (cmd.cc:118)
| | | |       |     ->27.31% (331,349,984B) 0x6D6AD0: cpp_main(int, char**) 
(monotone.cc:285)
| | | |       |       ->27.31% (331,349,984B) 0x6D7385: main (main.cc:148)
| | | |       |
| | | |       ->00.00% (0B) in 1+ places, all below ms_print's threshold 
(01.00%)


--
Jack (John) Cummings                           http://mudshark.org/
PGP fingerprint: F18B 13A3 6D06 D48A 598D  42EA 3D53 BDC8 7917 F802

massif.out.21169.lzma
Description: Binary data

pgpBAFlyiAeea.pgp
Description: PGP signature