recent memory cleanup changes

[John W. Eaton]

Over the last few days, I checked in a series of changes to clean up
some memory issues.  There were two types of problems I was trying to
address.

  1. Real leaks, objects for which memory was allocated but never
     released and the pointers to the memory were no longer
     available.  An example some changes that fix this type of problem
     is http://hg.savannah.gnu.org/hgweb/octave/rev/c93b953f7d54

  2. Memory allocated for buffers that are handled by static pointers
     but that is not reclaimed before Octave exits.  I don't think of
     these as leaks because we still have pointers to the data and the
     buffers are needed for the duration of the program so not freeing
     them doesn't really cause trouble.  Even if Octave is embedded in
     some other program and Octave is started (by calling octave_main)
     and stopped (by calling clean_up_and_exit) repeatedly, there is
     no leak for this type of data because the static pointers will be
     initialized on the first call and will not need to be
     reinitialized on subsequent calls.  However, the buffers will
     remain allocated when the Octave interpreter is not active, so I
     think it is reasonable to try to have clean_up_and_exit release
     these buffers as well.  Most of my recent changes attempt to
     address this kind of memory issue.

With these changes, valgrind is reporting much less memory allocated
when Octave exits.  For example, simply starting Octave and issuing
a "quit" command with 3.4.3 resulted in

  HEAP SUMMARY:
      in use at exit: 36,854,349 bytes in 10,678 blocks
    total heap usage: 460,252 allocs, 449,574 frees, 73,495,857 bytes allocated

  LEAK SUMMARY:
     definitely lost: 4,764 bytes in 115 blocks
     indirectly lost: 32,742 bytes in 201 blocks
       possibly lost: 1,231,898 bytes in 3,676 blocks
     still reachable: 35,584,945 bytes in 6,686 blocks
          suppressed: 0 bytes in 0 blocks
  Rerun with --leak-check=full to see details of leaked memory

  For counts of detected and suppressed errors, rerun with: -v
  ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 13 from 7)

The majority of the "still reachable" memory was the 32MB static
buffer allocated for the octave_local_buffer class.  The majority of
the "possibly lost" memory was static pointers to allocated buffers.

Now I see

  HEAP SUMMARY:
      in use at exit: 150,387 bytes in 3,272 blocks
    total heap usage: 523,804 allocs, 520,532 frees, 75,586,634 bytes allocated

  LEAK SUMMARY:
     definitely lost: 60 bytes in 1 blocks
     indirectly lost: 240 bytes in 10 blocks
       possibly lost: 129 bytes in 1 blocks
     still reachable: 149,958 bytes in 3,260 blocks
          suppressed: 0 bytes in 0 blocks
  Rerun with --leak-check=full to see details of leaked memory

  For counts of detected and suppressed errors, rerun with: -v
  ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 13 from 7)

This is better, but not perfect.  Running the test suite with valgrind
shows the following:

  HEAP SUMMARY:
      in use at exit: 172,102 bytes in 1,739 blocks
    total heap usage: 173,052,686 allocs, 173,050,947 frees, 8,690,690,208 
bytes allocated

  LEAK SUMMARY:
     definitely lost: 2,483 bytes in 43 blocks
     indirectly lost: 36,371 bytes in 115 blocks
       possibly lost: 17,143 bytes in 317 blocks
     still reachable: 116,105 bytes in 1,264 blocks
          suppressed: 0 bytes in 0 blocks

  For counts of detected and suppressed errors, rerun with: -v
  Use --track-origins=yes to see where uninitialised values come from
  ERROR SUMMARY: 3672 errors from 112 contexts (suppressed: 265 from 7)

which is a bit worse, so there is still some work to be done.  I'm not
sure why "still reachable" is actually less in this case.

I'm attaching a copy of the full valgrind report for running the test
suite with

  valgrind --tool=memcheck --leak-check=full --show-reachable=yes

Some of the things that valgrind reports are apparently not real
problems, but are due to bugs in valgrind.  See for example the bug
report I filed with libc here:

  http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=650714

I know some people are seeing problems after these changes and I'd
like to fix those problems, but first I need to understand why they
are happening.  I'm not seeing any crashes related to these changes
and valgrind is not reporting any memory corruption problems to would
obviously lead to crashes.  So if you are seeing segfaults, I need to
have some detailed information about where the crashes are happening
and precisely how you compiled Octave, on what type of system, etc.
Also, have you rebuilt from scratch, including running autogen.sh and
configure?

jwe

memcheck-log.gz
Description: Binary data