bug#43040: grep 3.4: memory leak

[Shlomi Fish]

Hi Luca,

On Tue, 25 Aug 2020 17:17:55 +0200
Luca Borzacchiello via Bug reports for GNU grep <bug-grep@gnu.org> wrote:

> Dear maintainer,
> grep 3.4 is very slow and uses a lot of memory when executed with the
> attached inputs:
> 
> time ./grep-3.4/build/bin/grep -f ./mem_leak ./la_divin.txt
> [...]
> real 0m0,442s
> user 0m0,117s
> sys 0m0,326s
> 
> mem usage: ~1.5GB
> 
> it seems to be a regression, since grep 3.3 runs smoothly on the same
> inputs:
> 
> time ./grep-3.3/build/bin/grep -f ./mem_leak ./la_divin.txt
> [...]
> real 0m0,016s
> user 0m0,016s
> sys 0m0,000s
>

I can reproduce this issue on my mageia v8 x86-64 system (core i3/sandy
bridge). grep-v3.3 built from source is fine.

> mem usage: ~300KB
> 
> valgrind detects some memory leaks on grep 3.4, this is its output:
> ==156624== Memcheck, a memory error detector
> ==156624== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
> ==156624== Using Valgrind-3.15.0 and LibVEX; rerun with -h for copyright
> info
> ==156624== Command: ./grep-3.4/build/bin/grep -f ./mem_leak ./la_divin.txt
> ==156624==
> ==156624==
> ==156624== HEAP SUMMARY:
> ==156624==     in use at exit: 3,762,378,749 bytes in 281,181 blocks
> ==156624==   total heap usage: 396,668 allocs, 115,487 frees, 3,789,426,889
> bytes allocated
> ==156624==
> ==156624== 8 bytes in 1 blocks are possibly lost in loss record 2 of 114
> ==156624==    at 0x483B7F3: malloc (in
> /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==156624==    by 0x1300D2: analyze (regcomp.c:1169)
> ==156624==    by 0x1300D2: re_compile_internal (regcomp.c:795)
> ==156624==    by 0x130DDB: rpl_re_compile_pattern (regcomp.c:230)
> ==156624==    by 0x10D3E6: regex_compile (dfasearch.c:160)
> ==156624==    by 0x10D7AC: GEAcompile (dfasearch.c:247)
> ==156624==    by 0x10C782: main (grep.c:2900)
> ==156624==
> ==156624== 8 bytes in 1 blocks are possibly lost in loss record 3 of 114
> ==156624==    at 0x483B7F3: malloc (in
> /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==156624==    by 0x1255BA: re_node_set_alloc (regex_internal.c:972)
> ==156624==    by 0x1255BA: calc_eclosure_iter (regcomp.c:1700)
> ==156624==    by 0x130316: calc_eclosure (regcomp.c:1677)
> ==156624==    by 0x130316: analyze (regcomp.c:1204)
> ==156624==    by 0x130316: re_compile_internal (regcomp.c:795)
> ==156624==    by 0x130DDB: rpl_re_compile_pattern (regcomp.c:230)
> ==156624==    by 0x10D3E6: regex_compile (dfasearch.c:160)
> ==156624==    by 0x10D7AC: GEAcompile (dfasearch.c:247)
> ==156624==    by 0x10C782: main (grep.c:2900)
> ==156624==
> ==156624== 8 bytes in 1 blocks are possibly lost in loss record 4 of 114
> ==156624==    at 0x483B7F3: malloc (in
> /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==156624==    by 0x126009: re_node_set_init_copy (regex_internal.c:1033)
> ==156624==    by 0x1262D1: create_cd_newstate (regex_internal.c:1681)
> ==156624==    by 0x1262D1: re_acquire_state_context (regex_internal.c:1553)
> ==156624==    by 0x13091B: create_initial_state (regcomp.c:1050)
> ==156624==    by 0x13091B: re_compile_internal (regcomp.c:806)
> ==156624==    by 0x130DDB: rpl_re_compile_pattern (regcomp.c:230)
> ==156624==    by 0x10D3E6: regex_compile (dfasearch.c:160)
> ==156624==    by 0x10D7AC: GEAcompile (dfasearch.c:247)
> ==156624==    by 0x10C782: main (grep.c:2900)
> ==156624==
> ==156624== 8 bytes in 1 blocks are possibly lost in loss record 5 of 114
> ==156624==    at 0x483B7F3: malloc (in
> /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==156624==    by 0x123AAA: re_node_set_alloc (regex_internal.c:972)
> ==156624==    by 0x123AAA: register_state (regex_internal.c:1574)
> ==156624==    by 0x126430: create_cd_newstate (regex_internal.c:1737)
> ==156624==    by 0x126430: re_acquire_state_context (regex_internal.c:1553)
> ==156624==    by 0x13091B: create_initial_state (regcomp.c:1050)
> ==156624==    by 0x13091B: re_compile_internal (regcomp.c:806)
> ==156624==    by 0x130DDB: rpl_re_compile_pattern (regcomp.c:230)
> ==156624==    by 0x10D3E6: regex_compile (dfasearch.c:160)
> ==156624==    by 0x10D7AC: GEAcompile (dfasearch.c:247)
> ==156624==    by 0x10C782: main (grep.c:2900)
> ==156624==
> ==156624== 16 bytes in 1 blocks are possibly lost in loss record 17 of 114
> ==156624==    at 0x483B7F3: malloc (in
> /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==156624==    by 0x12FD2D: init_dfa (regcomp.c:859)
> ==156624==    by 0x12FD2D: re_compile_internal (regcomp.c:758)
> ==156624==    by 0x130DDB: rpl_re_compile_pattern (regcomp.c:230)
> ==156624==    by 0x10D3E6: regex_compile (dfasearch.c:160)
> ==156624==    by 0x10D7AC: GEAcompile (dfasearch.c:247)
> ==156624==    by 0x10C782: main (grep.c:2900)
> ==156624==
> ==156624== 16 bytes in 1 blocks are possibly lost in loss record 18 of 114
> ==156624==    at 0x483B723: malloc (in
> /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==156624==    by 0x483E017: realloc (in
> /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==156624==    by 0x123B50: register_state (regex_internal.c:1589)
> ==156624==    by 0x126430: create_cd_newstate (regex_internal.c:1737)
> ==156624==    by 0x126430: re_acquire_state_context (regex_internal.c:1553)
> ==156624==    by 0x13091B: create_initial_state (regcomp.c:1050)
> ==156624==    by 0x13091B: re_compile_internal (regcomp.c:806)
> ==156624==    by 0x130DDB: rpl_re_compile_pattern (regcomp.c:230)
> ==156624==    by 0x10D3E6: regex_compile (dfasearch.c:160)
> ==156624==    by 0x10D7AC: GEAcompile (dfasearch.c:247)
> ==156624==    by 0x10C782: main (grep.c:2900)
> ==156624==
> ==156624== 24 bytes in 1 blocks are possibly lost in loss record 34 of 114
> ==156624==    at 0x483DD99: calloc (in
> /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==156624==    by 0x12FD55: init_dfa (regcomp.c:866)
> ==156624==    by 0x12FD55: re_compile_internal (regcomp.c:758)
> ==156624==    by 0x130DDB: rpl_re_compile_pattern (regcomp.c:230)
> ==156624==    by 0x10D3E6: regex_compile (dfasearch.c:160)
> ==156624==    by 0x10D7AC: GEAcompile (dfasearch.c:247)
> ==156624==    by 0x10C782: main (grep.c:2900)
> ==156624==
> ==156624== 24 bytes in 1 blocks are possibly lost in loss record 35 of 114
> ==156624==    at 0x483B7F3: malloc (in
> /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==156624==    by 0x1300F6: analyze (regcomp.c:1171)
> ==156624==    by 0x1300F6: re_compile_internal (regcomp.c:795)
> ==156624==    by 0x130DDB: rpl_re_compile_pattern (regcomp.c:230)
> ==156624==    by 0x10D3E6: regex_compile (dfasearch.c:160)
> ==156624==    by 0x10D7AC: GEAcompile (dfasearch.c:247)
> ==156624==    by 0x10C782: main (grep.c:2900)
> ==156624==
> ==156624== 24 bytes in 1 blocks are possibly lost in loss record 36 of 114
> ==156624==    at 0x483B7F3: malloc (in
> /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==156624==    by 0x130108: analyze (regcomp.c:1172)
> ==156624==    by 0x130108: re_compile_internal (regcomp.c:795)
> ==156624==    by 0x130DDB: rpl_re_compile_pattern (regcomp.c:230)
> ==156624==    by 0x10D3E6: regex_compile (dfasearch.c:160)
> ==156624==    by 0x10D7AC: GEAcompile (dfasearch.c:247)
> ==156624==    by 0x10C782: main (grep.c:2900)
> ==156624==
> ==156624== 112 bytes in 1 blocks are possibly lost in loss record 54 of 114
> ==156624==    at 0x483DD99: calloc (in
> /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==156624==    by 0x1262B6: create_cd_newstate (regex_internal.c:1678)
> ==156624==    by 0x1262B6: re_acquire_state_context (regex_internal.c:1553)
> ==156624==    by 0x13091B: create_initial_state (regcomp.c:1050)
> ==156624==    by 0x13091B: re_compile_internal (regcomp.c:806)
> ==156624==    by 0x130DDB: rpl_re_compile_pattern (regcomp.c:230)
> ==156624==    by 0x10D3E6: regex_compile (dfasearch.c:160)
> ==156624==    by 0x10D7AC: GEAcompile (dfasearch.c:247)
> ==156624==    by 0x10C782: main (grep.c:2900)
> ==156624==
> ==156624== 128 bytes in 1 blocks are possibly lost in loss record 64 of 114
> ==156624==    at 0x483B723: malloc (in
> /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==156624==    by 0x483E017: realloc (in
> /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==156624==    by 0x11FFB9: xrealloc (xmalloc.c:61)
> ==156624==    by 0x114A9B: xpalloc (dfa.c:818)
> ==156624==    by 0x114CD3: realloc_trans_if_necessary (dfa.c:2853)
> ==156624==    by 0x11988B: dfaexec_main (dfa.c:3391)
> ==156624==    by 0x10E082: EGexecute (dfasearch.c:416)
> ==156624==    by 0x10C7C5: main (grep.c:2905)
> ==156624==
> ==156624== 232 bytes in 1 blocks are possibly lost in loss record 68 of 114
> ==156624==    at 0x483B723: malloc (in
> /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==156624==    by 0x483E017: realloc (in
> /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==156624==    by 0x130B24: re_compile_internal (regcomp.c:750)
> ==156624==    by 0x130DDB: rpl_re_compile_pattern (regcomp.c:230)
> ==156624==    by 0x10D3E6: regex_compile (dfasearch.c:160)
> ==156624==    by 0x10D7AC: GEAcompile (dfasearch.c:247)
> ==156624==    by 0x10C782: main (grep.c:2900)
> ==156624==
> ==156624== 25,222 bytes in 1 blocks are possibly lost in loss record 78 of
> 114
> ==156624==    at 0x483DFAF: realloc (in
> /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==156624==    by 0x11FFB9: xrealloc (xmalloc.c:61)
> ==156624==    by 0x10C024: main (grep.c:2598)
> ==156624==
> ==156624== 10,379,152 (5,013,056 direct, 5,366,096 indirect) bytes in
> 21,608 blocks are definitely lost in loss record 113 of 114
> ==156624==    at 0x483B723: malloc (in
> /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==156624==    by 0x483E017: realloc (in
> /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==156624==    by 0x130B24: re_compile_internal (regcomp.c:750)
> ==156624==    by 0x130DDB: rpl_re_compile_pattern (regcomp.c:230)
> ==156624==    by 0x10D3E6: regex_compile (dfasearch.c:160)
> ==156624==    by 0x10D7AC: GEAcompile (dfasearch.c:247)
> ==156624==    by 0x10C782: main (grep.c:2900)
> ==156624==
> ==156624== LEAK SUMMARY:
> ==156624==    definitely lost: 5,013,056 bytes in 21,608 blocks
> ==156624==    indirectly lost: 5,366,096 bytes in 216,158 blocks
> ==156624==      possibly lost: 25,830 bytes in 13 blocks
> ==156624==    still reachable: 3,751,973,767 bytes in 43,402 blocks
> ==156624==                       of which reachable via heuristic:
> ==156624==                         newarray           : 112 bytes in 1
> blocks
> ==156624==         suppressed: 0 bytes in 0 blocks
> ==156624== Reachable blocks (those to which a pointer was found) are not
> shown.
> ==156624== To see them, rerun with: --leak-check=full --show-leak-kinds=all
> ==156624==
> ==156624== For lists of detected and suppressed errors, rerun with: -s
> ==156624== ERROR SUMMARY: 14 errors from 14 contexts (suppressed: 0 from 0)
> 
> ---
> Regards,
> Luca Borzacchiello



-- 

Shlomi Fish       https://www.shlomifish.org/
Apple Inc. is Evil - https://www.shlomifish.org/open-source/anti/apple/

If the mountain does not come to Muhammad, then Chuck Norris will bring the
mountain over.
    — https://www.shlomifish.org/humour/bits/facts/Chuck-Norris/

Please reply to list if it's a mailing list post - https://shlom.in/reply .