Re: [Findutils-patches] [PATCH] find memory leak

[Goffredo Baroncelli]

Hi Bernard,

(please put me in CC in case of reply because I am not subscribed to the 
mailing list)



On 2017-02-02 00:34, Bernhard Voelker wrote:
> On 12/15/2016 10:19 PM, Goffredo Baroncelli wrote:
>> Hi All,
[...]
>>
>> # Without the patch, this command requires up to 602MB of memory !!!
>> $ /usr/bin/time ./find /lib/modules/ -ignore_readdir_race      \( -fstype 
>> NFS -o -fstype nfs -o -fstype nfs4 -o -fstype afs -o -fstype binfmt_misc -o 
>> -fstype proc -o -fstype smbfs -o -fstype autofs -o -fstype iso9660 -o 
>> -fstype ncpfs -o -fstype coda -o -fstype devpts -o -fstype ftpfs -o -fstype 
>> devfs -o -fstype mfs -o -fstype shfs -o -fstype sysfs -o -fstype cifs -o 
>> -fstype lustre_lite -o -fstype tmpfs -o -fstype usbfs -o -fstype udf -o 
>> -fstype ocfs2 -o      -type d -false  \) -prune -o -print0  | wc -l
>> 31.59user 27.17system 0:58.82elapsed 99%CPU (0avgtext+0avgdata 
>> 602476maxresident)k
>> 0inputs+0outputs (0major+150129minor)pagefaults 0swaps
>>
>> # With the patch, the same command as above requires only 2MB of memory !!!
>> $ /usr/bin/time ./find /lib/modules/ -ignore_readdir_race      \( -fstype 
>> NFS -o -fstype nfs -o -fstype nfs4 -o -fstype afs -o -fstype binfmt_misc -o 
>> -fstype proc -o -fstype smbfs -o -fstype autofs -o -fstype iso9660 -o 
>> -fstype ncpfs -o -fstype coda -o -fstype devpts -o -fstype ftpfs -o -fstype 
>> devfs -o -fstype mfs -o -fstype shfs -o -fstype sysfs -o -fstype cifs -o 
>> -fstype lustre_lite -o -fstype tmpfs -o -fstype usbfs -o -fstype udf -o 
>> -fstype ocfs2 -o      -type d -false  \) -prune -o -print0  | wc -l
>> 30.94user 26.14system 0:57.14elapsed 99%CPU (0avgtext+0avgdata 
>> 2764maxresident)k
>> 0inputs+0outputs (0major+165minor)pagefaults 0swaps
>>
>>
>> # for curiosity my /lib/modules contains
>> $ find /lib/modules/ | wc -l
>> 68101
>>
>> Finally, what about caching the value of read_file_system_list() instead of 
>> recomputing it every time ? On the basis of my tests I found an impressive 
>> gain in terms of cpu usage. But what are the cons ?
>>
>>
>> BR
>> G.Baroncelli
> 
> I cannot reproduce here - i.e. I do not see such a huge memory leak here ... 
> even
> if I add some 8 bind-mounts so that /proc/self/mountinfo has 30k.
> How does your test system look like? Is there anything special about your 
> directory
> structure?  

I don't think that my directory structure is special in any way. 

$ ls /lib/mnodules
3.16.0-4-amd64  4.3.0-trunk-amd64  4.6.0-1-amd64  4.8.15  4.9.5
3.16-3-amd64    4.4.0-rc8-amd64    4.8.0-2-amd64  4.9.3


In your testa, have you used all the "-fstype ...." arguments ? IIRC the leak 
size depends by the number of "-fstype ...."



> What does 'valgrind --leak-check=full --show-leak-kinds=all ./find ...'
> tell you about the leak(s)? How many lost blocks do you have?

Below the output:



$ valgrind --leak-check=full --show-leak-kinds=all  /usr/bin/find /lib/modules/ 
-ignore_readdir_race      \( -fstype NFS -o -fstype nfs -o -fstype nfs4 -o 
-fstype afs -o -fstype binfmt_misc -o -fstype proc -o -fstype smbfs -o -fstype 
autofs -o -fstype iso9660 -o -fstype ncpfs -o -fstype coda -o -fstype devpts -o 
-fstype ftpfs -o -fstype devfs -o -fstype mfs -o -fstype shfs -o -fstype sysfs 
-o -fstype cifs -o -fstype lustre_lite -o -fstype tmpfs -o -fstype usbfs -o 
-fstype udf -o -fstype ocfs2 -o      -type d -false  \) -prune -o -print0  | wc 
-l                     
==4520== Memcheck, a memory error detector                                      
                    
==4520== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.        
                    
==4520== Using Valgrind-3.12.0 and LibVEX; rerun with -h for copyright info     
                    
==4520== Command: /usr/bin/find /lib/modules/ -ignore_readdir_race ( -fstype 
NFS -o -fstype nfs -o -fstype nfs4 -o -fstype afs -o -fstype binfmt_misc -o 
-fstype proc -o -fstype smbfs -o -fstype autofs -o -fstype iso9660 -o -fstype 
ncpfs -o -fstype coda -o -fstype devpts -o -fstype ftpfs -o -fstype devfs -o 
-fstype mfs -o -fstype shfs -o -fstype sysfs -o -fstype cifs -o -fstype 
lustre_lite -o -fstype tmpfs -o -fstype usbfs -o -fstype udf -o -fstype ocfs2 
-o -type d -false ) -prune -o -print0
==4520== 



0
==4520== 
==4520== HEAP SUMMARY:
==4520==     in use at exit: 7,625,040 bytes in 3,340,960 blocks
==4520==   total heap usage: 17,220,046 allocs, 13,879,086 frees, 550,821,122 
bytes allocated
==4520== 
==4520== 8 bytes in 1 blocks are still reachable in loss record 1 of 30
==4520==    at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
==4520==    by 0x127E28: ??? (in /usr/bin/find)
==4520==    by 0x127FE3: ??? (in /usr/bin/find)
==4520==    by 0x117AE8: ??? (in /usr/bin/find)
==4520==    by 0x1106E2: ??? (in /usr/bin/find)
==4520==    by 0x1158A8: ??? (in /usr/bin/find)
==4520==    by 0x1158A8: ??? (in /usr/bin/find)
==4520==    by 0x1158A8: ??? (in /usr/bin/find)
==4520==    by 0x110CDD: pred_and (in /usr/bin/find)
==4520==    by 0x1158A8: ??? (in /usr/bin/find)
==4520==    by 0x1158A8: ??? (in /usr/bin/find)
==4520==    by 0x1111BD: pred_or (in /usr/bin/find)
==4520== 
==4520== 16 bytes in 1 blocks are still reachable in loss record 2 of 30
==4520==    at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
==4520==    by 0x115A12: ??? (in /usr/bin/find)
==4520==    by 0x10F0BF: ??? (in /usr/bin/find)
==4520==    by 0x53842B0: (below main) (libc-start.c:291)
==4520== 
==4520== 56 bytes in 1 blocks are still reachable in loss record 3 of 30
==4520==    at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
==4520==    by 0x127E28: ??? (in /usr/bin/find)
==4520==    by 0x127FE3: ??? (in /usr/bin/find)
==4520==    by 0x126EAC: ??? (in /usr/bin/find)
==4520==    by 0x118DF9: ??? (in /usr/bin/find)
==4520==    by 0x119003: ??? (in /usr/bin/find)
==4520==    by 0x113EC7: ??? (in /usr/bin/find)
==4520==    by 0x10F158: ??? (in /usr/bin/find)
==4520==    by 0x53842B0: (below main) (libc-start.c:291)
==4520== 
==4520== 312 bytes in 1 blocks are still reachable in loss record 4 of 30
==4520==    at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
==4520==    by 0x127E28: ??? (in /usr/bin/find)
==4520==    by 0x127F98: ??? (in /usr/bin/find)
==4520==    by 0x112C90: ??? (in /usr/bin/find)
==4520==    by 0x112DBE: ??? (in /usr/bin/find)
==4520==    by 0x114706: ??? (in /usr/bin/find)
==4520==    by 0x11A614: ??? (in /usr/bin/find)
==4520==    by 0x113EC7: ??? (in /usr/bin/find)
==4520==    by 0x10F158: ??? (in /usr/bin/find)
==4520==    by 0x53842B0: (below main) (libc-start.c:291)
==4520== 
==4520== 312 bytes in 1 blocks are still reachable in loss record 5 of 30
==4520==    at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
==4520==    by 0x127E28: ??? (in /usr/bin/find)
==4520==    by 0x127F98: ??? (in /usr/bin/find)
==4520==    by 0x112C90: ??? (in /usr/bin/find)
==4520==    by 0x112E04: ??? (in /usr/bin/find)
==4520==    by 0x119E7A: ??? (in /usr/bin/find)
==4520==    by 0x113EC7: ??? (in /usr/bin/find)
==4520==    by 0x10F158: ??? (in /usr/bin/find)
==4520==    by 0x53842B0: (below main) (libc-start.c:291)
==4520== 
==4520== 312 bytes in 1 blocks are still reachable in loss record 6 of 30
==4520==    at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
==4520==    by 0x127E28: ??? (in /usr/bin/find)
==4520==    by 0x127F98: ??? (in /usr/bin/find)
==4520==    by 0x112C90: ??? (in /usr/bin/find)
==4520==    by 0x112DBE: ??? (in /usr/bin/find)
==4520==    by 0x119E7A: ??? (in /usr/bin/find)
==4520==    by 0x113EC7: ??? (in /usr/bin/find)
==4520==    by 0x10F158: ??? (in /usr/bin/find)
==4520==    by 0x53842B0: (below main) (libc-start.c:291)
==4520== 
==4520== 312 bytes in 1 blocks are still reachable in loss record 7 of 30
==4520==    at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
==4520==    by 0x127E28: ??? (in /usr/bin/find)
==4520==    by 0x127F98: ??? (in /usr/bin/find)
==4520==    by 0x112C90: ??? (in /usr/bin/find)
==4520==    by 0x112DBE: ??? (in /usr/bin/find)
==4520==    by 0x1146C3: ??? (in /usr/bin/find)
==4520==    by 0x11804C: ??? (in /usr/bin/find)
==4520==    by 0x113EC7: ??? (in /usr/bin/find)
==4520==    by 0x10F158: ??? (in /usr/bin/find)
==4520==    by 0x53842B0: (below main) (libc-start.c:291)
==4520== 
==4520== 312 bytes in 1 blocks are still reachable in loss record 8 of 30
==4520==    at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
==4520==    by 0x127E28: ??? (in /usr/bin/find)
==4520==    by 0x127F98: ??? (in /usr/bin/find)
==4520==    by 0x112C90: ??? (in /usr/bin/find)
==4520==    by 0x112E04: ??? (in /usr/bin/find)
==4520==    by 0x114706: ??? (in /usr/bin/find)
==4520==    by 0x11BCD7: ??? (in /usr/bin/find)
==4520==    by 0x113EC7: ??? (in /usr/bin/find)
==4520==    by 0x10F158: ??? (in /usr/bin/find)
==4520==    by 0x53842B0: (below main) (libc-start.c:291)
==4520== 
==4520== 312 bytes in 1 blocks are still reachable in loss record 9 of 30
==4520==    at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
==4520==    by 0x127E28: ??? (in /usr/bin/find)
==4520==    by 0x127F98: ??? (in /usr/bin/find)
==4520==    by 0x112C90: ??? (in /usr/bin/find)
==4520==    by 0x112DBE: ??? (in /usr/bin/find)
==4520==    by 0x114706: ??? (in /usr/bin/find)
==4520==    by 0x11BCD7: ??? (in /usr/bin/find)
==4520==    by 0x113EC7: ??? (in /usr/bin/find)
==4520==    by 0x10F158: ??? (in /usr/bin/find)
==4520==    by 0x53842B0: (below main) (libc-start.c:291)
==4520== 
==4520== 312 bytes in 1 blocks are still reachable in loss record 10 of 30
==4520==    at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
==4520==    by 0x127E28: ??? (in /usr/bin/find)
==4520==    by 0x127F98: ??? (in /usr/bin/find)
==4520==    by 0x112C90: ??? (in /usr/bin/find)
==4520==    by 0x112D68: ??? (in /usr/bin/find)
==4520==    by 0x119C48: ??? (in /usr/bin/find)
==4520==    by 0x113EC7: ??? (in /usr/bin/find)
==4520==    by 0x10F158: ??? (in /usr/bin/find)
==4520==    by 0x53842B0: (below main) (libc-start.c:291)
==4520== 
==4520== 312 bytes in 1 blocks are still reachable in loss record 11 of 30
==4520==    at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
==4520==    by 0x127E28: ??? (in /usr/bin/find)
==4520==    by 0x127F98: ??? (in /usr/bin/find)
==4520==    by 0x112C90: ??? (in /usr/bin/find)
==4520==    by 0x112E04: ??? (in /usr/bin/find)
==4520==    by 0x114706: ??? (in /usr/bin/find)
==4520==    by 0x117ED8: ??? (in /usr/bin/find)
==4520==    by 0x113EC7: ??? (in /usr/bin/find)
==4520==    by 0x10F158: ??? (in /usr/bin/find)
==4520==    by 0x53842B0: (below main) (libc-start.c:291)
==4520== 
==4520== 312 bytes in 1 blocks are still reachable in loss record 12 of 30
==4520==    at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
==4520==    by 0x127E28: ??? (in /usr/bin/find)
==4520==    by 0x127F98: ??? (in /usr/bin/find)
==4520==    by 0x112C90: ??? (in /usr/bin/find)
==4520==    by 0x112DBE: ??? (in /usr/bin/find)
==4520==    by 0x114706: ??? (in /usr/bin/find)
==4520==    by 0x117ED8: ??? (in /usr/bin/find)
==4520==    by 0x113EC7: ??? (in /usr/bin/find)
==4520==    by 0x10F158: ??? (in /usr/bin/find)
==4520==    by 0x53842B0: (below main) (libc-start.c:291)
==4520== 
==4520== 312 bytes in 1 blocks are still reachable in loss record 13 of 30
==4520==    at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
==4520==    by 0x127E28: ??? (in /usr/bin/find)
==4520==    by 0x127F98: ??? (in /usr/bin/find)
==4520==    by 0x112C90: ??? (in /usr/bin/find)
==4520==    by 0x112DBE: ??? (in /usr/bin/find)
==4520==    by 0x114706: ??? (in /usr/bin/find)
==4520==    by 0x118FBD: ??? (in /usr/bin/find)
==4520==    by 0x113EC7: ??? (in /usr/bin/find)
==4520==    by 0x10F158: ??? (in /usr/bin/find)
==4520==    by 0x53842B0: (below main) (libc-start.c:291)
==4520== 
==4520== 312 bytes in 1 blocks are still reachable in loss record 14 of 30
==4520==    at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
==4520==    by 0x127E28: ??? (in /usr/bin/find)
==4520==    by 0x127F98: ??? (in /usr/bin/find)
==4520==    by 0x112475: ??? (in /usr/bin/find)
==4520==    by 0x11359A: ??? (in /usr/bin/find)
==4520==    by 0x114027: ??? (in /usr/bin/find)
==4520==    by 0x10F158: ??? (in /usr/bin/find)
==4520==    by 0x53842B0: (below main) (libc-start.c:291)
==4520== 
==4520== 312 bytes in 1 blocks are still reachable in loss record 15 of 30
==4520==    at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
==4520==    by 0x127E28: ??? (in /usr/bin/find)
==4520==    by 0x127F98: ??? (in /usr/bin/find)
==4520==    by 0x112475: ??? (in /usr/bin/find)
==4520==    by 0x113649: ??? (in /usr/bin/find)
==4520==    by 0x114027: ??? (in /usr/bin/find)
==4520==    by 0x10F158: ??? (in /usr/bin/find)
==4520==    by 0x53842B0: (below main) (libc-start.c:291)
==4520== 
==4520== 312 bytes in 1 blocks are still reachable in loss record 16 of 30
==4520==    at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
==4520==    by 0x127E28: ??? (in /usr/bin/find)
==4520==    by 0x127F98: ??? (in /usr/bin/find)
==4520==    by 0x112475: ??? (in /usr/bin/find)
==4520==    by 0x11359A: ??? (in /usr/bin/find)
==4520==    by 0x11360A: ??? (in /usr/bin/find)
==4520==    by 0x11360A: ??? (in /usr/bin/find)
==4520==    by 0x114027: ??? (in /usr/bin/find)
==4520==    by 0x10F158: ??? (in /usr/bin/find)
==4520==    by 0x53842B0: (below main) (libc-start.c:291)
==4520== 
==4520== 312 bytes in 1 blocks are still reachable in loss record 17 of 30
==4520==    at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
==4520==    by 0x127E28: ??? (in /usr/bin/find)
==4520==    by 0x127F98: ??? (in /usr/bin/find)
==4520==    by 0x112475: ??? (in /usr/bin/find)
==4520==    by 0x11359A: ??? (in /usr/bin/find)
==4520==    by 0x11360A: ??? (in /usr/bin/find)
==4520==    by 0x11360A: ??? (in /usr/bin/find)
==4520==    by 0x11360A: ??? (in /usr/bin/find)
==4520==    by 0x114027: ??? (in /usr/bin/find)
==4520==    by 0x10F158: ??? (in /usr/bin/find)
==4520==    by 0x53842B0: (below main) (libc-start.c:291)
==4520== 
==4520== 616 bytes in 273 blocks are definitely lost in loss record 18 of 30
==4520==    at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
==4520==    by 0x127E28: ??? (in /usr/bin/find)
==4520==    by 0x127FE3: ??? (in /usr/bin/find)
==4520==    by 0x12BE9A: ??? (in /usr/bin/find)
==4520==    by 0x117A1B: ??? (in /usr/bin/find)
==4520==    by 0x11BE2E: ??? (in /usr/bin/find)
==4520==    by 0x113EC7: ??? (in /usr/bin/find)
==4520==    by 0x10F158: ??? (in /usr/bin/find)
==4520==    by 0x53842B0: (below main) (libc-start.c:291)
==4520== 
==4520== 1,936 bytes in 858 blocks are indirectly lost in loss record 19 of 30
==4520==    at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
==4520==    by 0x127E28: ??? (in /usr/bin/find)
==4520==    by 0x127FE3: ??? (in /usr/bin/find)
==4520==    by 0x12BE9A: ??? (in /usr/bin/find)
==4520==    by 0x11BD68: ??? (in /usr/bin/find)
==4520==    by 0x113EC7: ??? (in /usr/bin/find)
==4520==    by 0x10F158: ??? (in /usr/bin/find)
==4520==    by 0x53842B0: (below main) (libc-start.c:291)
==4520== 
==4520== 2,184 bytes in 7 blocks are still reachable in loss record 20 of 30
==4520==    at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
==4520==    by 0x127E28: ??? (in /usr/bin/find)
==4520==    by 0x127F98: ??? (in /usr/bin/find)
==4520==    by 0x112C90: ??? (in /usr/bin/find)
==4520==    by 0x112DBE: ??? (in /usr/bin/find)
==4520==    by 0x114706: ??? (in /usr/bin/find)
==4520==    by 0x11BDC4: ??? (in /usr/bin/find)
==4520==    by 0x113EC7: ??? (in /usr/bin/find)
==4520==    by 0x10F158: ??? (in /usr/bin/find)
==4520==    by 0x53842B0: (below main) (libc-start.c:291)
==4520== 
==4520== 4,992 bytes in 16 blocks are still reachable in loss record 21 of 30
==4520==    at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
==4520==    by 0x127E28: ??? (in /usr/bin/find)
==4520==    by 0x127F98: ??? (in /usr/bin/find)
==4520==    by 0x112C90: ??? (in /usr/bin/find)
==4520==    by 0x112DBE: ??? (in /usr/bin/find)
==4520==    by 0x114706: ??? (in /usr/bin/find)
==4520==    by 0x11BCD7: ??? (in /usr/bin/find)
==4520==    by 0x11BDA2: ??? (in /usr/bin/find)
==4520==    by 0x113EC7: ??? (in /usr/bin/find)
==4520==    by 0x10F158: ??? (in /usr/bin/find)
==4520==    by 0x53842B0: (below main) (libc-start.c:291)
==4520== 
==4520== 6,424 bytes in 858 blocks are indirectly lost in loss record 22 of 30
==4520==    at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
==4520==    by 0x127E28: ??? (in /usr/bin/find)
==4520==    by 0x127FE3: ??? (in /usr/bin/find)
==4520==    by 0x12BEAB: ??? (in /usr/bin/find)
==4520==    by 0x11BD68: ??? (in /usr/bin/find)
==4520==    by 0x113EC7: ??? (in /usr/bin/find)
==4520==    by 0x10F158: ??? (in /usr/bin/find)
==4520==    by 0x53842B0: (below main) (libc-start.c:291)
==4520== 
==4520== 6,688 bytes in 858 blocks are indirectly lost in loss record 23 of 30
==4520==    at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
==4520==    by 0x127E28: ??? (in /usr/bin/find)
==4520==    by 0x127FE3: ??? (in /usr/bin/find)
==4520==    by 0x12BE75: ??? (in /usr/bin/find)
==4520==    by 0x11BD68: ??? (in /usr/bin/find)
==4520==    by 0x113EC7: ??? (in /usr/bin/find)
==4520==    by 0x10F158: ??? (in /usr/bin/find)
==4520==    by 0x53842B0: (below main) (libc-start.c:291)
==4520== 
==4520== 7,488 bytes in 24 blocks are still reachable in loss record 24 of 30
==4520==    at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
==4520==    by 0x127E28: ??? (in /usr/bin/find)
==4520==    by 0x127F98: ??? (in /usr/bin/find)
==4520==    by 0x112C90: ??? (in /usr/bin/find)
==4520==    by 0x112D68: ??? (in /usr/bin/find)
==4520==    by 0x119B88: ??? (in /usr/bin/find)
==4520==    by 0x113EC7: ??? (in /usr/bin/find)
==4520==    by 0x10F158: ??? (in /usr/bin/find)
==4520==    by 0x53842B0: (below main) (libc-start.c:291)
==4520== 
==4520== 14,080 bytes in 858 blocks are indirectly lost in loss record 25 of 30
==4520==    at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
==4520==    by 0x127E28: ??? (in /usr/bin/find)
==4520==    by 0x127FE3: ??? (in /usr/bin/find)
==4520==    by 0x12BE87: ??? (in /usr/bin/find)
==4520==    by 0x11BD68: ??? (in /usr/bin/find)
==4520==    by 0x113EC7: ??? (in /usr/bin/find)
==4520==    by 0x10F158: ??? (in /usr/bin/find)
==4520==    by 0x53842B0: (below main) (libc-start.c:291)
==4520== 
==4520== 46,816 bytes in 836 blocks are indirectly lost in loss record 26 of 30
==4520==    at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
==4520==    by 0x127E28: ??? (in /usr/bin/find)
==4520==    by 0x12BE65: ??? (in /usr/bin/find)
==4520==    by 0x11BD68: ??? (in /usr/bin/find)
==4520==    by 0x113EC7: ??? (in /usr/bin/find)
==4520==    by 0x10F158: ??? (in /usr/bin/find)
==4520==    by 0x53842B0: (below main) (libc-start.c:291)
==4520== 
==4520== 77,176 (1,232 direct, 75,944 indirect) bytes in 22 blocks are 
definitely lost in loss record 27 of 30
==4520==    at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
==4520==    by 0x127E28: ??? (in /usr/bin/find)
==4520==    by 0x12BE65: ??? (in /usr/bin/find)
==4520==    by 0x11BD68: ??? (in /usr/bin/find)
==4520==    by 0x113EC7: ??? (in /usr/bin/find)
==4520==    by 0x10F158: ??? (in /usr/bin/find)
==4520==    by 0x53842B0: (below main) (libc-start.c:291)
==4520== 
==4520== 1,075,448 bytes in 476,619 blocks are definitely lost in loss record 
28 of 30
==4520==    at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
==4520==    by 0x127E28: ??? (in /usr/bin/find)
==4520==    by 0x127FE3: ??? (in /usr/bin/find)
==4520==    by 0x12BE9A: ??? (in /usr/bin/find)
==4520==    by 0x117A1B: ??? (in /usr/bin/find)
==4520==    by 0x1106E2: ??? (in /usr/bin/find)
==4520==    by 0x1158A8: ??? (in /usr/bin/find)
==4520==    by 0x1158A8: ??? (in /usr/bin/find)
==4520==    by 0x1111BD: pred_or (in /usr/bin/find)
==4520==    by 0x1158A8: ??? (in /usr/bin/find)
==4520==    by 0x1158A8: ??? (in /usr/bin/find)
==4520==    by 0x110CDD: pred_and (in /usr/bin/find)
==4520== 
==4520== 1,075,448 bytes in 476,619 blocks are definitely lost in loss record 
29 of 30
==4520==    at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
==4520==    by 0x127E28: ??? (in /usr/bin/find)
==4520==    by 0x127FE3: ??? (in /usr/bin/find)
==4520==    by 0x12BE9A: ??? (in /usr/bin/find)
==4520==    by 0x117A1B: ??? (in /usr/bin/find)
==4520==    by 0x1106E2: ??? (in /usr/bin/find)
==4520==    by 0x1158A8: ??? (in /usr/bin/find)
==4520==    by 0x1158A8: ??? (in /usr/bin/find)
==4520==    by 0x1158A8: ??? (in /usr/bin/find)
==4520==    by 0x110CDD: pred_and (in /usr/bin/find)
==4520==    by 0x1158A8: ??? (in /usr/bin/find)
==4520==    by 0x1158A8: ??? (in /usr/bin/find)
==4520== 
==4520== 5,377,240 bytes in 2,383,095 blocks are definitely lost in loss record 
30 of 30
==4520==    at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
==4520==    by 0x127E28: ??? (in /usr/bin/find)
==4520==    by 0x127FE3: ??? (in /usr/bin/find)
==4520==    by 0x12BE9A: ??? (in /usr/bin/find)
==4520==    by 0x117A1B: ??? (in /usr/bin/find)
==4520==    by 0x1106E2: ??? (in /usr/bin/find)
==4520==    by 0x1158A8: ??? (in /usr/bin/find)
==4520==    by 0x1158A8: ??? (in /usr/bin/find)
==4520==    by 0x1111BD: pred_or (in /usr/bin/find)
==4520==    by 0x1158A8: ??? (in /usr/bin/find)
==4520==    by 0x1111BD: pred_or (in /usr/bin/find)
==4520==    by 0x1158A8: ??? (in /usr/bin/find)
==4520== 
==4520== LEAK SUMMARY:
==4520==    definitely lost: 7,529,984 bytes in 3,336,628 blocks
==4520==    indirectly lost: 75,944 bytes in 4,268 blocks
==4520==      possibly lost: 0 bytes in 0 blocks
==4520==    still reachable: 19,112 bytes in 64 blocks
==4520==         suppressed: 0 bytes in 0 blocks
==4520== 
==4520== For counts of detected and suppressed errors, rerun with: -v
==4520== ERROR SUMMARY: 5 errors from 5 contexts (suppressed: 0 from 0)


Pay attention that I removed some kernel modules directories from /lib/modules 
after my email, so now the memory usage is lower

/usr/bin/time /usr/bin/find /lib/modules/ -ignore_readdir_race      \( -fstype 
NFS -o -fstype nfs -o -fstype nfs4 -o -fstype afs -o -fstype binfmt_misc -o 
-fstype proc -o -fstype smbfs -o -fstype autofs -o -fstype iso9660 -o -fstype 
ncpfs -o -fstype coda -o -fstype devpts -o -fstype ftpfs -o -fstype devfs -o 
-fstype mfs -o -fstype shfs -o -fstype sysfs -o -fstype cifs -o -fstype 
lustre_lite -o -fstype tmpfs -o -fstype usbfs -o -fstype udf -o -fstype ocfs2 
-o      -type d -false  \) -prune -o -print0  | wc -l
5.62user 5.00system 0:10.63elapsed 99%CPU (0avgtext+0avgdata 110204maxresident)k
0inputs+0outputs (0major+26952minor)pagefaults 0swaps

(only 110MB used, but before the memory usage was ~600MB)



> However, your patch is fine, of course; I wrapped it into a nice commit:
> 
>   [PATCH 1/5] find: fix memory leak in mount list handling
> 
> I also added a wrapper around gnulib's read_file_system_list() in order to
> always free the list in the next invocation.  By this, we still get the same
> semantics.
> 
>   [PATCH 2/5] maint: remove unused get_mounted_filesystems
>   [PATCH 3/5] maint: inline now last use of must_read_fs_list
>   [PATCH 4/5] maint: move is_used_fs_type to fstype.c
>   [PATCH 5/5] find: avoid unneccessary reading of the mount list
> 
> One could argument that reading the mount list just once would be
> enough, yet this would maybe change the result for automounted file systems.
> Well, we could still do this further optimization at a later step.
> 
> Have a nice day,
> Berny
> 


-- 
gpg @keyserver.linux.it: Goffredo Baroncelli <kreijackATinwind.it>
Key fingerprint BBF5 1610 0B64 DAC6 5F7D  17B2 0EDA 9B37 8B82 E0B5