Re: awk leaks memory on RHEL5_64

[Aharon Robbins]

Greetings. Re this:

> Date: Thu, 7 Jan 2010 19:11:04 -0800
> Subject: awk leaks memory on RHEL5_64
> From: Jeetendra Mirchandani <address@hidden>
> To: address@hidden
>
> Hi,
>
> I have discovered that awk leaks memory on a rhel5_64 box
>
> awk '{if(FILENAME=="file1")interested[$1]=1; else if(interested[$1])print}'
> file1 file2
>
>
> Here file1 is about 40MB in size, with about 500K records. file2 has about
> 220MM records and is about 9GB in size.
>
> awk process grows to 13GB + in size
>
> $ cat /etc/redhat-release
> Red Hat Enterprise Linux Server release 5 (Tikanga)
>
> $ uname -r
> 2.6.18-53.1.14.el5
>
> $ awk --version
> GNU Awk 3.1.5
>
> $ rpm -q gawk
> gawk-3.1.5-14.el5
>
> $ rpm -q glibc
> glibc-2.5-12

Gawk is not leaking memory; your program is not doing what you think
it's doing.  Try:

   awk '{if(FILENAME=="file1")interested[$1]=1; else if($1 in interested)print}'

Then read up about associative arrays in the gawk manual.

Arnold