Re: Memory allocation problem with latest gawk - new testcase

[Aharon Robbins]

Greetings.  Re this:

> Date: Thu, 3 Apr 2003 14:55:15 +0400
> From: Stanislav Ievlev <address@hidden>
> To: Stepan Kasal <address@hidden>
> Cc: Aharon Robbins <address@hidden>, address@hidden,
>    address@hidden, address@hidden
> Subject: Re: Memory allocation problem with latest gawk - new testcase
>
> Problem #1: gawk behaviour depends on kernel version.
>
> In attachment you can find testcase. It's demonstrates bug on 2.2.x kernels.
> Diff between strace output on 2.2.x and 2.4.x kernels also included.

I'm very sorry, but I'm not responsible for kernel bugs.  If it works
correctly under 2.4.x, then use a 2.4 kernel. That's what's current anyway.

> Problem #2: memory leak (works both on 2.2.x and 2.4.x kernels)
>
> $ (ulimit -Sv 16384 && time gawk '/.*/ {;}' /dev/zero)
> gawk: cmd. line:2: fatal: grow_iop_buffer: iop->buf: can't allocate 16777218 
> bytes of memory (Cannot allocate memory)
> 0.03user 0.00system 0:00.12elapsed 24%CPU (0avgtext+0avgdata 0maxresident)k
> 0inputs+0outputs (186major+75minor)pagefaults 0swaps

And:

> > Problem #2: memory leak (works both on 2.2.x and 2.4.x kernels)
> > 
> > $(ulimit -Sv 16384 && time gawk '/.*/ {;}' /dev/zero)
> > gawk: cmd. line:2: fatal: grow_iop_buffer: iop->buf: can't allocate 
> > 16777218 bytes of memory (Cannot allocate memory)
> > 0.03user 0.00system 0:00.12elapsed 24%CPU (0avgtext+0avgdata 0maxresident)k
> > 0inputs+0outputs (186major+75minor)pagefaults 0swaps
>
> Note:
> this testcase works as expected (endless loop) with gawk 3.1.1.

I'm afraid you're not understanding what's going on.  The basic idea is that
gawk reads characters until it finds the record separator, by default a newline.
Since /dev/zero has no newlines in it, it will keep growing the buffer until
it runs out of memory.  You have limited the amount of memory to 16 meg or so,
so of course it eventually runs out of memory, and reports a fatal error.

I would state that the 3.1.1 behavior is a bug, fixed in 3.1.2.

Arnold