bug#43999: sed: couldn't re-allocate memory

[Assaf Gordon]

tag 43999 moreinfo
stop

Hello,

On 2020-10-14 2:40 p.m., Mark Filipak wrote:
> The file "1,073,709,056 bytes" provokes an error (& zero output), but 
> only if piped from 'tr' and only for a particular pattern: /00000100/.
> the file "1,073,739,776 bytes" succeeds with identical parameters.
> The pipe through 'tr' appears not to be the problem.
>
> $ sed --version
> sed (GNU sed) 4.2.2

First,
sed version 4.2.2 is 8 years old.
It is not supported any more.
Please try with the latest version of sed and report back if you still 
get the error.

Second,
please send some information about your environment, namely "uname -a",
are you using a 32bit or 64bit system,
and how much RAM and SWAP does your machine have.

> ...
> $ xxd -p -u "1,073,709,056 bytes" | tr -d '\n' | sed -r 
> 's/00000100/\x0D\x0A&/g' > foo.txt
> sed: couldn't re-allocate memory

Third,
You did not describe what your input file contains.
I implicitly assume from the file name that the file is large (~1GB?).
I did not try reproduce this error, but my hunch is that by deleting
input newlines you are asking sed to load a very large input string and
store it in memory - so a memory-related error is not unexpected.

regards,
 - assaf