Re: [Help-bash] xmalloc: ../../../bash/lib/sh/makepath.c:116: cannot allocate 265 bytes

[adrelanos]

Bob Proulx:
> adrelanos wrote:
>> #!/bin/bash
>> set -x
>> shopt -s globstar
>> echo /** > ~/testfile
> 
> Wow!  That will be a memory hog.  Megabytes and megabytes.  How much
> virtual memory does your machine have available?

768 MB

> Do you have any network mounted file systems?

No.

> If so then the above
> would try to expand all of those too.  That could cause a lot of
> network traffic and file system activity on the remote systems too.

I see.

> Do you really want all of the files printed on one line without any
> newlines between them?

No, I wasn't sure if this would turn into a bug report. And to make my
question as short as possible, I removed the "for file in /**; do...".

>> echo "end"
>>
>> I am getting:
>>
>> ./a: xmalloc: ../../../bash/lib/sh/makepath.c:116: cannot allocate 265 bytes 
>> (1123962880 bytes allocated)
>>
>> Is this expected?
> 
> It depends upon how much memory you have in your system.  If you do
> not have that much memory then yes this is expected.  I don't think I
> own a machine with enough memory so that would be expected on all of
> my systems.  If you are running a large 64-bit system with a lot of
> virtual memory then your machine might be able to struggle through it.
> It above has allocated 1,123,962,880 bytes.  Well over a gigabyte
> already.  At that point it could not allocate any more memory.

Ok, probably indeed due to too little RAM.

> What system are you using?

Debian Jessie 32 bit

>> Should I better use "find" for such large requests? Can "find" be
>> safely combined with "sort"?
> 
> Definitely better with find!  The 'echo /**' will attempt to do the
> same thing as find but will attempt to do it all in current memory.
> Then it will print out all of the files on one line.  That will be
> gigabytes that will need to be kept in memory for the final print of
> everything all at once.
> 
> Find it much better.  And it will put newlines between the file
> names.  You probably want -xdev to avoid crossing mount points and
> going down /dev and /proc and others.
> 
>   find / -xdev > ~/testfile

Okay, I will rather use find.

> Rather than answer your question about combining sort with find and
> about filenames with arbitrary whitespace such as newlines in them I
> will pause and ask you to say what you are trying to do?

When I run find on the same folder, the output needs to be deterministic
(files in same order) regardless of how often find is being run. Seems
like find already produces deterministic results and sort isn't required
at all.