Re: 'Argument list too long' in non-recursive setup

[Ralf Wildenhues]

[ adding bug-automake as a reminder for the distdir.am TODO item; this
  is <http://thread.gmane.org/gmane.comp.sysutils.autoconf.bugs/6807> ]

Hello Xan,

* Xan Lopez wrote on Wed, Jul 29, 2009 at 08:15:31AM CEST:
> On Wed, Jul 29, 2009 at 8:55 AM, Xan Lopez<address@hidden> wrote:
> >> On Linux before 2.6.23(?), the default limit for command line plus
> >> environment when executing a process is something like 128KB, but this
> >> can be modified in the kernel (or /proc, I don't remember).  On newer
> >> Linux kernels, this limit has been lifted to not exist any more.
[...]
> > That's right, this limit can be modified recompiling the kernel (it's
> > a #define IIRC), although that workaround is not very practical. It's
> > good to know this is going away in recent kernels though.
> 
> Oh, you wrote 2.6.23, not 2.6.32!

Yep.  :-)

> I'm using 2.6.27, so this should be working. I googled a bit, and
> found this page: http://www.in-ulm.de/~mascheck/various/argmax/, which
> indeed mentions ARG_MAX should be relative to the stack size in new
> kernels. I increased my stack size with ulimit -s, and checked that
> getconf ARG_MAX indeed grows with it. Still, distcheck still fails the
> same way no matter how much I increase the value.

Confirmed over here.  This test:

ulimit -s 32768
ulimit -s
l=some-command-line-argument
while /bin/true $l
do
  echo size ${#l} worked
  l="$l $l"
done

gives me this encouraging output on 2.6.26:

| 32768
| size 26 worked
| [...]
| size 3538943 worked
| size 7077887 worked
| bash: /bin/true: Argument list too long

but this test:

tr T '\t' >Makefile <<\END
l := some-command-line-argument
l := $l $l
l := $l $l
l := $l $l
l := $l $l
l := $l $l
l := $l $l
l := $l $l
l := $l $l
l := $l $l
l := $l $l
l := $l $l
l := $l $l
all:
T@/bin/true $l
T:
address@hidden'$l'; echo size $${#l} worked
T@/bin/true $l $l
T:
address@hidden'$l $l'; echo size $${#l} worked
END
make

maxes out even after the ulimit change:

| :
| size 110591 worked
| :
| make: execvp: /bin/sh: Argument list too long
| make: *** [all] Error 127

with GNU Make 3.81.  This shows the /bin/true commands both went
successful (because make bypassed the shell for these), but the second
command with shell constructs failed.

Looking at the strace:

| 20589 vfork()                           = 20591
| 20591 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
| 20591 execve("/bin/sh", ["/bin/sh", "-c", "l=\'some-command-line-argument 
so"...], [/* 56 vars */] <unfinished ...>
| 20589 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
| 20589 wait4(-1,  <unfinished ...>
| 20591 <... execve resumed> )            = -1 E2BIG (Argument list too long)
| 20591 open("/usr/share/locale/en_US.UTF-8/LC_MESSAGES/libc.mo", O_RDONLY) = 
-1 ENOENT (No such file or directory)
[...]
| 20591 write(2, "make: ", 6)             = 6
| 20591 write(2, "execvp: /bin/sh: Argument list t"..., 39) = 39
| 20591 write(2, "\n", 1)                 = 1
| 20591 exit_group(127)                   = ?
| 20589 <... wait4 resumed> [{WIFEXITED(s) && WEXITSTATUS(s) == 127}], 0, NULL) 
= 20591
| 20589 --- SIGCHLD (Child exited) @ 0 (0) ---
| 20589 rt_sigreturn(0xffffffff)          = 20591

we see that GNU make invokes
  /bin/sh -c "l=...; /bin/true ..."

and passes the whole command as one command line argument to execve.
Now, as Sven's page documents, Linux now has a per-argument limit of
MAX_ARG_STRLEN (131072), which is what we're hitting here now.

I'll take this up with bug-make.  Thank you again for bringing it up.

Cheers,
Ralf