Re: make install using "-j"

[Paul Eggert]

Ralf Wildenhues <address@hidden> writes in
<http://lists.gnu.org/archive/html/bug-automake/2006-06/msg00025.html>:

> Not sure about an easy way to fix the first bug as well here.
>       * lib/install-sh: HP-UX 11.23 `mkdir -m u=rwx,[...],u+wx -p'
>       may exit 0 but create intermediate directories with bogus
>       permissions.  So change the `$mkdirprog -m $test_mode' test to
>       create actual directories, and test for readability of the
>       intermediate one.  Clean up afterwards.

This patch assumes the working directory is writable, which isn't a
safe assumption.  Also, it won't work as root, since 'test -r D'
succeeds as root even if D has mode d-wx-w--w-.  And, as you mentioned,
the patch doesn't work around the "mkdir -m u=rwx,g=rx,o=rx -p -- foo/bar"
bug on UP-UX 11.23 and IRIX 6.5 that you reported in
<http://lists.gnu.org/archive/html/bug-automake/2006-06/msg00024.html>.

On my list of things to do is to fix GNU install to be safer and more
compatible with BSD install, with respect to the modes of intermediate
directories and temporary files.  Among other things, GNU install
should use mode 755 as modified by the umask to create temporary
directories, instead of what it currently does (which is an
undocumented algorithm that is similar to, but not quite identical to,
mkdir -p's algorithm).  I'll CC: this message to bug-coreutils to give
them a heads-up on this.

Anyway, to get back to install-sh: it needs a similar patch, so your
email prompted me to write it.  As a side effect I think this patch
works around both bugs you reported.  So I propose the following
Automake patch instead.  Does it work for you?

Index: ChangeLog
===================================================================
RCS file: /cvs/automake/automake/ChangeLog,v
retrieving revision 1.2901
diff -p -u -r1.2901 ChangeLog
--- ChangeLog   7 Jun 2006 06:01:22 -0000       1.2901
+++ ChangeLog   22 Jun 2006 21:08:06 -0000
@@ -1,3 +1,20 @@
+2006-06-22  Paul Eggert  <address@hidden>
+
+       * lib/install-sh: Don't incorrectly claim that this implementation
+       can install only one file at a time.
+       (test_mode, intermediate_mode): Remove.
+       Use octal modes if the invoker specifies an octal mode, and use
+       octal umask values if 'umask' outputs octal values; in the usual
+       case this works around bugs with symbolic modes with HP-UX 11.23
+       and IRIX 6.5 mkdir as reported by Ralf Wildenhues in
+       <http://lists.gnu.org/archive/html/bug-automake/2006-06/msg00024.html>
+       since Automake uses octal modes.
+       (cp_umask, mkdir_umask, mkdir_mode): New variables, to avoid
+       temporarily creating files or directories with too-permissive modes.
+       Use the FreeBSD method for computing modes of intermediate directories.
+       (posix_mkdir): Use ':' for true, not 'true'; this is a bit faster on
+       traditional implementations.
+
 2006-06-07  Alexandre Duret-Lutz  <address@hidden>
 
        * automake.in (handle_LIBOBJS): Don't rely on the caller defining $1.
@@ -508,6 +525,9 @@
        * doc/automake.texi (limitations on file names): New section.
        * lib/install-sh: Rewrite to support '*' in file names.
        Also, tune so that we don't invoke so many commands in the usual case.
+       This has the side effect of fixing `install-sh -d' to not fail if it
+       loses the race in creating the last path component against another
+       process.
        * tests/instspc.test: The "*" test is now fixed.
 
 2005-09-13  Stepan Kasal  <address@hidden>
Index: lib/install-sh
===================================================================
RCS file: /cvs/automake/automake/lib/install-sh,v
retrieving revision 1.34
diff -p -u -r1.34 install-sh
--- lib/install-sh      11 May 2006 19:52:08 -0000      1.34
+++ lib/install-sh      22 Jun 2006 21:08:06 -0000
@@ -39,8 +39,7 @@ scriptversion=2006-05-11.20
 # when there is no Makefile.
 #
 # This script is compatible with the BSD install script, but was written
-# from scratch.  It can only install one file at a time, a restriction
-# shared with many OS's install programs.
+# from scratch.
 
 nl='
 '
@@ -50,6 +49,10 @@ IFS=" ""     $nl"
 
 # Don't use :- since 4.3BSD and earlier shells don't like it.
 doit="${DOITPROG-}"
+case $doit in
+  '') doit_exec=exec ;;
+  ?*) doit_exec=$doit ;;
+esac
 
 # Put in absolute file names if you don't have them in your path;
 # or use environment vars.
@@ -66,17 +69,9 @@ mkdirprog="${MKDIRPROG-mkdir}"
 posix_glob=
 posix_mkdir=
 
-# Symbolic mode for testing mkdir with directories.
-# It is the same as 755, but also tests that "u+" works.
-test_mode=u=rwx,g=rx,o=rx,u+wx
-
 # Desired mode of installed file.
 mode=0755
 
-# Desired mode of newly created intermediate directories.
-# It is empty if not known yet.
-intermediate_mode=
-
 chmodcmd=$chmodprog
 chowncmd=
 chgrpcmd=
@@ -191,7 +186,31 @@ if test $# -eq 0; then
   exit 0
 fi
 
-test -n "$dir_arg" || trap '(exit $?); exit' 1 2 13 15
+if test -z "$dir_arg"; then
+  trap '(exit $?); exit' 1 2 13 15
+
+  # Set umask so as not to create temps with too-generous modes.
+  # However, 'strip' requires both read and write access to temps.
+  case $mode in
+    # Optimize common cases.
+    *644) cp_umask=133 ;;
+    *755) cp_umask=22 ;;
+
+    *[0-7])
+      case $stripcmd in
+       '') u_plus_rw= ;;
+       ?*) u_plus_rw='% 200' ;;
+      esac
+      cp_umask=`expr '(' 777 - $mode % 1000 ')' $u_plus_rw`
+      ;;
+    *)
+      case $stripcmd in
+       '') u_plus_rw= ;;
+       ?*) u_plus_rw=,u+rw ;;
+      esac
+      cp_umask=$mode$u_plus_rw ;;
+  esac
+fi
 
 for src
 do
@@ -274,40 +293,47 @@ do
   if test $dstdir_status != 0; then
     case $posix_mkdir in
       '')
+       # Create intermediate dirs using mode 755 as modified by the umask.
+       # This is like FreeBSD 'install' as of 1997-10-28.
+       umask=`umask`
+       case $stripcmd.$umask in
+         # Optimize common cases.
+         *[2367][2367]) mkdir_umask=$umask ;;
+         .*0[02][02] | .[02][02] | .[02]) mkdir_umask=22 ;;
+
+         *[0-7])
+           mkdir_umask=`expr $umask + 22 \
+             - $umask % 100 % 40 + $umask % 20 \
+             - $umask % 10 % 4 + $umask % 2
+           `;;
+         *) mkdir_umask=$umask,go-w ;;
+       esac
+
+       # With -d, create the new directory with the user-specified mode.
+       # Otherwise, rely on $mkdir_umask.
+       if test -n "$dir_arg"; then
+         mkdir_mode=-m$mode
+       else
+         mkdir_mode=
+       fi
+
        posix_mkdir=false
-       if $mkdirprog -m $test_mode -p -- / >/dev/null 2>&1; then
-         posix_mkdir=true
+       if $mkdirprog $mkdir_mode -p -- / >/dev/null 2>&1; then
+         posix_mkdir=:
        else
          # Remove any dirs left behind by ancient mkdir implementations.
-         rmdir ./-m "$test_mode" ./-p ./-- 2>/dev/null
+         case $mkdir_mode in
+           ?*) mkdir_mode=./$mkdir_mode ;;
+         esac
+         rmdir $mkdir_mode ./-p ./-- 2>/dev/null
        fi ;;
     esac
 
     if
-      $posix_mkdir && {
-
-       # With -d, create the new directory with the user-specified mode.
-       # Otherwise, create it using the same intermediate mode that
-       # mkdir -p would use when creating intermediate directories.
-       # POSIX says that this mode is "$(umask -S),u+wx", so use that
-       # if umask -S works.
-
-       if test -n "$dir_arg"; then
-         mkdir_mode=$mode
-       else
-         case $intermediate_mode in
-           '')
-             if umask_S=`(umask -S) 2>/dev/null`; then
-               intermediate_mode=$umask_S,u+wx
-             else
-               intermediate_mode=$test_mode
-             fi ;;
-         esac
-         mkdir_mode=$intermediate_mode
-       fi
-
-       $mkdirprog -m "$mkdir_mode" -p -- "$dstdir"
-      }
+      $posix_mkdir && (
+       umask $mkdir_umask
+       $doit_exec $mkdirprog $mkdir_mode -p -- "$dstdir"
+      )
     then :
     else
 
@@ -349,7 +375,8 @@ do
          prefixes=
        else
          if $posix_mkdir; then
-           $mkdirprog -m "$mkdir_mode" -p -- "$dstdir" && break
+           (umask=$mkdir_umask
+            $doit_exec $mkdirprog $mkdir_mode -p -- "$dstdir") && break
            # Don't fail if two instances are running concurrently.
            test -d "$prefix" || exit 1
          else
@@ -365,7 +392,9 @@ do
 
       if test -n "$prefixes"; then
        # Don't fail if two instances are running concurrently.
-       eval "\$mkdirprog $prefixes" || test -d "$dstdir" || exit 1
+       (umask $mkdir_umask
+        eval "\$doit_exec \$mkdirprog $prefixes") ||
+         test -d "$dstdir" || exit 1
        obsolete_mkdir_used=true
       fi
     fi
@@ -386,7 +415,7 @@ do
     trap 'ret=$?; rm -f "$dsttmp" "$rmtmp" && exit $ret' 0
 
     # Copy the file name to the temp name.
-    $doit $cpprog "$src" "$dsttmp" &&
+    (umask $cp_umask; $doit_exec $cpprog "$src" "$dsttmp") &&
 
     # and set any options; do chmod last to preserve setuid bits.
     #