[bug#67868] intentionally unreadable dirs make distcheck fail

[Karl Berry]

(Sorry, long message.)

As of a few days ago, in a normal working directory, running make
distcheck twice immediately fails the second time (for me):

  if test -d "automake-1.16i"; then find "automake-1.16i" -type d ! -perm -200 
-exec chmod u+w {} ';' && rm -rf "automake-1.16i" || { sleep 5 && rm -rf 
"automake-1.16i"; }; else :; fi
  find: 'automake-1.16i/_build/sub/t/uninstall-fail.dir/__inst-dir__/share': 
Permission denied
  rm: cannot remove 
'automake-1.16i/_build/sub/t/uninstall-fail.dir/__inst-dir__/share': Permission 
denied
  make[2]: *** [Makefile:3266: distdir-am] Error 1

where t/uninstall-fail.dir/__inst-dir__/share is the directory made
intentionally unreadable/unwritable/unsearchable by the uninstall-fail
test, in order to perform its job.

I'm not sure what caused the new failure. Maybe something changing in my
environment (Alma Linux 8), though I don't know what it was. ACL's and
SElinux are not (and never have been) involved, so far as I know.

The failure was apparently because am__remove_distdir only made
unwritable directories writable, but it also seems necessary to make
them readable and searchable. At least, changing distdir.am like this:

+++ b/lib/am/distdir.am
@@ -23,7 +23,7 @@
 
 am__remove_distdir = \
   if test -d "$(distdir)"; then \
-    find "$(distdir)" -type d ! -perm -200 -exec chmod u+w {} ';' \
+    find "$(distdir)" -type d ! -perm -700 -exec chmod u+rwx {} ';' \

got past that immediate failure on startup. Then make distcheck failed
again, for similar reasons, at the make distcleancheck at the end:

    cannot chdir to child for t/uninstall-fail.dir/__inst-dir__/share: 
Permission denied at ../../t/ax/deltree.pl line 29.
    cannot remove directory for t/uninstall-fail.dir/__inst-dir__: Directory 
not empty at ../../t/ax/deltree.pl line 29.
    cannot remove directory for t/uninstall-fail.dir: Directory not empty at 
../../t/ax/deltree.pl line 29.
    make[1]: *** [Makefile:4286: clean-local-check] Error 1
    make[1]: Leaving directory '/u/karl/gnu/src/akarl/automake-1.16i/_build/sub'
    make: *** [Makefile:3363: distcheck] Error 1

I eventually found that these errors come from the command:
/usr/local/bin/perl ../../t/ax/deltree.pl t/*.dir t/*/*.dir */t/*.dir

which is run as the clean-local-check target (defined in t/local.mk),
which is the dependency of clean-local, which in turn part of the usual
distclean.

Again, I could not see any good reason why this started failing now when
it's been fine for ages. The sub-second patch is an obvious candidate,
as a recent and pervasive change, although the failure is not about
parallelism (surprisingly), since I ran the whole thing without -j, and
the same errors resulted.

Lacking any better insight, I threw in another "fix the horrible mode 0
directories" command before the deltree:

+++ b/t/local.mk
@@ -278,6 +278,7 @@ EXTRA_DIST += $(perf_TESTS)
 clean-local: clean-local-check
 .PHONY: clean-local-check
 clean-local-check:
+       find . -type d ! -perm -700 -exec chmod u+rwx {} ';'
        $(AM_V_GEN)$(PERL) $(srcdir)/t/ax/deltree.pl t/*.dir t/*/*.dir */t/*.dir

which made the distclean(check) succeed again.

Furthermore, that mode 0 uninstall-fail.dir/__inst-dir__/share directory
has caused me (at least) quite a bit of unnecessary grief over the
years.  I didn't see any particular reason not to chmod it back to
something reasonable at the end of the test, instead of the whole rest
of the world having to work around the mode 0.  So I made that change too:

+++ b/t/uninstall-fail.sh
@@ -98,4 +98,7 @@ run_make -M -e FAIL uninstall
 
 $EGREP "(cd|sh)(\[[0-9]*[0-9]\])?: .*$inst/share" output
 
+# restore normal permissions so the rest of the world doesn't have to deal.
+chmod u+rwx $inst/share

If anyone has any ideas about what's really going on, and/or if there
are better fixes out there, I'm all ears. Assuming it fails for other
people too, maybe someone smarter than me can run git bisect or
something and discern why the problem started now, at this late date.

Sorry for the horribly verbose explanations of three lines of changes.

At any rate I'm committing the above changes; full patch below. --thanks, karl.

-----------------------------------------------------------------------------
dist: more forcefully deal with mode 0 directories created by tests.

* lib/am/distdir.am (am__remove_distdir): make directories
readable and searchable, not just writable. (Also typo.)
* t/local.mk (clean-local-check): ensure directories are
at least mode 700
* t/uninstall-fail.sh: restore reasonable permissions of
the mode 0 $inst/share directory at the end.
diff --git a/lib/am/distdir.am b/lib/am/distdir.am
index 264713c33..301239de1 100644
--- a/lib/am/distdir.am
+++ b/lib/am/distdir.am
@@ -23,7 +23,7 @@ top_distdir = $(distdir)

 am__remove_distdir = \
   if test -d "$(distdir)"; then \
-    find "$(distdir)" -type d ! -perm -200 -exec chmod u+w {} ';' \
+    find "$(distdir)" -type d ! -perm -700 -exec chmod u+rwx {} ';' \
       && rm -rf "$(distdir)" \
 ## On MSYS (1.0.17) it is not possible to remove a directory that is in
 ## use; so, if the first rm fails, we sleep some seconds and retry, to
@@ -213,7 +213,7 @@ endif %?TOPDIR_P%
          fi; \
        done
 ##
-## Test for directory existence here because previous automake
+## Test for directory existence here because a previous automake
 ## invocation might have created some directories.  Note that we
 ## explicitly set distdir for the subdir make; that lets us mix-n-match
 ## many automake-using packages into one large package, and have "dist"
diff --git a/t/local.mk b/t/local.mk
index 4fa46a071..51386fd00 100644
--- a/t/local.mk
+++ b/t/local.mk
@@ -278,6 +278,7 @@ EXTRA_DIST += $(perf_TESTS)
 clean-local: clean-local-check
 .PHONY: clean-local-check
 clean-local-check:
+       find . -type d ! -perm -700 -exec chmod u+rwx {} ';'
        $(AM_V_GEN)$(PERL) $(srcdir)/t/ax/deltree.pl t/*.dir t/*/*.dir */t/*.dir

 # vim: ft=automake noet
diff --git a/t/uninstall-fail.sh b/t/uninstall-fail.sh
index a1a365d90..60e131fc3 100644
--- a/t/uninstall-fail.sh
+++ b/t/uninstall-fail.sh
@@ -98,4 +98,7 @@ run_make -M -e FAIL uninstall

 $EGREP "(cd|sh)(\[[0-9]*[0-9]\])?: .*$inst/share" output

+# restore reasonable permissions so the rest of the world doesn't have to deal.
+chmod u+rwx $inst/share
+
 :