rm: avoiding a race condition on non-glibc systems

[Jim Meyering]

In reviewing parts of remove.c I noted (again) the race condition on
systems with an unlink that may remove directories, so added this comment:

+/* If anyone knows of another system for which unlink can never
+   remove a directory, please report it to address@hidden
+   The code below is slightly more efficient if it *knows* that
+   unlink(2) cannot possibly unlink a directory.  */

On non-glibc systems that race condition means that there's a window
via which a privileged rm user might be tricked into unlinking a
nonempty directory.

I investigated for Solaris-10 and found that this privilege
can be tested at run-time.  Here's a little code to test
whether its PRIV_SYS_LINKDIR is in the effective set:

/* This is Solaris-10-specific -- at least <priv.h> */
#include <priv.h>
#include <stdlib.h>
#include <stdbool.h>

static bool
can_unlink_directory (void)
{
#ifdef __GLIBC__
  return false;
#elif defined PRIV_EFFECTIVE && defined PRIV_SYS_LINKDIR
  /* Solaris 10 */
  priv_set_t *pset = priv_allocset ();
  if (getppriv (PRIV_EFFECTIVE, pset) != 0)
    return true;
  bool can = priv_ismember (pset, PRIV_SYS_LINKDIR);
  priv_freeset (pset);
  return can;
#else
  /* Be pessimistic: assume that unlink can remove a directory.  */
  return true;
#endif
}

int
main ()
{
  exit (can_unlink_directory () ? 0 : 1);
}

That little test program works fine, but I'm not sure I want to
rearrange remove.c enough to add such a run-time test.

At best, the race condition is purely theoretical (barring a manually
suspended `rm -r') and we needn't worry.  If anyone can give an idea
of how easy/hard it would be for a non-privileged user to trick a
running rm process into unlinking a directory, I'd appreciate it.