Re: FYI: a 256,000,000-byte file name

[Jim Meyering]

James Youngman <address@hidden> wrote:
> On Tue, Dec 06, 2005 at 03:30:54PM +0100, Jim Meyering wrote:
>
>> It does (at least for the 1M-level-deep, 2MB-long name).
>> Nice!  Will this version of find become the default soon?
>
> It will become the default for 4.3.0, which will be released as a
> development release in parallel with 4.2.x.
>
> I'm thinking I should develop some pre-release checks that I run
> before releases but which don't run as part of "make check" even
> though they're included in findutils.  There are a number of test that

There are some like that in coreutils, too.
Look for the RUN_EXPENSIVE_TESTS and RUN_VERY_EXPENSIVE_TESTS
environment variables under coreutils/tests/.

> would be useful but which don't get included right now since they
> require lots of disk space or have other requirements which it may not
> be possible to fulfil on some random machine.  This test is one of
> them.  Would it be feasible to include your test in findutils 4.3.x?
> I would be happy to do any modification/tidying etc.

Sure.  To test it, all I did was run this:

  export TMPDIR=/dev/shm
  z-mkdir-long 1000000
  rm -rf $TMPDIR/zzzzzzz*

Of course, you'll have to arrange to have plenty of RAM *and* i-nodes.

I've used the following little sh/perl scripts, over the years.
It's surprising how few non-trivial tools fail to operate on --
or even run from -- very deep hierarchies.  Even GNU ls and cp haven't
yet been rewritten to work under such extremes.

# Create an N-deep hierarchy of z's, starting in $TMPDIR (or /tmp).
# E.g., `z-mkdir 5' creates /tmp/z/z/z/z/z.
function z-mkdir
{
  local n
  case $# in 1) n=$1;; *) echo "Usage: $FUNCNAME N" 1>&2; return 1;; esac
  local root=${TMPDIR=/tmp}
  test -n "$ROOT" && root=$ROOT
  ( cd $root &&
      time perl -e '$i=0; do {mkdir "z",0700 or die "at depth $i: $!\n";
                    chdir "z"} until (++$i == '$n')' )
}

# Remove a hierarchy of z's, starting in $TMPDIR (or /tmp)
function z-rmdir
{
  case $# in 0) ;; *) echo "Usage: $FUNCNAME" 1>&2; return 1;; esac
  local root=${TMPDIR=/tmp}
  test -n "$ROOT" && root=$ROOT
  ( cd $root &&
      time perl -e 'while (1) {chdir "z" or last};' \
                -e 'while (1) {chdir ".." && rmdir "z" or last}' )
}

# Same as z-mkdir, but with each directory name being a 255-byte
# sequence of z's.
function z-mkdir-long
{
  local n
  case $# in 1) n=$1;; *) echo "Usage: $FUNCNAME N" 1>&2; return 1;; esac
  local root=${TMPDIR=/tmp}
  test -n "$ROOT" && root=$ROOT
  ( cd $root &&
      time perl -e '$i=0; $z="z" x 255;
                    do {mkdir $z,0700
                          or die "at depth $i: $!\n";
                        chdir $z} until (++$i == '$n')' )
}

function z-rmdir-long
{
  case $# in 0) ;; *) echo "Usage: $FUNCNAME" 1>&2; return 1;; esac
  local root=${TMPDIR=/tmp}
  test -n "$ROOT" && root=$ROOT
  ( cd $root &&
      time perl -e '$z= "z" x 255;' \
                -e 'while (1) {chdir $z or last};' \
                -e 'while (1) {chdir ".." && rmdir $z or last}' )
}

# Create a directory with name of the specified length.
function z2-mkdir
{
  local n
  case $# in 1) n=$1;; *) echo "Usage: $FUNCNAME N" 1>&2; return 1;; esac
  local root=${TMPDIR=/tmp}
  test -n "$ROOT" && root=$ROOT
  ( cd $root &&
    perl -e 'my $len='$n'-length "'$root'";$i=100;$d="z"x$i;
                  while ($i+2 < $len) {
                    $len -= $i + 1;
                    mkdir $d,0700 or die "$!\n";
                    chdir $d} $d="z"x($len-1);
                    mkdir $d or die "z2-mkdir: $d: $!\n"' )
}

# Print the depth of a hierarchy of z's, starting in $TMPDIR (or /tmp).
function z-depth
{
  local root=${TMPDIR=/tmp}
  test -n "$ROOT" && root=$ROOT
  ( cd $root &&
      perl -e '$n=0;while (1) {chdir "z" or last; ++$n} print "$n\n"' )
}

# Run a command from the deepest directory in of a hierarchy of z's.
function z-run
{
  case $# in 0) echo "Usage: $FUNCNAME COMMAND [ARGS...]" 1>&2; return 1;; esac
  local root=${TMPDIR=/tmp}
  test -n "$ROOT" && root=$ROOT
  ( cd $root &&
      perl -e 'while (1) {chdir "z" or last}; exec "'"$*"'"' )
}