Re: add some fd tests to head

[Eric Blake]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

According to Eric Blake on 10/6/2006 7:13 AM:
>>> Indeed, here are some more tests, and some (but not all) fixes.  In 
>>> particular, 
>>> this will not completely work until gnulib's closeout module checks for 
>>> write 
>>> errors on stderr, as was proposed a few days ago but not committed yet.
> 

Finally, all the new tests now pass[*][**] on cygwin with this patch.

[*] if you use my proposed patch to gnulib clean-temp, not yet committed
as of this writing
[**] cygwin 1.5.21 and earlier had a bug in popen: if stdout is closed,
popen fails to create a pipe.  This is fixed in cygwin CVS.

2006-10-06  Eric Blake  <address@hidden>

        Replace uses of tmpfile with clean-temp, since tmpfile is
        incompatible with closeout.
        * tests/builtins.at (esyscmd, syscmd): Don't use pipe; it loses
        exit status of m4.
        * tests/others.at (stderr closed): Likewise.
        * ltdl/m4/gnulib-cache.m4: Augment with gnulib-tool --import
        clean-temp.
        * m4/output.c [!HAVE_TMPFILE]: Delete dead code.
        (m4_tmpfile, cleanup_tmpfile): New functions.
        (make_room_for, m4_insert_diversion): Use clean-temp module.
        * configure.ac (AC_CHECK_FUNCS_ONCE): No longer check for
        tmpfile.
        * doc/m4.texinfo (Diversions): Document new use of $TMPDIR.

- --
Life is short - so eat dessert first!

Eric Blake             address@hidden
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2.1 (Cygwin)
Comment: Public key at home.comcast.net/~ericblake/eblake.gpg
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFFJp9e84KuGfSFAYARArYJAJ9EL33i17A+qNvbR1QQrYrUPWF8hACggt8B
C+9J/okhKwvKZFlnlpmbcbg=
=grT/
-----END PGP SIGNATURE-----

? ltdl/m4/allocsa.m4
? ltdl/m4/closeout.m4
? ltdl/m4/double-slash-root.m4
? ltdl/m4/eealloc.m4
? ltdl/m4/fatal-signal.m4
? ltdl/m4/gl_list.m4
? ltdl/m4/mbstate_t.m4
? ltdl/m4/mkdtemp.m4
? ltdl/m4/pathmax.m4
? ltdl/m4/quotearg.m4
? ltdl/m4/sig_atomic_t.m4
? ltdl/m4/signalblocking.m4
? ltdl/m4/tmpdir.m4
Index: configure.ac
===================================================================
RCS file: /sources/m4/m4/configure.ac,v
retrieving revision 1.56
diff -u -p -r1.56 configure.ac
--- configure.ac        26 Sep 2006 13:19:26 -0000      1.56
+++ configure.ac        6 Oct 2006 18:17:01 -0000
@@ -29,7 +29,7 @@ AC_CONFIG_AUX_DIR([ltdl/config])
 AC_CONFIG_MACRO_DIR([ltdl/m4])
 AC_CONFIG_LIBOBJ_DIR([gnu])
 AC_CONFIG_TESTDIR([tests])
-AC_CONFIG_HEADERS([gnu/config.h:config-h.in])
+AC_CONFIG_HEADERS([gnu/config.h:gnu/config-h.in])
 AC_CONFIG_FILES([tests/m4], [chmod +x tests/m4])
 
 
@@ -188,7 +188,7 @@ AC_SUBST([INCLUDE_STDBOOL_H])
 ## --------------------------------- ##
 ## Library functions required by M4. ##
 ## --------------------------------- ##
-AC_CHECK_FUNCS_ONCE([calloc strerror tmpfile])
+AC_CHECK_FUNCS_ONCE([calloc strerror])
 
 AM_WITH_DMALLOC
 
Index: doc/m4.texinfo
===================================================================
RCS file: /sources/m4/m4/doc/m4.texinfo,v
retrieving revision 1.59
diff -u -p -r1.59 m4.texinfo
--- doc/m4.texinfo      6 Oct 2006 15:24:10 -0000       1.59
+++ doc/m4.texinfo      6 Oct 2006 18:17:03 -0000
@@ -3919,6 +3919,8 @@ limit to the overall memory usable by al
 (512K, currently).  When this maximum is about to be exceeded,
 a temporary file is opened to receive the contents of the biggest
 diversion still in memory, freeing this memory for other diversions.
+When creating the temporary file, @code{m4} honors the value of the
+environment variable @env{TMPDIR}, and falls back to @file{/tmp}.
 So, it is theoretically possible that the number of diversions be
 limited by the number of available file descriptors.
 
Index: ltdl/m4/gnulib-cache.m4
===================================================================
RCS file: /sources/m4/m4/ltdl/m4/gnulib-cache.m4,v
retrieving revision 1.13
diff -u -p -r1.13 gnulib-cache.m4
--- ltdl/m4/gnulib-cache.m4     5 Oct 2006 23:07:28 -0000       1.13
+++ ltdl/m4/gnulib-cache.m4     6 Oct 2006 18:17:03 -0000
@@ -15,11 +15,11 @@
 
 
 # Specification in the form of a command-line invocation:
-#   gnulib-tool --import --dir=. --lib=libgnu --source-base=gnu 
--m4-base=ltdl/m4 --doc-base=doc --aux-dir=ltdl/config --libtool 
--macro-prefix=M4 assert binary-io cloexec close-stream closeout dirname error 
exit fdl filenamecat fopen-safer free gendocs gettext gnupload mkstemp obstack 
progname regex regexprops-generic stdbool stdlib-safer strnlen strtol 
unlocked-io verror xalloc xalloc-die xstrndup xstrtol xvasprintf
+#   gnulib-tool --import --dir=. --lib=libgnu --source-base=gnu 
--m4-base=ltdl/m4 --doc-base=doc --aux-dir=ltdl/config --libtool 
--macro-prefix=M4 assert binary-io clean-temp cloexec close-stream closeout 
dirname error exit fdl filenamecat fopen-safer free gendocs gettext gnupload 
mkstemp obstack progname regex regexprops-generic stdbool stdlib-safer strnlen 
strtol unlocked-io verror xalloc xalloc-die xstrndup xstrtol xvasprintf
 
 # Specification in the form of a few gnulib-tool.m4 macro invocations:
 gl_LOCAL_DIR([])
-gl_MODULES([assert binary-io cloexec close-stream closeout dirname error exit 
fdl filenamecat fopen-safer free gendocs gettext gnupload mkstemp obstack 
progname regex regexprops-generic stdbool stdlib-safer strnlen strtol 
unlocked-io verror xalloc xalloc-die xstrndup xstrtol xvasprintf])
+gl_MODULES([assert binary-io clean-temp cloexec close-stream closeout dirname 
error exit fdl filenamecat fopen-safer free gendocs gettext gnupload mkstemp 
obstack progname regex regexprops-generic stdbool stdlib-safer strnlen strtol 
unlocked-io verror xalloc xalloc-die xstrndup xstrtol xvasprintf])
 gl_AVOID([])
 gl_SOURCE_BASE([gnu])
 gl_M4_BASE([ltdl/m4])
Index: m4/output.c
===================================================================
RCS file: /sources/m4/m4/m4/output.c,v
retrieving revision 1.30
diff -u -p -r1.30 output.c
--- m4/output.c 4 Oct 2006 03:57:00 -0000       1.30
+++ m4/output.c 6 Oct 2006 18:17:03 -0000
@@ -28,6 +28,10 @@
 
 #include "m4private.h"
 
+#include "binary-io.h"
+#include "clean-temp.h"
+#include "xvasprintf.h"
+
 /* Define this to see runtime debug output.  Implied by DEBUG.  */
 /*#define DEBUG_OUTPUT */
 
@@ -42,19 +46,14 @@
 /* Size of buffer size to use while copying files.  */
 #define COPY_BUFFER_SIZE (32 * 512)
 
-#ifdef HAVE_TMPFILE
-extern FILE *tmpfile ();
-#endif
-
-/* FIXME - hack until we get clean-temp gnulib module going.  */
-#undef tmpfile
-
 /* Output functions.  Most of the complexity is for handling cpp like
    sync lines.
 
    This code is fairly entangled with the code in input.c, and maybe it
    belongs there?  */
 
+typedef struct temp_dir m4_temp_dir;
+
 /* In a struct diversion, only one of file or buffer be may non-NULL,
    depending on the fact output is diverted to a file or in memory
    buffer.  Further, if buffer is NULL, then pointer is NULL, size and
@@ -85,6 +84,9 @@ static FILE *output_file;     /* current val
 static char *output_cursor;    /* current value of (buffer + used) */
 static int output_unused;      /* current value of (size - used) */
 
+/* Temporary directory holding all spilled diversion files.  */
+static m4_temp_dir *output_temp_dir;
+
 
 
 /* --- OUTPUT INITIALIZATION --- */
@@ -114,27 +116,46 @@ m4_output_exit (void)
   DELETE (diversion_table);
 }
 
-#ifndef HAVE_TMPFILE
-
-/* Implement tmpfile(3) for non-USG systems.  */
+/* Clean up any temporary directory.  Designed for use as an atexit
+   handler.  */
+static void
+cleanup_tmpfile (void)
+{
+  cleanup_temp_dir (output_temp_dir);
+}
 
+/* Create a temporary file open for reading and writing in a secure
+   temp directory.  The file will be automatically closed and deleted
+   on a fatal signal.  When done with the file, close it with
+   close_stream_temp.  Exits on failure, so the return value is always
+   an open file.  */
 static FILE *
-tmpfile (void)
+m4_tmpfile (m4 *context)
 {
-  char buf[32];
-  int fd;
-
-  strcpy (buf, "/tmp/m4XXXXXX");
-  fd = mkstemp (buf);
-  if (fd < 0)
-    return NULL;
+  static unsigned int count;
+  char *name;
+  FILE *file;
 
-  unlink (buf);
-  return fdopen (fd, "w+");
+  if (output_temp_dir == NULL)
+    {
+      errno = 0;
+      output_temp_dir = create_temp_dir ("m4-", NULL, true);
+      if (output_temp_dir == NULL)
+       m4_error (context, EXIT_FAILURE, errno,
+                 _("cannot create temporary file for diversion"));
+      atexit (cleanup_tmpfile);
+    }
+  name = xasprintf ("%s/m4-%d", output_temp_dir->dir_name, count++);
+  register_temp_file (output_temp_dir, name);
+  errno = 0;
+  file = fopen_temp (name, O_BINARY ? "wb+" : "w+");
+  if (file == NULL)
+    m4_error (context, EXIT_FAILURE, errno,
+             _("cannot create temporary file for diversion"));
+  free (name);
+  return file;
 }
 
-#endif /* not HAVE_TMPFILE */
-
 /* Reorganize in-memory diversion buffers so the current diversion can
    accomodate LENGTH more characters without further reorganization.  The
    current diversion buffer is made bigger if possible.  But to make room
@@ -187,10 +208,7 @@ make_room_for (m4 *context, int length)
       /* Create a temporary file, write the in-memory buffer of the
         diversion to this file, then release the buffer.  */
 
-      selected_diversion->file = tmpfile ();
-      if (selected_diversion->file == NULL)
-       m4_error (context, EXIT_FAILURE, errno,
-                 _("cannot create temporary file for diversion"));
+      selected_diversion->file = m4_tmpfile (context);
       if (set_cloexec_flag (fileno (selected_diversion->file), true) != 0)
        m4_error (context, 0, errno,
                  _("cannot protect diversion across forks"));
@@ -538,7 +556,7 @@ m4_insert_diversion (m4 *context, int di
 
   if (diversion->file)
     {
-      fclose (diversion->file);
+      close_stream_temp (diversion->file);
       diversion->file = NULL;
     }
   else if (diversion->buffer)
Index: tests/builtins.at
===================================================================
RCS file: /sources/m4/m4/tests/builtins.at,v
retrieving revision 1.21
diff -u -p -r1.21 builtins.at
--- tests/builtins.at   5 Oct 2006 23:07:28 -0000       1.21
+++ tests/builtins.at   6 Oct 2006 18:17:03 -0000
@@ -152,9 +152,10 @@ world
 divert`'esyscmd(`echo hi >&3')
 hello
 ]])
-AT_CHECK_M4([3>&-], [0], [[hello
+AT_CHECK_M4([3>&-], [0], [stdout], [experr], [in.m4])
+AT_CHECK([sed -ne '/./p' stdout], [0], [[hello
 world
-]], [experr], [in.m4 | sed -ne '/./p'])
+]])
 
 dnl Ensure that esyscmd does not inherit any unnecessary fds from input files.
 AT_DATA([in.m4], [[hello esyscmd(`cat <&3')dnl
@@ -644,9 +645,10 @@ world
 divert`'syscmd(`echo hi >&3')
 hello
 ]])
-AT_CHECK_M4([3>&-], [0], [[hello
+AT_CHECK_M4([3>&-], [0], [stdout], [experr], [in.m4])
+AT_CHECK([sed -ne '/./p' stdout], [0], [[hello
 world
-]], [experr], [in.m4 | sed -ne '/./p'])
+]])
 
 dnl Ensure that syscmd does not inherit any unnecessary fds from input files.
 AT_DATA([in.m4], [[hello syscmd(`cat <&3')dnl
Index: tests/others.at
===================================================================
RCS file: /sources/m4/m4/tests/others.at,v
retrieving revision 1.21
diff -u -p -r1.21 others.at
--- tests/others.at     5 Oct 2006 23:07:28 -0000       1.21
+++ tests/others.at     6 Oct 2006 18:17:03 -0000
@@ -398,11 +398,12 @@ hello`'dnl(world)
 undivert
 goodbye
 ]])
-AT_CHECK_M4([2>&-], [1],
+AT_CHECK_M4([2>&-], [1], [stdout], [], [in.m4])
+AT_CHECK([sed -ne '/./p' stdout], [0],
 [[hello
 and
 goodbye
-]], [], [in.m4 | sed -ne '/./p'])
+]])
 
 dnl command line input file must not collide with closed stderr
 AT_DATA([in.m4], [[syscmd(`cat <&2')sysval