Re: defn() with multiple arguments

[Eric Blake]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

According to Eric Blake on 8/8/2007 10:31 PM:
> According to Schwarz, Konrad on 8/6/2007 1:54 AM:
>> However, I think the following example motivates this feature, should
>> you in fact have need of motivation.
> 
> But since POSIX requires it, and since it can, indeed, be used to
> concatenate macros that would otherwise contain unbalanced quotes if
> expanded in isolation, I think it is worth adding to M4 1.4.11.  Patch to
> follow soon.

Here's what I'm applying to the branch; the builtin.c portion should apply
relatively cleanly to the 1.4.10 tarball as well if you don't want to pull
from CVS or wait for 1.4.11.  CVS head already has this feature, but I
will be applying a followup patch to head that makes the documentation
more closely match what I just added to the branch.

2007-08-09  Eric Blake  <address@hidden>

        POSIX requires defn(`a',`b') to concatenate definitions.
        * src/builtin.c (m4_defn): Allow multiple arguments, but warn if
        trying to mix a builtin with anything else.
        * doc/m4.texinfo (Defn): Document a use for this POSIX
        requirement.
        (Incompatibilities): Update to match current status.
        * NEWS: Document this change.
        * THANKS: Update.
        Reported by Konrad Schwarz.

- --
Don't work too hard, make some time for fun as well!

Eric Blake             address@hidden
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Cygwin)
Comment: Public key at home.comcast.net/~ericblake/eblake.gpg
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFGuxjj84KuGfSFAYARAumWAKDCrhRzVL86gYi8N2EI0ZDTtjbGawCfacKC
0cJFR3Nb/pnBpdLTUjdDx04=
=9MHv
-----END PGP SIGNATURE-----

Index: NEWS
===================================================================
RCS file: /sources/m4/m4/NEWS,v
retrieving revision 1.1.1.1.2.110
diff -u -p -r1.1.1.1.2.110 NEWS
--- NEWS        22 Jul 2007 04:40:15 -0000      1.1.1.1.2.110
+++ NEWS        9 Aug 2007 13:31:12 -0000
@@ -7,6 +7,10 @@ Version 1.4.11 - ?? ??? 2007, by ????  (
 * Fix regression introduced in 1.4.9b in the `divert' builtin when more
   than 512 kibibytes are saved in diversions on platforms like NetBSD
   where fopen(name,"a+") seeks to the end of the file.
+* Enhance the `defn' builtin to support concatenation of multiple text
+  arguments, as required by POSIX.  However, at this time, it is not
+  possible to concatenate a builtin macro with anything else.
+* A number of portability improvements inherited from gnulib.
 
 Version 1.4.10 - 09 Jul 2007, by Eric Blake  (CVS version 1.4.9c)
 
Index: doc/m4.texinfo
===================================================================
RCS file: /sources/m4/m4/doc/m4.texinfo,v
retrieving revision 1.1.1.1.2.132
diff -u -p -r1.1.1.1.2.132 m4.texinfo
--- doc/m4.texinfo      15 Jul 2007 04:11:10 -0000      1.1.1.1.2.132
+++ doc/m4.texinfo      9 Aug 2007 13:31:13 -0000
@@ -2023,16 +2023,20 @@ case, @code{undefine} does nothing.
 It is possible to rename an already defined macro.  To do this, you need
 the builtin @code{defn}:
 
address@hidden Builtin defn (@var{name})
-Expands to the @emph{quoted definition} of @var{name}.  If the
-argument is not a defined macro, the expansion is void.
address@hidden Builtin defn (@address@hidden)
+Expands to the @emph{quoted definition} of each @var{name}.  If an
+argument is not a defined macro, the expansion for that argument is
+empty.
 
 If @var{name} is a user-defined macro, the quoted definition is simply
-the quoted expansion text.  If, instead, @var{name} is a builtin, the
+the quoted expansion text.  If, instead, there is only one @var{name}
+and it is a builtin, the
 expansion is a special token, which points to the builtin's internal
 definition.  This token is only meaningful as the second argument to
 @code{define} (and @code{pushdef}), and is silently converted to an
-empty string in most other contexts.
+empty string in most other contexts.  Combining a builtin with anything
+else is not supported; a warning is issued and the builtin is omitted
+from the final expansion.
 
 The macro @code{defn} is recognized only with parameters.
 @end deffn
@@ -2103,6 +2107,34 @@ echo(foo)
 @result{}AA'
 @end example
 
+On the other hand, it is possible to exploit the fact that @code{defn}
+can concatenate multiple macros prior to the rescanning phase, in order
+to join the definitions of macros that, in isolation, have unbalanced
+quotes.  In the example below, note how the use of @code{defn} on
address@hidden in isolation opens a string, which is not closed until the next
+line; but used on @code{l} and @code{r} together results in nested
+quoting.  Also note that @code{defn} with multiple arguments can only
+join text macros, not builtins, although a future version of
address@hidden M4 may lift this restriction.
+
address@hidden
+define(`l', `<[>')define(`r', `<]>')
address@hidden
+changequote(`[', `]')
address@hidden
+defn([l])defn([r])
+])
address@hidden<[>]defn([r])
address@hidden)
+defn([l], [r])
address@hidden<[>][<]>
+define([a], [A])
address@hidden
+defn([a], [defn], [a])
address@hidden:stdin:7: Warning: cannot concatenate builtin `defn'
address@hidden
address@hidden example
+
 @cindex builtins, special tokens
 @cindex tokens, builtin macro
 Using @code{defn} to generate special tokens for builtin macros outside
@@ -6167,18 +6199,21 @@ future release of @acronym{GNU} @code{m4
 
 @itemize @bullet
 @item
-System V @code{m4} supports multiple arguments to @code{defn}, and
address@hidden requires it.  This is not yet implemented in @acronym{GNU}
address@hidden  Unfortunately, this means it is not possible to mix builtins
-and other text into a single macro; a helper macro is required.
address@hidden requires support for multiple arguments to @code{defn},
+without any clarification on how @code{defn} behaves when one of the
+multiple arguments names a builtin.  System V @code{m4} and some other
+implementations allow mixing builtins and text macros into a single
+macro.  @acronym{GNU} @code{m4} only supports joining multiple text
+arguments, although a future implementation may lift this restriction to
+behave more like System V.  The only portable way to join text macros
+with builtins is via helper macros and implicit concatenation of macro
+results.
 
 @item
 @acronym{POSIX} requires an application to exit with non-zero status if
 it wrote an error message to stderr.  This has not yet been consistently
 implemented for the various builtins that are required to issue an error
-(such as @code{include} (@pxref{Include}) when a file is unreadable,
address@hidden (@pxref{Eval}) when an argument cannot be parsed, or using
address@hidden (@pxref{M4exit}) with a non-numeric argument).
+(such as @code{eval} (@pxref{Eval}) when an argument cannot be parsed).
 
 @item
 Some traditional implementations only allow reading standard input
@@ -6234,12 +6269,12 @@ definition on the stack, as if doing @co
 @code{pushdef(`f',`1')}.  @acronym{POSIX} allows either behavior.
 
 @item
address@hidden requires @code{syscmd} (@pxref{Syscmd}) to evaluate
-command output for macro expansion, but this appears to be a mistake
-in @acronym{POSIX} since traditional implementations did not do this.
address@hidden @code{m4} follows traditional behavior in @code{syscmd}, and
-provides the extension @code{esyscmd} that provides the @acronym{POSIX}
-semantics.
address@hidden 2001 requires @code{syscmd} (@pxref{Syscmd}) to evaluate
+command output for macro expansion, but this was a mistake that is
+anticipated to be corrected in the next version of @acronym{POSIX}.
address@hidden @code{m4} follows traditional behavior in @code{syscmd}
+where output is not rescanned, and provides the extension @code{esyscmd}
+that does scan the output.
 
 @item
 At one point, @acronym{POSIX} required @code{changequote(@var{arg})}
@@ -6270,8 +6305,8 @@ Most implementations of @code{m4} give m
 comments when parsing, meaning that if the start delimiter given to
 @code{changecom} (@pxref{Changecom}) starts with a macro name, comments
 are effectively disabled.  @acronym{POSIX} does not specify what the
-precedence is, so the @acronym{GNU} @code{m4} parser recognizes
-comments, then macros, then quoted strings.
+precedence is, so this version of @acronym{GNU} @code{m4} parser
+recognizes comments, then macros, then quoted strings.
 
 @item
 Traditional implementations allow argument collection, but not string
@@ -6315,9 +6350,11 @@ requires @samp{=} to cause an error, but
 implementations allowed it as an alias for @samp{==}.
 
 @item
address@hidden requires @code{translit} (@pxref{Translit}) to treat
-each character of the second and third arguments literally, but @acronym{GNU}
address@hidden treats @samp{-} as a range operator.
address@hidden 2001 requires @code{translit} (@pxref{Translit}) to
+treat each character of the second and third arguments literally.
+However, it is anticipated that the next version of @acronym{POSIX} will
+allow the @acronym{GNU} @code{m4} behavior of treating @samp{-} as a
+range operator.
 
 @item
 @acronym{POSIX} requires @code{m4} to honor the locale environment
Index: src/builtin.c
===================================================================
RCS file: /sources/m4/m4/src/Attic/builtin.c,v
retrieving revision 1.1.1.1.2.61
diff -u -p -r1.1.1.1.2.61 builtin.c
--- src/builtin.c       4 Aug 2007 20:40:11 -0000       1.1.1.1.2.61
+++ src/builtin.c       9 Aug 2007 13:31:14 -0000
@@ -890,38 +890,43 @@ m4_defn (struct obstack *obs, int argc, 
 {
   symbol *s;
   builtin_func *b;
+  int i;
 
-  if (bad_argc (argv[0], argc, 2, 2))
-    return;
-
-  s = lookup_symbol (ARG (1), SYMBOL_LOOKUP);
-  if (s == NULL)
+  if (bad_argc (argv[0], argc, 2, -1))
     return;
 
-  switch (SYMBOL_TYPE (s))
+  for (i = 1; i < argc; i++)
     {
-    case TOKEN_TEXT:
-      obstack_grow (obs, lquote.string, lquote.length);
-      obstack_grow (obs, SYMBOL_TEXT (s), strlen (SYMBOL_TEXT (s)));
-      obstack_grow (obs, rquote.string, rquote.length);
-      break;
+      s = lookup_symbol (ARG (i), SYMBOL_LOOKUP);
+      if (s == NULL)
+       continue;
 
-    case TOKEN_FUNC:
-      b = SYMBOL_FUNC (s);
-      if (b == m4_placeholder)
-       M4ERROR ((warning_status, 0, "\
-builtin `%s' requested by frozen file is not supported", ARG (1)));
-      else
-       push_macro (b);
-      break;
+      switch (SYMBOL_TYPE (s))
+       {
+       case TOKEN_TEXT:
+         obstack_grow (obs, lquote.string, lquote.length);
+         obstack_grow (obs, SYMBOL_TEXT (s), strlen (SYMBOL_TEXT (s)));
+         obstack_grow (obs, rquote.string, rquote.length);
+         break;
 
-    case TOKEN_VOID:
-      break;
+       case TOKEN_FUNC:
+         b = SYMBOL_FUNC (s);
+         if (b == m4_placeholder)
+           M4ERROR ((warning_status, 0, "\
+builtin `%s' requested by frozen file is not supported", ARG (i)));
+         else if (argc != 2)
+           M4ERROR ((warning_status, 0,
+                     "Warning: cannot concatenate builtin `%s'",
+                     ARG (i)));
+         else
+           push_macro (b);
+         break;
 
-    default:
-      M4ERROR ((warning_status, 0,
-               "INTERNAL ERROR: bad symbol type in m4_defn ()"));
-      abort ();
+       default:
+         M4ERROR ((warning_status, 0,
+                   "INTERNAL ERROR: bad symbol type in m4_defn ()"));
+         abort ();
+       }
     }
 }
 
@@ -1898,16 +1903,16 @@ Warning: \\0 will disappear, use \\& ins
        case '7': case '8': case '9':
          ch -= '0';
          if (regs->num_regs - 1 <= ch)
-           M4ERROR ((warning_status, 0, "\
-Warning: sub-expression %d not present", ch));
+           M4ERROR ((warning_status, 0,
+                     "Warning: sub-expression %d not present", ch));
          else if (regs->end[ch] > 0)
            obstack_grow (obs, victim + regs->start[ch],
                          regs->end[ch] - regs->start[ch]);
          break;
 
        case '\0':
-         M4ERROR ((warning_status, 0, "\
-Warning: trailing \\ ignored in replacement"));
+         M4ERROR ((warning_status, 0,
+                   "Warning: trailing \\ ignored in replacement"));
          return;
 
        default: