[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[SCM] GNU M4 source repository branch, master, updated. cvs-readonly-27-
|
From: |
Eric Blake |
|
Subject: |
[SCM] GNU M4 source repository branch, master, updated. cvs-readonly-27-gbcb92cf |
|
Date: |
Fri, 07 Dec 2007 14:12:51 +0000 |
This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "GNU M4 source repository".
http://git.sv.gnu.org/gitweb/?p=m4.git;a=commitdiff;h=bcb92cf23ae09a5e3a0c8f5f8d2a991245918ede
The branch, master has been updated
via bcb92cf23ae09a5e3a0c8f5f8d2a991245918ede (commit)
from edff3edfb0438a3047fb7208a34404c3b76513e5 (commit)
Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.
- Log -----------------------------------------------------------------
commit bcb92cf23ae09a5e3a0c8f5f8d2a991245918ede
Author: Eric Blake <address@hidden>
Date: Thu Dec 6 22:14:22 2007 -0700
Stage 5: add notion of quote age.
* m4/m4module.h (m4_get_symbol_value_quote_age): New prototype.
(m4_set_symbol_value_text): Adjust prototype.
(m4_has_syntax): Factor out the unsigned char cast.
* m4/m4private.h (struct m4_syntax_table): Add syntax_age and
quote_age members.
(m4__quote_age, m4__safe_quotes): New accessor macros, no need for
functions at this point.
(struct m4_symbol_value, struct m4_macro_args): Add quote_age
member.
(m4_set_symbol_value_text): Adjust fast accessor.
(m4_get_symbol_value_quote_age): New fast accessor.
* m4/symtab.c (m4_set_symbol_value_text): Add parameter.
(m4_get_symbol_value_quote_age): New function.
(m4_symbol_value_copy): Adjust callers.
* m4/macro.c (expand_token): Add parameter, and track quote age.
(expand_argument, collect_arguments): Track quote age.
(m4_macro_expand_input, process_macro, m4_make_argv_ref)
(m4_macro_expand_input): Update callers.
(m4_arg_text, m4_arg_len, m4_arg_func): Abort on type mismatch.
* m4/input.c: Comment cleanups.
(struct m4_input_block): Reduce size.
(m4__next_token): Report quote age.
(m4_push_builtin, init_builtin_token): Update callers.
* m4/utility.c (skip_space): Adjust callers.
* m4/module.c (install_macro_table): Likewise.
* m4/syntax.c (m4_set_syntax): Initialize and update quote age.
(m4_set_quotes, m4_set_comment): Detect no-op changes, and update
quote age.
(set_quote_age): New helper function.
(check_is_single_quotes, check_is_single_comments): Adjust
callers.
* src/freeze.c (reload_frozen_state): Likewise.
* src/main.c (main): Likewise.
* modules/m4.c (define, pushdef): No need to set macro text.
* tests/builtins.at (changequote, defn): New tests.
* examples/wrapfifo.m4: New file.
* examples/wraplifo.m4: New file.
* Makefile.am (dist_pkgdata_DATA): Distribute new examples.
Signed-off-by: Eric Blake <address@hidden>
-----------------------------------------------------------------------
Summary of changes:
ChangeLog | 42 +++++++++++++
examples/wrapfifo.m4 | 10 +++
examples/wraplifo.m4 | 10 +++
m4/input.c | 157 +++++++++++++++++++++++++-------------------------
m4/m4module.h | 18 +++++-
m4/m4private.h | 53 +++++++++++++----
m4/macro.c | 124 ++++++++++++++++++++++++++--------------
m4/module.c | 3 +-
m4/symtab.c | 15 ++++-
m4/syntax.c | 122 +++++++++++++++++++++++++++++++-------
m4/utility.c | 2 +-
modules/m4.c | 12 +---
src/freeze.c | 2 +-
src/main.c | 2 +-
tests/builtins.at | 71 ++++++++++++++++++++++
15 files changed, 466 insertions(+), 177 deletions(-)
create mode 100644 examples/wrapfifo.m4
create mode 100644 examples/wraplifo.m4
diff --git a/ChangeLog b/ChangeLog
index af39595..247eb37 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,45 @@
+2007-12-07 Eric Blake <address@hidden>
+
+ Stage 5: add notion of quote age.
+ * m4/m4module.h (m4_get_symbol_value_quote_age): New prototype.
+ (m4_set_symbol_value_text): Adjust prototype.
+ (m4_has_syntax): Factor out the unsigned char cast.
+ * m4/m4private.h (struct m4_syntax_table): Add syntax_age and
+ quote_age members.
+ (m4__quote_age, m4__safe_quotes): New accessor macros, no need for
+ functions at this point.
+ (struct m4_symbol_value, struct m4_macro_args): Add quote_age
+ member.
+ (m4_set_symbol_value_text): Adjust fast accessor.
+ (m4_get_symbol_value_quote_age): New fast accessor.
+ * m4/symtab.c (m4_set_symbol_value_text): Add parameter.
+ (m4_get_symbol_value_quote_age): New function.
+ (m4_symbol_value_copy): Adjust callers.
+ * m4/macro.c (expand_token): Add parameter, and track quote age.
+ (expand_argument, collect_arguments): Track quote age.
+ (m4_macro_expand_input, process_macro, m4_make_argv_ref)
+ (m4_macro_expand_input): Update callers.
+ (m4_arg_text, m4_arg_len, m4_arg_func): Abort on type mismatch.
+ * m4/input.c: Comment cleanups.
+ (struct m4_input_block): Reduce size.
+ (m4__next_token): Report quote age.
+ (m4_push_builtin, init_builtin_token): Update callers.
+ * m4/utility.c (skip_space): Adjust callers.
+ * m4/module.c (install_macro_table): Likewise.
+ * m4/syntax.c (m4_set_syntax): Initialize and update quote age.
+ (m4_set_quotes, m4_set_comment): Detect no-op changes, and update
+ quote age.
+ (set_quote_age): New helper function.
+ (check_is_single_quotes, check_is_single_comments): Adjust
+ callers.
+ * src/freeze.c (reload_frozen_state): Likewise.
+ * src/main.c (main): Likewise.
+ * modules/m4.c (define, pushdef): No need to set macro text.
+ * tests/builtins.at (changequote, defn): New tests.
+ * examples/wrapfifo.m4: New file.
+ * examples/wraplifo.m4: New file.
+ * Makefile.am (dist_pkgdata_DATA): Distribute new examples.
+
2007-12-04 Eric Blake <address@hidden>
Fix builds with OpenBSD make.
diff --git a/examples/wrapfifo.m4 b/examples/wrapfifo.m4
new file mode 100644
index 0000000..95ff87a
--- /dev/null
+++ b/examples/wrapfifo.m4
@@ -0,0 +1,10 @@
+dnl Redefine m4wrap to have FIFO semantics.
+define(`_m4wrap_level', `0')dnl
+define(`m4wrap',
+`ifdef(`m4wrap'_m4wrap_level,
+ `define(`m4wrap'_m4wrap_level,
+ defn(`m4wrap'_m4wrap_level)`$1')',
+ `builtin(`m4wrap', `define(`_m4wrap_level',
+ incr(_m4wrap_level))dnl
+m4wrap'_m4wrap_level)dnl
+define(`m4wrap'_m4wrap_level, `$1')')')dnl
diff --git a/examples/wraplifo.m4 b/examples/wraplifo.m4
new file mode 100644
index 0000000..bdbf3fb
--- /dev/null
+++ b/examples/wraplifo.m4
@@ -0,0 +1,10 @@
+dnl Redefine m4wrap to have LIFO semantics.
+define(`_m4wrap_level', `0')dnl
+define(`_m4wrap', defn(`m4wrap'))dnl
+define(`m4wrap',
+`ifdef(`m4wrap'_m4wrap_level,
+ `define(`m4wrap'_m4wrap_level,
+ `$1'defn(`m4wrap'_m4wrap_level))',
+ `_m4wrap(`define(`_m4wrap_level', incr(_m4wrap_level))dnl
+m4wrap'_m4wrap_level)dnl
+define(`m4wrap'_m4wrap_level, `$1')')')dnl
diff --git a/m4/input.c b/m4/input.c
index 08c5f64..e4228a9 100644
--- a/m4/input.c
+++ b/m4/input.c
@@ -28,13 +28,13 @@
/*#define DEBUG_INPUT */
/*
- Unread input can be either files that should be read (eg. included
- files), strings which should be rescanned (eg. macro expansion
- text), or quoted builtin definitions (as returned by the builtin
- "defn"). Unread input is organized in a stack, implemented with an
- obstack. Each input source is described by a "struct
- m4_input_block". The obstack is "input_stack". The top of the
- input stack is "isp".
+ Unread input can be either files that should be read (from the
+ command line or by include/sinclude), strings which should be
+ rescanned (normal macro expansion text), or quoted builtin
+ definitions (as returned by the builtin "defn"). Unread input is
+ organized in a stack, implemented with an obstack. Each input
+ source is described by a "struct m4_input_block". The obstack is
+ "input_stack". The top of the input stack is "isp".
Each input_block has an associated struct input_funcs, which is a
vtable that defines polymorphic functions for peeking, reading,
@@ -72,12 +72,12 @@
manage the coordination between the different push routines.
Normally, input sources behave in LIFO order, resembling a stack.
- But thanks to the defn macro, when collecting the expansion of a
- macro, it is possible that we must intermix multiple input blocks
- in FIFO order. This also applies to the POSIX requirements of
- m4wrap. Therefore, when collecting an expansion, a meta-input
- block is formed which will visit its children in FIFO order,
- without losing data when the obstack is cleared in LIFO order.
+ But thanks to the defn and m4wrap macros, when collecting the
+ expansion of a macro, it is possible that we must intermix multiple
+ input blocks in FIFO order. Therefore, when collecting an
+ expansion, a meta-input block is formed which will visit its
+ children in FIFO order, without losing data when the obstack is
+ cleared in LIFO order.
The current file and line number are stored in the context, for use
by the error handling functions in utility.c. When collecting a
@@ -117,16 +117,20 @@ static bool consume_syntax (m4 *,
m4_obstack *, unsigned int);
static int m4_print_token (const char *, m4__token_type, m4_symbol_value *);
#endif
+/* Vtable of callbacks for each input method. */
struct input_funcs
{
- /* Peek at input, return CHAR_RETRY if none available. */
+ /* Peek at input, return an unsigned char, CHAR_BUILTIN if it is a
+ builtin, or CHAR_RETRY if none available. */
int (*peek_func) (m4_input_block *);
- /* Read input, return CHAR_RETRY if none available. If the flag is
- false, then do not alter the current file or line. */
+ /* Read input, return an unsigned char, CHAR_BUILTIN if it is a
+ builtin, or CHAR_RETRY if none available. If the flag is false,
+ then do not alter the current file or line. */
int (*read_func) (m4_input_block *, m4 *, bool);
- /* Unread a single character, previously read by read_func. */
+ /* Unread a single unsigned character or CHAR_BUILTIN, must be the
+ same character previously read by read_func. */
void (*unget_func) (m4_input_block *, int);
/* Optional function to perform cleanup at end of input. */
@@ -137,46 +141,45 @@ struct input_funcs
void (*print_func) (m4_input_block *, m4 *, m4_obstack *);
};
+/* A block of input to be scanned. */
struct m4_input_block
{
- m4_input_block *prev; /* previous input_block on the input
stack */
- struct input_funcs *funcs; /* functions on this input_block */
- const char *file; /* file where this input is from */
- int line; /* line where this input is from */
+ m4_input_block *prev; /* Previous input_block on the input
stack. */
+ struct input_funcs *funcs; /* Virtual functions of this input_block. */
+ const char *file; /* File where this input is from. */
+ int line; /* Line where this input is from. */
union
{
struct
{
- char *str; /* string value */
- size_t len; /* remaining length */
+ char *str; /* String value. */
+ size_t len; /* Remaining length. */
}
- u_s;
+ u_s; /* See string_funcs. */
struct
{
- FILE *fp; /* input file handle */
- bool end; /* true iff peek returned EOF */
- bool close; /* true if file should be closed on EOF */
- bool advance_line; /* start_of_input_line from next_char () */
+ FILE *fp; /* Input file handle. */
+ bool_bitfield end : 1; /* True iff peek returned EOF. */
+ bool_bitfield close : 1; /* True to close file on pop. */
+ bool_bitfield line_start : 1; /* Saved start_of_input_line state. */
}
- u_f;
+ u_f; /* See file_funcs. */
struct
{
- const m4_builtin *builtin; /* pointer to builtin's function. */
- m4_module *module; /* originating module. */
- int flags; /* flags associated with the builtin. */
- m4_hash *arg_signature; /* argument signature for builtin. */
- unsigned int min_args; /* argv minima for the builtin. */
- unsigned int max_args; /* argv maxima for the builtin. */
- bool read; /* true iff block has been read. */
+ const m4_builtin *builtin; /* Pointer to builtin's function. */
+ m4_module *module; /* Originating module. */
+ bool_bitfield read : 1; /* True iff block has been read. */
+ int flags : 24; /* Flags tied to the builtin. */
+ m4_hash *arg_signature; /* Argument signature for builtin. */
}
- u_b;
+ u_b; /* See builtin_funcs. */
struct
{
- m4_input_block *current; /* pointer to current sub-block. */
- m4_input_block *tail; /* pointer to last sub-block. */
+ m4_input_block *current; /* Pointer to current sub-block. */
+ m4_input_block *tail; /* Pointer to last sub-block. */
}
- u_c;
+ u_c; /* See composite_funcs. */
}
u;
};
@@ -199,7 +202,7 @@ static m4_obstack *wrapup_stack;
static m4_obstack *current_input;
/* Bottom of token_stack, for obstack_free. */
-static char *token_bottom;
+static void *token_bottom;
/* Pointer to top of current_input. */
static m4_input_block *isp;
@@ -295,7 +298,7 @@ file_clean (m4_input_block *me, m4 *context)
}
else if (me->u.u_f.close && fclose (me->u.u_f.fp) == EOF)
m4_error (context, 0, errno, NULL, _("error reading file `%s'"), me->file);
- start_of_input_line = me->u.u_f.advance_line;
+ start_of_input_line = me->u.u_f.line_start;
m4_set_output_line (context, -1);
}
@@ -332,8 +335,7 @@ m4_push_file (m4 *context, FILE *fp, const char *title,
bool close_file)
m4_debug_message (context, M4_DEBUG_TRACE_INPUT,
_("input read from %s"), title);
- i = (m4_input_block *) obstack_alloc (current_input,
- sizeof (m4_input_block));
+ i = (m4_input_block *) obstack_alloc (current_input, sizeof *i);
i->funcs = &file_funcs;
/* Save title on a separate obstack, so that wrapped text can refer
to it even after the file is popped. */
@@ -343,7 +345,7 @@ m4_push_file (m4 *context, FILE *fp, const char *title,
bool close_file)
i->u.u_f.fp = fp;
i->u.u_f.end = false;
i->u.u_f.close = close_file;
- i->u.u_f.advance_line = start_of_input_line;
+ i->u.u_f.line_start = start_of_input_line;
m4_set_output_line (context, -1);
@@ -421,8 +423,7 @@ m4_push_builtin (m4 *context, m4_symbol_value *token)
next = NULL;
}
- i = (m4_input_block *) obstack_alloc (current_input,
- sizeof (m4_input_block));
+ i = (m4_input_block *) obstack_alloc (current_input, sizeof *i);
i->funcs = &builtin_funcs;
i->file = m4_get_current_file (context);
i->line = m4_get_current_line (context);
@@ -430,9 +431,11 @@ m4_push_builtin (m4 *context, m4_symbol_value *token)
i->u.u_b.builtin = m4_get_symbol_value_builtin (token);
i->u.u_b.module = VALUE_MODULE (token);
i->u.u_b.arg_signature = VALUE_ARG_SIGNATURE (token);
- i->u.u_b.min_args = VALUE_MIN_ARGS (token);
- i->u.u_b.max_args = VALUE_MAX_ARGS (token);
i->u.u_b.flags = VALUE_FLAGS (token);
+ /* Check for bitfield truncation. */
+ assert (i->u.u_b.flags == VALUE_FLAGS (token)
+ && i->u.u_b.builtin->min_args == VALUE_MIN_ARGS (token)
+ && i->u.u_b.builtin->max_args == VALUE_MAX_ARGS (token));
i->u.u_b.read = false;
i->prev = isp;
@@ -501,8 +504,7 @@ m4_push_string_init (m4 *context)
while (isp && pop_input (context, false));
/* Reserve the next location on the obstack. */
- next = (m4_input_block *) obstack_alloc (current_input,
- sizeof (m4_input_block));
+ next = (m4_input_block *) obstack_alloc (current_input, sizeof *next);
next->funcs = &string_funcs;
next->file = m4_get_current_file (context);
next->line = m4_get_current_line (context);
@@ -661,8 +663,7 @@ m4_push_wrapup (m4 *context, const char *s)
{
m4_input_block *i;
- i = (m4_input_block *) obstack_alloc (wrapup_stack,
- sizeof (m4_input_block));
+ i = (m4_input_block *) obstack_alloc (wrapup_stack, sizeof *i);
i->prev = wsp;
i->funcs = &string_funcs;
@@ -708,8 +709,8 @@ pop_input (m4 *context, bool cleanup)
}
/* To switch input over to the wrapup stack, main () calls pop_wrapup.
- Since wrapup text can install new wrapup text, pop_wrapup () returns
- false when there is no wrapup text on the stack, and true otherwise. */
+ Since wrapup text can install new wrapup text, pop_wrapup ()
+ returns true if there is more wrapped text to parse. */
bool
m4_pop_wrapup (m4 *context)
{
@@ -736,7 +737,7 @@ m4_pop_wrapup (m4 *context)
(unsigned long int) ++level);
current_input = wrapup_stack;
- wrapup_stack = (m4_obstack *) xmalloc (sizeof (m4_obstack));
+ wrapup_stack = (m4_obstack *) xmalloc (sizeof *wrapup_stack);
obstack_init (wrapup_stack);
isp = wsp;
@@ -760,8 +761,8 @@ init_builtin_token (m4 *context, m4_symbol_value *token)
VALUE_MODULE (token) = block->u.u_b.module;
VALUE_FLAGS (token) = block->u.u_b.flags;
VALUE_ARG_SIGNATURE (token) = block->u.u_b.arg_signature;
- VALUE_MIN_ARGS (token) = block->u.u_b.min_args;
- VALUE_MAX_ARGS (token) = block->u.u_b.max_args;
+ VALUE_MIN_ARGS (token) = block->u.u_b.builtin->min_args;
+ VALUE_MAX_ARGS (token) = block->u.u_b.builtin->max_args;
}
@@ -963,7 +964,7 @@ consume_syntax (m4 *context, m4_obstack *obs, unsigned int
syntax)
}
-/* Inititialize input stacks, and quote/comment characters. */
+/* Inititialize input stacks. */
void
m4_input_init (m4 *context)
{
@@ -971,16 +972,15 @@ m4_input_init (m4 *context)
m4_set_current_file (context, NULL);
m4_set_current_line (context, 0);
- current_input = (m4_obstack *) xmalloc (sizeof (m4_obstack));
+ current_input = (m4_obstack *) xmalloc (sizeof *current_input);
obstack_init (current_input);
- wrapup_stack = (m4_obstack *) xmalloc (sizeof (m4_obstack));
+ wrapup_stack = (m4_obstack *) xmalloc (sizeof *wrapup_stack);
obstack_init (wrapup_stack);
/* Allocate an object in the current chunk, so that obstack_free
will always work even if the first token parsed spills to a new
chunk. */
obstack_init (&token_stack);
- obstack_alloc (&token_stack, 1);
token_bottom = obstack_finish (&token_stack);
isp = NULL;
@@ -990,6 +990,7 @@ m4_input_init (m4 *context)
start_of_input_line = false;
}
+/* Free memory used by the input engine. */
void
m4_input_exit (void)
{
@@ -1000,20 +1001,17 @@ m4_input_exit (void)
}
-/* Parse and return a single token from the input stream. A token can
- be M4_TOKEN_EOF, if the input_stack is empty; it can be
- M4_TOKEN_STRING for a quoted string; M4_TOKEN_WORD for something
- that is a potential macro name; and M4_TOKEN_SIMPLE for any single
- character that is not a part of any of the previous types. If LINE
- is not NULL, set *LINE to the line number where the token starts.
- Report errors (unterminated comments or strings) on behalf of
- CALLER, if non-NULL.
-
- M4__next_token () returns the token type, and passes back a pointer to
- the token data through TOKEN. The token text is collected on the obstack
- token_stack, which never contains more than one token text at a time.
- The storage pointed to by the fields in TOKEN is therefore subject to
- change the next time m4__next_token () is called. */
+/* Parse and return a single token from the input stream, built in
+ TOKEN. See m4__token_type for the valid return types, along with a
+ description of what TOKEN will contain. If LINE is not NULL, set
+ *LINE to the line number where the token starts. Report errors
+ (unterminated comments or strings) on behalf of CALLER, if
+ non-NULL.
+
+ The token text is collected on the obstack token_stack, which never
+ contains more than one token text at a time. The storage pointed
+ to by the fields in TOKEN is therefore subject to change the next
+ time m4__next_token () is called. */
m4__token_type
m4__next_token (m4 *context, m4_symbol_value *token, int *line,
const char *caller)
@@ -1028,9 +1026,11 @@ m4__next_token (m4 *context, m4_symbol_value *token, int
*line,
assert (next == NULL);
if (!line)
line = &dummy;
+ memset (token, '\0', sizeof *token);
do {
obstack_free (&token_stack, token_bottom);
+
/* Must consume an input character, but not until CHAR_BUILTIN is
handled. */
ch = peek_char (context);
@@ -1229,9 +1229,8 @@ m4__next_token (m4 *context, m4_symbol_value *token, int
*line,
len = obstack_object_size (&token_stack);
obstack_1grow (&token_stack, '\0');
- memset (token, '\0', sizeof (m4_symbol_value));
-
- m4_set_symbol_value_text (token, obstack_finish (&token_stack), len);
+ m4_set_symbol_value_text (token, obstack_finish (&token_stack), len,
+ m4__quote_age (M4SYNTAX));
VALUE_MAX_ARGS (token) = -1;
#ifdef DEBUG_INPUT
diff --git a/m4/m4module.h b/m4/m4module.h
index 8f3f590..f70d414 100644
--- a/m4/m4module.h
+++ b/m4/m4module.h
@@ -267,13 +267,18 @@ extern bool m4_is_symbol_value_text
(m4_symbol_value *);
extern bool m4_is_symbol_value_func (m4_symbol_value *);
extern bool m4_is_symbol_value_placeholder (m4_symbol_value *);
extern bool m4_is_symbol_value_void (m4_symbol_value *);
+
extern const char * m4_get_symbol_value_text (m4_symbol_value *);
extern size_t m4_get_symbol_value_len (m4_symbol_value *);
+extern unsigned int m4_get_symbol_value_quote_age (m4_symbol_value *);
+
extern m4_builtin_func *m4_get_symbol_value_func (m4_symbol_value *);
extern const m4_builtin *m4_get_symbol_value_builtin (m4_symbol_value *);
extern const char * m4_get_symbol_value_placeholder (m4_symbol_value *);
+
extern void m4_set_symbol_value_text (m4_symbol_value *,
- const char *, size_t);
+ const char *, size_t,
+ unsigned int);
extern void m4_set_symbol_value_builtin (m4_symbol_value *,
const m4_builtin *);
extern void m4_set_symbol_value_placeholder (m4_symbol_value *,
@@ -301,12 +306,12 @@ extern bool m4_is_arg_text (m4_macro_args
*, unsigned int);
extern bool m4_is_arg_func (m4_macro_args *, unsigned int);
extern const char *m4_arg_text (m4_macro_args *, unsigned int);
extern bool m4_arg_equal (m4_macro_args *, unsigned int,
- unsigned int);
+ unsigned int);
extern bool m4_arg_empty (m4_macro_args *, unsigned int);
extern size_t m4_arg_len (m4_macro_args *, unsigned int);
extern m4_builtin_func *m4_arg_func (m4_macro_args *, unsigned int);
extern m4_macro_args *m4_make_argv_ref (m4_macro_args *, const char *, size_t,
- bool, bool);
+ bool, bool);
/* --- RUNTIME DEBUGGING --- */
@@ -410,7 +415,12 @@ enum {
#define M4_SYNTAX_VALUE (~(M4_SYNTAX_RQUOTE | M4_SYNTAX_ECOMM))
#define m4_syntab(S, C) ((S)->table[(C)])
-#define m4_has_syntax(S, C, T) ((m4_syntab ((S), (C)) & (T)) > 0)
+/* Determine if character C matches any of the bitwise-or'd syntax
+ categories T for the given syntax table S. C can be either an
+ unsigned int (including special values such as CHAR_BUILTIN) or a
+ char which will be interpreted as an unsigned char. */
+#define m4_has_syntax(S, C, T) \
+ ((m4_syntab ((S), sizeof (C) == 1 ? to_uchar (C) : (C)) & (T)) > 0)
extern void m4_set_quotes (m4_syntax_table*, const char*, const char*);
extern void m4_set_comment (m4_syntax_table*, const char*, const char*);
diff --git a/m4/m4private.h b/m4/m4private.h
index 8e23e00..db1f513 100644
--- a/m4/m4private.h
+++ b/m4/m4private.h
@@ -220,6 +220,9 @@ struct m4_symbol_value
{
size_t len; /* Length of string. */
const char * text; /* String contents. */
+ /* Quote age when this string was built, or zeroto force a
+ rescan of the string. Ignored for 0 len. */
+ unsigned int quote_age;
} u_t; /* Valid when type is TEXT, PLACEHOLDER. */
const m4_builtin * builtin;/* Valid when type is FUNC. */
m4_symbol_chain * chain; /* Valid when type is COMP. */
@@ -243,6 +246,10 @@ struct m4_macro_args
bool_bitfield has_ref : 1;
const char *argv0; /* The macro name being expanded. */
size_t argv0_len; /* Length of argv0. */
+ /* The value of quote_age for all tokens, or 0 if quote_age changed
+ during parsing or any token is potentially unsafe and requires a
+ rescan. */
+ unsigned int quote_age;
size_t arraylen; /* True length of allocated elements in array. */
/* Used as a variable-length array, storing information about each
argument. */
@@ -282,6 +289,7 @@ struct m4_macro_args
((V)->type == M4_SYMBOL_PLACEHOLDER)
# define m4_get_symbol_value_text(V) ((V)->u.u_t.text)
# define m4_get_symbol_value_len(V) ((V)->u.u_t.len)
+# define m4_get_symbol_value_quote_age(V) ((V)->u.u_t.quote_age)
# define m4_get_symbol_value_func(V) ((V)->u.builtin->func)
# define m4_get_symbol_value_builtin(V) ((V)->u.builtin)
# define m4_get_symbol_value_placeholder(V) \
@@ -289,8 +297,9 @@ struct m4_macro_args
# define m4_symbol_value_groks_macro(V) (BIT_TEST ((V)->flags,
\
VALUE_MACRO_ARGS_BIT))
-# define m4_set_symbol_value_text(V, T, L) \
- ((V)->type = M4_SYMBOL_TEXT, (V)->u.u_t.text = (T), (V)->u.u_t.len = (L))
+# define m4_set_symbol_value_text(V, T, L, A) \
+ ((V)->type = M4_SYMBOL_TEXT, (V)->u.u_t.text = (T), \
+ (V)->u.u_t.len = (L), (V)->u.u_t.quote_age = (A))
# define m4_set_symbol_value_builtin(V, B) \
((V)->type = M4_SYMBOL_FUNC, (V)->u.builtin = (B))
# define m4_set_symbol_value_placeholder(V, T)
\
@@ -335,14 +344,14 @@ extern void m4__symtab_remove_module_references
(m4_symbol_table*,
/* CHAR_RETRY must be last, because we size the syntax table to hold
all other characters and sentinels. */
-#define CHAR_EOF 256 /* character return on EOF */
-#define CHAR_BUILTIN 257 /* character return for BUILTIN token */
-#define CHAR_RETRY 258 /* character return for end of input block */
+#define CHAR_EOF 256 /* Character return on EOF. */
+#define CHAR_BUILTIN 257 /* Character return for BUILTIN token. */
+#define CHAR_RETRY 258 /* Character return for end of input block. */
-#define DEF_LQUOTE "`"
-#define DEF_RQUOTE "\'"
-#define DEF_BCOMM "#"
-#define DEF_ECOMM "\n"
+#define DEF_LQUOTE "`" /* Default left quote delimiter. */
+#define DEF_RQUOTE "\'" /* Default right quote delimiter. */
+#define DEF_BCOMM "#" /* Default begin comment delimiter. */
+#define DEF_ECOMM "\n" /* Default end comment delimiter. */
typedef struct {
char *string; /* characters of the string */
@@ -362,14 +371,26 @@ struct m4_syntax_table {
/* True iff strlen(lquote) == strlen(rquote) == 1 and lquote is not
interfering with macro names. */
- bool is_single_quotes;
+ bool_bitfield is_single_quotes : 1;
/* True iff strlen(bcomm) == strlen(ecomm) == 1 and bcomm is not
interfering with macros or quotes. */
- bool is_single_comments;
+ bool_bitfield is_single_comments : 1;
/* True iff some character has M4_SYNTAX_ESCAPE. */
- bool is_macro_escaped;
+ bool_bitfield is_macro_escaped : 1;
+
+ /* Track the number of changesyntax calls. This saturates at
+ 0xffff, so the idea is that most users won't be changing the
+ syntax that frequently; perhaps in the future we will cache
+ frequently used syntax schemes by index. */
+ unsigned short syntax_age;
+
+ /* Track the current quote age, determined by all significant
+ changequote, changecom, and changesyntax calls, since any of
+ these can alter the rescan of a prior parameter in a quoted
+ context. */
+ unsigned int quote_age;
};
/* Fast macro versions of syntax table accessor functions,
@@ -385,6 +406,14 @@ struct m4_syntax_table {
# define m4_is_syntax_macro_escaped(S) ((S)->is_macro_escaped)
#endif
+/* Return the current quote age. */
+#define m4__quote_age(S) ((S)->quote_age)
+
+/* Return true if the current quote age guarantees that parsing the
+ current token in the context of a quoted string of the same quote
+ age will give the same parse. */
+#define m4__safe_quotes(S) (((S)->quote_age & 0xffff) != 0)
+
/* --- MACRO MANAGEMENT --- */
diff --git a/m4/macro.c b/m4/macro.c
index 25fc7e7..56d43ac 100644
--- a/m4/macro.c
+++ b/m4/macro.c
@@ -32,8 +32,8 @@
static m4_macro_args *collect_arguments (m4 *, const char *, size_t,
m4_symbol *, m4_obstack *);
static void expand_macro (m4 *, const char *, size_t, m4_symbol *);
-static void expand_token (m4 *, m4_obstack *, m4__token_type,
- m4_symbol_value *, int);
+static bool expand_token (m4 *, m4_obstack *, m4__token_type,
+ m4_symbol_value *, int, bool);
static bool expand_argument (m4 *, m4_obstack *, m4_symbol_value *,
const char *);
static void process_macro (m4 *, m4_symbol_value *, m4_obstack *, int,
@@ -85,26 +85,35 @@ m4_macro_expand_input (m4 *context)
obstack_init (&argc_stack);
obstack_init (&argv_stack);
- m4_set_symbol_value_text (&empty_symbol, "", 0);
+ m4_set_symbol_value_text (&empty_symbol, "", 0, 0);
+ VALUE_MAX_ARGS (&empty_symbol) = -1;
while ((type = m4__next_token (context, &token, &line, NULL))
!= M4_TOKEN_EOF)
- expand_token (context, (m4_obstack *) NULL, type, &token, line);
+ expand_token (context, (m4_obstack *) NULL, type, &token, line, true);
obstack_free (&argc_stack, NULL);
obstack_free (&argv_stack, NULL);
}
-/* Expand one token, according to its type. Potential macro names
- (M4_TOKEN_WORD) are looked up in the symbol table, to see if they have a
- macro definition. If they have, they are expanded as macros, otherwise
- the text are just copied to the output. */
-static void
-expand_token (m4 *context, m4_obstack *obs,
- m4__token_type type, m4_symbol_value *token, int line)
+/* Expand one token onto OBS, according to its type. If OBS is NULL,
+ output the expansion to the current diversion. TYPE determines the
+ contents of TOKEN. Potential macro names (a TYPE of M4_TOKEN_WORD)
+ are looked up in the symbol table, to see if they have a macro
+ definition. If they have, they are expanded as macros, otherwise
+ the text are just copied to the output. LINE determines where
+ TOKEN began. FIRST is true if there is no prior content in the
+ current macro argument. Return true if the result is guranteed to
+ give the same parse on rescan in a quoted context with the same
+ quote age. Returning false is always safe, although it may lead to
+ slower performance. */
+static bool
+expand_token (m4 *context, m4_obstack *obs, m4__token_type type,
+ m4_symbol_value *token, int line, bool first)
{
m4_symbol *symbol;
+ bool result;
const char *text = (m4_is_symbol_value_text (token)
? m4_get_symbol_value_text (token) : NULL);
@@ -112,16 +121,31 @@ expand_token (m4 *context, m4_obstack *obs,
{ /* TOKSW */
case M4_TOKEN_EOF:
case M4_TOKEN_MACDEF:
+ /* Always safe, since there is no text to rescan. */
+ return true;
+
+ case M4_TOKEN_STRING:
+ /* Tokens and comments are safe in isolation (since quote_age
+ detects any change in delimiters). This is also returned for
+ sequences of benign characters, such as digits. But if other
+ text is already present, multi-character delimiters could be
+ formed by concatenation, so use a conservative heuristic. */
+ result = first || m4__safe_quotes (M4SYNTAX);
break;
case M4_TOKEN_OPEN:
case M4_TOKEN_COMMA:
case M4_TOKEN_CLOSE:
- case M4_TOKEN_SIMPLE:
- case M4_TOKEN_STRING:
case M4_TOKEN_SPACE:
- m4_shipout_text (context, obs, text, m4_get_symbol_value_len (token),
- line);
+ /* Conservative heuristic, thanks to multi-character delimiter
+ concatenation. */
+ result = m4__safe_quotes (M4SYNTAX);
+ break;
+
+ case M4_TOKEN_SIMPLE:
+ /* No guarantees here. */
+ assert (m4_get_symbol_value_len (token) == 1);
+ result = false;
break;
case M4_TOKEN_WORD:
@@ -130,7 +154,7 @@ expand_token (m4 *context, m4_obstack *obs,
size_t len = m4_get_symbol_value_len (token);
size_t len2 = len;
- if (m4_has_syntax (M4SYNTAX, to_uchar (*textp), M4_SYNTAX_ESCAPE))
+ if (m4_has_syntax (M4SYNTAX, *textp, M4_SYNTAX_ESCAPE))
{
textp++;
len2--;
@@ -142,16 +166,25 @@ expand_token (m4 *context, m4_obstack *obs,
|| (symbol->value->type == M4_SYMBOL_FUNC
&& BIT_TEST (SYMBOL_FLAGS (symbol), VALUE_BLIND_ARGS_BIT)
&& !m4__next_token_is_open (context)))
- m4_shipout_text (context, obs, text, len, line);
- else
- expand_macro (context, textp, len2, symbol);
+ {
+ m4_shipout_text (context, obs, text, len, line);
+ /* The word just output is unquoted, but we can trust the
+ heuristics of safe_quote. */
+ return m4__safe_quotes (M4SYNTAX);
+ }
+ expand_macro (context, textp, len2, symbol);
+ /* Expanding a macro may create new tokens to scan, and those
+ tokens may generate unsafe text, but we did not append any
+ text now. */
+ return true;
}
- break;
default:
assert (!"INTERNAL ERROR: bad token type in expand_token ()");
abort ();
}
+ m4_shipout_text (context, obs, text, m4_get_symbol_value_len (token), line);
+ return result;
}
@@ -173,6 +206,8 @@ expand_argument (m4 *context, m4_obstack *obs,
m4_symbol_value *argp,
const char *file = m4_get_current_file (context);
int line = m4_get_current_line (context);
size_t len;
+ unsigned int age = m4__quote_age (M4SYNTAX);
+ bool first = true;
argp->type = M4_SYMBOL_VOID;
@@ -201,7 +236,7 @@ expand_argument (m4 *context, m4_obstack *obs,
m4_symbol_value *argp,
len = obstack_object_size (obs);
obstack_1grow (obs, '\0');
VALUE_MODULE (argp) = NULL;
- m4_set_symbol_value_text (argp, obstack_finish (obs), len);
+ m4_set_symbol_value_text (argp, obstack_finish (obs), len, age);
return type == M4_TOKEN_COMMA;
}
/* fallthru */
@@ -211,7 +246,8 @@ expand_argument (m4 *context, m4_obstack *obs,
m4_symbol_value *argp,
paren_level++;
else if (type == M4_TOKEN_CLOSE)
paren_level--;
- expand_token (context, obs, type, &token, line);
+ if (!expand_token (context, obs, type, &token, line, first))
+ age = 0;
break;
case M4_TOKEN_EOF:
@@ -222,7 +258,8 @@ expand_argument (m4 *context, m4_obstack *obs,
m4_symbol_value *argp,
case M4_TOKEN_WORD:
case M4_TOKEN_SPACE:
case M4_TOKEN_STRING:
- expand_token (context, obs, type, &token, line);
+ if (!expand_token (context, obs, type, &token, line, first))
+ age = 0;
break;
case M4_TOKEN_MACDEF:
@@ -233,10 +270,12 @@ expand_argument (m4 *context, m4_obstack *obs,
m4_symbol_value *argp,
break;
default:
- assert (!"INTERNAL ERROR: bad token type in expand_argument ()");
+ assert (!"expand_argument");
abort ();
}
+ if (argp->type != M4_SYMBOL_VOID || obstack_object_size (obs))
+ first = false;
type = m4__next_token (context, &token, NULL, caller);
}
}
@@ -370,6 +409,7 @@ collect_arguments (m4 *context, const char *name, size_t
len,
table, so we don't have to copy it here. */
args.argv0 = (char *) obstack_copy0 (arguments, name, len);
args.argv0_len = len;
+ args.quote_age = m4__quote_age (M4SYNTAX);
args.arraylen = 0;
obstack_grow (&argv_stack, &args, offsetof (m4_macro_args, array));
name = args.argv0;
@@ -391,11 +431,20 @@ collect_arguments (m4 *context, const char *name, size_t
len,
obstack_ptr_grow (&argv_stack, tokenp);
args.arraylen++;
args.argc++;
+ /* Be conservative - any change in quoting while collecting
+ arguments, or any unsafe argument, will require a rescan
+ if $@ is reused. */
+ if (m4_is_symbol_value_text (tokenp)
+ && m4_get_symbol_value_len (tokenp)
+ && m4_get_symbol_value_quote_age (tokenp) != args.quote_age)
+ args.quote_age = 0;
}
while (more_args);
}
argv = (m4_macro_args *) obstack_finish (&argv_stack);
argv->argc = args.argc;
+ if (args.quote_age != m4__quote_age (M4SYNTAX))
+ argv->quote_age = 0;
argv->arraylen = args.arraylen;
return argv;
}
@@ -426,7 +475,7 @@ m4_macro_call (m4 *context, m4_symbol_value *value,
m4_obstack *expansion,
m4_get_symbol_value_placeholder (value));
else
{
- assert (!"INTERNAL ERROR: bad symbol type in m4_macro_call ()");
+ assert (!"m4_macro_call");
abort ();
}
}
@@ -447,7 +496,7 @@ process_macro (m4 *context, m4_symbol_value *value,
m4_obstack *obs,
{
char ch;
- if (!m4_has_syntax (M4SYNTAX, to_uchar (*text), M4_SYNTAX_DOLLAR))
+ if (!m4_has_syntax (M4SYNTAX, *text, M4_SYNTAX_DOLLAR))
{
obstack_1grow (obs, *text);
text++;
@@ -501,7 +550,7 @@ process_macro (m4 *context, m4_symbol_value *value,
m4_obstack *obs,
const char *key;
for (endp = ++text;
- *endp && m4_has_syntax (M4SYNTAX, to_uchar (*endp),
+ *endp && m4_has_syntax (M4SYNTAX, *endp,
(M4_SYNTAX_OTHER | M4_SYNTAX_ALPHA
| M4_SYNTAX_NUM));
++endp)
@@ -768,7 +817,7 @@ m4_is_arg_func (m4_macro_args *argv, unsigned int index)
return m4_is_symbol_value_func (m4_arg_symbol (argv, index));
}
-/* Given ARGV, return the text at argument INDEX, or NULL if the
+/* Given ARGV, return the text at argument INDEX. Abort if the
argument is not text. Index 0 is always text, and indices beyond
argc return the empty string. */
const char *
@@ -781,8 +830,6 @@ m4_arg_text (m4_macro_args *argv, unsigned int index)
if (argv->argc <= index)
return "";
value = m4_arg_symbol (argv, index);
- if (!m4_is_symbol_value_text (value))
- return NULL;
return m4_get_symbol_value_text (value);
}
@@ -815,7 +862,7 @@ m4_arg_empty (m4_macro_args *argv, unsigned int index)
: !argv->argv0_len);
}
-/* Given ARGV, return the length of argument INDEX, or SIZE_MAX if the
+/* Given ARGV, return the length of argument INDEX. Abort if the
argument is not text. Indices beyond argc return 0. */
size_t
m4_arg_len (m4_macro_args *argv, unsigned int index)
@@ -827,25 +874,15 @@ m4_arg_len (m4_macro_args *argv, unsigned int index)
if (argv->argc <= index)
return 0;
value = m4_arg_symbol (argv, index);
- if (!m4_is_symbol_value_text (value))
- return SIZE_MAX;
return m4_get_symbol_value_len (value);
}
/* Given ARGV, return the builtin function referenced by argument
- INDEX, or NULL if it is not a builtin. Index 0, and indices beyond
- argc, return NULL. */
+ INDEX. Abort if it is not a single builtin. */
m4_builtin_func *
m4_arg_func (m4_macro_args *argv, unsigned int index)
{
- m4_symbol_value *value;
-
- if (index == 0 || argv->argc <= index)
- return NULL;
- value = m4_arg_symbol (argv, index);
- if (!m4_is_symbol_value_func (value))
- return NULL;
- return m4_get_symbol_value_func (value);
+ return m4_get_symbol_value_func (m4_arg_symbol (argv, index));
}
/* Create a new argument object using the same obstack as ARGV; thus,
@@ -909,6 +946,7 @@ m4_make_argv_ref (m4_macro_args *argv, const char *argv0,
size_t argv0_len,
new_argv->inuse = false;
new_argv->argv0 = argv0;
new_argv->argv0_len = argv0_len;
+ new_argv->quote_age = argv->quote_age;
return new_argv;
}
diff --git a/m4/module.c b/m4/module.c
index afeece4..901a48f 100644
--- a/m4/module.c
+++ b/m4/module.c
@@ -195,7 +195,8 @@ install_macro_table (m4 *context, m4_module *module)
m4_symbol_value *value = m4_symbol_value_create ();
size_t len = strlen (mp->value);
- m4_set_symbol_value_text (value, xmemdup (mp->value, len + 1), len);
+ m4_set_symbol_value_text (value, xmemdup (mp->value, len + 1),
+ len, 0);
VALUE_MODULE (value) = module;
m4_symbol_pushdef (M4SYMTAB, mp->name, value);
diff --git a/m4/symtab.c b/m4/symtab.c
index 2f83f7b..932a31f 100644
--- a/m4/symtab.c
+++ b/m4/symtab.c
@@ -414,9 +414,10 @@ m4_symbol_value_copy (m4_symbol_value *dest,
m4_symbol_value *src)
if (m4_is_symbol_value_text (src))
{
size_t len = m4_get_symbol_value_len (src);
+ unsigned int age = m4_get_symbol_value_quote_age (src);
m4_set_symbol_value_text (dest,
xmemdup (m4_get_symbol_value_text (src),
- len + 1), len);
+ len + 1), len, age);
}
else if (m4_is_symbol_value_placeholder (src))
m4_set_symbol_value_placeholder (dest,
@@ -662,6 +663,14 @@ m4_get_symbol_value_len (m4_symbol_value *value)
return value->u.u_t.len;
}
+#undef m4_get_symbol_value_quote_age
+unsigned int
+m4_get_symbol_value_quote_age (m4_symbol_value *value)
+{
+ assert (value && value->type == M4_SYMBOL_TEXT);
+ return value->u.u_t.quote_age;
+}
+
#undef m4_get_symbol_value_func
m4_builtin_func *
m4_get_symbol_value_func (m4_symbol_value *value)
@@ -688,7 +697,8 @@ m4_get_symbol_value_placeholder (m4_symbol_value *value)
#undef m4_set_symbol_value_text
void
-m4_set_symbol_value_text (m4_symbol_value *value, const char *text, size_t len)
+m4_set_symbol_value_text (m4_symbol_value *value, const char *text, size_t len,
+ unsigned int quote_age)
{
assert (value && text);
/* TODO - this assertion enforces NUL-terminated text with no
@@ -701,6 +711,7 @@ m4_set_symbol_value_text (m4_symbol_value *value, const
char *text, size_t len)
value->type = M4_SYMBOL_TEXT;
value->u.u_t.text = text;
value->u.u_t.len = len;
+ value->u.u_t.quote_age = quote_age;
}
#undef m4_set_symbol_value_builtin
diff --git a/m4/syntax.c b/m4/syntax.c
index 9f2e122..0bce3c0 100644
--- a/m4/syntax.c
+++ b/m4/syntax.c
@@ -102,11 +102,12 @@
M4_SYNTAX_RQUOTE and M4_SYNTAX_ECOMM do not start tokens. */
-static bool check_is_single_quotes (m4_syntax_table *);
-static bool check_is_single_comments (m4_syntax_table *);
-static bool check_is_macro_escaped (m4_syntax_table *);
-static int add_syntax_attribute (m4_syntax_table *, int, int);
-static int remove_syntax_attribute (m4_syntax_table *, int, int);
+static bool check_is_single_quotes (m4_syntax_table *);
+static bool check_is_single_comments (m4_syntax_table *);
+static bool check_is_macro_escaped (m4_syntax_table *);
+static int add_syntax_attribute (m4_syntax_table *, int, int);
+static int remove_syntax_attribute (m4_syntax_table *, int, int);
+static void set_quote_age (m4_syntax_table *, bool, bool);
m4_syntax_table *
m4_syntax_create (void)
@@ -392,6 +393,7 @@ m4_set_syntax (m4_syntax_table *syntax, char key, char
action,
syntax->is_single_quotes = true;
syntax->is_single_comments = true;
syntax->is_macro_escaped = false;
+ set_quote_age (syntax, true, false);
return 0;
}
@@ -417,6 +419,7 @@ m4_set_syntax (m4_syntax_table *syntax, char key, char
action,
default:
assert (false);
}
+ set_quote_age (syntax, false, true);
return code;
}
@@ -431,10 +434,8 @@ check_is_single_quotes (m4_syntax_table *syntax)
return false;
assert (syntax->lquote.length == 1 && syntax->rquote.length == 1);
- if (m4_has_syntax (syntax, to_uchar (*syntax->lquote.string),
- M4_SYNTAX_LQUOTE)
- && m4_has_syntax (syntax, to_uchar (*syntax->rquote.string),
- M4_SYNTAX_RQUOTE))
+ if (m4_has_syntax (syntax, *syntax->lquote.string, M4_SYNTAX_LQUOTE)
+ && m4_has_syntax (syntax, *syntax->rquote.string, M4_SYNTAX_RQUOTE))
return true;
/* The most recent action invalidated our current lquote/rquote. If
@@ -486,10 +487,8 @@ check_is_single_comments (m4_syntax_table *syntax)
return false;
assert (syntax->bcomm.length == 1 && syntax->ecomm.length == 1);
- if (m4_has_syntax (syntax, to_uchar (*syntax->bcomm.string),
- M4_SYNTAX_BCOMM)
- && m4_has_syntax (syntax, to_uchar (*syntax->ecomm.string),
- M4_SYNTAX_ECOMM))
+ if (m4_has_syntax (syntax, *syntax->bcomm.string, M4_SYNTAX_BCOMM)
+ && m4_has_syntax (syntax, *syntax->ecomm.string, M4_SYNTAX_ECOMM))
return true;
/* The most recent action invalidated our current bcomm/ecomm. If
@@ -558,9 +557,6 @@ m4_set_quotes (m4_syntax_table *syntax, const char *lq,
const char *rq)
assert (syntax);
- free (syntax->lquote.string);
- free (syntax->rquote.string);
-
/* POSIX states that with 0 arguments, the default quotes are used.
POSIX XCU ERN 112 states that behavior is implementation-defined
if there was only one argument, or if there is an empty string in
@@ -576,6 +572,12 @@ m4_set_quotes (m4_syntax_table *syntax, const char *lq,
const char *rq)
else if (!rq || (*lq && !*rq))
rq = DEF_RQUOTE;
+ if (strcmp (syntax->lquote.string, lq) == 0
+ && strcmp (syntax->rquote.string, rq) == 0)
+ return;
+
+ free (syntax->lquote.string);
+ free (syntax->rquote.string);
syntax->lquote.string = xstrdup (lq);
syntax->lquote.length = strlen (syntax->lquote.string);
syntax->rquote.string = xstrdup (rq);
@@ -587,7 +589,7 @@ m4_set_quotes (m4_syntax_table *syntax, const char *lq,
const char *rq)
syntax->is_single_quotes
= (syntax->lquote.length == 1 && syntax->rquote.length == 1
- && !m4_has_syntax (syntax, to_uchar (*syntax->lquote.string),
+ && !m4_has_syntax (syntax, *syntax->lquote.string,
(M4_SYNTAX_IGNORE | M4_SYNTAX_ESCAPE
| M4_SYNTAX_ALPHA | M4_SYNTAX_NUM)));
@@ -608,9 +610,9 @@ m4_set_quotes (m4_syntax_table *syntax, const char *lq,
const char *rq)
add_syntax_attribute (syntax, to_uchar (syntax->rquote.string[0]),
M4_SYNTAX_RQUOTE);
}
-
if (syntax->is_macro_escaped)
check_is_macro_escaped (syntax);
+ set_quote_age (syntax, false, false);
}
void
@@ -620,9 +622,6 @@ m4_set_comment (m4_syntax_table *syntax, const char *bc,
const char *ec)
assert (syntax);
- free (syntax->bcomm.string);
- free (syntax->ecomm.string);
-
/* POSIX requires no arguments to disable comments, and that one
argument use newline as the close-comment. POSIX XCU ERN 131
states that empty arguments invoke implementation-defined
@@ -635,6 +634,12 @@ m4_set_comment (m4_syntax_table *syntax, const char *bc,
const char *ec)
else if (!ec || (*bc && !*ec))
ec = DEF_ECOMM;
+ if (strcmp (syntax->bcomm.string, bc) == 0
+ && strcmp (syntax->ecomm.string, ec) == 0)
+ return;
+
+ free (syntax->bcomm.string);
+ free (syntax->ecomm.string);
syntax->bcomm.string = xstrdup (bc);
syntax->bcomm.length = strlen (syntax->bcomm.string);
syntax->ecomm.string = xstrdup (ec);
@@ -646,7 +651,7 @@ m4_set_comment (m4_syntax_table *syntax, const char *bc,
const char *ec)
syntax->is_single_comments
= (syntax->bcomm.length == 1 && syntax->ecomm.length == 1
- && !m4_has_syntax (syntax, to_uchar (*syntax->bcomm.string),
+ && !m4_has_syntax (syntax, *syntax->bcomm.string,
(M4_SYNTAX_IGNORE | M4_SYNTAX_ESCAPE
| M4_SYNTAX_ALPHA | M4_SYNTAX_NUM
| M4_SYNTAX_LQUOTE)));
@@ -667,11 +672,82 @@ m4_set_comment (m4_syntax_table *syntax, const char *bc,
const char *ec)
add_syntax_attribute (syntax, to_uchar (syntax->ecomm.string[0]),
M4_SYNTAX_ECOMM);
}
-
if (syntax->is_macro_escaped)
check_is_macro_escaped (syntax);
+ set_quote_age (syntax, false, false);
}
+/* Call this when changing anything that might impact the quote age,
+ so that m4_quote_age and m4_safe_quotes will reflect the change.
+ If RESET, changesyntax was reset to its default stage; if CHANGE,
+ arbitrary syntax has changed; otherwise, just quotes or comment
+ delimiters have changed. */
+static void
+set_quote_age (m4_syntax_table *syntax, bool reset, bool change)
+{
+ /* Multi-character quotes are inherently unsafe, since concatenation
+ of individual characters can result in a quote delimiter,
+ consider:
+
+ define(echo,``$1'')define(a,A)changequote(<[,]>)echo(<[]]><[>a]>)
+ => A]> (not ]>a)
+
+ Also, unquoted close delimiters are unsafe, consider:
+
+ define(echo,``$1'')define(a,A)echo(`a''`a')
+ => aA' (not a'a)
+
+ Duplicated start and end quote delimiters, as well as comment
+ delimiters that overlap with quote delimiters or active characters,
+ also present a problem, consider:
+
+ define(echo,$*)echo(a,a,a`'define(a,A)changecom(`,',`,'))
+ => A,a,A (not A,A,A)
+
+ The impact of arbitrary changesyntax is difficult to characterize.
+ So if things are in their default state, we use 0 for the upper 16
+ bits of quote_age; otherwise we increment syntax_age for each
+ changesyntax, but saturate it at 0xffff rather than wrapping
+ around. Perhaps a cache of other frequently used states is
+ warranted, if changesyntax becomes more popular
+
+ Rather than check every token for an unquoted delimiter, we merely
+ encode current_quote_age to 0 when things are unsafe, and non-zero
+ when safe (namely, the syntax_age in the upper 16 bits, coupled
+ with the 16-bit value composed of the single-character start and
+ end quote delimiters). There may be other situations which are
+ safe even when this algorithm sets the quote_age to zero, but at
+ least a quote_age of zero always produces correct results (although
+ it may take more time in doing so). */
+
+ unsigned short local_syntax_age;
+ if (reset)
+ local_syntax_age = 0;
+ else if (change && syntax->syntax_age < 0xffff)
+ local_syntax_age = ++syntax->syntax_age;
+ else
+ local_syntax_age = syntax->syntax_age;
+ if (local_syntax_age < 0xffff && syntax->is_single_quotes
+ && !m4_has_syntax (syntax, *syntax->lquote.string,
+ (M4_SYNTAX_ALPHA | M4_SYNTAX_NUM | M4_SYNTAX_OPEN
+ | M4_SYNTAX_COMMA | M4_SYNTAX_CLOSE
+ | M4_SYNTAX_SPACE))
+ && !m4_has_syntax (syntax, *syntax->rquote.string,
+ (M4_SYNTAX_ALPHA | M4_SYNTAX_NUM | M4_SYNTAX_OPEN
+ | M4_SYNTAX_COMMA | M4_SYNTAX_CLOSE
+ | M4_SYNTAX_SPACE))
+ && *syntax->lquote.string != *syntax->rquote.string
+ && *syntax->bcomm.string != *syntax->lquote.string
+ && !m4_has_syntax (syntax, *syntax->bcomm.string,
+ M4_SYNTAX_OPEN | M4_SYNTAX_COMMA | M4_SYNTAX_CLOSE))
+ {
+ syntax->quote_age = ((local_syntax_age << 16)
+ | ((*syntax->lquote.string & 0xff) << 8)
+ | (*syntax->rquote.string & 0xff));
+ }
+ else
+ syntax->quote_age = 0;
+}
/* Define these functions at the end, so that calls in the file use the
diff --git a/m4/utility.c b/m4/utility.c
index 72205a8..b367717 100644
--- a/m4/utility.c
+++ b/m4/utility.c
@@ -62,7 +62,7 @@ m4_bad_argc (m4 *context, int argc, const char *caller,
unsigned int min,
static const char *
skip_space (m4 *context, const char *arg)
{
- while (m4_has_syntax (M4SYNTAX, to_uchar (*arg), M4_SYNTAX_SPACE))
+ while (m4_has_syntax (M4SYNTAX, *arg, M4_SYNTAX_SPACE))
arg++;
return arg;
}
diff --git a/modules/m4.c b/modules/m4.c
index f9d65ed..ca5cf45 100644
--- a/modules/m4.c
+++ b/modules/m4.c
@@ -165,11 +165,7 @@ M4BUILTIN_HANDLER (define)
{
m4_symbol_value *value = m4_symbol_value_create ();
- if (argc == 2)
- m4_set_symbol_value_text (value, xstrdup (""), 0);
- else
- m4_symbol_value_copy (value, m4_arg_symbol (argv, 2));
-
+ m4_symbol_value_copy (value, m4_arg_symbol (argv, 2));
m4_symbol_define (M4SYMTAB, M4ARG (1), value);
}
else
@@ -197,11 +193,7 @@ M4BUILTIN_HANDLER (pushdef)
{
m4_symbol_value *value = m4_symbol_value_create ();
- if (argc == 2)
- m4_set_symbol_value_text (value, xstrdup (""), 0);
- else
- m4_symbol_value_copy (value, m4_arg_symbol (argv, 2));
-
+ m4_symbol_value_copy (value, m4_arg_symbol (argv, 2));
m4_symbol_pushdef (M4SYMTAB, M4ARG (1), value);
}
else
diff --git a/src/freeze.c b/src/freeze.c
index 8430dda..cdadf60 100644
--- a/src/freeze.c
+++ b/src/freeze.c
@@ -757,7 +757,7 @@ ill-formed frozen file, version 2 directive `%c'
encountered"), 'T');
m4_set_symbol_value_text (token, xmemdup (string[1],
number[1] + 1),
- number[1]);
+ number[1], 0);
VALUE_MODULE (token) = module;
VALUE_MAX_ARGS (token) = -1;
diff --git a/src/main.c b/src/main.c
index ef9cb7b..7c35e64 100644
--- a/src/main.c
+++ b/src/main.c
@@ -676,7 +676,7 @@ main (int argc, char *const *argv, char *const *envp)
}
m4_set_symbol_value_text (value, xstrdup (macro_value
? macro_value : ""),
- len);
+ len, 0);
if (defn->code == 'D')
m4_symbol_define (M4SYMTAB, macro_name, value);
diff --git a/tests/builtins.at b/tests/builtins.at
index eeaf0d3..91586b8 100644
--- a/tests/builtins.at
+++ b/tests/builtins.at
@@ -88,6 +88,26 @@ AT_CHECK_M4([in], [0], [expout])
AT_CLEANUP
+## ----------- ##
+## changequote ##
+## ----------- ##
+
+AT_SETUP([changequote])
+
+AT_DATA([in.m4],
+[[define(`aaaaaaaaaaaaaaaaaaaa', `A')define(`q', `"$@"')
+changequote(`"', `"')
+q(q("aaaaaaaaaaaaaaaaaaaa", "a"))
+]])
+
+AT_CHECK_M4([in.m4], [0], [[
+
+A,a
+]])
+
+AT_CLEANUP
+
+
## ----- ##
## debug ##
## ----- ##
@@ -159,6 +179,57 @@ AT_CLEANUP
+## ---- ##
+## defn ##
+## ---- ##
+
+AT_SETUP([defn])
+
+dnl This test is a reminder that defn needs to be fixed to handle
+dnl concatenation of builtin tokens with text, and user macros need
+dnl to handle builtin tokens without flattening.
+AT_XFAIL_IF([:])
+
+AT_DATA([[in.m4]],
+[[define(`e', `$@')define(`q', ``$@'')define(`u', `$*')
+define(`cmp', `ifelse($1, $2, `yes', `no')')define(`d', defn(`defn'))
+cmp(`defn(`defn')', `defn(`d')')
+cmp(`defn(`defn')', ``<defn>'')
+cmp(`q(defn(`defn'))', `q(defn(`d'))')
+cmp(`q(defn(`defn'))', `q(`<defn>')')
+cmp(`q(defn(`defn'))', ``'')
+cmp(`q(`1', `2', defn(`defn'))', `q(`1', `2', defn(`d'))')
+cmp(`q(`1', `2', defn(`defn'))', `q(`1', `2', `<defn>')')
+cmp(`q(`1', `2', defn(`defn'))', ```1',`2',<defn>'')
+cmp(`q(`1', `2', defn(`defn'))', ```1',`2',`''')
+define(`cat', `$1`'ifelse(address@hidden:@', `1', `', `$0(shift($@))')')
+cat(`define(`foo',', defn(`divnum'), `)foo')
+cat(e(`define(`bar',', defn(`divnum'), `)bar'))
+m4wrap(`u('q(`cat(`define(`baz','', defn(`divnum'), ``)baz')')`)
+')
+]])
+
+AT_CHECK_M4([in.m4], [0], [[
+
+yes
+no
+yes
+no
+no
+yes
+no
+no
+no
+
+0
+0
+
+0
+]])
+
+AT_CLEANUP
+
+
## ------ ##
## divert ##
## ------ ##
hooks/post-receive
--
GNU M4 source repository
| [Prev in Thread] |
Current Thread |
[Next in Thread] |
- [SCM] GNU M4 source repository branch, master, updated. cvs-readonly-27-gbcb92cf,
Eric Blake <=