[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[SCM] GNU M4 source repository branch, master, updated. cvs-readonly-176
|
From: |
Eric Blake |
|
Subject: |
[SCM] GNU M4 source repository branch, master, updated. cvs-readonly-176-g64df649 |
|
Date: |
Tue, 16 Dec 2008 13:53:17 +0000 |
This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "GNU M4 source repository".
http://git.sv.gnu.org/gitweb/?p=m4.git;a=commitdiff;h=64df649442038cace6b063fa5bd36c7007193dcc
The branch, master has been updated
via 64df649442038cace6b063fa5bd36c7007193dcc (commit)
via c28c0d455f7e28ee1bac21ab398dbe3d010b502a (commit)
via 01c5807a56be04620dbe3ff50f72358ef1ed88ed (commit)
via 50aba86fb9b76d2371385d2a042101ee30fda337 (commit)
via 789d06f131983794a089ad6c09e79e3613ac132f (commit)
from 41d0c77062c8046730101d82b56f682edc70957e (commit)
Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.
- Log -----------------------------------------------------------------
commit 64df649442038cace6b063fa5bd36c7007193dcc
Author: Eric Blake <address@hidden>
Date: Wed Dec 10 07:16:51 2008 -0700
Double size of temp file cache.
* m4/output.c (tmp_file, tmp_file_owner): Split...
(tmp_file1, tmp_file2, tmp_file1_owner, tmp_file2_owner): ...into
two variables.
(tmp_file2_recent): New variable.
(m4_tmpopen, m4_tmpclose, m4_tmpremove, m4_tmprename)
(m4_output_exit): Adjust callers.
Signed-off-by: Eric Blake <address@hidden>
commit c28c0d455f7e28ee1bac21ab398dbe3d010b502a
Author: Eric Blake <address@hidden>
Date: Wed Dec 10 06:15:53 2008 -0700
Use fewer seeks on cached files.
* m4/output.c (m4_tmpfile): Use write, not append mode.
(m4_tmpopen): Add parameter to decide when to skip seeks.
(m4_tmprename, m4_make_diversion, insert_diversion_helper)
(m4_freeze_diversions): Adjust callers.
Signed-off-by: Eric Blake <address@hidden>
commit 01c5807a56be04620dbe3ff50f72358ef1ed88ed
Author: Eric Blake <address@hidden>
Date: Sat Dec 13 21:59:57 2008 -0700
Cache most recently spilled diversion.
* m4/output.c (tmp_file, tmp_file_owner): New variables, for
1-deep cache of spilled diversions.
(m4_tmpfile): Open in append mode, since we might revisit
diversion without closing it now.
(m4_tmpopen): Check cache first.
(m4_tmpclose): Update cache, rather than closing. Add parameter.
(m4_tmpremove): Close cache before removing.
(m4_tmprename): Deal with open files when renaming.
(m4_output_exit): Close cache before exiting.
(make_room_for, m4_make_diversion, insert_diversion_helper):
Adjust callers.
* ltdl/m4/m4-rename.m4 (M4_RENAME): New file.
* configure.ac (M4_RENAME): Invoke it.
Signed-off-by: Eric Blake <address@hidden>
commit 50aba86fb9b76d2371385d2a042101ee30fda337
Author: Eric Blake <address@hidden>
Date: Tue Dec 9 21:23:44 2008 -0700
Correctly track size of in-memory diversions.
* m4/output.c (insert_diversion_helper): Correctly track total
in-memory diversion size after undivert.
Signed-off-by: Eric Blake <address@hidden>
commit 789d06f131983794a089ad6c09e79e3613ac132f
Author: Eric Blake <address@hidden>
Date: Tue Dec 9 20:56:37 2008 -0700
Avoid quadratic behavior for some cases of divert/undivert.
* m4/output.c (struct m4_diversion): Improve comments.
(m4_tmpname, m4_make_diversion): Strengthen preconditions.
(m4_tmprename): New function.
(m4_output_init, m4_output_exit): Move after internal functions.
(make_room_for): Don't bother copying uninitialized bytes.
(insert_diversion_helper): Transfer metadata, rather than copying
contents, when undiverting into a previously unused diversion.
* tests/builtins.at (divert): Add a check to the test.
* doc/m4.texinfo (Undivert): Enhance test.
* NEWS: Document the speedup.
Signed-off-by: Eric Blake <address@hidden>
-----------------------------------------------------------------------
Summary of changes:
.gitignore | 2 +-
ChangeLog | 47 ++++++++
NEWS | 2 +-
configure.ac | 1 +
doc/m4.texinfo | 8 ++
ltdl/m4/m4-rename.m4 | 45 +++++++
m4/output.c | 315 ++++++++++++++++++++++++++++++++++++++------------
tests/builtins.at | 14 +++
8 files changed, 360 insertions(+), 74 deletions(-)
create mode 100644 ltdl/m4/m4-rename.m4
diff --git a/.gitignore b/.gitignore
index 5e8e96c..0041494 100644
--- a/.gitignore
+++ b/.gitignore
@@ -24,7 +24,7 @@ INSTALL
intl
libtool
*.log
-m4-*
+/m4-*
Makefile
Makefile.in
patches
diff --git a/ChangeLog b/ChangeLog
index 69849a9..a73ba6e 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,50 @@
+2008-12-15 Eric Blake <address@hidden>
+
+ Double size of temp file cache.
+ * m4/output.c (tmp_file, tmp_file_owner): Split...
+ (tmp_file1, tmp_file2, tmp_file1_owner, tmp_file2_owner): ...into
+ two variables.
+ (tmp_file2_recent): New variable.
+ (m4_tmpopen, m4_tmpclose, m4_tmpremove, m4_tmprename)
+ (m4_output_exit): Adjust callers.
+
+ Use fewer seeks on cached files.
+ * m4/output.c (m4_tmpfile): Use write, not append mode.
+ (m4_tmpopen): Add parameter to decide when to skip seeks.
+ (m4_tmprename, m4_make_diversion, insert_diversion_helper)
+ (m4_freeze_diversions): Adjust callers.
+
+ Cache most recently spilled diversion.
+ * m4/output.c (tmp_file, tmp_file_owner): New variables, for
+ 1-deep cache of spilled diversions.
+ (m4_tmpfile): Open in append mode, since we might revisit
+ diversion without closing it now.
+ (m4_tmpopen): Check cache first.
+ (m4_tmpclose): Update cache, rather than closing. Add parameter.
+ (m4_tmpremove): Close cache before removing.
+ (m4_tmprename): Deal with open files when renaming.
+ (m4_output_exit): Close cache before exiting.
+ (make_room_for, m4_make_diversion, insert_diversion_helper):
+ Adjust callers.
+ * ltdl/m4/m4-rename.m4 (M4_RENAME): New file.
+ * configure.ac (M4_RENAME): Invoke it.
+
+ Correctly track size of in-memory diversions.
+ * m4/output.c (insert_diversion_helper): Correctly track total
+ in-memory diversion size after undivert.
+
+ Avoid quadratic behavior for some cases of divert/undivert.
+ * m4/output.c (struct m4_diversion): Improve comments.
+ (m4_tmpname, m4_make_diversion): Strengthen preconditions.
+ (m4_tmprename): New function.
+ (m4_output_init, m4_output_exit): Move after internal functions.
+ (make_room_for): Don't bother copying uninitialized bytes.
+ (insert_diversion_helper): Transfer metadata, rather than copying
+ contents, when undiverting into a previously unused diversion.
+ * tests/builtins.at (divert): Add a check to the test.
+ * doc/m4.texinfo (Undivert): Enhance test.
+ * NEWS: Document the speedup.
+
2008-12-02 Eric Blake <address@hidden>
Stage 27: Allow embedded NUL in text processing macros.
diff --git a/NEWS b/NEWS
index 1332647..618a8f2 100644
--- a/NEWS
+++ b/NEWS
@@ -244,7 +244,7 @@ promoted to 2.0.
** The `divert' builtin now accepts an optional second argument of text
that is immediately placed in the new diversion, regardless of whether
the current expansion is nested within argument collection of another
- macro.
+ macro. It has also been optimized for faster performance.
** The `-d'/`--debug' command-line option now understands `-' and `+'
modifiers, the way the builtin `debugmode' has always done; this allows
diff --git a/configure.ac b/configure.ac
index 1aa9397..d8962cc 100644
--- a/configure.ac
+++ b/configure.ac
@@ -162,6 +162,7 @@ M4_ERROR
M4_GETOPT
M4_OBSTACK
M4_REGEX
+M4_RENAME
## ------------------------ ##
diff --git a/doc/m4.texinfo b/doc/m4.texinfo
index bee9aec..173921c 100644
--- a/doc/m4.texinfo
+++ b/doc/m4.texinfo
@@ -6267,6 +6267,14 @@ undivert(`0')
undivert
@result{}diverted text
@result{}
+divert(`1')more
+divert(`2')undivert(`1')diverted text`'divert
address@hidden
+undivert(`1')
address@hidden
+undivert(`2')
address@hidden
address@hidden text
@end example
When a diversion has been undiverted, the diverted text is discarded,
diff --git a/ltdl/m4/m4-rename.m4 b/ltdl/m4/m4-rename.m4
new file mode 100644
index 0000000..892032b
--- /dev/null
+++ b/ltdl/m4/m4-rename.m4
@@ -0,0 +1,45 @@
+# -*- Autoconf -*-
+# m4-rename.m4 -- Test the abilities of rename.
+# Written by Eric Blake <address@hidden>
+#
+# Copyright (C) 2008 Free Software Foundation, Inc
+#
+# This file is part of GNU M4.
+#
+# GNU M4 is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# GNU M4 is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# serial 1
+
+# M4_RENAME
+# ---------
+# Detect platforms like mingw where rename can't move open files.
+AC_DEFUN([M4_RENAME],
+[AC_CACHE_CHECK([whether an open file can be renamed],
+ [M4_cv_func_rename_open_file_works],
+ [AC_RUN_IFELSE([AC_LANG_PROGRAM([AC_INCLUDES_DEFAULT],
+ [FILE *f = fopen ("conftest.1", "w+");
+ int result = rename ("conftest.1", "conftest.2");
+ fclose (f); remove ("conftest.1"); remove ("conftest.2");
+ return result;])],
+ [M4_cv_func_rename_open_file_works=yes],
+ [M4_cv_func_rename_open_file_works=no],
+ [M4_cv_func_rename_open_file_works='guessing no'])])
+if test "$M4_cv_func_rename_open_file_works" = yes ; then
+ M4_rename_open_works=1
+else
+ M4_rename_open_works=0
+fi
+AC_DEFINE_UNQUOTED([RENAME_OPEN_FILE_WORKS], [$M4_rename_open_works],
+ [Define to 1 if a file can be renamed while open, or to 0 if not.])
+])
diff --git a/m4/output.c b/m4/output.c
index 7f6f9c1..baea127 100644
--- a/m4/output.c
+++ b/m4/output.c
@@ -65,13 +65,13 @@ struct m4_diversion
{
union
{
- FILE *file; /* diversion file on disk */
- char *buffer; /* in-memory diversion buffer */
- m4_diversion *next; /* free-list pointer */
+ FILE *file; /* Diversion file on disk. */
+ char *buffer; /* Malloc'd diversion buffer. */
+ m4_diversion *next; /* Free-list pointer */
} u;
- int divnum; /* which diversion this represents */
- size_t size; /* usable size before reallocation */
- size_t used; /* used length in characters */
+ int divnum; /* Which diversion this represents. */
+ size_t size; /* Usable size before reallocation. */
+ size_t used; /* Used buffer length, or tmp file exists. */
};
/* Sorted set of diversions 1 through INT_MAX. */
@@ -89,20 +89,42 @@ static m4_obstack diversion_storage;
/* Total size of all in-memory buffer sizes. */
static size_t total_buffer_size;
-/* Current output diversion, NULL if output is being currently discarded. */
+/* Current output diversion, NULL if output is being currently
+ discarded. output_diversion->u is guaranteed non-NULL except when
+ the diversion has never been used; use size to determine if it is a
+ malloc'd buffer or a FILE. output_diversion->used is 0 if u.file
+ is stdout, and non-zero if this is a malloc'd buffer or a temporary
+ diversion file. */
static m4_diversion *output_diversion;
-/* Values of some output_diversion fields, cached out for speed. */
-static FILE *output_file; /* current value of (file) */
-static char *output_cursor; /* current value of (buffer + used) */
-static size_t output_unused; /* current value of (size - used) */
+/* Cache of output_diversion->u.file, only valid when
+ output_diversion->size is 0. */
+static FILE *output_file;
+
+/* Cache of output_diversion->u.buffer + output_diversion->used, only
+ valid when output_diversion->size is non-zero. */
+static char *output_cursor;
+
+/* Cache of output_diversion->size - output_diversion->used, only
+ valid when output_diversion->size is non-zero. */
+static size_t output_unused;
/* Temporary directory holding all spilled diversion files. */
static m4_temp_dir *output_temp_dir;
+/* Cache of most recently used spilled diversion files. */
+static FILE *tmp_file1;
+static FILE *tmp_file2;
+
+/* Diversions that own tmp_file, or 0. */
+static int tmp_file1_owner;
+static int tmp_file2_owner;
+
+/* True if tmp_file2 is more recently used. */
+static bool tmp_file2_recent;
-/* --- OUTPUT INITIALIZATION --- */
+/* Internal routines. */
/* Callback for comparing list elements ELT1 and ELT2 for order in
diversion_table. */
@@ -126,32 +148,6 @@ threshold_diversion_CB (const void *elt, const void
*threshold)
return diversion->divnum >= *(const int *) threshold;
}
-/* Initialize the output engine. */
-void
-m4_output_init (m4 *context)
-{
- diversion_table = gl_oset_create_empty (GL_AVLTREE_OSET, cmp_diversion_CB,
- NULL);
- div0.u.file = stdout;
- m4_set_current_diversion (context, 0);
- output_diversion = &div0;
- output_file = stdout;
- obstack_init (&diversion_storage);
-}
-
-/* Clean up memory allocated during use. */
-void
-m4_output_exit (void)
-{
- /* Order is important, since we may have registered cleanup_tmpfile
- as an atexit handler, and it must not traverse stale memory. */
- gl_oset_t table = diversion_table;
- assert (gl_oset_size (diversion_table) == 0);
- diversion_table = NULL;
- gl_oset_free (table);
- obstack_free (&diversion_storage, NULL);
-}
-
/* Clean up any temporary directory. Designed for use as an atexit
handler, where it is not safe to call exit() recursively; so this
calls _exit if a problem is encountered. */
@@ -199,6 +195,7 @@ m4_tmpname (int divnum)
buffer = (char *) obstack_alloc (&diversion_storage,
INT_BUFSIZE_BOUND (divnum));
}
+ assert (0 < divnum);
if (snprintf (&buffer[offset], INT_BUFSIZE_BOUND (divnum), "%d", divnum) < 0)
abort ();
return buffer;
@@ -207,9 +204,10 @@ m4_tmpname (int divnum)
/* Create a temporary file for diversion DIVNUM open for reading and
writing in a secure temp directory. The file will be automatically
closed and deleted on a fatal signal. The file can be closed and
- reopened with m4_tmpclose and m4_tmpopen; when finally done with
- the file, close it and use m4_tmpremove. Exits on failure, so the
- return value is always an open file. */
+ reopened with m4_tmpclose and m4_tmpopen, or moved with
+ m4_tmprename; when finally done with the file, close it with
+ m4_tmpremove. Exits on failure, so the return value is always an
+ open file. */
static FILE *
m4_tmpfile (m4 *context, int divnum)
{
@@ -239,42 +237,177 @@ m4_tmpfile (m4 *context, int divnum)
}
/* Reopen a temporary file for diversion DIVNUM for reading and
- writing in a secure temp directory. Exits on failure, so the
- return value is always an open file. */
+ writing in a secure temp directory. If REREAD, the file is
+ positioned at offset 0, otherwise the file is positioned at the
+ end. Exits on failure, so the return value is always an open
+ file. */
static FILE *
-m4_tmpopen (m4 *context, int divnum)
+m4_tmpopen (m4 *context, int divnum, bool reread)
{
- const char *name = m4_tmpname (divnum);
+ const char *name;
FILE *file;
- file = fopen_temp (name, O_BINARY ? "ab+" : "a+");
+ if (tmp_file1_owner == divnum)
+ {
+ if (reread && fseeko (tmp_file1, 0, SEEK_SET) != 0)
+ m4_error (context, EXIT_FAILURE, errno, NULL,
+ _("cannot seek within diversion"));
+ tmp_file2_recent = false;
+ return tmp_file1;
+ }
+ else if (tmp_file2_owner == divnum)
+ {
+ if (reread && fseeko (tmp_file2, 0, SEEK_SET) != 0)
+ m4_error (context, EXIT_FAILURE, errno, NULL,
+ _("cannot seek to beginning of diversion"));
+ tmp_file2_recent = true;
+ return tmp_file2;
+ }
+ name = m4_tmpname (divnum);
+ /* We need update mode, to avoid truncation. */
+ file = fopen_temp (name, O_BINARY ? "rb+" : "r+");
if (file == NULL)
m4_error (context, EXIT_FAILURE, errno, NULL,
_("cannot create temporary file for diversion"));
else if (set_cloexec_flag (fileno (file), true) != 0)
m4_warn (context, errno, NULL, _("cannot protect diversion across forks"));
- /* POSIX states that it is undefined whether an append stream starts
- at offset 0 or at the end. We want the beginning. */
- else if (fseeko (file, 0, SEEK_SET) != 0)
+ /* Update mode starts at the beginning of the stream, but sometimes
+ we want the end. */
+ else if (!reread && fseeko (file, 0, SEEK_END) != 0)
m4_error (context, EXIT_FAILURE, errno, NULL,
- _("cannot seek to beginning of diversion"));
+ _("cannot seek within diversion"));
return file;
}
-/* Close, but don't delete, a temporary FILE. */
+/* Close, but don't delete, a temporary FILE for diversion DIVNUM. To
+ reduce the I/O overhead of repeatedly opening and closing the same
+ file, this implementation caches the most recent spilled diversion.
+ On the other hand, keeping every spilled diversion open would run
+ into EMFILE limits. */
static int
-m4_tmpclose (FILE *file)
+m4_tmpclose (FILE *file, int divnum)
{
- return close_stream_temp (file);
+ int result = 0;
+ if (divnum != tmp_file1_owner && divnum != tmp_file2_owner)
+ {
+ if (tmp_file2_recent)
+ {
+ if (tmp_file1_owner)
+ result = close_stream_temp (tmp_file1);
+ tmp_file1 = file;
+ tmp_file1_owner = divnum;
+ }
+ else
+ {
+ if (tmp_file2_owner)
+ result = close_stream_temp (tmp_file2);
+ tmp_file2 = file;
+ tmp_file2_owner = divnum;
+ }
+ }
+ return result;
}
/* Delete a closed temporary FILE for diversion DIVNUM. */
static int
m4_tmpremove (int divnum)
{
+ if (divnum == tmp_file1_owner)
+ {
+ int result = close_stream_temp (tmp_file1);
+ if (result)
+ return result;
+ tmp_file1_owner = 0;
+ }
+ else if (divnum == tmp_file2_owner)
+ {
+ int result = close_stream_temp (tmp_file2);
+ if (result)
+ return result;
+ tmp_file2_owner = 0;
+ }
return cleanup_temp_file (output_temp_dir, m4_tmpname (divnum));
}
+/* Transfer the temporary file for diversion OLDNUM to the previously
+ unused diversion NEWNUM. Return an open stream visiting the new
+ temporary file, positioned at the end, or exit on failure. */
+static FILE*
+m4_tmprename (m4 *context, int oldnum, int newnum)
+{
+ /* m4_tmpname reuses its return buffer. */
+ char *oldname = xstrdup (m4_tmpname (oldnum));
+ const char *newname = m4_tmpname (newnum);
+ register_temp_file (output_temp_dir, newname);
+ if (oldnum == tmp_file1_owner)
+ {
+ /* Be careful of mingw, which can't rename an open file. */
+ if (RENAME_OPEN_FILE_WORKS)
+ tmp_file1_owner = newnum;
+ else
+ {
+ if (close_stream_temp (tmp_file1))
+ m4_error (context, EXIT_FAILURE, errno, NULL,
+ _("cannot close temporary file for diversion"));
+ tmp_file1_owner = 0;
+ }
+ }
+ else if (oldnum == tmp_file2_owner)
+ {
+ /* Be careful of mingw, which can't rename an open file. */
+ if (RENAME_OPEN_FILE_WORKS)
+ tmp_file2_owner = newnum;
+ else
+ {
+ if (close_stream_temp (tmp_file2))
+ m4_error (context, EXIT_FAILURE, errno, NULL,
+ _("cannot close temporary file for diversion"));
+ tmp_file2_owner = 0;
+ }
+ }
+ /* Either it is safe to rename an open file, or no one should have
+ oldname open at this point. */
+ if (rename (oldname, newname))
+ m4_error (context, EXIT_FAILURE, errno, NULL,
+ _("cannot create temporary file for diversion"));
+ unregister_temp_file (output_temp_dir, oldname);
+ free (oldname);
+ return m4_tmpopen (context, newnum, false);
+}
+
+
+/* --- OUTPUT INITIALIZATION --- */
+
+/* Initialize the output engine. */
+void
+m4_output_init (m4 *context)
+{
+ diversion_table = gl_oset_create_empty (GL_AVLTREE_OSET, cmp_diversion_CB,
+ NULL);
+ div0.u.file = stdout;
+ m4_set_current_diversion (context, 0);
+ output_diversion = &div0;
+ output_file = stdout;
+ obstack_init (&diversion_storage);
+}
+
+/* Clean up memory allocated during use. */
+void
+m4_output_exit (void)
+{
+ /* Order is important, since we may have registered cleanup_tmpfile
+ as an atexit handler, and it must not traverse stale memory. */
+ gl_oset_t table = diversion_table;
+ assert (gl_oset_size (diversion_table) == 0);
+ if (tmp_file1_owner)
+ m4_tmpremove (tmp_file1_owner);
+ if (tmp_file2_owner)
+ m4_tmpremove (tmp_file2_owner);
+ diversion_table = NULL;
+ gl_oset_free (table);
+ obstack_free (&diversion_storage, NULL);
+}
+
/* Reorganize in-memory diversion buffers so the current diversion can
accomodate LENGTH more characters without further reorganization. The
current diversion buffer is made bigger if possible. But to make room
@@ -380,16 +513,19 @@ make_room_for (m4 *context, size_t length)
{
FILE *file = selected_diversion->u.file;
selected_diversion->u.file = NULL;
- if (m4_tmpclose (file) != 0)
+ if (m4_tmpclose (file, selected_diversion->divnum) != 0)
m4_error (context, 0, errno, NULL,
_("cannot close temporary file for diversion"));
}
- /* The buffer may be safely reallocated. */
-
+ /* The current buffer may be safely reallocated. */
assert (wanted_size >= length);
- output_diversion->u.buffer = xrealloc (output_diversion->u.buffer,
- wanted_size);
+ {
+ char *buffer = output_diversion->u.buffer;
+ output_diversion->u.buffer = xcharalloc ((size_t) wanted_size);
+ memcpy (output_diversion->u.buffer, buffer, output_diversion->used);
+ free (buffer);
+ }
total_buffer_size += wanted_size - output_diversion->size;
output_diversion->size = wanted_size;
@@ -664,10 +800,10 @@ m4_make_diversion (m4 *context, int divnum)
assert (output_diversion->divnum != divnum);
if (!output_diversion->size && !output_diversion->u.file)
{
+ assert (!output_diversion->used);
if (!gl_oset_remove (diversion_table, output_diversion))
assert (false);
output_diversion->u.next = free_list;
- output_diversion->used = 0;
free_list = output_diversion;
}
else if (output_diversion->size)
@@ -677,7 +813,7 @@ m4_make_diversion (m4 *context, int divnum)
assert (output_diversion->divnum != 0);
FILE *file = output_diversion->u.file;
output_diversion->u.file = NULL;
- if (m4_tmpclose (file) != 0)
+ if (m4_tmpclose (file, output_diversion->divnum) != 0)
m4_error (context, 0, errno, NULL,
_("cannot close temporary file for diversion"));
}
@@ -737,7 +873,8 @@ m4_make_diversion (m4 *context, int divnum)
{
if (!output_diversion->u.file && output_diversion->used)
output_diversion->u.file = m4_tmpopen (context,
- output_diversion->divnum);
+ output_diversion->divnum,
+ false);
output_file = output_diversion->u.file;
}
@@ -802,18 +939,50 @@ insert_diversion_helper (m4 *context, m4_diversion
*diversion, bool escaped)
{
if (diversion->size)
{
- char *str = diversion->u.buffer;
- size_t len = diversion->used;
- if (escaped)
- str = quotearg_style_mem (escape_quoting_style, str, len);
- m4_output_text (context, str, escaped ? strlen (str) : len);
+ if (!output_diversion->u.file)
+ {
+ /* Transferring diversion metadata is faster than
+ copying contents. */
+ assert (!output_diversion->used && output_diversion != &div0
+ && !output_file);
+ output_diversion->u.buffer = diversion->u.buffer;
+ output_diversion->size = diversion->size;
+ output_cursor = diversion->u.buffer + diversion->used;
+ output_unused = diversion->size - diversion->used;
+ diversion->u.buffer = NULL;
+ }
+ else
+ {
+ char *str = diversion->u.buffer;
+ size_t len = diversion->used;
+ /* Avoid double-charging the total in-memory size when
+ transferring from one in-memory diversion to
+ another. */
+ total_buffer_size -= diversion->size;
+ if (escaped)
+ str = quotearg_style_mem (escape_quoting_style, str, len);
+ m4_output_text (context, str, escaped ? strlen (str) : len);
+ }
+ }
+ else if (!output_diversion->u.file)
+ {
+ /* Transferring diversion metadata is faster than copying
+ contents. */
+ assert (!output_diversion->used && output_diversion != &div0
+ && !output_file);
+ output_diversion->u.file = m4_tmprename (context, diversion->divnum,
+ output_diversion->divnum);
+ output_diversion->used = 1;
+ output_file = output_diversion->u.file;
+ diversion->u.file = NULL;
+ diversion->size = 1;
}
else
{
assert (diversion->used);
if (!diversion->u.file)
- diversion->u.file = m4_tmpopen (context, diversion->divnum);
- m4_insert_file (context, diversion->u.file);
+ diversion->u.file = m4_tmpopen (context, diversion->divnum, true);
+ insert_file (context, diversion->u.file, escaped);
}
m4_set_output_line (context, -1);
@@ -822,9 +991,10 @@ insert_diversion_helper (m4 *context, m4_diversion
*diversion, bool escaped)
/* Return all space used by the diversion. */
if (diversion->size)
{
+ if (!output_diversion)
+ total_buffer_size -= diversion->size;
free (diversion->u.buffer);
diversion->size = 0;
- diversion->used = 0;
}
else
{
@@ -832,8 +1002,7 @@ insert_diversion_helper (m4 *context, m4_diversion
*diversion, bool escaped)
{
FILE *file = diversion->u.file;
diversion->u.file = NULL;
- diversion->used = 0;
- if (m4_tmpclose (file) != 0)
+ if (m4_tmpclose (file, diversion->divnum) != 0)
m4_error (context, 0, errno, NULL,
_("cannot clean temporary file for diversion"));
}
@@ -841,6 +1010,7 @@ insert_diversion_helper (m4 *context, m4_diversion
*diversion, bool escaped)
m4_error (context, 0, errno, NULL,
_("cannot clean temporary file for diversion"));
}
+ diversion->used = 0;
if (!gl_oset_remove (diversion_table, diversion))
assert (false);
diversion->u.next = free_list;
@@ -915,7 +1085,8 @@ m4_freeze_diversions (m4 *context, FILE *file)
{
struct stat file_stat;
assert (!diversion->u.file);
- diversion->u.file = m4_tmpopen (context, diversion->divnum);
+ diversion->u.file = m4_tmpopen (context, diversion->divnum,
+ true);
if (fstat (fileno (diversion->u.file), &file_stat) < 0)
m4_error (context, EXIT_FAILURE, errno, NULL,
_("cannot stat diversion"));
diff --git a/tests/builtins.at b/tests/builtins.at
index 83810fa..f819403 100644
--- a/tests/builtins.at
+++ b/tests/builtins.at
@@ -353,6 +353,20 @@ rm expout2
AT_CHECK_M4([in.m4], [0], [expout])
+dnl Avoid quadratic copying time when transferring diversions; test
+dnl both in-memory and diversions spilled to a file.
+AT_DATA([in.m4], [[include(`forloop2.m4')dnl
+divert(`1')format(`%10000s', `')dnl
+forloop(`i', `1', `10000',
+ `divert(incr(i))undivert(i)')dnl
+divert(`9001')format(`%1000000s', `')dnl
+forloop(`i', `9001', `10000',
+ `divert(incr(i))undivert(i)')dnl
+divert(`-1')undivert
+]])
+
+AT_CHECK_M4([-I "$top_srcdir/examples" in.m4])
+
AT_CLEANUP
hooks/post-receive
--
GNU M4 source repository
| [Prev in Thread] |
Current Thread |
[Next in Thread] |
- [SCM] GNU M4 source repository branch, master, updated. cvs-readonly-176-g64df649,
Eric Blake <=