[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[8/18] argv_ref speedup: lengthen lifetime of $@ references
|
From: |
Eric Blake |
|
Subject: |
[8/18] argv_ref speedup: lengthen lifetime of $@ references |
|
Date: |
Tue, 18 Dec 2007 07:21:32 -0700 |
|
User-agent: |
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.9) Gecko/20071031 Thunderbird/2.0.0.9 Mnenhy/0.7.5.666 |
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Next in the series. This patch merely adds the bookkeeping hooks
necessary to lengthen parameter lifetimes, so it adds both time and memory
to computation. Without this patch, we were freeing the collected
arguments to a macro call the moment the macro generated its expansion
text, making it impossible to reuse any cache data associated with those
parameters when rescanning the expansion. This patch makes it so that if
we create a reference to a macro's arguments, then those arguments will
remain allocated in memory until all rescanning of the expansion has
completed. At the same time, it still tries as hard as possible to avoid
keeping references in memory, when it can be proven that no references
were created (earlier versions of this patch blindly kept everything in
memory, then I watched my system crash as it ran out of resources). Since
input.c:push_token does not yet preserve any references (I split that
functionality into later patches, since this one was already getting big),
that means that the only additional memory used by this patch is due to
the fact that struct token_chain is larger.
2007-12-18 Eric Blake <address@hidden>
Stage 8: extend life of references into argv.
* src/m4.h (obstack_regrow): Delete, now that it is unused.
(struct token_chain): Add level field.
(push_token): Adjust prototype.
(adjust_refcount): New prototype.
* src/macro.c [DEBUG_MACRO]: Add debugging hooks for $@ reference
counting.
(argc_stack, argv_stack): Delete, replaced by...
(struct macro_arg_stacks, stacks, stacks_count): ...these new
structure for tracking $@ references.
(expand_input): Initialize new structure.
(collect_arguments): Alter signature.
(expand_macro): Rework obstack handling, in order to keep $@
references alive until they have been rescanned.
(adjust_refcount, arg_mark): New functions.
(make_argv_ref): Use new field.
(push_arg, push_args): Update callers.
* src/input.c (make_text_link): Use new field.
(push_token): Change signature.
(pop_input, next_char_1): Call new function.
* doc/m4.texinfo: Clean up some examples of -d option usage.
(Ifelse): Add some stress tests.
- --
Don't work too hard, make some time for fun as well!
Eric Blake address@hidden
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Cygwin)
Comment: Public key at home.comcast.net/~ericblake/eblake.gpg
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFHZ9ds84KuGfSFAYARAvETAJ9Z/l7DkkN3LDvZAqVKbjp6BBOkmACdHgWp
ZhsaCBPej9QnJ3uVCHH9q4g=
=wTA3
-----END PGP SIGNATURE-----
>From fa34c3f77687fac1d5264955e11a7c704a89e875 Mon Sep 17 00:00:00 2001
From: Eric Blake <address@hidden>
Date: Mon, 17 Dec 2007 09:28:39 -0700
Subject: [PATCH] Stage 8: extend life of references into argv.
* m4/system_.h (obstack_regrow): Delete.
* m4/m4private.h (struct m4_symbol_chain): Add level field.
(m4__push_symbol): Adjust prototype.
(m4__adjust_refcount): New prototype.
(DEBUG_MACRO) [DEBUG]: New debug control.
(struct m4__macro_arg_stacks): New structure.
(struct m4): Add arg_stacks, stacks_count fields.
* m4/m4module.h (m4_make_argv_ref): Add parameter.
* m4/macro.c (argc_stack, argv_stack): Delete, replaced by
context->arg_stacks.
(m4_macro_expand_input) [DEBUG_MACRO]: Add debug hooks,
conditional on M4_DEBUG_MACRO envvar.
(collect_arguments): Adjust signature.
(expand_macro): Rework obstack handling.
(m4__adjust_refcount, arg_mark): New functions.
(m4_make_argv_ref): Populate new field.
(m4_push_arg, m4_push_args): Track inuse.
(process_macro): One less cast.
* m4/m4.c (m4_delete): Clean up arg_stacks.
* m4/input.c (make_text_link): Use new field.
(m4__push_symbol, file_clean): Update signature.
(composite_read): Bump refcount when done with reference.
(composite_clean): New function.
(pop_input): Adjust caller.
* m4/debug.c (m4_debug_message): Make assertion match comment.
* modules/gnu.c (builtin, indir): Adjust callers.
* tests/builtins.at (ifelse): New test.
(exp): Move and rename...
* tests/others.at (countdown): ...to this.
* doc/m4.texinfo (Improved foreach): Fix tracing usage in
example.
Signed-off-by: Eric Blake <address@hidden>
---
ChangeLog | 35 ++++++
doc/m4.texinfo | 2 +-
m4/debug.c | 2 +-
m4/input.c | 65 ++++++++---
m4/m4.c | 15 +++
m4/m4module.h | 4 +-
m4/m4private.h | 35 +++++--
m4/macro.c | 327 +++++++++++++++++++++++++++++++++++++++++------------
m4/system_.h | 8 --
modules/gnu.c | 9 +-
tests/builtins.at | 63 +++++++----
tests/others.at | 26 ++++
12 files changed, 454 insertions(+), 137 deletions(-)
diff --git a/ChangeLog b/ChangeLog
index 60ffb58..f5cf9e1 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,38 @@
+2007-12-17 Eric Blake <address@hidden>
+
+ Stage 8: extend life of references into argv.
+ * m4/system_.h (obstack_regrow): Delete.
+ * m4/m4private.h (struct m4_symbol_chain): Add level field.
+ (m4__push_symbol): Adjust prototype.
+ (m4__adjust_refcount): New prototype.
+ (DEBUG_MACRO) [DEBUG]: New debug control.
+ (struct m4__macro_arg_stacks): New structure.
+ (struct m4): Add arg_stacks, stacks_count fields.
+ * m4/m4module.h (m4_make_argv_ref): Add parameter.
+ * m4/macro.c (argc_stack, argv_stack): Delete, replaced by
+ context->arg_stacks.
+ (m4_macro_expand_input) [DEBUG_MACRO]: Add debug hooks,
+ conditional on M4_DEBUG_MACRO envvar.
+ (collect_arguments): Adjust signature.
+ (expand_macro): Rework obstack handling.
+ (m4__adjust_refcount, arg_mark): New functions.
+ (m4_make_argv_ref): Populate new field.
+ (m4_push_arg, m4_push_args): Track inuse.
+ (process_macro): One less cast.
+ * m4/m4.c (m4_delete): Clean up arg_stacks.
+ * m4/input.c (make_text_link): Use new field.
+ (m4__push_symbol, file_clean): Update signature.
+ (composite_read): Bump refcount when done with reference.
+ (composite_clean): New function.
+ (pop_input): Adjust caller.
+ * m4/debug.c (m4_debug_message): Make assertion match comment.
+ * modules/gnu.c (builtin, indir): Adjust callers.
+ * tests/builtins.at (ifelse): New test.
+ (exp): Move and rename...
+ * tests/others.at (countdown): ...to this.
+ * doc/m4.texinfo (Improved foreach): Fix tracing usage in
+ example.
+
2007-12-13 Eric Blake <address@hidden>
Yet more rewording.
diff --git a/doc/m4.texinfo b/doc/m4.texinfo
index 29b0608..c8b2062 100644
--- a/doc/m4.texinfo
+++ b/doc/m4.texinfo
@@ -8401,7 +8401,7 @@ detrimental effects.
@comment examples
@example
-$ @kbd{m4 -I examples}
+$ @kbd{m4 -d -I examples}
include(`foreach2.m4')
@result{}
include(`foreachq2.m4')
diff --git a/m4/debug.c b/m4/debug.c
index 29272d0..5b54c4a 100644
--- a/m4/debug.c
+++ b/m4/debug.c
@@ -225,7 +225,7 @@ void
m4_debug_message (m4 *context, int mode, const char *format, ...)
{
/* Check that mode has exactly one bit set. */
- assert ((mode & (mode - 1)) == 0);
+ assert (mode && (mode & (mode - 1)) == 0);
assert (format);
if (m4_get_debug_file (context) != NULL
diff --git a/m4/input.c b/m4/input.c
index 28b00a8..bdacfbf 100644
--- a/m4/input.c
+++ b/m4/input.c
@@ -90,7 +90,7 @@
static int file_peek (m4_input_block *);
static int file_read (m4_input_block *, m4 *, bool);
static void file_unget (m4_input_block *, int);
-static void file_clean (m4_input_block *, m4 *);
+static bool file_clean (m4_input_block *, m4 *, bool);
static void file_print (m4_input_block *, m4 *, m4_obstack *);
static int builtin_peek (m4_input_block *);
static int builtin_read (m4_input_block *, m4 *, bool);
@@ -103,6 +103,7 @@ static void string_print (m4_input_block
*, m4 *, m4_obstack *);
static int composite_peek (m4_input_block *);
static int composite_read (m4_input_block *, m4 *, bool);
static void composite_unget (m4_input_block *, int);
+static bool composite_clean (m4_input_block *, m4 *, bool);
static void composite_print (m4_input_block *, m4 *, m4_obstack *);
static void make_text_link (m4_obstack *, m4_symbol_chain **,
@@ -135,8 +136,10 @@ struct input_funcs
same character previously read by read_func. */
void (*unget_func) (m4_input_block *, int);
- /* Optional function to perform cleanup at end of input. */
- void (*clean_func) (m4_input_block *, m4 *);
+ /* Optional function to perform cleanup at end of input. If
+ CLEANUP, it is safe to perform non-recoverable cleanup actions.
+ Return true only if no cleanup remains to be done. */
+ bool (*clean_func) (m4_input_block *, m4 *, bool cleanup);
/* Add a representation of the input block to the obstack, for use
in trace expansion output. */
@@ -238,7 +241,8 @@ static struct input_funcs string_funcs = {
/* Vtable for handling input from composite chains. */
static struct input_funcs composite_funcs = {
- composite_peek, composite_read, composite_unget, NULL, composite_print
+ composite_peek, composite_read, composite_unget, composite_clean,
+ composite_print
};
@@ -298,9 +302,11 @@ file_unget (m4_input_block *me, int ch)
start_of_input_line = false;
}
-static void
-file_clean (m4_input_block *me, m4 *context)
+static bool
+file_clean (m4_input_block *me, m4 *context, bool cleanup)
{
+ if (!cleanup)
+ return false;
if (me->prev)
m4_debug_message (context, M4_DEBUG_TRACE_INPUT,
_("input reverted to %s, line %d"),
@@ -318,6 +324,7 @@ file_clean (m4_input_block *me, m4 *context)
m4_error (context, 0, errno, NULL, _("error reading file `%s'"), me->file);
start_of_input_line = me->u.u_f.line_start;
m4_set_output_line (context, -1);
+ return true;
}
static void
@@ -519,8 +526,10 @@ m4_push_string_init (m4 *context)
current_input stack and VALUE lives in temporary storage. Allows
gathering input from multiple locations, rather than copying
everything consecutively onto the input stack. Must be called
- between push_string_init and push_string_finish. */
-void
+ between push_string_init and push_string_finish. Return true only
+ if LEVEL is less than SIZE_MAX and a reference was created to
+ VALUE. */
+bool
m4__push_symbol (m4_symbol_value *value, size_t level)
{
m4_symbol_chain *chain;
@@ -529,7 +538,7 @@ m4__push_symbol (m4_symbol_value *value, size_t level)
/* TODO - also accept TOKEN_COMP chains. */
assert (m4_is_symbol_value_text (value));
if (m4_get_symbol_value_len (value) == 0)
- return;
+ return false;
if (next->funcs == &string_funcs)
{
@@ -555,9 +564,11 @@ m4__push_symbol (m4_symbol_value *value, size_t level)
else
chain->str = m4_get_symbol_value_text (value);
chain->len = m4_get_symbol_value_len (value);
+ chain->level = SIZE_MAX;
chain->argv = NULL;
chain->index = 0;
chain->flatten = false;
+ return false; /* Only return true when data is reused, not copied. */
}
/* Last half of m4_push_string (). If next is now NULL, a call to
@@ -648,6 +659,8 @@ composite_read (m4_input_block *me, m4 *context, bool safe)
}
if (safe)
return CHAR_RETRY;
+ if (chain->level < SIZE_MAX)
+ m4__adjust_refcount (context, chain->level, false);
me->u.u_c.chain = chain = chain->next;
}
return CHAR_RETRY;
@@ -671,6 +684,28 @@ composite_unget (m4_input_block *me, int ch)
}
}
+static bool
+composite_clean (m4_input_block *me, m4 *context, bool cleanup)
+{
+ m4_symbol_chain *chain = me->u.u_c.chain;
+ assert (!chain || !cleanup);
+ while (chain)
+ {
+ if (chain->str)
+ assert (!chain->len);
+ else
+ {
+ /* TODO - peek into argv. */
+ assert (!"implemented yet");
+ abort ();
+ }
+ if (chain->level < SIZE_MAX)
+ m4__adjust_refcount (context, chain->level, false);
+ me->u.u_c.chain = chain = chain->next;
+ }
+ return true;
+}
+
static void
composite_print (m4_input_block *me, m4 *context, m4_obstack *obs)
{
@@ -718,6 +753,7 @@ make_text_link (m4_obstack *obs, m4_symbol_chain **start,
chain->next = NULL;
chain->str = str;
chain->len = len;
+ chain->level = SIZE_MAX;
chain->argv = NULL;
chain->index = 0;
chain->flatten = false;
@@ -787,16 +823,11 @@ pop_input (m4 *context, bool cleanup)
m4_input_block *tmp = isp->prev;
assert (isp);
- if (isp->funcs->peek_func (isp) != CHAR_RETRY)
+ if (isp->funcs->peek_func (isp) != CHAR_RETRY
+ || (isp->funcs->clean_func
+ && !isp->funcs->clean_func (isp, context, cleanup)))
return false;
- if (isp->funcs->clean_func != NULL)
- {
- if (!cleanup)
- return false;
- isp->funcs->clean_func (isp, context);
- }
-
if (tmp != NULL)
{
obstack_free (current_input, isp);
diff --git a/m4/m4.c b/m4/m4.c
index 651a943..066f97f 100644
--- a/m4/m4.c
+++ b/m4/m4.c
@@ -48,6 +48,7 @@ m4_create (void)
void
m4_delete (m4 *context)
{
+ size_t i;
assert (context);
if (context->symtab)
@@ -77,6 +78,20 @@ m4_delete (m4 *context)
free (context->search_path);
}
+ for (i = 0; i < context->stacks_count; i++)
+ {
+ assert (context->arg_stacks[i].refcount == 0
+ && context->arg_stacks[i].argcount == 0);
+ if (context->arg_stacks[i].args)
+ {
+ obstack_free (context->arg_stacks[i].args, NULL);
+ free (context->arg_stacks[i].args);
+ obstack_free (context->arg_stacks[i].argv, NULL);
+ free (context->arg_stacks[i].argv);
+ }
+ }
+ free (context->arg_stacks);
+
free (context);
}
diff --git a/m4/m4module.h b/m4/m4module.h
index 0bab08f..520bd4e 100644
--- a/m4/m4module.h
+++ b/m4/m4module.h
@@ -310,8 +310,8 @@ extern bool m4_arg_equal (m4_macro_args *,
unsigned int,
extern bool m4_arg_empty (m4_macro_args *, unsigned int);
extern size_t m4_arg_len (m4_macro_args *, unsigned int);
extern m4_builtin_func *m4_arg_func (m4_macro_args *, unsigned int);
-extern m4_macro_args *m4_make_argv_ref (m4_macro_args *, const char *, size_t,
- bool, bool);
+extern m4_macro_args *m4_make_argv_ref (m4 *, m4_macro_args *, const char *,
+ size_t, bool, bool);
extern void m4_push_arg (m4 *, m4_obstack *, m4_macro_args *,
unsigned int);
extern void m4_push_args (m4 *, m4_obstack *, m4_macro_args *,
diff --git a/m4/m4private.h b/m4/m4private.h
index b93d876..f24942f 100644
--- a/m4/m4private.h
+++ b/m4/m4private.h
@@ -28,6 +28,7 @@
#include "cloexec.h"
typedef struct m4__search_path_info m4__search_path_info;
+typedef struct m4__macro_arg_stacks m4__macro_arg_stacks;
typedef enum {
M4_SYMBOL_VOID, /* Traced but undefined, u is invalid. */
@@ -75,6 +76,8 @@ struct m4 {
/* __PRIVATE__: */
m4__search_path_info *search_path; /* The list of path directories. */
+ m4__macro_arg_stacks *arg_stacks; /* Array of current argv refs. */
+ size_t stacks_count; /* Size of arg_stacks. */
};
#define M4_OPT_PREFIX_BUILTINS_BIT (1 << 0) /* -P */
@@ -195,6 +198,7 @@ struct m4_symbol_chain
m4_symbol_chain *next;/* Pointer to next link of chain. */
const char *str; /* NUL-terminated string if text, or NULL. */
size_t len; /* Length of str, or 0. */
+ size_t level; /* Expansion level of content, or SIZE_MAX. */
m4_macro_args *argv; /* Reference to earlier address@hidden */
unsigned int index; /* Argument index within argv. */
bool flatten; /* True to treat builtins as text. */
@@ -256,6 +260,20 @@ struct m4_macro_args
m4_symbol_value *array[FLEXIBLE_ARRAY_MEMBER];
};
+/* Internal structure for managing multiple argv references. See
+ macro.c for a much more detailed comment on usage. */
+struct m4__macro_arg_stacks
+{
+ size_t refcount; /* Number of active $@ references at this level. */
+ size_t argcount; /* Number of argv at this level. */
+ m4_obstack *args; /* Content of arguments. */
+ m4_obstack *argv; /* Argv pointers into args. */
+ void *args_base; /* Location for clearing the args obstack. */
+ void *argv_base; /* Location for clearing the argv obstack. */
+};
+
+extern size_t m4__adjust_refcount (m4 *, size_t, bool);
+
#define VALUE_NEXT(T) ((T)->next)
#define VALUE_MODULE(T) ((T)->module)
#define VALUE_FLAGS(T) ((T)->flags)
@@ -432,7 +450,7 @@ typedef enum {
M4_TOKEN_MACDEF /* Macro's definition (see "defn"), M4_SYMBOL_FUNC. */
} m4__token_type;
-extern void m4__push_symbol (m4_symbol_value *, size_t);
+extern bool m4__push_symbol (m4_symbol_value *, size_t);
extern m4__token_type m4__next_token (m4 *, m4_symbol_value *, int *,
const char *);
extern bool m4__next_token_is_open (m4 *);
@@ -477,13 +495,14 @@ extern void m4__include_init (m4 *);
#if DEBUG
-# define DEBUG_INCL
-# define DEBUG_INPUT
-# define DEBUG_MODULES
-# define DEBUG_OUTPUT
-# define DEBUG_STKOVF
-# define DEBUG_SYM
-# define DEBUG_SYNTAX
+# define DEBUG_INCL 1
+# define DEBUG_INPUT 1
+# define DEBUG_MACRO 1
+# define DEBUG_MODULES 1
+# define DEBUG_OUTPUT 1
+# define DEBUG_STKOVF 1
+# define DEBUG_SYM 1
+# define DEBUG_SYNTAX 1
#endif
#endif /* m4private.h */
diff --git a/m4/macro.c b/m4/macro.c
index 1108d08..bb0d3b0 100644
--- a/m4/macro.c
+++ b/m4/macro.c
@@ -29,8 +29,102 @@
#include "intprops.h"
+/* Define this to 1 see runtime debug info. Implied by DEBUG. */
+/*#define DEBUG_INPUT 1 */
+#ifndef DEBUG_MACRO
+# define DEBUG_MACRO 0
+#endif /* DEBUG_MACRO */
+
+/* A note on argument memory lifetimes: We use an internal struct
+ (m4__macro_args_stacks) to maintain list of argument obstacks.
+ Within a recursion level, consecutive macros can share a stack, but
+ distinct recursion levels need different stacks since the nested
+ macro is interrupting the argument collection of the outer level.
+ Note that a reference can live as long as the expansion containing
+ the reference can participate as an argument in a future macro
+ call.
+
+ Therefore, we implement a reference counter for each expansion
+ level, tracking how many references exist into the obstack, as well
+ as associate a level with each reference. Of course, expand_macro
+ is actively using argv, so it increments the refcount on entry and
+ decrements it on exit. Additionally, any time the input engine is
+ handed a reference that it does not inline, it increases the
+ refcount in push_token, then decreases it in pop_input once the
+ reference has been rescanned. Finally, when the input engine hands
+ a reference back to expand_argument, the refcount increases, which
+ is then cleaned up at the end of expand_macro.
+
+ For a running example, consider this input:
+
+ define(a,A)define(b,`a(`$1')')define(c,$*)dnl
+ define(x,`a(1)`'c($@')define(y,`$@)')dnl
+ x(a(`b')``a'')y(`b')(`a')
+ => AAaA
+
+ Assuming all arguments are large enough to exceed the inlining
+ thresholds of the input engine, the interesting sequence of events
+ is as follows:
+
+ stacks[0] refs stacks[1] refs
+ after second dnl ends: `' 0 `' 0
+ expand_macro for x, level 0: `' 1 `' 0
+ expand_macro for a, level 1: `' 1 `' 1
+ after collect_arguments for a: `' 1 `b' 1
+ push `A' to input stack: `' 1 `b' 1
+ exit expand_macro for a: `' 1 `' 0
+ after collect_arguments for x: `A`a'' 1 `' 0
+ push `a(1)`'c(' to input stack: `A`a'' 1 `' 0
+ push_token saves $@(x) ref: `A`a'' 2 `' 0
+ exit expand_macro for x: `A`a'' 1 `' 0
+ expand_macro for a, level 0: `A`a'' 2 `' 0
+ after collect_arguments for a: `A`a''`1' 2 `' 0
+ push `A' to input stack: `A`a''`1' 2 `' 0
+ exit expand_macro for a: `A`a'' 1 `' 0
+ output `A': `A`a'' 1 `' 0
+ expand_macro for c, level 0: `A`a'' 2 `' 0
+ expand_argument gets $@(x) ref: `A`a''`$@(x)' 3 `' 0
+ pop_input ends $@(x) ref: `A`a''`$@(x)' 2 `' 0
+ expand_macro for y, level 1: `A`a''`$@(x)' 2 `' 1
+ after collect_arguments for y: `A`a''`$@(x)' 2 `b' 1
+ push_token saves $@(y) ref: `A`a''`$@(x)' 2 `b' 2
+ push `)' to input stack: `A`a''`$@(x)' 2 `b' 2
+ exit expand_macro for y: `A`a''`$@(x)' 2 `b' 1
+ expand_argument gets $@(y) ref: `A`a''`$@(x)$@(y)' 2 `b' 2
+ pop_input ends $@(y) ref: `A`a''`$@(x)$@(y)' 2 `b' 1
+ after collect_arguments for c: `A`a''`$@(x)$@(y)' 2 `b' 1
+ push_token saves $*(c) ref: `A`a''`$@(x)$@(y)' 3 `b' 2
+ expand_macro frees $@(x) ref: `A`a''`$@(x)$@(y)' 2 `b' 2
+ expand_macro frees $@(y) ref: `A`a''`$@(x)$@(y)' 2 `b' 1
+ exit expand_macro for c: `A`a''`$@(x)$@(y)' 1 `b' 1
+ output `Aa': `A`a''`$@(x)$@(y)' 0 `b' 1
+ pop_input ends $*(c)$@(x) ref: `' 0 `b' 1
+ expand_macro for b, level 0: `' 1 `b' 1
+ pop_input ends $*(c)$@(y) ref: `' 1 `' 0
+ after collect_arguments for b: `a' 1 `' 0
+ push `a(`' to input stack: `a' 1 `' 0
+ push_token saves $1(b) ref: `a' 2 `' 0
+ push `')' to input stack: `a' 2 `' 0
+ exit expand_macro for b: `a' 1 `' 0
+ expand_macro for a, level 0 : `a' 2 `' 0
+ expand_argument gets $1(b) ref: `a'`$1(b)' 3 `' 0
+ pop_input ends $1(b) ref: `a'`$1(b)' 2 `' 0
+ after collect_arguments for a: `a'`$1(b)' 2 `' 0
+ push `A' to input stack: `a'`$1(b)' 2 `' 0
+ expand_macro frees $1(b) ref: `a'`$1(b)' 1 `' 0
+ exit expand_macro for a: `' 0 `' 0
+ output `A': `' 0 `' 0
+
+ An obstack is only completely cleared when its refcount reaches
+ zero. However, as an optimization, expand_macro also frees
+ anything that it added to the obstack if no additional references
+ were added at the current expansion level, to reduce the amount of
+ memory left on the obstack while waiting for refcounts to drop.
+*/
+
static m4_macro_args *collect_arguments (m4 *, const char *, size_t,
- m4_symbol *, m4_obstack *);
+ m4_symbol *, m4_obstack *,
+ m4_obstack *);
static void expand_macro (m4 *, const char *, size_t, m4_symbol *);
static bool expand_token (m4 *, m4_obstack *, m4__token_type,
m4_symbol_value *, int, bool);
@@ -57,23 +151,24 @@ static size_t expansion_level = 0;
/* The number of the current call of expand_macro (). */
static size_t macro_call_id = 0;
-/* The shared stack of collected arguments for macro calls; as each
- argument is collected, it is finished and its location stored in
- argv_stack. This stack can be used simultaneously by multiple
- macro calls, using obstack_regrow to handle partial objects
- embedded in the stack. */
-static m4_obstack argc_stack;
-
-/* The shared stack of pointers to collected arguments for macro
- calls. This object is never finished; we exploit the fact that
- obstack_blank is documented to take a negative size to reduce the
- size again. */
-static m4_obstack argv_stack;
-
/* A placeholder symbol value representing the empty string, used to
optimize checks for emptiness. */
static m4_symbol_value empty_symbol;
+#if DEBUG_MACRO
+/* True if significant changes to stacks should be printed to the
+ trace stream. Primarily useful for debugging $@ ref memory leaks,
+ and controlled by M4_DEBUG_MACRO environment variable. */
+static int debug_macro_level;
+#else
+# define debug_macro_level 0
+#endif /* !DEBUG_MACRO */
+#define PRINT_ARGCOUNT_CHANGES 1 /* Any change to argcount > 1. */
+#define PRINT_REFCOUNT_INCREASE 2 /* Any increase to refcount. */
+#define PRINT_REFCOUNT_DECREASE 4 /* Any decrease to refcount. */
+
+
+
/* This function reads all input, and expands each token, one at a time. */
void
m4_macro_expand_input (m4 *context)
@@ -82,8 +177,11 @@ m4_macro_expand_input (m4 *context)
m4_symbol_value token;
int line;
- obstack_init (&argc_stack);
- obstack_init (&argv_stack);
+#if DEBUG_MACRO
+ const char *s = getenv ("M4_DEBUG_MACRO");
+ if (s)
+ debug_macro_level = strtol (s, NULL, 0);
+#endif /* DEBUG_MACRO */
m4_set_symbol_value_text (&empty_symbol, "", 0, 0);
VALUE_MAX_ARGS (&empty_symbol) = -1;
@@ -91,9 +189,6 @@ m4_macro_expand_input (m4 *context)
while ((type = m4__next_token (context, &token, &line, NULL))
!= M4_TOKEN_EOF)
expand_token (context, (m4_obstack *) NULL, type, &token, line, true);
-
- obstack_free (&argc_stack, NULL);
- obstack_free (&argv_stack, NULL);
}
@@ -295,17 +390,17 @@ expand_argument (m4 *context, m4_obstack *obs,
m4_symbol_value *argp,
static void
expand_macro (m4 *context, const char *name, size_t len, m4_symbol *symbol)
{
- void *argc_base = NULL; /* Base of argc_stack on entry. */
- void *argv_base = NULL; /* Base of argv_stack on entry. */
- unsigned int argc_size; /* Size of argc_stack on entry. */
- unsigned int argv_size; /* Size of argv_stack on entry. */
- m4_macro_args *argv;
- m4_obstack *expansion;
- m4_input_block *expanded;
- bool traced;
- bool trace_expansion = false;
- size_t my_call_id;
- m4_symbol_value *value;
+ void *args_base; /* Base of stack->args on entry. */
+ void *argv_base; /* Base of stack->argv on entry. */
+ m4_macro_args *argv; /* Arguments to the called macro. */
+ m4_obstack *expansion; /* Collects the macro's expansion. */
+ m4_input_block *expanded; /* The resulting expansion, for tracing. */
+ bool traced; /* True if this macro is traced. */
+ bool trace_expansion = false; /* True if trace and debugmode(`e'). */
+ size_t my_call_id; /* Sequence id for this macro. */
+ m4_symbol_value *value; /* Original value of this macro. */
+ size_t level = expansion_level; /* Expansion level of this macro. */
+ m4__macro_arg_stacks *stack; /* Storage for this macro. */
/* Report errors at the location where the open parenthesis (if any)
was found, but after expansion, restore global state back to the
@@ -318,6 +413,35 @@ expand_macro (m4 *context, const char *name, size_t len,
m4_symbol *symbol)
const char *loc_close_file;
int loc_close_line;
+ /* Obstack preparation. */
+ if (context->stacks_count <= level)
+ {
+ size_t count = context->stacks_count;
+ context->arg_stacks
+ = (m4__macro_arg_stacks *) x2nrealloc (context->arg_stacks,
+ &context->stacks_count,
+ sizeof *context->arg_stacks);
+ memset (&context->arg_stacks[count], 0,
+ sizeof *context->arg_stacks * (context->stacks_count - count));
+ }
+ stack = &context->arg_stacks[level];
+ if (!stack->args)
+ {
+ assert (!stack->refcount);
+ stack->args = xmalloc (sizeof *stack->args);
+ stack->argv = xmalloc (sizeof *stack->argv);
+ obstack_init (stack->args);
+ obstack_init (stack->argv);
+ stack->args_base = obstack_finish (stack->args);
+ stack->argv_base = obstack_finish (stack->argv);
+ }
+ assert (obstack_object_size (stack->args) == 0
+ && obstack_object_size (stack->argv) == 0);
+ args_base = obstack_finish (stack->args);
+ argv_base = obstack_finish (stack->argv);
+ m4__adjust_refcount (context, level, true);
+ stack->argcount++;
+
/* Grab the current value of this macro, because it may change while
collecting arguments. Likewise, grab any state needed during
tracing. */
@@ -333,21 +457,15 @@ expand_macro (m4 *context, const char *name, size_t len,
m4_symbol *symbol)
m4_error (context, EXIT_FAILURE, 0, NULL, _("\
recursion limit of %zu exceeded, use -L<N> to change it"),
m4_get_nesting_limit_opt (context));
-
- macro_call_id++;
- my_call_id = macro_call_id;
-
- argc_size = obstack_object_size (&argc_stack);
- argv_size = obstack_object_size (&argv_stack);
- argc_base = obstack_finish (&argc_stack);
- if (0 < argv_size)
- argv_base = obstack_finish (&argv_stack);
+ my_call_id = ++macro_call_id;
if (traced && m4_is_debug_bit (context, M4_DEBUG_TRACE_CALL))
trace_prepre (context, name, my_call_id, value);
- argv = collect_arguments (context, name, len, symbol, &argc_stack);
+ argv = collect_arguments (context, name, len, symbol, stack->args,
+ stack->argv);
+ /* The actual macro call. */
loc_close_file = m4_get_current_file (context);
loc_close_line = m4_get_current_line (context);
m4_set_current_file (context, loc_open_file);
@@ -363,6 +481,7 @@ recursion limit of %zu exceeded, use -L<N> to change it"),
if (traced)
trace_post (context, my_call_id, argv, expanded, trace_expansion);
+ /* Cleanup. */
m4_set_current_file (context, loc_close_file);
m4_set_current_line (context, loc_close_line);
@@ -371,15 +490,30 @@ recursion limit of %zu exceeded, use -L<N> to change it"),
if (BIT_TEST (VALUE_FLAGS (value), VALUE_DELETED_BIT))
m4_symbol_value_delete (value);
- /* TODO - pay attention to argv->inuse. */
- if (0 < argc_size)
- obstack_regrow (&argc_stack, argc_base, argc_size);
- else
- obstack_free (&argc_stack, argc_base);
- if (0 < argv_size)
- obstack_regrow (&argv_stack, argv_base, argv_size);
- else
- obstack_free (&argv_stack, argv);
+ /* If argv contains references, those refcounts can be reduced now. */
+ /* TODO - support references in argv. */
+
+ /* We no longer need argv, so reduce the refcount. Additionally, if
+ no other references to argv were created, we can free our portion
+ of the obstack, although we must leave earlier content alone. A
+ refcount of 0 implies that adjust_refcount already freed the
+ entire stack. */
+ if (m4__adjust_refcount (context, level, false))
+ {
+ if (argv->inuse)
+ {
+ if (debug_macro_level & PRINT_ARGCOUNT_CHANGES)
+ xfprintf (stderr, "m4debug: -%d- `%s' in use, level=%d, "
+ "refcount=%zu, argcount=%zu\n", my_call_id, argv->argv0,
+ level, stack->refcount, stack->argcount);
+ }
+ else
+ {
+ obstack_free (stack->args, args_base);
+ obstack_free (stack->argv, argv_base);
+ stack->argcount--;
+ }
+ }
}
/* Collect all the arguments to a call of the macro SYMBOL (called
@@ -389,7 +523,8 @@ recursion limit of %zu exceeded, use -L<N> to change it"),
arguments. */
static m4_macro_args *
collect_arguments (m4 *context, const char *name, size_t len,
- m4_symbol *symbol, m4_obstack *arguments)
+ m4_symbol *symbol, m4_obstack *arguments,
+ m4_obstack *argv_stack)
{
m4_symbol_value token;
m4_symbol_value *tokenp;
@@ -403,13 +538,13 @@ collect_arguments (m4 *context, const char *name, size_t
len,
args.argc = 1;
args.inuse = false;
args.has_ref = false;
- /* FIXME - add accessor to symtab that returns name from the hash
- table, so we don't have to copy it here. */
+ /* Must copy here, since we are consuming tokens, and since symbol
+ table can be changed during argument collection. */
args.argv0 = (char *) obstack_copy0 (arguments, name, len);
args.argv0_len = len;
args.quote_age = m4__quote_age (M4SYNTAX);
args.arraylen = 0;
- obstack_grow (&argv_stack, &args, offsetof (m4_macro_args, array));
+ obstack_grow (argv_stack, &args, offsetof (m4_macro_args, array));
name = args.argv0;
if (m4__next_token_is_open (context))
@@ -426,7 +561,7 @@ collect_arguments (m4 *context, const char *name, size_t
len,
else
tokenp = (m4_symbol_value *) obstack_copy (arguments, &token,
sizeof *tokenp);
- obstack_ptr_grow (&argv_stack, tokenp);
+ obstack_ptr_grow (argv_stack, tokenp);
args.arraylen++;
args.argc++;
/* Be conservative - any change in quoting while collecting
@@ -439,7 +574,7 @@ collect_arguments (m4 *context, const char *name, size_t
len,
}
while (more_args);
}
- argv = (m4_macro_args *) obstack_finish (&argv_stack);
+ argv = (m4_macro_args *) obstack_finish (argv_stack);
argv->argc = args.argc;
if (args.quote_age != m4__quote_age (M4SYNTAX))
argv->quote_age = 0;
@@ -545,7 +680,7 @@ process_macro (m4 *context, m4_symbol_value *value,
m4_obstack *obs,
{
size_t len = 0;
const char *endp;
- const char *key;
+ char *key;
for (endp = ++text;
*endp && m4_has_syntax (M4SYNTAX, *endp,
@@ -580,7 +715,7 @@ process_macro (m4 *context, m4_symbol_value *value,
m4_obstack *obs,
text = *endp ? 1 + endp : endp;
- free ((char *) key);
+ free (key);
break;
}
break;
@@ -755,6 +890,51 @@ trace_post (m4 *context, size_t id, m4_macro_args *argv,
/* Accessors into m4_macro_args. */
+/* Adjust the refcount of argument stack LEVEL. If INCREASE, then
+ increase the count, otherwise decrease the count and clear the
+ entire stack if the new count is zero. Return the new
+ refcount. */
+size_t
+m4__adjust_refcount (m4 *context, size_t level, bool increase)
+{
+ m4__macro_arg_stacks *stack = &context->arg_stacks[level];
+ assert (level < context->stacks_count && stack->args
+ && (increase || stack->refcount));
+ if (increase)
+ stack->refcount++;
+ else if (--stack->refcount == 0)
+ {
+ obstack_free (stack->args, stack->args_base);
+ obstack_free (stack->argv, stack->argv_base);
+ if ((debug_macro_level & PRINT_ARGCOUNT_CHANGES) && 1 < stack->argcount)
+ xfprintf (stderr, "m4debug: -%d- freeing %zu args, level=%d\n",
+ macro_call_id, stack->argcount, level);
+ stack->argcount = 0;
+ }
+ if (debug_macro_level
+ & (increase ? PRINT_REFCOUNT_INCREASE : PRINT_REFCOUNT_DECREASE))
+ xfprintf (stderr, "m4debug: level %d refcount=%d\n", level,
+ stack->refcount);
+ return stack->refcount;
+}
+
+/* Mark ARGV as being in use, along with any $@ references that it
+ wraps. */
+static void
+arg_mark (m4_macro_args *argv)
+{
+ argv->inuse = true;
+ if (argv->has_ref)
+ {
+ /* TODO for now we support only a single-length $@ chain. */
+ assert (argv->arraylen == 1
+ && argv->array[0]->type == M4_SYMBOL_COMP
+ && !argv->array[0]->u.chain->next
+ && !argv->array[0]->u.chain->str);
+ argv->array[0]->u.chain->argv->inuse = true;
+ }
+}
+
/* Given ARGV, return the symbol value at the specified INDEX, which
must be non-zero. */
m4_symbol_value *
@@ -891,15 +1071,16 @@ m4_arg_func (m4_macro_args *argv, unsigned int index)
FLATTEN, any builtins in ARGV are flattened to an empty string when
referenced through the new object. */
m4_macro_args *
-m4_make_argv_ref (m4_macro_args *argv, const char *argv0, size_t argv0_len,
- bool skip, bool flatten)
+m4_make_argv_ref (m4 *context, m4_macro_args *argv, const char *argv0,
+ size_t argv0_len, bool skip, bool flatten)
{
m4_macro_args *new_argv;
m4_symbol_value *value;
m4_symbol_chain *chain;
unsigned int index = skip ? 2 : 1;
+ m4_obstack *obs = context->arg_stacks[expansion_level - 1].argv;
- assert (obstack_object_size (&argv_stack) == 0);
+ assert (obstack_object_size (obs) == 0);
/* When making a reference through a reference, point to the
original if possible. */
if (argv->has_ref)
@@ -913,20 +1094,18 @@ m4_make_argv_ref (m4_macro_args *argv, const char
*argv0, size_t argv0_len,
}
if (argv->argc <= index)
{
- new_argv = (m4_macro_args *) obstack_alloc (&argv_stack,
- offsetof (m4_macro_args,
- array));
+ new_argv = (m4_macro_args *) obstack_alloc (obs, offsetof (m4_macro_args,
+ array));
new_argv->arraylen = 0;
new_argv->has_ref = false;
}
else
{
- new_argv = (m4_macro_args *) obstack_alloc (&argv_stack,
- (offsetof (m4_macro_args,
- array)
- + sizeof value));
- value = (m4_symbol_value *) obstack_alloc (&argv_stack, sizeof *value);
- chain = (m4_symbol_chain *) obstack_alloc (&argv_stack, sizeof *chain);
+ new_argv = (m4_macro_args *) obstack_alloc (obs, (offsetof
(m4_macro_args,
+ array)
+ + sizeof value));
+ value = (m4_symbol_value *) obstack_alloc (obs, sizeof *value);
+ chain = (m4_symbol_chain *) obstack_alloc (obs, sizeof *chain);
new_argv->arraylen = 1;
new_argv->array[0] = value;
new_argv->has_ref = true;
@@ -935,11 +1114,11 @@ m4_make_argv_ref (m4_macro_args *argv, const char
*argv0, size_t argv0_len,
chain->next = NULL;
chain->str = NULL;
chain->len = 0;
+ chain->level = expansion_level - 1;
chain->argv = argv;
chain->index = index;
chain->flatten = flatten;
}
- /* TODO - should argv->inuse be set? */
new_argv->argc = argv->argc - (index - 1);
new_argv->inuse = false;
new_argv->argv0 = argv0;
@@ -966,7 +1145,8 @@ m4_push_arg (m4 *context, m4_obstack *obs, m4_macro_args
*argv,
return;
/* TODO handle builtin tokens? */
assert (value->type == M4_SYMBOL_TEXT);
- m4__push_symbol (value, expansion_level - 1);
+ if (m4__push_symbol (value, expansion_level - 1))
+ arg_mark (argv);
}
/* Push series of comma-separated arguments from ARGV, which should
@@ -981,6 +1161,7 @@ m4_push_args (m4 *context, m4_obstack *obs, m4_macro_args
*argv, bool skip,
m4_symbol_value sep;
unsigned int i = skip ? 2 : 1;
bool use_sep = false;
+ bool inuse = false;
const char *lquote = m4_get_syntax_lquote (M4SYNTAX);
const char *rquote = m4_get_syntax_rquote (M4SYNTAX);
@@ -1017,10 +1198,12 @@ m4_push_args (m4 *context, m4_obstack *obs,
m4_macro_args *argv, bool skip,
use_sep = true;
/* TODO handle builtin tokens? */
assert (value->type == M4_SYMBOL_TEXT);
- m4__push_symbol (value, expansion_level - 1);
+ inuse |= m4__push_symbol (value, expansion_level - 1);
}
if (quote)
obstack_grow (obs, rquote, strlen (rquote));
+ if (inuse)
+ arg_mark (argv);
}
diff --git a/m4/system_.h b/m4/system_.h
index 64ca73c..f1e9602 100644
--- a/m4/system_.h
+++ b/m4/system_.h
@@ -49,14 +49,6 @@
#include <gnu/xprintf.h>
#include <gnu/xstrndup.h>
-/* glibc's obstack left out the ability to suspend and resume growth
- of an object on the stack. Reopen OBJECT (previously returned by
- obstack_alloc or obstack_finish) with SIZE for additional growth,
- freeing all objects that occur later in the stack. */
-#define obstack_regrow(OBS, OBJECT, SIZE) \
- (obstack_free (OBS, (char *) (OBJECT) + (SIZE)), \
- (OBS)->object_base = (char *) (OBJECT))
-
/* In addition to EXIT_SUCCESS and EXIT_FAILURE, m4 can fail with version
mismatch when trying to load a frozen file produced by a newer m4 than
the version doing the reload. */
diff --git a/modules/gnu.c b/modules/gnu.c
index 7205727..79ec5b2 100644
--- a/modules/gnu.c
+++ b/modules/gnu.c
@@ -446,8 +446,9 @@ M4BUILTIN_HANDLER (builtin)
{
m4_macro_args *new_argv;
bool flatten = (bp->flags & M4_BUILTIN_GROKS_MACRO) == 0;
- new_argv = m4_make_argv_ref (argv, name, m4_arg_len (argv, 1),
- true, flatten);
+ new_argv = m4_make_argv_ref (context, argv, name,
+ m4_arg_len (argv, 1),
+ true, flatten);
bp->func (context, obs, argc - 1, new_argv);
}
free (value);
@@ -681,8 +682,8 @@ M4BUILTIN_HANDLER (indir)
{
m4_macro_args *new_argv;
bool flatten = !m4_symbol_groks_macro (symbol);
- new_argv = m4_make_argv_ref (argv, name, m4_arg_len (argv, 1), true,
- flatten);
+ new_argv = m4_make_argv_ref (context, argv, name,
+ m4_arg_len (argv, 1), true, flatten);
m4_macro_call (context, m4_get_symbol_value (symbol), obs,
argc - 1, new_argv);
}
diff --git a/tests/builtins.at b/tests/builtins.at
index 91586b8..c68ab33 100644
--- a/tests/builtins.at
+++ b/tests/builtins.at
@@ -426,33 +426,48 @@ AT_CHECK_M4([in.m4 3>&-], [0], [[hello world
AT_CLEANUP
-## --- ##
-## exp ##
-## --- ##
-
-AT_SETUP([exp])
-
-AT_DATA([[exp.m4]],
-[[define(`countdown', `$1
-ifelse(eval($1 > 0), 1, `countdown(decr($1))', `Done')')dnl
-countdown(7)
-]])
+## ------ ##
+## ifelse ##
+## ------ ##
-AT_CHECK_M4([exp.m4], 0,
-[[7
-6
-5
-4
-3
-2
-1
-0
-Done
+AT_TEST_M4([ifelse],
+dnl ensure that comparisons work regardless of reference chains in the middle
+[[define(`e', `$@')define(`long', `01234567890123456789')
+ifelse(long, `01234567890123456789', `yes', `no')
+ifelse(`01234567890123456789', long, `yes', `no')
+ifelse(long, `01234567890123456789-', `yes', `no')
+ifelse(`01234567890123456789-', long, `yes', `no')
+ifelse(e(long), `01234567890123456789', `yes', `no')
+ifelse(`01234567890123456789', e(long), `yes', `no')
+ifelse(e(long), `01234567890123456789-', `yes', `no')
+ifelse(`01234567890123456789-', e(long), `yes', `no')
+ifelse(-e(long), `-01234567890123456789', `yes', `no')
+ifelse(-`01234567890123456789', -e(long), `yes', `no')
+ifelse(-e(long), `-01234567890123456789-', `yes', `no')
+ifelse(`-01234567890123456789-', -e(long), `yes', `no')
+ifelse(-e(long)-, `-01234567890123456789-', `yes', `no')
+ifelse(-`01234567890123456789-', -e(long)-, `yes', `no')
+ifelse(-e(long)-, `-01234567890123456789', `yes', `no')
+ifelse(`-01234567890123456789', -e(long)-, `yes', `no')
+]], [[
+yes
+yes
+no
+no
+yes
+yes
+no
+no
+yes
+yes
+no
+no
+yes
+yes
+no
+no
]])
-AT_CLEANUP
-
-
## ------- ##
## include ##
diff --git a/tests/others.at b/tests/others.at
index dd1e381..76a97eb 100644
--- a/tests/others.at
+++ b/tests/others.at
@@ -71,6 +71,32 @@ Macro foo expansion
]])
+## --------- ##
+## countdown ##
+## --------- ##
+
+AT_SETUP([countdown])
+
+AT_DATA([[exp.m4]],
+[[define(`countdown', `$1
+ifelse(eval($1 > 0), 1, `countdown(decr($1))', `Done')')dnl
+countdown(7)
+]])
+
+AT_CHECK_M4([exp.m4], 0,
+[[7
+6
+5
+4
+3
+2
+1
+0
+Done
+]])
+
+AT_CLEANUP
+
## ------- ##
## foreach ##
--
1.5.3.5
>From c947bf47f48f9132155bc623596771e4a4e3f732 Mon Sep 17 00:00:00 2001
From: Eric Blake <address@hidden>
Date: Thu, 25 Oct 2007 22:05:15 -0600
Subject: [PATCH] Stage 8: extend life of references into argv.
* src/m4.h (obstack_regrow): Delete, now that it is unused.
(struct token_chain): Add level field.
(push_token): Adjust prototype.
(adjust_refcount): New prototype.
* src/macro.c [DEBUG_MACRO]: Add debugging hooks for $@ reference
counting.
(argc_stack, argv_stack): Delete, replaced by...
(struct macro_arg_stacks, stacks, stacks_count): ...these new
structure for tracking $@ references.
(expand_input): Initialize new structure.
(collect_arguments): Alter signature.
(expand_macro): Rework obstack handling, in order to keep $@
references alive until they have been rescanned.
(adjust_refcount, arg_mark): New functions.
(make_argv_ref): Use new field.
(push_arg, push_args): Update callers.
* src/input.c (make_text_link): Use new field.
(push_token): Change signature.
(pop_input, next_char_1): Call new function.
* doc/m4.texinfo: Clean up some examples of -d option usage.
(Ifelse): Add some stress tests.
(cherry picked from commit b0daa4c96f734aab4d450594f64bc15d4321fd60)
Signed-off-by: Eric Blake <address@hidden>
---
ChangeLog | 25 +++++
doc/m4.texinfo | 47 ++++++++-
src/input.c | 16 ++-
src/m4.h | 12 +--
src/macro.c | 325 ++++++++++++++++++++++++++++++++++++++++++++++----------
5 files changed, 352 insertions(+), 73 deletions(-)
diff --git a/ChangeLog b/ChangeLog
index 14582ac..2b2a0d0 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,28 @@
+2007-12-18 Eric Blake <address@hidden>
+
+ Stage 8: extend life of references into argv.
+ * src/m4.h (obstack_regrow): Delete, now that it is unused.
+ (struct token_chain): Add level field.
+ (push_token): Adjust prototype.
+ (adjust_refcount): New prototype.
+ * src/macro.c [DEBUG_MACRO]: Add debugging hooks for $@ reference
+ counting.
+ (argc_stack, argv_stack): Delete, replaced by...
+ (struct macro_arg_stacks, stacks, stacks_count): ...these new
+ structure for tracking $@ references.
+ (expand_input): Initialize new structure.
+ (collect_arguments): Alter signature.
+ (expand_macro): Rework obstack handling, in order to keep $@
+ references alive until they have been rescanned.
+ (adjust_refcount, arg_mark): New functions.
+ (make_argv_ref): Use new field.
+ (push_arg, push_args): Update callers.
+ * src/input.c (make_text_link): Use new field.
+ (push_token): Change signature.
+ (pop_input, next_char_1): Call new function.
+ * doc/m4.texinfo: Clean up some examples of -d option usage.
+ (Ifelse): Add some stress tests.
+
2007-12-13 Eric Blake <address@hidden>
Yet more rewording.
diff --git a/doc/m4.texinfo b/doc/m4.texinfo
index 576a34c..eaec266 100644
--- a/doc/m4.texinfo
+++ b/doc/m4.texinfo
@@ -535,6 +535,7 @@ below as taking effect after any files that occurred
earlier in the
command line. The argument @option{--} is a marker to denote the end of
options.
address@hidden FIXME option -d+f only works on head right now...
With short options, options that do not take arguments may be combined
into a single command line argument with subsequent options, options
with mandatory arguments may be provided either as a single command line
@@ -2640,6 +2641,46 @@ ifelse(`foo', `bar', `3', `gnu', `gnats', `6', `7', `8')
@ignore
@comment Stress tests, not worth documenting.
+
address@hidden Ensure that references compared to strings work regardless of
address@hidden similar prefixes.
address@hidden
+define(`e', `$@@')define(`long', `01234567890123456789')
address@hidden
+ifelse(long, `01234567890123456789', `yes', `no')
address@hidden
+ifelse(`01234567890123456789', long, `yes', `no')
address@hidden
+ifelse(long, `01234567890123456789-', `yes', `no')
address@hidden
+ifelse(`01234567890123456789-', long, `yes', `no')
address@hidden
+ifelse(e(long), `01234567890123456789', `yes', `no')
address@hidden
+ifelse(`01234567890123456789', e(long), `yes', `no')
address@hidden
+ifelse(e(long), `01234567890123456789-', `yes', `no')
address@hidden
+ifelse(`01234567890123456789-', e(long), `yes', `no')
address@hidden
+ifelse(-e(long), `-01234567890123456789', `yes', `no')
address@hidden
+ifelse(-`01234567890123456789', -e(long), `yes', `no')
address@hidden
+ifelse(-e(long), `-01234567890123456789-', `yes', `no')
address@hidden
+ifelse(`-01234567890123456789-', -e(long), `yes', `no')
address@hidden
+ifelse(-e(long)-, `-01234567890123456789-', `yes', `no')
address@hidden
+ifelse(-`01234567890123456789-', -e(long)-, `yes', `no')
address@hidden
+ifelse(-e(long)-, `-01234567890123456789', `yes', `no')
address@hidden
+ifelse(`-01234567890123456789', -e(long)-, `yes', `no')
address@hidden
address@hidden example
+
@comment It would be nice to pass builtin tokens through ifelse, m4wrap,
@comment user macros; hence the fixmes.
@example
@@ -3237,6 +3278,7 @@ option @option{--nesting-limit} (or @option{-L},
@pxref{Limits control,
@option{-t}) can be used to invoke @code{traceon(@var{name})} before
parsing input.
address@hidden The explicit -dp neutralizes the testsuite default of -d.
@comment options: -dp -L3 -tifelse
@comment status: 1
@example
@@ -3399,6 +3441,7 @@ are reset to the default of @samp{aeq}.
The expansion of @code{debugmode} is void.
@end deffn
address@hidden The explicit -dp neutralizes the testsuite default of -d.
@comment options: -dp
@example
$ @kbd{m4}
@@ -3447,7 +3490,7 @@ The expansion of @code{debugfile} is void.
@end deffn
@example
-$ @kbd{m4}
+$ @kbd{m4 -d}
traceon(`divnum')
@result{}
divnum(`extra')
@@ -7070,7 +7113,7 @@ repeating the side effects of unquoted list elements will
have any
detrimental effects.
@example
-$ @kbd{m4 -I examples}
+$ @kbd{m4 -d -I examples}
include(`foreach2.m4')
@result{}
include(`foreachq2.m4')
diff --git a/src/input.c b/src/input.c
index 5c87217..8386061 100644
--- a/src/input.c
+++ b/src/input.c
@@ -215,6 +215,7 @@ make_text_link (struct obstack *obs, token_chain **start,
token_chain **end)
chain->next = NULL;
chain->str = str;
chain->len = len;
+ chain->level = -1;
chain->argv = NULL;
chain->index = 0;
chain->flatten = false;
@@ -317,9 +318,10 @@ push_string_init (void)
| current_input stack and TOKEN lives in temporary storage. Allows |
| gathering input from multiple locations, rather than copying |
| everything consecutively onto the input stack. Must be called |
-| between push_string_init and push_string_finish. |
+| between push_string_init and push_string_finish. Return true only |
+| if LEVEL is non-negative, and a reference was created to TOKEN. |
`-------------------------------------------------------------------*/
-void
+bool
push_token (token_data *token, int level)
{
token_chain *chain;
@@ -328,7 +330,7 @@ push_token (token_data *token, int level)
/* TODO - also accept TOKEN_COMP chains. */
assert (TOKEN_DATA_TYPE (token) == TOKEN_TEXT);
if (TOKEN_DATA_LEN (token) == 0)
- return;
+ return false;
if (next->type == INPUT_STRING)
{
@@ -353,9 +355,11 @@ push_token (token_data *token, int level)
else
chain->str = TOKEN_DATA_TEXT (token);
chain->len = TOKEN_DATA_LEN (token);
+ chain->level = -1;
chain->argv = NULL;
chain->index = 0;
chain->flatten = false;
+ return false; /* No reference exists when text is copied. */
}
/*-------------------------------------------------------------------.
@@ -469,7 +473,9 @@ pop_input (bool cleanup)
assert (!"implemented yet");
abort ();
}
- chain = chain->next;
+ if (chain->level >= 0)
+ adjust_refcount (chain->level, false);
+ isp->u.u_c.chain = chain = chain->next;
}
break;
@@ -760,6 +766,8 @@ next_char_1 (void)
assert (!"implemented yet");
abort ();
}
+ if (chain->level >= 0)
+ adjust_refcount (chain->level, false);
isp->u.u_c.chain = chain = chain->next;
}
break;
diff --git a/src/m4.h b/src/m4.h
index 111f167..854f28d 100644
--- a/src/m4.h
+++ b/src/m4.h
@@ -88,14 +88,6 @@ typedef struct string STRING;
#define obstack_chunk_alloc xmalloc
#define obstack_chunk_free free
-/* glibc's obstack left out the ability to suspend and resume growth
- of an object on the stack. Reopen OBJECT (previously returned by
- obstack_alloc or obstack_finish) with SIZE for additional growth,
- freeing all objects that occur later in the stack. */
-#define obstack_regrow(OBS, OBJECT, SIZE) \
- (obstack_free (OBS, (char *) (OBJECT) + (SIZE)), \
- (OBS)->object_base = (char *) (OBJECT))
-
/* These must come first. */
typedef struct input_block input_block;
typedef struct token_data token_data;
@@ -286,6 +278,7 @@ struct token_chain
token_chain *next; /* Pointer to next link of chain. */
const char *str; /* NUL-terminated string if text, else NULL. */
size_t len; /* Length of str, else 0. */
+ int level; /* Expansion level of link content, or -1. */
macro_arguments *argv;/* Reference to earlier address@hidden */
unsigned int index; /* Argument index within argv. */
bool flatten; /* True to treat builtins as text. */
@@ -350,7 +343,7 @@ void skip_line (const char *);
void push_file (FILE *, const char *, bool);
void push_macro (builtin_func *);
struct obstack *push_string_init (void);
-void push_token (token_data *, int);
+bool push_token (token_data *, int);
const input_block *push_string_finish (void);
void push_wrapup (const char *);
bool pop_wrapup (void);
@@ -460,6 +453,7 @@ macro_arguments *make_argv_ref (macro_arguments *, const
char *, size_t,
bool, bool);
void push_arg (struct obstack *, macro_arguments *, unsigned int);
void push_args (struct obstack *, macro_arguments *, bool, bool);
+size_t adjust_refcount (int, bool);
/* File: builtin.c --- builtins. */
diff --git a/src/macro.c b/src/macro.c
index 1e4b271..63d957d 100644
--- a/src/macro.c
+++ b/src/macro.c
@@ -24,6 +24,10 @@
#include "m4.h"
+#ifndef DEBUG_MACRO
+# define DEBUG_MACRO 0
+#endif /* DEBUG_MACRO */
+
/* Opaque structure describing all arguments to a macro, including the
macro name at index 0. */
struct macro_arguments
@@ -32,9 +36,9 @@ struct macro_arguments
arraylen since the array can refer to multiple arguments via a
single $@ reference. */
unsigned int argc;
- /* False unless the macro expansion refers to $@, determines whether
- this object can be freed at end of macro expansion or must wait
- until next byte read from file. */
+ /* False unless the macro expansion refers to $@; determines whether
+ this object can be freed immediately at the end of expand_macro,
+ or must wait until all recursion has completed. */
bool_bitfield inuse : 1;
/* False if all arguments are just text or func, true if this argv
refers to another one. */
@@ -51,32 +55,134 @@ struct macro_arguments
token_data *array[FLEXIBLE_ARRAY_MEMBER];
};
+/* Internal struct to maintain list of argument stacks. Within a
+ recursion level, consecutive macros can share a stack, but distinct
+ recursion levels need different stacks since the nested macro is
+ interrupting the argument collection of the outer level. Note that
+ a reference can live as long as the expansion containing the
+ reference can participate as an argument in a future macro call.
+
+ Therefore, we implement a reference counter for each expansion
+ level, tracking how many references exist into the obstack, as well
+ as associate a level with each reference. Of course, expand_macro
+ is actively using argv, so it increments the refcount on entry and
+ decrements it on exit. Additionally, any time the input engine is
+ handed a reference that it does not inline, it increases the
+ refcount in push_token, then decreases it in pop_input once the
+ reference has been rescanned. Finally, when the input engine hands
+ a reference back to expand_argument, the refcount increases, which
+ is then cleaned up at the end of expand_macro.
+
+ For a running example, consider this input:
+
+ define(a,A)define(b,`a(`$1')')define(c,$*)dnl
+ define(x,`a(1)`'c($@')define(y,`$@)')dnl
+ x(a(`b')``a'')y(`b')(`a')
+ => AAaA
+
+ Assuming all arguments are large enough to exceed the inlining
+ thresholds of the input engine, the interesting sequence of events
+ is as follows:
+
+ stacks[0] refs stacks[1] refs
+ after second dnl ends: `' 0 `' 0
+ expand_macro for x, level 0: `' 1 `' 0
+ expand_macro for a, level 1: `' 1 `' 1
+ after collect_arguments for a: `' 1 `b' 1
+ push `A' to input stack: `' 1 `b' 1
+ exit expand_macro for a: `' 1 `' 0
+ after collect_arguments for x: `A`a'' 1 `' 0
+ push `a(1)`'c(' to input stack: `A`a'' 1 `' 0
+ push_token saves $@(x) ref: `A`a'' 2 `' 0
+ exit expand_macro for x: `A`a'' 1 `' 0
+ expand_macro for a, level 0: `A`a'' 2 `' 0
+ after collect_arguments for a: `A`a''`1' 2 `' 0
+ push `A' to input stack: `A`a''`1' 2 `' 0
+ exit expand_macro for a: `A`a'' 1 `' 0
+ output `A': `A`a'' 1 `' 0
+ expand_macro for c, level 0: `A`a'' 2 `' 0
+ expand_argument gets $@(x) ref: `A`a''`$@(x)' 3 `' 0
+ pop_input ends $@(x) ref: `A`a''`$@(x)' 2 `' 0
+ expand_macro for y, level 1: `A`a''`$@(x)' 2 `' 1
+ after collect_arguments for y: `A`a''`$@(x)' 2 `b' 1
+ push_token saves $@(y) ref: `A`a''`$@(x)' 2 `b' 2
+ push `)' to input stack: `A`a''`$@(x)' 2 `b' 2
+ exit expand_macro for y: `A`a''`$@(x)' 2 `b' 1
+ expand_argument gets $@(y) ref: `A`a''`$@(x)$@(y)' 2 `b' 2
+ pop_input ends $@(y) ref: `A`a''`$@(x)$@(y)' 2 `b' 1
+ after collect_arguments for c: `A`a''`$@(x)$@(y)' 2 `b' 1
+ push_token saves $*(c) ref: `A`a''`$@(x)$@(y)' 3 `b' 2
+ expand_macro frees $@(x) ref: `A`a''`$@(x)$@(y)' 2 `b' 2
+ expand_macro frees $@(y) ref: `A`a''`$@(x)$@(y)' 2 `b' 1
+ exit expand_macro for c: `A`a''`$@(x)$@(y)' 1 `b' 1
+ output `Aa': `A`a''`$@(x)$@(y)' 0 `b' 1
+ pop_input ends $*(c)$@(x) ref: `' 0 `b' 1
+ expand_macro for b, level 0: `' 1 `b' 1
+ pop_input ends $*(c)$@(y) ref: `' 1 `' 0
+ after collect_arguments for b: `a' 1 `' 0
+ push `a(`' to input stack: `a' 1 `' 0
+ push_token saves $1(b) ref: `a' 2 `' 0
+ push `')' to input stack: `a' 2 `' 0
+ exit expand_macro for b: `a' 1 `' 0
+ expand_macro for a, level 0 : `a' 2 `' 0
+ expand_argument gets $1(b) ref: `a'`$1(b)' 3 `' 0
+ pop_input ends $1(b) ref: `a'`$1(b)' 2 `' 0
+ after collect_arguments for a: `a'`$1(b)' 2 `' 0
+ push `A' to input stack: `a'`$1(b)' 2 `' 0
+ expand_macro frees $1(b) ref: `a'`$1(b)' 1 `' 0
+ exit expand_macro for a: `' 0 `' 0
+ output `A': `' 0 `' 0
+
+ An obstack is only completely cleared when its refcount reaches
+ zero. However, as an optimization, expand_macro also frees
+ anything that it added to the obstack if no additional references
+ were added at the current expansion level, to reduce the amount of
+ memory left on the obstack while waiting for refcounts to drop.
+*/
+struct macro_arg_stacks
+{
+ size_t refcount; /* Number of active $@ references at this level. */
+ size_t argcount; /* Number of argv at this level. */
+ struct obstack *args; /* Content of arguments. */
+ struct obstack *argv; /* Argv pointers into args. */
+ void *args_base; /* Location for clearing the args obstack. */
+ void *argv_base; /* Location for clearing the argv obstack. */
+};
+
+typedef struct macro_arg_stacks macro_arg_stacks;
+
static void expand_macro (symbol *);
static bool expand_token (struct obstack *, token_type, token_data *, int,
bool);
+/* Array of stacks, one element per macro recursion level. */
+static macro_arg_stacks *stacks;
+
+/* Current size of stacks array. */
+static size_t stacks_count;
+
/* Current recursion level in expand_macro (). */
int expansion_level = 0;
/* The number of the current call of expand_macro (). */
static int macro_call_id = 0;
-/* The shared stack of collected arguments for macro calls; as each
- argument is collected, it is finished and its location stored in
- argv_stack. This stack can be used simultaneously by multiple
- macro calls, using obstack_regrow to handle partial objects
- embedded in the stack. */
-static struct obstack argc_stack;
-
-/* The shared stack of pointers to collected arguments for macro
- calls. This stack can be used simultaneously by multiple macro
- calls, using obstack_regrow to handle partial objects embedded in
- the stack. */
-static struct obstack argv_stack;
-
/* The empty string token. */
static token_data empty_token;
+#if DEBUG_MACRO
+/* True if significant changes to stacks should be printed to the
+ trace stream. Primarily useful for debugging $@ ref memory leaks,
+ and controlled by M4_DEBUG_MACRO environment variable. */
+static int debug_macro_level;
+#else
+# define debug_macro_level 0
+#endif /* !DEBUG_MACRO */
+#define PRINT_ARGCOUNT_CHANGES 1
+#define PRINT_REFCOUNT_INCREASE 2
+#define PRINT_REFCOUNT_DECREASE 4
+
+
/*----------------------------------------------------------------.
| This function reads all input, and expands each token, one at a |
| time. |
@@ -88,9 +194,13 @@ expand_input (void)
token_type t;
token_data td;
int line;
+ size_t i;
- obstack_init (&argc_stack);
- obstack_init (&argv_stack);
+#if DEBUG_MACRO
+ const char *s = getenv ("M4_DEBUG_MACRO");
+ if (s)
+ debug_macro_level = strtol (s, NULL, 0);
+#endif /* DEBUG_MACRO */
TOKEN_DATA_TYPE (&empty_token) = TOKEN_TEXT;
TOKEN_DATA_TEXT (&empty_token) = "";
@@ -102,8 +212,20 @@ expand_input (void)
while ((t = next_token (&td, &line, NULL)) != TOKEN_EOF)
expand_token ((struct obstack *) NULL, t, &td, line, true);
- obstack_free (&argc_stack, NULL);
- obstack_free (&argv_stack, NULL);
+ for (i = 0; i < stacks_count; i++)
+ {
+ assert (stacks[i].refcount == 0 && stacks[i].argcount == 0);
+ if (stacks[i].args)
+ {
+ obstack_free (stacks[i].args, NULL);
+ free (stacks[i].args);
+ obstack_free (stacks[i].argv, NULL);
+ free (stacks[i].argv);
+ }
+ }
+ free (stacks);
+ stacks = NULL;
+ stacks_count = 0;
}
@@ -317,7 +439,8 @@ expand_argument (struct obstack *obs, token_data *argp,
const char *caller)
`-------------------------------------------------------------------------*/
static macro_arguments *
-collect_arguments (symbol *sym, struct obstack *arguments)
+collect_arguments (symbol *sym, struct obstack *arguments,
+ struct obstack *argv_stack)
{
token_data td;
token_data *tdp;
@@ -333,7 +456,7 @@ collect_arguments (symbol *sym, struct obstack *arguments)
args.argv0_len = strlen (args.argv0);
args.quote_age = quote_age ();
args.arraylen = 0;
- obstack_grow (&argv_stack, &args, offsetof (macro_arguments, array));
+ obstack_grow (argv_stack, &args, offsetof (macro_arguments, array));
if (peek_token () == TOKEN_OPEN)
{
@@ -347,7 +470,7 @@ collect_arguments (symbol *sym, struct obstack *arguments)
tdp = &empty_token;
else
tdp = (token_data *) obstack_copy (arguments, &td, sizeof td);
- obstack_ptr_grow (&argv_stack, tdp);
+ obstack_ptr_grow (argv_stack, tdp);
args.arraylen++;
args.argc++;
/* Be conservative - any change in quoting while collecting
@@ -360,7 +483,7 @@ collect_arguments (symbol *sym, struct obstack *arguments)
}
while (more_args);
}
- argv = (macro_arguments *) obstack_finish (&argv_stack);
+ argv = (macro_arguments *) obstack_finish (argv_stack);
argv->argc = args.argc;
if (args.quote_age != quote_age ())
argv->quote_age = 0;
@@ -409,15 +532,14 @@ call_macro (symbol *sym, int argc, macro_arguments *argv,
static void
expand_macro (symbol *sym)
{
- void *argc_base = NULL; /* Base of argc_stack on entry. */
- void *argv_base = NULL; /* Base of argv_stack on entry. */
- unsigned int argc_size; /* Size of argc_stack on entry. */
- unsigned int argv_size; /* Size of argv_stack on entry. */
- macro_arguments *argv;
- struct obstack *expansion;
- const input_block *expanded;
- bool traced;
- int my_call_id;
+ void *args_base; /* Base of stacks[i].args on entry. */
+ void *argv_base; /* Base of stacks[i].argv on entry. */
+ macro_arguments *argv; /* Arguments to the called macro. */
+ struct obstack *expansion; /* Collects the macro's expansion. */
+ const input_block *expanded; /* The resulting expansion, for tracing. */
+ bool traced; /* True if this macro is traced. */
+ int my_call_id; /* Sequence id for this macro. */
+ int level = expansion_level; /* Expansion level of this macro. */
/* Report errors at the location where the open parenthesis (if any)
was found, but after expansion, restore global state back to the
@@ -430,6 +552,33 @@ expand_macro (symbol *sym)
const char *loc_close_file;
int loc_close_line;
+ /* Obstack preparation. */
+ if (level >= stacks_count)
+ {
+ size_t old_count = stacks_count;
+ stacks = (macro_arg_stacks *) x2nrealloc (stacks, &stacks_count,
+ sizeof *stacks);
+ memset (&stacks[old_count], 0,
+ sizeof *stacks * (stacks_count - old_count));
+ }
+ if (!stacks[level].args)
+ {
+ assert (!stacks[level].refcount);
+ stacks[level].args = xmalloc (sizeof (struct obstack));
+ stacks[level].argv = xmalloc (sizeof (struct obstack));
+ obstack_init (stacks[level].args);
+ obstack_init (stacks[level].argv);
+ stacks[level].args_base = obstack_finish (stacks[level].args);
+ stacks[level].argv_base = obstack_finish (stacks[level].argv);
+ }
+ assert (obstack_object_size (stacks[level].args) == 0
+ && obstack_object_size (stacks[level].argv) == 0);
+ args_base = obstack_finish (stacks[level].args);
+ argv_base = obstack_finish (stacks[level].argv);
+ adjust_refcount (level, true);
+ stacks[level].argcount++;
+
+ /* Prepare for argument collection. */
SYMBOL_PENDING_EXPANSIONS (sym)++;
expansion_level++;
if (nesting_limit > 0 && expansion_level > nesting_limit)
@@ -441,18 +590,12 @@ expand_macro (symbol *sym)
my_call_id = macro_call_id;
traced = (debug_level & DEBUG_TRACE_ALL) || SYMBOL_TRACED (sym);
-
- argc_size = obstack_object_size (&argc_stack);
- argv_size = obstack_object_size (&argv_stack);
- argc_base = obstack_finish (&argc_stack);
- if (0 < argv_size)
- argv_base = obstack_finish (&argv_stack);
-
if (traced && (debug_level & DEBUG_TRACE_CALL))
trace_prepre (SYMBOL_NAME (sym), my_call_id);
- argv = collect_arguments (sym, &argc_stack);
+ argv = collect_arguments (sym, stacks[level].args, stacks[level].argv);
+ /* The actual macro call. */
loc_close_file = current_file;
loc_close_line = current_line;
current_file = loc_open_file;
@@ -468,6 +611,7 @@ expand_macro (symbol *sym)
if (traced)
trace_post (SYMBOL_NAME (sym), my_call_id, argv, expanded);
+ /* Cleanup. */
current_file = loc_close_file;
current_line = loc_close_line;
@@ -477,15 +621,58 @@ expand_macro (symbol *sym)
if (SYMBOL_DELETED (sym))
free_symbol (sym);
- /* TODO pay attention to argv->inuse, in case someone is depending on
address@hidden */
- if (0 < argc_size)
- obstack_regrow (&argc_stack, argc_base, argc_size);
- else
- obstack_free (&argc_stack, argc_base);
- if (0 < argv_size)
- obstack_regrow (&argv_stack, argv_base, argv_size);
- else
- obstack_free (&argv_stack, argv);
+ /* If argv contains references, those refcounts can be reduced now. */
+ /* TODO - support references in argv. */
+
+ /* We no longer need argv, so reduce the refcount. Additionally, if
+ no other references to argv were created, we can free our portion
+ of the obstack, although we must leave earlier content alone. A
+ refcount of 0 implies that adjust_refcount already freed the
+ entire stack. */
+ if (adjust_refcount (level, false))
+ {
+ if (argv->inuse)
+ {
+ if (debug_macro_level & PRINT_ARGCOUNT_CHANGES)
+ xfprintf (debug, "m4debug: -%d- `%s' in use, level=%d, "
+ "refcount=%zu, argcount=%zu\n", my_call_id, argv->argv0,
+ level, stacks[level].refcount, stacks[level].argcount);
+ }
+ else
+ {
+ obstack_free (stacks[level].args, args_base);
+ obstack_free (stacks[level].argv, argv_base);
+ stacks[level].argcount--;
+ }
+ }
+}
+
+/* Adjust the refcount of argument stack LEVEL. If INCREASE, then
+ increase the count, otherwise decrease the count and clear the
+ entire stack if the new count is zero. Return the new
+ refcount. */
+size_t
+adjust_refcount (int level, bool increase)
+{
+ assert (level >= 0 && level < stacks_count && stacks[level].args);
+ assert (increase || stacks[level].refcount);
+ if (increase)
+ stacks[level].refcount++;
+ else if (--stacks[level].refcount == 0)
+ {
+ obstack_free (stacks[level].args, stacks[level].args_base);
+ obstack_free (stacks[level].argv, stacks[level].argv_base);
+ if ((debug_macro_level & PRINT_ARGCOUNT_CHANGES)
+ && stacks[level].argcount > 1)
+ xfprintf (debug, "m4debug: -%d- freeing %zu args, level=%d\n",
+ macro_call_id, stacks[level].argcount, level);
+ stacks[level].argcount = 0;
+ }
+ if (debug_macro_level
+ & (increase ? PRINT_REFCOUNT_INCREASE : PRINT_REFCOUNT_DECREASE))
+ xfprintf (debug, "m4debug: level %d refcount=%d\n", level,
+ stacks[level].refcount);
+ return stacks[level].refcount;
}
@@ -525,6 +712,23 @@ arg_token (macro_arguments *argv, unsigned int index)
return token;
}
+/* Mark ARGV as being in use, along with any $@ references that it
+ wraps. */
+static void
+arg_mark (macro_arguments *argv)
+{
+ argv->inuse = true;
+ if (argv->has_ref)
+ {
+ /* TODO for now we support only a single-length $@ chain. */
+ assert (argv->arraylen == 1
+ && TOKEN_DATA_TYPE (argv->array[0]) == TOKEN_COMP
+ && !argv->array[0]->u.chain->next
+ && !argv->array[0]->u.chain->str);
+ argv->array[0]->u.chain->argv->inuse = true;
+ }
+}
+
/* Given ARGV, return how many arguments it refers to. */
unsigned int
arg_argc (macro_arguments *argv)
@@ -661,8 +865,9 @@ make_argv_ref (macro_arguments *argv, const char *argv0,
size_t argv0_len,
token_data *token;
token_chain *chain;
unsigned int index = skip ? 2 : 1;
+ struct obstack *obs = stacks[expansion_level - 1].argv;
- assert (obstack_object_size (&argv_stack) == 0);
+ assert (obstack_object_size (obs) == 0);
/* When making a reference through a reference, point to the
original if possible. */
if (argv->has_ref)
@@ -678,17 +883,17 @@ make_argv_ref (macro_arguments *argv, const char *argv0,
size_t argv0_len,
if (argv->argc <= index)
{
new_argv = (macro_arguments *)
- obstack_alloc (&argv_stack, offsetof (macro_arguments, array));
+ obstack_alloc (obs, offsetof (macro_arguments, array));
new_argv->arraylen = 0;
new_argv->has_ref = false;
}
else
{
new_argv = (macro_arguments *)
- obstack_alloc (&argv_stack,
+ obstack_alloc (obs,
offsetof (macro_arguments, array) + sizeof token);
- token = (token_data *) obstack_alloc (&argv_stack, sizeof *token);
- chain = (token_chain *) obstack_alloc (&argv_stack, sizeof *chain);
+ token = (token_data *) obstack_alloc (obs, sizeof *token);
+ chain = (token_chain *) obstack_alloc (obs, sizeof *chain);
new_argv->arraylen = 1;
new_argv->array[0] = token;
new_argv->has_ref = true;
@@ -697,11 +902,11 @@ make_argv_ref (macro_arguments *argv, const char *argv0,
size_t argv0_len,
chain->next = NULL;
chain->str = NULL;
chain->len = 0;
+ chain->level = expansion_level - 1;
chain->argv = argv;
chain->index = index;
chain->flatten = flatten;
}
- /* TODO - should argv->inuse be set? */
new_argv->argc = argv->argc - (index - 1);
new_argv->inuse = false;
new_argv->argv0 = argv0;
@@ -727,7 +932,8 @@ push_arg (struct obstack *obs, macro_arguments *argv,
unsigned int index)
token = arg_token (argv, index);
/* TODO handle func tokens? */
assert (TOKEN_DATA_TYPE (token) == TOKEN_TEXT);
- push_token (token, expansion_level - 1);
+ if (push_token (token, expansion_level - 1))
+ arg_mark (argv);
}
/* Push series of comma-separated arguments from ARGV, which should
@@ -741,6 +947,7 @@ push_args (struct obstack *obs, macro_arguments *argv, bool
skip, bool quote)
token_data sep;
unsigned int i = skip ? 2 : 1;
bool use_sep = false;
+ bool inuse = false;
static char comma[2] = ",";
if (i >= argv->argc)
@@ -779,8 +986,10 @@ push_args (struct obstack *obs, macro_arguments *argv,
bool skip, bool quote)
use_sep = true;
/* TODO handle func tokens? */
assert (TOKEN_DATA_TYPE (token) == TOKEN_TEXT);
- push_token (token, expansion_level - 1);
+ inuse |= push_token (token, expansion_level - 1);
}
if (quote)
obstack_grow (obs, rquote.string, rquote.length);
+ if (inuse)
+ arg_mark (argv);
}
--
1.5.3.5
| [Prev in Thread] |
Current Thread |
[Next in Thread] |
- [8/18] argv_ref speedup: lengthen lifetime of $@ references,
Eric Blake <=