Re: M4 syntax $11 vs. ${11}

[Eric Blake]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

According to Eric Blake on 1/30/2007 7:23 AM:
> 
> Let me think on this for a couple more days (as it is, I still haven't
> completely implemented extended arguments in CVS m4; partly because I want
> to do it right, so I want this thread to come to a consensus before I
> commit to coding something), and get more feedback.  But after typing and
> re-reading this email, I'm starting to lean towards the following:
> - in m4 1.4.9: make --warn-syntax (or whatever name we decide on) take an
> optional argument of a regular expression to warn about if encountered in
> macro definitions, with the default regex being \$[{0-9][0-9].

Here's what I'm checking in for 1.4.9.  Anyone who has been testing the
- --warn-syntax option (available only in CVS for a couple of weeks) now
needs to test with --warn-macro-sequence.  I'm hoping this is the last
change prior to 1.4.9, but will not release it for a couple of weeks to
make sure things settle first.

2007-02-08  Eric Blake  <address@hidden>

        Rename --warn-syntax to --warn-macro-sequence[=regex], to make it
        more flexible, and so that autoconf can use it.
        * src/m4.h (set_macro_sequence, free_macro_sequence): New
        prototypes.
        * src/builtin.c (macro_sequence_buf, macro_sequence_regs)
        (macro_sequence_inuse, set_macro_sequence, free_macro_sequence):
        New variables and functions.
        (define_user_macro): Allow flexibility in regular expression used
        to trigger warning.
        * src/m4.c (warn_syntax): Delete.
        (usage, WARN_MACRO_SEQUENCE_OPTION, main): Implement changed
        spelling of option, along with optional argument.
        * doc/m4.texinfo (Operation modes, Arguments): Document this
        change.
        * NEWS: Document this change.

> - in m4 2.0: have ${1} be the out-of-the-box syntax for extended
> arguments, but add a new macro changeextarg that can be used to recognize
> ${{1}} instead

Still to be implemented (and it may be a couple of weeks, because I'm
about to head out on a two-week vacation).  I think once I get that
implemented, it will be time to release beta 1.9b.  But I am planning on
adding changeextarg({{,}}) as the means to print ${1} literally but treat
${{1}} like $1.

> - in autoconf 2.62: if --warn-syntax is supported by the installed m4,
> then turn it on with a regular expression to reject $10 and ${{1}},
> neither of which seem to be in many current autoconf snippets; and in
> m4sugar, if changesyntax is supported, then disable extended arguments;
> that way, ${1} is reserved for the shell no matter what

Here's an updated patch that lets autoconf 2.62 exploit
- --warn-macro-sequence to warn users about $10 and ${{, as well as avoid
false positives for ${1} even if the user does M4=path/to/m4\
- --warn-macro-sequence.  Plus, it completes last week's patch to require m4
1.4.5 or greater.  It does not tackle whether $[1] or $[]1 should be the
preferred style for escaping literal shell variables, which was a
secondary point raised by Ralf.  And I verified that I get no false
positives (and therefore don't have to change any .m4 files) in autoconf.

2007-02-08  Eric Blake  <address@hidden>

        * bin/autom4te.in (m4): Require GNU M4 1.4.5 or better.  Use
        --warn-macro-sequence if it is supported.  For now, warn users
        about ${{1}}, since we don't depend on M4 2.0.
        * lib/m4sugar/m4sugar.m4: Tell M4 2.0 to use ${{1}}, not ${1},
        for extended parameters.
        * doc/autoconf.texi (Quoting and Parameters): New section.
        (Quotation and Nested Macros): Touch up example.
        * NEWS: Document the new limitation on user macros not exceeding
        $9.

> - in autoconf 3.0: require m4 2.0, no longer use --warn-syntax, and use
> changeextarg to support ${{1}} m4 extended arguments
> 

We'll just leave allowing ${{1}} for the future :)

- --
Don't work too hard, make some time for fun as well!

Eric Blake             address@hidden
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Cygwin)
Comment: Public key at home.comcast.net/~ericblake/eblake.gpg
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFFyzka84KuGfSFAYARAiGSAKCnwZkcsIad3ZlyeCl/MrhoHvWRVQCgi+CH
qJxUpzaObENgnbgZUSJxsrc=
=Ciwu
-----END PGP SIGNATURE-----

Index: NEWS
===================================================================
RCS file: /sources/m4/m4/NEWS,v
retrieving revision 1.1.1.1.2.92
diff -u -p -r1.1.1.1.2.92 NEWS
--- NEWS        2 Feb 2007 02:55:11 -0000       1.1.1.1.2.92
+++ NEWS        8 Feb 2007 14:42:13 -0000
@@ -20,15 +20,19 @@ Version 1.4.9 - ?? ??? 2007, by ????  (C
   continues, so that you can see all warnings instead of fixing them one
   at a time.  To acheive 1.4.8 behavior, where the first warning
   immediately exits, specify -E twice on the command line.
-* A new `--warn-syntax' command-line option allows detection of
-  non-portable syntax that might be broken when upgrading to M4 2.0.  For
-  example, POSIX requires a macro definition containing `$11' to expand to
-  the first argument concatenated with 1, rather than the eleventh
-  argument; and allows implementations to choose whether `${11}' is treated
-  as literal text, as in M4 1.4.x, or as the eleventh argument, as in the
-  eventual M4 2.0.  Be aware that Autoconf 2.61 will not work with this
-  option enabled.
-* Improved portability to platforms such as BSD/OS.
+* A new `--warn-macro-sequence' command-line option allows detection of
+  sequences in `define' and `pushdef' definitions that match an optional
+  regular expression.  The default regular expression is
+  `\$\({[0-9][^}]*}\|[0-9][0-9]+\)', corresponding to the sequences that
+  might not behave correctly when upgrading to the eventual M4 2.0.  By
+  default, M4 2.0 will follow the POSIX requirement that a macro definition
+  containing `$11' must expand to the first argument concatenated with 1,
+  rather than the eleventh argument; and will take advantage of the POSIX
+  wording that allows implementations to treat `${11}' as the eleventh
+  argument instead of literal text.  Be aware that Autoconf 2.61 will not
+  work with this option enabled with the default regular expression; but
+  Autoconf 2.62 will be compatible with this option.
+* Improved portability to platforms such as BSD/OS and AIX.
 
 Version 1.4.8 - 20 November 2006, by Eric Blake  (CVS version 1.4.7a)
 
Index: doc/m4.texinfo
===================================================================
RCS file: /sources/m4/m4/doc/m4.texinfo,v
retrieving revision 1.1.1.1.2.113
diff -u -p -r1.1.1.1.2.113 m4.texinfo
--- doc/m4.texinfo      5 Feb 2007 13:25:06 -0000       1.1.1.1.2.113
+++ doc/m4.texinfo      8 Feb 2007 14:42:14 -0000
@@ -584,12 +584,19 @@ is also specified.
 Suppress warnings, such as missing or superfluous arguments in macro
 calls, or treating the empty string as zero.
 
address@hidden --warn-syntax
-Issue warnings when syntax is encountered that will change semantics in
address@hidden M4 2.0.  For now, the only semantics that will change have
-to do with how more than 9 arguments in a macro definition are handled
-(@pxref{Arguments}).  This warning is disabled by default because it
-triggers spurious failures in @acronym{GNU} Autoconf 2.61.
address@hidden address@hidden@address@hidden
+Issue a warning if the regular expression @var{REGEXP} has a non-empty
+match in any macro definition (either by @code{define} or
address@hidden).  Empty matches are ignored; therefore, supplying the
+empty string as @var{REGEXP} disables any warning.  If the optional
address@hidden is not supplied, then the default regular expression is
address@hidden(@address@hidden@}\|[0-9][0-9]+\)} (a literal @samp{$} followed
+by multiple digits or by an open brace and a digit), since these
+sequences will change semantics in the default operation of
address@hidden M4 2.0 (due to a change in how more than 9 arguments in a
+macro definition will be handled, @pxref{Arguments}).  Providing an
+alternate regular expression can provide a useful reverse lookup feature
+of finding where a macro is defined to have a given definition.
 
 @item -W @var{REGEXP}
 @itemx address@hidden
@@ -1635,13 +1642,24 @@ bar(`a', `b')
 
 To help you detect places in your M4 input files that might change in
 behavior due to the changed behavior of M4 2.0, you can use the
address@hidden command-line option (@pxref{Operation modes, ,
-Invoking m4}).  This will add a warning any time a macro definition
-includes @samp{$} followed by multiple digits, or by @address@hidden and a
-digit.  The warning is not enabled by default, because it triggers a
-number of warnings in Autoconf 2.61 (and Autoconf uses @option{-E} to
-treat warnings as errors), and because it will still be possible to
-restore traditional behavior in M4 2.0.
address@hidden command-line option (@pxref{Operation
+modes, , Invoking m4}) with the default regular expression.  This will
+add a warning any time a macro definition includes @samp{$} followed by
+multiple digits, or by @address@hidden and a digit.  The warning is not
+enabled by default, because it triggers a number of warnings in Autoconf
+2.61 (and Autoconf uses @option{-E} to treat warnings as errors), and
+because it will still be possible to restore older behavior in M4 2.0.
+
address@hidden ignore
address@hidden
+$ @kbd{m4 --warn-macro-sequence}
+define(`foo', `$001 address@hidden@} $1')
address@hidden:stdin:1: Warning: definition of `foo' contains sequence `$001'
address@hidden:stdin:1: Warning: definition of `foo' contains sequence 
address@hidden@}'
address@hidden
+foo(`bar')
address@hidden address@hidden@} bar
address@hidden example
 
 @node Pseudo Arguments
 @section Special arguments to macros
Index: src/builtin.c
===================================================================
RCS file: /sources/m4/m4/src/Attic/builtin.c,v
retrieving revision 1.1.1.1.2.56
diff -u -p -r1.1.1.1.2.56 builtin.c
--- src/builtin.c       28 Jan 2007 01:54:44 -0000      1.1.1.1.2.56
+++ src/builtin.c       8 Feb 2007 14:42:15 -0000
@@ -221,6 +221,68 @@ define_builtin (const char *name, const 
   SYMBOL_FUNC (sym) = bp->func;
 }
 
+/* Storage for the compiled regular expression of
+   --warn-macro-sequence.  */
+static struct re_pattern_buffer macro_sequence_buf;
+
+/* Storage for the matches of --warn-macro-sequence.  */
+static struct re_registers macro_sequence_regs;
+
+/* True if --warn-macro-sequence is in effect.  */
+static bool macro_sequence_inuse;
+
+/*----------------------------------------.
+| Clean up regular expression variables.  |
+`----------------------------------------*/
+
+static void
+free_pattern_buffer (struct re_pattern_buffer *buf, struct re_registers *regs)
+{
+  regfree (buf);
+  free (regs->start);
+  free (regs->end);
+}
+
+/*-----------------------------------------------------------------.
+| Set the regular expression of --warn-macro-sequence that will be |
+| checked during define and pushdef.  Exit on failure.             |
+`-----------------------------------------------------------------*/
+void
+set_macro_sequence (const char *regexp)
+{
+  const char *msg;
+
+  if (! regexp)
+    regexp = "\\$\\({[0-9][^}]*}\\|[0-9][0-9]+\\)";
+  else if (regexp[0] == '\0')
+    {
+      macro_sequence_inuse = false;
+      return;
+    }
+
+  msg = re_compile_pattern (regexp, strlen (regexp), &macro_sequence_buf);
+  if (msg != NULL)
+    {
+      M4ERROR ((EXIT_FAILURE, 0,
+               "--warn-macro-sequence: bad regular expression `%s': %s",
+               regexp, msg));
+    }
+  re_set_registers (&macro_sequence_buf, &macro_sequence_regs,
+                   macro_sequence_regs.num_regs,
+                   macro_sequence_regs.start, macro_sequence_regs.end);
+  macro_sequence_inuse = true;
+}
+
+/*------------------------------------------------------------.
+| Free dynamic memory utilized by the define sequence regular |
+| expression.                                                |
+`------------------------------------------------------------*/
+void
+free_macro_sequence (void)
+{
+  free_pattern_buffer (&macro_sequence_buf, &macro_sequence_regs);
+}
+
 /*-------------------------------------------------------------------------.
 | Define a predefined or user-defined macro, with name NAME, and expansion |
 | TEXT.  MODE destinguishes between the "define" and the "pushdef" case.   |
@@ -231,50 +293,43 @@ void
 define_user_macro (const char *name, const char *text, symbol_lookup mode)
 {
   symbol *s;
-  size_t len;
+  char *defn = xstrdup (text ? text : "");
 
   s = lookup_symbol (name, mode);
   if (SYMBOL_TYPE (s) == TOKEN_TEXT)
     free (SYMBOL_TEXT (s));
 
   SYMBOL_TYPE (s) = TOKEN_TEXT;
-  SYMBOL_TEXT (s) = xstrdup (text ? text : "");
+  SYMBOL_TEXT (s) = defn;
 
-  /* In M4 2.0, $11 will mean the first argument concatenated with 1,
-     not the eleventh argument.  Also, ${1} will mean the first
-     argument, rather than literal text (although for compatibility
-     sake, it will be possible to restore the traditional meaning of
-     ${1} using changesyntax).  Needing more than 9 arguments is
-     somewhat rare, but using M4 to process shell code is quite
-     common; either way, warn on usages that will change in
-     semantics.  */
-  if (warn_syntax && text && (len = strlen (text)) >= 3)
+  /* Implement --warn-macro-sequence.  */
+  if (macro_sequence_inuse && text)
     {
-      static struct re_pattern_buffer buf;
-      static bool init = false;
       regoff_t offset = 0;
+      size_t len = strlen (defn);
 
-      if (! init)
+      while ((offset = re_search (&macro_sequence_buf, defn, len, offset,
+                                 len - offset, &macro_sequence_regs)) >= 0)
        {
-         const char *msg = "\\$[{0-9][0-9]";
-         init_pattern_buffer (&buf, NULL);
-         msg = re_compile_pattern (msg, strlen (msg), &buf);
-         if (msg != NULL)
+         /* Skip empty matches.  */
+         if (macro_sequence_regs.start[0] == macro_sequence_regs.end[0])
+           offset++;
+         else
            {
-             M4ERROR ((EXIT_FAILURE, 0,
-                       "unable to check --warn-syntax: %s", msg));
+             char tmp;
+             offset = macro_sequence_regs.end[0];
+             tmp = defn[offset];
+             defn[offset] = '\0';
+             M4ERROR ((warning_status, 0,
+                       "Warning: definition of `%s' contains sequence `%s'",
+                       name, defn + macro_sequence_regs.start[0]));
+             defn[offset] = tmp;
            }
-         init = true;
-       }
-      while ((offset = re_search (&buf, text, len, offset, len - offset,
-                                 NULL)) >= 0)
-       {
-         M4ERROR ((warning_status, 0,
-                   "Warning: semantics of `$%c%c%s' in `%s' will change",
-                   text[offset + 1], text[offset + 2],
-                   text[offset + 1] == '{' ? "...}" : "", name));
-         offset += 3;
        }
+      if (offset == -2)
+       M4ERROR ((warning_status, 0,
+                 "error checking --warn-define-sequence for macro `%s'",
+                 name));
     }
 }
 
@@ -1880,18 +1935,6 @@ init_pattern_buffer (struct re_pattern_b
     }
 }
 
-/*----------------------------------------.
-| Clean up regular expression variables.  |
-`----------------------------------------*/
-
-static void
-free_pattern_buffer (struct re_pattern_buffer *buf, struct re_registers *regs)
-{
-  regfree (buf);
-  free (regs->start);
-  free (regs->end);
-}
-
 /*--------------------------------------------------------------------------.
 | Regular expression version of index.  Given two arguments, expand to the  |
 | index of the first match of the second argument (a regexp) in the first.  |
Index: src/m4.c
===================================================================
RCS file: /sources/m4/m4/src/Attic/m4.c,v
retrieving revision 1.1.1.1.2.43
diff -u -p -r1.1.1.1.2.43 m4.c
--- src/m4.c    2 Feb 2007 02:55:11 -0000       1.1.1.1.2.43
+++ src/m4.c    8 Feb 2007 14:42:15 -0000
@@ -58,9 +58,6 @@ static bool fatal_warnings = false;
 /* If not zero, then value of exit status for warning diagnostics.  */
 int warning_status = 0;
 
-/* If true, then warn about usage of ${1} in macro definitions.  */
-bool warn_syntax = false;
-
 /* Artificial limit for expansion_level in macro.c.  */
 int nesting_limit = 1024;
 
@@ -163,7 +160,9 @@ Operation modes:\n\
   -i, --interactive            unbuffer output, ignore interrupts\n\
   -P, --prefix-builtins        force a `m4_' prefix to all builtins\n\
   -Q, --quiet, --silent        suppress some warnings for builtins\n\
-      --warn-syntax            warn on syntax that will change in future\n\
+      --warn-macro-sequence[=REGEXP]\n\
+                               warn if macro definition matches REGEXP,\n\
+                               default \\$\\({[0-9][^}]*}\\|[0-9]+\\)\n\
 ", stdout);
 #ifdef ENABLE_CHANGEWORD
       fputs ("\
@@ -239,7 +238,7 @@ enum
 {
   DEBUGFILE_OPTION = CHAR_MAX + 1,     /* no short opt */
   DIVERSIONS_OPTION,                   /* not quite -N, because of message */
-  WARN_SYNTAX_OPTION,                  /* no short opt */
+  WARN_MACRO_SEQUENCE_OPTION,          /* no short opt */
 
   HELP_OPTION,                         /* no short opt */
   VERSION_OPTION                       /* no short opt */
@@ -269,7 +268,7 @@ static const struct option long_options[
 
   {"debugfile", required_argument, NULL, DEBUGFILE_OPTION},
   {"diversions", required_argument, NULL, DIVERSIONS_OPTION},
-  {"warn-syntax", no_argument, NULL, WARN_SYNTAX_OPTION},
+  {"warn-macro-sequence", optional_argument, NULL, WARN_MACRO_SEQUENCE_OPTION},
 
   {"help", no_argument, NULL, HELP_OPTION},
   {"version", no_argument, NULL, VERSION_OPTION},
@@ -337,6 +336,7 @@ main (int argc, char *const *argv, char 
   const char *debugfile = NULL;
   const char *frozen_file_to_read = NULL;
   const char *frozen_file_to_write = NULL;
+  const char *macro_sequence = "";
 
   program_name = argv[0];
   retcode = EXIT_SUCCESS;
@@ -474,8 +474,12 @@ main (int argc, char *const *argv, char 
        debugfile = optarg;
        break;
 
-      case WARN_SYNTAX_OPTION:
-       warn_syntax = true;
+      case WARN_MACRO_SEQUENCE_OPTION:
+         /* Don't call set_macro_sequence here, as it can exit.
+            --warn-macro-sequence sets optarg to NULL (which uses the
+            default regexp); --warn-macro-sequence= sets optarg to ""
+            (which disables these warnings).  */
+        macro_sequence = optarg;
        break;
 
       case VERSION_OPTION:
@@ -497,6 +501,7 @@ main (int argc, char *const *argv, char 
   input_init ();
   output_init ();
   symtab_init ();
+  set_macro_sequence (macro_sequence);
   include_env_init ();
 
   if (frozen_file_to_read)
@@ -595,5 +600,6 @@ main (int argc, char *const *argv, char 
       undivert_all ();
     }
   output_exit ();
+  free_macro_sequence ();
   exit (retcode);
 }
Index: src/m4.h
===================================================================
RCS file: /sources/m4/m4/src/m4.h,v
retrieving revision 1.1.1.1.2.38
diff -u -p -r1.1.1.1.2.38 m4.h
--- src/m4.h    5 Feb 2007 13:43:36 -0000       1.1.1.1.2.38
+++ src/m4.h    8 Feb 2007 14:42:15 -0000
@@ -119,7 +119,6 @@ extern int max_debug_argument_length;       /*
 extern int suppress_warnings;          /* -Q */
 extern int warning_status;             /* -E */
 extern int nesting_limit;              /* -L */
-extern bool warn_syntax;               /* --warn-syntax */
 #ifdef ENABLE_CHANGEWORD
 extern const char *user_word_regexp;   /* -W */
 #endif
@@ -411,6 +410,8 @@ struct re_registers;
 
 void builtin_init (void);
 void define_builtin (const char *, const builtin *, symbol_lookup);
+void set_macro_sequence (const char *);
+void free_macro_sequence (void);
 void define_user_macro (const char *, const char *, symbol_lookup);
 void undivert_all (void);
 void expand_user_macro (struct obstack *, symbol *, int, token_data **);

Index: NEWS
===================================================================
RCS file: /sources/autoconf/autoconf/NEWS,v
retrieving revision 1.424
diff -u -p -r1.424 NEWS
--- NEWS        2 Feb 2007 14:11:43 -0000       1.424
+++ NEWS        8 Feb 2007 14:47:54 -0000
@@ -9,8 +9,9 @@
    These warnings can be disabled with the new AC_DISABLE_OPTION_CHECKING
    macro, or by invoking 'configure' with --disable-option-checking.
 
-** For portability with the eventual M4 2.0, macros should no longer use
-   anything larger than $9 to refer to arguments.
+** For portability with POSIX and the eventual M4 2.0, macros should not
+   refer to arguments greater than $9.  If M4 1.4.9 or later is
+   available, a warning is issued that detects usage such as $10.
 
 * Major changes in Autoconf 2.61a (2006-12-11)
 
Index: bin/autom4te.in
===================================================================
RCS file: /sources/autoconf/autoconf/bin/autom4te.in,v
retrieving revision 1.105
diff -u -p -r1.105 autom4te.in
--- bin/autom4te.in     4 Jan 2007 16:43:06 -0000       1.105
+++ bin/autom4te.in     8 Feb 2007 14:47:55 -0000
@@ -89,9 +89,14 @@ my $freeze = 0;
 
 # $M4.
 my $m4 = $ENV{"M4"} || '@M4@';
-# Some non-GNU m4's don't reject the --help option, so give them /dev/null.
-fatal "need GNU m4 1.4 or later: $m4"
-  if system "$m4 --help </dev/null 2>&1 | grep reload-state >/dev/null";
+fatal "need GNU m4 1.4.5 or later: $m4"
+  unless system "echo 'ifdef(mac,bug)dnl' | $m4 --trace=mac 2>&1 | grep bug 
>/dev/null";
+
+# Detect if we can use feature added in M4 1.4.9 to warn user about $10.
+# Override the user's choice of --warn-macro-sequence, since the default
+# macro sequence rejects ${1} even though we guarantee it is safe.
+$m4 .= " --warn-macro-sequence='\\\$\\([0-9][0-9]+\\|{{\\)'"
+  if ! system "$m4 --help | grep warn-macro-sequence >/dev/null";
 
 # Set some high recursion limit as the default limit, 250, has already
 # been hit with AC_OUTPUT.  Don't override the user's choice.
Index: doc/autoconf.texi
===================================================================
RCS file: /sources/autoconf/autoconf/doc/autoconf.texi,v
retrieving revision 1.1131
diff -u -p -r1.1131 autoconf.texi
--- doc/autoconf.texi   7 Feb 2007 17:45:37 -0000       1.1131
+++ doc/autoconf.texi   8 Feb 2007 14:48:00 -0000
@@ -434,6 +434,7 @@ M4 Quotation
 
 * Active Characters::           Characters that change the behavior of M4
 * One Macro Call::              Quotation and one macro call
+* Quoting and Parameters::      M4 vs. shell parameters
 * Quotation and Nested Macros::  Macros calling macros
 * Changequote is Evil::         Worse than INTERCAL: M4 + changequote
 * Quadrigraphs::                Another way to escape special characters
@@ -8862,10 +8863,6 @@ and their interface might change in the 
 @cindex M4 quotation
 @cindex quotation
 
address@hidden FIXME: Grmph, yet another quoting myth: quotation has *never*
address@hidden prevented `expansion' of $1.  Unless it refers to the expansion
address@hidden of the value of $1?  Anyway, we need a rewrite address@hidden
-
 The most common problem with existing macros is an improper quotation.
 This section, which users of Autoconf can skip, but which macro writers
 @emph{must} read, first justifies the quotation scheme that was chosen
@@ -8875,6 +8872,7 @@ former helps one to follow the latter.
 @menu
 * Active Characters::           Characters that change the behavior of M4
 * One Macro Call::              Quotation and one macro call
+* Quoting and Parameters::      M4 vs. shell parameters
 * Quotation and Nested Macros::  Macros calling macros
 * Changequote is Evil::         Worse than INTERCAL: M4 + changequote
 * Quadrigraphs::                Another way to escape special characters
@@ -8888,8 +8886,8 @@ To fully understand where proper quotati
 to know what the special characters are in Autoconf: @samp{#} introduces
 a comment inside which no macro expansion is performed, @samp{,}
 separates arguments, @samp{[} and @samp{]} are the quotes themselves,
-and finally @samp{(} and @samp{)} (which M4 tries to match by
-pairs).
address@hidden(} and @samp{)} (which M4 tries to match by pairs), and finally
address@hidden inside a macro definition.
 
 In order to understand the delicate case of macro calls, we first have
 to present some obvious failures.  Below they are ``obvious-ified'',
@@ -9010,10 +9008,77 @@ car([[[a]]])
 @result{}[a]
 @end example
 
address@hidden Quoting and Parameters
address@hidden
+
+When M4 encounters @samp{$} within a macro definition, followed
+immediately by a character it recognizes, it will perform M4 parameter
+expansion.  This happens regardless of how many layers of quotes the
+parameter expansion is nested within, or even if it occurs in text that
+will be rescanned as a comment.
+
address@hidden
+define([none], [$1])
address@hidden
+define([one], [[$1]])
address@hidden
+define([two], [[[$1]]])
address@hidden
+define([comment], [# $1])
address@hidden
+define([active], [ACTIVE])
address@hidden
+none([active])
address@hidden
+one([active])
address@hidden
+two([active])
address@hidden
+comment([active])
address@hidden active
address@hidden example
+
+On the other hand, since autoconf generates shell code, you often want
+to output shell variable expansion, rather than performing M4 parameter
+expansion.  To do this, you must use M4 quoting to separate the @samp{$}
+from the next character in the definition of your macro.  If the macro
+definition occurs in single-quoted text, then insert another level of
+quoting; if the usage is already inside a double-quoted string, then
+split it into concatenated strings.
+
address@hidden
+define([single], [one set of quotes for $[1] in definition])
address@hidden
+define([double], [[two sets of quotes for $][1 in definition]])
address@hidden
+single
address@hidden set of quotes for $1 in definition
+double
address@hidden sets of quotes for $1 in definition
address@hidden example
+
address@hidden states that M4 implementations are free to use
address@hidden@{} in macro definitions however they like.  @acronym{GNU} M4
+1.4.x always outputs those two characters literally, whereas
address@hidden M4 2.0 has a new feature called extended arguments that is
+modeled after shell variable expansion semantics.  This is due in part
+to the fact that M4 1.4.x violates @acronym{POSIX} by treating
address@hidden as the eleventh parameter.  To comply with @acronym{POSIX},
+M4 2.0 defaults to treating @address@hidden@}} as the eleventh parameter,
+while treating @samp{$11} as the first parameter concatenated with
address@hidden  Meanwhile, m4sugar uses other features of M4 2.0 to reserve
+the notation @address@hidden@address@hidden@}} (note the doubled 
@address@hidden) for M4
+extended arguments instead of M4's default, so that you can safely use
address@hidden@address@hidden to get literal output meaning the shell's first 
argument
+regardless of your version of M4.  However, it does mean that you cannot
+portably refer to anything higher than @samp{$9} in your macro and still
+work with all supported versions of M4.  In fact, if you use M4 1.4.9 or
+greater, Autoconf is able to warn you of non-portable usage of a
+multi-digit parameter.
+
 With this in mind, we can explore the cases where macros invoke
 address@hidden
 
-
 @node Quotation and Nested Macros
 @subsection Quotation and Nested Macros
 
@@ -9107,17 +9172,20 @@ qar([int tab[10];])
 @noindent
 Ahhh!  That's much better.
 
-But note what you've done: now that the arguments are literal strings,
-if the user wants to use the results of expansions as arguments, she has
-to use an @emph{unquoted} macro call:
+But note what you've done: now that the result of @code{qar} is always
+a literal string, the only time a user can use the results of
+expansions is with an @emph{unquoted} macro call:
 
 @example
 qar(active)
 @result{}ACT
+qar([active])
address@hidden
 @end example
 
 @noindent
-where she wanted to reproduce what she used to do with @code{car}:
+leaving no way for the user to reproduce what she used to do with
address@hidden:
 
 @example
 car([active])
Index: lib/m4sugar/m4sugar.m4
===================================================================
RCS file: /sources/autoconf/autoconf/lib/m4sugar/m4sugar.m4,v
retrieving revision 2.100
diff -u -p -r2.100 m4sugar.m4
--- lib/m4sugar/m4sugar.m4      20 Oct 2006 01:34:33 -0000      2.100
+++ lib/m4sugar/m4sugar.m4      8 Feb 2007 14:48:00 -0000
@@ -3,8 +3,8 @@ divert(-1)#                             
 # Base M4 layer.
 # Requires GNU M4.
 #
-# Copyright (C) 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006 Free Software
-# Foundation, Inc.
+# Copyright (C) 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007 Free
+# Software Foundation, Inc.
 #
 # This program is free software; you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
@@ -54,6 +54,13 @@ divert(-1)#                             
 # Set the quotes, whatever the current quoting system.
 changequote()
 changequote([, ])
+# M4 1.9b and greater defaults to treating ${1} as an expansion of parameter
+# one, but too many existing autoconf macros expect the 1.4.x behavior of
+# literal output.  We can tell the difference by whether the changeextarg
+# builtin exists; in which case we reserve ${{1}} as extended parameters for
+# the day when we can rely on M4 2.0.  autom4te also warns the user that
+# $10 is non-portable.
+ifdef([changeextarg], [changeextarg({{,}})])
 
 # Some old m4's don't support m4exit.  But they provide
 # equivalent functionality by core dumping because of the