fixing the LIFO m4wrap example

[Eric Blake]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

In recently fixing the autoconf implementation of m4_wrap to guarantee
FIFO order, I discovered that the example distributed in the m4 manual had
a bug when wrapping text containing $1.  This fixes the example to exactly
match m4 1.4.10 LIFO behavior (so that once I switch to FIFO behavior, as
required by POSIX, in the next stage of the argv_ref branch, the manual
will give an accurate way to restore prior behavior for those who need
it).  Part of the fix meant documenting how to join an arbitrary number of
arguments with a space between each; useful enough that I added a
subsection on this common use of the shift builtin.

- --
Don't work too hard, make some time for fun as well!

Eric Blake             address@hidden
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (Cygwin)
Comment: Public key at home.comcast.net/~ericblake/eblake.gpg
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkfcjfIACgkQ84KuGfSFAYBXXgCgv9NbnE5FMMwl5TkN2Xh/7aGe
lmoAoINzzWStB57avEwOPzEs7QwQxacV
=Quqf
-----END PGP SIGNATURE-----

>From 4cc7916e4dd2c221f37aa7eec159b48d15273157 Mon Sep 17 00:00:00 2001
From: Eric Blake <address@hidden>
Date: Sat, 15 Mar 2008 15:12:47 -0600
Subject: [PATCH] Document join, in order to fix bug in m4wrap example.

* examples/join.m4: New file.
* examples/wraplifo2.m4: Likewise.
* examples/Makefile.am (EXTRA_DIST): Add new files.
* doc/m4.texinfo (Improved m4wrap): New node.
(Defn, Location): Enhance tests.
(Shift): Document the composit macro join.
(Incompatibilities): Move documentation of LIFO vs. FIFO...
(M4wrap): ...here, to match improved example.

Signed-off-by: Eric Blake <address@hidden>
---
 ChangeLog             |   12 ++
 doc/m4.texinfo        |  301 +++++++++++++++++++++++++++++++++++++++++++-----
 examples/Makefile.am  |    6 +-
 examples/join.m4      |   15 +++
 examples/wraplifo2.m4 |    9 ++
 5 files changed, 309 insertions(+), 34 deletions(-)
 create mode 100644 examples/join.m4
 create mode 100644 examples/wraplifo2.m4

diff --git a/ChangeLog b/ChangeLog
index bce309d..599d00f 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,15 @@
+2008-03-15  Eric Blake  <address@hidden>
+
+       Document join, in order to fix bug in m4wrap example.
+       * examples/join.m4: New file.
+       * examples/wraplifo2.m4: Likewise.
+       * examples/Makefile.am (EXTRA_DIST): Add new files.
+       * doc/m4.texinfo (Improved m4wrap): New node.
+       (Defn, Location): Enhance tests.
+       (Shift): Document the composit macro join.
+       (Incompatibilities): Move documentation of LIFO vs. FIFO...
+       (M4wrap): ...here, to match improved example.
+
 2008-03-14  Eric Blake  <address@hidden>
 
        Stage 19: allow builtin tokens in more macros.
diff --git a/doc/m4.texinfo b/doc/m4.texinfo
index 7ac9867..f0fbb96 100644
--- a/doc/m4.texinfo
+++ b/doc/m4.texinfo
@@ -269,6 +269,7 @@ Correct version of some examples
 * Improved exch::               Solution for @code{exch}
 * Improved forloop::            Solution for @code{forloop}
 * Improved foreach::            Solution for @code{foreach}
+* Improved m4wrap::             Solution for @code{m4wrap}
 * Improved cleardivert::        Solution for @code{cleardivert}
 * Improved capitalize::         Solution for @code{capitalize}
 * Improved fatal_error::        Solution for @code{fatal_error}
@@ -2284,25 +2285,40 @@ builtin token is preserved only when it occurs in 
isolation.  A future
 version of @acronym{GNU} M4 may lift these restrictions.
 
 @example
+$ @kbd{m4 -d}
 define(`a', `A')define(`AA', `b')
 @result{}
+traceon(`defn', `define')
address@hidden
 defn(`a', `divnum', `a')
address@hidden:stdin:2: Warning: defn: cannot concatenate builtin `divnum'
address@hidden:stdin:3: Warning: defn: cannot concatenate builtin `divnum'
address@hidden: -1- defn(`a', `divnum', `a') -> ``A'`A''
 @result{}AA
 define(`mydivnum', defn(`divnum', `divnum'))mydivnum
address@hidden:stdin:3: Warning: defn: cannot concatenate builtin `divnum'
address@hidden:stdin:3: Warning: defn: cannot concatenate builtin `divnum'
address@hidden:stdin:4: Warning: defn: cannot concatenate builtin `divnum'
address@hidden:stdin:4: Warning: defn: cannot concatenate builtin `divnum'
address@hidden: -2- defn(`divnum', `divnum')
address@hidden: -1- define(`mydivnum', `')
address@hidden
+traceoff(`defn', `define')
 @result{}
 define(`mydivnum', defn(`divnum')defn(`divnum'))mydivnum
address@hidden:stdin:4: Warning: define: cannot concatenate builtin `divnum'
address@hidden:stdin:4: Warning: define: cannot concatenate builtin `divnum'
address@hidden:stdin:6: Warning: define: cannot concatenate builtin `divnum'
address@hidden:stdin:6: Warning: define: cannot concatenate builtin `divnum'
 @result{}
 define(`mydivnum', defn(`divnum')`a')mydivnum
address@hidden:stdin:5: Warning: define: cannot concatenate builtin `divnum'
address@hidden:stdin:7: Warning: define: cannot concatenate builtin `divnum'
 @result{}A
 define(`mydivnum', `a'defn(`divnum'))mydivnum
address@hidden:stdin:6: Warning: define: cannot concatenate builtin `divnum'
address@hidden:stdin:8: Warning: define: cannot concatenate builtin `divnum'
 @result{}A
+define(`q', ``$@@'')
address@hidden
+define(`foo', q(`a', defn(`divnum')))foo
address@hidden:stdin:10: Warning: define: cannot quote builtin
address@hidden,
+ifdef(`foo', `yes', `no')
address@hidden
 @end example
 
 @node Pushdef
@@ -2931,6 +2947,8 @@ shift(`foo', `bar', `baz')
 
 An example of the use of @code{shift} is this macro:
 
address@hidden reversing arguments
address@hidden arguments, reversing
 @deffn Composite reverse (@dots{})
 Takes any number of arguments, and reverses their order.
 @end deffn
@@ -3008,6 +3026,113 @@ example2(`feeling rather indecisive today')
 @result{}default answer: 4
 @end example
 
address@hidden joining arguments
address@hidden arguments, joining
address@hidden concatenating arguments
+Another common task that requires iteration is joining a list of
+arguments into a single string.
+
address@hidden Composite join (@ovar{separator}, @address@hidden)
address@hidden Composite joinall (@ovar{separator}, @address@hidden)
+Generate a single-quoted string, consisting of each @var{arg} separated
+by @var{separator}.  While @code{joinall} always outputs a
address@hidden between arguments, @code{join} avoids the
address@hidden for an empty @var{arg}.
address@hidden deffn
+
+Here are some examples of its usage, based on the implementation
address@hidden@value{VERSION}/@/examples/@/join.m4} distributed in this
+package:
+
address@hidden examples
address@hidden
+$ @kbd{m4 -I examples}
+include(`join.m4')
address@hidden
+join,join(`-'),join(`-', `'),join(`-', `', `')
address@hidden,,,
+joinall,joinall(`-'),joinall(`-', `'),joinall(`-', `', `')
address@hidden,,,-
+join(`-', `1')
address@hidden
+join(`-', `1', `2', `3')
address@hidden
+join(`', `1', `2', `3')
address@hidden
+join(`-', `', `1', `', `', `2', `')
address@hidden
+joinall(`-', `', `1', `', `', `2', `')
address@hidden
+join(`,', `1', `2', `3')
address@hidden,2,3
+define(`nargs', `$#')dnl
+nargs(join(`,', `1', `2', `3'))
address@hidden
address@hidden example
+
+Examining the implementation shows some interesting points about several
+m4 programming idioms.
+
address@hidden examples
address@hidden
+$ @kbd{m4 -I examples}
+undivert(`join.m4')dnl
address@hidden(`-1')
address@hidden join(sep, args) - join each non-empty ARG into a single
address@hidden string, with each element separated by SEP
address@hidden(`join',
address@hidden(`$#', `2', ``$2'',
address@hidden  `ifelse(`$2', `', `', ``$2'_')$0(`$1', shift(shift($@@)))')')
address@hidden(`_join',
address@hidden(`$#$2', `2', `',
address@hidden  `ifelse(`$2', `', `', ``$1$2'')$0(`$1', shift(shift($@@)))')')
address@hidden joinall(sep, args) - join each ARG, including empty ones,
address@hidden into a single string, with each element separated by SEP
address@hidden(`joinall', ``$2'_$0(`$1', shift($@@))')
address@hidden(`_joinall',
address@hidden(`$#', `2', `', ``$1$3'$0(`$1', shift(shift($@@)))')')
address@hidden'dnl
address@hidden example
+
+First, notice that this implementation creates helper macros
address@hidden and @code{_joinall}.  This division of labor makes it
+easier to output the correct number of @var{separator} instances:
address@hidden and @code{joinall} are responsible for the first argument,
+without a separator, while @code{_join} and @code{_joinall} are
+responsible for all remaining arguments, always outputting a separator
+when outputting an argument.
+
+Next, observe how @code{join} decides to iterate to itself, because the
+first @var{arg} was empty, or to output the argument and swap over to
address@hidden  If the argument is non-empty, then the nested
address@hidden results in an unquoted @samp{_}, which is concatenated
+with the @samp{$0} to form the next macro name to invoke.  The
address@hidden implementation is simpler since it does not have to
+suppress empty @var{arg}; it always executes once then defers to
address@hidden
+
+Another important idiom is the idea that @var{separator} is reused for
+each iteration.  Each iteration has one less argument, but rather than
+discarding @samp{$1} by iterating with @code{$0(shift($@@))}, the macro
+discards @samp{$2} by using @code{$0(`$1', shift(shift($@@)))}.
+
+Next, notice that it is possible to compare more than one condition in a
+single @code{ifelse} test.  The test of @samp{$#$2} against @samp{2}
+allows @code{_join} to iterate for two separate reasons---either there
+are still more than two arguments, or there are exactly two arguments
+but the last argument is not empty.
+
+Finally, notice that these macros require exactly two arguments to
+terminate recursion, but that they still correctly result in empty
+output when given no @var{args} (i.e., zero or one macro argument).  On
+the first pass when there are too few arguments, the @code{shift}
+results in no output, but leaves an empty string to serve as the
+required second argument for the second pass.  Put another way,
address@hidden', shift($@@)} is not the same as @samp{$@@}, since only the
+former guarantees at least two arguments.
+
address@hidden quote manipulation
address@hidden manipulating quotes
 Sometimes, a recursive algorithm requires adding quotes to each element,
 or treating multiple arguments as a single element:
 
@@ -3074,6 +3199,9 @@ undivert(`quote.m4')dnl
 @result{}divert`'dnl
 @end example
 
+It is worth pointing out that @samp{quote(@var{args})} is more efficient
+than @samp{joinall(`,', @var{args})} for producing the same output.
+
 @cindex nine arguments, more than
 @cindex more than nine arguments
 @cindex arguments, more than nine
@@ -4484,6 +4612,64 @@ in which they were saved (LIFO---last in, first out).  
However, this
 behavior is likely to change in a future release, to match
 @acronym{POSIX}, so you should not depend on this order.
 
+It is possible to emulate @acronym{POSIX} behavior even
+with older versions of @acronym{GNU} M4 by including the file
address@hidden@value{VERSION}/@/examples/@/wrapfifo.m4} from the
+distribution:
+
address@hidden examples
address@hidden
+$ @kbd{m4 -I examples}
+undivert(`wrapfifo.m4')dnl
address@hidden Redefine m4wrap to have FIFO semantics.
address@hidden(`_m4wrap_level', `0')dnl
address@hidden(`m4wrap',
address@hidden(`m4wrap'_m4wrap_level,
address@hidden       `define(`m4wrap'_m4wrap_level,
address@hidden               defn(`m4wrap'_m4wrap_level)`$1')',
address@hidden       `builtin(`m4wrap', `define(`_m4wrap_level',
address@hidden                                  incr(_m4wrap_level))dnl
address@hidden'_m4wrap_level)dnl
address@hidden(`m4wrap'_m4wrap_level, `$1')')')dnl
+include(`wrapfifo.m4')
address@hidden
+m4wrap(`a`'m4wrap(`c
+', `d')')m4wrap(`b')
address@hidden
+^D
address@hidden
address@hidden example
+
+It is likewise possible to emulate LIFO behavior without resorting to
+the @acronym{GNU} M4 extension of @code{builtin}, by including the file
address@hidden@value{VERSION}/@/examples/@/wraplifo.m4} from the
+distribution.  (Unfortunately, both examples shown here share some
+subtle bugs.  See if you can find and correct them; or @pxref{Improved
+m4wrap, , Answers}).
+
address@hidden examples
address@hidden
+$ @kbd{m4 -I examples}
+undivert(`wraplifo.m4')dnl
address@hidden Redefine m4wrap to have LIFO semantics.
address@hidden(`_m4wrap_level', `0')dnl
address@hidden(`_m4wrap', defn(`m4wrap'))dnl
address@hidden(`m4wrap',
address@hidden(`m4wrap'_m4wrap_level,
address@hidden       `define(`m4wrap'_m4wrap_level,
address@hidden               `$1'defn(`m4wrap'_m4wrap_level))',
address@hidden       `_m4wrap(`define(`_m4wrap_level', incr(_m4wrap_level))dnl
address@hidden'_m4wrap_level)dnl
address@hidden(`m4wrap'_m4wrap_level, `$1')')')dnl
+include(`wraplifo.m4')
address@hidden
+m4wrap(`a`'m4wrap(`c
+', `d')')m4wrap(`b')
address@hidden
+^D
address@hidden
address@hidden example
+
 Here is an example of implementing a factorial function using
 @code{m4wrap}:
 
@@ -6423,7 +6609,11 @@ __line__
 @result{}8
 __line__
 @result{}11
+m4wrap(`__line__
+')
address@hidden
 ^D
address@hidden
 @result{}6
 @result{}6
 @end example
@@ -6873,31 +7063,6 @@ argument to @code{m4wrap} is saved for later evaluation, 
but
 @acronym{GNU} @code{m4} saves and processes all arguments, with output
 separated by spaces.
 
-However, it is possible to emulate @acronym{POSIX} behavior by
-including the file @address@hidden/@/examples/@/wrapfifo.m4}
-from the distribution:
-
address@hidden
-undivert(`wrapfifo.m4')dnl
address@hidden Redefine m4wrap to have FIFO semantics.
address@hidden(`_m4wrap_level', `0')dnl
address@hidden(`m4wrap',
address@hidden(`m4wrap'_m4wrap_level,
address@hidden       `define(`m4wrap'_m4wrap_level,
address@hidden               defn(`m4wrap'_m4wrap_level)`$1')',
address@hidden       `builtin(`m4wrap', `define(`_m4wrap_level',
address@hidden                                  incr(_m4wrap_level))dnl
address@hidden'_m4wrap_level)dnl
address@hidden(`m4wrap'_m4wrap_level, `$1')')')dnl
-include(`wrapfifo.m4')
address@hidden
-m4wrap(`a`'m4wrap(`c
-', `d')')m4wrap(`b')
address@hidden
-^D
address@hidden
address@hidden example
-
 @item
 @acronym{POSIX} states that builtins that require arguments, but are
 called without arguments, have undefined behavior.  Traditional
@@ -7104,6 +7269,7 @@ presented here.
 * Improved exch::               Solution for @code{exch}
 * Improved forloop::            Solution for @code{forloop}
 * Improved foreach::            Solution for @code{foreach}
+* Improved m4wrap::             Solution for @code{m4wrap}
 * Improved cleardivert::        Solution for @code{cleardivert}
 * Improved capitalize::         Solution for @code{capitalize}
 * Improved fatal_error::        Solution for @code{fatal_error}
@@ -7557,6 +7723,77 @@ include(`loop.m4')dnl
 
 @end ignore
 
address@hidden Improved m4wrap
address@hidden Solution for @code{m4wrap}
+
+The replacement @code{m4wrap} versions presented above, designed to
+guarantee FIFO or LIFO order regardless of the underlying M4
+implementation, share a bug when dealing with wrapped text that looks
+like parameter expansion.  Note how the invocation of
address@hidden@var{n}} interprets these parameters, while using the
+builtin preserves them for their intended use.
+
address@hidden examples
address@hidden
+$ @kbd{m4 -I examples}
+include(`wraplifo.m4')
address@hidden
+m4wrap(`define(`foo', ``$0:'-$1-$*-$#-')foo(`a', `b')
+')
address@hidden
+builtin(`m4wrap', ``'define(`bar', ``$0:'-$1-$*-$#-')bar(`a', `b')
+')
address@hidden
+^D
address@hidden:-a-a,b-2-
address@hidden:---0-
address@hidden example
+
+Additionally, the computation of @code{_m4wrap_level} and creation of
+multiple @address@hidden placeholders in the original examples is
+more expensive in time and memory than strictly necessary.  Notice how
+the improved version grabs the wrapped text via @code{defn} to avoid
+parameter expansion, then undefines @code{_m4wrap_text}, before
+stripping a level of quotes with @code{_arg1} to expand the text.  That
+way, each level of wrapping reuses the single placeholder, which starts
+each nesting level in an undefined state.
+
+Finally, it is worth emulating the @acronym{GNU} M4 extension of saving
+all arguments to @code{m4wrap}, separated by a space, rather than saving
+just the first argument.  This is done with the @code{join} macro
+documented previously (@pxref{Shift}).  The improved LIFO example is
+shipped as @address@hidden/@/examples/@/wraplifo2.m4}, and can
+easily be converted to a FIFO solution by swapping the adjacent
+invocations of @code{joinall} and @code{defn}.
+
address@hidden examples
address@hidden
+$ @kbd{m4 -I examples}
+include(`wraplifo2.m4')
address@hidden
+undivert(`wraplifo2.m4')dnl
address@hidden Redefine m4wrap to have LIFO semantics, improved example.
address@hidden(`join.m4')dnl
address@hidden(`_m4wrap', defn(`m4wrap'))dnl
address@hidden(`_arg1', `$1')dnl
address@hidden(`m4wrap',
address@hidden(`_$0_text',
address@hidden       `define(`_$0_text', joinall(` ', $@@)defn(`_$0_text'))',
address@hidden       `_$0(`_arg1(defn(`_$0_text')undefine(`_$0_text'))')dnl
address@hidden(`_$0_text', joinall(` ', $@@))')')dnl
+m4wrap(`define(`foo', ``$0:'-$1-$*-$#-')foo(`a', `b')
+')
address@hidden
+m4wrap(`lifo text
+m4wrap(`nested', `', `$@@
+')')
address@hidden
+^D
address@hidden text
address@hidden:-a-a,b-2-
address@hidden  $@@
address@hidden example
+
 @node Improved cleardivert
 @section Solution for @code{cleardivert}
 
diff --git a/examples/Makefile.am b/examples/Makefile.am
index 3450eac..254d2ab 100644
--- a/examples/Makefile.am
+++ b/examples/Makefile.am
@@ -1,6 +1,6 @@
 ## Makefile.am - template for generating Makefile via Automake
 ##
-## Copyright (C) 2006, 2007 Free Software Foundation, Inc.
+## Copyright (C) 2006, 2007, 2008 Free Software Foundation, Inc.
 ##
 ## This file is part of GNU M4.
 ##
@@ -42,6 +42,7 @@ incl-test.m4 \
 incl.m4 \
 include.m4 \
 indir.m4 \
+join.m4 \
 loop.m4 \
 misc.m4 \
 multiquotes.m4 \
@@ -62,4 +63,5 @@ undivert.incl \
 undivert.m4 \
 wrap.m4 \
 wrapfifo.m4 \
-wraplifo.m4
+wraplifo.m4 \
+wraplifo2.m4
diff --git a/examples/join.m4 b/examples/join.m4
new file mode 100644
index 0000000..8687ac7
--- /dev/null
+++ b/examples/join.m4
@@ -0,0 +1,15 @@
+divert(`-1')
+# join(sep, args) - join each non-empty ARG into a single
+# string, with each element separated by SEP
+define(`join',
+`ifelse(`$#', `2', ``$2'',
+  `ifelse(`$2', `', `', ``$2'_')$0(`$1', shift(shift($@)))')')
+define(`_join',
+`ifelse(`$#$2', `2', `',
+  `ifelse(`$2', `', `', ``$1$2'')$0(`$1', shift(shift($@)))')')
+# joinall(sep, args) - join each ARG, including empty ones,
+# into a single string, with each element separated by SEP
+define(`joinall', ``$2'_$0(`$1', shift($@))')
+define(`_joinall',
+`ifelse(`$#', `2', `', ``$1$3'$0(`$1', shift(shift($@)))')')
+divert`'dnl
diff --git a/examples/wraplifo2.m4 b/examples/wraplifo2.m4
new file mode 100644
index 0000000..5b450a7
--- /dev/null
+++ b/examples/wraplifo2.m4
@@ -0,0 +1,9 @@
+dnl Redefine m4wrap to have LIFO semantics, improved example.
+include(`join.m4')dnl
+define(`_m4wrap', defn(`m4wrap'))dnl
+define(`_arg1', `$1')dnl
+define(`m4wrap',
+`ifdef(`_$0_text',
+       `define(`_$0_text', joinall(` ', $@)defn(`_$0_text'))',
+       `_$0(`_arg1(defn(`_$0_text')undefine(`_$0_text'))')dnl
+define(`_$0_text', joinall(` ', $@))')')dnl
-- 
1.5.4