Re: testsuite: distributed depending on non-distributed

[Eric Blake]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

According to Eric Blake on 9/12/2006 6:56 AM:
>
> Thanks for the report.  Both of these are a PATH issue.  The testsuite
> currently expects to execute tests/m4 (built from tests/m4.in), which
> invokes the real m4 with -M modules (so it can find modtest), and
> which normalizes stderr.  But on installcheck, we are altering PATH to
> pick up the installed m4, and bypassing the wrapper.  I was not seeing
> the failures because I was skipping too many tests (I ran with
> TESTSUITEFLAGS=1, instead of TESTSUITEFLAGS='-59 61-').  Perhaps the
> thing to do here would be renaming tests/m4 to tests/m4wrap, altering
> AT_CHECK_M4 to call tests/m4wrap, and then having the wrapper script
> pick up the correct m4 from the PATH.  Or maybe follow autoconf's lead
> and do all m4 output normalization inside testsuite.at.  Also, since
> modtest is not installed, but currently lives in the same build
> directory as other modules that do get installed, that makes it hard
> to tell if installcheck is testing the installed module or the built
> one.  Maybe it is worth splitting built modules into two directories,
> based on whether they will be installed.
>
> I'll have to play with this some, but hope to have a patch soon.

Here we go.  The tests/m4 wrapper is still useful, and is similar to the
wrappers in autoconf/tests, so I kept it, with modifications.  I moved the
stderr filtering out of the tests/m4 wrapper into the testsuite (so that
installed m4 also gets filtered), updated the Makefile to build
uninstalled modules directly into tests (so that '-M tests' now picks up
only the test modules, and we can blindly use that instead of '-M modules'
which would also pick up uninstalled libraries and hide a broken
installation), and fixed a couple of shell portability bugs in the process.

2006-09-13  Eric Blake  <address@hidden>

        Fix installcheck.
        * Makefile.am (check_LTLIBRARIES): Build test libraries in the
        tests directory, so we can be sure an installed build is not
        picking up uninstalled non-test libraries.
        (check-local, installcheck-local): Depend on test libraries.
        * tests/modules.at (AT_CHECK_M4_MODTEST): Don't use options
        after file name.  Fix quoting.  Remove skipping the test, now
        that makefile guarantees test modules will exist.
        (Freezing modules, modules: shadow, modules: unload)
        (modules: trace): Always put test modules in module path.
        (modules: importing): Likewise, and rely on AT_CHECK_M4 for
        stderr munging.
        * src/main.c (usage): Document default module search path.
        (import_environment, frozen_file_to_read, frozen_file_to_write):
        Move...
        (main): ...here.
        * tests/testsuite.at (AT_CHECK_M4): Filter stderr here...
        * tests/m4.in: ...not here.
        Reported by Ralf Wildenhues.

- --
Life is short - so eat dessert first!

Eric Blake             address@hidden
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2.1 (Cygwin)
Comment: Public key at home.comcast.net/~ericblake/eblake.gpg
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFFCKQA84KuGfSFAYARAl7eAJ9amDqQR3tYAU6QxKnYFJDuWelxpwCeKr+W
P51UiJxhdxVQkPk8P2bS1tY=
=qBHX
-----END PGP SIGNATURE-----

Index: Makefile.am
===================================================================
RCS file: /sources/m4/m4/Makefile.am,v
retrieving revision 1.45
diff -u -p -r1.45 Makefile.am
--- Makefile.am 8 Sep 2006 15:47:13 -0000       1.45
+++ Makefile.am 14 Sep 2006 00:35:16 -0000
@@ -177,26 +177,31 @@ EXTRA_DIST += modules/perl.c
 
 module_check   = -rpath /dev/null
 check_LTLIBRARIES = \
-                 modules/import.la \
-                 modules/modtest.la \
-                 modules/shadow.la \
-                 modules/stdlib.la \
-                 modules/time.la
-
-modules_import_la_LDFLAGS      = $(module_ldflags) $(module_check)
-modules_import_la_LIBADD       = $(module_libadd)
-
-modules_modtest_la_LDFLAGS     = $(module_ldflags) $(module_check)
-modules_modtest_la_LIBADD      = $(module_libadd)
-
-modules_shadow_la_LDFLAGS      = $(module_ldflags) $(module_check)
-modules_shadow_la_LIBADD       = $(module_libadd)
-
-modules_stdlib_la_LDFLAGS      = $(module_ldflags) $(module_check)
-modules_stdlib_la_LIBADD       = $(module_libadd)
-
-modules_time_la_LDFLAGS                = $(module_ldflags) $(module_check)
-modules_time_la_LIBADD         = $(module_libadd)
+                 tests/import.la \
+                 tests/modtest.la \
+                 tests/shadow.la \
+                 tests/stdlib.la \
+                 tests/time.la
+
+tests_import_la_LDFLAGS                = $(module_ldflags) $(module_check)
+tests_import_la_LIBADD         = $(module_libadd)
+tests_import_la_SOURCES                = modules/import.c
+
+tests_modtest_la_LDFLAGS       = $(module_ldflags) $(module_check)
+tests_modtest_la_LIBADD                = $(module_libadd)
+tests_modtest_la_SOURCES       = modules/modtest.c
+
+tests_shadow_la_LDFLAGS                = $(module_ldflags) $(module_check)
+tests_shadow_la_LIBADD         = $(module_libadd)
+tests_shadow_la_SOURCES                = modules/shadow.c
+
+tests_stdlib_la_LDFLAGS                = $(module_ldflags) $(module_check)
+tests_stdlib_la_LIBADD         = $(module_libadd)
+tests_stdlib_la_SOURCES                = modules/stdlib.c
+
+tests_time_la_LDFLAGS          = $(module_ldflags) $(module_check)
+tests_time_la_LIBADD           = $(module_libadd)
+tests_time_la_SOURCES          = modules/time.c
 
 EXTRA_DIST     += \
                  modules/modtest.m4 \
@@ -414,13 +419,13 @@ DISTCLEANFILES += tests/atconfig
 CD_TESTDIR     = cd ./tests
 
 # Hook the test suite into the check rule
-check-local: tests/atconfig tests/m4 $(TESTSUITE)
+check-local: tests/atconfig tests/m4 $(TESTSUITE) $(check_LTLIBRARIES)
        $(CD_TESTDIR); \
        $(SHELL) '$(abs_srcdir)/tests/testsuite' $(TESTS_ENVIRONMENT) \
          $(TESTSUITEFLAGS)
 
 # Run the test suite on the *installed* tree.
-installcheck-local: tests/atconfig $(TESTSUITE)
+installcheck-local: tests/atconfig $(TESTSUITE) $(check_LTLIBRARIES)
        $(CD_TESTDIR); \
        $(SHELL) '$(abs_srcdir)/tests/testsuite' $(TESTS_ENVIRONMENT) \
          AUTOTEST_PATH="$(bindir)" $(TESTSUITEFLAGS)
Index: tests/m4.in
===================================================================
RCS file: /sources/m4/m4/tests/m4.in,v
retrieving revision 1.7
diff -u -p -r1.7 m4.in
--- tests/m4.in 5 Sep 2006 23:16:40 -0000       1.7
+++ tests/m4.in 14 Sep 2006 00:35:16 -0000
@@ -23,22 +23,19 @@
 #
 # Written by Gary V. Vaughan <address@hidden>
 
-"@abs_top_builddir@/src/m4" \
-      --module-directory="@abs_top_builddir@/modules" \
-      ${1+"$@"} 2>/tmp/m4-$$
-status=$?
-# Normalize stderr.
-# - If configure --disable-shared, then src/m4 is a real program,
-#   neutralize the full path display
-# - otherwise, it is a  libtool wrapper, relying on PATH to execute the
-#   actual program, either lt-m4, or m4.
-#
-# - In both cases, beware of .exe.
-# - Also, when tracing modules, lines of the form
-#      m4debug: module m4: opening file `m4.a'
-#   must be reduced, since module names are platform dependent.
-sed -e 's,^[^:]*[lt-]*m4[.ex]*:,m4:,' \
-    -e '/^m4debug: module/s/opening file.*/opening file/' /tmp/m4-$$ >&2
-rm /tmp/m4-$$
+# Be Bourne compatible
+if test -n "${ZSH_VERSION+set}" && (emulate sh) >/dev/null 2>&1; then
+  emulate sh
+  NULLCMD=:
+  # Zsh 3.x and 4.x performs word splitting on ${1+"$@"}, which
+  # is contrary to our usage.  Disable this feature.
+  alias -g '${1+"$@"}'='"$@"'
+  setopt NO_GLOB_SUBST
+else
+  case `(set -o) 2>/dev/null` in *posix*) set -o posix;; esac
+fi
 
-exit $status
+exec "@abs_top_builddir@/src/m4" \
+      --module-directory="@abs_top_builddir@/modules" \
+      ${1+"$@"}
+exit 1
Index: tests/modules.at
===================================================================
RCS file: /sources/m4/m4/tests/modules.at,v
retrieving revision 1.20
diff -u -p -r1.20 modules.at
--- tests/modules.at    5 Sep 2006 16:58:02 -0000       1.20
+++ tests/modules.at    14 Sep 2006 00:35:16 -0000
@@ -50,19 +50,19 @@ test3
 
 # First generate the `expout' ouput by running over the sources before
 # freezing.
-AT_CHECK_M4([-m load frozen.m4 unfrozen.m4], 0,
-            [stdout], [stderr])
+AT_CHECK_M4([-M "$abs_builddir" -m load frozen.m4 unfrozen.m4],
+           [0], [stdout], [stderr])
 
 mv stdout expout
 mv stderr experr
 
 # Now freeze the first source file.
-AT_CHECK_M4([-m load -F frozen.m4f frozen.m4], 0,
-            [ignore], [ignore])
+AT_CHECK_M4([-M "$abs_builddir" -m load -F frozen.m4f frozen.m4],
+           [0], [], [ignore])
 
 # Now rerun the original sequence, but using the frozen file.
-AT_CHECK_M4([-R frozen.m4f unfrozen.m4], 0,
-            [expout], [experr])
+AT_CHECK_M4([-M "$abs_builddir" -R frozen.m4f unfrozen.m4],
+           [0], [expout], [experr])
 
 AT_CLEANUP([frozen.m4f])
 
@@ -74,13 +74,12 @@ AT_CLEANUP([frozen.m4f])
 
 # AT_CHECK_M4_MODTEST(TITLE, ENV-VARS, M4-OPTIONS)
 # ------------------------------------------------
-# Skip if modtest is not present (we are not in the package).
+# Add a test named TITLE, running m4 with either ENV-VARS in the environment
+# or M4-OPTIONS set to pick up test modules.
 m4_define([AT_CHECK_M4_MODTEST],
 [AT_SETUP([$1])
 AT_CHECK_DYNAMIC_MODULE
 
-AT_CHECK([test -f "$abs_top_builddir/modules/modtest.la" || exit 77])
-
 AT_DATA([input.m4],
 [[load(`modtest')
 test
@@ -90,7 +89,7 @@ test
 Dumpdef: dumpdef(`test').
 ]])
 
-AT_CHECK([$2 m4 -m load -d input.m4 $3], 0,
+AT_CHECK([$2 m4 -m load -d $3 input.m4], [0],
 [[
 Test module called.
 Dumpdef: .
@@ -110,22 +109,22 @@ AT_CLEANUP
 
 
 AT_CHECK_M4_MODTEST([--module-directory: absolute path],
-                    [], [-M $top_buildpath/modules])
+                   [], [-M "$abs_builddir"])
 
 AT_CHECK_M4_MODTEST([--module-directory: relative path],
-                    [], [-M $top_builddir/modules])
+                   [], [-M "$top_build_prefix/tests"])
 
 AT_CHECK_M4_MODTEST([M4MODPATH: absolute path],
-                    [M4MODPATH=$top_buildpath/modules], [])
+                   [M4MODPATH="$abs_builddir"], [])
 
 AT_CHECK_M4_MODTEST([M4MODPATH: relative path],
-                    [M4MODPATH=$top_builddir/modules], [])
+                   [M4MODPATH="$top_build_prefix/tests"], [])
 
 AT_CHECK_M4_MODTEST([LTDL_LIBRARY_PATH: absolute path],
-                    [LTDL_LIBRARY_PATH=$top_buildpath/modules], [])
+                   [LTDL_LIBRARY_PATH="$abs_builddir"], [])
 
 AT_CHECK_M4_MODTEST([LTDL_LIBRARY_PATH: relative path],
-                    [LTDL_LIBRARY_PATH=$top_builddir/modules], [])
+                   [LTDL_LIBRARY_PATH="$top_build_prefix/tests"], [])
 
 
 
@@ -264,8 +263,8 @@ test:       <test>
 shadow:        <shadow>
 ]])
 
-AT_CHECK_M4([-m load input.m4], 0,
-            [expout], [experr])
+AT_CHECK_M4([-M "$abs_builddir" -m load input.m4], [0],
+           [expout], [experr])
 
 AT_CLEANUP
 
@@ -332,7 +331,8 @@ Test module unloaded.
 ]])
 
 
-AT_CHECK_M4([-m load input.m4], 0, expout, experr)
+AT_CHECK_M4([-M "$abs_builddir" -m load input.m4],
+           [0], [expout], [experr])
 
 AT_CLEANUP
 
@@ -380,22 +380,11 @@ Test module loaded.
 
 TRUE
 m4:input.m4:6: cannot load symbol `no_such' from module `modtest'
-m4:input.m4:7: cannot open module `no_such': no_such.so: cannot open shared 
object file: No such file or directory
+m4:input.m4:7: cannot open module `no_such'
 ]])
 
-AT_CHECK_M4([-m load input.m4], 1, expout, stderr)
-
-# The expected stderr from the m4 invocation above becomes the expected
-# stdout of the following sed invocation to canonicalize the strerror message.
-# Note that the choice of file name affects the behaviour of AT_CHECK, so
-# we retain experr above for consistency with the rest of the testsuite, but
-# rename it here to make it play nicely with AT_CHECK:
-rm -f expout
-mv experr expout
-
-AT_CHECK([sed '/^m4:input.m4:7:/ {
-    s,\(7:[[^:]]*:\).*$,\1 no_such.so: cannot open shared object file: No such 
file or directory,;q; }' stderr], 0, expout)
-
+AT_CHECK_M4([-M "$abs_builddir" -m load input.m4],
+           [1], [expout], [experr])
 
 AT_CLEANUP
 
@@ -437,6 +426,7 @@ m4trace: -1- test -> `Shadow::`test' cal
 ]])
 
 
-AT_CHECK_M4([-m load -t test input.m4], 0, expout, experr)
+AT_CHECK_M4([-M "$abs_builddir" -m load -t test input.m4],
+           [0], [expout], [experr])
 
 AT_CLEANUP
Index: tests/testsuite.at
===================================================================
RCS file: /sources/m4/m4/tests/testsuite.at,v
retrieving revision 1.21
diff -u -p -r1.21 testsuite.at
--- tests/testsuite.at  5 Sep 2006 16:58:02 -0000       1.21
+++ tests/testsuite.at  14 Sep 2006 00:35:16 -0000
@@ -24,9 +24,32 @@ m4_version_prereq([2.52e])
 # AT_CHECK_M4(ARGS, [EXIT-STATUS = 0], [STDOUT = `'], [STDERR = `'])
 # ------------------------------------------------------------------
 # Run m4 with ARGS, and stdin redirected from /dev/null.  Expect EXIT-STATUS,
-# with output matching STDOUT and STDERR as in AT_CHECK.
+# with output matching STDOUT and STDERR as in AT_CHECK.  If stderr is
+# specified, normalize the observed error output.
+#
+# When testing an uninstalled wrapper, tests/m4 is on the path, but invokes
+# src/m4, which may itself be a libtool wrapper.  Also, some platforms
+# choose to display argv[0] differently:
+#    path/to/lt-m4:file:line: message
+# or m4.exe:file:line: message
+# to m4:file:line
+#
+# When tracing modules, the module name is platform-dependent:
+#    m4debug: module gnu: opening file `gnu.so'
+# or m4debug: module gnu: opening file `gnu.a'
+# to m4debug: module gnu: opening file
+#
+# When testing modules, a failed module name is platform-dependent:
+#    m4:input.m4:7: cannot open module `no_such': no_such.so: cannot open 
shared object file: No such file or directory
+# or m4:input.m4:7: cannot open module `no_such': can't open the module
+# to m4:input.m4:7: cannot open module `no_such'
 m4_define([AT_CHECK_M4],
-[AT_CHECK([m4 -b -d $1 < /dev/null], [$2], [$3], [$4])
+[AT_CHECK([m4 -b -d $1 < /dev/null], [$2], [$3], m4_ifval([$4], [stderr]))
+m4_ifval([$4],
+[AT_CHECK([[sed 's/^[^:]*[lt-]*m4[.ex]*:/m4:/
+       /^m4debug: module/s/opening file.*/opening file/
+       s/\(cannot open module [^:]*\):.*/\1/
+    ' stderr >&2]], [0], [], [$4])])
 ])
 
 # AT_TEST_M4(TITLE, INPUT, [STDOUT = `'], [STDERR = `'])
Index: src/main.c
===================================================================
RCS file: /sources/m4/m4/src/main.c,v
retrieving revision 1.80
diff -u -p -r1.80 main.c
--- src/main.c  7 Sep 2006 23:53:04 -0000       1.80
+++ src/main.c  14 Sep 2006 00:35:16 -0000
@@ -24,21 +24,12 @@
 #include "getopt.h"
 #include "version-etc.h"
 #include "gnu/progname.h"
+#include "pathconf.h"
 
 #include <limits.h>
 
 #define AUTHORS _("Rene' Seindal"), "Gary V. Vaughan"
 
-
-/* Name of frozen file to digest after initialization.  */
-const char *frozen_file_to_read = NULL;
-
-/* Name of frozen file to produce near completion.  */
-const char *frozen_file_to_write = NULL;
-
-/* If nonzero, import the environment as macros.  */
-static bool import_environment = false;
-
 typedef struct macro_definition
 {
   struct macro_definition *next;
@@ -105,9 +96,10 @@ SPEC is any one of:\n\
       printf (_("\
 \n\
 Dynamic loading features:\n\
-  -M, --module-directory=DIR   add DIR to the module search path\n\
-  -m, --load-module=MODULE     load dynamic MODULE from %s\n\
-"), USER_MODULE_PATH_ENV);
+  -M, --module-directory=DIR   add DIR to module search path before\n\
+                               `%s'\n\
+  -m, --load-module=MODULE     load dynamic MODULE\n\
+"), MODULE_PATH);
       fputs (_("\
 \n\
 Preprocessor features:\n\
@@ -237,12 +229,15 @@ main (int argc, char *const *argv, char 
   FILE *fp;
   char *filename;
   bool read_stdin = false;     /* true iff we have read from stdin */
+  bool import_environment = false; /* true to import environment */
+  const char *frozen_file_to_read = NULL;
+  const char *frozen_file_to_write = NULL;
 
   m4 *context;
 
   int exit_status;
 
-  /* Initialise gnulib error module.  */
+  /* Initialize gnulib error module.  */
   set_program_name (argv[0]);
 
   setlocale (LC_ALL, "");