testsuite -C dir

[Eric Blake]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I've had the idea for quite some time now [1], but finally got tired of
this ugly hunk in m4's non-recursive Makefile.am enough to do something
about it:

# FIXME - Someday, Autoconf might add an option to testsuite to specify
# which directory it should output testsuite.log.  Until then, we want it
# to drop in tests, so we have to do a cd inside the targets that invoke
# testsuite.  As a result, we need an absolute path since the relative
# $(srcdir) inside $(TESTSUITE) does not work after cd.
CD_TESTDIR      = cd ./tests
check-local: tests/atconfig tests/m4 $(TESTSUITE) $(check_LTLIBRARIES)
        $(CD_TESTDIR); \
        $(SHELL) '$(abs_srcdir)/tests/testsuite' $(TESTS_ENVIRONMENT) \
          $(TESTSUITEFLAGS)

In the spirit of make -C (and tar -C), autotest testsuites should support
- -C, as in this patch.  Then the above can be written as:

check-local: tests/atconfig tests/m4 $(TESTSUITE) $(check_LTLIBRARIES)
        $(SHELL) tests/testsuite -C tests $(TESTS_ENVIRONMENT) \
          $(TESTSUITEFLAGS)

In the meantime, I discovered a bug: './testsuite -k' ran the entire
testsuite, instead of complaining that -k requires an argument, but only
if a user did not add an additional option that requires an argument via
the (undocumented) AT_ARG_OPTION*.

By the way, AT_ARG_OPTION looks kind of cool - I had no idea it existed.
Does anyone want to extract this into m4sh, so that more than just
autotest can support the generation of portable shell code for parsing
arbitrary command-line options given a relatively simple interface of macros?

Any objections to applying this patch?  Also, how should './testsuite -C
- -' behave?  Since I didn't special case it, it currently dumps
testsuite.log in $OLDPWD (if your shell's cd is POSIX-compliant), rather
than checking for `./-'.

[1] http://lists.gnu.org/archive/html/autoconf/2006-06/msg00085.html

- --
Don't work too hard, make some time for fun as well!

Eric Blake             address@hidden
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Cygwin)
Comment: Public key at home.comcast.net/~ericblake/eblake.gpg
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFHm/ZY84KuGfSFAYARAilpAKC+5EtXysINBRpwEdEZJ19tgY9NOwCeNx4v
+Z+w/DG3U+LtkRYkC1SNHK4=
=jeCm
-----END PGP SIGNATURE-----

>From c4aaffd028b90df26a5e7ab2e1ef4b622d4b7320 Mon Sep 17 00:00:00 2001
From: Eric Blake <address@hidden>
Date: Sat, 26 Jan 2008 19:57:41 -0700
Subject: [PATCH] Add 'testsuite -C dir'.

* lib/autotest/general.m4 (_AT_ARG_OPTION): Move missing argument
detection...
(AT_INIT) <PARSE_ARGS_END>: ...here, since -k always takes
argument.
<DEFAULTS>: Delay computation of variables based on $at_dir...
<PREPARE_TESTS>: ...to here, since -C can change $at_dir.
<PARSE_ARGS>: Parse new option.
<HELP_TUNING>: Document it.
* tests/autotest.at (Choosing where testsuite is run): New test
for this feature.
(Keywords and ranges): Add test for missing -k argument.
* NEWS: Document this.
* doc/autoconf.texi (testsuite Invocation): Likewise.

Signed-off-by: Eric Blake <address@hidden>
---
 ChangeLog               |   17 +++++++++++
 NEWS                    |    3 ++
 doc/autoconf.texi       |    6 ++++
 lib/autotest/general.m4 |   69 +++++++++++++++++++++++++++++-----------------
 tests/autotest.at       |   27 +++++++++++++++++-
 5 files changed, 95 insertions(+), 27 deletions(-)

diff --git a/ChangeLog b/ChangeLog
index 27fb894..71b9b94 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,20 @@
+2008-01-27  Eric Blake  <address@hidden>
+
+       Add 'testsuite -C dir'.
+       * lib/autotest/general.m4 (_AT_ARG_OPTION): Move missing argument
+       detection...
+       (AT_INIT) <PARSE_ARGS_END>: ...here, since -k always takes
+       argument.
+       <DEFAULTS>: Delay computation of variables based on $at_dir...
+       <PREPARE_TESTS>: ...to here, since -C can change $at_dir.
+       <PARSE_ARGS>: Parse new option.
+       <HELP_TUNING>: Document it.
+       * tests/autotest.at (Choosing where testsuite is run): New test
+       for this feature.
+       (Keywords and ranges): Add test for missing -k argument.
+       * NEWS: Document this.
+       * doc/autoconf.texi (testsuite Invocation): Likewise.
+
 2008-01-24  Ralf Wildenhues  <address@hidden>
 
        * build-aux/config.guess, build-aux/config.sub,
diff --git a/NEWS b/NEWS
index ba278ad..855ed52 100644
--- a/NEWS
+++ b/NEWS
@@ -81,6 +81,9 @@ GNU Autoconf NEWS - User visible changes.
 
 ** Autotest now determines $srcdir correctly.
 
+** Testsuites built by autotest now accept a -C/--directory=DIR option
+   to adjust the working directory prior to creating files.
+
 ** Autoconf now requires GNU M4 1.4.5 or later.  Earlier versions of M4 have
    a bug in macro tracing that interferes with the interaction between
    Autoconf and Automake.  GNU M4 1.4.8 or later is recommended.  The
diff --git a/doc/autoconf.texi b/doc/autoconf.texi
index 78628e5..59e6f97 100644
--- a/doc/autoconf.texi
+++ b/doc/autoconf.texi
@@ -20191,6 +20191,12 @@ Display the list of options and exit successfully.
 @itemx -V
 Display the version of the test suite and exit successfully.
 
address@hidden address@hidden
address@hidden -C @var{dir}
+Change the current directory to @var{dir} before creating any files.
+Useful for running the testsuite in a subdirectory from a top-level
+Makefile.
+
 @item --clean
 @itemx -c
 Remove all the files the test suite might have created and exit.  Meant
diff --git a/lib/autotest/general.m4 b/lib/autotest/general.m4
index 11ff925..963cd52 100644
--- a/lib/autotest/general.m4
+++ b/lib/autotest/general.m4
@@ -1,8 +1,8 @@
 # This file is part of Autoconf.                          -*- Autoconf -*-
 # M4 macros used in building test suites.
 
-# Copyright (C) 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007 Free
-# Software Foundation, Inc.
+# Copyright (C) 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008
+# Free Software Foundation, Inc.
 
 # This program is free software: you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
@@ -413,7 +413,7 @@ at_groups=
 # Whether a write failure occurred
 at_write_fail=0
 
-# The directory we are in.
+# The directory we run the suite in.  Default to . if no -C option.
 at_dir=`pwd`
 # An absolute reference to this testsuite script.
 dnl m4-double quote, to preserve []
@@ -421,23 +421,8 @@ dnl m4-double quote, to preserve []
   [\\/]* | ?:[\\/]* ) at_myself=$as_myself ;;
   * ) at_myself=$at_dir/$as_myself ;;
 esac]
-# The directory the whole suite works in.
-# Should be absolute to let the user `cd' at will.
-at_suite_dir=$at_dir/$as_me.dir
-# The file containing the suite.
-at_suite_log=$at_dir/$as_me.log
-# The file containing the location of the last AT_CHECK.
-at_check_line_file=$at_suite_dir/at-check-line
-# The file containing the exit status of the last command.
-at_status_file=$at_suite_dir/at-status
-# The files containing the output of the tested commands.
-at_stdout=$at_suite_dir/at-stdout
-at_stder1=$at_suite_dir/at-stder1
-at_stderr=$at_suite_dir/at-stderr
-# The file containing the function to run a test group.
-at_test_source=$at_suite_dir/at-test-source
-# The file containing dates.
-at_times_file=$at_suite_dir/at-times
+# Whether -C is in effect.
+at_change_dir=false
 m4_divert_pop([DEFAULTS])dnl
 m4_wrap([m4_divert_text([DEFAULTS],
 [
@@ -563,6 +548,15 @@ do
        at_groups="$at_groups$at_range "
        ;;
 
+    # Directory selection.
+    --directory | -C )
+        at_prev=--directory
+       ;;
+    --directory=* )
+        at_change_dir=:
+        at_dir=$at_optarg
+       ;;
+
     # Keywords.
     --keywords | -k )
        at_prev=--keywords
@@ -616,6 +610,9 @@ m4_divert_push([PARSE_ARGS_END])dnl
   esac
 done
 
+# Verify our last option didn't require an argument
+AS_IF([test -n "$at_prev"], [AS_ERROR([`$at_prev' requires an argument.])])
+
 # Selected test groups.
 if test -z "$at_groups"; then
   at_groups=$at_groups_all
@@ -661,6 +658,8 @@ cat <<_ATEOF || at_write_fail=1
 
 dnl extra quoting prevents emacs whitespace mode from putting tabs in output
 Execution tuning:
+  -C, --directory=DIR
+[                 change to directory DIR before starting]
   -k, --keywords=KEYWORDS
 [                 select the tests matching all the comma-separated KEYWORDS]
 [                 multiple \`-k' accumulate; prefixed \`!' negates a KEYWORD]
@@ -724,6 +723,30 @@ fi
 m4_divert_pop([VERSION_END])dnl
 m4_divert_push([PREPARE_TESTS])dnl
 
+# Take any -C into account.
+if $at_change_dir ; then
+  test x != x"$at_dir" && cd "$at_dir" \
+    || AS_ERROR(["unable to change directory"])
+  at_dir=`pwd`
+fi
+# The directory the whole suite works in.
+# Should be absolute to let the user `cd' at will.
+at_suite_dir=$at_dir/$as_me.dir
+# The file containing the suite.
+at_suite_log=$at_dir/$as_me.log
+# The file containing the location of the last AT_CHECK.
+at_check_line_file=$at_suite_dir/at-check-line
+# The file containing the exit status of the last command.
+at_status_file=$at_suite_dir/at-status
+# The files containing the output of the tested commands.
+at_stdout=$at_suite_dir/at-stdout
+at_stder1=$at_suite_dir/at-stder1
+at_stderr=$at_suite_dir/at-stderr
+# The file containing the function to run a test group.
+at_test_source=$at_suite_dir/at-test-source
+# The file containing dates.
+at_times_file=$at_suite_dir/at-times
+
 # Don't take risks: use only absolute directories in PATH.
 #
 # For stand-alone test suites, AUTOTEST_PATH is relative to `.'.
@@ -1249,12 +1272,6 @@ m4_divert_text([PARSE_ARGS_BEGIN],
 m4_ifvaln([$3],,[at_arg_[]m4_bpatsubst([AT_first_option], -, _)=false])dnl
 at_arg_given_[]m4_bpatsubst([AT_first_option], -, _)=false
 ])dnl m4_divert_text DEFAULTS
-m4_ifval([$3],[m4_divert_once([PARSE_ARGS_END],
-[
-##
-## Verify our last option didn't require an argument
-##
-AS_IF([test -n "$at_prev"],[AS_ERROR([`$at_prev' requires an argument.])])])])
 m4_divert_text([PARSE_ARGS],
 [dnl Parse the options and args when necessary.
 m4_ifvaln([$3],
diff --git a/tests/autotest.at b/tests/autotest.at
index e043d7a..13ed63f 100644
--- a/tests/autotest.at
+++ b/tests/autotest.at
@@ -2,7 +2,8 @@
 
 AT_BANNER([Autotest.])
 
-# Copyright (C) 2004, 2005, 2006, 2007 Free Software Foundation, Inc.
+# Copyright (C) 2004, 2005, 2006, 2007, 2008 Free Software Foundation,
+# Inc.
 #
 # This program is free software: you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
@@ -395,6 +396,28 @@ AT_CHECK([(cd micro-suite.dir/1 && ./run MY_VAR='one 
space')],
 ])
 
 
+# Controlling where the testsuite is run.
+AT_CHECK_AT_TEST([Choosing where testsuite is run],
+  [AT_CHECK([:])], [], [], [], [], [
+dnl AT_CHECK_AT_TEST tests the default of running in `.'.
+AT_CHECK([rm micro-suite.log && mkdir sub1 sub2])
+dnl check specifying a different relative path to run in.
+AT_CHECK([./micro-suite -C sub1], [0], [ignore], [])
+AT_CHECK([test -f micro-suite.log], [1])
+AT_CHECK([test -f sub1/micro-suite.log], [0])
+dnl check specifying an absolute path to run in.
+AT_CHECK([./micro-suite --directory="`pwd`/sub2"], [0], [ignore], [])
+AT_CHECK([test -f micro-suite.log], [1])
+AT_CHECK([test -f sub2/micro-suite.log], [0])
+dnl check for failure detection with bad, missing, or empty directory.
+AT_CHECK([./micro-suite -C nonesuch], [1], [ignore], [ignore])
+AT_CHECK([./micro-suite -C ''], [1], [ignore], [ignore])
+AT_CHECK([./micro-suite -C], [1], [ignore], [ignore])
+dnl check that --help overrides bad directory selection.
+AT_CHECK([./micro-suite -C nonesuch --help], [0], [ignore], [])
+])
+
+
 ## -------- ##
 ## Banners. ##
 ## -------- ##
@@ -491,6 +514,8 @@ AT_CLEANUP
 AT_CHECK_AUTOM4TE([--language=autotest -o k k.at])
 dnl check that AT_KEYWORDS does not duplicate words
 AT_CHECK([grep 'key1.*key1' k], [1])
+dnl check that -k requires an argument
+AT_CHECK([./k -k], [1], [], [ignore])
 
 # AT_CHECK_KEYS(TESTSUITE-OPTIONS, PATTERN1, COUNT1, PATTERN2, COUNT2)
 m4_define([AT_CHECK_KEYS],
-- 
1.5.3.8