Re: testsuite failures when test scripts are run with zsh
|
View:
New views
18 Messages
—
Rating Filter:
Alert me
|
 |
|
|
|
Re: testsuite failures when test scripts are run with zsh
by Stefano Lattarini
::
Rate this Message:
At Thursday 08 October 2009, Ralf Wildenhues <Ralf.Wildenhues@...>
wrote: > Hello Stefano, > > > Thanks. A few nits (and I hope you don't mind that I'm being > fairly nit-picky below): > > [CUT] > Cheers, and thanks for your work on this, > Ralf Wow, that was a lot of stuff! I think it will take me some time to resolve all or most the issues you brought up. Meantime, I can at least send the two small changesets you suggested should be separated from the big patch. I'll send them after proper testing. Regards, Stefano |
|
|
Re: testsuite failures when test scripts are run with zsh
by Stefano Lattarini
::
Rate this Message:
The updated patch is still not complete, but i'd like to hear comments
and criticisms about the work-in-progress. A "temporary" partial patch is attached. At Friday 09 October 2009, Ralf Wildenhues <Ralf.Wildenhues@...> wrote: > Hello Stefano, > > [CUT] > > Thanks. A few nits (and I hope you don't mind that I'm being > fairly nit-picky below): > Have you checked whether any of the changed testsuite files need > updated copyright years? I didn't. Now the years should have been updated (I hope there's no mistake, as I used `grep'/`sed -i'). > The format of ChangeLog entries for GNU is described in the GNU > Coding Standards (GCS). > [CUT] > The GCS further state to not use parentheses after function names. I must admit that I just skimmed through the GCS, but never read them completely and carefully. I guess I'll have to. In the meantime, I corrected the ChangeLog entry as you suggested (except for a small difference, which is reported below). > I personally don't like mixed-case function names; can we rename > the thing to run_cmd, or even run_command instead? I think > AUTOMAKE_run etc were only done this way because they use > $AUTOMAKE and the similarity was desired. Agreed. Let's rename it `run_command'. > So, in this case it would IMHO be sufficient to write something > like: > > Fix testsuite: avoid Zsh-related problems with `set -x'. > * tests/README: Describe Zsh 4.x `set -x' aka. `xtrace' issue > and workaround with run_command. > * tests/defs.in (run_command): New function, to be used for > commands where standard error needs to be captured. > > > + * tests/acloca14.test: > > + * tests/acloca17.test: > > [...] > For such file lists, there are basically three possibilities to > list them, according to GCS: > ... > 2) > * file1: Blurb. > * file2: Likewise. > ... > * fileN: Likewise. > Settled for this. > > Unset variable `TEST_LOG_COMPILER', that might cause spurious > > failures by leaking in the environment of the make processes > > executed by the test scripts. Updated the code which ensures > > Bourne-compatibility in Zsh, by adding a call to `setopt > > NO_GLOB_SUBST' (as done by autoconf 2.64). > These changes should be in two separate patches. Thanks. You're right, I became sloppy and lazy after too much editing and tweaking, and I ended up cramming too many things in the patch. I already separated these changes and sent them in two distinct mails to the list. > > * tests/version8.test: > > use new subroutine `run_CMD()' instead of hand-crafted stdout > > and/or stderr redirections; this, togheter with the changes in > > file `tests/defs.in', fixes misbehaviour w.r.t. zsh for (at > > least) the following tests: > > - aclocal8.test > > [...] > > I'd shorten this to: > > Use run_command throughout. of stdout and/or stderr." Are you OK with this? > > --- a/tests/README > > +++ b/tests/README > > @@ -157,6 +157,29 @@ Do not > > reason, but at least it makes sure the original error is still > > here.) > > > > + If you must run a program and later analize its stderr, do > > *not* run it > > I'd shorten this whole description a lot. Portability issues > should be explained in sufficient detail in the portability section > of the Autoconf manual, and we shouldn't repeat them in detail > elsewhere, only briefly mention them (and maybe refer to the > Autoconf manual). That avoids fragmenting the documentation files > in the long run. Also, don't write what shouldn't be done, but > what should be done, unless inferring the former from the latter is > not obvious. If possible, add a new maintainer-check rule in the > toplevel that ensures that we follow the new guide line (this rule > then checks what shouldn't be done!). > > For example, I'd merely write the following here: > > To run a program and analyze its stderr, use the run_command > function: run_command PROG [ARGS...] > grep $pattern stderr > > as 'info Autoconf "File Descriptors"' already describes the issue > in question. is expected to be empty (this was the concrete case where the zsh "bug" revealed itself). > > --- a/tests/defs.in > > +++ b/tests/defs.in > > [...] > > > @@ -396,26 +397,100 @@ is_newest () > > test -z "$is_newest_files" > > } > > > > # run_CMD [-e STATUS] [-i FILE] [-m] [--] COMMAND [ARGUMENTS..] > > # ------------------------------------------------------------- > > # Run the given COMMAND (can be an external commnds, a function > I'd just elide the part in parentheses, BTW; that's expected. > > # and with standard input taken from FILE if option `-i' is given > What is -i needed for? Why can't we just use > run_command command < input > in that case? AFAICS none of the other commands inside that > function do anything with their standard input. Well, frankly I don't remember why I added that `-i' option, and now I don't see why it should be needed. Moreover, it's not used anywhere in the test scripts. So let's remove it. > > and fail if COMMAND does not exit > > with STATUS. +# If status is "VOID" or "IGNORE", any exit value > > of the command is +# acceptable. If STATUS is "FAIL", then any > > exit value of the command +# *but 0* is acceptable. Default > > STATUS is "IGNORE". > > Since 0 is the default expected STATUS, can we omit '-e 0' from > tests, unless that serves to be stressed somewhere? Yes, it's probably less obtrusive this way. Changed as you suggested. > Hmm, I see a few inconsistencies cropping up here. First, we > already have AUTOMAKE_run. It has slightly different syntax. With > your patch, some automake invocations that capture output use > AUTOMAKE_run, while others use run_command. > > These inconsistencies should be resolved. I'm fine with having all > automake invocations use AUTOMAKE_run. Don't you think this should be done in a separate patch? [STILL NOT CHANGED IN THE ATTACHED PATCH] > > +# Also, save standard output and standard error from COMMAND, by > > default +# respectively in files `stdout' and `stderr' (in the > > current directory), +# or togheter in the file `stdall' (in the > > current directory) if the `-m' +# option is given. > > > > All the redirections are done without > > triggering the +# zsh4 bug related to `-x' shell switch > > (described in details in the +# tests/README file). > > I'd drop this sentence, too. > > > +run_CMD () > > +{ > > + # NOTE: all internal variables used here starts with the > > `_run' + # prefix, to minimize possibility of name clashes with > > global + # variables defined in user code. > > + : 'entering run_CMD(): become quiet' > > + set +x # xtrace verbosity stops here > > No need to comment the obvious, here as well as at the end of this > function; these 5 lines can be replaced with > set +x annoying, I think it's better to remove them. I'd just like to keep two comments: set +x # xtrace verbosity temporarly disabled in `run_command' at the beginning, and: set -x # reactivating temporarly turned-off xtrace verbosity at the end. Objections? > > If you care about reusability of this function in a context where > you can't be sure whether xtrace is enabled at this point, you can > use something like this instead: > case $i in *x*) run_xtrace=;; *) run_xtrace=:;; esac > $run_xtrace set +x > ... > $run_xtrace set -x Well, ATM I'd prefer to keep the function simpler. It can be made more reusable and general later, if the need will arise. What do you think? > > > + _run_stdin=- > > + _run_expected_exitcode=0 > > + _run_mix_stdout_and_stderr=no > > I don't see a need to use underscore-prefixed variables. We > control both the defs.in file as well as all files that use it. > IMHO a run_ prefix is sufficient. Agreed. > But please don't rename _run_cmd > below to run_cmd if that's also the function name (some shells > don't have separate name spaces for functions and variables). Since I renamed the function as `run_command', I'd go with the name `run_cmd' for the variable. > Missing function-scope variable names are a problem not really > solved well by underscores: as soon as you have more than one level > of functions, you'd need yet another way to avoid clashes. IOW: > let's just not use the ugly notation in the first place. Thanks. Agreed. > > > + while test $# -gt 0; do > > + case "$1" in > > No need to quote the argument after 'case'. OK, fixed. > > > + -e) _run_expected_exitcode=$2; shift;; > > + -i) _run_stdin=$2; shift;; > > + -m) _run_mix_stdout_and_stderr=yes;; > > + --) shift; break;; > > + -?) echo "run_CMD(): invalid switch '$1'" >&2; Exit 99;; > > Please use \` for left quotes in messages. I know this is ugly, > but inconsistent usage is even uglier, so a change should be done > across the board, and when unicode or similar can be assumed. > > > + *) break;; > > + esac > > + shift > > + done > > + case $# in > > + 0) echo "run_CMD(): missing COMMAND argument" >&2; Exit 99;; > > + *) _run_cmd=$1; shift;; > > + esac > > + _run_exitcode=0 > > + if test x"${_run_mix_stdout_and_stderr}" = x"yes"; then > > How come you're starting to mix $foo and ${foo} style here? > Several instances below. prefer ${_foo}. But this won't be a problem anymore, as I took your suggestion and removed leading underscores from variable names. > > + _run_evald_cmd='"${_run_cmd}" ${1+"$@"} >stdall 2>&1' > > + else > > + _run_evald_cmd='"${_run_cmd}" ${1+"$@"} >stdout 2>stderr' > > + fi > > + if test x"${_run_stdin}" != x"-"; then > > + _run_evald_cmd="${_run_evald_cmd}"' <"${_run_stdin}"' > > + fi > > + _run_evald_cmd="${_run_evald_cmd} || _run_exitcode=\$?" > > + eval "${_run_evald_cmd}" > > Why not simplify these two lines to > eval "${_run_evald_cmd}" > run_exitcode=$? > and drop the _run_exitcode initialization above? [STILL NOT CHANGED IN THE ATTACHED PATCH] > > + if test x"${_run_mix_stdout_and_stderr}" = x"yes"; then > > + echo "=== stdout and stderr" > > Instead of all the file boundary markers, can we just re-enable > xtrace here? That makes the trace output look more similar to that > we get from commands not run through this function. You can > reorder this chunk of code to be after the exit code handling to > not deal with xtrace twice. Well, I liked the `===' marker: it sticked out clearly without being obtrusive. But if you prefer a stricter consistency, I might just use 'echo "+ cat stdout" >&2' etc. instead of 'echo "=== stdout"' etc. Note that for now I left the `===' markers in place. Feedback needed. [STILL NOT CHANGED IN THE ATTACHED PATCH] > > + cat stdall > > + echo "===" > > + else > > + echo "=== stderr" >&2 > > + cat stderr >&2 > > + echo "===" >&2 > > + echo "=== stdout" > > + cat stdout > > + echo "===" > > + fi > > + case ${_run_expected_exitcode} in > > + VOID|void|IGNORE|ignore|IGNORED|ignored|${_run_exitcode}) > > + _run_rc=0 > > + ;; > > + FAIL|fail|FAILURE|failure) > > + test ${_run_exitcode} -gt 0 && _run_rc=0 || _run_rc=1 > > Please make this > if ... ; then run_rc=0; else ... ; fi > as this may run with `set -e' set, and I'm not sure whether some > broken shell may err-exit here otherwise. (See the Autoconf manual > for details.) > > + ;; > > + *) > > + _run_rc=1 > > + ;; > > + esac > > + set -x # xtrace verbosity restart here > > + : "exit status ${_run_exitcode} (expecting > > ${_run_expected_exitcode})" + : "leaving run_CMD()" > > + return ${_run_rc} > > +} > > > > > > # AUTOMAKE_run status [options...] > > # -------------------------------- > > -# Run Automake with OPTIONS, and fail if automake > > +# Run Automake with OPTIONS, and make the testcase FAIL if > > automake > > Why this change? I'd drop it, likewise for the comment to > AUTOMAKE_fails. [STILL NOT CHANGED IN THE ATTACHED PATCH] > > # does not exit with STATUS. > > AUTOMAKE_run () > > { > > - expected_exitcode=$1 > > + _am_run_expected_exitcode=$1 > Why? For pseudo-consistency with `run_command'. Do you prefer this change removed completely, or is OK to use `am_run_expected_exitcode' instead of `_am_run_expected_exitcode'? Feedback needed. [STILL NOT CHANGED IN THE ATTACHED PATCH] > > shift > > - exitcode=0 > > - $AUTOMAKE ${1+"$@"} >stdout 2>stderr || exitcode=$? > > - cat stderr >&2 > > - cat stdout > > - test $exitcode = $expected_exitcode || Exit 1 > > + run_CMD -e ${_am_run_expected_exitcode} -- $AUTOMAKE ${1+"$@"} > > || Exit 1 > > This change needs to be listed in the ChangeLog entry. > In some of the tests, you use `test -s FILE || Exit 1', while in > others, you use plain `test ! -s FILE'. If you use the latter, or > any other command that you'd like to fail the test with nonzero > status, you have to ensure that the test has enabled `set -e' > earlier on, or use the former variant. Good catch. Luckily, it seems that the only test scripts using `test -s' or `test ! -s' and which also don't have `set -e' enabled are: $ grep 'test.*-s' $(grep -L '\<set *-e\>' tests/*.test) tests/acsilent.test:test -s stdall && Exit 1 tests/dejagnu-p.test:# Test to make sure dejagnu ... tests/dejagnu.test:# Test to make sure dejagnu ... tests/dejagnu7.test:# Check that "make check" fails, when ... tests/dejagnu7.test:runtest --help | $FGREP -e --status || Exit 77 tests/mdate4.test:test -f sub/mdate-sh tests/unused.test:test ! -s stderr which prunig the false positives boil down to: tests/acsilent.test tests/unused.test and neither of these two required a correction (the `test ! -s stderr' is the last command in `unused.test', so that if it fails, the whole test case fails). -*-*-*- By the way, your observation has made me think: wouldn't it be better to enable `set -e' in defs.in, so that all the test cases could have a more uniform environment? This would reduce the possibility of repeating mistakes like the one you pointed out above (which was very small, but only for sheer luck). Of course this propesed change would require to change also some exising tests (those which don't use `set -e'), but this can be done in a later patch. And another aside: I was about to modify also the file `tests/dejagnu-p.test', before remembering it is automatically generated (and, well, also before noticing it was a false positive). I think it would be useful to make the autogoenerated files (tests/defs, tests/*-p.test, etc) readonly. It could be done in a separate patch. What do you think? -*-*-*- > Cheers, and thanks for your work on this, > Ralf Thanks for your review. I hope this time I'm making fewer mistakes. Regards, Stefano [Fix-testsuite-avoid-Zsh-related-problems-with-set-x.patch] From 84c967dec7f12332879d86c3fbedfd9837f540b6 Mon Sep 17 00:00:00 2001 From: Stefano Lattarini <stefano.lattarini@...> Date: Wed, 9 Sep 2009 21:49:03 +0200 Subject: [PATCH] Fix testsuite: avoid Zsh-related problems with `set -x'. * tests/README: Describe Zsh 4.x `set -x' a.k.a. `xtrace' issue and workaround with run_command. * tests/defs.in (run_command): New function, to be used for commands where standard error needs to be captured. (AUTOMAKE_run): reimplemented using run_command. * tests/acloca14.test: Use new subroutine run_command instead of hand-crafted redirections of stdout and/or stderr. * tests/acloca17.test: Likewise. * tests/acloca18.test: Likewise. * tests/aclocal.test: Likewise. * tests/aclocal3.test: Likewise. * tests/aclocal8.test: Likewise. * tests/acsilent.test: Likewise. * tests/alpha.test: Likewise. * tests/ammissing.test: Likewise. * tests/ansi3.test: Likewise. * tests/ansi3b.test: Likewise. * tests/ccnoco3.test: Likewise. * tests/check8.test: Likewise. * tests/cond39.test: Likewise. * tests/defs.in: Likewise. * tests/depcomp2.test: Likewise. * tests/depcomp4.test: Likewise. * tests/depend5.test: Likewise. * tests/depend6.test: Likewise. * tests/discover.test: Likewise. * tests/distcom3.test: Likewise. * tests/fn99.test: Likewise. * tests/fn99subdir.test: Likewise. * tests/gnits2.test: Likewise. * tests/gnits3.test: Likewise. * tests/help.test: Likewise. * tests/init.test: Likewise. * tests/instsh.test: Likewise. * tests/ldadd.test: Likewise. * tests/libtool7.test: Likewise. * tests/libtool9.test: Likewise. * tests/lisp3.test: Likewise. * tests/ltorder.test: Likewise. * tests/man4.test: Likewise. * tests/mdate.test: Likewise. * tests/missing3.test: Likewise. * tests/missing4.test: Likewise. * tests/missing6.test: Likewise. * tests/obsolete.test: Likewise. * tests/output-order.test: Likewise. * tests/pr220.test: Likewise. * tests/pr300-ltlib.test: Likewise. * tests/python11.test: Likewise. * tests/python4.test: Likewise. * tests/python5.test: Likewise. * tests/python6.test: Likewise. * tests/python7.test: Likewise. * tests/python8.test: Likewise. * tests/python9.test: Likewise. * tests/sanity.test: Likewise. * tests/subobj.test: Likewise. * tests/subobj9.test: Likewise. * tests/unused.test: Likewise. * tests/vars3.test: Likewise. * tests/version8.test: Likewise. --- ChangeLog | 65 +++++++++++++++++++++++++++++++ tests/README | 10 +++++ tests/acloca14.test | 4 +- tests/acloca17.test | 6 +-- tests/acloca18.test | 10 ++--- tests/aclocal.test | 8 ++-- tests/aclocal3.test | 5 +- tests/aclocal8.test | 7 +-- tests/acsilent.test | 7 ++- tests/alpha.test | 5 +- tests/ammissing.test | 5 +- tests/ansi3.test | 5 +- tests/ansi3b.test | 5 +- tests/ccnoco3.test | 3 +- tests/check8.test | 8 +--- tests/cond39.test | 8 +-- tests/defs.in | 96 +++++++++++++++++++++++++++++++++++++++------- tests/depcomp2.test | 9 +++- tests/depcomp4.test | 7 +-- tests/depend5.test | 5 +-- tests/depend6.test | 7 +-- tests/discover.test | 5 +- tests/distcom3.test | 7 +-- tests/fn99.test | 5 +- tests/fn99subdir.test | 5 +- tests/gnits2.test | 6 +- tests/gnits3.test | 6 +- tests/help.test | 4 +- tests/init.test | 10 +++- tests/instsh.test | 7 +-- tests/ldadd.test | 5 +- tests/libtool7.test | 23 +++++------ tests/libtool9.test | 9 ++-- tests/lisp3.test | 7 +-- tests/ltorder.test | 5 +- tests/man4.test | 8 +-- tests/mdate.test | 5 +- tests/missing3.test | 18 ++++---- tests/missing4.test | 8 +-- tests/missing6.test | 7 +-- tests/obsolete.test | 6 +- tests/output-order.test | 11 ++--- tests/pr220.test | 5 +- tests/pr300-ltlib.test | 6 +-- tests/python11.test | 6 +-- tests/python4.test | 5 +- tests/python5.test | 5 +- tests/python6.test | 5 +- tests/python7.test | 5 +- tests/python8.test | 5 +- tests/python9.test | 5 +- tests/sanity.test | 6 +-- tests/subobj.test | 5 +- tests/subobj9.test | 6 +- tests/unused.test | 6 +- tests/vars3.test | 5 +- tests/version8.test | 5 +- 57 files changed, 315 insertions(+), 217 deletions(-) diff --git a/ChangeLog b/ChangeLog index b54cf75..b137f9a 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,68 @@ +2009-10-08 Stefano Lattarini <stefano.lattarini@...> + + Fix testsuite: avoid Zsh-related problems with `set -x'. + * tests/README: Describe Zsh 4.x `set -x' a.k.a. `xtrace' issue + and workaround with run_command. + * tests/defs.in (run_command): New function, to be used for + commands where standard error needs to be captured. + (AUTOMAKE_run): reimplemented using run_command. + * tests/acloca14.test: Use new subroutine run_command instead + of hand-crafted redirections of stdout and/or stderr. + * tests/acloca17.test: Likewise. + * tests/acloca18.test: Likewise. + * tests/aclocal.test: Likewise. + * tests/aclocal3.test: Likewise. + * tests/aclocal8.test: Likewise. + * tests/acsilent.test: Likewise. + * tests/alpha.test: Likewise. + * tests/ammissing.test: Likewise. + * tests/ansi3.test: Likewise. + * tests/ansi3b.test: Likewise. + * tests/ccnoco3.test: Likewise. + * tests/check8.test: Likewise. + * tests/cond39.test: Likewise. + * tests/defs.in: Likewise. + * tests/depcomp2.test: Likewise. + * tests/depcomp4.test: Likewise. + * tests/depend5.test: Likewise. + * tests/depend6.test: Likewise. + * tests/discover.test: Likewise. + * tests/distcom3.test: Likewise. + * tests/fn99.test: Likewise. + * tests/fn99subdir.test: Likewise. + * tests/gnits2.test: Likewise. + * tests/gnits3.test: Likewise. + * tests/help.test: Likewise. + * tests/init.test: Likewise. + * tests/instsh.test: Likewise. + * tests/ldadd.test: Likewise. + * tests/libtool7.test: Likewise. + * tests/libtool9.test: Likewise. + * tests/lisp3.test: Likewise. + * tests/ltorder.test: Likewise. + * tests/man4.test: Likewise. + * tests/mdate.test: Likewise. + * tests/missing3.test: Likewise. + * tests/missing4.test: Likewise. + * tests/missing6.test: Likewise. + * tests/obsolete.test: Likewise. + * tests/output-order.test: Likewise. + * tests/pr220.test: Likewise. + * tests/pr300-ltlib.test: Likewise. + * tests/python11.test: Likewise. + * tests/python4.test: Likewise. + * tests/python5.test: Likewise. + * tests/python6.test: Likewise. + * tests/python7.test: Likewise. + * tests/python8.test: Likewise. + * tests/python9.test: Likewise. + * tests/sanity.test: Likewise. + * tests/subobj.test: Likewise. + * tests/subobj9.test: Likewise. + * tests/unused.test: Likewise. + * tests/vars3.test: Likewise. + * tests/version8.test: Likewise. + 2009-10-06 Bruno Haible <bruno@...> Ralf Wildenhues <Ralf.Wildenhues@...> diff --git a/tests/README b/tests/README index 4068ad6..9b25be9 100644 --- a/tests/README +++ b/tests/README @@ -115,6 +115,16 @@ Do Use $sleep when you have to make sure that some file is newer than another. + To run a program and analyze its stderr, use the run_command function. + For example: + run_command PROG [ARGS...] + grep $pattern stderr + or (if you must check that nothing is written on stderr): + run_command PROG2 [ARGS...] + test ! -s stderr + See section "File Descriptors" in the Autoconf manual to understand why + this can be useful, or even required. + Use `cat' or `grep' to display (part of) files that may be interesting for debugging, so that when a user send a verbose output we don't have to ask him for more details. Display stderr diff --git a/tests/acloca14.test b/tests/acloca14.test index 8ddbb1a..cf9c484 100755 --- a/tests/acloca14.test +++ b/tests/acloca14.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2004, 2008 Free Software Foundation, Inc. +# Copyright (C) 2004, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -108,5 +108,5 @@ $MAKE testdist2 # Make sure aclocal diagnose missing included files with correct `file:line:'. rm -f b.m4 -$ACLOCAL 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e 1 $ACLOCAL grep 'a.m4:1:.*b.m4.*does not exist' stderr diff --git a/tests/acloca17.test b/tests/acloca17.test index 884f24b..788cb9d 100755 --- a/tests/acloca17.test +++ b/tests/acloca17.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2004, 2008 Free Software Foundation, Inc. +# Copyright (C) 2004, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -33,7 +33,5 @@ EOF # FIXME: We want autom4te's 'undefined required macro' warning to be fatal, # but have no means to say so to aclocal. We use WARNINGS=error instead. - -WARNINGS=error $ACLOCAL -I m4 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_command -e 1 env WARNINGS=error $ACLOCAL -I m4 grep 'configure.in:4:.*UNDEFINED_MACRO' stderr diff --git a/tests/acloca18.test b/tests/acloca18.test index 22feb23..7047eea 100755 --- a/tests/acloca18.test +++ b/tests/acloca18.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2005, 2006, 2008 Free Software Foundation, Inc. +# Copyright (C) 2005, 2006, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -86,7 +86,7 @@ grep macro23 foo ACLOCAL_TESTSUITE_FLAGS='-I 4 -I 1 -I 2' rm -f foo -$ACLOCAL --install 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e 1 $ACLOCAL --install grep AM_MACRO2 stderr ACLOCAL_TESTSUITE_FLAGS='-I 4 -I 1' @@ -105,9 +105,7 @@ AC_DEFUN([AM_MACRO1], [echo macro1d >> foo]) AC_DEFUN([AM_MACRO2], [echo macro2d >> foo]) EOF rm -f foo -$ACLOCAL --diff=diff >output 2>stderr -cat stderr -cat output -grep '#serial 456' output +run_command $ACLOCAL --diff=diff +grep '#serial 456' stdout test ! -f 4/m1.m4 grep 'installing.*4/m1.m4' stderr diff --git a/tests/aclocal.test b/tests/aclocal.test index 6b87a9d..945b363 100755 --- a/tests/aclocal.test +++ b/tests/aclocal.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1998, 2001, 2002, 2004, 2008 Free Software Foundation, Inc. +# Copyright (C) 1998, 2001, 2002, 2004, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -23,15 +23,15 @@ set -e $ACLOCAL --output=fred test -f fred -$ACLOCAL --output 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e 1 $ACLOCAL --output grep 'option.*--output.*an argument' stderr grep help stderr -$ACLOCAL --unknown-option 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e 1 $ACLOCAL --unknown-option grep 'unrecognized.*--unknown-option' stderr grep help stderr -$ACLOCAL --ver 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e 1 $ACLOCAL --ver grep 'unrecognized.*--ver' stderr grep help stderr diff --git a/tests/aclocal3.test b/tests/aclocal3.test index a550e50..a6f26a5 100755 --- a/tests/aclocal3.test +++ b/tests/aclocal3.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1998, 2002, 2004 Free Software Foundation, Inc. +# Copyright (C) 1998, 2002, 2004, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -27,6 +27,5 @@ AC_DEFUN([GNOME_X_CHECKS], [ ]) END -$ACLOCAL -I macros 2>stderr -cat stderr +run_command $ACLOCAL -I macros grep 'macros/gnome.m4:2:.*AM_PATH_GTK.*not found' stderr diff --git a/tests/aclocal8.test b/tests/aclocal8.test index ac10aff..4cf7d73 100755 --- a/tests/aclocal8.test +++ b/tests/aclocal8.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2003, 2004 Free Software Foundation, Inc. +# Copyright (C) 2003, 2004, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -34,9 +34,8 @@ EOF echo 'AC_DEFUN([MACRO1],)' >m4/macro1.m4 echo 'AC_DEFUN([MACRO2], [AC_REQUIRE([AM_UNUSED_MACRO])])' >m4/macro2.m4 -$ACLOCAL -I m4 >output 2>&1 -cat output -test 0 = `wc -l <output` +run_command -m $ACLOCAL -I m4 +test -s stdall && Exit 1 grep macro1.m4 aclocal.m4 grep macro2.m4 aclocal.m4 && Exit 1 : diff --git a/tests/acsilent.test b/tests/acsilent.test index 6ac7863..e250092 100755 --- a/tests/acsilent.test +++ b/tests/acsilent.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1996, 2001, 2002, 2008 Free Software Foundation, Inc. +# Copyright (C) 1996, 2001, 2002, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -32,5 +32,6 @@ module=[$1] AC_SUBST(module)]) END -$ACLOCAL > output 2>&1 || { cat output; Exit 1; } -test -z "`cat output`" +run_command -m $ACLOCAL +test -s stdall && Exit 1 +: diff --git a/tests/alpha.test b/tests/alpha.test index 1ee8ed6..1410b9d 100755 --- a/tests/alpha.test +++ b/tests/alpha.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1998, 2001, 2002, 2003, 2008 Free Software Foundation, Inc. +# Copyright (C) 1998, 2001, 2002, 2003, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -62,8 +62,7 @@ $AUTOMAKE ./configure # make distdir should fail because NEWS does not mention 1.0a -$MAKE check 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_command -e FAIL $MAKE check grep 'NEWS not updated' stderr test ! -f works diff --git a/tests/ammissing.test b/tests/ammissing.test index 61d360a..cbed710 100755 --- a/tests/ammissing.test +++ b/tests/ammissing.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1997, 2002, 2004 Free Software Foundation, Inc. +# Copyright (C) 1997, 2002, 2004, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -20,6 +20,5 @@ echo AM_ZARDOZ >> configure.in -$ACLOCAL 2>stderr -cat stderr +run_command $ACLOCAL grep 'configure.in:.*AM_ZARDOZ.*not found' stderr diff --git a/tests/ansi3.test b/tests/ansi3.test index 2bbcaf7..95e4fc1 100755 --- a/tests/ansi3.test +++ b/tests/ansi3.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1998, 1999, 2001, 2002, 2003 Free Software Foundation, Inc. +# Copyright (C) 1998, 1999, 2001, 2002, 2003, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -53,8 +53,7 @@ export CFLAGS # Linux boxes (Red Hat 5.1 in particular). $ACLOCAL $AUTOCONF -$AUTOMAKE -a 2>stderr -cat stderr +run_command $AUTOMAKE -a grep 'Makefile.am:1:.*ansi2knr\.c' stderr CC='gcc' ./configure ANSI2KNR=./ansi2knr U=_ $MAKE -e diff --git a/tests/ansi3b.test b/tests/ansi3b.test index 8f254ad..447b690 100755 --- a/tests/ansi3b.test +++ b/tests/ansi3b.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1998, 1999, 2001, 2002, 2003 Free Software Foundation, Inc. +# Copyright (C) 1998, 1999, 2001, 2002, 2003, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -52,8 +52,7 @@ export CFLAGS # Linux boxes (Red Hat 5.1 in particular). $ACLOCAL $AUTOCONF -$AUTOMAKE -a 2>stderr -cat stderr +run_command $AUTOMAKE -a grep 'configure.in:2:.*ansi2knr\.1' stderr CC='gcc' ./configure ANSI2KNR=./ansi2knr U=_ $MAKE -e diff --git a/tests/ccnoco3.test b/tests/ccnoco3.test index 2e9cd37..93731a1 100755 --- a/tests/ccnoco3.test +++ b/tests/ccnoco3.test @@ -74,8 +74,7 @@ CC=`pwd`/../Mycomp export CC ../configure -$MAKE 2>stderr || { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_command $MAKE grep 'mv.*the same file' stderr && Exit 1 Exit 0 diff --git a/tests/check8.test b/tests/check8.test index 7936582..a087cb7 100755 --- a/tests/check8.test +++ b/tests/check8.test @@ -67,10 +67,7 @@ $AUTOMAKE -a unset TESTS || : ./configure -AM_COLOR_TESTS=always $MAKE -e check >stdout 2>stderr && - { cat stdout; cat stderr >&2; Exit 1; } -cat stdout -cat stderr >&2 +run_command -e FAIL -- env AM_COLOR_TESTS=always $MAKE -e check grep 'XPASS.* foo$' stdout grep '^[^X]*PASS.* sub/foo$' stdout grep '^[^X]*PASS.* bar' stdout @@ -87,8 +84,7 @@ $MAKE distclean mkdir build cd build ../configure -$MAKE check >stdout && { cat stdout; Exit 1; } -cat stdout +run_command -e FAIL $MAKE check # Note: we are not grepping for the space here, due to the Solaris make VPATH # rewriting (if we fix that, we can still write a separate test for it). grep 'XPASS.*foo$' stdout diff --git a/tests/cond39.test b/tests/cond39.test index 79eaa70..1e44866 100755 --- a/tests/cond39.test +++ b/tests/cond39.test @@ -1,5 +1,5 @@ #!/bin/sh -# Copyright (C) 2008 Free Software Foundation, Inc. +# Copyright (C) 2008, 2009 Free Software Foundation, Inc. # # This file is part of GNU Automake. # @@ -82,8 +82,7 @@ $AUTOCONF $AUTOMAKE --add-missing ./configure COND=true -$MAKE 2>stderr -cat stderr +run_command $MAKE grep 'overriding commands' stderr && Exit 1 $MAKE sure-exist ./prog1 && Exit 1 @@ -97,8 +96,7 @@ $MAKE sure-exist $MAKE distclean ./configure COND=false -$MAKE 2>stderr -cat stderr +run_command $MAKE grep 'overriding commands' stderr && Exit 1 ./prog1 ./sub/prog2 diff --git a/tests/defs.in b/tests/defs.in index 61182b6..2b0d98d 100644 --- a/tests/defs.in +++ b/tests/defs.in @@ -20,19 +20,19 @@ # Defines for Automake testing environment. # Tom Tromey <tromey@...> -# Be Bourne compatible. -# (Snippet copied from configure's initialization in Autoconf 2.59c.) -if test -n "${ZSH_VERSION+set}" && (emulate sh) >/dev/null 2>&1; then +# Be more Bourne compatible +# (Snippet copied from configure's initialization in Autoconf 2.64) +DUALCASE=1; export DUALCASE # for MKS sh +if test -n "${ZSH_VERSION+set}" && (emulate sh) >/dev/null 2>&1; then : emulate sh NULLCMD=: - # Zsh 3.x and 4.x performs word splitting on ${1+"$@"}, which + # Pre-4.2 versions of Zsh do word splitting on ${1+"$@"}, which # is contrary to our usage. Disable this feature. alias -g '${1+"$@"}'='"$@"' + setopt NO_GLOB_SUBST else case `(set -o) 2>/dev/null` in *posix*) set -o posix;; esac fi -BIN_SH=xpg4; export BIN_SH # for Tru64 -DUALCASE=1; export DUALCASE # for MKS sh # Ensure we are running from the right directory. test -f ./defs || { @@ -314,6 +314,7 @@ unset oldincludedir pdfdir psdir sbindir sharedstatedir sysconfdir # backward-compatible behavior. unset DISABLE_HARD_ERRORS unset TESTS +unset TEST_LOG_COMPILER unset TEST_LOGS unset RECHECK_LOGS unset VERBOSE @@ -396,26 +397,91 @@ is_newest () test -z "$is_newest_files" } +# run_command [-e STATUS] [-i FILE] [-m] [--] COMMAND [ARGUMENTS..] +# ----------------------------------------------------------------- +# Run the given COMMAND with ARGUMENTS and fail if COMMAND does not exit +# with STATUS. If status is "VOID" or "IGNORE", any exit value of the +# command is acceptable. If STATUS is "FAIL", then any exit value of the +# command *but 0* is acceptable. Default STATUS is `0'. +# Also, save standard output and standard error from COMMAND, by default +# respectively in files `stdout' and `stderr' (in the current directory), +# or togheter in the file `stdall' (in the current directory) if the `-m' +# option is given. +run_command () +{ + set +x # xtrace verbosity temporarly disabled in `run_command' + run_exitcode_expected=0 + run_mix_stdout_and_stderr=no + while test $# -gt 0; do + case $1 in + -e) run_exitcode_expected=$2; shift;; + -m) run_mix_stdout_and_stderr=yes;; + --) shift; break;; + -?) echo "run_commmand(): invalid switch \`$1'" >&2; Exit 99;; + *) break;; + esac + shift + done + case $# in + 0) echo "run_command(): missing COMMAND argument" >&2; Exit 99;; + *) run_cmd=$1; shift;; + esac + run_exitcode_got=0 + if test x"$run_mix_stdout_and_stderr" = x"yes"; then + run_evald_cmd='"$run_cmd" ${1+"$@"} >stdall 2>&1' + else + run_evald_cmd='"$run_cmd" ${1+"$@"} >stdout 2>stderr' + fi + run_evald_cmd="$run_evald_cmd || run_exitcode_got=\$?" + eval "$run_evald_cmd" + if test x"$run_mix_stdout_and_stderr" = x"yes"; then + echo "=== stdout and stderr" + cat stdall + echo "===" + else + echo "=== stderr" >&2 + cat stderr >&2 + echo "===" >&2 + echo "=== stdout" + cat stdout + echo "===" + fi + case $run_exitcode_expected in + VOID|void|IGNORE|ignore|IGNORED|ignored|$run_exitcode_got) + run_rc=0 + ;; + FAIL|fail|FAILURE|failure) + if test $run_exitcode_got -gt 0; then + run_rc=0 + else + run_rc=1 + fi + ;; + *) + run_rc=1 + ;; + esac + echo "run_command: exit status $run_exitcode_got (expecting" \ + "$run_exitcode_expected)" + set -x # reactivating temporarly turned-off xtrace verbosity + return $run_rc +} # AUTOMAKE_run status [options...] # -------------------------------- -# Run Automake with OPTIONS, and fail if automake +# Run Automake with OPTIONS, and make the testcase FAIL if automake # does not exit with STATUS. AUTOMAKE_run () { - expected_exitcode=$1 + am_run_expected_exitcode=$1 shift - exitcode=0 - $AUTOMAKE ${1+"$@"} >stdout 2>stderr || exitcode=$? - cat stderr >&2 - cat stdout - test $exitcode = $expected_exitcode || Exit 1 + run_command -e $am_run_expected_exitcode -- $AUTOMAKE ${1+"$@"} || Exit 1 } # AUTOMAKE_fails [options...] # --------------------------- -# Run Automake with OPTIONS, and fail if automake -# does not exit with STATUS. +# Run Automake with OPTIONS, and make the testcase FAIL if automake +# does not exit with status 1. AUTOMAKE_fails () { AUTOMAKE_run 1 ${1+"$@"} diff --git a/tests/depcomp2.test b/tests/depcomp2.test index 4653c90..3abe7ba 100755 --- a/tests/depcomp2.test +++ b/tests/depcomp2.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2001, 2002 Free Software Foundation, Inc. +# Copyright (C) 2001, 2002, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -20,6 +20,8 @@ required=gcc . ./defs || Exit 1 +set -e + cat > configure.in << 'END' AC_INIT(subdir/foo.c) AM_INIT_AUTOMAKE(nonesuch, nonesuch) @@ -48,5 +50,6 @@ export CFLAGS $ACLOCAL || Exit 1 $AUTOMAKE --add-missing || Exit 1 $AUTOCONF || Exit 1 -./configure CC='gcc' 2>error.log -test -z "`cat error.log`" +run_command ./configure CC='gcc' +test -s stderr && Exit 1 +: diff --git a/tests/depcomp4.test b/tests/depcomp4.test index 7d22894..70cbd57 100755 --- a/tests/depcomp4.test +++ b/tests/depcomp4.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2003 Free Software Foundation, Inc. +# Copyright (C) 2003, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -51,7 +51,6 @@ $AUTOMAKE -a grep am_cv_CC_dependencies_compiler_type configure ./configure am_cv_CC_dependencies_compiler_type=makedepend -$MAKE >output 2>&1 -cat output -grep 'unknown directive' output && Exit 1 +run_command -m $MAKE +grep 'unknown directive' stdall && Exit 1 $MAKE grepdeps diff --git a/tests/depend5.test b/tests/depend5.test index 0d9be3b..52e4169 100755 --- a/tests/depend5.test +++ b/tests/depend5.test @@ -65,10 +65,7 @@ for arg in Makefile \ "--file=$name" do rm -rf .deps _deps - ./config.status "$arg" depfiles >stdout 2>stderr || - { cat stdout; cat stderr >&2; Exit 1; } - cat stdout - cat stderr >&2 + run_command ./config.status "$arg" depfiles grep '[Nn]o such file' stderr && Exit 1 if test -n "$depdir"; then diff --git a/tests/depend6.test b/tests/depend6.test index de2adf9..5ce2208 100755 --- a/tests/depend6.test +++ b/tests/depend6.test @@ -39,9 +39,6 @@ END $ACLOCAL $AUTOMAKE $AUTOCONF -./configure --disable-dependency-tracking 2>stderr || { - stat=$?; cat stderr >&2; Exit $stat; -} -cat stderr >&2 -grep shift stderr && Exit 1 +run_command ./configure --disable-dependency-tracking +grep 'shift' stderr && Exit 1 : diff --git a/tests/discover.test b/tests/discover.test index 36293f7..c62f1b4 100755 --- a/tests/discover.test +++ b/tests/discover.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1996, 1997, 2000, 2001, 2002 Free Software Foundation, Inc. +# Copyright (C) 1996, 1997, 2000, 2001, 2002, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -34,5 +34,6 @@ END : > fsusage.c $ACLOCAL || Exit 1 -$AUTOMAKE 2>&1 | $FGREP fsusage.c || Exit 1 +run_command -e IGNORE -m $AUTOMAKE +$FGREP fsusage.c stdall || Exit 1 Exit 0 diff --git a/tests/distcom3.test b/tests/distcom3.test index 04bf365..fdcc31a 100755 --- a/tests/distcom3.test +++ b/tests/distcom3.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2001, 2002, 2003, 2004, 2006 Free Software Foundation, Inc. +# Copyright (C) 2001, 2002, 2003, 2004, 2006, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -34,9 +34,8 @@ END $ACLOCAL # Should not warn about missing README, since it is a target. -$AUTOMAKE --add-missing --gnu >stdout 2>&1 -cat stdout -grep README stdout && Exit 1 +run_command -m $AUTOMAKE --add-missing --gnu +grep README stdall && Exit 1 sed -n -e '/^DIST_COMMON =.*\\$/ { :loop diff --git a/tests/fn99.test b/tests/fn99.test index 982a989..611c767 100755 --- a/tests/fn99.test +++ b/tests/fn99.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2004, 2008 Free Software Foundation, Inc. +# Copyright (C) 2004, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -47,8 +47,7 @@ do touch x done) || Exit 77 -$MAKE dist 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_command -e FAIL $MAKE dist grep 'filenames are too long' stderr test 2 = `grep 12345678 stderr | wc -l` : diff --git a/tests/fn99subdir.test b/tests/fn99subdir.test index 465f2af..7055e85 100755 --- a/tests/fn99subdir.test +++ b/tests/fn99subdir.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2006, 2008 Free Software Foundation, Inc. +# Copyright (C) 2006, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -73,8 +73,7 @@ do touch x done) -$MAKE dist 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_command -e FAIL $MAKE dist grep 'filenames are too long' stderr test 1 = `grep 12345678 stderr | wc -l` : diff --git a/tests/gnits2.test b/tests/gnits2.test index 4c5d54f..a85b096 100755 --- a/tests/gnits2.test +++ b/tests/gnits2.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2002, 2003, 2004, 2007 Free Software Foundation, Inc. +# Copyright (C) 2002, 2003, 2004, 2007, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -105,6 +105,6 @@ cd build ../configure "--prefix=`pwd`/../inst-dir" --program-prefix=p $MAKE all $MAKE test-install -$MAKE -k installcheck 2>stderr || : # Never trust the exit status of make -k. -cat stderr +# Never trust the exit status of make -k. +run_command -e IGNORE $MAKE -k installcheck $MAKE grep-stderr diff --git a/tests/gnits3.test b/tests/gnits3.test index 7a08fb8..4d90684 100755 --- a/tests/gnits3.test +++ b/tests/gnits3.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2002, 2007 Free Software Foundation, Inc. +# Copyright (C) 2002, 2007, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -83,8 +83,8 @@ cd build ../configure "--prefix=`pwd`/../inst-dir" --program-prefix=p $MAKE $MAKE install -$MAKE -k installcheck 2>stderr || : # Never trust the exit status of make -k. -cat stderr +# Never trust the exit status of make -k. +run_command -e IGNORE $MAKE -k installcheck $MAKE grep-stderr # Make sure there is no more error when all targets are exempted. diff --git a/tests/help.test b/tests/help.test index d4d7110..cae8967 100755 --- a/tests/help.test +++ b/tests/help.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2003, 2008 Free Software Foundation, Inc. +# Copyright (C) 2003, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -32,7 +32,7 @@ $AUTOMAKE --version $AUTOMAKE --help # aclocal and automake cannot work without configure.ac or configure.in -$ACLOCAL 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e 1 $ACLOCAL grep configure.ac stderr grep configure.in stderr AUTOMAKE_fails diff --git a/tests/init.test b/tests/init.test index 38ec681..f793691 100755 --- a/tests/init.test +++ b/tests/init.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2006, 2008 Free Software Foundation, Inc. +# Copyright (C) 2006, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -27,7 +27,11 @@ AM_INIT_AUTOMAKE END # The error message should mension AC_INIT, not AC_PACKAGE_VERSION. -($ACLOCAL && $AUTOCONF) 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_command -e FAILURE $ACLOCAL || aclocal_success=yes +mv stderr aclocal-stderr +run_command -e FAILURE $AUTOCONF || autoconf_success=yes +mv stderr autoconf-stderr +test x"$aclocal_sucess#$autoconf_success" = x"yes#yes" && Exit 1 +cat aclocal-stderr autoconf-stderr > stderr grep AC_PACKAGE_VERSION stderr && Exit 1 grep AC_INIT stderr diff --git a/tests/instsh.test b/tests/instsh.test index 61f5f9e..fe3d485 100755 --- a/tests/instsh.test +++ b/tests/instsh.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1996, 1999, 2001, 2002, 2003 Free Software Foundation, Inc. +# Copyright (C) 1996, 1999, 2001, 2002, 2003, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -32,10 +32,9 @@ mv Makefile.am configure.in frob/ cd frob $ACLOCAL -$AUTOMAKE --add-missing > output 2>&1 +run_command -m $AUTOMAKE --add-missing # Only one `/' should appear in the output. -cat output -grep '/.*/' output && Exit 1 +grep '/.*/' stdall && Exit 1 test -f install-sh diff --git a/tests/ldadd.test b/tests/ldadd.test index 005f9d8..0a9907d 100755 --- a/tests/ldadd.test +++ b/tests/ldadd.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1997, 1999, 2000, 2001, 2002 Free Software Foundation, Inc. +# Copyright (C) 1997, 1999, 2000, 2001, 2002, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -39,5 +39,6 @@ END : > q.c $ACLOCAL || Exit 1 -$AUTOMAKE 2>&1 | grep LDADD || Exit 1 +run_command -e 1 -m $AUTOMAKE +grep LDADD stdall || Exit 1 Exit 0 diff --git a/tests/libtool7.test b/tests/libtool7.test index a2f912f..f8706d9 100755 --- a/tests/libtool7.test +++ b/tests/libtool7.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2003, 2004, 2007 Free Software Foundation, Inc. +# Copyright (C) 2003, 2004, 2007, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -82,19 +82,16 @@ $AUTOCONF $AUTOMAKE --add-missing --copy ./configure "--prefix=`pwd`/_inst" -env LIBTOOLFLAGS=--silent $MAKE print >output 2>&1 -cat output -grep '1BEG: libmod1.la mod2.la :END1' output -grep '2BEG: mod2.la :END2' output -grep '3BEG: .*silent.*silent.* :END3' output +run_command -m env LIBTOOLFLAGS=--silent $MAKE print +grep '1BEG: libmod1.la mod2.la :END1' stdall +grep '2BEG: mod2.la :END2' stdall +grep '3BEG: .*silent.*silent.* :END3' stdall test 2 -le `grep mod2_la_LIBTOOLFLAGS Makefile | wc -l` $MAKE -env LIBTOOLFLAGS=--silent $MAKE install >output 2>&1 -cat output -grep 'silent.*silent.*prg' output -grep 'silent.*silent.*libmod1' output +run_command -m env LIBTOOLFLAGS=--silent $MAKE install +grep 'silent.*silent.*prg' stdall +grep 'silent.*silent.*libmod1' stdall -env LIBTOOLFLAGS=--silent $MAKE uninstall >output 2>&1 -cat output -grep 'silent.*silent.*libmod1' output +run_command -m env LIBTOOLFLAGS=--silent $MAKE uninstall +grep 'silent.*silent.*libmod1' stdall diff --git a/tests/libtool9.test b/tests/libtool9.test index c0d9688..94e617b 100755 --- a/tests/libtool9.test +++ b/tests/libtool9.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2005 Free Software Foundation, Inc. +# Copyright (C) 2005, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -88,10 +88,11 @@ $AUTOCONF $AUTOMAKE --add-missing --copy ./configure -env LDFLAGS=ldflags AM_LDFLAGS=am_ldflags libmod1_la_LDFLAGS=lm1_la_ldflags \ +run_command -m env \ + LDFLAGS=ldflags AM_LDFLAGS=am_ldflags libmod1_la_LDFLAGS=lm1_la_ldflags \ CFLAGS=cflags AM_CFLAGS=am_cflags prg2_CFLAGS=prg2_cflags \ - $MAKE -e print >output 2>&1 -cat output + $MAKE -e print +mv stdall output grep '1BEG: libmod1.la mod2.la :END1' output grep '2BEG: mod2.la :END2' output grep '3BEG:.* am_cflags cflags .*lm1_la_ldflags ldflags.* :END3' output diff --git a/tests/lisp3.test b/tests/lisp3.test index 809d491..086173d 100755 --- a/tests/lisp3.test +++ b/tests/lisp3.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2003, 2008 Free Software Foundation, Inc. +# Copyright (C) 2003, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -61,9 +61,8 @@ test -f am-one.elc unique=0a3346e2af8a689b85002b53df09142a $sleep echo "(message \"$unique\")(provide 'am-three)" > am-three.el -$MAKE >output 2>&1 -cat output -grep $unique output +run_command -m $MAKE +grep $unique stdall # It should also work for VPATH-builds. $MAKE distcheck diff --git a/tests/ltorder.test b/tests/ltorder.test index 0f69211..1e6a03a 100755 --- a/tests/ltorder.test +++ b/tests/ltorder.test @@ -1,5 +1,5 @@ #!/bin/sh -# Copyright (C) 2008 Free Software Foundation, Inc. +# Copyright (C) 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -53,8 +53,7 @@ $AUTOMAKE --add-missing ./configure "--prefix=`pwd`/inst" $MAKE -$MAKE install 2>stderr || { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_command $MAKE install grep 'has not been installed' stderr && Exit 1 $MAKE uninstall diff --git a/tests/man4.test b/tests/man4.test index aa86a9e..7424c72 100755 --- a/tests/man4.test +++ b/tests/man4.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2008 Free Software Foundation, Inc. +# Copyright (C) 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -88,11 +88,9 @@ mkdir build cd build ../configure $MAKE -$MAKE dist 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_command -e FAIL $MAKE dist grep 'install help2man' stderr -$MAKE distcheck 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_command -e FAIL $MAKE distcheck grep 'install help2man' stderr rm -f ../help2man diff --git a/tests/mdate.test b/tests/mdate.test index 704b393..e63bff9 100755 --- a/tests/mdate.test +++ b/tests/mdate.test @@ -1,5 +1,5 @@ #!/bin/sh -# Copyright (C) 1996, 1997, 2001, 2002 Free Software Foundation, Inc. +# Copyright (C) 1996, 1997, 2001, 2002, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -31,4 +31,5 @@ END : > texinfo.tex $ACLOCAL || Exit 1 -$AUTOMAKE 2>&1 > /dev/null | grep 'required file.*mdate-sh' +run_command -e IGNORED $AUTOMAKE +grep 'required file.*mdate-sh' stderr diff --git a/tests/missing3.test b/tests/missing3.test index 07ef779..9b48368 100755 --- a/tests/missing3.test +++ b/tests/missing3.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2004, 2008 Free Software Foundation, Inc. +# Copyright (C) 2004, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -22,24 +22,24 @@ set -e # b7cb8259 assumed not to exist. -./missing b7cb8259 --version 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e FAIL ./missing b7cb8259 --version grep . stderr && Exit 1 -./missing b7cb8259 --grep 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e FAIL ./missing b7cb8259 --grep grep WARNING stderr -./missing --run b7cb8259 --version && Exit 1 -./missing --run b7cb8259 --grep 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e FAIL ./missing --run b7cb8259 --version +run_command -e FAIL ./missing --run b7cb8259 --grep grep WARNING stderr # missing itself it known to exist :) -./missing ./missing --version 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e FAIL ./missing ./missing --version grep . stderr && Exit 1 -./missing ./missing --grep 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e FAIL ./missing ./missing --grep grep WARNING stderr -./missing --run ./missing --version 2>stderr +run_command ./missing --run ./missing --version grep . stderr && Exit 1 -./missing --run ./missing --grep 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e FAIL ./missing --run ./missing --grep grep WARNING stderr && Exit 1 grep Unknown stderr diff --git a/tests/missing4.test b/tests/missing4.test index 1015828..c6c3b7e 100755 --- a/tests/missing4.test +++ b/tests/missing4.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2006, 2007, 2008 Free Software Foundation, Inc. +# Copyright (C) 2006, 2007, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -36,13 +36,11 @@ cmp aclocal.m4 aclocal.tmp && Exit 1 mv aclocal.tmp aclocal.m4 -$MAKE 2>stderr -cat stderr +run_command $MAKE grep 'You have another version of autoconf' stderr grep -c 'aclocal.m4:.*this file was generated for' stderr -$MAKE 2>stderr -cat stderr +run_command $MAKE grep 'You have another version of autoconf' stderr && Exit 1 grep 'aclocal.m4:.*this file was generated for' stderr && Exit 1 diff --git a/tests/missing6.test b/tests/missing6.test index b63a0d1..bf6df2d 100755 --- a/tests/missing6.test +++ b/tests/missing6.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2008 Free Software Foundation, Inc. +# Copyright (C) 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -31,7 +31,7 @@ rm -f configure.in touch Makefile.am $ACLOCAL -$AUTOCONF 2>stderr +run_command $AUTOCONF grep 'You have another version of autoconf' stderr grep 'aclocal.m4:.*this file was generated for' stderr $AUTOMAKE @@ -43,8 +43,7 @@ cmp configure.ac configure.tmp && Exit 1 mv configure.tmp configure.ac -$MAKE 2>stderr -cat stderr +run_command $MAKE grep 'You have another version of autoconf' stderr grep 'aclocal.m4:.*this file was generated for autoconf 9999a' stderr diff --git a/tests/obsolete.test b/tests/obsolete.test index e6601c9..daae69c 100755 --- a/tests/obsolete.test +++ b/tests/obsolete.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1996, 2001, 2002, 2007, 2008 Free Software Foundation, Inc. +# Copyright (C) 1996, 2001, 2002, 2007, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -27,7 +27,7 @@ END $PERL -ne '/AU_DEFUN\(\[(\w+)\]/ && print "$1\n"' \ "$testsrcdir/../m4/obsolete.m4" >> obs cat obs >> configure.in -$PERL -ne 'chomp; print "grep $_ stderr || Exit 1\n"; ' obs > obs.1 +$PERL -ne 'chomp; print "grep $_ stdall || Exit 1\n"; ' obs > obs.1 $PERL -ne 'chomp; print "grep $_ configure.in && Exit 1\n"; ' obs > obs.2 # Sanity check. Make sure we have added something to configure.in. @@ -36,7 +36,7 @@ test `cat configure.in | wc -l` -gt 1 || Exit 1 $ACLOCAL || Exit 1 # Expect Autoconf to complain about each of the macros in obs. -$AUTOCONF -Wobsolete >stderr 2>&1 +run_command -m $AUTOCONF -Wobsolete . ./obs.1 # Make sure Autoupdate remove each of these macros. $AUTOUPDATE || Exit 1 diff --git a/tests/output-order.test b/tests/output-order.test index 856baa2..852c194 100755 --- a/tests/output-order.test +++ b/tests/output-order.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2008 Free Software Foundation, Inc. +# Copyright (C) 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -43,13 +43,12 @@ END $ACLOCAL $AUTOCONF rm -f missing install-sh -$AUTOMAKE --add-missing --copy 2>stderr -cat stderr >&2 +run_command $AUTOMAKE --add-missing --copy +mv stderr stder0 for i in 1 2 3 4 5 6; do rm -f missing install-sh INSTALL COPYING + run_command -e IGNORE $AUTOMAKE --add-missing --copy # The grep prevents a Heisenbug with the HP-UX shell and VERBOSE=yes. - $AUTOMAKE --add-missing --copy 2>&1 >/dev/null | - grep -v /dev/null | - diff - stderr + grep -v /dev/null stder0 | diff - stderr done diff --git a/tests/pr220.test b/tests/pr220.test index 25e1f0e..6644d28 100755 --- a/tests/pr220.test +++ b/tests/pr220.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2001, 2002, 2004, 2008 Free Software Foundation, Inc. +# Copyright (C) 2001, 2002, 2004, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -55,6 +55,5 @@ $AUTOMAKE -a cd build # configure should fail since we've done something invalid. -../configure 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_command -e FAIL ../configure grep NEVER_TRUE stderr diff --git a/tests/pr300-ltlib.test b/tests/pr300-ltlib.test index 01814a9..7ed0a2f 100755 --- a/tests/pr300-ltlib.test +++ b/tests/pr300-ltlib.test @@ -48,8 +48,7 @@ $ACLOCAL $AUTOCONF $AUTOMAKE --copy --add-missing ./configure --prefix "`pwd`/inst" -$MAKE V=1 >stdout -cat stdout +run_command $MAKE V=1 grep 'liba.la .*-rpath .*lib' stdout grep 'liba.la .*-rpath .*lib/subdir' stdout && Exit 1 @@ -58,8 +57,7 @@ grep 'libb.la .*-rpath .*lib/subdir' stdout test -f subdir/liba.la test -f subdir/libb.la -$MAKE install 2>stderr -cat stderr >&2 +run_command $MAKE install grep 'remember.*--finish' stderr && Exit 1 test -f inst/lib/liba.la diff --git a/tests/python11.test b/tests/python11.test index 0f15f37..920de88 100755 --- a/tests/python11.test +++ b/tests/python11.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2004, 2006, 2008 Free Software Foundation, Inc. +# Copyright (C) 2004, 2006, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -35,9 +35,7 @@ EOF $ACLOCAL $AUTOCONF -./configure >stdout 2>stderr && { cat stdout; cat stderr >&2; Exit 1; } -cat stdout -cat stderr >&2 +run_command -e FAILURE ./configure grep 'checking for IShouldNotExist1' stdout grep 'checking for IShouldNotExist2' stdout grep 'no suitable Python interpreter found' stderr diff --git a/tests/python4.test b/tests/python4.test index 11bb3ed..3ff44af 100755 --- a/tests/python4.test +++ b/tests/python4.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2003, 2008 Free Software Foundation, Inc. +# Copyright (C) 2003, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -33,6 +33,5 @@ $AUTOCONF $AUTOMAKE --add-missing # Simulate no Python -./configure PYTHON=: 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_command -e FAILURE ./configure PYTHON=: grep 'no suitable Python interpreter found' stderr diff --git a/tests/python5.test b/tests/python5.test index 86f5863..5695876 100755 --- a/tests/python5.test +++ b/tests/python5.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2003, 2008 Free Software Foundation, Inc. +# Copyright (C) 2003, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -34,6 +34,5 @@ $ACLOCAL $AUTOCONF $AUTOMAKE --add-missing -./configure 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_command -e FAILURE ./configure grep 'no suitable Python interpreter found' stderr diff --git a/tests/python6.test b/tests/python6.test index eb4fb4d..ed60e03 100755 --- a/tests/python6.test +++ b/tests/python6.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2003 Free Software Foundation, Inc. +# Copyright (C) 2003, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -34,6 +34,5 @@ $AUTOCONF $AUTOMAKE --add-missing # Simulate no Python -./configure PYTHON=: 2>stderr -cat stderr +run_command ./configure PYTHON=: grep 'GREP ME:' stderr diff --git a/tests/python7.test b/tests/python7.test index 62c1a1b..eb6a4dc 100755 --- a/tests/python7.test +++ b/tests/python7.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2003 Free Software Foundation, Inc. +# Copyright (C) 2003, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -34,6 +34,5 @@ $ACLOCAL $AUTOCONF $AUTOMAKE --add-missing -./configure 2>stderr -cat stderr +run_command ./configure grep 'GREP ME:' stderr diff --git a/tests/python8.test b/tests/python8.test index a50af22..5f100ab 100755 --- a/tests/python8.test +++ b/tests/python8.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2003 Free Software Foundation, Inc. +# Copyright (C) 2003, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -32,6 +32,5 @@ $ACLOCAL $AUTOCONF $AUTOMAKE --add-missing -./configure 2>stderr -cat stderr +run_command ./configure grep 'GREP ME' stderr diff --git a/tests/python9.test b/tests/python9.test index 9b23deb..c0fa7a5 100755 --- a/tests/python9.test +++ b/tests/python9.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2003 Free Software Foundation, Inc. +# Copyright (C) 2003, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -33,6 +33,5 @@ $ACLOCAL $AUTOCONF $AUTOMAKE --add-missing -./configure 2>stderr -cat stderr +run_command ./configure grep 'GREP ME' stderr diff --git a/tests/sanity.test b/tests/sanity.test index 1c96c78..1296381 100755 --- a/tests/sanity.test +++ b/tests/sanity.test @@ -36,14 +36,12 @@ cp ../install-sh ../missing . $ACLOCAL $AUTOCONF $AUTOMAKE -./configure 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr +run_command -e FAILURE ./configure grep 'unsafe absolute working directory' stderr cd .. mkdir build cd build -../unsafe$/configure 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr +run_command -e FAILURE ../unsafe$/configure grep 'unsafe srcdir' stderr : diff --git a/tests/subobj.test b/tests/subobj.test index df297f4..9d7882d 100755 --- a/tests/subobj.test +++ b/tests/subobj.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1999, 2001, 2002, 2004 Free Software Foundation, Inc. +# Copyright (C) 1999, 2001, 2002, 2004, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -32,8 +32,7 @@ wish_SOURCES = generic/a.c generic/b.c END $ACLOCAL -$AUTOMAKE --add-missing 2>stderr -cat stderr +run_command $AUTOMAKE --add-missing # Make sure compile is installed, and that Automake says so. grep 'install.*compile' stderr test -f compile diff --git a/tests/subobj9.test b/tests/subobj9.test index 99fe6e8..1f68721 100755 --- a/tests/subobj9.test +++ b/tests/subobj9.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2002, 2004, 2005 Free Software Foundation, Inc. +# Copyright (C) 2002, 2004, 2005, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -74,8 +74,8 @@ if test -n "`./libtool --help | grep tag=TAG`"; then fi $MAKE -$MAKE distcheck 2>&1 | tee out +run_command -m $MAKE distcheck # GNU Make used to complain that the Makefile contained two rules # for `src/.dirstamp' and `.//src/.dirstamp'. -grep 'overriding commands' out && Exit 1 +grep 'overriding commands' stdall && Exit 1 : diff --git a/tests/unused.test b/tests/unused.test index a5b4da6..68d5613 100755 --- a/tests/unused.test +++ b/tests/unused.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1998, 2002, 2003, 2008 Free Software Foundation, Inc. +# Copyright (C) 1998, 2002, 2003, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -29,5 +29,5 @@ AC_DEFUN([MACRO_1_2], echo 12) AC_DEFUN([MACRO_1_2_3], echo 123) END -$ACLOCAL 2> stderr || { cat stderr >&2; Exit 1; } -test -z "`cat stderr`" +run_command $ACLOCAL +test ! -s stderr diff --git a/tests/vars3.test b/tests/vars3.test index 3d10a90..9f11262 100755 --- a/tests/vars3.test +++ b/tests/vars3.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2002 Free Software Foundation, Inc. +# Copyright (C) 2002, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -38,8 +38,7 @@ EOF $ACLOCAL # Make sure this warning is print in the `portability' category. -$AUTOMAKE --warnings=no-error,none,portability 2>stderr -cat stderr +run_command $AUTOMAKE --warnings=no-error,none,portability # Lines number are printed in error message. # Use them to make sure errors are diagnosed against the right lines. diff --git a/tests/version8.test b/tests/version8.test index 91b11a0..ffe4a3e 100755 --- a/tests/version8.test +++ b/tests/version8.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2005, 2008 Free Software Foundation, Inc. +# Copyright (C) 2005, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -20,6 +20,5 @@ set -e echo 'AM_AUTOMAKE_VERSION([1.9])' >>configure.in -$ACLOCAL 2>stderr && { cat stderr >&2; Exit 0; } -cat stderr >&2 +run_command -e 1 $ACLOCAL $FGREP 'AM_INIT_AUTOMAKE([1.9])' stderr -- 1.6.3.3 |
|
|
Re: testsuite failures when test scripts are run with zsh
by Ralf Wildenhues
::
Rate this Message:
* Stefano Lattarini wrote on Sat, Oct 10, 2009 at 03:42:52AM CEST:
> At Friday 09 October 2009, Ralf Wildenhues wrote: > > I'd shorten this to: > > > > Use run_command throughout. > I'd prefer to keep it a bit more verbose. I settled for this: > "Use new subroutine run_command instead of hand-crafted redirections > of stdout and/or stderr." > Are you OK with this? Sure. > > Hmm, I see a few inconsistencies cropping up here. First, we > > already have AUTOMAKE_run. It has slightly different syntax. With > > your patch, some automake invocations that capture output use > > AUTOMAKE_run, while others use run_command. > > > > These inconsistencies should be resolved. I'm fine with having all > > automake invocations use AUTOMAKE_run. > Don't you think this should be done in a separate patch? I didn't make myself clear enough, sorry. What I meant was that we shouldn't change uses of $AUTOMAKE with redirections to 'run_command $AUTOMAKE' when we also have AUTOMAKE_run. We should only use one of the two, and the latter is already used. There is no need to convert uses of $AUTOMAKE which do not have redirections. > > > +run_CMD () > > > +{ > > > + # NOTE: all internal variables used here starts with the > > > `_run' + # prefix, to minimize possibility of name clashes with > > > global + # variables defined in user code. > > > + : 'entering run_CMD(): become quiet' > > > + set +x # xtrace verbosity stops here > > > > No need to comment the obvious, here as well as at the end of this > > function; these 5 lines can be replaced with > > set +x > Well, I thought that things were clearer the way I did, but if more > experienced developers find that comments to be too "patronizing" or > annoying, I think it's better to remove them. I'd just like to keep > two comments: > set +x # xtrace verbosity temporarly disabled in `run_command' > at the beginning, and: > set -x # reactivating temporarly turned-off xtrace verbosity > at the end. Objections? Fine with me. > > If you care about reusability of this function in a context where > > you can't be sure whether xtrace is enabled at this point, you can > > use something like this instead: > > case $i in *x*) run_xtrace=;; *) run_xtrace=:;; esac > > $run_xtrace set +x > > ... > > $run_xtrace set -x > Well, ATM I'd prefer to keep the function simpler. It can be made > more reusable and general later, if the need will arise. > What do you think? Sure. > > > + _run_evald_cmd="${_run_evald_cmd} || _run_exitcode=\$?" > > > + eval "${_run_evald_cmd}" > > > > Why not simplify these two lines to > > eval "${_run_evald_cmd}" > > run_exitcode=$? > > and drop the _run_exitcode initialization above? > This way, a failing `eval' would make the shell abort if `set -e' is > active. Or am I mistaken? Feedback needed. Yeah, so how about if eval "${_run_evald_cmd}"; then run_exitcode=0 else run_exitcode=$? fi > > > + if test x"${_run_mix_stdout_and_stderr}" = x"yes"; then > > > + echo "=== stdout and stderr" > > > > Instead of all the file boundary markers, can we just re-enable > > xtrace here? That makes the trace output look more similar to that > > we get from commands not run through this function. You can > > reorder this chunk of code to be after the exit code handling to > > not deal with xtrace twice. > Well, I liked the `===' marker: it sticked out clearly without being > obtrusive. The question is: why do you want it to stick out further than the rest of the trace output? I can't see any reason. > But if you prefer a stricter consistency, I might just > use 'echo "+ cat stdout" >&2' etc. instead of 'echo "=== stdout"' etc. You can have that easier by just using wrapping the output in "set -x"/ "set +x". BTW, your run_command doesn't do what it advertizes to do: it doesn't necessarily cause a test failure when it should, esp. when a test doesn't use 'set -e'. I didn't think of the need to Exit within run_command when I wrote my previous review; it makes the issue moot of reordering output and exit handling: output handling should come first. I don't understand why you'd want to make run_command not exit in case of failure, if case that was intentional. BTW2, do you have access to a Solaris or OpenBSD box to test this function with its shells? > > > # AUTOMAKE_run status [options...] > > > # -------------------------------- > > > -# Run Automake with OPTIONS, and fail if automake > > > +# Run Automake with OPTIONS, and make the testcase FAIL if > > > automake > > > > Why this change? I'd drop it, likewise for the comment to > > AUTOMAKE_fails. > It seemed clearer: it's not the function that it's failing, but the > whole testcase. OK, but please s/testcase/test/g, and the ChangeLog entry should mention this ((AUTOMAKE_run): Update comment.). > > > # does not exit with STATUS. > > > AUTOMAKE_run () > > > { > > > - expected_exitcode=$1 > > > + _am_run_expected_exitcode=$1 > or is OK to use `am_run_expected_exitcode' instead > of `_am_run_expected_exitcode'? Sure. > By the way, your observation has made me think: wouldn't it be better > to enable `set -e' in defs.in, so that all the test cases could have a > more uniform environment? This would require an audit of all tests that currently don't use set -e. This needs testing on Solaris, OpenBSD, NetBSD, Tru64, because of bugs and warts in their shell's implementation of errexit. > And another aside: I was about to modify also the file > `tests/dejagnu-p.test', before remembering it is automatically > generated (and, well, also before noticing it was a false positive). > > I think it would be useful to make the autogoenerated files (tests/defs, > tests/*-p.test, etc) readonly. It could be done in a separate patch. > What do you think? Hmm, I'm a bit leery of making things read-only, but making sure the files contain a "generated by ... " line near the top seems a good idea. Thanks, Ralf |
|
|
Re: testsuite failures when test scripts are run with zsh
by Stefano Lattarini
::
Rate this Message:
At Saturday 10 October 2009, Ralf Wildenhues <Ralf.Wildenhues@...>
wrote: > * Stefano Lattarini wrote on Sat, Oct 10, 2009 at 03:42:52AM CEST: > [CUT] > > > Hmm, I see a few inconsistencies cropping up here. First, we > > > already have AUTOMAKE_run. It has slightly different syntax. > > > With your patch, some automake invocations that capture output > > > use AUTOMAKE_run, while others use run_command. > > > > > > These inconsistencies should be resolved. I'm fine with having > > > all automake invocations use AUTOMAKE_run. > > > > Don't you think this should be done in a separate patch? > > I didn't make myself clear enough, sorry. What I meant was that we > shouldn't change uses of $AUTOMAKE with redirections to > 'run_command $AUTOMAKE' when we also have AUTOMAKE_run. We should > only use one of the two, and the latter is already used. There is > no need to convert uses of $AUTOMAKE which do not have > redirections. `run_command $AUTOMAKE' instead of `AUTOMAKE_run' and `run_command -e 1 $AUTOMAKE' instead of `AUTOMAKE_fails' (of course remembering to add a proper `|| Exit 1' in tests not using `set -e')? If you think about it, the testsuite don't have `ACLOCAL_run' or `ACLOCAL_fails', but simply uses `run_command $ACLOCAL' and `run_COMMAND -e 1 $ACLOCAL'. By the way, this change should require a small change in `tests/README' too. If you agree with it, I think it should be done with a distinct patch. Opinions? > [CUT] > > > > > + _run_evald_cmd="${_run_evald_cmd} || _run_exitcode=\$?" > > > > + eval "${_run_evald_cmd}" > > > > > > Why not simplify these two lines to > > > eval "${_run_evald_cmd}" > > > run_exitcode=$? > > > and drop the _run_exitcode initialization above? > > > > This way, a failing `eval' would make the shell abort if `set -e' > > is active. Or am I mistaken? Feedback needed. > > Yeah, so how about > if eval "${_run_evald_cmd}"; then > run_exitcode=0 > else > run_exitcode=$? > fi reliable there? In the meantime, I modified the patch to follow your advice. > > > > + if test x"${_run_mix_stdout_and_stderr}" = x"yes"; then > > > > + echo "=== stdout and stderr" > > > > > > Instead of all the file boundary markers, can we just re-enable > > > xtrace here? That makes the trace output look more similar to > > > that we get from commands not run through this function. You > > > can reorder this chunk of code to be after the exit code > > > handling to not deal with xtrace twice. > > > > Well, I liked the `===' marker: it sticked out clearly without > > being obtrusive. > > The question is: why do you want it to stick out further than the > rest of the trace output? I can't see any reason. > > > But if you prefer a stricter consistency, I might just > > use 'echo "+ cat stdout" >&2' etc. instead of 'echo "=== stdout"' > > etc. > > You can have that easier by just using wrapping the output in "set > -x"/ "set +x". different shells, so it's more consistent to temporarly re-enable traces). Patch modified accordingly to your suggestion. > > BTW, your run_command doesn't do what it advertizes to do: it > doesn't necessarily cause a test failure when it should, esp. when > a test doesn't use 'set -e'. I think that here there's a misunderstanding about the meaning of `fail': you mean "the testcase fails", while I mean "the function fails", i.e. it return a value != 0. Do you have a rewording to suggest to make things clearer? > I didn't think of the need to Exit within run_command when I wrote > my previous review; it makes the issue moot of reordering output > and exit handling: output handling should come first. Sorry, but I failed to parse this sentence. Can you reformulate it in a simpler way? > I don't understand why you'd want to make run_command not exit in > case of failure, in case that was intentional. Yes, it was intentional. I think it provides more flexibility (which is useful for example in the test `init.test',). Moreover, in the usual case, the test scripts are run with the `-e' flag on, so that an "uncatched" failing `run_command' call makes the testcase fail. In case `set -e' is not used, the testcase's writer should anyway be prepared to add manually add `|| Exit 1' wherever deemed appropriate. So I'd prefer not to ever call `Exit' in run_command. Objections? > BTW2, do you have access to a Solaris or OpenBSD box to test this > function with its shells? Alas no, but at least I installed the "Heirloom Shell": http://heirloom.sourceforge.net/sh.html which should be quite similar to the Solaris Shell: no `$(...)' for command substitution, no `${var#prefix}' expansions, common namespace for variables and functions, `while ...; do ...; done <file' spawning a subshell, and many other "goodies". I will test the final patch with that shell (and of course with zsh too, which is the very reason to have this patch in the first place). > > > > > # AUTOMAKE_run status [options...] > > > > # -------------------------------- > > > > -# Run Automake with OPTIONS, and fail if automake > > > > +# Run Automake with OPTIONS, and make the testcase FAIL if > > > > automake > > > > > > Why this change? I'd drop it, likewise for the comment to > > > AUTOMAKE_fails. > > > > It seemed clearer: it's not the function that it's failing, but > > the whole testcase. > > OK, but please s/testcase/test/g, and the ChangeLog entry should > mention this ((AUTOMAKE_run): Update comment.). > [CUT] > > By the way, your observation has made me think: wouldn't it be > > better to enable `set -e' in defs.in, so that all the test cases > > could have a more uniform environment? > > This would require an audit of all tests that currently don't use > set -e. This needs testing on Solaris, OpenBSD, NetBSD, Tru64, > because of bugs and warts in their shell's implementation of > errexit. Unfortunately, I don't have access to any of those system, so I'll have to drop the ball on this. > > And another aside: I was about to modify also the file > > `tests/dejagnu-p.test', before remembering it is automatically > > generated (and, well, also before noticing it was a false > > positive). > > > > I think it would be useful to make the autogoenerated files > > (tests/defs, tests/*-p.test, etc) readonly. It could be done in > > a separate patch. What do you think? > > Hmm, I'm a bit leery of making things read-only, but making sure > the files contain a "generated by ... " line near the top seems a > good idea. "unintentional" editing: I have ended up too many times modifying tests/defs instead of tests/defs.in, even if it has the customary `@configure_input@' line at the top (yeah, stupid me, but I lost time anyway). On the contrary, the fact that a file is read-only is a much clearer and outstanding indicator of the fact it should not be modified, and is anyway easily circumvented in the case you really need to modify that file. I'm stressing this because I think that making generated files readonly would be very very useful to the absent-minded developer or contributor (e.g., me). Regards, Stefano |
|
|
Re: testsuite failures when test scripts are run with zsh
by Jim Meyering
::
Rate this Message:
Stefano Lattarini wrote:
... >> Hmm, I'm a bit leery of making things read-only, but making sure >> the files contain a "generated by ... " line near the top seems a >> good idea. > In truth, I don't find that much useful in order to prevent > "unintentional" editing: I have ended up too many times modifying > tests/defs instead of tests/defs.in, even if it has the customary > `@configure_input@' line at the top (yeah, stupid me, but I lost time > anyway). On the contrary, the fact that a file is read-only is a much > clearer and outstanding indicator of the fact it should not be > modified, and is anyway easily circumvented in the case you really > need to modify that file. I'm stressing this because I think that > making generated files readonly would be very very useful to the > absent-minded developer or contributor (e.g., me). Hi Stefano and Ralf, Ralf, Why are you reluctant to make any generated file read-only? I've been lobbying for this in many projects for a long time, for exactly the same reasons. I've wasted too much time (even if it's just a minute or two per incident) by inadvertently modifying a generated file rather than the template from which it is generated. The only potential gotcha is that when redirecting to the temporary file, you have to be careful to remove the read-only file beforehand, or to ensure that it writable. |
|
|
Re: testsuite failures when test scripts are run with zsh
by Ralf Wildenhues
::
Rate this Message:
Hi Jim, Stefano,
* Jim Meyering wrote on Sun, Oct 11, 2009 at 01:41:25PM CEST: > Stefano Lattarini wrote: > ... > >> Hmm, I'm a bit leery of making things read-only, but making sure > >> the files contain a "generated by ... " line near the top seems a > >> good idea. > > In truth, I don't find that much useful in order to prevent > > "unintentional" editing: I have ended up too many times modifying > > tests/defs instead of tests/defs.in, even if it has the customary > > `@configure_input@' line at the top (yeah, stupid me, but I lost time > > anyway). On the contrary, the fact that a file is read-only is a much > > clearer and outstanding indicator of the fact it should not be > > modified, and is anyway easily circumvented in the case you really > > need to modify that file. I'm stressing this because I think that > > making generated files readonly would be very very useful to the > > absent-minded developer or contributor (e.g., me). > Ralf, > Why are you reluctant to make any generated file read-only? Basically, because I'm lazy. Some non-GNU systems have slightly different semantics wrt. read-only files, and I keep forgetting what the exact differences are, and am afraid to introduce silent bugs there. Also, here is another big warning sign that files are not pristine sources: these files live in the build tree, not the source tree (hmm, that's not true for a distribution tar ball currently). I realize that I'm weighing my time against other developer time here, and in any case the generated tests are not a big issue this way or another, so I guess these would be alright to change. > I've been lobbying for this in many projects for a long time, > for exactly the same reasons. I've wasted too much time > (even if it's just a minute or two per incident) by inadvertently > modifying a generated file rather than the template from which > it is generated. There are several prominent instances of generated files that are not read-only. The Makefile.in files that automake generates, the config.status file that configure generates, and the Makefile files that config.status generate, are not made read-only, neither is the libtool file that the Libtool macros/config.status generate. Making those files read-only will break other packages out there; while the practice to modify these files (either manually, but quite often also automatically) may not be best practices, it is definitely a practice we should not (try to) forbid, or code that we should break. So I'd oppose changing their modes. > The only potential gotcha is that when redirecting to the temporary > file, you have to be careful to remove the read-only file beforehand, > or to ensure that it writable. Yes, that's an issue. I think you also need -f with rm; and I think not all vi instances will overwrite such a file even with :w!, but haven't checked now. Cheers, Ralf |
|
|
Re: testsuite failures when test scripts are run with zsh
by Ralf Wildenhues
::
Rate this Message:
Hi Stefano,
* Stefano Lattarini wrote on Sat, Oct 10, 2009 at 03:04:59PM CEST: > At Saturday 10 October 2009, Ralf Wildenhues wrote: > > I didn't make myself clear enough, sorry. What I meant was that we > > shouldn't change uses of $AUTOMAKE with redirections to > > 'run_command $AUTOMAKE' when we also have AUTOMAKE_run. We should > > only use one of the two, and the latter is already used. There is > > no need to convert uses of $AUTOMAKE which do not have > > redirections. > I understand. But what about instead susbstituting everywhere > `run_command $AUTOMAKE' instead of `AUTOMAKE_run' and > `run_command -e 1 $AUTOMAKE' instead of `AUTOMAKE_fails' > (of course remembering to add a proper `|| Exit 1' in tests not > using `set -e')? First off, I think that run_command really should Exit when the command does not produce the intended status. It should not be necessary to do run_command -e 1 $command || Exit 1 That is much safer, and less maintenance. If we really need to run some command where we need to ignore the exit status, then we can still use run_command '$command || :' or make this command another function or so. Which brings me to the second inconsistent issue with this API: the 'eval'uation level of the command is part of the API. This is important, because when the absolute source and build directories contain white space in the name (and Automake mostly works in this case now), we should be doing the right thing. Then to your question above: yes it is ok to replace all instances of AUTOMAKE_run and AUTOMAKE_fails (there is no need to replace plain $AUTOMAKE without redirection). > If you think about it, the testsuite don't have `ACLOCAL_run' or > `ACLOCAL_fails', but simply uses `run_command $ACLOCAL' and > `run_COMMAND -e 1 $ACLOCAL'. > By the way, this change should require a small change in > `tests/README' too. > If you agree with it, I think it should be done with a distinct patch. Sounds good. > > Yeah, so how about > > if eval "${_run_evald_cmd}"; then > > run_exitcode=0 > > else > > run_exitcode=$? > > fi > This seems better; however, are we sure that the value of `$?' is > reliable there? Yes. > > BTW, your run_command doesn't do what it advertizes to do: it > > doesn't necessarily cause a test failure when it should, esp. when > > a test doesn't use 'set -e'. > I think that here there's a misunderstanding about the meaning of > `fail': you mean "the testcase fails", while I mean "the function > fails", i.e. it return a value != 0. Do you have a rewording to > suggest to make things clearer? "fail" is FAIL, and exit status != 0 is returning a nonzero exit status. > > I didn't think of the need to Exit within run_command when I wrote > > my previous review; it makes the issue moot of reordering output > > and exit handling: output handling should come first. > Sorry, but I failed to parse this sentence. Can you reformulate it in > a simpler way? What I was trying to say here is what I wrote above in my first paragraph of this message. > > I don't understand why you'd want to make run_command not exit in > > case of failure, in case that was intentional. > Yes, it was intentional. I think it provides more flexibility (which > is useful for example in the test `init.test',). Moreover, in the > usual case, the test scripts are run with the `-e' flag on, so that an > "uncatched" failing `run_command' call makes the testcase fail. > In case `set -e' is not used, the testcase's writer should anyway be > prepared to add manually add `|| Exit 1' wherever deemed appropriate. > So I'd prefer not to ever call `Exit' in run_command. Objections? Yes; see above. > > > > > # AUTOMAKE_run status [options...] > > > > > # -------------------------------- > > > > > -# Run Automake with OPTIONS, and fail if automake > > > > > +# Run Automake with OPTIONS, and make the testcase FAIL if > > > > > automake > > > > > > > > Why this change? I'd drop it, likewise for the comment to > > > > AUTOMAKE_fails. > > > > > > It seemed clearer: it's not the function that it's failing, but > > > the whole testcase. > > > > OK, but please s/testcase/test/g, and the ChangeLog entry should > > mention this ((AUTOMAKE_run): Update comment.). > Done. Also added `(AUTOMAKE_fail): Updated comment.' to the ChangeLog. Thinking about this again "make the test fail" is both bad English and not any clearer than "fail". But this API is going away anyway when you replace it all with run_command, so this is all moot. > > [CUT] > > > By the way, your observation has made me think: wouldn't it be > > > better to enable `set -e' in defs.in, so that all the test cases > > > could have a more uniform environment? > > > > This would require an audit of all tests that currently don't use > > set -e. This needs testing on Solaris, OpenBSD, NetBSD, Tru64, > > because of bugs and warts in their shell's implementation of > > errexit. > Unfortunately, I don't have access to any of those system, so I'll > have to drop the ball on this. I can test the final iteration of this patch. BTW, now that we have TEST_LOG_COMPILER, and correctly unset it in defs.in, too, we can set it in tests/Makefile.am and worry less about shells like Solaris /bin/sh. Of course, that would require us to warn that running tests directly (i.e., without 'make' in-between), might require to use a decent shell. Cheers, Ralf |
|
|
Re: testsuite failures when test scripts are run with zsh
by Stefano Lattarini
::
Rate this Message:
I attached a new version of the patch. Please note that it still
doesn't address the remarks and objections made in the present mail. At Tuesday 13 October 2009, Ralf Wildenhues <Ralf.Wildenhues@...> wrote: > Hi Stefano, > > > I understand. But what about instead susbstituting everywhere > > `run_command $AUTOMAKE' instead of `AUTOMAKE_run' and > > `run_command -e 1 $AUTOMAKE' instead of `AUTOMAKE_fails' > > (of course remembering to add a proper `|| Exit 1' in tests not > > using `set -e')? > > First off, I think that run_command really should Exit when the > command does not produce the intended status. It should not be > necessary to do run_command -e 1 $command || Exit 1 > > That is much safer, and less maintenance. scenario. However, we should then really add a similar subroutine (say `run_redirect' -- tell me if you have a better name) which only takes care of portably redirecting stdout/stderr of a command (and, obviously, rewrite `run_command' implementation to advantage of the new function). Is this OK with you? > If we really need to run some command where we need to ignore > the exit status, then we can still use > > run_command '$command || :' This is wrong, as currently `run_command' do not `eval' its COMMAND. And the exit status of $command would be lost, which might not be what we really want. > > or make this command another function or so. Yes, please. > > Which brings me to the second inconsistent issue with this API: the > 'eval'uation level of the command is part of the API. > This is important, because when the absolute source and build > directories contain white space in the name (and Automake mostly > works in this case now), we should be doing the right thing. I don't understand. The `eval' used in the run_command implementation is there just to provide a shortand: no argument passed by the caller is ever eval'd (and this is the right thing to do, I think). > Then to your question above: yes it is ok to replace all instances > of AUTOMAKE_run and AUTOMAKE_fails (there is no need to replace > plain $AUTOMAKE without redirection). > > > If you think about it, the testsuite don't have `ACLOCAL_run' or > > `ACLOCAL_fails', but simply uses `run_command $ACLOCAL' and > > `run_COMMAND -e 1 $ACLOCAL'. > > By the way, this change should require a small change in > > `tests/README' too. > > If you agree with it, I think it should be done with a distinct > > patch. > Sounds good. to say is that we should get rid of AUTOMAKE_run and AUTOMAKE_fails, not add ACLOCAL_run and ACLOCAL_fails (especially now that I'm going to follow your advice and make run_command use `Exit 1' on failures). Is this OK? > > > Yeah, so how about > > > if eval "${_run_evald_cmd}"; then > > > run_exitcode=0 > > > else > > > run_exitcode=$? > > > fi > > > > This seems better; however, are we sure that the value of `$?' is > > reliable there? > Yes. > > > BTW, your run_command doesn't do what it advertizes to do: it > > > doesn't necessarily cause a test failure when it should, esp. > > > when a test doesn't use 'set -e'. > > > > I think that here there's a misunderstanding about the meaning of > > `fail': you mean "the testcase fails", while I mean "the function > > fails", i.e. it return a value != 0. Do you have a rewording to > > suggest to make things clearer? > > "fail" is FAIL, and exit status != 0 is returning a nonzero exit > status. instead of "fail"? > > > [CUT] > > > > > > > By the way, your observation has made me think: wouldn't it > > > > be better to enable `set -e' in defs.in, so that all the test > > > > cases could have a more uniform environment? > > > > > > This would require an audit of all tests that currently don't > > > use set -e. This needs testing on Solaris, OpenBSD, NetBSD, > > > Tru64, because of bugs and warts in their shell's > > > implementation of errexit. > > > > Unfortunately, I don't have access to any of those system, so > > I'll have to drop the ball on this. > > I can test the final iteration of this patch. > BTW, now that we have TEST_LOG_COMPILER, and correctly unset it in > defs.in, too, we can set it in tests/Makefile.am and worry less > about shells like Solaris /bin/sh. Of course, that would require > us to warn that running tests directly (i.e., without 'make' > in-between), might require to use a decent shell. Or we could add proper code in `tests/defs.in', to make it re-execute the test scripts with CONFIG_SHELL. But this should be obviously done in a distinct patch. Regards, Stefano [Fix-testsuite-avoid-Zsh-related-problems-with-set-x.patch] From 4da7f4d6926f3001366be8e71b5288877ef34d73 Mon Sep 17 00:00:00 2001 From: Stefano Lattarini <stefano.lattarini@...> Date: Sun, 11 Oct 2009 14:49:58 +0200 Subject: [PATCH] Fix testsuite: avoid Zsh-related problems with `set -x' --- ChangeLog | 67 ++++++++++++++++++++++++++++++++++++++ tests/README | 10 ++++++ tests/acloca14.test | 4 +- tests/acloca17.test | 6 +-- tests/acloca18.test | 10 ++--- tests/aclocal.test | 8 ++-- tests/aclocal3.test | 5 +-- tests/aclocal8.test | 7 ++-- tests/acsilent.test | 7 ++-- tests/alpha.test | 5 +-- tests/ammissing.test | 5 +-- tests/ansi3.test | 5 +-- tests/ansi3b.test | 5 +-- tests/ccnoco3.test | 3 +- tests/check8.test | 8 +--- tests/cond39.test | 8 ++--- tests/defs.in | 83 +++++++++++++++++++++++++++++++++++++++++----- tests/depcomp2.test | 9 +++-- tests/depcomp4.test | 7 ++-- tests/depend5.test | 5 +-- tests/depend6.test | 7 +--- tests/discover.test | 5 ++- tests/distcom3.test | 7 ++-- tests/fn99.test | 5 +-- tests/fn99subdir.test | 5 +-- tests/gnits2.test | 6 ++-- tests/gnits3.test | 6 ++-- tests/help.test | 4 +- tests/init.test | 13 ++++++-- tests/instsh.test | 7 ++-- tests/ldadd.test | 5 ++- tests/libtool7.test | 23 ++++++------- tests/libtool9.test | 9 +++-- tests/lisp3.test | 7 ++-- tests/ltorder.test | 5 +-- tests/man4.test | 8 ++--- tests/mdate.test | 5 ++- tests/missing3.test | 18 +++++----- tests/missing4.test | 8 ++--- tests/missing6.test | 7 ++-- tests/obsolete.test | 6 ++-- tests/output-order.test | 11 +++--- tests/pr220.test | 5 +-- tests/pr300-ltlib.test | 6 +-- tests/python11.test | 6 +-- tests/python4.test | 5 +-- tests/python5.test | 5 +-- tests/python6.test | 5 +-- tests/python7.test | 5 +-- tests/python8.test | 5 +-- tests/python9.test | 5 +-- tests/sanity.test | 6 +-- tests/subobj.test | 5 +-- tests/subobj9.test | 6 ++-- tests/unused.test | 6 ++-- tests/vars3.test | 5 +-- tests/version8.test | 5 +-- 57 files changed, 313 insertions(+), 211 deletions(-) diff --git a/ChangeLog b/ChangeLog index 3286548..25223d1 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,70 @@ +2009-10-12 Stefano Lattarini <stefano.lattarini@...> + + Fix testsuite: avoid Zsh-related problems with `set -x'. + * tests/README: Describe Zsh 4.x `set -x' a.k.a. `xtrace' issue + and workaround with run_command. + * tests/defs.in (run_command): New function, to be used for + commands where standard error needs to be captured. + (AUTOMAKE_run): Reimplemented using run_command. Updated + comment. + (AUTOMAKE_fails): Updated comment. + * tests/acloca14.test: Use new subroutine run_command instead + of hand-crafted redirections of stdout and/or stderr. + * tests/acloca17.test: Likewise. + * tests/acloca18.test: Likewise. + * tests/aclocal.test: Likewise. + * tests/aclocal3.test: Likewise. + * tests/aclocal8.test: Likewise. + * tests/acsilent.test: Likewise. + * tests/alpha.test: Likewise. + * tests/ammissing.test: Likewise. + * tests/ansi3.test: Likewise. + * tests/ansi3b.test: Likewise. + * tests/ccnoco3.test: Likewise. + * tests/check8.test: Likewise. + * tests/cond39.test: Likewise. + * tests/defs.in: Likewise. + * tests/depcomp2.test: Likewise. + * tests/depcomp4.test: Likewise. + * tests/depend5.test: Likewise. + * tests/depend6.test: Likewise. + * tests/discover.test: Likewise. + * tests/distcom3.test: Likewise. + * tests/fn99.test: Likewise. + * tests/fn99subdir.test: Likewise. + * tests/gnits2.test: Likewise. + * tests/gnits3.test: Likewise. + * tests/help.test: Likewise. + * tests/init.test: Likewise. + * tests/instsh.test: Likewise. + * tests/ldadd.test: Likewise. + * tests/libtool7.test: Likewise. + * tests/libtool9.test: Likewise. + * tests/lisp3.test: Likewise. + * tests/ltorder.test: Likewise. + * tests/man4.test: Likewise. + * tests/mdate.test: Likewise. + * tests/missing3.test: Likewise. + * tests/missing4.test: Likewise. + * tests/missing6.test: Likewise. + * tests/obsolete.test: Likewise. + * tests/output-order.test: Likewise. + * tests/pr220.test: Likewise. + * tests/pr300-ltlib.test: Likewise. + * tests/python11.test: Likewise. + * tests/python4.test: Likewise. + * tests/python5.test: Likewise. + * tests/python6.test: Likewise. + * tests/python7.test: Likewise. + * tests/python8.test: Likewise. + * tests/python9.test: Likewise. + * tests/sanity.test: Likewise. + * tests/subobj.test: Likewise. + * tests/subobj9.test: Likewise. + * tests/unused.test: Likewise. + * tests/vars3.test: Likewise. + * tests/version8.test: Likewise. + 2009-10-11 Ralf Wildenhues <Ralf.Wildenhues@...> Improve description of the various *LINK variables. diff --git a/tests/README b/tests/README index 4068ad6..9b25be9 100644 --- a/tests/README +++ b/tests/README @@ -115,6 +115,16 @@ Do Use $sleep when you have to make sure that some file is newer than another. + To run a program and analyze its stderr, use the run_command function. + For example: + run_command PROG [ARGS...] + grep $pattern stderr + or (if you must check that nothing is written on stderr): + run_command PROG2 [ARGS...] + test ! -s stderr + See section "File Descriptors" in the Autoconf manual to understand why + this can be useful, or even required. + Use `cat' or `grep' to display (part of) files that may be interesting for debugging, so that when a user send a verbose output we don't have to ask him for more details. Display stderr diff --git a/tests/acloca14.test b/tests/acloca14.test index 8ddbb1a..cf9c484 100755 --- a/tests/acloca14.test +++ b/tests/acloca14.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2004, 2008 Free Software Foundation, Inc. +# Copyright (C) 2004, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -108,5 +108,5 @@ $MAKE testdist2 # Make sure aclocal diagnose missing included files with correct `file:line:'. rm -f b.m4 -$ACLOCAL 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e 1 $ACLOCAL grep 'a.m4:1:.*b.m4.*does not exist' stderr diff --git a/tests/acloca17.test b/tests/acloca17.test index 884f24b..788cb9d 100755 --- a/tests/acloca17.test +++ b/tests/acloca17.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2004, 2008 Free Software Foundation, Inc. +# Copyright (C) 2004, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -33,7 +33,5 @@ EOF # FIXME: We want autom4te's 'undefined required macro' warning to be fatal, # but have no means to say so to aclocal. We use WARNINGS=error instead. - -WARNINGS=error $ACLOCAL -I m4 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_command -e 1 env WARNINGS=error $ACLOCAL -I m4 grep 'configure.in:4:.*UNDEFINED_MACRO' stderr diff --git a/tests/acloca18.test b/tests/acloca18.test index 22feb23..7047eea 100755 --- a/tests/acloca18.test +++ b/tests/acloca18.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2005, 2006, 2008 Free Software Foundation, Inc. +# Copyright (C) 2005, 2006, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -86,7 +86,7 @@ grep macro23 foo ACLOCAL_TESTSUITE_FLAGS='-I 4 -I 1 -I 2' rm -f foo -$ACLOCAL --install 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e 1 $ACLOCAL --install grep AM_MACRO2 stderr ACLOCAL_TESTSUITE_FLAGS='-I 4 -I 1' @@ -105,9 +105,7 @@ AC_DEFUN([AM_MACRO1], [echo macro1d >> foo]) AC_DEFUN([AM_MACRO2], [echo macro2d >> foo]) EOF rm -f foo -$ACLOCAL --diff=diff >output 2>stderr -cat stderr -cat output -grep '#serial 456' output +run_command $ACLOCAL --diff=diff +grep '#serial 456' stdout test ! -f 4/m1.m4 grep 'installing.*4/m1.m4' stderr diff --git a/tests/aclocal.test b/tests/aclocal.test index 6b87a9d..945b363 100755 --- a/tests/aclocal.test +++ b/tests/aclocal.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1998, 2001, 2002, 2004, 2008 Free Software Foundation, Inc. +# Copyright (C) 1998, 2001, 2002, 2004, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -23,15 +23,15 @@ set -e $ACLOCAL --output=fred test -f fred -$ACLOCAL --output 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e 1 $ACLOCAL --output grep 'option.*--output.*an argument' stderr grep help stderr -$ACLOCAL --unknown-option 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e 1 $ACLOCAL --unknown-option grep 'unrecognized.*--unknown-option' stderr grep help stderr -$ACLOCAL --ver 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e 1 $ACLOCAL --ver grep 'unrecognized.*--ver' stderr grep help stderr diff --git a/tests/aclocal3.test b/tests/aclocal3.test index a550e50..a6f26a5 100755 --- a/tests/aclocal3.test +++ b/tests/aclocal3.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1998, 2002, 2004 Free Software Foundation, Inc. +# Copyright (C) 1998, 2002, 2004, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -27,6 +27,5 @@ AC_DEFUN([GNOME_X_CHECKS], [ ]) END -$ACLOCAL -I macros 2>stderr -cat stderr +run_command $ACLOCAL -I macros grep 'macros/gnome.m4:2:.*AM_PATH_GTK.*not found' stderr diff --git a/tests/aclocal8.test b/tests/aclocal8.test index ac10aff..4cf7d73 100755 --- a/tests/aclocal8.test +++ b/tests/aclocal8.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2003, 2004 Free Software Foundation, Inc. +# Copyright (C) 2003, 2004, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -34,9 +34,8 @@ EOF echo 'AC_DEFUN([MACRO1],)' >m4/macro1.m4 echo 'AC_DEFUN([MACRO2], [AC_REQUIRE([AM_UNUSED_MACRO])])' >m4/macro2.m4 -$ACLOCAL -I m4 >output 2>&1 -cat output -test 0 = `wc -l <output` +run_command -m $ACLOCAL -I m4 +test -s stdall && Exit 1 grep macro1.m4 aclocal.m4 grep macro2.m4 aclocal.m4 && Exit 1 : diff --git a/tests/acsilent.test b/tests/acsilent.test index 6ac7863..e250092 100755 --- a/tests/acsilent.test +++ b/tests/acsilent.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1996, 2001, 2002, 2008 Free Software Foundation, Inc. +# Copyright (C) 1996, 2001, 2002, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -32,5 +32,6 @@ module=[$1] AC_SUBST(module)]) END -$ACLOCAL > output 2>&1 || { cat output; Exit 1; } -test -z "`cat output`" +run_command -m $ACLOCAL +test -s stdall && Exit 1 +: diff --git a/tests/alpha.test b/tests/alpha.test index 1ee8ed6..1410b9d 100755 --- a/tests/alpha.test +++ b/tests/alpha.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1998, 2001, 2002, 2003, 2008 Free Software Foundation, Inc. +# Copyright (C) 1998, 2001, 2002, 2003, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -62,8 +62,7 @@ $AUTOMAKE ./configure # make distdir should fail because NEWS does not mention 1.0a -$MAKE check 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_command -e FAIL $MAKE check grep 'NEWS not updated' stderr test ! -f works diff --git a/tests/ammissing.test b/tests/ammissing.test index 61d360a..cbed710 100755 --- a/tests/ammissing.test +++ b/tests/ammissing.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1997, 2002, 2004 Free Software Foundation, Inc. +# Copyright (C) 1997, 2002, 2004, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -20,6 +20,5 @@ echo AM_ZARDOZ >> configure.in -$ACLOCAL 2>stderr -cat stderr +run_command $ACLOCAL grep 'configure.in:.*AM_ZARDOZ.*not found' stderr diff --git a/tests/ansi3.test b/tests/ansi3.test index 2bbcaf7..95e4fc1 100755 --- a/tests/ansi3.test +++ b/tests/ansi3.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1998, 1999, 2001, 2002, 2003 Free Software Foundation, Inc. +# Copyright (C) 1998, 1999, 2001, 2002, 2003, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -53,8 +53,7 @@ export CFLAGS # Linux boxes (Red Hat 5.1 in particular). $ACLOCAL $AUTOCONF -$AUTOMAKE -a 2>stderr -cat stderr +run_command $AUTOMAKE -a grep 'Makefile.am:1:.*ansi2knr\.c' stderr CC='gcc' ./configure ANSI2KNR=./ansi2knr U=_ $MAKE -e diff --git a/tests/ansi3b.test b/tests/ansi3b.test index 8f254ad..447b690 100755 --- a/tests/ansi3b.test +++ b/tests/ansi3b.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1998, 1999, 2001, 2002, 2003 Free Software Foundation, Inc. +# Copyright (C) 1998, 1999, 2001, 2002, 2003, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -52,8 +52,7 @@ export CFLAGS # Linux boxes (Red Hat 5.1 in particular). $ACLOCAL $AUTOCONF -$AUTOMAKE -a 2>stderr -cat stderr +run_command $AUTOMAKE -a grep 'configure.in:2:.*ansi2knr\.1' stderr CC='gcc' ./configure ANSI2KNR=./ansi2knr U=_ $MAKE -e diff --git a/tests/ccnoco3.test b/tests/ccnoco3.test index 2e9cd37..93731a1 100755 --- a/tests/ccnoco3.test +++ b/tests/ccnoco3.test @@ -74,8 +74,7 @@ CC=`pwd`/../Mycomp export CC ../configure -$MAKE 2>stderr || { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_command $MAKE grep 'mv.*the same file' stderr && Exit 1 Exit 0 diff --git a/tests/check8.test b/tests/check8.test index 7936582..a087cb7 100755 --- a/tests/check8.test +++ b/tests/check8.test @@ -67,10 +67,7 @@ $AUTOMAKE -a unset TESTS || : ./configure -AM_COLOR_TESTS=always $MAKE -e check >stdout 2>stderr && - { cat stdout; cat stderr >&2; Exit 1; } -cat stdout -cat stderr >&2 +run_command -e FAIL -- env AM_COLOR_TESTS=always $MAKE -e check grep 'XPASS.* foo$' stdout grep '^[^X]*PASS.* sub/foo$' stdout grep '^[^X]*PASS.* bar' stdout @@ -87,8 +84,7 @@ $MAKE distclean mkdir build cd build ../configure -$MAKE check >stdout && { cat stdout; Exit 1; } -cat stdout +run_command -e FAIL $MAKE check # Note: we are not grepping for the space here, due to the Solaris make VPATH # rewriting (if we fix that, we can still write a separate test for it). grep 'XPASS.*foo$' stdout diff --git a/tests/cond39.test b/tests/cond39.test index 79eaa70..1e44866 100755 --- a/tests/cond39.test +++ b/tests/cond39.test @@ -1,5 +1,5 @@ #!/bin/sh -# Copyright (C) 2008 Free Software Foundation, Inc. +# Copyright (C) 2008, 2009 Free Software Foundation, Inc. # # This file is part of GNU Automake. # @@ -82,8 +82,7 @@ $AUTOCONF $AUTOMAKE --add-missing ./configure COND=true -$MAKE 2>stderr -cat stderr +run_command $MAKE grep 'overriding commands' stderr && Exit 1 $MAKE sure-exist ./prog1 && Exit 1 @@ -97,8 +96,7 @@ $MAKE sure-exist $MAKE distclean ./configure COND=false -$MAKE 2>stderr -cat stderr +run_command $MAKE grep 'overriding commands' stderr && Exit 1 ./prog1 ./sub/prog2 diff --git a/tests/defs.in b/tests/defs.in index ecdf069..67f0856 100644 --- a/tests/defs.in +++ b/tests/defs.in @@ -395,26 +395,91 @@ is_newest () test -z "$is_newest_files" } +# run_command [-e STATUS] [-i FILE] [-m] [--] COMMAND [ARGUMENTS..] +# ----------------------------------------------------------------- +# Run the given COMMAND with ARGUMENTS and fail if COMMAND does not exit +# with STATUS. If status is "VOID" or "IGNORE", any exit value of the +# command is acceptable. If STATUS is "FAIL", then any exit value of the +# command *but 0* is acceptable. Default STATUS is `0'. +# Also, save standard output and standard error from COMMAND, by default +# respectively in files `stdout' and `stderr' (in the current directory), +# or togheter in the file `stdall' (in the current directory) if the `-m' +# option is given. +run_command () +{ + set +x # xtrace verbosity temporarly disabled in `run_command' + run_exitcode_expected=0 + run_mix_stdout_and_stderr=no + while test $# -gt 0; do + case $1 in + -e) run_exitcode_expected=$2; shift;; + -m) run_mix_stdout_and_stderr=yes;; + --) shift; break;; + -?) echo "run_commmand(): invalid switch \`$1'" >&2; Exit 99;; + *) break;; + esac + shift + done + case $# in + 0) echo "run_command(): missing COMMAND argument" >&2; Exit 99;; + *) run_cmd=$1; shift;; + esac + if test x"$run_mix_stdout_and_stderr" = x"yes"; then + run_evald_cmd='"$run_cmd" ${1+"$@"} >stdall 2>&1' + else + run_evald_cmd='"$run_cmd" ${1+"$@"} >stdout 2>stderr' + fi + if eval "$run_evald_cmd"; then + run_exitcode_got=0 + else + run_exitcode_got=$? + fi + if test x"$run_mix_stdout_and_stderr" = x"yes"; then + set -x + cat stdall + { set +x; } 2>/dev/null + else + set -x + cat stderr >&2 + cat stdout + { set +x; } 2>/dev/null + fi + case $run_exitcode_expected in + VOID|void|IGNORE|ignore|IGNORED|ignored|$run_exitcode_got) + run_rc=0 + ;; + FAIL|fail|FAILURE|failure) + if test $run_exitcode_got -gt 0; then + run_rc=0 + else + run_rc=1 + fi + ;; + *) + run_rc=1 + ;; + esac + echo "run_command: exit status $run_exitcode_got (expecting" \ + "$run_exitcode_expected)" + set -x # reactivating temporarly turned-off xtrace verbosity + return $run_rc +} # AUTOMAKE_run status [options...] # -------------------------------- -# Run Automake with OPTIONS, and fail if automake +# Run Automake with OPTIONS, and make the test FAIL if automake # does not exit with STATUS. AUTOMAKE_run () { - expected_exitcode=$1 + am_run_expected_exitcode=$1 shift - exitcode=0 - $AUTOMAKE ${1+"$@"} >stdout 2>stderr || exitcode=$? - cat stderr >&2 - cat stdout - test $exitcode = $expected_exitcode || Exit 1 + run_command -e $am_run_expected_exitcode -- $AUTOMAKE ${1+"$@"} || Exit 1 } # AUTOMAKE_fails [options...] # --------------------------- -# Run Automake with OPTIONS, and fail if automake -# does not exit with STATUS. +# Run Automake with OPTIONS, and make the test FAIL if automake +# does not exit with status 1. AUTOMAKE_fails () { AUTOMAKE_run 1 ${1+"$@"} diff --git a/tests/depcomp2.test b/tests/depcomp2.test index 4653c90..3abe7ba 100755 --- a/tests/depcomp2.test +++ b/tests/depcomp2.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2001, 2002 Free Software Foundation, Inc. +# Copyright (C) 2001, 2002, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -20,6 +20,8 @@ required=gcc . ./defs || Exit 1 +set -e + cat > configure.in << 'END' AC_INIT(subdir/foo.c) AM_INIT_AUTOMAKE(nonesuch, nonesuch) @@ -48,5 +50,6 @@ export CFLAGS $ACLOCAL || Exit 1 $AUTOMAKE --add-missing || Exit 1 $AUTOCONF || Exit 1 -./configure CC='gcc' 2>error.log -test -z "`cat error.log`" +run_command ./configure CC='gcc' +test -s stderr && Exit 1 +: diff --git a/tests/depcomp4.test b/tests/depcomp4.test index 7d22894..70cbd57 100755 --- a/tests/depcomp4.test +++ b/tests/depcomp4.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2003 Free Software Foundation, Inc. +# Copyright (C) 2003, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -51,7 +51,6 @@ $AUTOMAKE -a grep am_cv_CC_dependencies_compiler_type configure ./configure am_cv_CC_dependencies_compiler_type=makedepend -$MAKE >output 2>&1 -cat output -grep 'unknown directive' output && Exit 1 +run_command -m $MAKE +grep 'unknown directive' stdall && Exit 1 $MAKE grepdeps diff --git a/tests/depend5.test b/tests/depend5.test index 0d9be3b..52e4169 100755 --- a/tests/depend5.test +++ b/tests/depend5.test @@ -65,10 +65,7 @@ for arg in Makefile \ "--file=$name" do rm -rf .deps _deps - ./config.status "$arg" depfiles >stdout 2>stderr || - { cat stdout; cat stderr >&2; Exit 1; } - cat stdout - cat stderr >&2 + run_command ./config.status "$arg" depfiles grep '[Nn]o such file' stderr && Exit 1 if test -n "$depdir"; then diff --git a/tests/depend6.test b/tests/depend6.test index de2adf9..5ce2208 100755 --- a/tests/depend6.test +++ b/tests/depend6.test @@ -39,9 +39,6 @@ END $ACLOCAL $AUTOMAKE $AUTOCONF -./configure --disable-dependency-tracking 2>stderr || { - stat=$?; cat stderr >&2; Exit $stat; -} -cat stderr >&2 -grep shift stderr && Exit 1 +run_command ./configure --disable-dependency-tracking +grep 'shift' stderr && Exit 1 : diff --git a/tests/discover.test b/tests/discover.test index 36293f7..c62f1b4 100755 --- a/tests/discover.test +++ b/tests/discover.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1996, 1997, 2000, 2001, 2002 Free Software Foundation, Inc. +# Copyright (C) 1996, 1997, 2000, 2001, 2002, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -34,5 +34,6 @@ END : > fsusage.c $ACLOCAL || Exit 1 -$AUTOMAKE 2>&1 | $FGREP fsusage.c || Exit 1 +run_command -e IGNORE -m $AUTOMAKE +$FGREP fsusage.c stdall || Exit 1 Exit 0 diff --git a/tests/distcom3.test b/tests/distcom3.test index 04bf365..fdcc31a 100755 --- a/tests/distcom3.test +++ b/tests/distcom3.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2001, 2002, 2003, 2004, 2006 Free Software Foundation, Inc. +# Copyright (C) 2001, 2002, 2003, 2004, 2006, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -34,9 +34,8 @@ END $ACLOCAL # Should not warn about missing README, since it is a target. -$AUTOMAKE --add-missing --gnu >stdout 2>&1 -cat stdout -grep README stdout && Exit 1 +run_command -m $AUTOMAKE --add-missing --gnu +grep README stdall && Exit 1 sed -n -e '/^DIST_COMMON =.*\\$/ { :loop diff --git a/tests/fn99.test b/tests/fn99.test index 982a989..611c767 100755 --- a/tests/fn99.test +++ b/tests/fn99.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2004, 2008 Free Software Foundation, Inc. +# Copyright (C) 2004, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -47,8 +47,7 @@ do touch x done) || Exit 77 -$MAKE dist 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_command -e FAIL $MAKE dist grep 'filenames are too long' stderr test 2 = `grep 12345678 stderr | wc -l` : diff --git a/tests/fn99subdir.test b/tests/fn99subdir.test index 465f2af..7055e85 100755 --- a/tests/fn99subdir.test +++ b/tests/fn99subdir.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2006, 2008 Free Software Foundation, Inc. +# Copyright (C) 2006, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -73,8 +73,7 @@ do touch x done) -$MAKE dist 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_command -e FAIL $MAKE dist grep 'filenames are too long' stderr test 1 = `grep 12345678 stderr | wc -l` : diff --git a/tests/gnits2.test b/tests/gnits2.test index 4c5d54f..a85b096 100755 --- a/tests/gnits2.test +++ b/tests/gnits2.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2002, 2003, 2004, 2007 Free Software Foundation, Inc. +# Copyright (C) 2002, 2003, 2004, 2007, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -105,6 +105,6 @@ cd build ../configure "--prefix=`pwd`/../inst-dir" --program-prefix=p $MAKE all $MAKE test-install -$MAKE -k installcheck 2>stderr || : # Never trust the exit status of make -k. -cat stderr +# Never trust the exit status of make -k. +run_command -e IGNORE $MAKE -k installcheck $MAKE grep-stderr diff --git a/tests/gnits3.test b/tests/gnits3.test index 7a08fb8..4d90684 100755 --- a/tests/gnits3.test +++ b/tests/gnits3.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2002, 2007 Free Software Foundation, Inc. +# Copyright (C) 2002, 2007, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -83,8 +83,8 @@ cd build ../configure "--prefix=`pwd`/../inst-dir" --program-prefix=p $MAKE $MAKE install -$MAKE -k installcheck 2>stderr || : # Never trust the exit status of make -k. -cat stderr +# Never trust the exit status of make -k. +run_command -e IGNORE $MAKE -k installcheck $MAKE grep-stderr # Make sure there is no more error when all targets are exempted. diff --git a/tests/help.test b/tests/help.test index d4d7110..cae8967 100755 --- a/tests/help.test +++ b/tests/help.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2003, 2008 Free Software Foundation, Inc. +# Copyright (C) 2003, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -32,7 +32,7 @@ $AUTOMAKE --version $AUTOMAKE --help # aclocal and automake cannot work without configure.ac or configure.in -$ACLOCAL 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e 1 $ACLOCAL grep configure.ac stderr grep configure.in stderr AUTOMAKE_fails diff --git a/tests/init.test b/tests/init.test index 38ec681..e4f308c 100755 --- a/tests/init.test +++ b/tests/init.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2006, 2008 Free Software Foundation, Inc. +# Copyright (C) 2006, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -27,7 +27,14 @@ AM_INIT_AUTOMAKE END # The error message should mension AC_INIT, not AC_PACKAGE_VERSION. -($ACLOCAL && $AUTOCONF) 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +success=no +run_command -e FAILURE $ACLOCAL || success=yes +if test x"$success" = x"yes"; then + mv stderr stder1 + run_command -e FAILURE $AUTOCONF || success=yes + cat stderr >>stder1 + mv -f stder1 stderr +fi +test x"$success" = x"no" || Exit 1 grep AC_PACKAGE_VERSION stderr && Exit 1 grep AC_INIT stderr diff --git a/tests/instsh.test b/tests/instsh.test index 61f5f9e..fe3d485 100755 --- a/tests/instsh.test +++ b/tests/instsh.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1996, 1999, 2001, 2002, 2003 Free Software Foundation, Inc. +# Copyright (C) 1996, 1999, 2001, 2002, 2003, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -32,10 +32,9 @@ mv Makefile.am configure.in frob/ cd frob $ACLOCAL -$AUTOMAKE --add-missing > output 2>&1 +run_command -m $AUTOMAKE --add-missing # Only one `/' should appear in the output. -cat output -grep '/.*/' output && Exit 1 +grep '/.*/' stdall && Exit 1 test -f install-sh diff --git a/tests/ldadd.test b/tests/ldadd.test index 005f9d8..0a9907d 100755 --- a/tests/ldadd.test +++ b/tests/ldadd.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1997, 1999, 2000, 2001, 2002 Free Software Foundation, Inc. +# Copyright (C) 1997, 1999, 2000, 2001, 2002, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -39,5 +39,6 @@ END : > q.c $ACLOCAL || Exit 1 -$AUTOMAKE 2>&1 | grep LDADD || Exit 1 +run_command -e 1 -m $AUTOMAKE +grep LDADD stdall || Exit 1 Exit 0 diff --git a/tests/libtool7.test b/tests/libtool7.test index a2f912f..f8706d9 100755 --- a/tests/libtool7.test +++ b/tests/libtool7.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2003, 2004, 2007 Free Software Foundation, Inc. +# Copyright (C) 2003, 2004, 2007, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -82,19 +82,16 @@ $AUTOCONF $AUTOMAKE --add-missing --copy ./configure "--prefix=`pwd`/_inst" -env LIBTOOLFLAGS=--silent $MAKE print >output 2>&1 -cat output -grep '1BEG: libmod1.la mod2.la :END1' output -grep '2BEG: mod2.la :END2' output -grep '3BEG: .*silent.*silent.* :END3' output +run_command -m env LIBTOOLFLAGS=--silent $MAKE print +grep '1BEG: libmod1.la mod2.la :END1' stdall +grep '2BEG: mod2.la :END2' stdall +grep '3BEG: .*silent.*silent.* :END3' stdall test 2 -le `grep mod2_la_LIBTOOLFLAGS Makefile | wc -l` $MAKE -env LIBTOOLFLAGS=--silent $MAKE install >output 2>&1 -cat output -grep 'silent.*silent.*prg' output -grep 'silent.*silent.*libmod1' output +run_command -m env LIBTOOLFLAGS=--silent $MAKE install +grep 'silent.*silent.*prg' stdall +grep 'silent.*silent.*libmod1' stdall -env LIBTOOLFLAGS=--silent $MAKE uninstall >output 2>&1 -cat output -grep 'silent.*silent.*libmod1' output +run_command -m env LIBTOOLFLAGS=--silent $MAKE uninstall +grep 'silent.*silent.*libmod1' stdall diff --git a/tests/libtool9.test b/tests/libtool9.test index c0d9688..94e617b 100755 --- a/tests/libtool9.test +++ b/tests/libtool9.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2005 Free Software Foundation, Inc. +# Copyright (C) 2005, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -88,10 +88,11 @@ $AUTOCONF $AUTOMAKE --add-missing --copy ./configure -env LDFLAGS=ldflags AM_LDFLAGS=am_ldflags libmod1_la_LDFLAGS=lm1_la_ldflags \ +run_command -m env \ + LDFLAGS=ldflags AM_LDFLAGS=am_ldflags libmod1_la_LDFLAGS=lm1_la_ldflags \ CFLAGS=cflags AM_CFLAGS=am_cflags prg2_CFLAGS=prg2_cflags \ - $MAKE -e print >output 2>&1 -cat output + $MAKE -e print +mv stdall output grep '1BEG: libmod1.la mod2.la :END1' output grep '2BEG: mod2.la :END2' output grep '3BEG:.* am_cflags cflags .*lm1_la_ldflags ldflags.* :END3' output diff --git a/tests/lisp3.test b/tests/lisp3.test index 809d491..086173d 100755 --- a/tests/lisp3.test +++ b/tests/lisp3.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2003, 2008 Free Software Foundation, Inc. +# Copyright (C) 2003, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -61,9 +61,8 @@ test -f am-one.elc unique=0a3346e2af8a689b85002b53df09142a $sleep echo "(message \"$unique\")(provide 'am-three)" > am-three.el -$MAKE >output 2>&1 -cat output -grep $unique output +run_command -m $MAKE +grep $unique stdall # It should also work for VPATH-builds. $MAKE distcheck diff --git a/tests/ltorder.test b/tests/ltorder.test index 0f69211..1e6a03a 100755 --- a/tests/ltorder.test +++ b/tests/ltorder.test @@ -1,5 +1,5 @@ #!/bin/sh -# Copyright (C) 2008 Free Software Foundation, Inc. +# Copyright (C) 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -53,8 +53,7 @@ $AUTOMAKE --add-missing ./configure "--prefix=`pwd`/inst" $MAKE -$MAKE install 2>stderr || { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_command $MAKE install grep 'has not been installed' stderr && Exit 1 $MAKE uninstall diff --git a/tests/man4.test b/tests/man4.test index aa86a9e..7424c72 100755 --- a/tests/man4.test +++ b/tests/man4.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2008 Free Software Foundation, Inc. +# Copyright (C) 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -88,11 +88,9 @@ mkdir build cd build ../configure $MAKE -$MAKE dist 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_command -e FAIL $MAKE dist grep 'install help2man' stderr -$MAKE distcheck 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_command -e FAIL $MAKE distcheck grep 'install help2man' stderr rm -f ../help2man diff --git a/tests/mdate.test b/tests/mdate.test index 704b393..e63bff9 100755 --- a/tests/mdate.test +++ b/tests/mdate.test @@ -1,5 +1,5 @@ #!/bin/sh -# Copyright (C) 1996, 1997, 2001, 2002 Free Software Foundation, Inc. +# Copyright (C) 1996, 1997, 2001, 2002, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -31,4 +31,5 @@ END : > texinfo.tex $ACLOCAL || Exit 1 -$AUTOMAKE 2>&1 > /dev/null | grep 'required file.*mdate-sh' +run_command -e IGNORED $AUTOMAKE +grep 'required file.*mdate-sh' stderr diff --git a/tests/missing3.test b/tests/missing3.test index 07ef779..9b48368 100755 --- a/tests/missing3.test +++ b/tests/missing3.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2004, 2008 Free Software Foundation, Inc. +# Copyright (C) 2004, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -22,24 +22,24 @@ set -e # b7cb8259 assumed not to exist. -./missing b7cb8259 --version 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e FAIL ./missing b7cb8259 --version grep . stderr && Exit 1 -./missing b7cb8259 --grep 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e FAIL ./missing b7cb8259 --grep grep WARNING stderr -./missing --run b7cb8259 --version && Exit 1 -./missing --run b7cb8259 --grep 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e FAIL ./missing --run b7cb8259 --version +run_command -e FAIL ./missing --run b7cb8259 --grep grep WARNING stderr # missing itself it known to exist :) -./missing ./missing --version 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e FAIL ./missing ./missing --version grep . stderr && Exit 1 -./missing ./missing --grep 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e FAIL ./missing ./missing --grep grep WARNING stderr -./missing --run ./missing --version 2>stderr +run_command ./missing --run ./missing --version grep . stderr && Exit 1 -./missing --run ./missing --grep 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e FAIL ./missing --run ./missing --grep grep WARNING stderr && Exit 1 grep Unknown stderr diff --git a/tests/missing4.test b/tests/missing4.test index 1015828..c6c3b7e 100755 --- a/tests/missing4.test +++ b/tests/missing4.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2006, 2007, 2008 Free Software Foundation, Inc. +# Copyright (C) 2006, 2007, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -36,13 +36,11 @@ cmp aclocal.m4 aclocal.tmp && Exit 1 mv aclocal.tmp aclocal.m4 -$MAKE 2>stderr -cat stderr +run_command $MAKE grep 'You have another version of autoconf' stderr grep -c 'aclocal.m4:.*this file was generated for' stderr -$MAKE 2>stderr -cat stderr +run_command $MAKE grep 'You have another version of autoconf' stderr && Exit 1 grep 'aclocal.m4:.*this file was generated for' stderr && Exit 1 diff --git a/tests/missing6.test b/tests/missing6.test index b63a0d1..bf6df2d 100755 --- a/tests/missing6.test +++ b/tests/missing6.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2008 Free Software Foundation, Inc. +# Copyright (C) 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -31,7 +31,7 @@ rm -f configure.in touch Makefile.am $ACLOCAL -$AUTOCONF 2>stderr +run_command $AUTOCONF grep 'You have another version of autoconf' stderr grep 'aclocal.m4:.*this file was generated for' stderr $AUTOMAKE @@ -43,8 +43,7 @@ cmp configure.ac configure.tmp && Exit 1 mv configure.tmp configure.ac -$MAKE 2>stderr -cat stderr +run_command $MAKE grep 'You have another version of autoconf' stderr grep 'aclocal.m4:.*this file was generated for autoconf 9999a' stderr diff --git a/tests/obsolete.test b/tests/obsolete.test index e6601c9..daae69c 100755 --- a/tests/obsolete.test +++ b/tests/obsolete.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1996, 2001, 2002, 2007, 2008 Free Software Foundation, Inc. +# Copyright (C) 1996, 2001, 2002, 2007, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -27,7 +27,7 @@ END $PERL -ne '/AU_DEFUN\(\[(\w+)\]/ && print "$1\n"' \ "$testsrcdir/../m4/obsolete.m4" >> obs cat obs >> configure.in -$PERL -ne 'chomp; print "grep $_ stderr || Exit 1\n"; ' obs > obs.1 +$PERL -ne 'chomp; print "grep $_ stdall || Exit 1\n"; ' obs > obs.1 $PERL -ne 'chomp; print "grep $_ configure.in && Exit 1\n"; ' obs > obs.2 # Sanity check. Make sure we have added something to configure.in. @@ -36,7 +36,7 @@ test `cat configure.in | wc -l` -gt 1 || Exit 1 $ACLOCAL || Exit 1 # Expect Autoconf to complain about each of the macros in obs. -$AUTOCONF -Wobsolete >stderr 2>&1 +run_command -m $AUTOCONF -Wobsolete . ./obs.1 # Make sure Autoupdate remove each of these macros. $AUTOUPDATE || Exit 1 diff --git a/tests/output-order.test b/tests/output-order.test index 856baa2..852c194 100755 --- a/tests/output-order.test +++ b/tests/output-order.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2008 Free Software Foundation, Inc. +# Copyright (C) 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -43,13 +43,12 @@ END $ACLOCAL $AUTOCONF rm -f missing install-sh -$AUTOMAKE --add-missing --copy 2>stderr -cat stderr >&2 +run_command $AUTOMAKE --add-missing --copy +mv stderr stder0 for i in 1 2 3 4 5 6; do rm -f missing install-sh INSTALL COPYING + run_command -e IGNORE $AUTOMAKE --add-missing --copy # The grep prevents a Heisenbug with the HP-UX shell and VERBOSE=yes. - $AUTOMAKE --add-missing --copy 2>&1 >/dev/null | - grep -v /dev/null | - diff - stderr + grep -v /dev/null stder0 | diff - stderr done diff --git a/tests/pr220.test b/tests/pr220.test index 25e1f0e..6644d28 100755 --- a/tests/pr220.test +++ b/tests/pr220.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2001, 2002, 2004, 2008 Free Software Foundation, Inc. +# Copyright (C) 2001, 2002, 2004, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -55,6 +55,5 @@ $AUTOMAKE -a cd build # configure should fail since we've done something invalid. -../configure 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_command -e FAIL ../configure grep NEVER_TRUE stderr diff --git a/tests/pr300-ltlib.test b/tests/pr300-ltlib.test index 01814a9..7ed0a2f 100755 --- a/tests/pr300-ltlib.test +++ b/tests/pr300-ltlib.test @@ -48,8 +48,7 @@ $ACLOCAL $AUTOCONF $AUTOMAKE --copy --add-missing ./configure --prefix "`pwd`/inst" -$MAKE V=1 >stdout -cat stdout +run_command $MAKE V=1 grep 'liba.la .*-rpath .*lib' stdout grep 'liba.la .*-rpath .*lib/subdir' stdout && Exit 1 @@ -58,8 +57,7 @@ grep 'libb.la .*-rpath .*lib/subdir' stdout test -f subdir/liba.la test -f subdir/libb.la -$MAKE install 2>stderr -cat stderr >&2 +run_command $MAKE install grep 'remember.*--finish' stderr && Exit 1 test -f inst/lib/liba.la diff --git a/tests/python11.test b/tests/python11.test index 0f15f37..920de88 100755 --- a/tests/python11.test +++ b/tests/python11.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2004, 2006, 2008 Free Software Foundation, Inc. +# Copyright (C) 2004, 2006, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -35,9 +35,7 @@ EOF $ACLOCAL $AUTOCONF -./configure >stdout 2>stderr && { cat stdout; cat stderr >&2; Exit 1; } -cat stdout -cat stderr >&2 +run_command -e FAILURE ./configure grep 'checking for IShouldNotExist1' stdout grep 'checking for IShouldNotExist2' stdout grep 'no suitable Python interpreter found' stderr diff --git a/tests/python4.test b/tests/python4.test index 11bb3ed..3ff44af 100755 --- a/tests/python4.test +++ b/tests/python4.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2003, 2008 Free Software Foundation, Inc. +# Copyright (C) 2003, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -33,6 +33,5 @@ $AUTOCONF $AUTOMAKE --add-missing # Simulate no Python -./configure PYTHON=: 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_command -e FAILURE ./configure PYTHON=: grep 'no suitable Python interpreter found' stderr diff --git a/tests/python5.test b/tests/python5.test index 86f5863..5695876 100755 --- a/tests/python5.test +++ b/tests/python5.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2003, 2008 Free Software Foundation, Inc. +# Copyright (C) 2003, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -34,6 +34,5 @@ $ACLOCAL $AUTOCONF $AUTOMAKE --add-missing -./configure 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_command -e FAILURE ./configure grep 'no suitable Python interpreter found' stderr diff --git a/tests/python6.test b/tests/python6.test index eb4fb4d..ed60e03 100755 --- a/tests/python6.test +++ b/tests/python6.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2003 Free Software Foundation, Inc. +# Copyright (C) 2003, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -34,6 +34,5 @@ $AUTOCONF $AUTOMAKE --add-missing # Simulate no Python -./configure PYTHON=: 2>stderr -cat stderr +run_command ./configure PYTHON=: grep 'GREP ME:' stderr diff --git a/tests/python7.test b/tests/python7.test index 62c1a1b..eb6a4dc 100755 --- a/tests/python7.test +++ b/tests/python7.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2003 Free Software Foundation, Inc. +# Copyright (C) 2003, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -34,6 +34,5 @@ $ACLOCAL $AUTOCONF $AUTOMAKE --add-missing -./configure 2>stderr -cat stderr +run_command ./configure grep 'GREP ME:' stderr diff --git a/tests/python8.test b/tests/python8.test index a50af22..5f100ab 100755 --- a/tests/python8.test +++ b/tests/python8.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2003 Free Software Foundation, Inc. +# Copyright (C) 2003, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -32,6 +32,5 @@ $ACLOCAL $AUTOCONF $AUTOMAKE --add-missing -./configure 2>stderr -cat stderr +run_command ./configure grep 'GREP ME' stderr diff --git a/tests/python9.test b/tests/python9.test index 9b23deb..c0fa7a5 100755 --- a/tests/python9.test +++ b/tests/python9.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2003 Free Software Foundation, Inc. +# Copyright (C) 2003, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -33,6 +33,5 @@ $ACLOCAL $AUTOCONF $AUTOMAKE --add-missing -./configure 2>stderr -cat stderr +run_command ./configure grep 'GREP ME' stderr diff --git a/tests/sanity.test b/tests/sanity.test index 1c96c78..1296381 100755 --- a/tests/sanity.test +++ b/tests/sanity.test @@ -36,14 +36,12 @@ cp ../install-sh ../missing . $ACLOCAL $AUTOCONF $AUTOMAKE -./configure 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr +run_command -e FAILURE ./configure grep 'unsafe absolute working directory' stderr cd .. mkdir build cd build -../unsafe$/configure 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr +run_command -e FAILURE ../unsafe$/configure grep 'unsafe srcdir' stderr : diff --git a/tests/subobj.test b/tests/subobj.test index df297f4..9d7882d 100755 --- a/tests/subobj.test +++ b/tests/subobj.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1999, 2001, 2002, 2004 Free Software Foundation, Inc. +# Copyright (C) 1999, 2001, 2002, 2004, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -32,8 +32,7 @@ wish_SOURCES = generic/a.c generic/b.c END $ACLOCAL -$AUTOMAKE --add-missing 2>stderr -cat stderr +run_command $AUTOMAKE --add-missing # Make sure compile is installed, and that Automake says so. grep 'install.*compile' stderr test -f compile diff --git a/tests/subobj9.test b/tests/subobj9.test index 99fe6e8..1f68721 100755 --- a/tests/subobj9.test +++ b/tests/subobj9.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2002, 2004, 2005 Free Software Foundation, Inc. +# Copyright (C) 2002, 2004, 2005, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -74,8 +74,8 @@ if test -n "`./libtool --help | grep tag=TAG`"; then fi $MAKE -$MAKE distcheck 2>&1 | tee out +run_command -m $MAKE distcheck # GNU Make used to complain that the Makefile contained two rules # for `src/.dirstamp' and `.//src/.dirstamp'. -grep 'overriding commands' out && Exit 1 +grep 'overriding commands' stdall && Exit 1 : diff --git a/tests/unused.test b/tests/unused.test index a5b4da6..68d5613 100755 --- a/tests/unused.test +++ b/tests/unused.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1998, 2002, 2003, 2008 Free Software Foundation, Inc. +# Copyright (C) 1998, 2002, 2003, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -29,5 +29,5 @@ AC_DEFUN([MACRO_1_2], echo 12) AC_DEFUN([MACRO_1_2_3], echo 123) END -$ACLOCAL 2> stderr || { cat stderr >&2; Exit 1; } -test -z "`cat stderr`" +run_command $ACLOCAL +test ! -s stderr diff --git a/tests/vars3.test b/tests/vars3.test index 3d10a90..9f11262 100755 --- a/tests/vars3.test +++ b/tests/vars3.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2002 Free Software Foundation, Inc. +# Copyright (C) 2002, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -38,8 +38,7 @@ EOF $ACLOCAL # Make sure this warning is print in the `portability' category. -$AUTOMAKE --warnings=no-error,none,portability 2>stderr -cat stderr +run_command $AUTOMAKE --warnings=no-error,none,portability # Lines number are printed in error message. # Use them to make sure errors are diagnosed against the right lines. diff --git a/tests/version8.test b/tests/version8.test index 91b11a0..ffe4a3e 100755 --- a/tests/version8.test +++ b/tests/version8.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2005, 2008 Free Software Foundation, Inc. +# Copyright (C) 2005, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -20,6 +20,5 @@ set -e echo 'AM_AUTOMAKE_VERSION([1.9])' >>configure.in -$ACLOCAL 2>stderr && { cat stderr >&2; Exit 0; } -cat stderr >&2 +run_command -e 1 $ACLOCAL $FGREP 'AM_INIT_AUTOMAKE([1.9])' stderr -- 1.6.3.3 |
|
|
Re: testsuite failures when test scripts are run with zsh
by Ralf Wildenhues
::
Rate this Message:
Hi Stefano,
* Stefano Lattarini wrote on Tue, Oct 13, 2009 at 04:24:38PM CEST: > At Tuesday 13 October 2009, Ralf Wildenhues wrote: > > > > First off, I think that run_command really should Exit when the > > command does not produce the intended status. It should not be > > necessary to do run_command -e 1 $command || Exit 1 > > > > That is much safer, and less maintenance. > I see your point. It's OK with me to have `run_command' calling Exit > on failures, since (as you stressed) that's by far the most common > scenario. However, we should then really add a similar subroutine > (say `run_redirect' -- tell me if you have a better name) which only > takes care of portably redirecting stdout/stderr of a command (and, > obviously, rewrite `run_command' implementation to advantage of the > new function). > Is this OK with you? This is making things too complicated for my taste. Your run_command already has an -e option. If we're going to return the exit status anyway from the function, then we wouldn't need that -e *at all*, we could just return whatever status the command caused. So if it pleases you better, then we can change it to either of the following: - '-e ignore' ignores the status of the command; run_command returns it; - no '-e' just lets run_command return the status of the command, rather than checking the command against zero. I prefer the first, but only on the grounds that more typing is done only in the rare case. > > If we really need to run some command where we need to ignore > > the exit status, then we can still use > > > > run_command '$command || :' > This is wrong, as currently `run_command' do not `eval' its COMMAND. > And the exit status of $command would be lost, which might not be what > we really want. OK, sorry about that glitch. > > Which brings me to the second inconsistent issue with this API: the > > 'eval'uation level of the command is part of the API. > > This is important, because when the absolute source and build > > directories contain white space in the name (and Automake mostly > > works in this case now), we should be doing the right thing. > I don't understand. The `eval' used in the run_command implementation > is there just to provide a shortand: no argument passed by the caller > is ever eval'd (and this is the right thing to do, I think). OK. > > Then to your question above: yes it is ok to replace all instances > > of AUTOMAKE_run and AUTOMAKE_fails (there is no need to replace > > plain $AUTOMAKE without redirection). > > > > > If you think about it, the testsuite don't have `ACLOCAL_run' or > > > `ACLOCAL_fails', but simply uses `run_command $ACLOCAL' and > > > `run_COMMAND -e 1 $ACLOCAL'. > > > By the way, this change should require a small change in > > > `tests/README' too. > > > If you agree with it, I think it should be done with a distinct > > > patch. > > Sounds good. > Mmhh, I see another possible misunderstanding creeping in here. > Better to clear it out, just to be absolutely sure. What I was trying > to say is that we should get rid of AUTOMAKE_run and AUTOMAKE_fails, > not add ACLOCAL_run and ACLOCAL_fails (especially now that I'm going > to follow your advice and make run_command use `Exit 1' on failures). > Is this OK? OK. > > > > BTW, your run_command doesn't do what it advertizes to do: it > > > > doesn't necessarily cause a test failure when it should, esp. > > > > when a test doesn't use 'set -e'. > > > > > > I think that here there's a misunderstanding about the meaning of > > > `fail': you mean "the testcase fails", while I mean "the function > > > fails", i.e. it return a value != 0. Do you have a rewording to > > > suggest to make things clearer? > > > > "fail" is FAIL, and exit status != 0 is returning a nonzero exit > > status. > This will become mostly a moot issue once run_command will call > `Exit 1' on unexpected exit status. Should I anyway use "FAIL" > instead of "fail"? Either way is fine. > > > > > By the way, your observation has made me think: wouldn't it > > > > > be better to enable `set -e' in defs.in, so that all the test > > > > > cases could have a more uniform environment? > > > > > > > > This would require an audit of all tests that currently don't > > > > use set -e. This needs testing on Solaris, OpenBSD, NetBSD, > > > > Tru64, because of bugs and warts in their shell's > > > > implementation of errexit. > > > > > > Unfortunately, I don't have access to any of those system, so > > > I'll have to drop the ball on this. > > > > I can test the final iteration of this patch. > Well, even if we are going to make `set -e' the default, I think that > this change should be introduced with patch. Do you mean "should be introduced with a separate patch"? If yes, then I agree. > > BTW, now that we have TEST_LOG_COMPILER, and correctly unset it in > > defs.in, too, we can set it in tests/Makefile.am and worry less > > about shells like Solaris /bin/sh. Of course, that would require > > us to warn that running tests directly (i.e., without 'make' > > in-between), might require to use a decent shell. > Or we could add proper code in `tests/defs.in', to make it re-execute > the test scripts with CONFIG_SHELL. But this should be obviously done > in a distinct patch. Yes. > --- a/tests/defs.in > +++ b/tests/defs.in > @@ -395,26 +395,91 @@ is_newest () > test -z "$is_newest_files" > } > > +# run_command [-e STATUS] [-i FILE] [-m] [--] COMMAND [ARGUMENTS..] > +# ----------------------------------------------------------------- > +# Run the given COMMAND with ARGUMENTS and fail if COMMAND does not exit > +# with STATUS. If status is "VOID" or "IGNORE", any exit value of the > +# command is acceptable. If STATUS is "FAIL", then any exit value of the > +# command *but 0* is acceptable. Default STATUS is `0'. > +# Also, save standard output and standard error from COMMAND, by default > +# respectively in files `stdout' and `stderr' (in the current directory), > +# or togheter in the file `stdall' (in the current directory) if the `-m' typo togheter > +# option is given. > +run_command () > +{ > + set +x # xtrace verbosity temporarly disabled in `run_command' > + run_exitcode_expected=0 > + run_mix_stdout_and_stderr=no > + while test $# -gt 0; do > + case $1 in > + -e) run_exitcode_expected=$2; shift;; > + -m) run_mix_stdout_and_stderr=yes;; > + --) shift; break;; > + -?) echo "run_commmand(): invalid switch \`$1'" >&2; Exit 99;; s/()// > + *) break;; > + esac > + shift > + done > + case $# in > + 0) echo "run_command(): missing COMMAND argument" >&2; Exit 99;; Likewise. > + *) run_cmd=$1; shift;; > + esac > + if test x"$run_mix_stdout_and_stderr" = x"yes"; then > + run_evald_cmd='"$run_cmd" ${1+"$@"} >stdall 2>&1' > + else > + run_evald_cmd='"$run_cmd" ${1+"$@"} >stdout 2>stderr' > + fi > + if eval "$run_evald_cmd"; then > + run_exitcode_got=0 > + else > + run_exitcode_got=$? > + fi > + if test x"$run_mix_stdout_and_stderr" = x"yes"; then > + set -x > + cat stdall > + { set +x; } 2>/dev/null > + else > + set -x > + cat stderr >&2 > + cat stdout > + { set +x; } 2>/dev/null > + fi > + case $run_exitcode_expected in > + VOID|void|IGNORE|ignore|IGNORED|ignored|$run_exitcode_got) > + run_rc=0 > + ;; > + FAIL|fail|FAILURE|failure) > + if test $run_exitcode_got -gt 0; then > + run_rc=0 > + else > + run_rc=1 > + fi > + ;; > + *) > + run_rc=1 > + ;; > + esac > + echo "run_command: exit status $run_exitcode_got (expecting" \ > + "$run_exitcode_expected)" > + set -x # reactivating temporarly turned-off xtrace verbosity > + return $run_rc > +} Thanks, Ralf |
|
|
Re: testsuite failures when test scripts are run with zsh
by Stefano Lattarini
::
Rate this Message:
Hi Ralf.
A new version of the patch is attached. It's still not definitive, but I hope most things are settled by now. I tested it with `zsh3 -o no_function_argzero' (passes all tests) and with heirloom-sh (passes all tests but vala4.test, but this is not a regression, as you should remember: http://lists.gnu.org/archive/html/automake-patches/2009-10/msg00024.html If you say this version of the patch is OK to go, I'll test it a bit more thoroughly. > > [CUT] > > I see your point. It's OK with me to have `run_command' calling > > Exit on failures, since (as you stressed) that's by far the most > > common scenario. However, we should then really add a similar > > subroutine (say `run_redirect' -- tell me if you have a better > > name) which only takes care of portably redirecting stdout/stderr > > of a command (and, obviously, rewrite `run_command' > > implementation to advantage of the new function). > > Is this OK with you? > > This is making things too complicated for my taste. > Your run_command already has an -e option. If we're going to > return the exit status anyway from the function, then we wouldn't > need that -e *at all*, we could just return whatever status the > command caused. > > So if it pleases you better, then we can change it to either of the > following: > > - '-e ignore' ignores the status of the command; run_command > returns it; > - no '-e' just lets run_command return the status of the command, > rather than checking the command against zero. > I prefer the first, but only on the grounds that more typing is > done only in the rare case. prefer that `run_command -e IGNORE' continues to do what it states, e.g. *ignore* the failures instead of making the test case abort if `set -e' is on. Instead, what about adding support for a new special argument to `-e', say `RETURN' (which is what I did in the attached patch)? It's easy, far less obtrusive than adding another function, and it also keeps the "extra-typing" low (no need to ever add `-e 0'). Objections? > [CUT] > > > Well, even if we are going to make `set -e' the default, I think > > that this change should be introduced with patch. > > Do you mean "should be introduced with a separate patch"? Yes. I somehow managed to mess up the wording in the mail. Sorry. > If yes, then I agree. Good. > > > > BTW, now that we have TEST_LOG_COMPILER, and correctly unset it > > > in defs.in, too, we can set it in tests/Makefile.am and worry > > > less about shells like Solaris /bin/sh. Of course, that would > > > require us to warn that running tests directly (i.e., without > > > 'make' in-between), might require to use a decent shell. > > > > Or we could add proper code in `tests/defs.in', to make it > > re-execute the test scripts with CONFIG_SHELL. But this should > > be obviously done in a distinct patch. > Yes. > > --- a/tests/defs.in > > +++ b/tests/defs.in > > @@ -395,26 +395,91 @@ is_newest () > > test -z "$is_newest_files" > > } > > > > +# run_command [-e STATUS] [-i FILE] [-m] [--] COMMAND > > [ARGUMENTS..] +# > > ----------------------------------------------------------------- > > +# Run the given COMMAND with ARGUMENTS and fail if COMMAND does > > not exit +# with STATUS. If status is "VOID" or "IGNORE", any > > exit value of the +# command is acceptable. If STATUS is "FAIL", > > then any exit value of the +# command *but 0* is acceptable. > > Default STATUS is `0'. +# Also, save standard output and standard > > error from COMMAND, by default +# respectively in files `stdout' > > and `stderr' (in the current directory), +# or togheter in the > > file `stdall' (in the current directory) if the `-m' > > typo togheter > > +# option is given. > > +run_command () > > +{ > > + set +x # xtrace verbosity temporarly disabled in `run_command' > > + run_exitcode_expected=0 > > + run_mix_stdout_and_stderr=no > > + while test $# -gt 0; do > > + case $1 in > > + -e) run_exitcode_expected=$2; shift;; > > + -m) run_mix_stdout_and_stderr=yes;; > > + --) shift; break;; > > + -?) echo "run_commmand(): invalid switch \`$1'" >&2; Exit > > 99;; > > s/()// > > > + *) break;; > > + esac > > + shift > > + done > > + case $# in > > + 0) echo "run_command(): missing COMMAND argument" >&2; Exit > > 99;; > > Likewise. Regards, Stefano [0001-Testsuite-avoid-Zsh-related-problems-with-set-x.patch] From 599942d560a23dabe042db13066f190aa662ef66 Mon Sep 17 00:00:00 2001 From: Stefano Lattarini <stefano.lattarini@...> Date: Sun, 11 Oct 2009 14:49:58 +0200 Subject: [PATCH] Testsuite: avoid Zsh-related problems with `set -x'. * tests/README: Describe Zsh 4.x `set -x' a.k.a. `xtrace' issue and workaround with run_command. * tests/defs.in (run_command): New function, to be used for commands whose standard error needs to be captured and/or whose exit status must be checked. (save_shell_flags, restore_shell_flags): New functions, used in run_command. (AUTOMAKE_run): Reimplemented using run_command. Updated comment. (AUTOMAKE_fails): Updated comment. * tests/acloca14.test: Use new subroutine run_command instead of hand-crafted redirections of stdout and/or stderr. * tests/acloca17.test: Likewise. * tests/acloca18.test: Likewise. * tests/aclocal.test: Likewise. * tests/aclocal3.test: Likewise. * tests/aclocal8.test: Likewise. * tests/acsilent.test: Likewise. * tests/alpha.test: Likewise. * tests/ammissing.test: Likewise. * tests/ansi3.test: Likewise. * tests/ansi3b.test: Likewise. * tests/ccnoco3.test: Likewise. * tests/check8.test: Likewise. * tests/cond39.test: Likewise. * tests/defs.in: Likewise. * tests/depcomp2.test: Likewise. * tests/depcomp4.test: Likewise. * tests/depend5.test: Likewise. * tests/depend6.test: Likewise. * tests/discover.test: Likewise. * tests/distcom3.test: Likewise. * tests/fn99.test: Likewise. * tests/fn99subdir.test: Likewise. * tests/gnits2.test: Likewise. * tests/gnits3.test: Likewise. * tests/help.test: Likewise. * tests/init.test: Likewise. * tests/instsh.test: Likewise. * tests/ldadd.test: Likewise. * tests/libtool7.test: Likewise. * tests/libtool9.test: Likewise. * tests/lisp3.test: Likewise. * tests/ltorder.test: Likewise. * tests/man4.test: Likewise. * tests/mdate.test: Likewise. * tests/missing3.test: Likewise. * tests/missing4.test: Likewise. * tests/missing6.test: Likewise. * tests/obsolete.test: Likewise. * tests/output-order.test: Likewise. * tests/pr220.test: Likewise. * tests/pr300-ltlib.test: Likewise. * tests/python11.test: Likewise. * tests/python4.test: Likewise. * tests/python5.test: Likewise. * tests/python6.test: Likewise. * tests/python7.test: Likewise. * tests/python8.test: Likewise. * tests/python9.test: Likewise. * tests/sanity.test: Likewise. * tests/subobj.test: Likewise. * tests/subobj9.test: Likewise. * tests/unused.test: Likewise. * tests/vars3.test: Likewise. * tests/version8.test: Likewise. --- ChangeLog | 69 +++++++++++++++++++++++++++++ tests/README | 10 ++++ tests/acloca14.test | 4 +- tests/acloca17.test | 6 +-- tests/acloca18.test | 10 ++--- tests/aclocal.test | 8 ++-- tests/aclocal3.test | 5 +- tests/aclocal8.test | 7 +-- tests/acsilent.test | 7 ++- tests/alpha.test | 5 +- tests/ammissing.test | 5 +- tests/ansi3.test | 5 +- tests/ansi3b.test | 5 +- tests/ccnoco3.test | 3 +- tests/check8.test | 8 +--- tests/cond39.test | 8 +-- tests/defs.in | 111 +++++++++++++++++++++++++++++++++++++++++++---- tests/depcomp2.test | 14 +++--- tests/depcomp4.test | 7 +-- tests/depend5.test | 5 +-- tests/depend6.test | 7 +-- tests/discover.test | 5 +- tests/distcom3.test | 7 +-- tests/fn99.test | 5 +- tests/fn99subdir.test | 5 +- tests/gnits2.test | 6 +- tests/gnits3.test | 6 +- tests/help.test | 4 +- tests/init.test | 18 ++++++-- tests/instsh.test | 7 +-- tests/ldadd.test | 5 +- tests/libtool7.test | 23 ++++------ tests/libtool9.test | 9 ++-- tests/lisp3.test | 7 +-- tests/ltorder.test | 5 +- tests/man4.test | 8 +-- tests/mdate.test | 5 +- tests/missing3.test | 18 ++++---- tests/missing4.test | 8 +-- tests/missing6.test | 7 +-- tests/obsolete.test | 6 +- tests/output-order.test | 11 ++--- tests/pr220.test | 5 +- tests/pr300-ltlib.test | 6 +-- tests/python11.test | 6 +-- tests/python4.test | 5 +- tests/python5.test | 5 +- tests/python6.test | 5 +- tests/python7.test | 5 +- tests/python8.test | 5 +- tests/python9.test | 5 +- tests/sanity.test | 6 +-- tests/subobj.test | 5 +- tests/subobj9.test | 6 +- tests/unused.test | 6 +- tests/vars3.test | 5 +- tests/version8.test | 5 +- 57 files changed, 349 insertions(+), 215 deletions(-) diff --git a/ChangeLog b/ChangeLog index 3286548..4abd546 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,72 @@ +2009-10-16 Stefano Lattarini <stefano.lattarini@...> + + Fix testsuite: avoid Zsh-related problems with `set -x'. + * tests/README: Describe Zsh 4.x `set -x' a.k.a. `xtrace' issue + and workaround with run_command. + * tests/defs.in (run_command): New function, to be used for + commands whose standard error needs to be captured and/or whose + exit status must be checked. + (save_shell_flags, restore_shell_flags): New functions, used in + run_command. + (AUTOMAKE_run): Reimplemented using run_command. Updated comment. + (AUTOMAKE_fails): Updated comment. + * tests/acloca14.test: Use new subroutine run_command instead + of hand-crafted redirections of stdout and/or stderr. + * tests/acloca17.test: Likewise. + * tests/acloca18.test: Likewise. + * tests/aclocal.test: Likewise. + * tests/aclocal3.test: Likewise. + * tests/aclocal8.test: Likewise. + * tests/acsilent.test: Likewise. + * tests/alpha.test: Likewise. + * tests/ammissing.test: Likewise. + * tests/ansi3.test: Likewise. + * tests/ansi3b.test: Likewise. + * tests/ccnoco3.test: Likewise. + * tests/check8.test: Likewise. + * tests/cond39.test: Likewise. + * tests/defs.in: Likewise. + * tests/depcomp2.test: Likewise. + * tests/depcomp4.test: Likewise. + * tests/depend5.test: Likewise. + * tests/depend6.test: Likewise. + * tests/discover.test: Likewise. + * tests/distcom3.test: Likewise. + * tests/fn99.test: Likewise. + * tests/fn99subdir.test: Likewise. + * tests/gnits2.test: Likewise. + * tests/gnits3.test: Likewise. + * tests/help.test: Likewise. + * tests/init.test: Likewise. + * tests/instsh.test: Likewise. + * tests/ldadd.test: Likewise. + * tests/libtool7.test: Likewise. + * tests/libtool9.test: Likewise. + * tests/lisp3.test: Likewise. + * tests/ltorder.test: Likewise. + * tests/man4.test: Likewise. + * tests/mdate.test: Likewise. + * tests/missing3.test: Likewise. + * tests/missing4.test: Likewise. + * tests/missing6.test: Likewise. + * tests/obsolete.test: Likewise. + * tests/output-order.test: Likewise. + * tests/pr220.test: Likewise. + * tests/pr300-ltlib.test: Likewise. + * tests/python11.test: Likewise. + * tests/python4.test: Likewise. + * tests/python5.test: Likewise. + * tests/python6.test: Likewise. + * tests/python7.test: Likewise. + * tests/python8.test: Likewise. + * tests/python9.test: Likewise. + * tests/sanity.test: Likewise. + * tests/subobj.test: Likewise. + * tests/subobj9.test: Likewise. + * tests/unused.test: Likewise. + * tests/vars3.test: Likewise. + * tests/version8.test: Likewise. + 2009-10-11 Ralf Wildenhues <Ralf.Wildenhues@...> Improve description of the various *LINK variables. diff --git a/tests/README b/tests/README index 4068ad6..9b25be9 100644 --- a/tests/README +++ b/tests/README @@ -115,6 +115,16 @@ Do Use $sleep when you have to make sure that some file is newer than another. + To run a program and analyze its stderr, use the run_command function. + For example: + run_command PROG [ARGS...] + grep $pattern stderr + or (if you must check that nothing is written on stderr): + run_command PROG2 [ARGS...] + test ! -s stderr + See section "File Descriptors" in the Autoconf manual to understand why + this can be useful, or even required. + Use `cat' or `grep' to display (part of) files that may be interesting for debugging, so that when a user send a verbose output we don't have to ask him for more details. Display stderr diff --git a/tests/acloca14.test b/tests/acloca14.test index 8ddbb1a..cf9c484 100755 --- a/tests/acloca14.test +++ b/tests/acloca14.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2004, 2008 Free Software Foundation, Inc. +# Copyright (C) 2004, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -108,5 +108,5 @@ $MAKE testdist2 # Make sure aclocal diagnose missing included files with correct `file:line:'. rm -f b.m4 -$ACLOCAL 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e 1 $ACLOCAL grep 'a.m4:1:.*b.m4.*does not exist' stderr diff --git a/tests/acloca17.test b/tests/acloca17.test index 884f24b..788cb9d 100755 --- a/tests/acloca17.test +++ b/tests/acloca17.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2004, 2008 Free Software Foundation, Inc. +# Copyright (C) 2004, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -33,7 +33,5 @@ EOF # FIXME: We want autom4te's 'undefined required macro' warning to be fatal, # but have no means to say so to aclocal. We use WARNINGS=error instead. - -WARNINGS=error $ACLOCAL -I m4 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_command -e 1 env WARNINGS=error $ACLOCAL -I m4 grep 'configure.in:4:.*UNDEFINED_MACRO' stderr diff --git a/tests/acloca18.test b/tests/acloca18.test index 22feb23..7047eea 100755 --- a/tests/acloca18.test +++ b/tests/acloca18.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2005, 2006, 2008 Free Software Foundation, Inc. +# Copyright (C) 2005, 2006, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -86,7 +86,7 @@ grep macro23 foo ACLOCAL_TESTSUITE_FLAGS='-I 4 -I 1 -I 2' rm -f foo -$ACLOCAL --install 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e 1 $ACLOCAL --install grep AM_MACRO2 stderr ACLOCAL_TESTSUITE_FLAGS='-I 4 -I 1' @@ -105,9 +105,7 @@ AC_DEFUN([AM_MACRO1], [echo macro1d >> foo]) AC_DEFUN([AM_MACRO2], [echo macro2d >> foo]) EOF rm -f foo -$ACLOCAL --diff=diff >output 2>stderr -cat stderr -cat output -grep '#serial 456' output +run_command $ACLOCAL --diff=diff +grep '#serial 456' stdout test ! -f 4/m1.m4 grep 'installing.*4/m1.m4' stderr diff --git a/tests/aclocal.test b/tests/aclocal.test index 6b87a9d..945b363 100755 --- a/tests/aclocal.test +++ b/tests/aclocal.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1998, 2001, 2002, 2004, 2008 Free Software Foundation, Inc. +# Copyright (C) 1998, 2001, 2002, 2004, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -23,15 +23,15 @@ set -e $ACLOCAL --output=fred test -f fred -$ACLOCAL --output 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e 1 $ACLOCAL --output grep 'option.*--output.*an argument' stderr grep help stderr -$ACLOCAL --unknown-option 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e 1 $ACLOCAL --unknown-option grep 'unrecognized.*--unknown-option' stderr grep help stderr -$ACLOCAL --ver 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e 1 $ACLOCAL --ver grep 'unrecognized.*--ver' stderr grep help stderr diff --git a/tests/aclocal3.test b/tests/aclocal3.test index a550e50..34c5335 100755 --- a/tests/aclocal3.test +++ b/tests/aclocal3.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1998, 2002, 2004 Free Software Foundation, Inc. +# Copyright (C) 1998, 2002, 2004, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -27,6 +27,5 @@ AC_DEFUN([GNOME_X_CHECKS], [ ]) END -$ACLOCAL -I macros 2>stderr -cat stderr +run_command -e 1 $ACLOCAL -I macros grep 'macros/gnome.m4:2:.*AM_PATH_GTK.*not found' stderr diff --git a/tests/aclocal8.test b/tests/aclocal8.test index ac10aff..4cf7d73 100755 --- a/tests/aclocal8.test +++ b/tests/aclocal8.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2003, 2004 Free Software Foundation, Inc. +# Copyright (C) 2003, 2004, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -34,9 +34,8 @@ EOF echo 'AC_DEFUN([MACRO1],)' >m4/macro1.m4 echo 'AC_DEFUN([MACRO2], [AC_REQUIRE([AM_UNUSED_MACRO])])' >m4/macro2.m4 -$ACLOCAL -I m4 >output 2>&1 -cat output -test 0 = `wc -l <output` +run_command -m $ACLOCAL -I m4 +test -s stdall && Exit 1 grep macro1.m4 aclocal.m4 grep macro2.m4 aclocal.m4 && Exit 1 : diff --git a/tests/acsilent.test b/tests/acsilent.test index 6ac7863..e250092 100755 --- a/tests/acsilent.test +++ b/tests/acsilent.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1996, 2001, 2002, 2008 Free Software Foundation, Inc. +# Copyright (C) 1996, 2001, 2002, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -32,5 +32,6 @@ module=[$1] AC_SUBST(module)]) END -$ACLOCAL > output 2>&1 || { cat output; Exit 1; } -test -z "`cat output`" +run_command -m $ACLOCAL +test -s stdall && Exit 1 +: diff --git a/tests/alpha.test b/tests/alpha.test index 1ee8ed6..1410b9d 100755 --- a/tests/alpha.test +++ b/tests/alpha.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1998, 2001, 2002, 2003, 2008 Free Software Foundation, Inc. +# Copyright (C) 1998, 2001, 2002, 2003, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -62,8 +62,7 @@ $AUTOMAKE ./configure # make distdir should fail because NEWS does not mention 1.0a -$MAKE check 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_command -e FAIL $MAKE check grep 'NEWS not updated' stderr test ! -f works diff --git a/tests/ammissing.test b/tests/ammissing.test index 61d360a..42aa481 100755 --- a/tests/ammissing.test +++ b/tests/ammissing.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1997, 2002, 2004 Free Software Foundation, Inc. +# Copyright (C) 1997, 2002, 2004, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -20,6 +20,5 @@ echo AM_ZARDOZ >> configure.in -$ACLOCAL 2>stderr -cat stderr +run_command -e 1 $ACLOCAL grep 'configure.in:.*AM_ZARDOZ.*not found' stderr diff --git a/tests/ansi3.test b/tests/ansi3.test index 2bbcaf7..95e4fc1 100755 --- a/tests/ansi3.test +++ b/tests/ansi3.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1998, 1999, 2001, 2002, 2003 Free Software Foundation, Inc. +# Copyright (C) 1998, 1999, 2001, 2002, 2003, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -53,8 +53,7 @@ export CFLAGS # Linux boxes (Red Hat 5.1 in particular). $ACLOCAL $AUTOCONF -$AUTOMAKE -a 2>stderr -cat stderr +run_command $AUTOMAKE -a grep 'Makefile.am:1:.*ansi2knr\.c' stderr CC='gcc' ./configure ANSI2KNR=./ansi2knr U=_ $MAKE -e diff --git a/tests/ansi3b.test b/tests/ansi3b.test index 8f254ad..447b690 100755 --- a/tests/ansi3b.test +++ b/tests/ansi3b.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1998, 1999, 2001, 2002, 2003 Free Software Foundation, Inc. +# Copyright (C) 1998, 1999, 2001, 2002, 2003, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -52,8 +52,7 @@ export CFLAGS # Linux boxes (Red Hat 5.1 in particular). $ACLOCAL $AUTOCONF -$AUTOMAKE -a 2>stderr -cat stderr +run_command $AUTOMAKE -a grep 'configure.in:2:.*ansi2knr\.1' stderr CC='gcc' ./configure ANSI2KNR=./ansi2knr U=_ $MAKE -e diff --git a/tests/ccnoco3.test b/tests/ccnoco3.test index 2e9cd37..93731a1 100755 --- a/tests/ccnoco3.test +++ b/tests/ccnoco3.test @@ -74,8 +74,7 @@ CC=`pwd`/../Mycomp export CC ../configure -$MAKE 2>stderr || { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_command $MAKE grep 'mv.*the same file' stderr && Exit 1 Exit 0 diff --git a/tests/check8.test b/tests/check8.test index 7936582..a087cb7 100755 --- a/tests/check8.test +++ b/tests/check8.test @@ -67,10 +67,7 @@ $AUTOMAKE -a unset TESTS || : ./configure -AM_COLOR_TESTS=always $MAKE -e check >stdout 2>stderr && - { cat stdout; cat stderr >&2; Exit 1; } -cat stdout -cat stderr >&2 +run_command -e FAIL -- env AM_COLOR_TESTS=always $MAKE -e check grep 'XPASS.* foo$' stdout grep '^[^X]*PASS.* sub/foo$' stdout grep '^[^X]*PASS.* bar' stdout @@ -87,8 +84,7 @@ $MAKE distclean mkdir build cd build ../configure -$MAKE check >stdout && { cat stdout; Exit 1; } -cat stdout +run_command -e FAIL $MAKE check # Note: we are not grepping for the space here, due to the Solaris make VPATH # rewriting (if we fix that, we can still write a separate test for it). grep 'XPASS.*foo$' stdout diff --git a/tests/cond39.test b/tests/cond39.test index 79eaa70..1e44866 100755 --- a/tests/cond39.test +++ b/tests/cond39.test @@ -1,5 +1,5 @@ #!/bin/sh -# Copyright (C) 2008 Free Software Foundation, Inc. +# Copyright (C) 2008, 2009 Free Software Foundation, Inc. # # This file is part of GNU Automake. # @@ -82,8 +82,7 @@ $AUTOCONF $AUTOMAKE --add-missing ./configure COND=true -$MAKE 2>stderr -cat stderr +run_command $MAKE grep 'overriding commands' stderr && Exit 1 $MAKE sure-exist ./prog1 && Exit 1 @@ -97,8 +96,7 @@ $MAKE sure-exist $MAKE distclean ./configure COND=false -$MAKE 2>stderr -cat stderr +run_command $MAKE grep 'overriding commands' stderr && Exit 1 ./prog1 ./sub/prog2 diff --git a/tests/defs.in b/tests/defs.in index ecdf069..bfdbc37 100644 --- a/tests/defs.in +++ b/tests/defs.in @@ -395,26 +395,119 @@ is_newest () test -z "$is_newest_files" } +# save_shell_flags [VAR-BASENAME] +# ------------------------------- +# Save current shell flags in variable VAR_saved_shell_flags +# (default: _saved_shell_flags) +save_shell_flags () +{ + eval "${1}_saved_shell_flags=\$-"; +} + +# restore_shell_flags [VAR] +# ------------------------- +# Restore shell flags saved in variable VAR_saved_shell_flags +# (default: _saved_shell_flags) +restore_shell_flags () +{ + eval set -\${"${1}_saved_shell_flags"} +} + +# run_command [-e STATUS] [-m] [--] COMMAND [ARGUMENTS..] +# ------------------------------------------------------- +# Run the given COMMAND with ARGUMENTS, and cause the test to FAIL if +# COMMAND does not exit with STATUS (more details are given below). +# Also, save standard output and standard error from COMMAND, by default +# respectively in files `stdout' and `stderr' (in the current directory), +# or together in the file `stdall' (in the current directory) if the `-m' +# option is given. +# If STATUS is numeric, the exit value of COMMAND must match it exactly. +# If STATUS is "FAIL" or "FAILURE", then any exit value of COMMAND *but 0* +# is acceptable. If STATUS is "IGNORE" or "IGNORED", any exit value of +# COMMAND is acceptable, and run_command returns with success regardless +# of that value. Finally, if STATUS is "RETURN", any exit value of COMMAND +# is acceptable, *and* such exit value is returned by run_command. +# Default STATUS is `0'. +run_command () +{ + save_shell_flags 'run_command' + set +x # xtrace verbosity temporarly disabled in this function + run_exitcode_expected=0 + run_mix_stdout_and_stderr=no + while test $# -gt 0; do + case $1 in + -e) run_exitcode_expected=$2; shift;; + -m) run_mix_stdout_and_stderr=yes;; + --) shift; break;; + -?) echo "run_commmand: invalid switch \`$1'" >&2; Exit 99;; + *) break;; + esac + shift + done + case $# in + 0) echo "run_command: missing COMMAND argument" >&2; Exit 99;; + *) run_cmd=$1; shift;; + esac + if test x"$run_mix_stdout_and_stderr" = x"yes"; then + run_evald_cmd='"$run_cmd" ${1+"$@"} >stdall 2>&1' + else + run_evald_cmd='"$run_cmd" ${1+"$@"} >stdout 2>stderr' + fi + if eval "$run_evald_cmd"; then + run_exitcode_got=0 + else + run_exitcode_got=$? + fi + if test x"$run_mix_stdout_and_stderr" = x"yes"; then + set -x + cat stdall + { set +x; } 2>/dev/null + else + set -x + cat stderr >&2 + cat stdout + { set +x; } 2>/dev/null + fi + case $run_exitcode_expected in + RETURN|return) + restore_shell_flags 'run_command' + return $run_exitcode_got # RETURN POINT HERE + ;; + IGNORE|ignore|IGNORED|ignored|$run_exitcode_got) + run_ok=yes + ;; + FAIL|fail|FAILURE|failure) + if test $run_exitcode_got -gt 0; then + run_ok=yes + else + run_ok=no + fi + ;; + *) + run_ok=no + ;; + esac + echo "run_command: exit status $run_exitcode_got (expecting" \ + "$run_exitcode_expected)" + restore_shell_flags 'run_command' + test x"$run_ok" = x"yes" || Exit 1 +} # AUTOMAKE_run status [options...] # -------------------------------- -# Run Automake with OPTIONS, and fail if automake +# Run Automake with OPTIONS, and cause the test to FAIL if automake # does not exit with STATUS. AUTOMAKE_run () { - expected_exitcode=$1 + am_run_expected_exitcode=$1 shift - exitcode=0 - $AUTOMAKE ${1+"$@"} >stdout 2>stderr || exitcode=$? - cat stderr >&2 - cat stdout - test $exitcode = $expected_exitcode || Exit 1 + run_command -e $am_run_expected_exitcode -- $AUTOMAKE ${1+"$@"} } # AUTOMAKE_fails [options...] # --------------------------- -# Run Automake with OPTIONS, and fail if automake -# does not exit with STATUS. +# Run Automake with OPTIONS, and cuse the test to FAIL if automake +# does not exit with status 1. AUTOMAKE_fails () { AUTOMAKE_run 1 ${1+"$@"} diff --git a/tests/depcomp2.test b/tests/depcomp2.test index 4653c90..0daf2f0 100755 --- a/tests/depcomp2.test +++ b/tests/depcomp2.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2001, 2002 Free Software Foundation, Inc. +# Copyright (C) 2001, 2002, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -20,6 +20,8 @@ required=gcc . ./defs || Exit 1 +set -e + cat > configure.in << 'END' AC_INIT(subdir/foo.c) AM_INIT_AUTOMAKE(nonesuch, nonesuch) @@ -45,8 +47,8 @@ END CFLAGS= export CFLAGS -$ACLOCAL || Exit 1 -$AUTOMAKE --add-missing || Exit 1 -$AUTOCONF || Exit 1 -./configure CC='gcc' 2>error.log -test -z "`cat error.log`" +$ACLOCAL +$AUTOMAKE --add-missing +$AUTOCONF +run_command ./configure CC='gcc' +test ! -s stderr diff --git a/tests/depcomp4.test b/tests/depcomp4.test index 7d22894..70cbd57 100755 --- a/tests/depcomp4.test +++ b/tests/depcomp4.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2003 Free Software Foundation, Inc. +# Copyright (C) 2003, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -51,7 +51,6 @@ $AUTOMAKE -a grep am_cv_CC_dependencies_compiler_type configure ./configure am_cv_CC_dependencies_compiler_type=makedepend -$MAKE >output 2>&1 -cat output -grep 'unknown directive' output && Exit 1 +run_command -m $MAKE +grep 'unknown directive' stdall && Exit 1 $MAKE grepdeps diff --git a/tests/depend5.test b/tests/depend5.test index 0d9be3b..52e4169 100755 --- a/tests/depend5.test +++ b/tests/depend5.test @@ -65,10 +65,7 @@ for arg in Makefile \ "--file=$name" do rm -rf .deps _deps - ./config.status "$arg" depfiles >stdout 2>stderr || - { cat stdout; cat stderr >&2; Exit 1; } - cat stdout - cat stderr >&2 + run_command ./config.status "$arg" depfiles grep '[Nn]o such file' stderr && Exit 1 if test -n "$depdir"; then diff --git a/tests/depend6.test b/tests/depend6.test index de2adf9..5ce2208 100755 --- a/tests/depend6.test +++ b/tests/depend6.test @@ -39,9 +39,6 @@ END $ACLOCAL $AUTOMAKE $AUTOCONF -./configure --disable-dependency-tracking 2>stderr || { - stat=$?; cat stderr >&2; Exit $stat; -} -cat stderr >&2 -grep shift stderr && Exit 1 +run_command ./configure --disable-dependency-tracking +grep 'shift' stderr && Exit 1 : diff --git a/tests/discover.test b/tests/discover.test index 36293f7..c62f1b4 100755 --- a/tests/discover.test +++ b/tests/discover.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1996, 1997, 2000, 2001, 2002 Free Software Foundation, Inc. +# Copyright (C) 1996, 1997, 2000, 2001, 2002, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -34,5 +34,6 @@ END : > fsusage.c $ACLOCAL || Exit 1 -$AUTOMAKE 2>&1 | $FGREP fsusage.c || Exit 1 +run_command -e IGNORE -m $AUTOMAKE +$FGREP fsusage.c stdall || Exit 1 Exit 0 diff --git a/tests/distcom3.test b/tests/distcom3.test index 04bf365..fdcc31a 100755 --- a/tests/distcom3.test +++ b/tests/distcom3.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2001, 2002, 2003, 2004, 2006 Free Software Foundation, Inc. +# Copyright (C) 2001, 2002, 2003, 2004, 2006, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -34,9 +34,8 @@ END $ACLOCAL # Should not warn about missing README, since it is a target. -$AUTOMAKE --add-missing --gnu >stdout 2>&1 -cat stdout -grep README stdout && Exit 1 +run_command -m $AUTOMAKE --add-missing --gnu +grep README stdall && Exit 1 sed -n -e '/^DIST_COMMON =.*\\$/ { :loop diff --git a/tests/fn99.test b/tests/fn99.test index 982a989..611c767 100755 --- a/tests/fn99.test +++ b/tests/fn99.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2004, 2008 Free Software Foundation, Inc. +# Copyright (C) 2004, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -47,8 +47,7 @@ do touch x done) || Exit 77 -$MAKE dist 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_command -e FAIL $MAKE dist grep 'filenames are too long' stderr test 2 = `grep 12345678 stderr | wc -l` : diff --git a/tests/fn99subdir.test b/tests/fn99subdir.test index 465f2af..7055e85 100755 --- a/tests/fn99subdir.test +++ b/tests/fn99subdir.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2006, 2008 Free Software Foundation, Inc. +# Copyright (C) 2006, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -73,8 +73,7 @@ do touch x done) -$MAKE dist 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_command -e FAIL $MAKE dist grep 'filenames are too long' stderr test 1 = `grep 12345678 stderr | wc -l` : diff --git a/tests/gnits2.test b/tests/gnits2.test index 4c5d54f..a85b096 100755 --- a/tests/gnits2.test +++ b/tests/gnits2.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2002, 2003, 2004, 2007 Free Software Foundation, Inc. +# Copyright (C) 2002, 2003, 2004, 2007, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -105,6 +105,6 @@ cd build ../configure "--prefix=`pwd`/../inst-dir" --program-prefix=p $MAKE all $MAKE test-install -$MAKE -k installcheck 2>stderr || : # Never trust the exit status of make -k. -cat stderr +# Never trust the exit status of make -k. +run_command -e IGNORE $MAKE -k installcheck $MAKE grep-stderr diff --git a/tests/gnits3.test b/tests/gnits3.test index 7a08fb8..4d90684 100755 --- a/tests/gnits3.test +++ b/tests/gnits3.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2002, 2007 Free Software Foundation, Inc. +# Copyright (C) 2002, 2007, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -83,8 +83,8 @@ cd build ../configure "--prefix=`pwd`/../inst-dir" --program-prefix=p $MAKE $MAKE install -$MAKE -k installcheck 2>stderr || : # Never trust the exit status of make -k. -cat stderr +# Never trust the exit status of make -k. +run_command -e IGNORE $MAKE -k installcheck $MAKE grep-stderr # Make sure there is no more error when all targets are exempted. diff --git a/tests/help.test b/tests/help.test index d4d7110..cae8967 100755 --- a/tests/help.test +++ b/tests/help.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2003, 2008 Free Software Foundation, Inc. +# Copyright (C) 2003, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -32,7 +32,7 @@ $AUTOMAKE --version $AUTOMAKE --help # aclocal and automake cannot work without configure.ac or configure.in -$ACLOCAL 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e 1 $ACLOCAL grep configure.ac stderr grep configure.in stderr AUTOMAKE_fails diff --git a/tests/init.test b/tests/init.test index 38ec681..b6a7375 100755 --- a/tests/init.test +++ b/tests/init.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2006, 2008 Free Software Foundation, Inc. +# Copyright (C) 2006, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -27,7 +27,17 @@ AM_INIT_AUTOMAKE END # The error message should mension AC_INIT, not AC_PACKAGE_VERSION. -($ACLOCAL && $AUTOCONF) 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +if run_command -e RETURN $ACLOCAL; then + success=yes +else + success=no +fi +if test x"$success" != x"no"; then + mv stderr stder1 + run_command -e RETURN $AUTOCONF || success=no + cat stderr >>stder1 + mv -f stder1 stderr +fi +test x"$success" = x"no" || Exit 1 grep AC_PACKAGE_VERSION stderr && Exit 1 -grep AC_INIT stderr +grep AC_INIT stderr || Exit 1 diff --git a/tests/instsh.test b/tests/instsh.test index 61f5f9e..fe3d485 100755 --- a/tests/instsh.test +++ b/tests/instsh.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1996, 1999, 2001, 2002, 2003 Free Software Foundation, Inc. +# Copyright (C) 1996, 1999, 2001, 2002, 2003, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -32,10 +32,9 @@ mv Makefile.am configure.in frob/ cd frob $ACLOCAL -$AUTOMAKE --add-missing > output 2>&1 +run_command -m $AUTOMAKE --add-missing # Only one `/' should appear in the output. -cat output -grep '/.*/' output && Exit 1 +grep '/.*/' stdall && Exit 1 test -f install-sh diff --git a/tests/ldadd.test b/tests/ldadd.test index 005f9d8..0a9907d 100755 --- a/tests/ldadd.test +++ b/tests/ldadd.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1997, 1999, 2000, 2001, 2002 Free Software Foundation, Inc. +# Copyright (C) 1997, 1999, 2000, 2001, 2002, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -39,5 +39,6 @@ END : > q.c $ACLOCAL || Exit 1 -$AUTOMAKE 2>&1 | grep LDADD || Exit 1 +run_command -e 1 -m $AUTOMAKE +grep LDADD stdall || Exit 1 Exit 0 diff --git a/tests/libtool7.test b/tests/libtool7.test index a2f912f..f8706d9 100755 --- a/tests/libtool7.test +++ b/tests/libtool7.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2003, 2004, 2007 Free Software Foundation, Inc. +# Copyright (C) 2003, 2004, 2007, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -82,19 +82,16 @@ $AUTOCONF $AUTOMAKE --add-missing --copy ./configure "--prefix=`pwd`/_inst" -env LIBTOOLFLAGS=--silent $MAKE print >output 2>&1 -cat output -grep '1BEG: libmod1.la mod2.la :END1' output -grep '2BEG: mod2.la :END2' output -grep '3BEG: .*silent.*silent.* :END3' output +run_command -m env LIBTOOLFLAGS=--silent $MAKE print +grep '1BEG: libmod1.la mod2.la :END1' stdall +grep '2BEG: mod2.la :END2' stdall +grep '3BEG: .*silent.*silent.* :END3' stdall test 2 -le `grep mod2_la_LIBTOOLFLAGS Makefile | wc -l` $MAKE -env LIBTOOLFLAGS=--silent $MAKE install >output 2>&1 -cat output -grep 'silent.*silent.*prg' output -grep 'silent.*silent.*libmod1' output +run_command -m env LIBTOOLFLAGS=--silent $MAKE install +grep 'silent.*silent.*prg' stdall +grep 'silent.*silent.*libmod1' stdall -env LIBTOOLFLAGS=--silent $MAKE uninstall >output 2>&1 -cat output -grep 'silent.*silent.*libmod1' output +run_command -m env LIBTOOLFLAGS=--silent $MAKE uninstall +grep 'silent.*silent.*libmod1' stdall diff --git a/tests/libtool9.test b/tests/libtool9.test index c0d9688..94e617b 100755 --- a/tests/libtool9.test +++ b/tests/libtool9.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2005 Free Software Foundation, Inc. +# Copyright (C) 2005, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -88,10 +88,11 @@ $AUTOCONF $AUTOMAKE --add-missing --copy ./configure -env LDFLAGS=ldflags AM_LDFLAGS=am_ldflags libmod1_la_LDFLAGS=lm1_la_ldflags \ +run_command -m env \ + LDFLAGS=ldflags AM_LDFLAGS=am_ldflags libmod1_la_LDFLAGS=lm1_la_ldflags \ CFLAGS=cflags AM_CFLAGS=am_cflags prg2_CFLAGS=prg2_cflags \ - $MAKE -e print >output 2>&1 -cat output + $MAKE -e print +mv stdall output grep '1BEG: libmod1.la mod2.la :END1' output grep '2BEG: mod2.la :END2' output grep '3BEG:.* am_cflags cflags .*lm1_la_ldflags ldflags.* :END3' output diff --git a/tests/lisp3.test b/tests/lisp3.test index 809d491..086173d 100755 --- a/tests/lisp3.test +++ b/tests/lisp3.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2003, 2008 Free Software Foundation, Inc. +# Copyright (C) 2003, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -61,9 +61,8 @@ test -f am-one.elc unique=0a3346e2af8a689b85002b53df09142a $sleep echo "(message \"$unique\")(provide 'am-three)" > am-three.el -$MAKE >output 2>&1 -cat output -grep $unique output +run_command -m $MAKE +grep $unique stdall # It should also work for VPATH-builds. $MAKE distcheck diff --git a/tests/ltorder.test b/tests/ltorder.test index 0f69211..1e6a03a 100755 --- a/tests/ltorder.test +++ b/tests/ltorder.test @@ -1,5 +1,5 @@ #!/bin/sh -# Copyright (C) 2008 Free Software Foundation, Inc. +# Copyright (C) 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -53,8 +53,7 @@ $AUTOMAKE --add-missing ./configure "--prefix=`pwd`/inst" $MAKE -$MAKE install 2>stderr || { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_command $MAKE install grep 'has not been installed' stderr && Exit 1 $MAKE uninstall diff --git a/tests/man4.test b/tests/man4.test index aa86a9e..7424c72 100755 --- a/tests/man4.test +++ b/tests/man4.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2008 Free Software Foundation, Inc. +# Copyright (C) 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -88,11 +88,9 @@ mkdir build cd build ../configure $MAKE -$MAKE dist 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_command -e FAIL $MAKE dist grep 'install help2man' stderr -$MAKE distcheck 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_command -e FAIL $MAKE distcheck grep 'install help2man' stderr rm -f ../help2man diff --git a/tests/mdate.test b/tests/mdate.test index 704b393..e63bff9 100755 --- a/tests/mdate.test +++ b/tests/mdate.test @@ -1,5 +1,5 @@ #!/bin/sh -# Copyright (C) 1996, 1997, 2001, 2002 Free Software Foundation, Inc. +# Copyright (C) 1996, 1997, 2001, 2002, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -31,4 +31,5 @@ END : > texinfo.tex $ACLOCAL || Exit 1 -$AUTOMAKE 2>&1 > /dev/null | grep 'required file.*mdate-sh' +run_command -e IGNORED $AUTOMAKE +grep 'required file.*mdate-sh' stderr diff --git a/tests/missing3.test b/tests/missing3.test index 07ef779..9b48368 100755 --- a/tests/missing3.test +++ b/tests/missing3.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2004, 2008 Free Software Foundation, Inc. +# Copyright (C) 2004, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -22,24 +22,24 @@ set -e # b7cb8259 assumed not to exist. -./missing b7cb8259 --version 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e FAIL ./missing b7cb8259 --version grep . stderr && Exit 1 -./missing b7cb8259 --grep 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e FAIL ./missing b7cb8259 --grep grep WARNING stderr -./missing --run b7cb8259 --version && Exit 1 -./missing --run b7cb8259 --grep 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e FAIL ./missing --run b7cb8259 --version +run_command -e FAIL ./missing --run b7cb8259 --grep grep WARNING stderr # missing itself it known to exist :) -./missing ./missing --version 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e FAIL ./missing ./missing --version grep . stderr && Exit 1 -./missing ./missing --grep 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e FAIL ./missing ./missing --grep grep WARNING stderr -./missing --run ./missing --version 2>stderr +run_command ./missing --run ./missing --version grep . stderr && Exit 1 -./missing --run ./missing --grep 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e FAIL ./missing --run ./missing --grep grep WARNING stderr && Exit 1 grep Unknown stderr diff --git a/tests/missing4.test b/tests/missing4.test index 1015828..c6c3b7e 100755 --- a/tests/missing4.test +++ b/tests/missing4.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2006, 2007, 2008 Free Software Foundation, Inc. +# Copyright (C) 2006, 2007, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -36,13 +36,11 @@ cmp aclocal.m4 aclocal.tmp && Exit 1 mv aclocal.tmp aclocal.m4 -$MAKE 2>stderr -cat stderr +run_command $MAKE grep 'You have another version of autoconf' stderr grep -c 'aclocal.m4:.*this file was generated for' stderr -$MAKE 2>stderr -cat stderr +run_command $MAKE grep 'You have another version of autoconf' stderr && Exit 1 grep 'aclocal.m4:.*this file was generated for' stderr && Exit 1 diff --git a/tests/missing6.test b/tests/missing6.test index b63a0d1..bf6df2d 100755 --- a/tests/missing6.test +++ b/tests/missing6.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2008 Free Software Foundation, Inc. +# Copyright (C) 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -31,7 +31,7 @@ rm -f configure.in touch Makefile.am $ACLOCAL -$AUTOCONF 2>stderr +run_command $AUTOCONF grep 'You have another version of autoconf' stderr grep 'aclocal.m4:.*this file was generated for' stderr $AUTOMAKE @@ -43,8 +43,7 @@ cmp configure.ac configure.tmp && Exit 1 mv configure.tmp configure.ac -$MAKE 2>stderr -cat stderr +run_command $MAKE grep 'You have another version of autoconf' stderr grep 'aclocal.m4:.*this file was generated for autoconf 9999a' stderr diff --git a/tests/obsolete.test b/tests/obsolete.test index e6601c9..daae69c 100755 --- a/tests/obsolete.test +++ b/tests/obsolete.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1996, 2001, 2002, 2007, 2008 Free Software Foundation, Inc. +# Copyright (C) 1996, 2001, 2002, 2007, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -27,7 +27,7 @@ END $PERL -ne '/AU_DEFUN\(\[(\w+)\]/ && print "$1\n"' \ "$testsrcdir/../m4/obsolete.m4" >> obs cat obs >> configure.in -$PERL -ne 'chomp; print "grep $_ stderr || Exit 1\n"; ' obs > obs.1 +$PERL -ne 'chomp; print "grep $_ stdall || Exit 1\n"; ' obs > obs.1 $PERL -ne 'chomp; print "grep $_ configure.in && Exit 1\n"; ' obs > obs.2 # Sanity check. Make sure we have added something to configure.in. @@ -36,7 +36,7 @@ test `cat configure.in | wc -l` -gt 1 || Exit 1 $ACLOCAL || Exit 1 # Expect Autoconf to complain about each of the macros in obs. -$AUTOCONF -Wobsolete >stderr 2>&1 +run_command -m $AUTOCONF -Wobsolete . ./obs.1 # Make sure Autoupdate remove each of these macros. $AUTOUPDATE || Exit 1 diff --git a/tests/output-order.test b/tests/output-order.test index 856baa2..852c194 100755 --- a/tests/output-order.test +++ b/tests/output-order.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2008 Free Software Foundation, Inc. +# Copyright (C) 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -43,13 +43,12 @@ END $ACLOCAL $AUTOCONF rm -f missing install-sh -$AUTOMAKE --add-missing --copy 2>stderr -cat stderr >&2 +run_command $AUTOMAKE --add-missing --copy +mv stderr stder0 for i in 1 2 3 4 5 6; do rm -f missing install-sh INSTALL COPYING + run_command -e IGNORE $AUTOMAKE --add-missing --copy # The grep prevents a Heisenbug with the HP-UX shell and VERBOSE=yes. - $AUTOMAKE --add-missing --copy 2>&1 >/dev/null | - grep -v /dev/null | - diff - stderr + grep -v /dev/null stder0 | diff - stderr done diff --git a/tests/pr220.test b/tests/pr220.test index 25e1f0e..6644d28 100755 --- a/tests/pr220.test +++ b/tests/pr220.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2001, 2002, 2004, 2008 Free Software Foundation, Inc. +# Copyright (C) 2001, 2002, 2004, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -55,6 +55,5 @@ $AUTOMAKE -a cd build # configure should fail since we've done something invalid. -../configure 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_command -e FAIL ../configure grep NEVER_TRUE stderr diff --git a/tests/pr300-ltlib.test b/tests/pr300-ltlib.test index 01814a9..7ed0a2f 100755 --- a/tests/pr300-ltlib.test +++ b/tests/pr300-ltlib.test @@ -48,8 +48,7 @@ $ACLOCAL $AUTOCONF $AUTOMAKE --copy --add-missing ./configure --prefix "`pwd`/inst" -$MAKE V=1 >stdout -cat stdout +run_command $MAKE V=1 grep 'liba.la .*-rpath .*lib' stdout grep 'liba.la .*-rpath .*lib/subdir' stdout && Exit 1 @@ -58,8 +57,7 @@ grep 'libb.la .*-rpath .*lib/subdir' stdout test -f subdir/liba.la test -f subdir/libb.la -$MAKE install 2>stderr -cat stderr >&2 +run_command $MAKE install grep 'remember.*--finish' stderr && Exit 1 test -f inst/lib/liba.la diff --git a/tests/python11.test b/tests/python11.test index 0f15f37..920de88 100755 --- a/tests/python11.test +++ b/tests/python11.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2004, 2006, 2008 Free Software Foundation, Inc. +# Copyright (C) 2004, 2006, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -35,9 +35,7 @@ EOF $ACLOCAL $AUTOCONF -./configure >stdout 2>stderr && { cat stdout; cat stderr >&2; Exit 1; } -cat stdout -cat stderr >&2 +run_command -e FAILURE ./configure grep 'checking for IShouldNotExist1' stdout grep 'checking for IShouldNotExist2' stdout grep 'no suitable Python interpreter found' stderr diff --git a/tests/python4.test b/tests/python4.test index 11bb3ed..3ff44af 100755 --- a/tests/python4.test +++ b/tests/python4.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2003, 2008 Free Software Foundation, Inc. +# Copyright (C) 2003, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -33,6 +33,5 @@ $AUTOCONF $AUTOMAKE --add-missing # Simulate no Python -./configure PYTHON=: 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_command -e FAILURE ./configure PYTHON=: grep 'no suitable Python interpreter found' stderr diff --git a/tests/python5.test b/tests/python5.test index 86f5863..5695876 100755 --- a/tests/python5.test +++ b/tests/python5.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2003, 2008 Free Software Foundation, Inc. +# Copyright (C) 2003, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -34,6 +34,5 @@ $ACLOCAL $AUTOCONF $AUTOMAKE --add-missing -./configure 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_command -e FAILURE ./configure grep 'no suitable Python interpreter found' stderr diff --git a/tests/python6.test b/tests/python6.test index eb4fb4d..ed60e03 100755 --- a/tests/python6.test +++ b/tests/python6.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2003 Free Software Foundation, Inc. +# Copyright (C) 2003, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -34,6 +34,5 @@ $AUTOCONF $AUTOMAKE --add-missing # Simulate no Python -./configure PYTHON=: 2>stderr -cat stderr +run_command ./configure PYTHON=: grep 'GREP ME:' stderr diff --git a/tests/python7.test b/tests/python7.test index 62c1a1b..eb6a4dc 100755 --- a/tests/python7.test +++ b/tests/python7.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2003 Free Software Foundation, Inc. +# Copyright (C) 2003, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -34,6 +34,5 @@ $ACLOCAL $AUTOCONF $AUTOMAKE --add-missing -./configure 2>stderr -cat stderr +run_command ./configure grep 'GREP ME:' stderr diff --git a/tests/python8.test b/tests/python8.test index a50af22..5f100ab 100755 --- a/tests/python8.test +++ b/tests/python8.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2003 Free Software Foundation, Inc. +# Copyright (C) 2003, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -32,6 +32,5 @@ $ACLOCAL $AUTOCONF $AUTOMAKE --add-missing -./configure 2>stderr -cat stderr +run_command ./configure grep 'GREP ME' stderr diff --git a/tests/python9.test b/tests/python9.test index 9b23deb..c0fa7a5 100755 --- a/tests/python9.test +++ b/tests/python9.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2003 Free Software Foundation, Inc. +# Copyright (C) 2003, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -33,6 +33,5 @@ $ACLOCAL $AUTOCONF $AUTOMAKE --add-missing -./configure 2>stderr -cat stderr +run_command ./configure grep 'GREP ME' stderr diff --git a/tests/sanity.test b/tests/sanity.test index 1c96c78..1296381 100755 --- a/tests/sanity.test +++ b/tests/sanity.test @@ -36,14 +36,12 @@ cp ../install-sh ../missing . $ACLOCAL $AUTOCONF $AUTOMAKE -./configure 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr +run_command -e FAILURE ./configure grep 'unsafe absolute working directory' stderr cd .. mkdir build cd build -../unsafe$/configure 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr +run_command -e FAILURE ../unsafe$/configure grep 'unsafe srcdir' stderr : diff --git a/tests/subobj.test b/tests/subobj.test index df297f4..9d7882d 100755 --- a/tests/subobj.test +++ b/tests/subobj.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1999, 2001, 2002, 2004 Free Software Foundation, Inc. +# Copyright (C) 1999, 2001, 2002, 2004, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -32,8 +32,7 @@ wish_SOURCES = generic/a.c generic/b.c END $ACLOCAL -$AUTOMAKE --add-missing 2>stderr -cat stderr +run_command $AUTOMAKE --add-missing # Make sure compile is installed, and that Automake says so. grep 'install.*compile' stderr test -f compile diff --git a/tests/subobj9.test b/tests/subobj9.test index 99fe6e8..1f68721 100755 --- a/tests/subobj9.test +++ b/tests/subobj9.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2002, 2004, 2005 Free Software Foundation, Inc. +# Copyright (C) 2002, 2004, 2005, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -74,8 +74,8 @@ if test -n "`./libtool --help | grep tag=TAG`"; then fi $MAKE -$MAKE distcheck 2>&1 | tee out +run_command -m $MAKE distcheck # GNU Make used to complain that the Makefile contained two rules # for `src/.dirstamp' and `.//src/.dirstamp'. -grep 'overriding commands' out && Exit 1 +grep 'overriding commands' stdall && Exit 1 : diff --git a/tests/unused.test b/tests/unused.test index a5b4da6..68d5613 100755 --- a/tests/unused.test +++ b/tests/unused.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1998, 2002, 2003, 2008 Free Software Foundation, Inc. +# Copyright (C) 1998, 2002, 2003, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -29,5 +29,5 @@ AC_DEFUN([MACRO_1_2], echo 12) AC_DEFUN([MACRO_1_2_3], echo 123) END -$ACLOCAL 2> stderr || { cat stderr >&2; Exit 1; } -test -z "`cat stderr`" +run_command $ACLOCAL +test ! -s stderr diff --git a/tests/vars3.test b/tests/vars3.test index 3d10a90..9f11262 100755 --- a/tests/vars3.test +++ b/tests/vars3.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2002 Free Software Foundation, Inc. +# Copyright (C) 2002, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -38,8 +38,7 @@ EOF $ACLOCAL # Make sure this warning is print in the `portability' category. -$AUTOMAKE --warnings=no-error,none,portability 2>stderr -cat stderr +run_command $AUTOMAKE --warnings=no-error,none,portability # Lines number are printed in error message. # Use them to make sure errors are diagnosed against the right lines. diff --git a/tests/version8.test b/tests/version8.test index 91b11a0..ffe4a3e 100755 --- a/tests/version8.test +++ b/tests/version8.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2005, 2008 Free Software Foundation, Inc. +# Copyright (C) 2005, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -20,6 +20,5 @@ set -e echo 'AM_AUTOMAKE_VERSION([1.9])' >>configure.in -$ACLOCAL 2>stderr && { cat stderr >&2; Exit 0; } -cat stderr >&2 +run_command -e 1 $ACLOCAL $FGREP 'AM_INIT_AUTOMAKE([1.9])' stderr -- 1.6.3.3 |
|
|
Re: testsuite failures when test scripts are run with zsh
by Stefano Lattarini
::
Rate this Message:
Stefano Lattarini <stefano.lattarini@...> wrote:
> Hi Ralf. > > A new version of the patch is attached. It's still not definitive, > but I hope most things are settled by now. And, of course, they're not. Sigh. With FreeBSD-7.2 /bin/sh, the construct: if eval "$run_evald_cmd"; then run_exitcode_got=0 else run_exitcode_got=$? fi causes the shell to exit if `set -e' is on. Minimal testcase exposing the bug: $ /bin/sh -ec 'if false; then :; else :; fi; echo OK'; echo $? OK 0 $ /bin/sh -ec 'if eval false; then :; else :; fi; echo OK'; echo $? OK 1 l thought to revert to the uglier form: run_exitcode_got=0 eval "$run_evald_cmd" || run_exitcode_got=$? but this does not work either. Minimal testcase: /bin/sh -ec 'false || :; echo OK'; echo $? OK 0 /bin/sh -ec 'eval false || :; echo OK'; echo $? 1 Finally I settled to the following code: if (eval "exec $run_evald_cmd"); then run_exitcode_got=0 else run_exitcode_got=$? fi which works, and has also the advantage of ensuring to run only external programs, not builtins or shell functions. After testing on FreeBSD and re-testing on Debian, I will send an amended patch (and let's hope it will work eventually). In the meantime... > > Your run_command already has an -e option. If we're going to > > return the exit status anyway from the function, then we wouldn't > > need that -e *at all*, we could just return whatever status the > > command caused. > > > > So if it pleases you better, then we can change it to either of > > the following: > > > > - '-e ignore' ignores the status of the command; run_command > > returns it; > > - no '-e' just lets run_command return the status of the command, > > rather than checking the command against zero. > > I prefer the first, but only on the grounds that more typing is > > done only in the rare case. > > I thought about it, and now I strongly agree with you: adding > another subroutine is going to make things unnecessarly > complicated. But I'd prefer that `run_command -e IGNORE' continues > to do what it states, e.g. ignore the failures instead of making > the test case abort if `set -e' is on. Instead, what about adding > support for a new special argument to `-e', say `RETURN' (which is > what I did in the attached patch)? It's easy, far less obtrusive > than adding another function, and it also keeps the "extra-typing" > low (no need to ever add `-e 0'). Objections? Regards, Stefano |
|
|
Re: testsuite failures when test scripts are run with zsh
by Stefano Lattarini
::
Rate this Message:
Hello automakers.
The final version of the patch is attached. Sorry for the long delay, but being new to FreeeBSD I messed up things more than one time while runnning tests on that system. The patched testsuite runs succesfully on Debian/Linux with either bash, zsh 3 or zsh 4. With heirloom-sh, only the test `vala4.test' fails, but this is not a regression (it fails on the master branch too). On FreeBSD, there is no regression, but the following tests fails both on the master branch and with the patched testsuite: FAIL: cscope.test (exit: 1) FAIL: man4.test (exit: 1) FAIL: silent8.test (exit: 1) FAIL: subpkg.test (exit: 1) FAIL: txinfo21.test (exit: 1) Regards, Stefano [0001-Testsuite-avoid-Zsh-related-problem-with-set-x.patch] From 4c66b1f7e060271f4dff63ace3b49e4c3bc3cdf3 Mon Sep 17 00:00:00 2001 From: Stefano Lattarini <stefano.lattarini@...> Date: Fri, 23 Oct 2009 18:27:55 +0200 Subject: [PATCH] Testsuite: avoid Zsh-related problem with `set -x'. * tests/README: Describe Zsh 4.x `set -x' a.k.a. `xtrace' issue and workaround with run_command. * tests/defs.in (run_command): New function, to be used for commands whose standard error needs to be captured. (save_shell_flags, restore_shell_flags): New functions, used in run_command. (AUTOMAKE_run): Reimplemented using run_command. Updated comment. (AUTOMAKE_fails): Updated comment. * tests/acloca14.test: Use new subroutine run_command instead of hand-crafted redirections of stdout and/or stderr. * tests/acloca17.test: Likewise. * tests/acloca18.test: Likewise. * tests/aclocal.test: Likewise. * tests/aclocal3.test: Likewise. * tests/aclocal8.test: Likewise. * tests/acsilent.test: Likewise. * tests/alpha.test: Likewise. * tests/ammissing.test: Likewise. * tests/ansi3.test: Likewise. * tests/ansi3b.test: Likewise. * tests/ccnoco3.test: Likewise. * tests/check8.test: Likewise. * tests/cond39.test: Likewise. * tests/depcomp2.test: Likewise. * tests/depcomp4.test: Likewise. * tests/depend5.test: Likewise. * tests/depend6.test: Likewise. * tests/discover.test: Likewise. * tests/distcom3.test: Likewise. * tests/fn99.test: Likewise. * tests/fn99subdir.test: Likewise. * tests/gnits2.test: Likewise. * tests/gnits3.test: Likewise. * tests/help.test: Likewise. * tests/init.test: Likewise. * tests/instsh.test: Likewise. * tests/ldadd.test: Likewise. * tests/libtool7.test: Likewise. * tests/libtool9.test: Likewise. * tests/lisp3.test: Likewise. * tests/ltorder.test: Likewise. * tests/man4.test: Likewise. * tests/mdate.test: Likewise. * tests/missing3.test: Likewise. * tests/missing4.test: Likewise. * tests/missing6.test: Likewise. * tests/obsolete.test: Likewise. * tests/output-order.test: Likewise. * tests/pr220.test: Likewise. * tests/pr300-ltlib.test: Likewise. * tests/python11.test: Likewise. * tests/python4.test: Likewise. * tests/python5.test: Likewise. * tests/python6.test: Likewise. * tests/python7.test: Likewise. * tests/python8.test: Likewise. * tests/python9.test: Likewise. * tests/sanity.test: Likewise. * tests/subobj.test: Likewise. * tests/subobj9.test: Likewise. * tests/unused.test: Likewise. * tests/vars3.test: Likewise. * tests/version8.test: Likewise. --- ChangeLog | 67 ++++++++++++++++++++++++++ tests/README | 10 ++++ tests/acloca14.test | 4 +- tests/acloca17.test | 6 +-- tests/acloca18.test | 10 ++-- tests/aclocal.test | 8 ++-- tests/aclocal3.test | 5 +- tests/aclocal8.test | 7 +-- tests/acsilent.test | 7 ++- tests/alpha.test | 5 +- tests/ammissing.test | 5 +- tests/ansi3.test | 5 +- tests/ansi3b.test | 5 +- tests/ccnoco3.test | 3 +- tests/check8.test | 8 +-- tests/cond39.test | 8 +-- tests/defs.in | 118 +++++++++++++++++++++++++++++++++++++++++++---- tests/depcomp2.test | 14 +++-- tests/depcomp4.test | 7 +-- tests/depend5.test | 5 +-- tests/depend6.test | 7 +-- tests/discover.test | 5 +- tests/distcom3.test | 7 +-- tests/fn99.test | 5 +- tests/fn99subdir.test | 5 +- tests/gnits2.test | 6 +- tests/gnits3.test | 6 +- tests/help.test | 4 +- tests/init.test | 18 ++++++-- tests/instsh.test | 7 +-- tests/ldadd.test | 5 +- tests/libtool7.test | 23 ++++----- tests/libtool9.test | 9 ++-- tests/lisp3.test | 7 +-- tests/ltorder.test | 5 +- tests/man4.test | 8 +-- tests/mdate.test | 5 +- tests/missing3.test | 18 ++++---- tests/missing4.test | 8 +-- tests/missing6.test | 7 +-- tests/obsolete.test | 6 +- tests/output-order.test | 11 ++-- tests/pr220.test | 5 +- tests/pr300-ltlib.test | 6 +-- tests/python11.test | 6 +-- tests/python4.test | 5 +- tests/python5.test | 5 +- tests/python6.test | 5 +- tests/python7.test | 5 +- tests/python8.test | 5 +- tests/python9.test | 5 +- tests/sanity.test | 6 +-- tests/subobj.test | 5 +- tests/subobj9.test | 6 +- tests/unused.test | 6 +- tests/vars3.test | 5 +- tests/version8.test | 5 +- 57 files changed, 354 insertions(+), 215 deletions(-) diff --git a/ChangeLog b/ChangeLog index 2fe0566..8300a5b 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,70 @@ +2009-10-23 Stefano Lattarini <stefano.lattarini@...> + + Testsuite: avoid Zsh-related problem with `set -x'. + * tests/README: Describe Zsh 4.x `set -x' a.k.a. `xtrace' issue + and workaround with run_command. + * tests/defs.in (run_command): New function, to be used for + commands whose standard error needs to be captured. + (save_shell_flags, restore_shell_flags): New functions, used in + run_command. + (AUTOMAKE_run): Reimplemented using run_command. Updated comment. + (AUTOMAKE_fails): Updated comment. + * tests/acloca14.test: Use new subroutine run_command instead + of hand-crafted redirections of stdout and/or stderr. + * tests/acloca17.test: Likewise. + * tests/acloca18.test: Likewise. + * tests/aclocal.test: Likewise. + * tests/aclocal3.test: Likewise. + * tests/aclocal8.test: Likewise. + * tests/acsilent.test: Likewise. + * tests/alpha.test: Likewise. + * tests/ammissing.test: Likewise. + * tests/ansi3.test: Likewise. + * tests/ansi3b.test: Likewise. + * tests/ccnoco3.test: Likewise. + * tests/check8.test: Likewise. + * tests/cond39.test: Likewise. + * tests/depcomp2.test: Likewise. + * tests/depcomp4.test: Likewise. + * tests/depend5.test: Likewise. + * tests/depend6.test: Likewise. + * tests/discover.test: Likewise. + * tests/distcom3.test: Likewise. + * tests/fn99.test: Likewise. + * tests/fn99subdir.test: Likewise. + * tests/gnits2.test: Likewise. + * tests/gnits3.test: Likewise. + * tests/help.test: Likewise. + * tests/init.test: Likewise. + * tests/instsh.test: Likewise. + * tests/ldadd.test: Likewise. + * tests/libtool7.test: Likewise. + * tests/libtool9.test: Likewise. + * tests/lisp3.test: Likewise. + * tests/ltorder.test: Likewise. + * tests/man4.test: Likewise. + * tests/mdate.test: Likewise. + * tests/missing3.test: Likewise. + * tests/missing4.test: Likewise. + * tests/missing6.test: Likewise. + * tests/obsolete.test: Likewise. + * tests/output-order.test: Likewise. + * tests/pr220.test: Likewise. + * tests/pr300-ltlib.test: Likewise. + * tests/python11.test: Likewise. + * tests/python4.test: Likewise. + * tests/python5.test: Likewise. + * tests/python6.test: Likewise. + * tests/python7.test: Likewise. + * tests/python8.test: Likewise. + * tests/python9.test: Likewise. + * tests/sanity.test: Likewise. + * tests/subobj.test: Likewise. + * tests/subobj9.test: Likewise. + * tests/unused.test: Likewise. + * tests/vars3.test: Likewise. + * tests/version8.test: Likewise. + 2009-10-18 Ralf Wildenhues <Ralf.Wildenhues@...> Simplify Variable::_check_ambiguous_condition. diff --git a/tests/README b/tests/README index d494b10..0b26211 100644 --- a/tests/README +++ b/tests/README @@ -133,6 +133,16 @@ Do Use $sleep when you have to make sure that some file is newer than another. + To run a program and analyze its stderr, use the run_command function. + For example: + run_command PROG [ARGS...] + grep $pattern stderr + or (if you must check that nothing is written on stderr): + run_command PROG2 [ARGS...] + test ! -s stderr + See section "File Descriptors" in the Autoconf manual to understand why + this can be useful, or even required. + Use `cat' or `grep' to display (part of) files that may be interesting for debugging, so that when a user send a verbose output we don't have to ask him for more details. Display stderr diff --git a/tests/acloca14.test b/tests/acloca14.test index 8ddbb1a..cf9c484 100755 --- a/tests/acloca14.test +++ b/tests/acloca14.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2004, 2008 Free Software Foundation, Inc. +# Copyright (C) 2004, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -108,5 +108,5 @@ $MAKE testdist2 # Make sure aclocal diagnose missing included files with correct `file:line:'. rm -f b.m4 -$ACLOCAL 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e 1 $ACLOCAL grep 'a.m4:1:.*b.m4.*does not exist' stderr diff --git a/tests/acloca17.test b/tests/acloca17.test index 884f24b..788cb9d 100755 --- a/tests/acloca17.test +++ b/tests/acloca17.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2004, 2008 Free Software Foundation, Inc. +# Copyright (C) 2004, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -33,7 +33,5 @@ EOF # FIXME: We want autom4te's 'undefined required macro' warning to be fatal, # but have no means to say so to aclocal. We use WARNINGS=error instead. - -WARNINGS=error $ACLOCAL -I m4 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_command -e 1 env WARNINGS=error $ACLOCAL -I m4 grep 'configure.in:4:.*UNDEFINED_MACRO' stderr diff --git a/tests/acloca18.test b/tests/acloca18.test index 22feb23..7047eea 100755 --- a/tests/acloca18.test +++ b/tests/acloca18.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2005, 2006, 2008 Free Software Foundation, Inc. +# Copyright (C) 2005, 2006, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -86,7 +86,7 @@ grep macro23 foo ACLOCAL_TESTSUITE_FLAGS='-I 4 -I 1 -I 2' rm -f foo -$ACLOCAL --install 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e 1 $ACLOCAL --install grep AM_MACRO2 stderr ACLOCAL_TESTSUITE_FLAGS='-I 4 -I 1' @@ -105,9 +105,7 @@ AC_DEFUN([AM_MACRO1], [echo macro1d >> foo]) AC_DEFUN([AM_MACRO2], [echo macro2d >> foo]) EOF rm -f foo -$ACLOCAL --diff=diff >output 2>stderr -cat stderr -cat output -grep '#serial 456' output +run_command $ACLOCAL --diff=diff +grep '#serial 456' stdout test ! -f 4/m1.m4 grep 'installing.*4/m1.m4' stderr diff --git a/tests/aclocal.test b/tests/aclocal.test index 6b87a9d..945b363 100755 --- a/tests/aclocal.test +++ b/tests/aclocal.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1998, 2001, 2002, 2004, 2008 Free Software Foundation, Inc. +# Copyright (C) 1998, 2001, 2002, 2004, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -23,15 +23,15 @@ set -e $ACLOCAL --output=fred test -f fred -$ACLOCAL --output 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e 1 $ACLOCAL --output grep 'option.*--output.*an argument' stderr grep help stderr -$ACLOCAL --unknown-option 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e 1 $ACLOCAL --unknown-option grep 'unrecognized.*--unknown-option' stderr grep help stderr -$ACLOCAL --ver 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e 1 $ACLOCAL --ver grep 'unrecognized.*--ver' stderr grep help stderr diff --git a/tests/aclocal3.test b/tests/aclocal3.test index a550e50..34c5335 100755 --- a/tests/aclocal3.test +++ b/tests/aclocal3.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1998, 2002, 2004 Free Software Foundation, Inc. +# Copyright (C) 1998, 2002, 2004, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -27,6 +27,5 @@ AC_DEFUN([GNOME_X_CHECKS], [ ]) END -$ACLOCAL -I macros 2>stderr -cat stderr +run_command -e 1 $ACLOCAL -I macros grep 'macros/gnome.m4:2:.*AM_PATH_GTK.*not found' stderr diff --git a/tests/aclocal8.test b/tests/aclocal8.test index ac10aff..4cf7d73 100755 --- a/tests/aclocal8.test +++ b/tests/aclocal8.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2003, 2004 Free Software Foundation, Inc. +# Copyright (C) 2003, 2004, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -34,9 +34,8 @@ EOF echo 'AC_DEFUN([MACRO1],)' >m4/macro1.m4 echo 'AC_DEFUN([MACRO2], [AC_REQUIRE([AM_UNUSED_MACRO])])' >m4/macro2.m4 -$ACLOCAL -I m4 >output 2>&1 -cat output -test 0 = `wc -l <output` +run_command -m $ACLOCAL -I m4 +test -s stdall && Exit 1 grep macro1.m4 aclocal.m4 grep macro2.m4 aclocal.m4 && Exit 1 : diff --git a/tests/acsilent.test b/tests/acsilent.test index 6ac7863..e250092 100755 --- a/tests/acsilent.test +++ b/tests/acsilent.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1996, 2001, 2002, 2008 Free Software Foundation, Inc. +# Copyright (C) 1996, 2001, 2002, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -32,5 +32,6 @@ module=[$1] AC_SUBST(module)]) END -$ACLOCAL > output 2>&1 || { cat output; Exit 1; } -test -z "`cat output`" +run_command -m $ACLOCAL +test -s stdall && Exit 1 +: diff --git a/tests/alpha.test b/tests/alpha.test index 1ee8ed6..1410b9d 100755 --- a/tests/alpha.test +++ b/tests/alpha.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1998, 2001, 2002, 2003, 2008 Free Software Foundation, Inc. +# Copyright (C) 1998, 2001, 2002, 2003, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -62,8 +62,7 @@ $AUTOMAKE ./configure # make distdir should fail because NEWS does not mention 1.0a -$MAKE check 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_command -e FAIL $MAKE check grep 'NEWS not updated' stderr test ! -f works diff --git a/tests/ammissing.test b/tests/ammissing.test index 61d360a..42aa481 100755 --- a/tests/ammissing.test +++ b/tests/ammissing.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1997, 2002, 2004 Free Software Foundation, Inc. +# Copyright (C) 1997, 2002, 2004, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -20,6 +20,5 @@ echo AM_ZARDOZ >> configure.in -$ACLOCAL 2>stderr -cat stderr +run_command -e 1 $ACLOCAL grep 'configure.in:.*AM_ZARDOZ.*not found' stderr diff --git a/tests/ansi3.test b/tests/ansi3.test index 2bbcaf7..95e4fc1 100755 --- a/tests/ansi3.test +++ b/tests/ansi3.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1998, 1999, 2001, 2002, 2003 Free Software Foundation, Inc. +# Copyright (C) 1998, 1999, 2001, 2002, 2003, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -53,8 +53,7 @@ export CFLAGS # Linux boxes (Red Hat 5.1 in particular). $ACLOCAL $AUTOCONF -$AUTOMAKE -a 2>stderr -cat stderr +run_command $AUTOMAKE -a grep 'Makefile.am:1:.*ansi2knr\.c' stderr CC='gcc' ./configure ANSI2KNR=./ansi2knr U=_ $MAKE -e diff --git a/tests/ansi3b.test b/tests/ansi3b.test index 8f254ad..447b690 100755 --- a/tests/ansi3b.test +++ b/tests/ansi3b.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1998, 1999, 2001, 2002, 2003 Free Software Foundation, Inc. +# Copyright (C) 1998, 1999, 2001, 2002, 2003, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -52,8 +52,7 @@ export CFLAGS # Linux boxes (Red Hat 5.1 in particular). $ACLOCAL $AUTOCONF -$AUTOMAKE -a 2>stderr -cat stderr +run_command $AUTOMAKE -a grep 'configure.in:2:.*ansi2knr\.1' stderr CC='gcc' ./configure ANSI2KNR=./ansi2knr U=_ $MAKE -e diff --git a/tests/ccnoco3.test b/tests/ccnoco3.test index 2e9cd37..93731a1 100755 --- a/tests/ccnoco3.test +++ b/tests/ccnoco3.test @@ -74,8 +74,7 @@ CC=`pwd`/../Mycomp export CC ../configure -$MAKE 2>stderr || { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_command $MAKE grep 'mv.*the same file' stderr && Exit 1 Exit 0 diff --git a/tests/check8.test b/tests/check8.test index 7936582..a087cb7 100755 --- a/tests/check8.test +++ b/tests/check8.test @@ -67,10 +67,7 @@ $AUTOMAKE -a unset TESTS || : ./configure -AM_COLOR_TESTS=always $MAKE -e check >stdout 2>stderr && - { cat stdout; cat stderr >&2; Exit 1; } -cat stdout -cat stderr >&2 +run_command -e FAIL -- env AM_COLOR_TESTS=always $MAKE -e check grep 'XPASS.* foo$' stdout grep '^[^X]*PASS.* sub/foo$' stdout grep '^[^X]*PASS.* bar' stdout @@ -87,8 +84,7 @@ $MAKE distclean mkdir build cd build ../configure -$MAKE check >stdout && { cat stdout; Exit 1; } -cat stdout +run_command -e FAIL $MAKE check # Note: we are not grepping for the space here, due to the Solaris make VPATH # rewriting (if we fix that, we can still write a separate test for it). grep 'XPASS.*foo$' stdout diff --git a/tests/cond39.test b/tests/cond39.test index 79eaa70..1e44866 100755 --- a/tests/cond39.test +++ b/tests/cond39.test @@ -1,5 +1,5 @@ #!/bin/sh -# Copyright (C) 2008 Free Software Foundation, Inc. +# Copyright (C) 2008, 2009 Free Software Foundation, Inc. # # This file is part of GNU Automake. # @@ -82,8 +82,7 @@ $AUTOCONF $AUTOMAKE --add-missing ./configure COND=true -$MAKE 2>stderr -cat stderr +run_command $MAKE grep 'overriding commands' stderr && Exit 1 $MAKE sure-exist ./prog1 && Exit 1 @@ -97,8 +96,7 @@ $MAKE sure-exist $MAKE distclean ./configure COND=false -$MAKE 2>stderr -cat stderr +run_command $MAKE grep 'overriding commands' stderr && Exit 1 ./prog1 ./sub/prog2 diff --git a/tests/defs.in b/tests/defs.in index 6eec344..a130c8a 100644 --- a/tests/defs.in +++ b/tests/defs.in @@ -399,26 +399,126 @@ is_newest () test -z "$is_newest_files" } +# save_shell_flags [VAR-BASENAME] +# ------------------------------- +# Save current shell flags in variable VAR_saved_shell_flags +# (default: _saved_shell_flags) +save_shell_flags () +{ + eval "${1}_saved_shell_flags=\$-"; +} + +# restore_shell_flags [VAR] +# ------------------------- +# Restore shell flags saved in variable VAR_saved_shell_flags +# (default: _saved_shell_flags) +restore_shell_flags () +{ + eval set -\${"${1}_saved_shell_flags"} +} + +# run_command [-e STATUS] [-m] [--] COMMAND [ARGUMENTS..] +# ------------------------------------------------------- +# Run the given COMMAND with ARGUMENTS, and cause the test to FAIL if +# COMMAND does not exit with STATUS (more details are given below). +# Also, save standard output and standard error from COMMAND, by default +# respectively in files `stdout' and `stderr' (in the current directory), +# or together in the file `stdall' (in the current directory) if the `-m' +# option is given. +# If STATUS is numeric, the exit value of COMMAND must match it exactly. +# If STATUS is "FAIL" or "FAILURE", then any exit value of COMMAND *but 0* +# is acceptable. If STATUS is "IGNORE" or "IGNORED", any exit value of +# COMMAND is acceptable, and run_command returns with success regardless +# of that value. Finally, if STATUS is "RETURN", any exit value of COMMAND +# is acceptable, *and* such exit value is returned by run_command. +# Default STATUS is `0'. +run_command () +{ + save_shell_flags 'run_command' + set +x # xtrace verbosity temporarly disabled in this function + run_exitcode_expected=0 + run_mix_stdout_and_stderr=no + while test $# -gt 0; do + case $1 in + -e) run_exitcode_expected=$2; shift;; + -m) run_mix_stdout_and_stderr=yes;; + --) shift; break;; + -?) echo "run_commmand: invalid switch \`$1'" >&2; Exit 99;; + *) break;; + esac + shift + done + case $# in + 0) echo "run_command: missing COMMAND argument" >&2; Exit 99;; + *) run_cmd=$1; shift;; + esac + if test x"$run_mix_stdout_and_stderr" = x"yes"; then + run_evald_cmd='"$run_cmd" ${1+"$@"} >stdall 2>&1' + else + run_evald_cmd='"$run_cmd" ${1+"$@"} >stdout 2>stderr' + fi + # NOTE: do not use the apparently simpler construct: + # if eval "$run_evald_cmd"; then + # run_exitcode_got=0 + # else + # run_exitcode_got=$? + # fi + # as FreeBSD /bin/sh chokes on it when `set -e' is on. + if (eval "exec $run_evald_cmd"); then + run_exitcode_got=0 + else + run_exitcode_got=$? + fi + if test x"$run_mix_stdout_and_stderr" = x"yes"; then + set -x + cat stdall + { set +x; } 2>/dev/null + else + set -x + cat stderr >&2 + cat stdout + { set +x; } 2>/dev/null + fi + case $run_exitcode_expected in + RETURN|return) + restore_shell_flags 'run_command' + return $run_exitcode_got # RETURN POINT HERE + ;; + IGNORE|ignore|IGNORED|ignored|$run_exitcode_got) + run_ok=yes + ;; + FAIL|fail|FAILURE|failure) + if test $run_exitcode_got -gt 0; then + run_ok=yes + else + run_ok=no + fi + ;; + *) + run_ok=no + ;; + esac + echo "run_command: exit status $run_exitcode_got (expecting" \ + "$run_exitcode_expected)" + restore_shell_flags 'run_command' + test x"$run_ok" = x"yes" || Exit 1 +} # AUTOMAKE_run status [options...] # -------------------------------- -# Run Automake with OPTIONS, and fail if automake +# Run Automake with OPTIONS, and cause the test to FAIL if automake # does not exit with STATUS. AUTOMAKE_run () { - expected_exitcode=$1 + am_run_expected_exitcode=$1 shift - exitcode=0 - $AUTOMAKE ${1+"$@"} >stdout 2>stderr || exitcode=$? - cat stderr >&2 - cat stdout - test $exitcode = $expected_exitcode || Exit 1 + run_command -e $am_run_expected_exitcode -- $AUTOMAKE ${1+"$@"} } # AUTOMAKE_fails [options...] # --------------------------- -# Run Automake with OPTIONS, and fail if automake -# does not exit with STATUS. +# Run Automake with OPTIONS, and cuse the test to FAIL if automake +# does not exit with status 1. AUTOMAKE_fails () { AUTOMAKE_run 1 ${1+"$@"} diff --git a/tests/depcomp2.test b/tests/depcomp2.test index 4653c90..0daf2f0 100755 --- a/tests/depcomp2.test +++ b/tests/depcomp2.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2001, 2002 Free Software Foundation, Inc. +# Copyright (C) 2001, 2002, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -20,6 +20,8 @@ required=gcc . ./defs || Exit 1 +set -e + cat > configure.in << 'END' AC_INIT(subdir/foo.c) AM_INIT_AUTOMAKE(nonesuch, nonesuch) @@ -45,8 +47,8 @@ END CFLAGS= export CFLAGS -$ACLOCAL || Exit 1 -$AUTOMAKE --add-missing || Exit 1 -$AUTOCONF || Exit 1 -./configure CC='gcc' 2>error.log -test -z "`cat error.log`" +$ACLOCAL +$AUTOMAKE --add-missing +$AUTOCONF +run_command ./configure CC='gcc' +test ! -s stderr diff --git a/tests/depcomp4.test b/tests/depcomp4.test index 7d22894..70cbd57 100755 --- a/tests/depcomp4.test +++ b/tests/depcomp4.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2003 Free Software Foundation, Inc. +# Copyright (C) 2003, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -51,7 +51,6 @@ $AUTOMAKE -a grep am_cv_CC_dependencies_compiler_type configure ./configure am_cv_CC_dependencies_compiler_type=makedepend -$MAKE >output 2>&1 -cat output -grep 'unknown directive' output && Exit 1 +run_command -m $MAKE +grep 'unknown directive' stdall && Exit 1 $MAKE grepdeps diff --git a/tests/depend5.test b/tests/depend5.test index 0d9be3b..52e4169 100755 --- a/tests/depend5.test +++ b/tests/depend5.test @@ -65,10 +65,7 @@ for arg in Makefile \ "--file=$name" do rm -rf .deps _deps - ./config.status "$arg" depfiles >stdout 2>stderr || - { cat stdout; cat stderr >&2; Exit 1; } - cat stdout - cat stderr >&2 + run_command ./config.status "$arg" depfiles grep '[Nn]o such file' stderr && Exit 1 if test -n "$depdir"; then diff --git a/tests/depend6.test b/tests/depend6.test index de2adf9..5ce2208 100755 --- a/tests/depend6.test +++ b/tests/depend6.test @@ -39,9 +39,6 @@ END $ACLOCAL $AUTOMAKE $AUTOCONF -./configure --disable-dependency-tracking 2>stderr || { - stat=$?; cat stderr >&2; Exit $stat; -} -cat stderr >&2 -grep shift stderr && Exit 1 +run_command ./configure --disable-dependency-tracking +grep 'shift' stderr && Exit 1 : diff --git a/tests/discover.test b/tests/discover.test index 36293f7..c62f1b4 100755 --- a/tests/discover.test +++ b/tests/discover.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1996, 1997, 2000, 2001, 2002 Free Software Foundation, Inc. +# Copyright (C) 1996, 1997, 2000, 2001, 2002, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -34,5 +34,6 @@ END : > fsusage.c $ACLOCAL || Exit 1 -$AUTOMAKE 2>&1 | $FGREP fsusage.c || Exit 1 +run_command -e IGNORE -m $AUTOMAKE +$FGREP fsusage.c stdall || Exit 1 Exit 0 diff --git a/tests/distcom3.test b/tests/distcom3.test index 04bf365..fdcc31a 100755 --- a/tests/distcom3.test +++ b/tests/distcom3.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2001, 2002, 2003, 2004, 2006 Free Software Foundation, Inc. +# Copyright (C) 2001, 2002, 2003, 2004, 2006, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -34,9 +34,8 @@ END $ACLOCAL # Should not warn about missing README, since it is a target. -$AUTOMAKE --add-missing --gnu >stdout 2>&1 -cat stdout -grep README stdout && Exit 1 +run_command -m $AUTOMAKE --add-missing --gnu +grep README stdall && Exit 1 sed -n -e '/^DIST_COMMON =.*\\$/ { :loop diff --git a/tests/fn99.test b/tests/fn99.test index 982a989..611c767 100755 --- a/tests/fn99.test +++ b/tests/fn99.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2004, 2008 Free Software Foundation, Inc. +# Copyright (C) 2004, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -47,8 +47,7 @@ do touch x done) || Exit 77 -$MAKE dist 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_command -e FAIL $MAKE dist grep 'filenames are too long' stderr test 2 = `grep 12345678 stderr | wc -l` : diff --git a/tests/fn99subdir.test b/tests/fn99subdir.test index 465f2af..7055e85 100755 --- a/tests/fn99subdir.test +++ b/tests/fn99subdir.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2006, 2008 Free Software Foundation, Inc. +# Copyright (C) 2006, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -73,8 +73,7 @@ do touch x done) -$MAKE dist 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_command -e FAIL $MAKE dist grep 'filenames are too long' stderr test 1 = `grep 12345678 stderr | wc -l` : diff --git a/tests/gnits2.test b/tests/gnits2.test index 4c5d54f..a85b096 100755 --- a/tests/gnits2.test +++ b/tests/gnits2.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2002, 2003, 2004, 2007 Free Software Foundation, Inc. +# Copyright (C) 2002, 2003, 2004, 2007, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -105,6 +105,6 @@ cd build ../configure "--prefix=`pwd`/../inst-dir" --program-prefix=p $MAKE all $MAKE test-install -$MAKE -k installcheck 2>stderr || : # Never trust the exit status of make -k. -cat stderr +# Never trust the exit status of make -k. +run_command -e IGNORE $MAKE -k installcheck $MAKE grep-stderr diff --git a/tests/gnits3.test b/tests/gnits3.test index 7a08fb8..4d90684 100755 --- a/tests/gnits3.test +++ b/tests/gnits3.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2002, 2007 Free Software Foundation, Inc. +# Copyright (C) 2002, 2007, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -83,8 +83,8 @@ cd build ../configure "--prefix=`pwd`/../inst-dir" --program-prefix=p $MAKE $MAKE install -$MAKE -k installcheck 2>stderr || : # Never trust the exit status of make -k. -cat stderr +# Never trust the exit status of make -k. +run_command -e IGNORE $MAKE -k installcheck $MAKE grep-stderr # Make sure there is no more error when all targets are exempted. diff --git a/tests/help.test b/tests/help.test index d4d7110..cae8967 100755 --- a/tests/help.test +++ b/tests/help.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2003, 2008 Free Software Foundation, Inc. +# Copyright (C) 2003, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -32,7 +32,7 @@ $AUTOMAKE --version $AUTOMAKE --help # aclocal and automake cannot work without configure.ac or configure.in -$ACLOCAL 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e 1 $ACLOCAL grep configure.ac stderr grep configure.in stderr AUTOMAKE_fails diff --git a/tests/init.test b/tests/init.test index 38ec681..b6a7375 100755 --- a/tests/init.test +++ b/tests/init.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2006, 2008 Free Software Foundation, Inc. +# Copyright (C) 2006, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -27,7 +27,17 @@ AM_INIT_AUTOMAKE END # The error message should mension AC_INIT, not AC_PACKAGE_VERSION. -($ACLOCAL && $AUTOCONF) 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +if run_command -e RETURN $ACLOCAL; then + success=yes +else + success=no +fi +if test x"$success" != x"no"; then + mv stderr stder1 + run_command -e RETURN $AUTOCONF || success=no + cat stderr >>stder1 + mv -f stder1 stderr +fi +test x"$success" = x"no" || Exit 1 grep AC_PACKAGE_VERSION stderr && Exit 1 -grep AC_INIT stderr +grep AC_INIT stderr || Exit 1 diff --git a/tests/instsh.test b/tests/instsh.test index 61f5f9e..fe3d485 100755 --- a/tests/instsh.test +++ b/tests/instsh.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1996, 1999, 2001, 2002, 2003 Free Software Foundation, Inc. +# Copyright (C) 1996, 1999, 2001, 2002, 2003, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -32,10 +32,9 @@ mv Makefile.am configure.in frob/ cd frob $ACLOCAL -$AUTOMAKE --add-missing > output 2>&1 +run_command -m $AUTOMAKE --add-missing # Only one `/' should appear in the output. -cat output -grep '/.*/' output && Exit 1 +grep '/.*/' stdall && Exit 1 test -f install-sh diff --git a/tests/ldadd.test b/tests/ldadd.test index 005f9d8..0a9907d 100755 --- a/tests/ldadd.test +++ b/tests/ldadd.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1997, 1999, 2000, 2001, 2002 Free Software Foundation, Inc. +# Copyright (C) 1997, 1999, 2000, 2001, 2002, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -39,5 +39,6 @@ END : > q.c $ACLOCAL || Exit 1 -$AUTOMAKE 2>&1 | grep LDADD || Exit 1 +run_command -e 1 -m $AUTOMAKE +grep LDADD stdall || Exit 1 Exit 0 diff --git a/tests/libtool7.test b/tests/libtool7.test index a2f912f..f8706d9 100755 --- a/tests/libtool7.test +++ b/tests/libtool7.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2003, 2004, 2007 Free Software Foundation, Inc. +# Copyright (C) 2003, 2004, 2007, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -82,19 +82,16 @@ $AUTOCONF $AUTOMAKE --add-missing --copy ./configure "--prefix=`pwd`/_inst" -env LIBTOOLFLAGS=--silent $MAKE print >output 2>&1 -cat output -grep '1BEG: libmod1.la mod2.la :END1' output -grep '2BEG: mod2.la :END2' output -grep '3BEG: .*silent.*silent.* :END3' output +run_command -m env LIBTOOLFLAGS=--silent $MAKE print +grep '1BEG: libmod1.la mod2.la :END1' stdall +grep '2BEG: mod2.la :END2' stdall +grep '3BEG: .*silent.*silent.* :END3' stdall test 2 -le `grep mod2_la_LIBTOOLFLAGS Makefile | wc -l` $MAKE -env LIBTOOLFLAGS=--silent $MAKE install >output 2>&1 -cat output -grep 'silent.*silent.*prg' output -grep 'silent.*silent.*libmod1' output +run_command -m env LIBTOOLFLAGS=--silent $MAKE install +grep 'silent.*silent.*prg' stdall +grep 'silent.*silent.*libmod1' stdall -env LIBTOOLFLAGS=--silent $MAKE uninstall >output 2>&1 -cat output -grep 'silent.*silent.*libmod1' output +run_command -m env LIBTOOLFLAGS=--silent $MAKE uninstall +grep 'silent.*silent.*libmod1' stdall diff --git a/tests/libtool9.test b/tests/libtool9.test index c0d9688..94e617b 100755 --- a/tests/libtool9.test +++ b/tests/libtool9.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2005 Free Software Foundation, Inc. +# Copyright (C) 2005, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -88,10 +88,11 @@ $AUTOCONF $AUTOMAKE --add-missing --copy ./configure -env LDFLAGS=ldflags AM_LDFLAGS=am_ldflags libmod1_la_LDFLAGS=lm1_la_ldflags \ +run_command -m env \ + LDFLAGS=ldflags AM_LDFLAGS=am_ldflags libmod1_la_LDFLAGS=lm1_la_ldflags \ CFLAGS=cflags AM_CFLAGS=am_cflags prg2_CFLAGS=prg2_cflags \ - $MAKE -e print >output 2>&1 -cat output + $MAKE -e print +mv stdall output grep '1BEG: libmod1.la mod2.la :END1' output grep '2BEG: mod2.la :END2' output grep '3BEG:.* am_cflags cflags .*lm1_la_ldflags ldflags.* :END3' output diff --git a/tests/lisp3.test b/tests/lisp3.test index 809d491..086173d 100755 --- a/tests/lisp3.test +++ b/tests/lisp3.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2003, 2008 Free Software Foundation, Inc. +# Copyright (C) 2003, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -61,9 +61,8 @@ test -f am-one.elc unique=0a3346e2af8a689b85002b53df09142a $sleep echo "(message \"$unique\")(provide 'am-three)" > am-three.el -$MAKE >output 2>&1 -cat output -grep $unique output +run_command -m $MAKE +grep $unique stdall # It should also work for VPATH-builds. $MAKE distcheck diff --git a/tests/ltorder.test b/tests/ltorder.test index 0f69211..1e6a03a 100755 --- a/tests/ltorder.test +++ b/tests/ltorder.test @@ -1,5 +1,5 @@ #!/bin/sh -# Copyright (C) 2008 Free Software Foundation, Inc. +# Copyright (C) 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -53,8 +53,7 @@ $AUTOMAKE --add-missing ./configure "--prefix=`pwd`/inst" $MAKE -$MAKE install 2>stderr || { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_command $MAKE install grep 'has not been installed' stderr && Exit 1 $MAKE uninstall diff --git a/tests/man4.test b/tests/man4.test index aa86a9e..7424c72 100755 --- a/tests/man4.test +++ b/tests/man4.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2008 Free Software Foundation, Inc. +# Copyright (C) 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -88,11 +88,9 @@ mkdir build cd build ../configure $MAKE -$MAKE dist 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_command -e FAIL $MAKE dist grep 'install help2man' stderr -$MAKE distcheck 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_command -e FAIL $MAKE distcheck grep 'install help2man' stderr rm -f ../help2man diff --git a/tests/mdate.test b/tests/mdate.test index 704b393..e63bff9 100755 --- a/tests/mdate.test +++ b/tests/mdate.test @@ -1,5 +1,5 @@ #!/bin/sh -# Copyright (C) 1996, 1997, 2001, 2002 Free Software Foundation, Inc. +# Copyright (C) 1996, 1997, 2001, 2002, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -31,4 +31,5 @@ END : > texinfo.tex $ACLOCAL || Exit 1 -$AUTOMAKE 2>&1 > /dev/null | grep 'required file.*mdate-sh' +run_command -e IGNORED $AUTOMAKE +grep 'required file.*mdate-sh' stderr diff --git a/tests/missing3.test b/tests/missing3.test index 07ef779..9b48368 100755 --- a/tests/missing3.test +++ b/tests/missing3.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2004, 2008 Free Software Foundation, Inc. +# Copyright (C) 2004, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -22,24 +22,24 @@ set -e # b7cb8259 assumed not to exist. -./missing b7cb8259 --version 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e FAIL ./missing b7cb8259 --version grep . stderr && Exit 1 -./missing b7cb8259 --grep 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e FAIL ./missing b7cb8259 --grep grep WARNING stderr -./missing --run b7cb8259 --version && Exit 1 -./missing --run b7cb8259 --grep 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e FAIL ./missing --run b7cb8259 --version +run_command -e FAIL ./missing --run b7cb8259 --grep grep WARNING stderr # missing itself it known to exist :) -./missing ./missing --version 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e FAIL ./missing ./missing --version grep . stderr && Exit 1 -./missing ./missing --grep 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e FAIL ./missing ./missing --grep grep WARNING stderr -./missing --run ./missing --version 2>stderr +run_command ./missing --run ./missing --version grep . stderr && Exit 1 -./missing --run ./missing --grep 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e FAIL ./missing --run ./missing --grep grep WARNING stderr && Exit 1 grep Unknown stderr diff --git a/tests/missing4.test b/tests/missing4.test index 1015828..c6c3b7e 100755 --- a/tests/missing4.test +++ b/tests/missing4.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2006, 2007, 2008 Free Software Foundation, Inc. +# Copyright (C) 2006, 2007, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -36,13 +36,11 @@ cmp aclocal.m4 aclocal.tmp && Exit 1 mv aclocal.tmp aclocal.m4 -$MAKE 2>stderr -cat stderr +run_command $MAKE grep 'You have another version of autoconf' stderr grep -c 'aclocal.m4:.*this file was generated for' stderr -$MAKE 2>stderr -cat stderr +run_command $MAKE grep 'You have another version of autoconf' stderr && Exit 1 grep 'aclocal.m4:.*this file was generated for' stderr && Exit 1 diff --git a/tests/missing6.test b/tests/missing6.test index b63a0d1..bf6df2d 100755 --- a/tests/missing6.test +++ b/tests/missing6.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2008 Free Software Foundation, Inc. +# Copyright (C) 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -31,7 +31,7 @@ rm -f configure.in touch Makefile.am $ACLOCAL -$AUTOCONF 2>stderr +run_command $AUTOCONF grep 'You have another version of autoconf' stderr grep 'aclocal.m4:.*this file was generated for' stderr $AUTOMAKE @@ -43,8 +43,7 @@ cmp configure.ac configure.tmp && Exit 1 mv configure.tmp configure.ac -$MAKE 2>stderr -cat stderr +run_command $MAKE grep 'You have another version of autoconf' stderr grep 'aclocal.m4:.*this file was generated for autoconf 9999a' stderr diff --git a/tests/obsolete.test b/tests/obsolete.test index e6601c9..daae69c 100755 --- a/tests/obsolete.test +++ b/tests/obsolete.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1996, 2001, 2002, 2007, 2008 Free Software Foundation, Inc. +# Copyright (C) 1996, 2001, 2002, 2007, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -27,7 +27,7 @@ END $PERL -ne '/AU_DEFUN\(\[(\w+)\]/ && print "$1\n"' \ "$testsrcdir/../m4/obsolete.m4" >> obs cat obs >> configure.in -$PERL -ne 'chomp; print "grep $_ stderr || Exit 1\n"; ' obs > obs.1 +$PERL -ne 'chomp; print "grep $_ stdall || Exit 1\n"; ' obs > obs.1 $PERL -ne 'chomp; print "grep $_ configure.in && Exit 1\n"; ' obs > obs.2 # Sanity check. Make sure we have added something to configure.in. @@ -36,7 +36,7 @@ test `cat configure.in | wc -l` -gt 1 || Exit 1 $ACLOCAL || Exit 1 # Expect Autoconf to complain about each of the macros in obs. -$AUTOCONF -Wobsolete >stderr 2>&1 +run_command -m $AUTOCONF -Wobsolete . ./obs.1 # Make sure Autoupdate remove each of these macros. $AUTOUPDATE || Exit 1 diff --git a/tests/output-order.test b/tests/output-order.test index 856baa2..852c194 100755 --- a/tests/output-order.test +++ b/tests/output-order.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2008 Free Software Foundation, Inc. +# Copyright (C) 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -43,13 +43,12 @@ END $ACLOCAL $AUTOCONF rm -f missing install-sh -$AUTOMAKE --add-missing --copy 2>stderr -cat stderr >&2 +run_command $AUTOMAKE --add-missing --copy +mv stderr stder0 for i in 1 2 3 4 5 6; do rm -f missing install-sh INSTALL COPYING + run_command -e IGNORE $AUTOMAKE --add-missing --copy # The grep prevents a Heisenbug with the HP-UX shell and VERBOSE=yes. - $AUTOMAKE --add-missing --copy 2>&1 >/dev/null | - grep -v /dev/null | - diff - stderr + grep -v /dev/null stder0 | diff - stderr done diff --git a/tests/pr220.test b/tests/pr220.test index 25e1f0e..6644d28 100755 --- a/tests/pr220.test +++ b/tests/pr220.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2001, 2002, 2004, 2008 Free Software Foundation, Inc. +# Copyright (C) 2001, 2002, 2004, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -55,6 +55,5 @@ $AUTOMAKE -a cd build # configure should fail since we've done something invalid. -../configure 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_command -e FAIL ../configure grep NEVER_TRUE stderr diff --git a/tests/pr300-ltlib.test b/tests/pr300-ltlib.test index 01814a9..7ed0a2f 100755 --- a/tests/pr300-ltlib.test +++ b/tests/pr300-ltlib.test @@ -48,8 +48,7 @@ $ACLOCAL $AUTOCONF $AUTOMAKE --copy --add-missing ./configure --prefix "`pwd`/inst" -$MAKE V=1 >stdout -cat stdout +run_command $MAKE V=1 grep 'liba.la .*-rpath .*lib' stdout grep 'liba.la .*-rpath .*lib/subdir' stdout && Exit 1 @@ -58,8 +57,7 @@ grep 'libb.la .*-rpath .*lib/subdir' stdout test -f subdir/liba.la test -f subdir/libb.la -$MAKE install 2>stderr -cat stderr >&2 +run_command $MAKE install grep 'remember.*--finish' stderr && Exit 1 test -f inst/lib/liba.la diff --git a/tests/python11.test b/tests/python11.test index 0f15f37..920de88 100755 --- a/tests/python11.test +++ b/tests/python11.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2004, 2006, 2008 Free Software Foundation, Inc. +# Copyright (C) 2004, 2006, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -35,9 +35,7 @@ EOF $ACLOCAL $AUTOCONF -./configure >stdout 2>stderr && { cat stdout; cat stderr >&2; Exit 1; } -cat stdout -cat stderr >&2 +run_command -e FAILURE ./configure grep 'checking for IShouldNotExist1' stdout grep 'checking for IShouldNotExist2' stdout grep 'no suitable Python interpreter found' stderr diff --git a/tests/python4.test b/tests/python4.test index 11bb3ed..3ff44af 100755 --- a/tests/python4.test +++ b/tests/python4.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2003, 2008 Free Software Foundation, Inc. +# Copyright (C) 2003, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -33,6 +33,5 @@ $AUTOCONF $AUTOMAKE --add-missing # Simulate no Python -./configure PYTHON=: 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_command -e FAILURE ./configure PYTHON=: grep 'no suitable Python interpreter found' stderr diff --git a/tests/python5.test b/tests/python5.test index 86f5863..5695876 100755 --- a/tests/python5.test +++ b/tests/python5.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2003, 2008 Free Software Foundation, Inc. +# Copyright (C) 2003, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -34,6 +34,5 @@ $ACLOCAL $AUTOCONF $AUTOMAKE --add-missing -./configure 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_command -e FAILURE ./configure grep 'no suitable Python interpreter found' stderr diff --git a/tests/python6.test b/tests/python6.test index eb4fb4d..ed60e03 100755 --- a/tests/python6.test +++ b/tests/python6.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2003 Free Software Foundation, Inc. +# Copyright (C) 2003, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -34,6 +34,5 @@ $AUTOCONF $AUTOMAKE --add-missing # Simulate no Python -./configure PYTHON=: 2>stderr -cat stderr +run_command ./configure PYTHON=: grep 'GREP ME:' stderr diff --git a/tests/python7.test b/tests/python7.test index 62c1a1b..eb6a4dc 100755 --- a/tests/python7.test +++ b/tests/python7.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2003 Free Software Foundation, Inc. +# Copyright (C) 2003, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -34,6 +34,5 @@ $ACLOCAL $AUTOCONF $AUTOMAKE --add-missing -./configure 2>stderr -cat stderr +run_command ./configure grep 'GREP ME:' stderr diff --git a/tests/python8.test b/tests/python8.test index a50af22..5f100ab 100755 --- a/tests/python8.test +++ b/tests/python8.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2003 Free Software Foundation, Inc. +# Copyright (C) 2003, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -32,6 +32,5 @@ $ACLOCAL $AUTOCONF $AUTOMAKE --add-missing -./configure 2>stderr -cat stderr +run_command ./configure grep 'GREP ME' stderr diff --git a/tests/python9.test b/tests/python9.test index 9b23deb..c0fa7a5 100755 --- a/tests/python9.test +++ b/tests/python9.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2003 Free Software Foundation, Inc. +# Copyright (C) 2003, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -33,6 +33,5 @@ $ACLOCAL $AUTOCONF $AUTOMAKE --add-missing -./configure 2>stderr -cat stderr +run_command ./configure grep 'GREP ME' stderr diff --git a/tests/sanity.test b/tests/sanity.test index 1c96c78..1296381 100755 --- a/tests/sanity.test +++ b/tests/sanity.test @@ -36,14 +36,12 @@ cp ../install-sh ../missing . $ACLOCAL $AUTOCONF $AUTOMAKE -./configure 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr +run_command -e FAILURE ./configure grep 'unsafe absolute working directory' stderr cd .. mkdir build cd build -../unsafe$/configure 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr +run_command -e FAILURE ../unsafe$/configure grep 'unsafe srcdir' stderr : diff --git a/tests/subobj.test b/tests/subobj.test index df297f4..9d7882d 100755 --- a/tests/subobj.test +++ b/tests/subobj.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1999, 2001, 2002, 2004 Free Software Foundation, Inc. +# Copyright (C) 1999, 2001, 2002, 2004, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -32,8 +32,7 @@ wish_SOURCES = generic/a.c generic/b.c END $ACLOCAL -$AUTOMAKE --add-missing 2>stderr -cat stderr +run_command $AUTOMAKE --add-missing # Make sure compile is installed, and that Automake says so. grep 'install.*compile' stderr test -f compile diff --git a/tests/subobj9.test b/tests/subobj9.test index 99fe6e8..1f68721 100755 --- a/tests/subobj9.test +++ b/tests/subobj9.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2002, 2004, 2005 Free Software Foundation, Inc. +# Copyright (C) 2002, 2004, 2005, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -74,8 +74,8 @@ if test -n "`./libtool --help | grep tag=TAG`"; then fi $MAKE -$MAKE distcheck 2>&1 | tee out +run_command -m $MAKE distcheck # GNU Make used to complain that the Makefile contained two rules # for `src/.dirstamp' and `.//src/.dirstamp'. -grep 'overriding commands' out && Exit 1 +grep 'overriding commands' stdall && Exit 1 : diff --git a/tests/unused.test b/tests/unused.test index a5b4da6..68d5613 100755 --- a/tests/unused.test +++ b/tests/unused.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1998, 2002, 2003, 2008 Free Software Foundation, Inc. +# Copyright (C) 1998, 2002, 2003, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -29,5 +29,5 @@ AC_DEFUN([MACRO_1_2], echo 12) AC_DEFUN([MACRO_1_2_3], echo 123) END -$ACLOCAL 2> stderr || { cat stderr >&2; Exit 1; } -test -z "`cat stderr`" +run_command $ACLOCAL +test ! -s stderr diff --git a/tests/vars3.test b/tests/vars3.test index 3d10a90..9f11262 100755 --- a/tests/vars3.test +++ b/tests/vars3.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2002 Free Software Foundation, Inc. +# Copyright (C) 2002, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -38,8 +38,7 @@ EOF $ACLOCAL # Make sure this warning is print in the `portability' category. -$AUTOMAKE --warnings=no-error,none,portability 2>stderr -cat stderr +run_command $AUTOMAKE --warnings=no-error,none,portability # Lines number are printed in error message. # Use them to make sure errors are diagnosed against the right lines. diff --git a/tests/version8.test b/tests/version8.test index 91b11a0..ffe4a3e 100755 --- a/tests/version8.test +++ b/tests/version8.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2005, 2008 Free Software Foundation, Inc. +# Copyright (C) 2005, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -20,6 +20,5 @@ set -e echo 'AM_AUTOMAKE_VERSION([1.9])' >>configure.in -$ACLOCAL 2>stderr && { cat stderr >&2; Exit 0; } -cat stderr >&2 +run_command -e 1 $ACLOCAL $FGREP 'AM_INIT_AUTOMAKE([1.9])' stderr -- 1.6.3.3 |
|
|
Re: testsuite failures when test scripts are run with zsh
by Stefano Lattarini
::
Rate this Message:
Aaargh! There is a bug (another), with heirloom-sh this time.
Apparently, shell traces are not enabled/disabled in run_command() the way they should. Please do not apply the patch until I'll had figured out and fixed the problem (I've no time right now, maybe tomorrow I can do something). Regards, Stefano |
|
|
Re: testsuite failures when test scripts are run with zsh
by Stefano Lattarini
::
Rate this Message:
Stefano Lattarini wrote:
> Aaargh! There is a bug (another), with heirloom-sh this time. > Apparently, shell traces are not enabled/disabled in run_command() > the way they should. > Please do not apply the patch until I'll had figured out and fixed the > problem (I've no time right now, maybe tomorrow I can do something). That was an easy fix. Apparently, heirloom-sh executes a construct like `{ set +x; } 2>/dev/null' in a subshell, and this of course defies the purpose of the `set +x' directive (which ought to disable shell tracing in the *current* shell). The easy fix is to repeat `set +x' without redirection; this way, heirloom-sh works correctly (even if it displays an extra `+ set +x' on stderr, but this is not a big deal IMHO), while more decent shells continue to have the expected verbosity, and to correctly disable traces. The updated patch is attached. Regards, Stefano From 4eca77011ccaa38d1491c86d8cfd368d0f851b9c Mon Sep 17 00:00:00 2001 From: Stefano Lattarini <stefano.lattarini@...> Date: Fri, 23 Oct 2009 18:27:55 +0200 Subject: [PATCH] Testsuite: avoid Zsh-related problem with `set -x'. * tests/README: Describe Zsh 4.x `set -x' a.k.a. `xtrace' issue and workaround with run_command. * tests/defs.in (run_command): New function, to be used for commands whose standard error needs to be captured. (save_shell_flags, restore_shell_flags): New functions, used in run_command. (AUTOMAKE_run): Reimplemented using run_command. Updated comment. (AUTOMAKE_fails): Updated comment. * tests/acloca14.test: Use new subroutine run_command instead of hand-crafted redirections of stdout and/or stderr. * tests/acloca17.test: Likewise. * tests/acloca18.test: Likewise. * tests/aclocal.test: Likewise. * tests/aclocal3.test: Likewise. * tests/aclocal8.test: Likewise. * tests/acsilent.test: Likewise. * tests/alpha.test: Likewise. * tests/ammissing.test: Likewise. * tests/ansi3.test: Likewise. * tests/ansi3b.test: Likewise. * tests/ccnoco3.test: Likewise. * tests/check8.test: Likewise. * tests/cond39.test: Likewise. * tests/depcomp2.test: Likewise. * tests/depcomp4.test: Likewise. * tests/depend5.test: Likewise. * tests/depend6.test: Likewise. * tests/discover.test: Likewise. * tests/distcom3.test: Likewise. * tests/fn99.test: Likewise. * tests/fn99subdir.test: Likewise. * tests/gnits2.test: Likewise. * tests/gnits3.test: Likewise. * tests/help.test: Likewise. * tests/init.test: Likewise. * tests/instsh.test: Likewise. * tests/ldadd.test: Likewise. * tests/libtool7.test: Likewise. * tests/libtool9.test: Likewise. * tests/lisp3.test: Likewise. * tests/ltorder.test: Likewise. * tests/man4.test: Likewise. * tests/mdate.test: Likewise. * tests/missing3.test: Likewise. * tests/missing4.test: Likewise. * tests/missing6.test: Likewise. * tests/obsolete.test: Likewise. * tests/output-order.test: Likewise. * tests/pr220.test: Likewise. * tests/pr300-ltlib.test: Likewise. * tests/python11.test: Likewise. * tests/python4.test: Likewise. * tests/python5.test: Likewise. * tests/python6.test: Likewise. * tests/python7.test: Likewise. * tests/python8.test: Likewise. * tests/python9.test: Likewise. * tests/sanity.test: Likewise. * tests/subobj.test: Likewise. * tests/subobj9.test: Likewise. * tests/unused.test: Likewise. * tests/vars3.test: Likewise. * tests/version8.test: Likewise. --- ChangeLog | 67 ++++++++++++++++++++++++++ tests/README | 10 ++++ tests/acloca14.test | 4 +- tests/acloca17.test | 6 +-- tests/acloca18.test | 10 ++-- tests/aclocal.test | 8 ++-- tests/aclocal3.test | 5 +- tests/aclocal8.test | 7 +-- tests/acsilent.test | 7 ++- tests/alpha.test | 5 +- tests/ammissing.test | 5 +- tests/ansi3.test | 5 +- tests/ansi3b.test | 5 +- tests/ccnoco3.test | 3 +- tests/check8.test | 8 +-- tests/cond39.test | 8 +-- tests/defs.in | 119 +++++++++++++++++++++++++++++++++++++++++++---- tests/depcomp2.test | 14 +++-- tests/depcomp4.test | 7 +-- tests/depend5.test | 5 +-- tests/depend6.test | 7 +-- tests/discover.test | 5 +- tests/distcom3.test | 7 +-- tests/fn99.test | 5 +- tests/fn99subdir.test | 5 +- tests/gnits2.test | 6 +- tests/gnits3.test | 6 +- tests/help.test | 4 +- tests/init.test | 18 ++++++-- tests/instsh.test | 7 +-- tests/ldadd.test | 5 +- tests/libtool7.test | 23 ++++----- tests/libtool9.test | 9 ++-- tests/lisp3.test | 7 +-- tests/ltorder.test | 5 +- tests/man4.test | 8 +-- tests/mdate.test | 5 +- tests/missing3.test | 18 ++++---- tests/missing4.test | 8 +-- tests/missing6.test | 7 +-- tests/obsolete.test | 6 +- tests/output-order.test | 11 ++-- tests/pr220.test | 5 +- tests/pr300-ltlib.test | 6 +-- tests/python11.test | 6 +-- tests/python4.test | 5 +- tests/python5.test | 5 +- tests/python6.test | 5 +- tests/python7.test | 5 +- tests/python8.test | 5 +- tests/python9.test | 5 +- tests/sanity.test | 6 +-- tests/subobj.test | 5 +- tests/subobj9.test | 6 +- tests/unused.test | 6 +- tests/vars3.test | 5 +- tests/version8.test | 5 +- 57 files changed, 355 insertions(+), 215 deletions(-) diff --git a/ChangeLog b/ChangeLog index 2fe0566..8300a5b 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,70 @@ +2009-10-23 Stefano Lattarini <stefano.lattarini@...> + + Testsuite: avoid Zsh-related problem with `set -x'. + * tests/README: Describe Zsh 4.x `set -x' a.k.a. `xtrace' issue + and workaround with run_command. + * tests/defs.in (run_command): New function, to be used for + commands whose standard error needs to be captured. + (save_shell_flags, restore_shell_flags): New functions, used in + run_command. + (AUTOMAKE_run): Reimplemented using run_command. Updated comment. + (AUTOMAKE_fails): Updated comment. + * tests/acloca14.test: Use new subroutine run_command instead + of hand-crafted redirections of stdout and/or stderr. + * tests/acloca17.test: Likewise. + * tests/acloca18.test: Likewise. + * tests/aclocal.test: Likewise. + * tests/aclocal3.test: Likewise. + * tests/aclocal8.test: Likewise. + * tests/acsilent.test: Likewise. + * tests/alpha.test: Likewise. + * tests/ammissing.test: Likewise. + * tests/ansi3.test: Likewise. + * tests/ansi3b.test: Likewise. + * tests/ccnoco3.test: Likewise. + * tests/check8.test: Likewise. + * tests/cond39.test: Likewise. + * tests/depcomp2.test: Likewise. + * tests/depcomp4.test: Likewise. + * tests/depend5.test: Likewise. + * tests/depend6.test: Likewise. + * tests/discover.test: Likewise. + * tests/distcom3.test: Likewise. + * tests/fn99.test: Likewise. + * tests/fn99subdir.test: Likewise. + * tests/gnits2.test: Likewise. + * tests/gnits3.test: Likewise. + * tests/help.test: Likewise. + * tests/init.test: Likewise. + * tests/instsh.test: Likewise. + * tests/ldadd.test: Likewise. + * tests/libtool7.test: Likewise. + * tests/libtool9.test: Likewise. + * tests/lisp3.test: Likewise. + * tests/ltorder.test: Likewise. + * tests/man4.test: Likewise. + * tests/mdate.test: Likewise. + * tests/missing3.test: Likewise. + * tests/missing4.test: Likewise. + * tests/missing6.test: Likewise. + * tests/obsolete.test: Likewise. + * tests/output-order.test: Likewise. + * tests/pr220.test: Likewise. + * tests/pr300-ltlib.test: Likewise. + * tests/python11.test: Likewise. + * tests/python4.test: Likewise. + * tests/python5.test: Likewise. + * tests/python6.test: Likewise. + * tests/python7.test: Likewise. + * tests/python8.test: Likewise. + * tests/python9.test: Likewise. + * tests/sanity.test: Likewise. + * tests/subobj.test: Likewise. + * tests/subobj9.test: Likewise. + * tests/unused.test: Likewise. + * tests/vars3.test: Likewise. + * tests/version8.test: Likewise. + 2009-10-18 Ralf Wildenhues <Ralf.Wildenhues@...> Simplify Variable::_check_ambiguous_condition. diff --git a/tests/README b/tests/README index d494b10..0b26211 100644 --- a/tests/README +++ b/tests/README @@ -133,6 +133,16 @@ Do Use $sleep when you have to make sure that some file is newer than another. + To run a program and analyze its stderr, use the run_command function. + For example: + run_command PROG [ARGS...] + grep $pattern stderr + or (if you must check that nothing is written on stderr): + run_command PROG2 [ARGS...] + test ! -s stderr + See section "File Descriptors" in the Autoconf manual to understand why + this can be useful, or even required. + Use `cat' or `grep' to display (part of) files that may be interesting for debugging, so that when a user send a verbose output we don't have to ask him for more details. Display stderr diff --git a/tests/acloca14.test b/tests/acloca14.test index 8ddbb1a..cf9c484 100755 --- a/tests/acloca14.test +++ b/tests/acloca14.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2004, 2008 Free Software Foundation, Inc. +# Copyright (C) 2004, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -108,5 +108,5 @@ $MAKE testdist2 # Make sure aclocal diagnose missing included files with correct `file:line:'. rm -f b.m4 -$ACLOCAL 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e 1 $ACLOCAL grep 'a.m4:1:.*b.m4.*does not exist' stderr diff --git a/tests/acloca17.test b/tests/acloca17.test index 884f24b..788cb9d 100755 --- a/tests/acloca17.test +++ b/tests/acloca17.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2004, 2008 Free Software Foundation, Inc. +# Copyright (C) 2004, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -33,7 +33,5 @@ EOF # FIXME: We want autom4te's 'undefined required macro' warning to be fatal, # but have no means to say so to aclocal. We use WARNINGS=error instead. - -WARNINGS=error $ACLOCAL -I m4 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_command -e 1 env WARNINGS=error $ACLOCAL -I m4 grep 'configure.in:4:.*UNDEFINED_MACRO' stderr diff --git a/tests/acloca18.test b/tests/acloca18.test index 22feb23..7047eea 100755 --- a/tests/acloca18.test +++ b/tests/acloca18.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2005, 2006, 2008 Free Software Foundation, Inc. +# Copyright (C) 2005, 2006, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -86,7 +86,7 @@ grep macro23 foo ACLOCAL_TESTSUITE_FLAGS='-I 4 -I 1 -I 2' rm -f foo -$ACLOCAL --install 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e 1 $ACLOCAL --install grep AM_MACRO2 stderr ACLOCAL_TESTSUITE_FLAGS='-I 4 -I 1' @@ -105,9 +105,7 @@ AC_DEFUN([AM_MACRO1], [echo macro1d >> foo]) AC_DEFUN([AM_MACRO2], [echo macro2d >> foo]) EOF rm -f foo -$ACLOCAL --diff=diff >output 2>stderr -cat stderr -cat output -grep '#serial 456' output +run_command $ACLOCAL --diff=diff +grep '#serial 456' stdout test ! -f 4/m1.m4 grep 'installing.*4/m1.m4' stderr diff --git a/tests/aclocal.test b/tests/aclocal.test index 6b87a9d..945b363 100755 --- a/tests/aclocal.test +++ b/tests/aclocal.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1998, 2001, 2002, 2004, 2008 Free Software Foundation, Inc. +# Copyright (C) 1998, 2001, 2002, 2004, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -23,15 +23,15 @@ set -e $ACLOCAL --output=fred test -f fred -$ACLOCAL --output 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e 1 $ACLOCAL --output grep 'option.*--output.*an argument' stderr grep help stderr -$ACLOCAL --unknown-option 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e 1 $ACLOCAL --unknown-option grep 'unrecognized.*--unknown-option' stderr grep help stderr -$ACLOCAL --ver 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e 1 $ACLOCAL --ver grep 'unrecognized.*--ver' stderr grep help stderr diff --git a/tests/aclocal3.test b/tests/aclocal3.test index a550e50..34c5335 100755 --- a/tests/aclocal3.test +++ b/tests/aclocal3.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1998, 2002, 2004 Free Software Foundation, Inc. +# Copyright (C) 1998, 2002, 2004, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -27,6 +27,5 @@ AC_DEFUN([GNOME_X_CHECKS], [ ]) END -$ACLOCAL -I macros 2>stderr -cat stderr +run_command -e 1 $ACLOCAL -I macros grep 'macros/gnome.m4:2:.*AM_PATH_GTK.*not found' stderr diff --git a/tests/aclocal8.test b/tests/aclocal8.test index ac10aff..4cf7d73 100755 --- a/tests/aclocal8.test +++ b/tests/aclocal8.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2003, 2004 Free Software Foundation, Inc. +# Copyright (C) 2003, 2004, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -34,9 +34,8 @@ EOF echo 'AC_DEFUN([MACRO1],)' >m4/macro1.m4 echo 'AC_DEFUN([MACRO2], [AC_REQUIRE([AM_UNUSED_MACRO])])' >m4/macro2.m4 -$ACLOCAL -I m4 >output 2>&1 -cat output -test 0 = `wc -l <output` +run_command -m $ACLOCAL -I m4 +test -s stdall && Exit 1 grep macro1.m4 aclocal.m4 grep macro2.m4 aclocal.m4 && Exit 1 : diff --git a/tests/acsilent.test b/tests/acsilent.test index 6ac7863..e250092 100755 --- a/tests/acsilent.test +++ b/tests/acsilent.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1996, 2001, 2002, 2008 Free Software Foundation, Inc. +# Copyright (C) 1996, 2001, 2002, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -32,5 +32,6 @@ module=[$1] AC_SUBST(module)]) END -$ACLOCAL > output 2>&1 || { cat output; Exit 1; } -test -z "`cat output`" +run_command -m $ACLOCAL +test -s stdall && Exit 1 +: diff --git a/tests/alpha.test b/tests/alpha.test index 1ee8ed6..1410b9d 100755 --- a/tests/alpha.test +++ b/tests/alpha.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1998, 2001, 2002, 2003, 2008 Free Software Foundation, Inc. +# Copyright (C) 1998, 2001, 2002, 2003, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -62,8 +62,7 @@ $AUTOMAKE ./configure # make distdir should fail because NEWS does not mention 1.0a -$MAKE check 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_command -e FAIL $MAKE check grep 'NEWS not updated' stderr test ! -f works diff --git a/tests/ammissing.test b/tests/ammissing.test index 61d360a..42aa481 100755 --- a/tests/ammissing.test +++ b/tests/ammissing.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1997, 2002, 2004 Free Software Foundation, Inc. +# Copyright (C) 1997, 2002, 2004, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -20,6 +20,5 @@ echo AM_ZARDOZ >> configure.in -$ACLOCAL 2>stderr -cat stderr +run_command -e 1 $ACLOCAL grep 'configure.in:.*AM_ZARDOZ.*not found' stderr diff --git a/tests/ansi3.test b/tests/ansi3.test index 2bbcaf7..95e4fc1 100755 --- a/tests/ansi3.test +++ b/tests/ansi3.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1998, 1999, 2001, 2002, 2003 Free Software Foundation, Inc. +# Copyright (C) 1998, 1999, 2001, 2002, 2003, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -53,8 +53,7 @@ export CFLAGS # Linux boxes (Red Hat 5.1 in particular). $ACLOCAL $AUTOCONF -$AUTOMAKE -a 2>stderr -cat stderr +run_command $AUTOMAKE -a grep 'Makefile.am:1:.*ansi2knr\.c' stderr CC='gcc' ./configure ANSI2KNR=./ansi2knr U=_ $MAKE -e diff --git a/tests/ansi3b.test b/tests/ansi3b.test index 8f254ad..447b690 100755 --- a/tests/ansi3b.test +++ b/tests/ansi3b.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1998, 1999, 2001, 2002, 2003 Free Software Foundation, Inc. +# Copyright (C) 1998, 1999, 2001, 2002, 2003, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -52,8 +52,7 @@ export CFLAGS # Linux boxes (Red Hat 5.1 in particular). $ACLOCAL $AUTOCONF -$AUTOMAKE -a 2>stderr -cat stderr +run_command $AUTOMAKE -a grep 'configure.in:2:.*ansi2knr\.1' stderr CC='gcc' ./configure ANSI2KNR=./ansi2knr U=_ $MAKE -e diff --git a/tests/ccnoco3.test b/tests/ccnoco3.test index 2e9cd37..93731a1 100755 --- a/tests/ccnoco3.test +++ b/tests/ccnoco3.test @@ -74,8 +74,7 @@ CC=`pwd`/../Mycomp export CC ../configure -$MAKE 2>stderr || { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_command $MAKE grep 'mv.*the same file' stderr && Exit 1 Exit 0 diff --git a/tests/check8.test b/tests/check8.test index 7936582..a087cb7 100755 --- a/tests/check8.test +++ b/tests/check8.test @@ -67,10 +67,7 @@ $AUTOMAKE -a unset TESTS || : ./configure -AM_COLOR_TESTS=always $MAKE -e check >stdout 2>stderr && - { cat stdout; cat stderr >&2; Exit 1; } -cat stdout -cat stderr >&2 +run_command -e FAIL -- env AM_COLOR_TESTS=always $MAKE -e check grep 'XPASS.* foo$' stdout grep '^[^X]*PASS.* sub/foo$' stdout grep '^[^X]*PASS.* bar' stdout @@ -87,8 +84,7 @@ $MAKE distclean mkdir build cd build ../configure -$MAKE check >stdout && { cat stdout; Exit 1; } -cat stdout +run_command -e FAIL $MAKE check # Note: we are not grepping for the space here, due to the Solaris make VPATH # rewriting (if we fix that, we can still write a separate test for it). grep 'XPASS.*foo$' stdout diff --git a/tests/cond39.test b/tests/cond39.test index 79eaa70..1e44866 100755 --- a/tests/cond39.test +++ b/tests/cond39.test @@ -1,5 +1,5 @@ #!/bin/sh -# Copyright (C) 2008 Free Software Foundation, Inc. +# Copyright (C) 2008, 2009 Free Software Foundation, Inc. # # This file is part of GNU Automake. # @@ -82,8 +82,7 @@ $AUTOCONF $AUTOMAKE --add-missing ./configure COND=true -$MAKE 2>stderr -cat stderr +run_command $MAKE grep 'overriding commands' stderr && Exit 1 $MAKE sure-exist ./prog1 && Exit 1 @@ -97,8 +96,7 @@ $MAKE sure-exist $MAKE distclean ./configure COND=false -$MAKE 2>stderr -cat stderr +run_command $MAKE grep 'overriding commands' stderr && Exit 1 ./prog1 ./sub/prog2 diff --git a/tests/defs.in b/tests/defs.in index 6eec344..530f1e0 100644 --- a/tests/defs.in +++ b/tests/defs.in @@ -399,26 +399,127 @@ is_newest () test -z "$is_newest_files" } +# save_shell_flags [VAR-BASENAME] +# ------------------------------- +# Save current shell flags in variable VAR_saved_shell_flags +# (default: _saved_shell_flags) +save_shell_flags () +{ + eval "${1}_saved_shell_flags=\$-"; +} + +# restore_shell_flags [VAR] +# ------------------------- +# Restore shell flags saved in variable VAR_saved_shell_flags +# (default: _saved_shell_flags) +restore_shell_flags () +{ + eval set -\${"${1}_saved_shell_flags"} +} + +# run_command [-e STATUS] [-m] [--] COMMAND [ARGUMENTS..] +# ------------------------------------------------------- +# Run the given COMMAND with ARGUMENTS, and cause the test to FAIL if +# COMMAND does not exit with STATUS (more details are given below). +# Also, save standard output and standard error from COMMAND, by default +# respectively in files `stdout' and `stderr' (in the current directory), +# or together in the file `stdall' (in the current directory) if the `-m' +# option is given. +# If STATUS is numeric, the exit value of COMMAND must match it exactly. +# If STATUS is "FAIL" or "FAILURE", then any exit value of COMMAND *but 0* +# is acceptable. If STATUS is "IGNORE" or "IGNORED", any exit value of +# COMMAND is acceptable, and run_command returns with success regardless +# of that value. Finally, if STATUS is "RETURN", any exit value of COMMAND +# is acceptable, *and* such exit value is returned by run_command. +# Default STATUS is `0'. +run_command () +{ + save_shell_flags 'run_command' + set +x # xtrace verbosity temporarly disabled in this function + run_exitcode_expected=0 + run_mix_stdout_and_stderr=no + while test $# -gt 0; do + case $1 in + -e) run_exitcode_expected=$2; shift;; + -m) run_mix_stdout_and_stderr=yes;; + --) shift; break;; + -?) echo "run_commmand: invalid switch \`$1'" >&2; Exit 99;; + *) break;; + esac + shift + done + case $# in + 0) echo "run_command: missing COMMAND argument" >&2; Exit 99;; + *) run_cmd=$1; shift;; + esac + if test x"$run_mix_stdout_and_stderr" = x"yes"; then + run_evald_cmd='"$run_cmd" ${1+"$@"} >stdall 2>&1' + else + run_evald_cmd='"$run_cmd" ${1+"$@"} >stdout 2>stderr' + fi + # NOTE: do not use the apparently simpler construct: + # if eval "$run_evald_cmd"; then + # run_exitcode_got=0 + # else + # run_exitcode_got=$? + # fi + # as FreeBSD /bin/sh chokes on it when `set -e' is on. + if (eval "exec $run_evald_cmd"); then + run_exitcode_got=0 + else + run_exitcode_got=$? + fi + if test x"$run_mix_stdout_and_stderr" = x"yes"; then + set -x + cat stdall + else + set -x + cat stderr >&2 + cat stdout + fi + # Apparently, the `set +x' with redirected stderr is executed in a + # subshell by Solaris Sh, so repeat it without redirection. + { set +x; } 2>/dev/null; set +x + case $run_exitcode_expected in + RETURN|return) + restore_shell_flags 'run_command' + return $run_exitcode_got # RETURN POINT HERE + ;; + IGNORE|ignore|IGNORED|ignored|$run_exitcode_got) + run_ok=yes + ;; + FAIL|fail|FAILURE|failure) + if test $run_exitcode_got -gt 0; then + run_ok=yes + else + run_ok=no + fi + ;; + *) + run_ok=no + ;; + esac + echo "run_command: exit status $run_exitcode_got (expecting" \ + "$run_exitcode_expected)" + restore_shell_flags 'run_command' + test x"$run_ok" = x"yes" || Exit 1 +} # AUTOMAKE_run status [options...] # -------------------------------- -# Run Automake with OPTIONS, and fail if automake +# Run Automake with OPTIONS, and cause the test to FAIL if automake # does not exit with STATUS. AUTOMAKE_run () { - expected_exitcode=$1 + am_run_expected_exitcode=$1 shift - exitcode=0 - $AUTOMAKE ${1+"$@"} >stdout 2>stderr || exitcode=$? - cat stderr >&2 - cat stdout - test $exitcode = $expected_exitcode || Exit 1 + run_command -e $am_run_expected_exitcode -- $AUTOMAKE ${1+"$@"} } # AUTOMAKE_fails [options...] # --------------------------- -# Run Automake with OPTIONS, and fail if automake -# does not exit with STATUS. +# Run Automake with OPTIONS, and cuse the test to FAIL if automake +# does not exit with status 1. AUTOMAKE_fails () { AUTOMAKE_run 1 ${1+"$@"} diff --git a/tests/depcomp2.test b/tests/depcomp2.test index 4653c90..0daf2f0 100755 --- a/tests/depcomp2.test +++ b/tests/depcomp2.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2001, 2002 Free Software Foundation, Inc. +# Copyright (C) 2001, 2002, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -20,6 +20,8 @@ required=gcc . ./defs || Exit 1 +set -e + cat > configure.in << 'END' AC_INIT(subdir/foo.c) AM_INIT_AUTOMAKE(nonesuch, nonesuch) @@ -45,8 +47,8 @@ END CFLAGS= export CFLAGS -$ACLOCAL || Exit 1 -$AUTOMAKE --add-missing || Exit 1 -$AUTOCONF || Exit 1 -./configure CC='gcc' 2>error.log -test -z "`cat error.log`" +$ACLOCAL +$AUTOMAKE --add-missing +$AUTOCONF +run_command ./configure CC='gcc' +test ! -s stderr diff --git a/tests/depcomp4.test b/tests/depcomp4.test index 7d22894..70cbd57 100755 --- a/tests/depcomp4.test +++ b/tests/depcomp4.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2003 Free Software Foundation, Inc. +# Copyright (C) 2003, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -51,7 +51,6 @@ $AUTOMAKE -a grep am_cv_CC_dependencies_compiler_type configure ./configure am_cv_CC_dependencies_compiler_type=makedepend -$MAKE >output 2>&1 -cat output -grep 'unknown directive' output && Exit 1 +run_command -m $MAKE +grep 'unknown directive' stdall && Exit 1 $MAKE grepdeps diff --git a/tests/depend5.test b/tests/depend5.test index 0d9be3b..52e4169 100755 --- a/tests/depend5.test +++ b/tests/depend5.test @@ -65,10 +65,7 @@ for arg in Makefile \ "--file=$name" do rm -rf .deps _deps - ./config.status "$arg" depfiles >stdout 2>stderr || - { cat stdout; cat stderr >&2; Exit 1; } - cat stdout - cat stderr >&2 + run_command ./config.status "$arg" depfiles grep '[Nn]o such file' stderr && Exit 1 if test -n "$depdir"; then diff --git a/tests/depend6.test b/tests/depend6.test index de2adf9..5ce2208 100755 --- a/tests/depend6.test +++ b/tests/depend6.test @@ -39,9 +39,6 @@ END $ACLOCAL $AUTOMAKE $AUTOCONF -./configure --disable-dependency-tracking 2>stderr || { - stat=$?; cat stderr >&2; Exit $stat; -} -cat stderr >&2 -grep shift stderr && Exit 1 +run_command ./configure --disable-dependency-tracking +grep 'shift' stderr && Exit 1 : diff --git a/tests/discover.test b/tests/discover.test index 36293f7..c62f1b4 100755 --- a/tests/discover.test +++ b/tests/discover.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1996, 1997, 2000, 2001, 2002 Free Software Foundation, Inc. +# Copyright (C) 1996, 1997, 2000, 2001, 2002, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -34,5 +34,6 @@ END : > fsusage.c $ACLOCAL || Exit 1 -$AUTOMAKE 2>&1 | $FGREP fsusage.c || Exit 1 +run_command -e IGNORE -m $AUTOMAKE +$FGREP fsusage.c stdall || Exit 1 Exit 0 diff --git a/tests/distcom3.test b/tests/distcom3.test index 04bf365..fdcc31a 100755 --- a/tests/distcom3.test +++ b/tests/distcom3.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2001, 2002, 2003, 2004, 2006 Free Software Foundation, Inc. +# Copyright (C) 2001, 2002, 2003, 2004, 2006, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -34,9 +34,8 @@ END $ACLOCAL # Should not warn about missing README, since it is a target. -$AUTOMAKE --add-missing --gnu >stdout 2>&1 -cat stdout -grep README stdout && Exit 1 +run_command -m $AUTOMAKE --add-missing --gnu +grep README stdall && Exit 1 sed -n -e '/^DIST_COMMON =.*\\$/ { :loop diff --git a/tests/fn99.test b/tests/fn99.test index 982a989..611c767 100755 --- a/tests/fn99.test +++ b/tests/fn99.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2004, 2008 Free Software Foundation, Inc. +# Copyright (C) 2004, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -47,8 +47,7 @@ do touch x done) || Exit 77 -$MAKE dist 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_command -e FAIL $MAKE dist grep 'filenames are too long' stderr test 2 = `grep 12345678 stderr | wc -l` : diff --git a/tests/fn99subdir.test b/tests/fn99subdir.test index 465f2af..7055e85 100755 --- a/tests/fn99subdir.test +++ b/tests/fn99subdir.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2006, 2008 Free Software Foundation, Inc. +# Copyright (C) 2006, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -73,8 +73,7 @@ do touch x done) -$MAKE dist 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_command -e FAIL $MAKE dist grep 'filenames are too long' stderr test 1 = `grep 12345678 stderr | wc -l` : diff --git a/tests/gnits2.test b/tests/gnits2.test index 4c5d54f..a85b096 100755 --- a/tests/gnits2.test +++ b/tests/gnits2.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2002, 2003, 2004, 2007 Free Software Foundation, Inc. +# Copyright (C) 2002, 2003, 2004, 2007, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -105,6 +105,6 @@ cd build ../configure "--prefix=`pwd`/../inst-dir" --program-prefix=p $MAKE all $MAKE test-install -$MAKE -k installcheck 2>stderr || : # Never trust the exit status of make -k. -cat stderr +# Never trust the exit status of make -k. +run_command -e IGNORE $MAKE -k installcheck $MAKE grep-stderr diff --git a/tests/gnits3.test b/tests/gnits3.test index 7a08fb8..4d90684 100755 --- a/tests/gnits3.test +++ b/tests/gnits3.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2002, 2007 Free Software Foundation, Inc. +# Copyright (C) 2002, 2007, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -83,8 +83,8 @@ cd build ../configure "--prefix=`pwd`/../inst-dir" --program-prefix=p $MAKE $MAKE install -$MAKE -k installcheck 2>stderr || : # Never trust the exit status of make -k. -cat stderr +# Never trust the exit status of make -k. +run_command -e IGNORE $MAKE -k installcheck $MAKE grep-stderr # Make sure there is no more error when all targets are exempted. diff --git a/tests/help.test b/tests/help.test index d4d7110..cae8967 100755 --- a/tests/help.test +++ b/tests/help.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2003, 2008 Free Software Foundation, Inc. +# Copyright (C) 2003, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -32,7 +32,7 @@ $AUTOMAKE --version $AUTOMAKE --help # aclocal and automake cannot work without configure.ac or configure.in -$ACLOCAL 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e 1 $ACLOCAL grep configure.ac stderr grep configure.in stderr AUTOMAKE_fails diff --git a/tests/init.test b/tests/init.test index 38ec681..b6a7375 100755 --- a/tests/init.test +++ b/tests/init.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2006, 2008 Free Software Foundation, Inc. +# Copyright (C) 2006, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -27,7 +27,17 @@ AM_INIT_AUTOMAKE END # The error message should mension AC_INIT, not AC_PACKAGE_VERSION. -($ACLOCAL && $AUTOCONF) 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +if run_command -e RETURN $ACLOCAL; then + success=yes +else + success=no +fi +if test x"$success" != x"no"; then + mv stderr stder1 + run_command -e RETURN $AUTOCONF || success=no + cat stderr >>stder1 + mv -f stder1 stderr +fi +test x"$success" = x"no" || Exit 1 grep AC_PACKAGE_VERSION stderr && Exit 1 -grep AC_INIT stderr +grep AC_INIT stderr || Exit 1 diff --git a/tests/instsh.test b/tests/instsh.test index 61f5f9e..fe3d485 100755 --- a/tests/instsh.test +++ b/tests/instsh.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1996, 1999, 2001, 2002, 2003 Free Software Foundation, Inc. +# Copyright (C) 1996, 1999, 2001, 2002, 2003, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -32,10 +32,9 @@ mv Makefile.am configure.in frob/ cd frob $ACLOCAL -$AUTOMAKE --add-missing > output 2>&1 +run_command -m $AUTOMAKE --add-missing # Only one `/' should appear in the output. -cat output -grep '/.*/' output && Exit 1 +grep '/.*/' stdall && Exit 1 test -f install-sh diff --git a/tests/ldadd.test b/tests/ldadd.test index 005f9d8..0a9907d 100755 --- a/tests/ldadd.test +++ b/tests/ldadd.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1997, 1999, 2000, 2001, 2002 Free Software Foundation, Inc. +# Copyright (C) 1997, 1999, 2000, 2001, 2002, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -39,5 +39,6 @@ END : > q.c $ACLOCAL || Exit 1 -$AUTOMAKE 2>&1 | grep LDADD || Exit 1 +run_command -e 1 -m $AUTOMAKE +grep LDADD stdall || Exit 1 Exit 0 diff --git a/tests/libtool7.test b/tests/libtool7.test index a2f912f..f8706d9 100755 --- a/tests/libtool7.test +++ b/tests/libtool7.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2003, 2004, 2007 Free Software Foundation, Inc. +# Copyright (C) 2003, 2004, 2007, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -82,19 +82,16 @@ $AUTOCONF $AUTOMAKE --add-missing --copy ./configure "--prefix=`pwd`/_inst" -env LIBTOOLFLAGS=--silent $MAKE print >output 2>&1 -cat output -grep '1BEG: libmod1.la mod2.la :END1' output -grep '2BEG: mod2.la :END2' output -grep '3BEG: .*silent.*silent.* :END3' output +run_command -m env LIBTOOLFLAGS=--silent $MAKE print +grep '1BEG: libmod1.la mod2.la :END1' stdall +grep '2BEG: mod2.la :END2' stdall +grep '3BEG: .*silent.*silent.* :END3' stdall test 2 -le `grep mod2_la_LIBTOOLFLAGS Makefile | wc -l` $MAKE -env LIBTOOLFLAGS=--silent $MAKE install >output 2>&1 -cat output -grep 'silent.*silent.*prg' output -grep 'silent.*silent.*libmod1' output +run_command -m env LIBTOOLFLAGS=--silent $MAKE install +grep 'silent.*silent.*prg' stdall +grep 'silent.*silent.*libmod1' stdall -env LIBTOOLFLAGS=--silent $MAKE uninstall >output 2>&1 -cat output -grep 'silent.*silent.*libmod1' output +run_command -m env LIBTOOLFLAGS=--silent $MAKE uninstall +grep 'silent.*silent.*libmod1' stdall diff --git a/tests/libtool9.test b/tests/libtool9.test index c0d9688..94e617b 100755 --- a/tests/libtool9.test +++ b/tests/libtool9.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2005 Free Software Foundation, Inc. +# Copyright (C) 2005, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -88,10 +88,11 @@ $AUTOCONF $AUTOMAKE --add-missing --copy ./configure -env LDFLAGS=ldflags AM_LDFLAGS=am_ldflags libmod1_la_LDFLAGS=lm1_la_ldflags \ +run_command -m env \ + LDFLAGS=ldflags AM_LDFLAGS=am_ldflags libmod1_la_LDFLAGS=lm1_la_ldflags \ CFLAGS=cflags AM_CFLAGS=am_cflags prg2_CFLAGS=prg2_cflags \ - $MAKE -e print >output 2>&1 -cat output + $MAKE -e print +mv stdall output grep '1BEG: libmod1.la mod2.la :END1' output grep '2BEG: mod2.la :END2' output grep '3BEG:.* am_cflags cflags .*lm1_la_ldflags ldflags.* :END3' output diff --git a/tests/lisp3.test b/tests/lisp3.test index 809d491..086173d 100755 --- a/tests/lisp3.test +++ b/tests/lisp3.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2003, 2008 Free Software Foundation, Inc. +# Copyright (C) 2003, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -61,9 +61,8 @@ test -f am-one.elc unique=0a3346e2af8a689b85002b53df09142a $sleep echo "(message \"$unique\")(provide 'am-three)" > am-three.el -$MAKE >output 2>&1 -cat output -grep $unique output +run_command -m $MAKE +grep $unique stdall # It should also work for VPATH-builds. $MAKE distcheck diff --git a/tests/ltorder.test b/tests/ltorder.test index 0f69211..1e6a03a 100755 --- a/tests/ltorder.test +++ b/tests/ltorder.test @@ -1,5 +1,5 @@ #!/bin/sh -# Copyright (C) 2008 Free Software Foundation, Inc. +# Copyright (C) 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -53,8 +53,7 @@ $AUTOMAKE --add-missing ./configure "--prefix=`pwd`/inst" $MAKE -$MAKE install 2>stderr || { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_command $MAKE install grep 'has not been installed' stderr && Exit 1 $MAKE uninstall diff --git a/tests/man4.test b/tests/man4.test index aa86a9e..7424c72 100755 --- a/tests/man4.test +++ b/tests/man4.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2008 Free Software Foundation, Inc. +# Copyright (C) 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -88,11 +88,9 @@ mkdir build cd build ../configure $MAKE -$MAKE dist 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_command -e FAIL $MAKE dist grep 'install help2man' stderr -$MAKE distcheck 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_command -e FAIL $MAKE distcheck grep 'install help2man' stderr rm -f ../help2man diff --git a/tests/mdate.test b/tests/mdate.test index 704b393..e63bff9 100755 --- a/tests/mdate.test +++ b/tests/mdate.test @@ -1,5 +1,5 @@ #!/bin/sh -# Copyright (C) 1996, 1997, 2001, 2002 Free Software Foundation, Inc. +# Copyright (C) 1996, 1997, 2001, 2002, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -31,4 +31,5 @@ END : > texinfo.tex $ACLOCAL || Exit 1 -$AUTOMAKE 2>&1 > /dev/null | grep 'required file.*mdate-sh' +run_command -e IGNORED $AUTOMAKE +grep 'required file.*mdate-sh' stderr diff --git a/tests/missing3.test b/tests/missing3.test index 07ef779..9b48368 100755 --- a/tests/missing3.test +++ b/tests/missing3.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2004, 2008 Free Software Foundation, Inc. +# Copyright (C) 2004, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -22,24 +22,24 @@ set -e # b7cb8259 assumed not to exist. -./missing b7cb8259 --version 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e FAIL ./missing b7cb8259 --version grep . stderr && Exit 1 -./missing b7cb8259 --grep 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e FAIL ./missing b7cb8259 --grep grep WARNING stderr -./missing --run b7cb8259 --version && Exit 1 -./missing --run b7cb8259 --grep 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e FAIL ./missing --run b7cb8259 --version +run_command -e FAIL ./missing --run b7cb8259 --grep grep WARNING stderr # missing itself it known to exist :) -./missing ./missing --version 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e FAIL ./missing ./missing --version grep . stderr && Exit 1 -./missing ./missing --grep 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e FAIL ./missing ./missing --grep grep WARNING stderr -./missing --run ./missing --version 2>stderr +run_command ./missing --run ./missing --version grep . stderr && Exit 1 -./missing --run ./missing --grep 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e FAIL ./missing --run ./missing --grep grep WARNING stderr && Exit 1 grep Unknown stderr diff --git a/tests/missing4.test b/tests/missing4.test index 1015828..c6c3b7e 100755 --- a/tests/missing4.test +++ b/tests/missing4.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2006, 2007, 2008 Free Software Foundation, Inc. +# Copyright (C) 2006, 2007, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -36,13 +36,11 @@ cmp aclocal.m4 aclocal.tmp && Exit 1 mv aclocal.tmp aclocal.m4 -$MAKE 2>stderr -cat stderr +run_command $MAKE grep 'You have another version of autoconf' stderr grep -c 'aclocal.m4:.*this file was generated for' stderr -$MAKE 2>stderr -cat stderr +run_command $MAKE grep 'You have another version of autoconf' stderr && Exit 1 grep 'aclocal.m4:.*this file was generated for' stderr && Exit 1 diff --git a/tests/missing6.test b/tests/missing6.test index b63a0d1..bf6df2d 100755 --- a/tests/missing6.test +++ b/tests/missing6.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2008 Free Software Foundation, Inc. +# Copyright (C) 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -31,7 +31,7 @@ rm -f configure.in touch Makefile.am $ACLOCAL -$AUTOCONF 2>stderr +run_command $AUTOCONF grep 'You have another version of autoconf' stderr grep 'aclocal.m4:.*this file was generated for' stderr $AUTOMAKE @@ -43,8 +43,7 @@ cmp configure.ac configure.tmp && Exit 1 mv configure.tmp configure.ac -$MAKE 2>stderr -cat stderr +run_command $MAKE grep 'You have another version of autoconf' stderr grep 'aclocal.m4:.*this file was generated for autoconf 9999a' stderr diff --git a/tests/obsolete.test b/tests/obsolete.test index e6601c9..daae69c 100755 --- a/tests/obsolete.test +++ b/tests/obsolete.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1996, 2001, 2002, 2007, 2008 Free Software Foundation, Inc. +# Copyright (C) 1996, 2001, 2002, 2007, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -27,7 +27,7 @@ END $PERL -ne '/AU_DEFUN\(\[(\w+)\]/ && print "$1\n"' \ "$testsrcdir/../m4/obsolete.m4" >> obs cat obs >> configure.in -$PERL -ne 'chomp; print "grep $_ stderr || Exit 1\n"; ' obs > obs.1 +$PERL -ne 'chomp; print "grep $_ stdall || Exit 1\n"; ' obs > obs.1 $PERL -ne 'chomp; print "grep $_ configure.in && Exit 1\n"; ' obs > obs.2 # Sanity check. Make sure we have added something to configure.in. @@ -36,7 +36,7 @@ test `cat configure.in | wc -l` -gt 1 || Exit 1 $ACLOCAL || Exit 1 # Expect Autoconf to complain about each of the macros in obs. -$AUTOCONF -Wobsolete >stderr 2>&1 +run_command -m $AUTOCONF -Wobsolete . ./obs.1 # Make sure Autoupdate remove each of these macros. $AUTOUPDATE || Exit 1 diff --git a/tests/output-order.test b/tests/output-order.test index 856baa2..852c194 100755 --- a/tests/output-order.test +++ b/tests/output-order.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2008 Free Software Foundation, Inc. +# Copyright (C) 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -43,13 +43,12 @@ END $ACLOCAL $AUTOCONF rm -f missing install-sh -$AUTOMAKE --add-missing --copy 2>stderr -cat stderr >&2 +run_command $AUTOMAKE --add-missing --copy +mv stderr stder0 for i in 1 2 3 4 5 6; do rm -f missing install-sh INSTALL COPYING + run_command -e IGNORE $AUTOMAKE --add-missing --copy # The grep prevents a Heisenbug with the HP-UX shell and VERBOSE=yes. - $AUTOMAKE --add-missing --copy 2>&1 >/dev/null | - grep -v /dev/null | - diff - stderr + grep -v /dev/null stder0 | diff - stderr done diff --git a/tests/pr220.test b/tests/pr220.test index 25e1f0e..6644d28 100755 --- a/tests/pr220.test +++ b/tests/pr220.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2001, 2002, 2004, 2008 Free Software Foundation, Inc. +# Copyright (C) 2001, 2002, 2004, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -55,6 +55,5 @@ $AUTOMAKE -a cd build # configure should fail since we've done something invalid. -../configure 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_command -e FAIL ../configure grep NEVER_TRUE stderr diff --git a/tests/pr300-ltlib.test b/tests/pr300-ltlib.test index 01814a9..7ed0a2f 100755 --- a/tests/pr300-ltlib.test +++ b/tests/pr300-ltlib.test @@ -48,8 +48,7 @@ $ACLOCAL $AUTOCONF $AUTOMAKE --copy --add-missing ./configure --prefix "`pwd`/inst" -$MAKE V=1 >stdout -cat stdout +run_command $MAKE V=1 grep 'liba.la .*-rpath .*lib' stdout grep 'liba.la .*-rpath .*lib/subdir' stdout && Exit 1 @@ -58,8 +57,7 @@ grep 'libb.la .*-rpath .*lib/subdir' stdout test -f subdir/liba.la test -f subdir/libb.la -$MAKE install 2>stderr -cat stderr >&2 +run_command $MAKE install grep 'remember.*--finish' stderr && Exit 1 test -f inst/lib/liba.la diff --git a/tests/python11.test b/tests/python11.test index 0f15f37..920de88 100755 --- a/tests/python11.test +++ b/tests/python11.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2004, 2006, 2008 Free Software Foundation, Inc. +# Copyright (C) 2004, 2006, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -35,9 +35,7 @@ EOF $ACLOCAL $AUTOCONF -./configure >stdout 2>stderr && { cat stdout; cat stderr >&2; Exit 1; } -cat stdout -cat stderr >&2 +run_command -e FAILURE ./configure grep 'checking for IShouldNotExist1' stdout grep 'checking for IShouldNotExist2' stdout grep 'no suitable Python interpreter found' stderr diff --git a/tests/python4.test b/tests/python4.test index 11bb3ed..3ff44af 100755 --- a/tests/python4.test +++ b/tests/python4.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2003, 2008 Free Software Foundation, Inc. +# Copyright (C) 2003, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -33,6 +33,5 @@ $AUTOCONF $AUTOMAKE --add-missing # Simulate no Python -./configure PYTHON=: 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_command -e FAILURE ./configure PYTHON=: grep 'no suitable Python interpreter found' stderr diff --git a/tests/python5.test b/tests/python5.test index 86f5863..5695876 100755 --- a/tests/python5.test +++ b/tests/python5.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2003, 2008 Free Software Foundation, Inc. +# Copyright (C) 2003, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -34,6 +34,5 @@ $ACLOCAL $AUTOCONF $AUTOMAKE --add-missing -./configure 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_command -e FAILURE ./configure grep 'no suitable Python interpreter found' stderr diff --git a/tests/python6.test b/tests/python6.test index eb4fb4d..ed60e03 100755 --- a/tests/python6.test +++ b/tests/python6.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2003 Free Software Foundation, Inc. +# Copyright (C) 2003, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -34,6 +34,5 @@ $AUTOCONF $AUTOMAKE --add-missing # Simulate no Python -./configure PYTHON=: 2>stderr -cat stderr +run_command ./configure PYTHON=: grep 'GREP ME:' stderr diff --git a/tests/python7.test b/tests/python7.test index 62c1a1b..eb6a4dc 100755 --- a/tests/python7.test +++ b/tests/python7.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2003 Free Software Foundation, Inc. +# Copyright (C) 2003, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -34,6 +34,5 @@ $ACLOCAL $AUTOCONF $AUTOMAKE --add-missing -./configure 2>stderr -cat stderr +run_command ./configure grep 'GREP ME:' stderr diff --git a/tests/python8.test b/tests/python8.test index a50af22..5f100ab 100755 --- a/tests/python8.test +++ b/tests/python8.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2003 Free Software Foundation, Inc. +# Copyright (C) 2003, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -32,6 +32,5 @@ $ACLOCAL $AUTOCONF $AUTOMAKE --add-missing -./configure 2>stderr -cat stderr +run_command ./configure grep 'GREP ME' stderr diff --git a/tests/python9.test b/tests/python9.test index 9b23deb..c0fa7a5 100755 --- a/tests/python9.test +++ b/tests/python9.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2003 Free Software Foundation, Inc. +# Copyright (C) 2003, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -33,6 +33,5 @@ $ACLOCAL $AUTOCONF $AUTOMAKE --add-missing -./configure 2>stderr -cat stderr +run_command ./configure grep 'GREP ME' stderr diff --git a/tests/sanity.test b/tests/sanity.test index 1c96c78..1296381 100755 --- a/tests/sanity.test +++ b/tests/sanity.test @@ -36,14 +36,12 @@ cp ../install-sh ../missing . $ACLOCAL $AUTOCONF $AUTOMAKE -./configure 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr +run_command -e FAILURE ./configure grep 'unsafe absolute working directory' stderr cd .. mkdir build cd build -../unsafe$/configure 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr +run_command -e FAILURE ../unsafe$/configure grep 'unsafe srcdir' stderr : diff --git a/tests/subobj.test b/tests/subobj.test index df297f4..9d7882d 100755 --- a/tests/subobj.test +++ b/tests/subobj.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1999, 2001, 2002, 2004 Free Software Foundation, Inc. +# Copyright (C) 1999, 2001, 2002, 2004, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -32,8 +32,7 @@ wish_SOURCES = generic/a.c generic/b.c END $ACLOCAL -$AUTOMAKE --add-missing 2>stderr -cat stderr +run_command $AUTOMAKE --add-missing # Make sure compile is installed, and that Automake says so. grep 'install.*compile' stderr test -f compile diff --git a/tests/subobj9.test b/tests/subobj9.test index 99fe6e8..1f68721 100755 --- a/tests/subobj9.test +++ b/tests/subobj9.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2002, 2004, 2005 Free Software Foundation, Inc. +# Copyright (C) 2002, 2004, 2005, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -74,8 +74,8 @@ if test -n "`./libtool --help | grep tag=TAG`"; then fi $MAKE -$MAKE distcheck 2>&1 | tee out +run_command -m $MAKE distcheck # GNU Make used to complain that the Makefile contained two rules # for `src/.dirstamp' and `.//src/.dirstamp'. -grep 'overriding commands' out && Exit 1 +grep 'overriding commands' stdall && Exit 1 : diff --git a/tests/unused.test b/tests/unused.test index a5b4da6..68d5613 100755 --- a/tests/unused.test +++ b/tests/unused.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1998, 2002, 2003, 2008 Free Software Foundation, Inc. +# Copyright (C) 1998, 2002, 2003, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -29,5 +29,5 @@ AC_DEFUN([MACRO_1_2], echo 12) AC_DEFUN([MACRO_1_2_3], echo 123) END -$ACLOCAL 2> stderr || { cat stderr >&2; Exit 1; } -test -z "`cat stderr`" +run_command $ACLOCAL +test ! -s stderr diff --git a/tests/vars3.test b/tests/vars3.test index 3d10a90..9f11262 100755 --- a/tests/vars3.test +++ b/tests/vars3.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2002 Free Software Foundation, Inc. +# Copyright (C) 2002, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -38,8 +38,7 @@ EOF $ACLOCAL # Make sure this warning is print in the `portability' category. -$AUTOMAKE --warnings=no-error,none,portability 2>stderr -cat stderr +run_command $AUTOMAKE --warnings=no-error,none,portability # Lines number are printed in error message. # Use them to make sure errors are diagnosed against the right lines. diff --git a/tests/version8.test b/tests/version8.test index 91b11a0..ffe4a3e 100755 --- a/tests/version8.test +++ b/tests/version8.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2005, 2008 Free Software Foundation, Inc. +# Copyright (C) 2005, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -20,6 +20,5 @@ set -e echo 'AM_AUTOMAKE_VERSION([1.9])' >>configure.in -$ACLOCAL 2>stderr && { cat stderr >&2; Exit 0; } -cat stderr >&2 +run_command -e 1 $ACLOCAL $FGREP 'AM_INIT_AUTOMAKE([1.9])' stderr -- 1.6.3.3 |
|
|
Re: testsuite failures when test scripts are run with zsh
by Stefano Lattarini
::
Rate this Message:
I have rebased my private branch against master, and also amended the
patch a bit (to deal with the test `cscope.test', which used a `$MAKE 2>stderr' redirection that I missed). The updated patch is attached. Regards, Stefano From e9666da90920c1373bfe6f9cbf8aa1e4e50c2b8a Mon Sep 17 00:00:00 2001 From: Stefano Lattarini <stefano.lattarini@...> Date: Sat, 5 Dec 2009 15:03:54 +0100 Subject: [PATCH] Testsuite: avoid Zsh-related problem with `set -x'. * tests/README: Describe Zsh 4.x `set -x' a.k.a. `xtrace' issue and workaround with run_command. * tests/defs.in (run_command): New function, to be used for commands whose standard error needs to be captured. (save_shell_flags, restore_shell_flags): New functions, used in run_command. (AUTOMAKE_run): Reimplemented using run_command. Updated comment. (AUTOMAKE_fails): Updated comment. * tests/acloca14.test: Use new subroutine run_command instead of hand-crafted redirections of stdout and/or stderr. * tests/acloca17.test: Likewise. * tests/acloca18.test: Likewise. * tests/aclocal.test: Likewise. * tests/aclocal3.test: Likewise. * tests/aclocal8.test: Likewise. * tests/acsilent.test: Likewise. * tests/alpha.test: Likewise. * tests/ammissing.test: Likewise. * tests/ansi3.test: Likewise. * tests/ansi3b.test: Likewise. * tests/ccnoco3.test: Likewise. * tests/check8.test: Likewise. * tests/cond39.test: Likewise. * tests/cscope.test: Likewise. * tests/depcomp2.test: Likewise. * tests/depcomp4.test: Likewise. * tests/depend5.test: Likewise. * tests/depend6.test: Likewise. * tests/discover.test: Likewise. * tests/distcom3.test: Likewise. * tests/fn99.test: Likewise. * tests/fn99subdir.test: Likewise. * tests/gnits2.test: Likewise. * tests/gnits3.test: Likewise. * tests/help.test: Likewise. * tests/init.test: Likewise. * tests/instsh.test: Likewise. * tests/ldadd.test: Likewise. * tests/libtool7.test: Likewise. * tests/libtool9.test: Likewise. * tests/lisp3.test: Likewise. * tests/ltorder.test: Likewise. * tests/man4.test: Likewise. * tests/mdate.test: Likewise. * tests/missing3.test: Likewise. * tests/missing4.test: Likewise. * tests/missing6.test: Likewise. * tests/obsolete.test: Likewise. * tests/output-order.test: Likewise. * tests/pr220.test: Likewise. * tests/pr300-ltlib.test: Likewise. * tests/python11.test: Likewise. * tests/python4.test: Likewise. * tests/python5.test: Likewise. * tests/python6.test: Likewise. * tests/python7.test: Likewise. * tests/python8.test: Likewise. * tests/python9.test: Likewise. * tests/sanity.test: Likewise. * tests/subobj.test: Likewise. * tests/subobj9.test: Likewise. * tests/unused.test: Likewise. * tests/vars3.test: Likewise. * tests/version8.test: Likewise. --- ChangeLog | 68 +++++++++++++++++++++++++++ tests/README | 10 ++++ tests/acloca14.test | 4 +- tests/acloca17.test | 6 +-- tests/acloca18.test | 10 ++-- tests/aclocal.test | 8 ++-- tests/aclocal3.test | 5 +- tests/aclocal8.test | 7 +-- tests/acsilent.test | 7 ++- tests/alpha.test | 5 +- tests/ammissing.test | 5 +- tests/ansi3.test | 5 +- tests/ansi3b.test | 5 +- tests/ccnoco3.test | 3 +- tests/check8.test | 8 +-- tests/cond39.test | 8 +-- tests/cscope.test | 6 +-- tests/defs.in | 119 +++++++++++++++++++++++++++++++++++++++++++---- tests/depcomp2.test | 14 +++-- tests/depcomp4.test | 7 +-- tests/depend5.test | 5 +-- tests/depend6.test | 7 +-- tests/discover.test | 5 +- tests/distcom3.test | 7 +-- tests/fn99.test | 5 +- tests/fn99subdir.test | 5 +- tests/gnits2.test | 6 +- tests/gnits3.test | 6 +- tests/help.test | 4 +- tests/init.test | 18 ++++++-- tests/instsh.test | 7 +-- tests/ldadd.test | 5 +- tests/libtool7.test | 23 ++++----- tests/libtool9.test | 9 ++-- tests/lisp3.test | 7 +-- tests/ltorder.test | 5 +- tests/man4.test | 8 +-- tests/mdate.test | 5 +- tests/missing3.test | 18 ++++---- tests/missing4.test | 8 +-- tests/missing6.test | 7 +-- tests/obsolete.test | 6 +- tests/output-order.test | 11 ++-- tests/pr220.test | 5 +- tests/pr300-ltlib.test | 6 +-- tests/python11.test | 6 +-- tests/python4.test | 5 +- tests/python5.test | 5 +- tests/python6.test | 5 +- tests/python7.test | 5 +- tests/python8.test | 5 +- tests/python9.test | 5 +- tests/sanity.test | 6 +-- tests/subobj.test | 5 +- tests/subobj9.test | 6 +- tests/unused.test | 6 +- tests/vars3.test | 5 +- tests/version8.test | 5 +- 58 files changed, 358 insertions(+), 219 deletions(-) diff --git a/ChangeLog b/ChangeLog index a5c98fe..189d6d6 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,71 @@ +2009-12-05 Stefano Lattarini <stefano.lattarini@...> + + Testsuite: avoid Zsh-related problem with `set -x'. + * tests/README: Describe Zsh 4.x `set -x' a.k.a. `xtrace' issue + and workaround with run_command. + * tests/defs.in (run_command): New function, to be used for + commands whose standard error needs to be captured. + (save_shell_flags, restore_shell_flags): New functions, used in + run_command. + (AUTOMAKE_run): Reimplemented using run_command. Updated comment. + (AUTOMAKE_fails): Updated comment. + * tests/acloca14.test: Use new subroutine run_command instead + of hand-crafted redirections of stdout and/or stderr. + * tests/acloca17.test: Likewise. + * tests/acloca18.test: Likewise. + * tests/aclocal.test: Likewise. + * tests/aclocal3.test: Likewise. + * tests/aclocal8.test: Likewise. + * tests/acsilent.test: Likewise. + * tests/alpha.test: Likewise. + * tests/ammissing.test: Likewise. + * tests/ansi3.test: Likewise. + * tests/ansi3b.test: Likewise. + * tests/ccnoco3.test: Likewise. + * tests/check8.test: Likewise. + * tests/cond39.test: Likewise. + * tests/cscope.test: Likewise. + * tests/depcomp2.test: Likewise. + * tests/depcomp4.test: Likewise. + * tests/depend5.test: Likewise. + * tests/depend6.test: Likewise. + * tests/discover.test: Likewise. + * tests/distcom3.test: Likewise. + * tests/fn99.test: Likewise. + * tests/fn99subdir.test: Likewise. + * tests/gnits2.test: Likewise. + * tests/gnits3.test: Likewise. + * tests/help.test: Likewise. + * tests/init.test: Likewise. + * tests/instsh.test: Likewise. + * tests/ldadd.test: Likewise. + * tests/libtool7.test: Likewise. + * tests/libtool9.test: Likewise. + * tests/lisp3.test: Likewise. + * tests/ltorder.test: Likewise. + * tests/man4.test: Likewise. + * tests/mdate.test: Likewise. + * tests/missing3.test: Likewise. + * tests/missing4.test: Likewise. + * tests/missing6.test: Likewise. + * tests/obsolete.test: Likewise. + * tests/output-order.test: Likewise. + * tests/pr220.test: Likewise. + * tests/pr300-ltlib.test: Likewise. + * tests/python11.test: Likewise. + * tests/python4.test: Likewise. + * tests/python5.test: Likewise. + * tests/python6.test: Likewise. + * tests/python7.test: Likewise. + * tests/python8.test: Likewise. + * tests/python9.test: Likewise. + * tests/sanity.test: Likewise. + * tests/subobj.test: Likewise. + * tests/subobj9.test: Likewise. + * tests/unused.test: Likewise. + * tests/vars3.test: Likewise. + * tests/version8.test: Likewise. + 2009-11-28 Jim Meyering <meyering@...> do not put world-writable directories in distribution tarballs diff --git a/tests/README b/tests/README index d494b10..0b26211 100644 --- a/tests/README +++ b/tests/README @@ -133,6 +133,16 @@ Do Use $sleep when you have to make sure that some file is newer than another. + To run a program and analyze its stderr, use the run_command function. + For example: + run_command PROG [ARGS...] + grep $pattern stderr + or (if you must check that nothing is written on stderr): + run_command PROG2 [ARGS...] + test ! -s stderr + See section "File Descriptors" in the Autoconf manual to understand why + this can be useful, or even required. + Use `cat' or `grep' to display (part of) files that may be interesting for debugging, so that when a user send a verbose output we don't have to ask him for more details. Display stderr diff --git a/tests/acloca14.test b/tests/acloca14.test index 8ddbb1a..cf9c484 100755 --- a/tests/acloca14.test +++ b/tests/acloca14.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2004, 2008 Free Software Foundation, Inc. +# Copyright (C) 2004, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -108,5 +108,5 @@ $MAKE testdist2 # Make sure aclocal diagnose missing included files with correct `file:line:'. rm -f b.m4 -$ACLOCAL 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e 1 $ACLOCAL grep 'a.m4:1:.*b.m4.*does not exist' stderr diff --git a/tests/acloca17.test b/tests/acloca17.test index 884f24b..788cb9d 100755 --- a/tests/acloca17.test +++ b/tests/acloca17.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2004, 2008 Free Software Foundation, Inc. +# Copyright (C) 2004, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -33,7 +33,5 @@ EOF # FIXME: We want autom4te's 'undefined required macro' warning to be fatal, # but have no means to say so to aclocal. We use WARNINGS=error instead. - -WARNINGS=error $ACLOCAL -I m4 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_command -e 1 env WARNINGS=error $ACLOCAL -I m4 grep 'configure.in:4:.*UNDEFINED_MACRO' stderr diff --git a/tests/acloca18.test b/tests/acloca18.test index 22feb23..7047eea 100755 --- a/tests/acloca18.test +++ b/tests/acloca18.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2005, 2006, 2008 Free Software Foundation, Inc. +# Copyright (C) 2005, 2006, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -86,7 +86,7 @@ grep macro23 foo ACLOCAL_TESTSUITE_FLAGS='-I 4 -I 1 -I 2' rm -f foo -$ACLOCAL --install 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e 1 $ACLOCAL --install grep AM_MACRO2 stderr ACLOCAL_TESTSUITE_FLAGS='-I 4 -I 1' @@ -105,9 +105,7 @@ AC_DEFUN([AM_MACRO1], [echo macro1d >> foo]) AC_DEFUN([AM_MACRO2], [echo macro2d >> foo]) EOF rm -f foo -$ACLOCAL --diff=diff >output 2>stderr -cat stderr -cat output -grep '#serial 456' output +run_command $ACLOCAL --diff=diff +grep '#serial 456' stdout test ! -f 4/m1.m4 grep 'installing.*4/m1.m4' stderr diff --git a/tests/aclocal.test b/tests/aclocal.test index 6b87a9d..945b363 100755 --- a/tests/aclocal.test +++ b/tests/aclocal.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1998, 2001, 2002, 2004, 2008 Free Software Foundation, Inc. +# Copyright (C) 1998, 2001, 2002, 2004, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -23,15 +23,15 @@ set -e $ACLOCAL --output=fred test -f fred -$ACLOCAL --output 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e 1 $ACLOCAL --output grep 'option.*--output.*an argument' stderr grep help stderr -$ACLOCAL --unknown-option 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e 1 $ACLOCAL --unknown-option grep 'unrecognized.*--unknown-option' stderr grep help stderr -$ACLOCAL --ver 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e 1 $ACLOCAL --ver grep 'unrecognized.*--ver' stderr grep help stderr diff --git a/tests/aclocal3.test b/tests/aclocal3.test index a550e50..34c5335 100755 --- a/tests/aclocal3.test +++ b/tests/aclocal3.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1998, 2002, 2004 Free Software Foundation, Inc. +# Copyright (C) 1998, 2002, 2004, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -27,6 +27,5 @@ AC_DEFUN([GNOME_X_CHECKS], [ ]) END -$ACLOCAL -I macros 2>stderr -cat stderr +run_command -e 1 $ACLOCAL -I macros grep 'macros/gnome.m4:2:.*AM_PATH_GTK.*not found' stderr diff --git a/tests/aclocal8.test b/tests/aclocal8.test index ac10aff..4cf7d73 100755 --- a/tests/aclocal8.test +++ b/tests/aclocal8.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2003, 2004 Free Software Foundation, Inc. +# Copyright (C) 2003, 2004, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -34,9 +34,8 @@ EOF echo 'AC_DEFUN([MACRO1],)' >m4/macro1.m4 echo 'AC_DEFUN([MACRO2], [AC_REQUIRE([AM_UNUSED_MACRO])])' >m4/macro2.m4 -$ACLOCAL -I m4 >output 2>&1 -cat output -test 0 = `wc -l <output` +run_command -m $ACLOCAL -I m4 +test -s stdall && Exit 1 grep macro1.m4 aclocal.m4 grep macro2.m4 aclocal.m4 && Exit 1 : diff --git a/tests/acsilent.test b/tests/acsilent.test index 6ac7863..e250092 100755 --- a/tests/acsilent.test +++ b/tests/acsilent.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1996, 2001, 2002, 2008 Free Software Foundation, Inc. +# Copyright (C) 1996, 2001, 2002, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -32,5 +32,6 @@ module=[$1] AC_SUBST(module)]) END -$ACLOCAL > output 2>&1 || { cat output; Exit 1; } -test -z "`cat output`" +run_command -m $ACLOCAL +test -s stdall && Exit 1 +: diff --git a/tests/alpha.test b/tests/alpha.test index 1ee8ed6..1410b9d 100755 --- a/tests/alpha.test +++ b/tests/alpha.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1998, 2001, 2002, 2003, 2008 Free Software Foundation, Inc. +# Copyright (C) 1998, 2001, 2002, 2003, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -62,8 +62,7 @@ $AUTOMAKE ./configure # make distdir should fail because NEWS does not mention 1.0a -$MAKE check 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_command -e FAIL $MAKE check grep 'NEWS not updated' stderr test ! -f works diff --git a/tests/ammissing.test b/tests/ammissing.test index 61d360a..42aa481 100755 --- a/tests/ammissing.test +++ b/tests/ammissing.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1997, 2002, 2004 Free Software Foundation, Inc. +# Copyright (C) 1997, 2002, 2004, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -20,6 +20,5 @@ echo AM_ZARDOZ >> configure.in -$ACLOCAL 2>stderr -cat stderr +run_command -e 1 $ACLOCAL grep 'configure.in:.*AM_ZARDOZ.*not found' stderr diff --git a/tests/ansi3.test b/tests/ansi3.test index 2bbcaf7..95e4fc1 100755 --- a/tests/ansi3.test +++ b/tests/ansi3.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1998, 1999, 2001, 2002, 2003 Free Software Foundation, Inc. +# Copyright (C) 1998, 1999, 2001, 2002, 2003, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -53,8 +53,7 @@ export CFLAGS # Linux boxes (Red Hat 5.1 in particular). $ACLOCAL $AUTOCONF -$AUTOMAKE -a 2>stderr -cat stderr +run_command $AUTOMAKE -a grep 'Makefile.am:1:.*ansi2knr\.c' stderr CC='gcc' ./configure ANSI2KNR=./ansi2knr U=_ $MAKE -e diff --git a/tests/ansi3b.test b/tests/ansi3b.test index 8f254ad..447b690 100755 --- a/tests/ansi3b.test +++ b/tests/ansi3b.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1998, 1999, 2001, 2002, 2003 Free Software Foundation, Inc. +# Copyright (C) 1998, 1999, 2001, 2002, 2003, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -52,8 +52,7 @@ export CFLAGS # Linux boxes (Red Hat 5.1 in particular). $ACLOCAL $AUTOCONF -$AUTOMAKE -a 2>stderr -cat stderr +run_command $AUTOMAKE -a grep 'configure.in:2:.*ansi2knr\.1' stderr CC='gcc' ./configure ANSI2KNR=./ansi2knr U=_ $MAKE -e diff --git a/tests/ccnoco3.test b/tests/ccnoco3.test index 2e9cd37..93731a1 100755 --- a/tests/ccnoco3.test +++ b/tests/ccnoco3.test @@ -74,8 +74,7 @@ CC=`pwd`/../Mycomp export CC ../configure -$MAKE 2>stderr || { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_command $MAKE grep 'mv.*the same file' stderr && Exit 1 Exit 0 diff --git a/tests/check8.test b/tests/check8.test index 7936582..a087cb7 100755 --- a/tests/check8.test +++ b/tests/check8.test @@ -67,10 +67,7 @@ $AUTOMAKE -a unset TESTS || : ./configure -AM_COLOR_TESTS=always $MAKE -e check >stdout 2>stderr && - { cat stdout; cat stderr >&2; Exit 1; } -cat stdout -cat stderr >&2 +run_command -e FAIL -- env AM_COLOR_TESTS=always $MAKE -e check grep 'XPASS.* foo$' stdout grep '^[^X]*PASS.* sub/foo$' stdout grep '^[^X]*PASS.* bar' stdout @@ -87,8 +84,7 @@ $MAKE distclean mkdir build cd build ../configure -$MAKE check >stdout && { cat stdout; Exit 1; } -cat stdout +run_command -e FAIL $MAKE check # Note: we are not grepping for the space here, due to the Solaris make VPATH # rewriting (if we fix that, we can still write a separate test for it). grep 'XPASS.*foo$' stdout diff --git a/tests/cond39.test b/tests/cond39.test index 79eaa70..1e44866 100755 --- a/tests/cond39.test +++ b/tests/cond39.test @@ -1,5 +1,5 @@ #!/bin/sh -# Copyright (C) 2008 Free Software Foundation, Inc. +# Copyright (C) 2008, 2009 Free Software Foundation, Inc. # # This file is part of GNU Automake. # @@ -82,8 +82,7 @@ $AUTOCONF $AUTOMAKE --add-missing ./configure COND=true -$MAKE 2>stderr -cat stderr +run_command $MAKE grep 'overriding commands' stderr && Exit 1 $MAKE sure-exist ./prog1 && Exit 1 @@ -97,8 +96,7 @@ $MAKE sure-exist $MAKE distclean ./configure COND=false -$MAKE 2>stderr -cat stderr +run_command $MAKE grep 'overriding commands' stderr && Exit 1 ./prog1 ./sub/prog2 diff --git a/tests/cscope.test b/tests/cscope.test index 716cc38..8669169 100755 --- a/tests/cscope.test +++ b/tests/cscope.test @@ -73,8 +73,7 @@ $MAKE -n cscope : ${CSCOPE=cscope} ( $CSCOPE --version ) >/dev/null 2>&1 || Exit 77 -$MAKE cscope 2>stderr -cat stderr >&2 +run_command $MAKE cscope grep 'cannot find file' stderr && Exit 1 test -f cscope.files @@ -96,8 +95,7 @@ test ! -f cscope.po.out `pwd`/../configure || Exit 77 -$MAKE cscope 2>stderr -cat stderr >&2 +run_command $MAKE cscope grep 'cannot find file' stderr && Exit 1 $MAKE distclean diff --git a/tests/defs.in b/tests/defs.in index 6eec344..530f1e0 100644 --- a/tests/defs.in +++ b/tests/defs.in @@ -399,26 +399,127 @@ is_newest () test -z "$is_newest_files" } +# save_shell_flags [VAR-BASENAME] +# ------------------------------- +# Save current shell flags in variable VAR_saved_shell_flags +# (default: _saved_shell_flags) +save_shell_flags () +{ + eval "${1}_saved_shell_flags=\$-"; +} + +# restore_shell_flags [VAR] +# ------------------------- +# Restore shell flags saved in variable VAR_saved_shell_flags +# (default: _saved_shell_flags) +restore_shell_flags () +{ + eval set -\${"${1}_saved_shell_flags"} +} + +# run_command [-e STATUS] [-m] [--] COMMAND [ARGUMENTS..] +# ------------------------------------------------------- +# Run the given COMMAND with ARGUMENTS, and cause the test to FAIL if +# COMMAND does not exit with STATUS (more details are given below). +# Also, save standard output and standard error from COMMAND, by default +# respectively in files `stdout' and `stderr' (in the current directory), +# or together in the file `stdall' (in the current directory) if the `-m' +# option is given. +# If STATUS is numeric, the exit value of COMMAND must match it exactly. +# If STATUS is "FAIL" or "FAILURE", then any exit value of COMMAND *but 0* +# is acceptable. If STATUS is "IGNORE" or "IGNORED", any exit value of +# COMMAND is acceptable, and run_command returns with success regardless +# of that value. Finally, if STATUS is "RETURN", any exit value of COMMAND +# is acceptable, *and* such exit value is returned by run_command. +# Default STATUS is `0'. +run_command () +{ + save_shell_flags 'run_command' + set +x # xtrace verbosity temporarly disabled in this function + run_exitcode_expected=0 + run_mix_stdout_and_stderr=no + while test $# -gt 0; do + case $1 in + -e) run_exitcode_expected=$2; shift;; + -m) run_mix_stdout_and_stderr=yes;; + --) shift; break;; + -?) echo "run_commmand: invalid switch \`$1'" >&2; Exit 99;; + *) break;; + esac + shift + done + case $# in + 0) echo "run_command: missing COMMAND argument" >&2; Exit 99;; + *) run_cmd=$1; shift;; + esac + if test x"$run_mix_stdout_and_stderr" = x"yes"; then + run_evald_cmd='"$run_cmd" ${1+"$@"} >stdall 2>&1' + else + run_evald_cmd='"$run_cmd" ${1+"$@"} >stdout 2>stderr' + fi + # NOTE: do not use the apparently simpler construct: + # if eval "$run_evald_cmd"; then + # run_exitcode_got=0 + # else + # run_exitcode_got=$? + # fi + # as FreeBSD /bin/sh chokes on it when `set -e' is on. + if (eval "exec $run_evald_cmd"); then + run_exitcode_got=0 + else + run_exitcode_got=$? + fi + if test x"$run_mix_stdout_and_stderr" = x"yes"; then + set -x + cat stdall + else + set -x + cat stderr >&2 + cat stdout + fi + # Apparently, the `set +x' with redirected stderr is executed in a + # subshell by Solaris Sh, so repeat it without redirection. + { set +x; } 2>/dev/null; set +x + case $run_exitcode_expected in + RETURN|return) + restore_shell_flags 'run_command' + return $run_exitcode_got # RETURN POINT HERE + ;; + IGNORE|ignore|IGNORED|ignored|$run_exitcode_got) + run_ok=yes + ;; + FAIL|fail|FAILURE|failure) + if test $run_exitcode_got -gt 0; then + run_ok=yes + else + run_ok=no + fi + ;; + *) + run_ok=no + ;; + esac + echo "run_command: exit status $run_exitcode_got (expecting" \ + "$run_exitcode_expected)" + restore_shell_flags 'run_command' + test x"$run_ok" = x"yes" || Exit 1 +} # AUTOMAKE_run status [options...] # -------------------------------- -# Run Automake with OPTIONS, and fail if automake +# Run Automake with OPTIONS, and cause the test to FAIL if automake # does not exit with STATUS. AUTOMAKE_run () { - expected_exitcode=$1 + am_run_expected_exitcode=$1 shift - exitcode=0 - $AUTOMAKE ${1+"$@"} >stdout 2>stderr || exitcode=$? - cat stderr >&2 - cat stdout - test $exitcode = $expected_exitcode || Exit 1 + run_command -e $am_run_expected_exitcode -- $AUTOMAKE ${1+"$@"} } # AUTOMAKE_fails [options...] # --------------------------- -# Run Automake with OPTIONS, and fail if automake -# does not exit with STATUS. +# Run Automake with OPTIONS, and cuse the test to FAIL if automake +# does not exit with status 1. AUTOMAKE_fails () { AUTOMAKE_run 1 ${1+"$@"} diff --git a/tests/depcomp2.test b/tests/depcomp2.test index 4653c90..0daf2f0 100755 --- a/tests/depcomp2.test +++ b/tests/depcomp2.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2001, 2002 Free Software Foundation, Inc. +# Copyright (C) 2001, 2002, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -20,6 +20,8 @@ required=gcc . ./defs || Exit 1 +set -e + cat > configure.in << 'END' AC_INIT(subdir/foo.c) AM_INIT_AUTOMAKE(nonesuch, nonesuch) @@ -45,8 +47,8 @@ END CFLAGS= export CFLAGS -$ACLOCAL || Exit 1 -$AUTOMAKE --add-missing || Exit 1 -$AUTOCONF || Exit 1 -./configure CC='gcc' 2>error.log -test -z "`cat error.log`" +$ACLOCAL +$AUTOMAKE --add-missing +$AUTOCONF +run_command ./configure CC='gcc' +test ! -s stderr diff --git a/tests/depcomp4.test b/tests/depcomp4.test index 7d22894..70cbd57 100755 --- a/tests/depcomp4.test +++ b/tests/depcomp4.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2003 Free Software Foundation, Inc. +# Copyright (C) 2003, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -51,7 +51,6 @@ $AUTOMAKE -a grep am_cv_CC_dependencies_compiler_type configure ./configure am_cv_CC_dependencies_compiler_type=makedepend -$MAKE >output 2>&1 -cat output -grep 'unknown directive' output && Exit 1 +run_command -m $MAKE +grep 'unknown directive' stdall && Exit 1 $MAKE grepdeps diff --git a/tests/depend5.test b/tests/depend5.test index 0d9be3b..52e4169 100755 --- a/tests/depend5.test +++ b/tests/depend5.test @@ -65,10 +65,7 @@ for arg in Makefile \ "--file=$name" do rm -rf .deps _deps - ./config.status "$arg" depfiles >stdout 2>stderr || - { cat stdout; cat stderr >&2; Exit 1; } - cat stdout - cat stderr >&2 + run_command ./config.status "$arg" depfiles grep '[Nn]o such file' stderr && Exit 1 if test -n "$depdir"; then diff --git a/tests/depend6.test b/tests/depend6.test index de2adf9..5ce2208 100755 --- a/tests/depend6.test +++ b/tests/depend6.test @@ -39,9 +39,6 @@ END $ACLOCAL $AUTOMAKE $AUTOCONF -./configure --disable-dependency-tracking 2>stderr || { - stat=$?; cat stderr >&2; Exit $stat; -} -cat stderr >&2 -grep shift stderr && Exit 1 +run_command ./configure --disable-dependency-tracking +grep 'shift' stderr && Exit 1 : diff --git a/tests/discover.test b/tests/discover.test index 36293f7..c62f1b4 100755 --- a/tests/discover.test +++ b/tests/discover.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1996, 1997, 2000, 2001, 2002 Free Software Foundation, Inc. +# Copyright (C) 1996, 1997, 2000, 2001, 2002, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -34,5 +34,6 @@ END : > fsusage.c $ACLOCAL || Exit 1 -$AUTOMAKE 2>&1 | $FGREP fsusage.c || Exit 1 +run_command -e IGNORE -m $AUTOMAKE +$FGREP fsusage.c stdall || Exit 1 Exit 0 diff --git a/tests/distcom3.test b/tests/distcom3.test index 04bf365..fdcc31a 100755 --- a/tests/distcom3.test +++ b/tests/distcom3.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2001, 2002, 2003, 2004, 2006 Free Software Foundation, Inc. +# Copyright (C) 2001, 2002, 2003, 2004, 2006, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -34,9 +34,8 @@ END $ACLOCAL # Should not warn about missing README, since it is a target. -$AUTOMAKE --add-missing --gnu >stdout 2>&1 -cat stdout -grep README stdout && Exit 1 +run_command -m $AUTOMAKE --add-missing --gnu +grep README stdall && Exit 1 sed -n -e '/^DIST_COMMON =.*\\$/ { :loop diff --git a/tests/fn99.test b/tests/fn99.test index 982a989..611c767 100755 --- a/tests/fn99.test +++ b/tests/fn99.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2004, 2008 Free Software Foundation, Inc. +# Copyright (C) 2004, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -47,8 +47,7 @@ do touch x done) || Exit 77 -$MAKE dist 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_command -e FAIL $MAKE dist grep 'filenames are too long' stderr test 2 = `grep 12345678 stderr | wc -l` : diff --git a/tests/fn99subdir.test b/tests/fn99subdir.test index 465f2af..7055e85 100755 --- a/tests/fn99subdir.test +++ b/tests/fn99subdir.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2006, 2008 Free Software Foundation, Inc. +# Copyright (C) 2006, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -73,8 +73,7 @@ do touch x done) -$MAKE dist 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_command -e FAIL $MAKE dist grep 'filenames are too long' stderr test 1 = `grep 12345678 stderr | wc -l` : diff --git a/tests/gnits2.test b/tests/gnits2.test index 4c5d54f..a85b096 100755 --- a/tests/gnits2.test +++ b/tests/gnits2.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2002, 2003, 2004, 2007 Free Software Foundation, Inc. +# Copyright (C) 2002, 2003, 2004, 2007, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -105,6 +105,6 @@ cd build ../configure "--prefix=`pwd`/../inst-dir" --program-prefix=p $MAKE all $MAKE test-install -$MAKE -k installcheck 2>stderr || : # Never trust the exit status of make -k. -cat stderr +# Never trust the exit status of make -k. +run_command -e IGNORE $MAKE -k installcheck $MAKE grep-stderr diff --git a/tests/gnits3.test b/tests/gnits3.test index 7a08fb8..4d90684 100755 --- a/tests/gnits3.test +++ b/tests/gnits3.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2002, 2007 Free Software Foundation, Inc. +# Copyright (C) 2002, 2007, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -83,8 +83,8 @@ cd build ../configure "--prefix=`pwd`/../inst-dir" --program-prefix=p $MAKE $MAKE install -$MAKE -k installcheck 2>stderr || : # Never trust the exit status of make -k. -cat stderr +# Never trust the exit status of make -k. +run_command -e IGNORE $MAKE -k installcheck $MAKE grep-stderr # Make sure there is no more error when all targets are exempted. diff --git a/tests/help.test b/tests/help.test index d4d7110..cae8967 100755 --- a/tests/help.test +++ b/tests/help.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2003, 2008 Free Software Foundation, Inc. +# Copyright (C) 2003, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -32,7 +32,7 @@ $AUTOMAKE --version $AUTOMAKE --help # aclocal and automake cannot work without configure.ac or configure.in -$ACLOCAL 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e 1 $ACLOCAL grep configure.ac stderr grep configure.in stderr AUTOMAKE_fails diff --git a/tests/init.test b/tests/init.test index 38ec681..b6a7375 100755 --- a/tests/init.test +++ b/tests/init.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2006, 2008 Free Software Foundation, Inc. +# Copyright (C) 2006, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -27,7 +27,17 @@ AM_INIT_AUTOMAKE END # The error message should mension AC_INIT, not AC_PACKAGE_VERSION. -($ACLOCAL && $AUTOCONF) 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +if run_command -e RETURN $ACLOCAL; then + success=yes +else + success=no +fi +if test x"$success" != x"no"; then + mv stderr stder1 + run_command -e RETURN $AUTOCONF || success=no + cat stderr >>stder1 + mv -f stder1 stderr +fi +test x"$success" = x"no" || Exit 1 grep AC_PACKAGE_VERSION stderr && Exit 1 -grep AC_INIT stderr +grep AC_INIT stderr || Exit 1 diff --git a/tests/instsh.test b/tests/instsh.test index 61f5f9e..fe3d485 100755 --- a/tests/instsh.test +++ b/tests/instsh.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1996, 1999, 2001, 2002, 2003 Free Software Foundation, Inc. +# Copyright (C) 1996, 1999, 2001, 2002, 2003, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -32,10 +32,9 @@ mv Makefile.am configure.in frob/ cd frob $ACLOCAL -$AUTOMAKE --add-missing > output 2>&1 +run_command -m $AUTOMAKE --add-missing # Only one `/' should appear in the output. -cat output -grep '/.*/' output && Exit 1 +grep '/.*/' stdall && Exit 1 test -f install-sh diff --git a/tests/ldadd.test b/tests/ldadd.test index 005f9d8..0a9907d 100755 --- a/tests/ldadd.test +++ b/tests/ldadd.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1997, 1999, 2000, 2001, 2002 Free Software Foundation, Inc. +# Copyright (C) 1997, 1999, 2000, 2001, 2002, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -39,5 +39,6 @@ END : > q.c $ACLOCAL || Exit 1 -$AUTOMAKE 2>&1 | grep LDADD || Exit 1 +run_command -e 1 -m $AUTOMAKE +grep LDADD stdall || Exit 1 Exit 0 diff --git a/tests/libtool7.test b/tests/libtool7.test index a2f912f..f8706d9 100755 --- a/tests/libtool7.test +++ b/tests/libtool7.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2003, 2004, 2007 Free Software Foundation, Inc. +# Copyright (C) 2003, 2004, 2007, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -82,19 +82,16 @@ $AUTOCONF $AUTOMAKE --add-missing --copy ./configure "--prefix=`pwd`/_inst" -env LIBTOOLFLAGS=--silent $MAKE print >output 2>&1 -cat output -grep '1BEG: libmod1.la mod2.la :END1' output -grep '2BEG: mod2.la :END2' output -grep '3BEG: .*silent.*silent.* :END3' output +run_command -m env LIBTOOLFLAGS=--silent $MAKE print +grep '1BEG: libmod1.la mod2.la :END1' stdall +grep '2BEG: mod2.la :END2' stdall +grep '3BEG: .*silent.*silent.* :END3' stdall test 2 -le `grep mod2_la_LIBTOOLFLAGS Makefile | wc -l` $MAKE -env LIBTOOLFLAGS=--silent $MAKE install >output 2>&1 -cat output -grep 'silent.*silent.*prg' output -grep 'silent.*silent.*libmod1' output +run_command -m env LIBTOOLFLAGS=--silent $MAKE install +grep 'silent.*silent.*prg' stdall +grep 'silent.*silent.*libmod1' stdall -env LIBTOOLFLAGS=--silent $MAKE uninstall >output 2>&1 -cat output -grep 'silent.*silent.*libmod1' output +run_command -m env LIBTOOLFLAGS=--silent $MAKE uninstall +grep 'silent.*silent.*libmod1' stdall diff --git a/tests/libtool9.test b/tests/libtool9.test index c0d9688..94e617b 100755 --- a/tests/libtool9.test +++ b/tests/libtool9.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2005 Free Software Foundation, Inc. +# Copyright (C) 2005, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -88,10 +88,11 @@ $AUTOCONF $AUTOMAKE --add-missing --copy ./configure -env LDFLAGS=ldflags AM_LDFLAGS=am_ldflags libmod1_la_LDFLAGS=lm1_la_ldflags \ +run_command -m env \ + LDFLAGS=ldflags AM_LDFLAGS=am_ldflags libmod1_la_LDFLAGS=lm1_la_ldflags \ CFLAGS=cflags AM_CFLAGS=am_cflags prg2_CFLAGS=prg2_cflags \ - $MAKE -e print >output 2>&1 -cat output + $MAKE -e print +mv stdall output grep '1BEG: libmod1.la mod2.la :END1' output grep '2BEG: mod2.la :END2' output grep '3BEG:.* am_cflags cflags .*lm1_la_ldflags ldflags.* :END3' output diff --git a/tests/lisp3.test b/tests/lisp3.test index 809d491..086173d 100755 --- a/tests/lisp3.test +++ b/tests/lisp3.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2003, 2008 Free Software Foundation, Inc. +# Copyright (C) 2003, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -61,9 +61,8 @@ test -f am-one.elc unique=0a3346e2af8a689b85002b53df09142a $sleep echo "(message \"$unique\")(provide 'am-three)" > am-three.el -$MAKE >output 2>&1 -cat output -grep $unique output +run_command -m $MAKE +grep $unique stdall # It should also work for VPATH-builds. $MAKE distcheck diff --git a/tests/ltorder.test b/tests/ltorder.test index 0f69211..1e6a03a 100755 --- a/tests/ltorder.test +++ b/tests/ltorder.test @@ -1,5 +1,5 @@ #!/bin/sh -# Copyright (C) 2008 Free Software Foundation, Inc. +# Copyright (C) 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -53,8 +53,7 @@ $AUTOMAKE --add-missing ./configure "--prefix=`pwd`/inst" $MAKE -$MAKE install 2>stderr || { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_command $MAKE install grep 'has not been installed' stderr && Exit 1 $MAKE uninstall diff --git a/tests/man4.test b/tests/man4.test index aa86a9e..7424c72 100755 --- a/tests/man4.test +++ b/tests/man4.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2008 Free Software Foundation, Inc. +# Copyright (C) 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -88,11 +88,9 @@ mkdir build cd build ../configure $MAKE -$MAKE dist 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_command -e FAIL $MAKE dist grep 'install help2man' stderr -$MAKE distcheck 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_command -e FAIL $MAKE distcheck grep 'install help2man' stderr rm -f ../help2man diff --git a/tests/mdate.test b/tests/mdate.test index 704b393..e63bff9 100755 --- a/tests/mdate.test +++ b/tests/mdate.test @@ -1,5 +1,5 @@ #!/bin/sh -# Copyright (C) 1996, 1997, 2001, 2002 Free Software Foundation, Inc. +# Copyright (C) 1996, 1997, 2001, 2002, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -31,4 +31,5 @@ END : > texinfo.tex $ACLOCAL || Exit 1 -$AUTOMAKE 2>&1 > /dev/null | grep 'required file.*mdate-sh' +run_command -e IGNORED $AUTOMAKE +grep 'required file.*mdate-sh' stderr diff --git a/tests/missing3.test b/tests/missing3.test index 07ef779..9b48368 100755 --- a/tests/missing3.test +++ b/tests/missing3.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2004, 2008 Free Software Foundation, Inc. +# Copyright (C) 2004, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -22,24 +22,24 @@ set -e # b7cb8259 assumed not to exist. -./missing b7cb8259 --version 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e FAIL ./missing b7cb8259 --version grep . stderr && Exit 1 -./missing b7cb8259 --grep 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e FAIL ./missing b7cb8259 --grep grep WARNING stderr -./missing --run b7cb8259 --version && Exit 1 -./missing --run b7cb8259 --grep 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e FAIL ./missing --run b7cb8259 --version +run_command -e FAIL ./missing --run b7cb8259 --grep grep WARNING stderr # missing itself it known to exist :) -./missing ./missing --version 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e FAIL ./missing ./missing --version grep . stderr && Exit 1 -./missing ./missing --grep 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e FAIL ./missing ./missing --grep grep WARNING stderr -./missing --run ./missing --version 2>stderr +run_command ./missing --run ./missing --version grep . stderr && Exit 1 -./missing --run ./missing --grep 2>stderr && { cat stderr >&2; Exit 1; } +run_command -e FAIL ./missing --run ./missing --grep grep WARNING stderr && Exit 1 grep Unknown stderr diff --git a/tests/missing4.test b/tests/missing4.test index 1015828..c6c3b7e 100755 --- a/tests/missing4.test +++ b/tests/missing4.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2006, 2007, 2008 Free Software Foundation, Inc. +# Copyright (C) 2006, 2007, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -36,13 +36,11 @@ cmp aclocal.m4 aclocal.tmp && Exit 1 mv aclocal.tmp aclocal.m4 -$MAKE 2>stderr -cat stderr +run_command $MAKE grep 'You have another version of autoconf' stderr grep -c 'aclocal.m4:.*this file was generated for' stderr -$MAKE 2>stderr -cat stderr +run_command $MAKE grep 'You have another version of autoconf' stderr && Exit 1 grep 'aclocal.m4:.*this file was generated for' stderr && Exit 1 diff --git a/tests/missing6.test b/tests/missing6.test index b63a0d1..bf6df2d 100755 --- a/tests/missing6.test +++ b/tests/missing6.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2008 Free Software Foundation, Inc. +# Copyright (C) 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -31,7 +31,7 @@ rm -f configure.in touch Makefile.am $ACLOCAL -$AUTOCONF 2>stderr +run_command $AUTOCONF grep 'You have another version of autoconf' stderr grep 'aclocal.m4:.*this file was generated for' stderr $AUTOMAKE @@ -43,8 +43,7 @@ cmp configure.ac configure.tmp && Exit 1 mv configure.tmp configure.ac -$MAKE 2>stderr -cat stderr +run_command $MAKE grep 'You have another version of autoconf' stderr grep 'aclocal.m4:.*this file was generated for autoconf 9999a' stderr diff --git a/tests/obsolete.test b/tests/obsolete.test index e6601c9..daae69c 100755 --- a/tests/obsolete.test +++ b/tests/obsolete.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1996, 2001, 2002, 2007, 2008 Free Software Foundation, Inc. +# Copyright (C) 1996, 2001, 2002, 2007, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -27,7 +27,7 @@ END $PERL -ne '/AU_DEFUN\(\[(\w+)\]/ && print "$1\n"' \ "$testsrcdir/../m4/obsolete.m4" >> obs cat obs >> configure.in -$PERL -ne 'chomp; print "grep $_ stderr || Exit 1\n"; ' obs > obs.1 +$PERL -ne 'chomp; print "grep $_ stdall || Exit 1\n"; ' obs > obs.1 $PERL -ne 'chomp; print "grep $_ configure.in && Exit 1\n"; ' obs > obs.2 # Sanity check. Make sure we have added something to configure.in. @@ -36,7 +36,7 @@ test `cat configure.in | wc -l` -gt 1 || Exit 1 $ACLOCAL || Exit 1 # Expect Autoconf to complain about each of the macros in obs. -$AUTOCONF -Wobsolete >stderr 2>&1 +run_command -m $AUTOCONF -Wobsolete . ./obs.1 # Make sure Autoupdate remove each of these macros. $AUTOUPDATE || Exit 1 diff --git a/tests/output-order.test b/tests/output-order.test index 856baa2..852c194 100755 --- a/tests/output-order.test +++ b/tests/output-order.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2008 Free Software Foundation, Inc. +# Copyright (C) 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -43,13 +43,12 @@ END $ACLOCAL $AUTOCONF rm -f missing install-sh -$AUTOMAKE --add-missing --copy 2>stderr -cat stderr >&2 +run_command $AUTOMAKE --add-missing --copy +mv stderr stder0 for i in 1 2 3 4 5 6; do rm -f missing install-sh INSTALL COPYING + run_command -e IGNORE $AUTOMAKE --add-missing --copy # The grep prevents a Heisenbug with the HP-UX shell and VERBOSE=yes. - $AUTOMAKE --add-missing --copy 2>&1 >/dev/null | - grep -v /dev/null | - diff - stderr + grep -v /dev/null stder0 | diff - stderr done diff --git a/tests/pr220.test b/tests/pr220.test index 25e1f0e..6644d28 100755 --- a/tests/pr220.test +++ b/tests/pr220.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2001, 2002, 2004, 2008 Free Software Foundation, Inc. +# Copyright (C) 2001, 2002, 2004, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -55,6 +55,5 @@ $AUTOMAKE -a cd build # configure should fail since we've done something invalid. -../configure 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_command -e FAIL ../configure grep NEVER_TRUE stderr diff --git a/tests/pr300-ltlib.test b/tests/pr300-ltlib.test index 01814a9..7ed0a2f 100755 --- a/tests/pr300-ltlib.test +++ b/tests/pr300-ltlib.test @@ -48,8 +48,7 @@ $ACLOCAL $AUTOCONF $AUTOMAKE --copy --add-missing ./configure --prefix "`pwd`/inst" -$MAKE V=1 >stdout -cat stdout +run_command $MAKE V=1 grep 'liba.la .*-rpath .*lib' stdout grep 'liba.la .*-rpath .*lib/subdir' stdout && Exit 1 @@ -58,8 +57,7 @@ grep 'libb.la .*-rpath .*lib/subdir' stdout test -f subdir/liba.la test -f subdir/libb.la -$MAKE install 2>stderr -cat stderr >&2 +run_command $MAKE install grep 'remember.*--finish' stderr && Exit 1 test -f inst/lib/liba.la diff --git a/tests/python11.test b/tests/python11.test index 0f15f37..920de88 100755 --- a/tests/python11.test +++ b/tests/python11.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2004, 2006, 2008 Free Software Foundation, Inc. +# Copyright (C) 2004, 2006, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -35,9 +35,7 @@ EOF $ACLOCAL $AUTOCONF -./configure >stdout 2>stderr && { cat stdout; cat stderr >&2; Exit 1; } -cat stdout -cat stderr >&2 +run_command -e FAILURE ./configure grep 'checking for IShouldNotExist1' stdout grep 'checking for IShouldNotExist2' stdout grep 'no suitable Python interpreter found' stderr diff --git a/tests/python4.test b/tests/python4.test index 11bb3ed..3ff44af 100755 --- a/tests/python4.test +++ b/tests/python4.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2003, 2008 Free Software Foundation, Inc. +# Copyright (C) 2003, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -33,6 +33,5 @@ $AUTOCONF $AUTOMAKE --add-missing # Simulate no Python -./configure PYTHON=: 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_command -e FAILURE ./configure PYTHON=: grep 'no suitable Python interpreter found' stderr diff --git a/tests/python5.test b/tests/python5.test index 86f5863..5695876 100755 --- a/tests/python5.test +++ b/tests/python5.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2003, 2008 Free Software Foundation, Inc. +# Copyright (C) 2003, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -34,6 +34,5 @@ $ACLOCAL $AUTOCONF $AUTOMAKE --add-missing -./configure 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr >&2 +run_command -e FAILURE ./configure grep 'no suitable Python interpreter found' stderr diff --git a/tests/python6.test b/tests/python6.test index eb4fb4d..ed60e03 100755 --- a/tests/python6.test +++ b/tests/python6.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2003 Free Software Foundation, Inc. +# Copyright (C) 2003, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -34,6 +34,5 @@ $AUTOCONF $AUTOMAKE --add-missing # Simulate no Python -./configure PYTHON=: 2>stderr -cat stderr +run_command ./configure PYTHON=: grep 'GREP ME:' stderr diff --git a/tests/python7.test b/tests/python7.test index 62c1a1b..eb6a4dc 100755 --- a/tests/python7.test +++ b/tests/python7.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2003 Free Software Foundation, Inc. +# Copyright (C) 2003, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -34,6 +34,5 @@ $ACLOCAL $AUTOCONF $AUTOMAKE --add-missing -./configure 2>stderr -cat stderr +run_command ./configure grep 'GREP ME:' stderr diff --git a/tests/python8.test b/tests/python8.test index a50af22..5f100ab 100755 --- a/tests/python8.test +++ b/tests/python8.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2003 Free Software Foundation, Inc. +# Copyright (C) 2003, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -32,6 +32,5 @@ $ACLOCAL $AUTOCONF $AUTOMAKE --add-missing -./configure 2>stderr -cat stderr +run_command ./configure grep 'GREP ME' stderr diff --git a/tests/python9.test b/tests/python9.test index 9b23deb..c0fa7a5 100755 --- a/tests/python9.test +++ b/tests/python9.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2003 Free Software Foundation, Inc. +# Copyright (C) 2003, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -33,6 +33,5 @@ $ACLOCAL $AUTOCONF $AUTOMAKE --add-missing -./configure 2>stderr -cat stderr +run_command ./configure grep 'GREP ME' stderr diff --git a/tests/sanity.test b/tests/sanity.test index 1c96c78..1296381 100755 --- a/tests/sanity.test +++ b/tests/sanity.test @@ -36,14 +36,12 @@ cp ../install-sh ../missing . $ACLOCAL $AUTOCONF $AUTOMAKE -./configure 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr +run_command -e FAILURE ./configure grep 'unsafe absolute working directory' stderr cd .. mkdir build cd build -../unsafe$/configure 2>stderr && { cat stderr >&2; Exit 1; } -cat stderr +run_command -e FAILURE ../unsafe$/configure grep 'unsafe srcdir' stderr : diff --git a/tests/subobj.test b/tests/subobj.test index df297f4..9d7882d 100755 --- a/tests/subobj.test +++ b/tests/subobj.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1999, 2001, 2002, 2004 Free Software Foundation, Inc. +# Copyright (C) 1999, 2001, 2002, 2004, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -32,8 +32,7 @@ wish_SOURCES = generic/a.c generic/b.c END $ACLOCAL -$AUTOMAKE --add-missing 2>stderr -cat stderr +run_command $AUTOMAKE --add-missing # Make sure compile is installed, and that Automake says so. grep 'install.*compile' stderr test -f compile diff --git a/tests/subobj9.test b/tests/subobj9.test index 99fe6e8..1f68721 100755 --- a/tests/subobj9.test +++ b/tests/subobj9.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2002, 2004, 2005 Free Software Foundation, Inc. +# Copyright (C) 2002, 2004, 2005, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -74,8 +74,8 @@ if test -n "`./libtool --help | grep tag=TAG`"; then fi $MAKE -$MAKE distcheck 2>&1 | tee out +run_command -m $MAKE distcheck # GNU Make used to complain that the Makefile contained two rules # for `src/.dirstamp' and `.//src/.dirstamp'. -grep 'overriding commands' out && Exit 1 +grep 'overriding commands' stdall && Exit 1 : diff --git a/tests/unused.test b/tests/unused.test index a5b4da6..68d5613 100755 --- a/tests/unused.test +++ b/tests/unused.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 1998, 2002, 2003, 2008 Free Software Foundation, Inc. +# Copyright (C) 1998, 2002, 2003, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -29,5 +29,5 @@ AC_DEFUN([MACRO_1_2], echo 12) AC_DEFUN([MACRO_1_2_3], echo 123) END -$ACLOCAL 2> stderr || { cat stderr >&2; Exit 1; } -test -z "`cat stderr`" +run_command $ACLOCAL +test ! -s stderr diff --git a/tests/vars3.test b/tests/vars3.test index 3d10a90..9f11262 100755 --- a/tests/vars3.test +++ b/tests/vars3.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2002 Free Software Foundation, Inc. +# Copyright (C) 2002, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -38,8 +38,7 @@ EOF $ACLOCAL # Make sure this warning is print in the `portability' category. -$AUTOMAKE --warnings=no-error,none,portability 2>stderr -cat stderr +run_command $AUTOMAKE --warnings=no-error,none,portability # Lines number are printed in error message. # Use them to make sure errors are diagnosed against the right lines. diff --git a/tests/version8.test b/tests/version8.test index 91b11a0..ffe4a3e 100755 --- a/tests/version8.test +++ b/tests/version8.test @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2005, 2008 Free Software Foundation, Inc. +# Copyright (C) 2005, 2008, 2009 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -20,6 +20,5 @@ set -e echo 'AM_AUTOMAKE_VERSION([1.9])' >>configure.in -$ACLOCAL 2>stderr && { cat stderr >&2; Exit 0; } -cat stderr >&2 +run_command -e 1 $ACLOCAL $FGREP 'AM_INIT_AUTOMAKE([1.9])' stderr -- 1.6.5 |
|
|
Re: testsuite failures when test scripts are run with zsh
by Ralf Wildenhues
::
Rate this Message:
Hi Stefano,
* Stefano Lattarini wrote on Sat, Dec 05, 2009 at 03:09:34PM CET: > I have rebased my private branch against master, and also amended the > patch a bit (to deal with the test `cscope.test', which used a > `$MAKE 2>stderr' redirection that I missed). The updated patch is > attached. Thanks for continuously updating your patch; and sorry for not getting it in before 1.11.1/1.10.3. Your patch requires some portability fixes and some testing on several systems, and for the releases I decided to play safe and worry more about not getting any regressions in testing functionality than getting us to work well with another shell (which, almost by definition, isn't the only one on the system; if it were, you could likely invoke zsh as /bin/sh and it would work well with the current code). I will try to work on it soon though. I've also omitted a number of other patches that introduced new features; they are for master, not the release branches. Thanks again, Ralf |
|
|
Re: testsuite failures when test scripts are run with zsh
by Stefano Lattarini
::
Rate this Message:
At Tuesday 08 December 2009, Ralf Wildenhues <Ralf.Wildenhues@...> wrote:
> Hi Stefano, > > Thanks for continuously updating your patch; and sorry for not > getting it in before 1.11.1/1.10.3. Don't worry, I see that there are more important patches pending and more urgent work to be done. Anyway, in the meantime I had taken some further steps, and created a series of three patches (my original one, plus other releated two), which I will soon post in a new thread. Hope you are OK with this. > Your patch requires some portability fixes and some testing on > several systems, and for the releases I decided to play safe and > worry more about not getting any regressions in testing > functionality That's a good policy, and I fully agree with it. > than getting us to work well with another shell (which, almost by > definition, isn't the only one on the system; if it were, you could > likely invoke zsh as /bin/sh and it would work well with the > current code). > Mmhh... it seems that you are confusing the issue tackled by this patch with a different, minor problem due to Zsh redefining `$0' in sourced scripts (but your conclusion is mostly correct anyway, as I'll tell later). The problem with `$0' is admittedly a minor one, and a very simple workaround for it is already described in `tests/README' (changes introduced in commits 53fe020252642f4b923b8c77392e37e4c5b7038d and 674ee2f48686aa67507f9fa6dd67487f8a0e3b93 on master branch). There is also a different patch of mine (still pending) which automatically enables a workaround for this problem (Zsh 4.x only); it's posted in the thread at the following address (please don't stop at the first message, as I made a couple of ugly mistakes in my first two attempts): http://lists.gnu.org/archive/html/automake-patches/2009-11/msg00003.html The issue the current patch tries to address is described in details in this older post: http://lists.gnu.org/archive/html/bug-automake/2009-08/msg00020.html You can see that it's quite a different issue. But the good thing is that it shouldn't occur when /bin/sh is zsh, because (cut&paste from the linked post): "... the only systems I heard of that use zsh as /bin/sh were MacOS X 10.0 or 10.1, and google tells me that they used a 3.0.something version of zsh, which does not show the aforementioned bug." > I will try to work on it soon though. Take all the time you need. Thanks, Stefano |
| Free embeddable forum powered by Nabble | Forum Help |
