Commit graph

41 commits

Author SHA1 Message Date
Evan Hunt
a8b583e78b test interfaces directly in runall.sh and parallel.mk 2018-02-26 14:58:16 +01:00
Evan Hunt
4c753f5880 Merge branch 'tidy-runall.sh-environment-variables' into 'master'
Tidy runall.sh environment variables

See merge request isc-projects/bind9!18
2018-02-24 15:43:22 -05:00
Ondřej Surý
843d389661 Update license headers to not include years in copyright in all applicable files 2018-02-23 10:12:02 +01:00
Evan Hunt
0c559199bf final cleanup
- add CHANGES note
- update copyrights and license headers
- add -j to the make commands in .gitlab-ci.yml to take
  advantage of parallelization in the gitlab CI process
2018-02-22 22:58:15 -08:00
Michał Kępień
d989d20fe5 Preserve values of environment variables which are already set
This enables the environment variables controlling run.sh behavior to be
permanently set in a working environment (e.g. to automatically force
colored output without using "-c" in each runall.sh invocation).

Relevant runall.sh command line arguments still have a higher priority.
2018-02-21 14:59:38 +01:00
Michał Kępień
a324031a82 Allow forcing colored system test output
As parallel.mk and runsequential.sh both pipe system test output through
"tee" (for the purpose of creating test.output), run.sh invoked from
these two files detects it is not writing to a terminal, which prevents
colored output from being generated.  Allow forcing colored output using
a new command line argument for runall.sh, "-c", which sets an
environment variable (SYSTEMTEST_FORCE_COLOR) causing conf.sh to
unconditionally enable colored output.

The same environment variable can also be used directly to force colored
output when using "make test" instead of runall.sh.
2018-02-21 14:59:35 +01:00
Michał Kępień
3862043879 Allow retaining system test output using an environment variable
Instead of exporting an environment variable containing a command line
argument (NOCLEAN="-n"), extend run.sh to handle a "boolean" environment
variable (SYSTEMTEST_NO_CLEAN) itself.  The former method is buggy
because the value of NOCLEAN is set in parallel.mk when that file is
first created, but it is not subsequently updated upon each test run
(because make considers parallel.mk to be up to date).

To retain backward compatibility, the "-n" command line argument for
run.sh is still supported (and has a higher priority than the relevant
environment variable).

The SYSTEMTEST_NO_CLEAN environment variable can also be used directly
to prevent cleanup when using "make test" instead of runall.sh.

Apart from fixing a bug, this simplifies the way runall.sh controls
run.sh behavior due to the Makefile being bypassed.  Direct processing
of environment variables in run.sh is more scalable in the long run,
given that the previously utilized technique, even with its
implementation fixed, would still require Makefile.in to be modified in
two places each time a new flag needed to be passed from runall.sh to
run.sh.
2018-02-21 14:59:33 +01:00
Stephen Morris
7bb9a97904 [rt46602] Changes to get tests running on Windows
1. Track changes to conf.sh.in in conf.sh.win32
2. Modifications to prevent Windows "Configure" script replacing
   the sed "@PORT@" substitution tokens in conf.sh.win32.
3. runall.sh now runs Windows tests sequentially
2018-02-16 16:14:41 +00:00
Michał Kępień
2c519531a7 [rt46602] Export environment variables in a portable manner 2018-02-16 16:14:40 +00:00
Stephen Morris
59d88ee192 [rt46602] Pass noclean option to Makefile via exported variable 2018-02-16 16:09:49 +00:00
Michał Kępień
f0721de624 [rt46602] Fix various typos, formatting issues and stylistic nits 2018-02-16 16:09:49 +00:00
Michał Kępień
fe1ccafc34 [rt46602] Consistently print usage for runall.sh to stderr 2018-02-16 16:09:49 +00:00
Michał Kępień
0dcee1cad3 [rt46602] Clean up getopts usage
Ensure case clauses are in sync with the string passed to getopts.
Remove catch-all clauses as they will never be evaluated.
2018-02-16 16:09:49 +00:00
Stephen Morris
cdf18a5009 [rt46602] Minor documentation changes 2018-02-16 16:09:49 +00:00
Stephen Morris
00bc29640b [rt46602] Add ability to prevent deletion of output files when running all tests
Some tests may need to examine all output files from all system tests.
Allow the deletion of these files to be inhibited if required.
2018-02-16 15:19:44 +00:00
Stephen Morris
055e5be9fd [rt46602] Ensure test output in systests.output is not mixed up
When running all the system tests, output from a test is sent to a
test.output file in the test directory.  These are combined in to
systests.output when the run finishes.
2018-02-16 15:19:44 +00:00
Stephen Morris
32fe6f7682 [rt46602] Tidy up run management
Miscellaneous tidying up of run management.  The most significant
change is that "runall.sh" now runs _all_ the tests, even the
ones that can run in parallel.  runsequential.sh is the script
to run tests that have not been converted to parallel running.
2018-02-16 14:57:51 +00:00
Stephen Morris
e7429b124b [rt46602] Ensure that tests running in parallel use unique ports
Via an intermediate make file, tests that have been modified to be able
to run in parallel are assigned unique query and control port numbers
(other than 5300 and 9953 respectively).  Tests that have not yet been
modified all use ports 5300 and 9953, so must be run sequentially.
2018-02-16 14:57:51 +00:00
Ondřej Surý
278c72fef6 [rt46602] Prefix R:<result> as R:<test>:<result>
Resolve conflict in run.sh during cherry-pick from experimental branch.
2018-02-16 14:57:50 +00:00
Mark Andrews
0c27b3fe77 4401. [misc] Change LICENSE to MPL 2.0. 2016-06-27 14:56:38 +10:00
Tinderbox User
4688741c5c update copyright notice / whitespace 2015-12-16 23:45:26 +00:00
Mark Andrews
ecfedec0e0 perform a more complete cleanup after running system tests [rt41255] 2015-12-16 11:29:18 +11:00
Evan Hunt
60988462e5 [master] use posix-compatible shell in system tests
3839.	[test]		Use only posix-compatible shell in system tests.
			[RT #35625]
2014-05-06 22:06:04 -07:00
Tinderbox User
8ab8cd1fa6 update copyright notice 2014-03-01 23:46:15 +00:00
Evan Hunt
ec88c1fdff [master] capture stderr in systests.output
- also tidied up runall.sh summary output
2014-02-28 21:59:28 -08:00
Tinderbox User
a3128c1995 update copyright notice 2012-06-29 23:45:57 +00:00
Mark Andrews
bf8267aa45 reverse bad copyright update 2012-06-29 11:39:47 +10:00
Tinderbox User
247bf37860 update copyright notice 2012-06-29 01:22:18 +00:00
Mark Andrews
d884e8ee4d set status to 1 on R:FAIL 2011-12-19 23:08:50 +00:00
Automatic Updater
89d1324270 update copyright notice 2011-11-01 23:47:00 +00:00
Evan Hunt
e2271ee953 3189. [test] Added a summary report after system tests. [RT #25517] 2011-11-01 18:35:53 +00:00
Automatic Updater
4d42b714be update copyright notice 2010-03-04 23:50:34 +00:00
Mark Andrews
56c2c3835f 10.53.0.1 through 10.53.0.5 -> 10.53.0.1 through 10.53.0.7 2010-03-04 20:34:16 +00:00
Automatic Updater
70e5a7403f update copyright notice 2007-06-19 23:47:24 +00:00
Automatic Updater
ec5347e2c7 update copyright notice 2007-06-18 23:47:57 +00:00
Mark Andrews
dafcb997e3 update copyright notice 2004-03-05 05:14:21 +00:00
Andreas Gustafsson
f8d86ef2f6 if network interfaces are not not configured,
urge the user to configure them [RT #1757]
2001-09-22 01:10:41 +00:00
Brian Wellington
499b34cea0 copyright update 2001-01-09 22:01:04 +00:00
David Lawrence
40f53fa8d9 Trailing whitespace trimmed. Perhaps running "perl util/spacewhack.pl in your
own CVS tree will help minimize CVS conflicts.  Maybe not.
Blame Graff for getting me to trim all trailing whitespace.
2000-08-01 01:33:37 +00:00
David Lawrence
15a4474541 word wrap copyright notice at column 70 2000-07-27 09:55:03 +00:00
Andreas Gustafsson
2c9fdfdfbe added script for running all system tests in sequence 2000-07-06 19:32:37 +00:00