Commit graph

26645 commits

Author SHA1 Message Date
Stephen Morris
59d88ee192 [rt46602] Pass noclean option to Makefile via exported variable 2018-02-16 16:09:49 +00:00
Stephen Morris
23dfa27a84 [rt46602] Mention conf.sh.win32 in instructions for adding a test. 2018-02-16 16:09:49 +00:00
Michał Kępień
f0721de624 [rt46602] Fix various typos, formatting issues and stylistic nits 2018-02-16 16:09:49 +00:00
Michał Kępień
42cf18d239 [rt46602] Remove redundant variable assignment 2018-02-16 16:09:49 +00:00
Michał Kępień
1ac4c3e4e5 [rt46602] named.port is no longer created by setup.sh for allow_query 2018-02-16 16:09:49 +00:00
Michał Kępień
fe1ccafc34 [rt46602] Consistently print usage for runall.sh to stderr 2018-02-16 16:09:49 +00:00
Michał Kępień
0dcee1cad3 [rt46602] Clean up getopts usage
Ensure case clauses are in sync with the string passed to getopts.
Remove catch-all clauses as they will never be evaluated.
2018-02-16 16:09:49 +00:00
Michał Kępień
fe51c34df3 [rt46602] Assign default values to variables set by command line options and consistently treat them as integers 2018-02-16 16:09:49 +00:00
Michał Kępień
1960de5530 [rt46602] named.port cannot be ignored by git because it is required to be present by notify/ns4 2018-02-16 16:09:49 +00:00
Michał Kępień
344d05063f [rt46602] Re-add clean.sh invocation to catz/setup.sh
Commit 57aa7b60fd caused catz/setup.sh to no longer call clean.sh, which
results in the catz system test failing on subsequent runs if the first
run is interrupted or fails.
2018-02-16 16:09:49 +00:00
Stephen Morris
451b7309ca [rt46602] Remove option to format the printed start time of a test
There was too much incompatibility between operating systems as to
the allowed options for "date".
2018-02-16 16:09:49 +00:00
Stephen Morris
462766cd76 [rt46602] Escape line ends in shell script variable assignment
Escape the line ends in a multi-line variable assignment.  Under some
circumstances, substituting the variable caused syntax errors when
used as the list of values in a shell script "for" statement.
2018-02-16 16:09:49 +00:00
Stephen Morris
fb78f82ba1 [rt46602] Fix EXTRAPORT substitutions in copy_setport
Fix problems where all EXTRAPORT macros were being replaced by
the EXTRAPORT1 value. (Copy and paste issue.)
2018-02-16 16:09:49 +00:00
Stephen Morris
cdf18a5009 [rt46602] Minor documentation changes 2018-02-16 16:09:49 +00:00
Stephen Morris
7f2d41d4c4 [rt46602] Correct way allow_query test produces RNDC output 2018-02-16 16:09:49 +00:00
Stephen Morris
0855a8a6cc [rt46602] Simplify the cleanup code 2018-02-16 16:09:48 +00:00
Stephen Morris
f606b17dfb [rt46602] Extensions to README
Added descriptions of how nameservers are started and stopped
during the tests, and how the framework cleans up the files created.
2018-02-16 16:09:31 +00:00
Stephen Morris
b88b508b22 [rt46602] Miscellenous small changes 2018-02-16 15:19:44 +00:00
Stephen Morris
1f07c69f25 [rt46602] Handle echoing of multi-line output from tests 2018-02-16 15:19:44 +00:00
Stephen Morris
c12e06755f [rt46602] Rename "test" variable to "systest"
... to avoid confusion between the variable and the command of the
same name.
2018-02-16 15:19:44 +00:00
Stephen Morris
00bc29640b [rt46602] Add ability to prevent deletion of output files when running all tests
Some tests may need to examine all output files from all system tests.
Allow the deletion of these files to be inhibited if required.
2018-02-16 15:19:44 +00:00
Stephen Morris
244d1c30e2 [rt46602] Error messages before start of test are output to stderr
Error messages concerning the invocation of run.sh are output to
stderr.  Messages after the test has started are output to stdout.
2018-02-16 15:19:44 +00:00
Stephen Morris
a5cf6bd12a [rt46602] Ensure that output sent to stderr is routed to the output file 2018-02-16 15:19:44 +00:00
Stephen Morris
ce66752adc [rt46602] Minor tidying up of documentation 2018-02-16 15:19:44 +00:00
Stephen Morris
549a6c6bc8 [rt46602] Check prerequsities before network interfaces when running test. 2018-02-16 15:19:44 +00:00
Stephen Morris
f0f5b5f045 [rt46602] Minor formatting changes to Perl files 2018-02-16 15:19:44 +00:00
Stephen Morris
c344e1d768 [rt46603] Provide echostart() and echoend() to output test start/end messages. 2018-02-16 15:19:44 +00:00
Stephen Morris
055e5be9fd [rt46602] Ensure test output in systests.output is not mixed up
When running all the system tests, output from a test is sent to a
test.output file in the test directory.  These are combined in to
systests.output when the run finishes.
2018-02-16 15:19:44 +00:00
Stephen Morris
3c25e5fa02 [rt46602] Miscellaneous minor fixes
... for misspellings, typos, trailing spaces, duplicated lines etc.
2018-02-16 15:19:43 +00:00
Stephen Morris
edc7a18d84 [rt46602] Allocate 100 ports to each test 2018-02-16 15:19:43 +00:00
Stephen Morris
f5d8f07900 [rt46602] Pass port numbers to tests via environment variables 2018-02-16 15:19:43 +00:00
Stephen Morris
b2d90da0c2 [rt46602] Ensure parallel tests are in alphabetical order 2018-02-16 15:19:43 +00:00
Stephen Morris
af005cdbcf [rt46602] Expanded system tests README
Add more information on running the tests, together with a section
on how the tests are organised, aimed at new developers.
2018-02-16 15:19:43 +00:00
Stephen Morris
b24c2e11d8 [rt46602] More updates to the test running framework
Tidy up the stop/start files and make switch usage consistent. Also
tidy up the various "clean" targets in the Makefile.
2018-02-16 14:57:52 +00:00
Stephen Morris
517d990bd4 [rt46602] Modify some test files to cope with the "-p" flag 2018-02-16 14:57:51 +00:00
Stephen Morris
32fe6f7682 [rt46602] Tidy up run management
Miscellaneous tidying up of run management.  The most significant
change is that "runall.sh" now runs _all_ the tests, even the
ones that can run in parallel.  runsequential.sh is the script
to run tests that have not been converted to parallel running.
2018-02-16 14:57:51 +00:00
Stephen Morris
e0ff77f9d3 [rt46602] Assign block of ports for each test
Some tests use more ports than just the query and control ports.
Each test that can run in parallel with other tests is now assigned
a unique block of 10 ports.
2018-02-16 14:57:51 +00:00
Stephen Morris
07341693e1 [rt46602] Allow catz system test to run in parallel with other tests 2018-02-16 14:57:51 +00:00
Stephen Morris
78f2b9ca01 [rt46602] Miscellaneous changes to existing parallelised tests
Currently these tests are allow_query, rpzrecurse and serve-stale

1. Function to copy files and set port numbers renamed from copy_config
to copy_setports, as this is used to change the ports in Perl and Python
test scripts as well.
2. Changes to rpzrecurse/tests.sh to handle two calls to getopts (one to
parse port numbers, the other to parse rpzrecurse-specific options). Also
fixed various commands to use correct ports.
3. Updates to "clean.sh" scripts to ensure that all files created in the
test are removed.
2018-02-16 14:57:51 +00:00
Stephen Morris
e7429b124b [rt46602] Ensure that tests running in parallel use unique ports
Via an intermediate make file, tests that have been modified to be able
to run in parallel are assigned unique query and control port numbers
(other than 5300 and 9953 respectively).  Tests that have not yet been
modified all use ports 5300 and 9953, so must be run sequentially.
2018-02-16 14:57:51 +00:00
Stephen Morris
e1a69e5d1f [rt46602] Miscellaneous tidying up of code cherry-picked from experimental branch 2018-02-16 14:57:51 +00:00
Ondřej Surý
3b14d54fd7 [rt46602] Allow parallel run of new dynamic port tests and the old ones 2018-02-16 14:57:51 +00:00
Ondřej Surý
caaec1ef6d [rt46602] Put back original .gitignore files in the system tests 2018-02-16 14:57:51 +00:00
Ondřej Surý
2a435708e4 [rt46602] Adjust rpzrecurse for dynamic port usage
Resolve conflict in run.sh during cherry-pick from an experimental branch.
2018-02-16 14:57:51 +00:00
Ondřej Surý
e6802badc0 [rt46602] Add $SEDPORTS variable to help with @PORT@ and @CONTROLPORT@ replacement 2018-02-16 14:57:51 +00:00
Ondřej Surý
d2c8286163 [rt46602] Move most of ignored files to top of system tests directory 2018-02-16 14:57:51 +00:00
Ondřej Surý
53bade5ece [rt46602] Fix output from rndc reload 2018-02-16 14:57:51 +00:00
Ondřej Surý
bc276f6c05 [rt46602] Add .gitignore to allow_query 2018-02-16 14:57:51 +00:00
Ondřej Surý
7e3c96c867 [rt46602] Utilize echo_i to allow mixed test output 2018-02-16 14:57:51 +00:00
Ondřej Surý
278c72fef6 [rt46602] Prefix R:<result> as R:<test>:<result>
Resolve conflict in run.sh during cherry-pick from experimental branch.
2018-02-16 14:57:50 +00:00