We do hourly test builds of the bind9 tree.  This is an attempt to
document how they work.


* How things work

The scripts driving the build system are in ~wpk/b9t.  They are not
under CVS control.  The builds are driven by cron jobs separately
installed on each build system, running as user wpk.  Here is a
relevant extract from wpk's crontab on bb.rc.vix.com (reformatted to
fit in 80 columns):

The sources are checked out, and the web reports are generated,
on bb, as driven by the following cron jobs:

    #
    # check out the current bind 9 version and make the source tarball
    #
    45 2-21 * * * PLATFORM=BSD-3.1 && . $HOME/b9t/hosts/$PLATFORM/env && \
        nice make PLATFORM=$PLATFORM -e -f $HOME/b9t/bin/b9t.mk tarsrc \
        > $HOME/b9t/hosts/$PLATFORM/b9t-status 2>&1

    #
    # run the bind 9 build status report generator
    #
    30 3-22 * * * perl $HOME/b9t/bin/b9status.pl \
         > /proj/build-reports/bind9/bind9.html 2> /dev/null


Each host has a separate crontab entry for building the server and
running tests.  Here are examples from bb and sol:

    #
    # build the BSD-3.1 version of bind 9
    #
    0 3-22 * * * $HOME/b9t/bin/b9t.cron BSD-3.1

    #
    # bind 9 build for Solaris 5.6
    #
    0 3-22 * * * $HOME/b9t/bin/b9t.cron SunOS-5.6

Do not confuse the shell script ~wpk/b9t/bin/b9t.cron with the crontab
template (?) ~wpk/b9t/b9t.cron.  Although they have the same name,
they are not related.  The shell script b9t.cron then calls make,
using the makefile b9t.mk in the same location.  This makefile moves
the old status files out of the way and runs through the tests.

The actual builds take place in a directory whose location differs
among systems.  On most of them, it's on a local disk, under /build.
On some, it's on NFS; in this case the location is defined in
~wpk/b9t/hosts/$PLATFORM/env.

The output from the make process is in
~wpk/b9t/hosts/$PLATFORM/b9t-status, and the output from
The output from the later stages of the process is under
/proj/build-reports/bind9/hosts/$PLATFORM.  To make the files 
harder to find (?), they have names starting with a period:

  .populate
  .config
  .build
  .test


* Common problems

Sometime named processes fail to die when the tests are done,
interfering with the next test.  Just kill them.

On hp.rc.vix.com, the tests often fail because of NFS I/O errors.
When this happens, the machine needs to be rebooted.  It will not
come up again without manually entering commands on the console.

On bb, the tests sometimes fail because .nfs* files stuck in the build
tree keep it from being completely deleted when the next test runs.
The .nfs* files cannot be deleted, but they can be moved, so one way
of fixing this is to move them to ~wpk.