Testing GPAW

Testing of gpaw is done by a nightly test suite consisting of many small and quick tests and by a weekly set of larger test.

Quick test suite

Use the gpaw command to run the tests:

$ gpaw test --help
usage: gpaw test [-h] [-x test1.py,test2.py,...] [-f] [--from TESTFILE]
                 [--after TESTFILE] [--range test_i.py,test_j.py] [-j JOBS]
                 [--reverse] [-k] [-d DIRECTORY] [-s] [--list]
                 [tests [tests ...]]

Run the GPAW test suite. The test suite can be run in parallel with MPI
through gpaw-python. The test suite supports 1, 2, 4 or 8 CPUs although some
tests are skipped for some parallelizations. If no TESTs are given, run all
tests supporting the parallelization.

positional arguments:

optional arguments:
  -h, --help            show this help message and exit
  -x test1.py,test2.py,..., --exclude test1.py,test2.py,...
                        Exclude tests (comma separated list of tests).
  -f, --run-failed-tests-only
                        Run failed tests only.
  --from TESTFILE       Run remaining tests, starting from TESTFILE
  --after TESTFILE      Run remaining tests, starting after TESTFILE
  --range test_i.py,test_j.py
                        Run tests in range test_i.py to test_j.py (inclusive)
  -j JOBS, --jobs JOBS  Run JOBS threads. Each test will be executed in serial
                        by one thread. This option cannot be used for
                        parallelization together with MPI.
  --reverse             Run tests in reverse order (less overhead with
                        multiple jobs)
  -k, --keep-temp-dir   Do not delete temporary files.
  -d DIRECTORY, --directory DIRECTORY
                        Run test in this directory
  -s, --show-output     Show standard output from tests.
  --list                list the full list of tests, then exit

A temporary directory will be made and the tests will run in that directory. If all tests pass, the directory is removed.

The test suite consists of a large number of small and quick tests found in the gpaw/test/ directory. The tests run nightly in serial and in parallel.

Adding new tests

A test script should fulfill a number of requirements:

  • It should be quick. Preferably a few seconds, but a few minutes is OK. If the test takes several minutes or more, consider making the test a big test.
  • It should not depend on other scripts.
  • It should be possible to run it on 1, 2, 4, and 8 cores.

A test can produce standard output and files - it doesn’t have to clean up. Remember to add the new test to list of all tests specified in the gpaw/test/__init__.py file.

Use this function to check results:

gpaw.test.equal(x, y, tolerance=0, fail=True, msg='')

Big tests

The directory in gpaw/test/big/ contains a set of longer and more realistic tests that we run every weekend. These are submitted to a queueing system of a large computer.

Adding new tests

To add a new test, create a script somewhere in the file hierarchy ending with agts.py (e.g. submit.agts.py). AGTS is short for Advanced GPAW Test System (or Another Great Time Sink). This script defines how a number of scripts should be submitted to niflheim and how they depend on each other. Consider an example where one script, calculate.py, calculates something and saves a .gpw file and another script, analyse.py, analyses this output. Then the submit script should look something like:

def workflow():
    from myqueue.job import Job
    return [Job('calculate.py', cores=8, tmax='25m'),
            Job('analyse.py', cores=1, tmax='5m', deps=['calculate.py'])

As shown, this script has to contain the definition of the function workflow(). See https://gitlab.com/jensj/myqueue for more details.