Oxs regression tests: oxsregression

The oxsregression runs a test suite for the Oxs solver. For each test, an instance of boxsi is run and the results are compared against reference results stored in subdirectories under oommf/app/oxs/regression_tests/.

Launching
The oxsregression launch command is:

tclsh oommf.tcl runtests [-autoadd] [-alttestdir] [-cleanup] [-ignoreextra]
   [-keepfail] [-listtests] [-loglevel level] [-noexcludes] [-parallel n]
   [-resultsfile stemname] [-showoutput] [-sigfigs digits] [-threads count]
   [-timeout seconds] [-updaterefdata] [-v] [testa testb ...]

where

-autoadd

Automatically adds new tests from MIF files found in the examples directory oommf/app/oxs/examples/.

-alttestdir

Specify an alternative test directory to use in place of the default directory list oommf/app/oxs/examples/, oommf/app/oxs/regression_tests/bug_tests/, and oommf/app/oxs/regression_tests/local_tests/.

-cleanup

If oxsregression is killed or crashes mid-run, then some temporary result files may be left on disk. This command searches for and offers to delete these stray files.

-ignoreextra

Ignore extra columns, if any, in test results as compared to reference results. This is useful in development work when changes to a MIF file introduce additional data table output.

-keepfail

Keep results from failed tests. Normally test results are automatically deleted.

-listtests

List all selected tests and exit without running any tests.

-loglevel level

Controls the amount of log information written to oxsregression.log in the regression test directory oommf/app/oxs/regression_tests/. The default setting is 0.

-noexcludes

Some tests suffer from various numerical problems. These are excluded from testing, unless this option is specified.

-parallel n

Run n tests concurrently, with default n=1. This option is only available when using Tcl 8.6 or later.

-resultsfile stemname

Test results are written to temporary files; by default these files have the stem oxsregression-test-output. If oxsregression is run simultaneously, perhaps on different machines on a shared file system, then overwriting of files from one process can interfere with the processing by another. The -resultsfile option can be used to cordon off results between simultaneous runs.

-showoutput

If this switch is not specified, then stdout and stderr output from boxsi is swallowed by oxsregression.

-sigfigs digits

Number of significant (decimal) digits to use in comparing test to reference results; the default setting is eight.

-threads count

Number of threads to run boxsi with. This option is available for threaded builds only. The default is the default thread count for boxsi.

-timeout seconds

Maximum number of seconds to wait for one test to finish; any individual test that fails to complete within this time span is summarily terminated. The default time is 150 seconds; use 0 to indicate no timeout.

-updaterefdata

For developer use only; this option causes the reference results to be replaced (overwritten) with new test results.

-v

Enable verbose output.

testa testb ...

Tests to run, with glob-style wildcards (*, ?) accepted. If no tests are specified then all (non-excluded) tests are selected. Subtest selection must be quoted with the test to appear as a single argument, e.g., "exch6ngbr 1,7,9". If no subtests are specified then all subtests are run.

Take notice of the interplay between the -parallel n and -threads count options. The former is the number of tests run in parallel, and the latter is the number of computation threads active in each test run. The total number of threads active at one time can therefore be as many as n x count.

OOMMF Documentation Team
September 30, 2022