Automatic Program Test (Elk Test)

General

This documentation contains some general information on how to test a new version of the MCTDH program. The testing procedure is referred to as the elk-test , named after the famous test that showed up the fatal flaw in the Mercedes A-class. It is not a replacement for good programming practice and testing, but provides an automatic test on a standard set of input files which ensures that features of the program do not get lost by new developments.

Running the Test

The automatic test program is run by a shell script elk_test. To test a new version of the program set up a directory, type
elk_test prog_name
where prog_name is the name of the new code, and wait. Depending on the machine, and how the code is compiled, the test takes approximately 15 mins to 1 hour. If you are not sure whether the path to your new code is correctly set, better give the complete (i.e. absolute) name of the new code.

The script file is in the $MCTDH_DIR/bin directory of the package. If the command elk_test is not found, check that your path contains this directory. If not, add the path.

If there is a problem with finding files or programs, check that the variables in the elk_test script are set correctly:

testcode the name of the executable to be tested, e.g mctdh81.
ovlcode a version of the overlap program, e.g. overlap81.
compcode a version of the compare program, e.g. compare81.
comppath the path to the comparison set of test files, i.e. those that have been previously calculated by elk_test_gen and are known to be correct.

The script elk_test knows several options. If one types   elk_test -h one obtains:

Purpose: Test of the MCTDH program
Usage: elk_test [-h -c compname -o overlap -t compare -i inppath
                 -n number -r -b num -e num] progname
   -h:  Print this help text.
   -c:  Use "compname" as comparison directory.
        If "compname" ends with a slash, $MCTDH_PLATFORM will be appened.
   -o:  Use "overlap" program for overlaps.
   -t:  Use "compare" program for tests.
   -i:  Use test?.inp files from "inppath".
        If inppath=c, i.e. "-i c", comppath is used as inppath.
   -r:  Re-do tests that failed in a previous elk_test-run.
   -n:  Do tests only for the number(s) that follow -n.
        Quote the argument of -n if it contains blanks,
        e.g.:  -n 7   or   -n '3 7 21'
        With option -n elk_test will run the required tests
        regardless whether they already exist or not.
   -w:  Do tests only for wavepackets.
   -1:  Do tests only for density operators of Typ I.
   -2:  Do tests only for density operators of Typ II.
   -b:  Begin the test with the number following -b.
   -e:  End the test with the number following -e.

 Default overlap    : \
 Default compare    :  \ (run "elk_test -h" to see how these
 Default comparison :  /  are set for you)
 Default tests      : /


One may have to use the -c option if the directory which contains the data for comparison is not the default one. In fact, it is recommended that one adjust the default setting to ones demand before runing elk_test the first time. The defaults are set at the top of the shell script.
If some test did fail, fix the code and re-run elk_test. If the name-directory of a particular test already exists, elk_test will read and analyse the data, but will not run mctdh again, if a name-file already exisit. In order to re-do the failed tests use the -r or -n option.
If you want to perform a partial test, you may use the options -b (begin) and -e (end). I.e.   elk_test -b 3 -e 15 mctdh83   will run the tests 3,4,..,15. Test50 and higher are testing density operator propagation.

Results Given by the Test

The results are written to two files.

summary:

The title of the test is followed by the test results, which will be one of the following:

testXX: Passed hard limit
testXX: Passed *SOFT* limit
testXX: FAILED

Hard limit passed means that the maximum difference found in the numerical comparisons between the source code versions is < 10-12. The soft limit is < 10-6.

results:

This file contains more information on the various tests, e.g. CPU-time and memory required by the two versions, and more details on the numerical checks carried out. Thus if the limits are not met, one can look in the results file to see where the problem lies.

Checks Carried Out

The following comparisons are made.
  1. Overlaps between the wavefunctions. The average and maximum difference between the overlaps are calculated.
  2. The numbers in the auto files are compared.
  3. The numbers in the check file are compared.
  4. The numbers in the dicht file are compared.

Generating the Test Set

To generate the comparison set of files, set up a directory, and run the script
elk_test_gen prog_name
where prog_name is the name of the program to be used for the generation. The test set is generated using the input files in $MCTDH_DIR/elk_inputs. The elk_test_gen script also contains variables specifying the code to be used. The script file is in $MCTDH_DIR/bin. If the command elk_test_gen is not found, check that your path contains this directory. The output directory is the current directory with $MCTDH_PLATFORM appended if the pwd-path ends otherwise. elk_test_gen will make the output directory if it does not exist.

One may use options to change the paths of the input and output files. Running elk_test_gen -h yields:

Purpose: Generates a set of test files for the MCTDH program.
Usage:   elk_test_gen [-h] [-i inpname] [-o outname] progname
   -h:   Print this help text.
   -i:   Get input files from "inpname" directory.
   -o:   Write output in "outname" directory.
         If outname ends with a slash, $MCTDH_PLATFORM will be appened.
         The default outname is the current directory (with /$MCTDH_PLATFORM).

 The default setting reads:
 Potfit code: potfit81
 Input set  : /worka/dieter/mctdh81.102/elk_inputs
 Output path: /worka/dieter/mctdh81.102/elktests/i686


If one or some of the runs failed, fix the error and run elk_test_gen again. The script will not re-do succesfully installed sets.

Adding New Tests

To add new tests, simply set up a new test.inp file in the comparison code test directory, using the next available number, e.g. test31.inp. Wavefunction test may run from 1 to 49, density-operator test have numbers above 50. (there may be 'holes' in the list of tests). The .inp file must contain a 3 line title at the beginning. This is written to the output files. The title should start with ### Title of (see other test*.inp files). The psi, auto, and dicht files must be created. If any special checks are to be made, the program compare.F, found in $MCTDH_DIR/source/analyse, must be altered. Finally, add the title to the list in this document, $MCTDH_DIR/doc/mctdh/elk_docu.html

By typing elk_test_gen when in the comparison test directory the new test files will be generated. The elk_test can then be re-run, including this new comparison.

List of Tests

1:###           Test of NOCl1 / CMF integrator                   ###

 2:###           Test of NOCl1 / default VMF integrator           ###

 3:###           Test of NOCl1 / build option                     ###

 4:###           Test of NOCl1 / FFT basis-section                ###

 5:###           Test of NOCl1 with full potential                ###

 6:###           Test of NOCl1 with full potential / CDVR         ###

 7:###           Test of pyrazine 4-mode multi-set                ###

 8:###           Test of pyrazine 4-mode single-set               ###

 9:###           Test of pyrazine 4-mode interaction picture      ###

10:###  Test of Propagation of CH3I  (combined mode multi-packet version) ###

11:###     Test of    Relaxation of NOCl0                              ###

12:###     Test of    Relaxation of CH3I  (Single-packet version)      ###

13:###     Test of    Relaxation of CH3I  (Multi-packet version)       ###

14:###     Test of    Propagation of CH3I  (Single-packet version)     ###

15:###     Test of    Propagation of CH3I  (multi-packet version)      ###

16:###           Test of NOCl restart adding spfs                      ###

17:###           Test of NOCL1 continuation                            ###

18:###    Test of pyrazine 4-mode: CMF integrator + natural orbitals   ###

19:###    Test of pyrazine 4-mode: VMF integrator + natural orbitals   ###

20:###             Test of H+D2 (J=0) reactive scattering              ###
20:### 3D H+D2(nu=0) reactive scattering in Jacobian coordinates       ###
20:###   Adiabatic correction, natpot, sinus-dvr, leg-dvr, CMF,        ###

21:###    Test of H2/surface scattering.  sphfbr, fft periodic         ###

22:###  Test of Propagation of CH3I (multi-packet, CMF, h-proj option) ###

23:###    Test of H2/surface scattering.   test for autocap            ###

24:###                      Test of NOCl, multi-packet                 ###

25:###         Test of pyrazine 4-mode multi-set multi-packet          ###

26:###    Test of KLeg.  H+D2 reactive scattering (J>0)                ###
26:###      4D H+D2 reactive scattering in Jacobian coordinates        ###
26:###   Adiabatic correction, natpot, sinus-dvr, kleg-dvr,  CMF       ###

27:###  Test of expectation /  Butatriene linear 5-mode model          ###

28:###    Test of N2/surface scattering. PLeg, fft, exp -DVRs. CMF     ###

29:###     Test of   Relaxation=0 of CH3I  (Multi-packet version)      ###

30:###           Test of NOCl / Time-dependent operator                ###

31:###           Test of CO2 / CMF integrator                          ###

32:### Test of NOCl / Time-dep. operator / RK8 integrator & imp-ortho  ###

33:### Test of HONO, N=O excitation / Impr. Relaxation, lock, precon   ###



50:###  Test of: density2 / VMF / closed / Temp / Henon-Heiles         ###

51:###  Test of: density2 / VMF / open / Temp /   Henon-Heiles         ###

52:###  Test of density2 / Pyrazine 4 modes / CMF / s= 3               ###

53:###  Test of density2 / Pyrazine 4 modes / VMF / s=3                ###

54:###  Test of density2 / Pyrazine 4 modes / CMF / s=2 / Temperature  ###

55:###  Test of density2 / Pyrazine 4 modes / CMF / s=2 / linear       ###

56:###  Test of density2 / linear / Relax / NOCl S0 surface            ###

57:###  Test of density2 / linear /  Propagation of  NOCl1  to 10 fs.  ###

58:###  Test of density1 / VMF / closed / Temp / Henon-Heiles system   ###

59:###  Test of density1 / VMF / open /  Temp /  Henon-Heiles system   ###

60:###  Test of density1 / VMF / closed /  Temp /  Pyrazine 4 moden    ###

61:###  Test of density1 / VMF / open /     Pyrazine 4 moden           ###