Automatic Program Test (Elk Test)

General

This documentation contains some general information on how to test a new version of the MCTDH program. The testing procedure is referred to as the elk-test , named after the famous test that showed up the fatal flaw in the Mercedes A-class. It is not a replacement for good programming practice and testing, but provides an automatic test on a standard set of input files which ensures that features of the program do not get lost by new developments. Note, before one can run the elk_test, one has to generate a comparison data set (if not already existing). The path of the comparison data set is given by the environment variable MCTDH_ELK, which usually is set in the script ~./mctdhrc .

Running the Test

The automatic test program is run by a shell script elk_test. To test a new version of the program set up a directory, type
elk_test prog_name
where prog_name is the name of the new code, and wait. Depending on the machine, and how the code is compiled, the test takes approximately 15 mins to 1 hour. If you are not sure whether the path to your new code is correctly set, better give the complete (i.e. absolute) name of the new code.

The script file is in the $MCTDH_DIR/bin directory of the package. If the command elk_test is not found, check that your path contains this directory. If not, add the path.

If there is a problem with finding files or programs, check that the variables in the elk_test script are set correctly:

testcode the name of the executable to be tested, e.g. mctdh84.
ovlcode a version of the overlap program, e.g. overlap84.
compcode a version of the compare program, e.g. compare84.
comppath the path to the comparison set of test files, i.e. those that have been previously calculated by elk_test_gen and are known to be correct.

The script elk_test knows several options. If one types   elk_test -h one obtains:

Purpose: Test of the MCTDH program
Usage: elk_test [-h -c compname -o overlap -t compare -i inppath
       -n number -r -b num -e num -m mpinum -p pthreads] progname
   -h:  Print this help text.
   -c:  Use "compname" as comparison directory.
        If "compname" ends with a slash, $MCTDH_PLATFORM will be appended.
   -o:  Use "overlap" program for overlaps.
   -t:  Use "compare" program for tests.
   -i:  Use test?.inp files from "inppath".
        If inppath=c, i.e. "-i c", comppath is used as inppath.
   -q:  Re-do the compare-checks but do not re-do tests.
   -r:  Re-do tests that failed in a previous elk_test-run.
   -s:  Skip the test with the number that follows -s.
   -n:  Do tests only for the number(s) that follow -n.
        Quote the argument of -n if it contains blanks,
        e.g.:  -n 7   or   -n '3 7 21'
        With option -n elk_test will run the required tests
        regardless whether they already exist or not.
   -b:  Begin the test with the number following -b.
   -e:  End the test with the number following -e.
   -C:  Perform only the comparison step for a single test.
        The number of the test must follow the -C option.
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!  -m:  Specify the number of MPI processes for the elk-test.!
!  -p:  Specify the number of POSIX threads for the elk-test.!
!  If parallel code is tested the appropriate options must be!
!  used. Further a comparison set must be available for the  !
!  chosen number of POSIX-threads and MPI processes. Even if !
!  the appropriate comparison set is used numerical          !
!  differences can occur.                                    !
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

 Default overlap    : \
 Default compare    :  \ (run "elk_test -h" to see how these
 Default comparison :  /  are set for you)
 Default tests      : /

One may have to use the -c option if the directory which contains the data for comparison is not the default one. In fact, it is recommended that one adjust the default setting to ones demand before running elk_test the first time. The defaults are set at the top of the shell script.
If some test did fail, fix the code and re-run elk_test. If the name-directory of a particular test already exists, elk_test will read and analyse the data, but will not run mctdh again, if a name-file already exist. In order to re-do the failed tests use the -r or -n option.
If you want to perform a partial test, you may use the options -b (begin) and -e (end). I.e.   elk_test -b 3 -e 15 mctdh83   will run the tests 3,4,..,15. Test50 and higher are testing density operator propagation.
Also the parallel MCTDH-code can be tested for any number of POSIX-threads and MPI-processes. But for such runs an appropriate comparison set is needed because the parallelization itself and the different compiler settings for MPI influence the numerical behavior of the code.

Results Given by the Test

The results are written to two files.

summary:

The title of the test is followed by the test results, which will be one of the following:

testXX : Passed hard limit
testXX : Passed *SOFT* limit
testXX : FAILED
testXX : skipped

Hard limit passed means that the maximum difference found in the numerical comparisons between the source code versions is < 10-12. The soft limit is < 10-6.

results:

This file contains more information on the various tests, e.g. CPU-time and memory required by the two versions, and more details on the numerical checks carried out. Thus if the limits are not met, one can look in the results file to see where the problem lies.

Checks Carried Out

The following comparisons are made.
  1. Overlaps between the wavefunctions. The average and maximum difference between the overlaps are calculated.
  2. The numbers in the auto files are compared.
  3. The numbers in the check file are compared.
  4. The numbers in the dicht file are compared.

Generating the Test Set

To generate the comparison set of files, set up a directory, and run the script
elk_test_gen prog_name
where prog_name is the name of the program to be used for the generation. The test set is generated using the input files in $MCTDH_DIR/elk_inputs. The elk_test_gen script also contains variables specifying the code to be used. The script file is in $MCTDH_DIR/bin. If the command elk_test_gen is not found, check that your path contains this directory. The output directory is the current directory with $MCTDH_PLATFORM appended if the pwd-path ends otherwise. elk_test_gen will make the output directory if it does not exist.
Important note: The program projection must be compiled with the -i h4bmkp option to link with the bmkp PES of H4. One may need to run "mklinks h4bmkp" first. For this the addsurf directory must be installed and its path must appear in the environment variable MCTDH_ADDSURF. Finally, the path of the comparison data set should appear in the environment variable MCTDH_ELK. Both variables are usually set in ~/.mctdhrc .

One may use options to change the paths of the input and output files. Running elk_test_gen -h yields:

Purpose: Generates a set of test files for the MCTDH program.
Usage:   elk_test_gen [-h] [-M mpipro] [-P pthreads]
              [-i inpname] [-o outname] [-p potcode]
              [-j projcode] [-m mlpfcode] [-s skip] program
   -h:   Print this help text.
   -i:   Get input files from "inpname" directory.
   -j:   Use the projection-program "projcode".
   -p:   Use the potfit-program "potcode".
   -m:   Use the MLPF-program "mlpfcode".
   -o:   Write output in "outname" directory.
         If "outname" ends with a slash, $MCTDH_PLATFORM will be appended.
         The default outname is the current directory (with /$MCTDH_PLATFORM).
   -s:   Do not generate the first "skip" elk-test
   -M:   Number of MPI processes for the elk-test
   -P:   Number of POSIX-threads for the elk-test

 The default setting reads:
 [...]

If one or some of the runs failed, fix the error and run elk_test_gen again. The script will not re-do successfully installed sets.

As the mlpf program is not distributed with MCTDH, elk_test_gen will look for this program in the PATH by default. If the program is not available, the test data involving MLPFs will not be generated. The corresponding tests will be skipped when elk_test is run.

Adding New Tests

To add new tests, simply set up a new test.inp file in the comparison code test directory, using the next available number, e.g. test34.inp. The .inp file must contain a 3 line title at the beginning. This is written to the output files. The title should start with ### Title of (see other test*.inp files). The psi, auto, and dicht files must be created. If any special checks are to be made, the program compare.F, found in $MCTDH_DIR/source/analyse, must be altered. Finally, add the title to the list in this document, $MCTDH_DIR/doc/mctdh/elk_docu.html

By typing elk_test_gen when in the comparison test directory the new test files will be generated. The elk_test can then be re-run, including this new comparison.

List of Tests

 1:###           Test of NOCl1 / CMF integrator                   ###

 2:###           Test of NOCl1 / default VMF integrator           ###

 3:###           Test of NOCl1 / build option                     ###

 4:###           Test of NOCl1 / FFT basis-section                ###

 5:###           Test of NOCl1 with full potential                ###

 6:###           Test of NOCl1 with full potential / CDVR         ###

 7:###           Test of pyrazine 4-mode multi-set                ###

 8:###           Test of pyrazine 4-mode single-set               ###

 9:###           Test of pyrazine 4-mode interaction picture      ###

10:###  Test of Propagation of CH3I  (combined mode multi-packet version) ###

11:###     Test of    Relaxation of NOCl0                              ###

12:###     Test of    Relaxation of CH3I  (Single-packet version)      ###

13:###     Test of    Relaxation of CH3I  (Multi-packet version)       ###

14:###     Test of    Propagation of CH3I  (Single-packet version)     ###

15:###     Test of    Propagation of CH3I  (multi-packet version)      ###

16:###           Test of NOCl restart adding spfs                      ###

17:###           Test of NOCL1 continuation                            ###

18:###    Test of pyrazine 4-mode: CMF integrator + natural orbitals   ###

19:###    Test of pyrazine 4-mode: VMF integrator + natural orbitals   ###

20:###             Test of H+D2 (J=0) reactive scattering              ###
20:### 3D H+D2(nu=0) reactive scattering in Jacobian coordinates       ###
20:###   Adiabatic correction, natpot, sinus-dvr, leg-dvr, CMF,        ###

21:###    Test of H2/surface scattering.  sphfbr, fft periodic         ###

22:###  Test of Propagation of CH3I (multi-packet, CMF, h-proj option) ###

23:###    Test of H2/surface scattering.   test for autocap            ###

24:###                      Test of NOCl, multi-packet                 ###

25:###         Test of pyrazine 4-mode multi-set multi-packet          ###

26:###    Test of KLeg.  H+D2 reactive scattering (J>0)                ###
26:###      4D H+D2 reactive scattering in Jacobian coordinates        ###
26:###   Adiabatic correction, natpot, sinus-dvr, kleg-dvr,  CMF       ###

27:###  Test of expectation /  Butatriene linear 5-mode model          ###

28:###    Test of N2/surface scattering. PLeg, fft, exp -DVRs. CMF     ###

29:###     Test of   Relaxation=0 of CH3I  (Multi-packet version)      ###

30:###           Test of NOCl / Time-dependent operator                ###

31:###           Test of CO2 / CMF integrator                          ###

32:### Test of NOCl / Time-dep. operator / RK8 integrator & imp-ortho  ###

33:### Test of HONO, N=O excitation / Impr. Relaxation, lock, precon   ###

34:###     Test: Relaxation 3 bosonic particles, 1D, in a har. trap    ###

35:###     Test: Propagation 3 bosonic particles, 1D, in a har. trap   ###

36:###      Test of      H_2+H_2-Scattering (Propagation) with KLeg     ###

37:###      Test of    H_2+H_2-Scattering, ML-MCTDH, VMF                ###

38:###      Test of    H_2+H_2-Scattering, ML-MCTDH, CMF unified        ###

39:###      Test of    H_2+H_2-Scattering, ML-MCTDH, CMF split          ###

40:###      Test of    H_2+H_2-Scattering, ML-MCTDH, CMF split, MLPF    ###