Update information about test suite

Co-Authored-By: 1Regina <46968488+1Regina@users.noreply.github.com>
This commit is contained in:
Inari Listenmaa
2021-07-14 15:16:23 +08:00
parent 80d16fcf94
commit f345f615f4

View File

@@ -458,7 +458,6 @@ extra-lib-dirs:
and then run ``stack install``, also from the top directory. and then run ``stack install``, also from the top directory.
The Python library can also be installed from PyPI using `pip install pgf`.
== Compilation of RGL == == Compilation of RGL ==
@@ -552,36 +551,63 @@ the GF ``.rpm`` package.
When building ``.rpm`` packages for GF 3.4, we also had to build ``.rpm``s for When building ``.rpm`` packages for GF 3.4, we also had to build ``.rpm``s for
``fst`` and ``httpd-shed``. ``fst`` and ``httpd-shed``.
== Running the testsuite == == Running the test suite ==
**NOTE:** The test suite has not been maintained recently, so expect many The GF test suite is run with one of the following commands from the top directory:
tests to fail.
%% // TH 2012-08-06
GF has testsuite. It is run with the following command:
``` ```
$ cabal test $ cabal test
``` ```
or
```
$ stack test
```
The testsuite architecture for GF is very simple but still very flexible. The testsuite architecture for GF is very simple but still very flexible.
GF by itself is an interpreter and could execute commands in batch mode. GF by itself is an interpreter and could execute commands in batch mode.
This is everything that we need to organize a testsuite. The root of the This is everything that we need to organize a testsuite. The root of the
testsuite is the testsuite/ directory. It contains subdirectories which testsuite is the ``testsuite/`` directory. It contains subdirectories
themself contain GF batch files (with extension .gfs). The above command which themselves contain GF batch files (with extension ``.gfs``).
searches the subdirectories of the testsuite/ directory for files with extension The above command searches the subdirectories of the ``testsuite/`` directory
.gfs and when it finds one it is executed with the GF interpreter. for files with extension ``.gfs`` and when it finds one, it is executed with
The output of the script is stored in file with extension .out and is compared the GF interpreter. The output of the script is stored in file with extension ``.out``
with the content of the corresponding file with extension .gold, if there is one. and is compared with the content of the corresponding file with extension ``.gold``, if there is one.
If the contents are identical the command reports that the test was passed successfully.
Otherwise the test had failed.
Every time when you make some changes to GF that have to be tested, instead of Every time when you make some changes to GF that have to be tested,
writing the commands by hand in the GF shell, add them to one .gfs file in the testsuite instead of writing the commands by hand in the GF shell, add them to one ``.gfs``
and run the test. In this way you can use the same test later and we will be sure file in the testsuite subdirectory where its ``.gf`` file resides and run the test.
that we will not incidentaly break your code later. In this way you can use the same test later and we will be sure that we will not
accidentally break your code later.
**Test Outcome - Passed:** If the contents of the files with the ``.out`` extension
are identical to their correspondingly-named files with the extension ``.gold``,
the command will report that the tests passed successfully, e.g.
If you don't want to run the whole testsuite you can write the path to the subdirectory
in which you are interested. For example:
``` ```
$ cabal test testsuite/compiler Running 1 test suites...
Test suite gf-tests: RUNNING...
Test suite gf-tests: PASS
1 of 1 test suites (1 of 1 test cases) passed.
``` ```
will run only the testsuite for the compiler.
**Test Outcome - Failed:** If there is a contents mismatch between the files
with the ``.out`` extension and their corresponding files with the extension ``.gold``,
the test diagnostics will show a fail and the areas that failed. e.g.
```
testsuite/compiler/compute/Records.gfs: OK
testsuite/compiler/compute/Variants.gfs: FAIL
testsuite/compiler/params/params.gfs: OK
Test suite gf-tests: FAIL
0 of 1 test suites (0 of 1 test cases) passed.
```
The fail results overview is available in gf-tests.html which shows 4 columns:
+ //Results// - only areas that fail will appear. (Note: There are 3 failures in the gf-tests.html which are labelled as (expected). These failures should be ignored.)
+ //Input// - which is the test written in the .gfs file
+ //Gold// - the expected output from running the test set out in the .gfs file. This column refers to the contents from the .gold extension files.
+ //Output// - This column refers to the contents from the .out extension files which are generated as test output.
After fixing the areas which fail, rerun the test command. Repeat the entire process of fix-and-test until the test suite passes before submitting a pull request to include your changes.