mirror of
https://github.com/GrammaticalFramework/gf-core.git
synced 2026-04-23 11:42:49 -06:00
Update information about test suite
Co-Authored-By: 1Regina <46968488+1Regina@users.noreply.github.com>
This commit is contained in:
@@ -458,7 +458,6 @@ extra-lib-dirs:
|
|||||||
and then run ``stack install``, also from the top directory.
|
and then run ``stack install``, also from the top directory.
|
||||||
|
|
||||||
|
|
||||||
The Python library can also be installed from PyPI using `pip install pgf`.
|
|
||||||
|
|
||||||
== Compilation of RGL ==
|
== Compilation of RGL ==
|
||||||
|
|
||||||
@@ -552,36 +551,63 @@ the GF ``.rpm`` package.
|
|||||||
When building ``.rpm`` packages for GF 3.4, we also had to build ``.rpm``s for
|
When building ``.rpm`` packages for GF 3.4, we also had to build ``.rpm``s for
|
||||||
``fst`` and ``httpd-shed``.
|
``fst`` and ``httpd-shed``.
|
||||||
|
|
||||||
== Running the testsuite ==
|
== Running the test suite ==
|
||||||
|
|
||||||
**NOTE:** The test suite has not been maintained recently, so expect many
|
The GF test suite is run with one of the following commands from the top directory:
|
||||||
tests to fail.
|
|
||||||
%% // TH 2012-08-06
|
|
||||||
|
|
||||||
GF has testsuite. It is run with the following command:
|
|
||||||
```
|
```
|
||||||
$ cabal test
|
$ cabal test
|
||||||
```
|
```
|
||||||
|
|
||||||
|
or
|
||||||
|
|
||||||
|
```
|
||||||
|
$ stack test
|
||||||
|
```
|
||||||
|
|
||||||
The testsuite architecture for GF is very simple but still very flexible.
|
The testsuite architecture for GF is very simple but still very flexible.
|
||||||
GF by itself is an interpreter and could execute commands in batch mode.
|
GF by itself is an interpreter and could execute commands in batch mode.
|
||||||
This is everything that we need to organize a testsuite. The root of the
|
This is everything that we need to organize a testsuite. The root of the
|
||||||
testsuite is the testsuite/ directory. It contains subdirectories which
|
testsuite is the ``testsuite/`` directory. It contains subdirectories
|
||||||
themself contain GF batch files (with extension .gfs). The above command
|
which themselves contain GF batch files (with extension ``.gfs``).
|
||||||
searches the subdirectories of the testsuite/ directory for files with extension
|
The above command searches the subdirectories of the ``testsuite/`` directory
|
||||||
.gfs and when it finds one it is executed with the GF interpreter.
|
for files with extension ``.gfs`` and when it finds one, it is executed with
|
||||||
The output of the script is stored in file with extension .out and is compared
|
the GF interpreter. The output of the script is stored in file with extension ``.out``
|
||||||
with the content of the corresponding file with extension .gold, if there is one.
|
and is compared with the content of the corresponding file with extension ``.gold``, if there is one.
|
||||||
If the contents are identical the command reports that the test was passed successfully.
|
|
||||||
Otherwise the test had failed.
|
|
||||||
|
|
||||||
Every time when you make some changes to GF that have to be tested, instead of
|
Every time when you make some changes to GF that have to be tested,
|
||||||
writing the commands by hand in the GF shell, add them to one .gfs file in the testsuite
|
instead of writing the commands by hand in the GF shell, add them to one ``.gfs``
|
||||||
and run the test. In this way you can use the same test later and we will be sure
|
file in the testsuite subdirectory where its ``.gf`` file resides and run the test.
|
||||||
that we will not incidentaly break your code later.
|
In this way you can use the same test later and we will be sure that we will not
|
||||||
|
accidentally break your code later.
|
||||||
|
|
||||||
|
**Test Outcome - Passed:** If the contents of the files with the ``.out`` extension
|
||||||
|
are identical to their correspondingly-named files with the extension ``.gold``,
|
||||||
|
the command will report that the tests passed successfully, e.g.
|
||||||
|
|
||||||
If you don't want to run the whole testsuite you can write the path to the subdirectory
|
|
||||||
in which you are interested. For example:
|
|
||||||
```
|
```
|
||||||
$ cabal test testsuite/compiler
|
Running 1 test suites...
|
||||||
|
Test suite gf-tests: RUNNING...
|
||||||
|
Test suite gf-tests: PASS
|
||||||
|
1 of 1 test suites (1 of 1 test cases) passed.
|
||||||
```
|
```
|
||||||
will run only the testsuite for the compiler.
|
|
||||||
|
**Test Outcome - Failed:** If there is a contents mismatch between the files
|
||||||
|
with the ``.out`` extension and their corresponding files with the extension ``.gold``,
|
||||||
|
the test diagnostics will show a fail and the areas that failed. e.g.
|
||||||
|
|
||||||
|
```
|
||||||
|
testsuite/compiler/compute/Records.gfs: OK
|
||||||
|
testsuite/compiler/compute/Variants.gfs: FAIL
|
||||||
|
testsuite/compiler/params/params.gfs: OK
|
||||||
|
Test suite gf-tests: FAIL
|
||||||
|
0 of 1 test suites (0 of 1 test cases) passed.
|
||||||
|
```
|
||||||
|
|
||||||
|
The fail results overview is available in gf-tests.html which shows 4 columns:
|
||||||
|
|
||||||
|
+ //Results// - only areas that fail will appear. (Note: There are 3 failures in the gf-tests.html which are labelled as (expected). These failures should be ignored.)
|
||||||
|
+ //Input// - which is the test written in the .gfs file
|
||||||
|
+ //Gold// - the expected output from running the test set out in the .gfs file. This column refers to the contents from the .gold extension files.
|
||||||
|
+ //Output// - This column refers to the contents from the .out extension files which are generated as test output.
|
||||||
|
After fixing the areas which fail, rerun the test command. Repeat the entire process of fix-and-test until the test suite passes before submitting a pull request to include your changes.
|
||||||
|
|||||||
Reference in New Issue
Block a user