Caching parse results uses a lot of memory, even if they expire after
2 minutes, so it won't scale up to many simultaneous users.
But some excessive memory use seems to be caused by space leaks in
(the Haskell binding to) the C run-time system, and these should be fixed.
For example, flushing the PGF cache does not release the memory allocated
by the C run-time system when loading a PGF file.
Most PGF web API commands that produce linearizations now accept an
unlexer parameter. Possible values are "text", "code" and "mixed".
The web service now include Date and Last-Modified headers in the HTTP,
responses. This means that browsers can treat responses as static content and
cache them, so it becomes less critical to cache parse results in the server.
Also did some cleanup in PGFService.hs, e.g. removed a couple of functions
that can now be imported from PGF.Lexing instead.
The old type was [String] -> String. This function was only used
in GF.Text.Lexing.stringOp, which now uses (unwords . bindTok) instead,
with no change in behaviour.
The capitalization of the first word was done in GF.Text.Lexing.stringOp,
but is now done in the functions unlexText and unlexMixed in PGF.Lexing.
These functions are only used in stringOp and in PGFService (where the change
is needed), so the subtle change in behaviour should not cause any bugs.
Only change the first word to lowercase if the original input is not found in
the grammar's morphology. This allows parsing of sentenses starting with "I" in
English, nouns in German and proper names in other languages, but it can make
the wrong choice for multi-words.
The PGF web API commands parse, translate, translategroup, c-parse and
c-translate now support a lexer parameter. Possible values are "text",
"code" and "mixed".
This is used in the Wide Coverage Demo App to save one server roundtrip.
+ The current type checker for concrete syntax is in
GF.Compile.TypeCheck.RConcrete, but GF.Compile.TypeCheck.Concrete was
still imported in GFI.
+ Fixed a bug that allowed Ints n as a subtype of Ints m, regardless of
m and n. It now requires n<=m. Note: the type checker still allows Int
as a subtype of Ints m, regardless of m.
+ Fixed a potential efficiency problem with large record types, by reducing
the number of recursive calls from |R|*|S| to |R| when checking if R<=S.
+ Fixed a misleading comment: "alpha g t u" checks that u is a subtype of t,
the other way around. Similarly, "checkIfEqLType gr g t u trm" checks that
u is a subtype of t, not the other way around, and not that t is equal to u.
The layout now resembles a certain other online translation service.
The translation is started automatically after text is entered, no need to
press the Translate button.
This bug was introduced sometime between 2013-08-21 and 2013-11-01 and caused
the function convertTerm in GF.Compile.GeneratePMCFG to encounter a EPatt where
it expected Strs. I fixed it by applying the function getPatts (from the old
partial evaluator) to the pattern.
+ Adapt language selection menus to the languages supported by the grammar.
+ Add translation quality colors to the links to alternative translations.
+ Also added more country codes in langcode.js (needed for speech synthesis
voice selection).
The background color of the translation changes to pale green, yellow or red
according to the quality.
Two types of quality indications are supported:
+ From the grammar: linearizations starting with "+" or "*" are assumed to
be of high or low quality, respectively.
+ From the parse tree: if the root function is "?", then the translation is
assumed to be of low quality.
If no quality indications are detected, medium quality is assumed.
In support.js, the functions http_get_json, ajax_http_get_json
and ajax_http_post_json now calls the supplied error callback if the server
returns invalid JSON (e.g. because of a crash).
The function gftranslate.translate in gftranslate.js returns
a JSON value containing an error message (since it doesn't have an error
callback).
This should result in fewer situations where "nothing happens" and the user
doesn't know if it is beacuse the server is slow, or if there was an error.
The directory should be the same as the one used in Setup.hs when building
the RGL, where a build directory supplied by cabal is used. This is usually
dist/build/rgl, but it could be different, e.g. when using a cabal sandbox.
Two of the fixes were in the custom version of the binary package. We
should get rid of it at some point, to reduce the maintenance burden.
The third fix was to add new version constraints for happy and alex in
gf.cabal. New versions of them are needed because of the ugly, low-level,
GHC-specific code they produce need to be different for GHC 7.8.
More fixes might be needed for -server mode, but the cgi package is not
compatible with GHC 7.8 at the moment, so it will have to wait.
+ PGFService.hs: add command c-grammar, include probability in parse results
+ js/gftranslation.js: add start position and limit parameters, return more
info to applications
+ Simple Translator: show two wide coverage translations
+ Wide coverage demo: show parse tree and probability (intended as grammar
debugging aids)
PGF service requests are stateless and can run in parallel, but some other
requests handled by the server are not and might even change the current
working directory temporarily, and this affects all threads, so it is
important that the PGF service requests access PGF files by absolute paths.
The interface to the wide coverage translation is in js/gftranslate.js
and it assumes that the grammar is installed on the cloud server
as /robust/Translate8.pgf.
The list of supported languages is hardwired in gftranslate.js, since
there is no support for obtaining this info from the C run-time system
at the moment.
If the C run-time library is compiled and installed on your system, you can now
do 'cabal configure -fc-runtime' to get the following extras:
+ The haskell binding to the C run-time library will be included in the
PGF library (so you can import it in Haskell applications).
Documentation on the new modules will be included when you run
'cabal haddock'.
+ The new command 'pgf-shell', implemented on top of haskell binding to
the C run-time system.
+ Three new commands in the web API: c-parse, c-linearize and
c-translate. Their interfaces are similar to the corresponding commands
without the "c-" prefix, but they should be considered preliminary.
* The haskell-bind.cabal file makes it easy to build the haskell binding and
use it in ghci.
* pgf-shell.hs is a simple example of how to use the haskell binding.
Change the type of getConcr from
getConcr :: PGF -> Language -> Concr
to
getConcr :: PGF -> Language -> Maybe Concr
This is to prevent programs from crashing later if you try to select a
concrete syntax that is not present in a grammar.
The parse grammar Parse8.pgf for 8 languages can now be built in parallel with
make -j. Finnish requires a lot of RAM. French requires a lot of time.
Other related make targets available for convenience:
make Parse: create Parse???.pgf for the 8 individual languages
make Parse8: create Parse8.pgf assuming all Parse???.pgf already exist
Note: these make targets (and some others) use the gf executable and RGL
library that are installed on the system (found via $PATH and $GF_LIB_PATH,
respectively).
When running a command like
gf -make L_1.gf ... L_n.gf
gf now avoids recreating the target PGF file if it already exists and is
up-to-date.
gf still reads all required .gfo files, so significant additional speed
improvements are still possible. This could be done by reading .gfo files
more lazily...
When running a command like
gf -make -name=T L_1.pgf ... L_n.pgf
gf now checks if T.pgf exists and is up-to-date before reading and computing
the union of the L_i.pgf files.
The name (T) of the target PGF file has to be given explicitly for this to work,
since otherwise the name is not known until the union has been computed.
If the functions for reading PGF files and computing the union were lazier,
this would not be necessary...
Some backwards incompatible changes were made to the PGF file format after
the release of GF 3.5. This patch adds a module for reading PGF files in the
old format.
This means that old PGF files on the grammaticalframework.org server will
continue to work after we install the latest version of GF.
There were two differences between the current output and the old gold file:
1. The trees are no longer generated with increasing depth
2. The meaning of the -depth flag has changed: for example,
"gt -cat=Nat -depth=1" used to generate only "zero",
now you also get "succ zero".
* The old way: a user hook in Setup.hs
* The new way: specify it in gf.cabal
* The test suite is now called gf-tests, and it runs testsuite/run.hs.
* You can run it manually with 'runhaskell testsuite/run.hs'. It also runs,
together with rgl-tests, when you do 'cabal test'
* Currently only 9 of 34 tests pass. Many failures have silly causes:
- Error messages that look slightly different
- Same output but in a different order
- Absolute paths in output
hallgren