This bug was introduced sometime between 2013-08-21 and 2013-11-01 and caused
the function convertTerm in GF.Compile.GeneratePMCFG to encounter a EPatt where
it expected Strs. I fixed it by applying the function getPatts (from the old
partial evaluator) to the pattern.
+ Adapt language selection menus to the languages supported by the grammar.
+ Add translation quality colors to the links to alternative translations.
+ Also added more country codes in langcode.js (needed for speech synthesis
voice selection).
The background color of the translation changes to pale green, yellow or red
according to the quality.
Two types of quality indications are supported:
+ From the grammar: linearizations starting with "+" or "*" are assumed to
be of high or low quality, respectively.
+ From the parse tree: if the root function is "?", then the translation is
assumed to be of low quality.
If no quality indications are detected, medium quality is assumed.
In support.js, the functions http_get_json, ajax_http_get_json
and ajax_http_post_json now calls the supplied error callback if the server
returns invalid JSON (e.g. because of a crash).
The function gftranslate.translate in gftranslate.js returns
a JSON value containing an error message (since it doesn't have an error
callback).
This should result in fewer situations where "nothing happens" and the user
doesn't know if it is beacuse the server is slow, or if there was an error.
The directory should be the same as the one used in Setup.hs when building
the RGL, where a build directory supplied by cabal is used. This is usually
dist/build/rgl, but it could be different, e.g. when using a cabal sandbox.
Two of the fixes were in the custom version of the binary package. We
should get rid of it at some point, to reduce the maintenance burden.
The third fix was to add new version constraints for happy and alex in
gf.cabal. New versions of them are needed because of the ugly, low-level,
GHC-specific code they produce need to be different for GHC 7.8.
More fixes might be needed for -server mode, but the cgi package is not
compatible with GHC 7.8 at the moment, so it will have to wait.
+ PGFService.hs: add command c-grammar, include probability in parse results
+ js/gftranslation.js: add start position and limit parameters, return more
info to applications
+ Simple Translator: show two wide coverage translations
+ Wide coverage demo: show parse tree and probability (intended as grammar
debugging aids)
PGF service requests are stateless and can run in parallel, but some other
requests handled by the server are not and might even change the current
working directory temporarily, and this affects all threads, so it is
important that the PGF service requests access PGF files by absolute paths.
The interface to the wide coverage translation is in js/gftranslate.js
and it assumes that the grammar is installed on the cloud server
as /robust/Translate8.pgf.
The list of supported languages is hardwired in gftranslate.js, since
there is no support for obtaining this info from the C run-time system
at the moment.
If the C run-time library is compiled and installed on your system, you can now
do 'cabal configure -fc-runtime' to get the following extras:
+ The haskell binding to the C run-time library will be included in the
PGF library (so you can import it in Haskell applications).
Documentation on the new modules will be included when you run
'cabal haddock'.
+ The new command 'pgf-shell', implemented on top of haskell binding to
the C run-time system.
+ Three new commands in the web API: c-parse, c-linearize and
c-translate. Their interfaces are similar to the corresponding commands
without the "c-" prefix, but they should be considered preliminary.
* The haskell-bind.cabal file makes it easy to build the haskell binding and
use it in ghci.
* pgf-shell.hs is a simple example of how to use the haskell binding.
Change the type of getConcr from
getConcr :: PGF -> Language -> Concr
to
getConcr :: PGF -> Language -> Maybe Concr
This is to prevent programs from crashing later if you try to select a
concrete syntax that is not present in a grammar.
When running a command like
gf -make L_1.gf ... L_n.gf
gf now avoids recreating the target PGF file if it already exists and is
up-to-date.
gf still reads all required .gfo files, so significant additional speed
improvements are still possible. This could be done by reading .gfo files
more lazily...
When running a command like
gf -make -name=T L_1.pgf ... L_n.pgf
gf now checks if T.pgf exists and is up-to-date before reading and computing
the union of the L_i.pgf files.
The name (T) of the target PGF file has to be given explicitly for this to work,
since otherwise the name is not known until the union has been computed.
If the functions for reading PGF files and computing the union were lazier,
this would not be necessary...
Some backwards incompatible changes were made to the PGF file format after
the release of GF 3.5. This patch adds a module for reading PGF files in the
old format.
This means that old PGF files on the grammaticalframework.org server will
continue to work after we install the latest version of GF.
There were two differences between the current output and the old gold file:
1. The trees are no longer generated with increasing depth
2. The meaning of the -depth flag has changed: for example,
"gt -cat=Nat -depth=1" used to generate only "zero",
now you also get "succ zero".
* The old way: a user hook in Setup.hs
* The new way: specify it in gf.cabal
* The test suite is now called gf-tests, and it runs testsuite/run.hs.
* You can run it manually with 'runhaskell testsuite/run.hs'. It also runs,
together with rgl-tests, when you do 'cabal test'
* Currently only 9 of 34 tests pass. Many failures have silly causes:
- Error messages that look slightly different
- Same output but in a different order
- Absolute paths in output
This means that the -old-comp and -new-comp flags are not recognized anymore.
The only functional difference is that printnames were still normalized with
the old partial evaluator. Now that is done with the new partial evaluator.
This was a fairly simple change thanks to previous work on making the Ident
type abstract and the fact that PGF.CId already uses UTF-8-encoded
ByteStrings.
One potential pitfall is that Data.ByteString.UTF8 uses the same type for
ByteStrings as Data.ByteString. I renamed ident2bs to ident2utf8 and
bsCId to utf8CId, to make it clearer that they work with UTF-8-encoded
ByteStrings.
Since both the compiler input and identifiers are now UTF-8-encoded
ByteStrings, the lexer now creates identifiers without copying any characters.
**END OF DESCRIPTION***
Place the long patch description above the ***END OF DESCRIPTION*** marker.
The first line of this file will be the patch name.
This patch contains the following changes:
M ./src/compiler/GF/Compile/CheckGrammar.hs -3 +3
M ./src/compiler/GF/Compile/GrammarToPGF.hs -2 +2
M ./src/compiler/GF/Grammar/Binary.hs -5 +1
M ./src/compiler/GF/Grammar/Lexer.x -11 +13
M ./src/compiler/GF/Infra/Ident.hs -19 +36
M ./src/runtime/haskell/PGF.hs -1 +1
M ./src/runtime/haskell/PGF/CId.hs -2 +3
1. The default encoding is changed from Latin-1 to UTF-8.
2. Alternate encodings should be specified as "--# -coding=enc", the old
"flags coding=enc" declarations have no effect but are still checked for
consistency.
3. A transitional warning is generated for files that contain non-ASCII
characters without specifying a character encoding:
"Warning: default encoding has changed from Latin-1 to UTF-8"
4. Conversion to Unicode is now done *before* lexing. This makes it possible
to allow arbitrary Unicode characters in identifiers. But identifiers are
still stored as ByteStrings, so they are limited to Latin-1 characters
for now.
5. Lexer.hs is no longer part of the repository. We now generate the lexer
from Lexer.x with alex>=3. Some workarounds for bugs in alex-3.0 were
needed. These bugs might already be fixed in newer versions of alex, but
we should be compatible with what is shipped in the Haskell Platform.
Makefile2 specificies the dependencies accurately enough that you can use
make's support for parallel compilation. Run
make -f Makefile2 -j
to utilize all the cores in your computer to significantly speed up the
creation of Phrasebook.pgf.
+ Eliminated vairous ad-hoc coersion functions between specific monads
(IO, Err, IOE, Check) in favor of more general lifting functions
(liftIO, liftErr).
+ Generalized many basic monadic operations from specific monads to
arbitrary monads in the appropriate class (MonadIO and/or ErrorMonad),
thereby completely eliminating the need for lifting functions in lots
of places.
This can be considered a small step forward towards a cleaner
compiler API and more malleable compiler code in general.
1. No temporary files are created.
2. The output of a system command is read lazily, making it feasible to
process large or even infinite output, e.g. the following works as
expected:
? "yes" | ? "head -5" | ps -lextext
The system_pipe (aka "?") command creates a temporary file _tmpi containing
the input of the system command. It *both* appends _tmpi as an extra argument
to the system command line *and* adds an input redirection "< _tmpi". (It
also uses and output redirection "> _tmpo" to captures the output of the
command.)
With this patch, the _tmpi argument is no longer appended to the command line.
This allows system_pipe to work with pure filters, such as the "tr" commands,
but it will no longer work with commands that require an input file name.
(It is possible to use write_file instead...)
TODO: it would also be fairly easy to eliminate the creation of the _tmpi and
_tmpo files altogether.
Add proper type checking of course-of-values tables:
+ Make sure that all subterms have the same type.
+ Resolve overloaded operators.
Note though that the GF book states in C.4.12 that the "course-of-values
table [...] format is not recommended for GF source code, since the
ordering of parameter values is not specified and therefore a
compiler-internal decision."
The CF parser in GF.Grammar.CF assigns function names to the rules, but they
are not always unique, causing rules to be dropped in the follwing CF->GF
conversion. So a pass has been added before the CF->GF conversion, to make
sure that function names are unique.
A comment says "rules have an amazingly easy parser", but the parser looks
like quick hack. It is very sloppy and silently ignores many errors, e.g.
- Explicitly given function names should end with '.', but if the do not, the
last character in the function name is silently dropped.
- Everything following a ';' is silently dropped.
Trailing spaces caused the command line parse to be ambiguous, and
ambiguous parses were rejected by function readCommandLine, causing
the cryptic error message "command not parsed".
The only use of PGF.Tree outside the PGF library was in GF.Command.Commands,
and it was eliminated by using PGF.Expr directly instead.
PGF.Paraphrase still uses PGF.Tree.
This module should not be part of the public PGF library API, and it was only
used in GF.CompileToAPI, so the code was moved there. The module defined
constFuncs and syntaxFuncs, but only syntaxFuncs was used.
hallgren