The script bin/build-binary-dist.sh has been updated to build either a plain
.tar.gz package or OS X Installer package (.pkg).
Note that bin/build-binary-dist.sh is designed to build and include the
C run-time system in the binary package. If the C run-time system fails to
build, no binary package will be created.
On my laptop these changes speed up the full build of the RGL and example
grammars with 'cabal build' from ~95s to ~43s and the zero build from ~18s
to ~5s.
The main change is the introduction of the module GF.CompileInParallel that
replaces GF.Compile and the function GF.Compile.ReadFiles.getAllFiles. At
present, it is activated with the new -j flag, and it is only used when
combined with --make or --batch. In addition, to get parallel computations,
you need to add GHC run-time flags, e.g., +RTS -N -A20M -RTS, to the command
line.
The Setup.hs script has been modified to pass the appropriate flags to GF
for parallel compilation when compiling the RGL and example grammars, but you
need a recent version of Cabal for this to work (probably >=1.20).
Some additonal refactoring were made during this work. A new monad is used to
avoid warnings/error messages from different modules to be intertwined when
compiling in parallel, so some functios that were hardiwred to the IO or IOE
monads have been lifted to work in arbitrary monads that are instances in
the appropriate classes.
I prefer small functions with descriptive names over large monilithic chunks
of code, so I grouped the compiler passes called from compileSourceModule
into funcitons named frontend, middle and backend. This also makes decisions
about which passes to run clearly visible up front.
Also made some small changes in GF.Compile.
In particular, the function compileOne has been moved to the new module
GF.CompileOne and its type has been changed from
compileOne :: ... -> CompileEnv -> FilePath -> IOE CompileEnv
to
compileOne :: ... -> SourceGrammar -> FilePath -> IOE OneCompiledModule
making it more suitable for use in a parallel compiler.
The RGL is now compile with only three calls to GF (prelude, present,
alltenses). This also makes even more parallelism available to GF for speeding
up full builds of the RGL.
This speedup is obtained by instead of generating 211 shell commands to compile the RGL one file at a time, generate 7 shell commands, passing a number of
related files to GF in one go.
In addition to cutting down the zero build time, this opens up for speeding up
full builds of the RGL, by adding more parallelism in GF's grammar compilation
machinery.
(Note: gf.cabal already takes advantage of GHC's parallel build option to
speed up the compilation of GF itself, if GHC>=7.8 is used.)
GF.Text.Pretty provides the class Pretty and overloaded versions of the pretty
printing combinators in Text.PrettyPrint, allowing pretty printable values to
be used directly instead of first having to convert them to Doc with functions
like text, int, char and ppIdent. Some modules have been converted to use
GF.Text.Pretty, but not all. Precedences could be added to simplify the pretty
printers for terms and patterns.
GF.Infra.Location contains the types Location and L, factored out from
GF.Grammar.Grammar, and the class HasSourcePath. This allowed the import
of GF.Grammar.Grammar to be removed from GF.Infra.CheckM, making it more
like a pure library module.
The wide coverage demo apps now shows a "please wait" message while the grammar
is loading, and a red error message if the grammar is not found on the server.
The Simple Translation Tool also show red error messages if the grammar is not
found.
The configuration script takes significantly longer to run than the compilation
of all the C code, which is a hint that is in need of a major simplification.
PGF exports the public, stable API.
PGF.Internal exports additional things needed in the GF compiler & shell,
including the nonstardard version of Data.Binary.
From the documentation: the throwIO variant should be used in preference to
throw to raise an exception within the IO monad because it guarantees ordering
with respect to other IO operations, whereas throw does not.
Also removed some unused imports.
(table { p_i => t_i } ! x).l ==> table { p_i => t_i.l } ! x
This was used in the old partial evaluator and can significantly reduce term
sizes in some cases.
Eta expansion is applied between partial evaluation and PMCFG generation.
The buggy version generated type incorrect terms, but PMCFG generation
apparently worked anyway.
Also improve behaviour for languages that are missing in Phrasebook (Chinese).
Also update the initial set of languages in the menus to match Translate11.pgf.
1. Like pgf-translate, it now shows one result at a time, press Enter to get
more results.
2. You can load a new grammar with the command 'i <path-to-pgf>'
This means that the new translation is available instantly when switching
target languages. It can also reduce space leaks problems in the server
somewhat by avoiding repeated parsing of the source text.
+ Programs compiled with ghc<7.8 have a default built-in 8M stack size limit.
With ghc>7.0 this built-in default can be changed, so it is now 64M, allowing
GF to work with larger grammars without manually increasing the stack size.
(But if GF is compiled with ghc 6.12, the built-in default is still 8M.)
+ Enable ghc>=7.8 support for compiling modules in parallel, to speed up the
compilation of GF.
For example, you can now reorder paragraphs by cut & paste, without causing
anything to be sent to the server for re-translation.
Also some color & layout tweaks.
Also,
+ Get up to 10 (instead of 5) alternative robust translations.
+ Remove duplicates from list of alternative robust translations
+ Add GF Cloud logo to the simple translator page
Caching parse results uses a lot of memory, even if they expire after
2 minutes, so it won't scale up to many simultaneous users.
But some excessive memory use seems to be caused by space leaks in
(the Haskell binding to) the C run-time system, and these should be fixed.
For example, flushing the PGF cache does not release the memory allocated
by the C run-time system when loading a PGF file.
Most PGF web API commands that produce linearizations now accept an
unlexer parameter. Possible values are "text", "code" and "mixed".
The web service now include Date and Last-Modified headers in the HTTP,
responses. This means that browsers can treat responses as static content and
cache them, so it becomes less critical to cache parse results in the server.
Also did some cleanup in PGFService.hs, e.g. removed a couple of functions
that can now be imported from PGF.Lexing instead.
hallgren