The parser works on raw byte sequences read from source files. If parsing
succeeds the raw byte sequences are converted to proper Unicode characters
in a later phase. But the parser calls the function buildAnyTree, which can
fail and generate error messages containing source code fragments, which might
then containing raw byte sequences. To render these error messages correctly,
they need to be converted in accordance with the coding flag in the source
file. This is now done for UTF-8-encoded source files, but should ideally also
be done for other character encodings. (Latin-1-encoded files never suffered
from this problem, since raw bytes are proper Unicode characters in this case.)
* Evaluate operators once, not every time they are looked up
* Remember the list of parameter values instead of recomputing it from the
pattern type every time a table selection is made.
* Quick fix for partial application of some predefined functions.
The pretty printer produced
mkDet pre {"a"; "an" / vowel} Sg
which is not accepted by the parser. The parser assigns pre { ... }, to
prededence level 4, and this is now reflected in the pretty printer, so
it prints
mkDet (pre {"a"; "an" / vowel}) Sg
(This caused a problem in GFSE since it parsers pretty printed grammars...)
This is a simple change in GF.Grammar.Lookup.allOpers, which is used only in
the implementation of the show_operations command in the shell.
This is useful when importing a concrete syntax (like LexiconEng) as a resource.
However, the types don't always look as nice as I hoped...
In GF.Compile.CheckGrammar, use a new topological sorting function that
groups independent judgements, allowing them all to be checked before
continuing or reporting errors.
This turns error messages like
gf: too few bytes. Failed reading at byte position 1
gf: /some/path/somefile.gfo: too few bytes. Failed reading at byte position 1
but a better fix would be to ignore bad .gfo files and compile from source.
The problem is the way this decision is made in
GF.Compile.ReadFiles.selectFormat...
As a temporary workaround, alex is no longer invoked automatically when
building with cabal. Developers who want to modify the lexer need to run
alex on Lexer.x manually and record the modified Lexer.hs.
src/compiler/GF/Grammar/lexer/Lexer.x -- hidden from cabal
src/compiler/GF/Grammar/Lexer.hs -- update it manually
This speeds up the compilation of PhrasebookFin.pgf by 12%, mosly by speeding
up calls to lookupModule in calls from lookupParamValues, in calls
from allParamValues.
The invariant "modules are stored in dependency order" is no longer respected!
But the type MGrammar is now abstract, making it easier to maintain this or
other invariants in the future.
+ Avoids some code duplication by combinging lookupModule and lookupIdentInfo.
+ Also removed lookupIdentInfo from export list, since it is not used anywhere
else.
