1
0
forked from GitHub/gf-core
Commit Graph

5 Commits

Author SHA1 Message Date
aarne
42cfacbf86 lexer for latex code: don't separate backslash from the macro name that it marks 2014-10-17 06:40:06 +00:00
hallgren
50ea3d265c Change the type of PGF.Lexing.bindTok to [String] -> [String]
The old type was [String] -> String. This function was only used
in GF.Text.Lexing.stringOp, which now uses (unwords . bindTok) instead,
with no change in behaviour.
2014-04-09 17:39:21 +00:00
hallgren
677d849840 Unlexers: move capitalization of first word from GF.Text.Lexing to PGF.Lexing
The capitalization of the first word was done in GF.Text.Lexing.stringOp,
but is now done in the functions unlexText and unlexMixed in PGF.Lexing.
These functions are only used in stringOp and in PGFService (where the change
is needed), so the subtle change in behaviour should not cause any bugs.
2014-04-09 17:26:23 +00:00
hallgren
04a6260eea PGF Service: a bit more clever lexer=text
Only change the first word to lowercase if the original input is not found in
the grammar's morphology. This allows parsing of sentenses starting with "I" in 
English, nouns in German and proper names in other languages, but it can make
the wrong choice for multi-words.
2014-04-09 14:13:18 +00:00
hallgren
9cac98a356 Move basic lexing functions from GF.Text.Lexing to the new module PGF.Lexing
They are thus part of the PGF Run-Time Library, making it possible to add
lexing functionality in PGF service in a natural way.
2014-04-08 14:07:49 +00:00