Commit Graph

4 Commits

Author SHA1 Message Date
hallgren
f2cc1d2c68 Change the type of PGF.Lexing.bindTok to [String] -> [String]
The old type was [String] -> String. This function was only used
in GF.Text.Lexing.stringOp, which now uses (unwords . bindTok) instead,
with no change in behaviour.
2014-04-09 17:39:21 +00:00
hallgren
0d1ad2f0e5 Unlexers: move capitalization of first word from GF.Text.Lexing to PGF.Lexing
The capitalization of the first word was done in GF.Text.Lexing.stringOp,
but is now done in the functions unlexText and unlexMixed in PGF.Lexing.
These functions are only used in stringOp and in PGFService (where the change
is needed), so the subtle change in behaviour should not cause any bugs.
2014-04-09 17:26:23 +00:00
hallgren
1602eab5b6 PGF Service: a bit more clever lexer=text
Only change the first word to lowercase if the original input is not found in
the grammar's morphology. This allows parsing of sentenses starting with "I" in 
English, nouns in German and proper names in other languages, but it can make
the wrong choice for multi-words.
2014-04-09 14:13:18 +00:00
hallgren
cf34ee8890 Move basic lexing functions from GF.Text.Lexing to the new module PGF.Lexing
They are thus part of the PGF Run-Time Library, making it possible to add
lexing functionality in PGF service in a natural way.
2014-04-08 14:07:49 +00:00