krasimir
f6796ced2d
bugfix in lookup_morpho in the C runtime
2015-08-06 11:51:23 +00:00
krasimir
6f2afdd53e
bugfix in the parser for lexical lookup
2015-06-30 12:54:19 +00:00
krasimir
c7f1803410
bugfix in the bottom-up prediction for the parser
2015-06-25 09:46:43 +00:00
krasimir
e352742b8c
the orthographic tokens were broken when they appear inside a dictionary entry. Now this is fixed by treating those as syntactic rules
2015-06-12 13:03:19 +00:00
krasimir
ce05465365
the callbacks in C now take the concrete syntax as a parameter
2015-05-18 14:41:33 +00:00
krasimir
773a33979b
pgf_lookup_morpho and pgf_lookup_word_prefix are now also case insensitive if case_sensitive=off
2015-05-18 14:20:35 +00:00
krasimir
b4e41cabdd
"flags case_sensitive=off" makes the parser case insensitive
2015-05-11 14:11:42 +00:00
krasimir
1e0d7be4f4
added all orthographic primitives
2015-05-11 13:01:39 +00:00
krasimir
13998e3287
implemented CAPIT in parsing
2015-05-08 18:54:50 +00:00
krasimir
483df35954
fix in the dictionary lookup
2015-05-08 18:12:24 +00:00
krasimir
ea0f74d455
a new nice and elegant algorithm for dictionary lookup in the parser
2015-05-08 14:38:24 +00:00
krasimir
365c7bb1d8
the parser is now fully Unicode compatible
2015-05-08 09:23:29 +00:00
krasimir
8f0624bc30
remove the meta_item variable from the parse state
2015-03-06 15:18:50 +00:00
krasimir
a5817ff492
fix some out-of-date code for the PGF_RESULT_DEBUG mode in the parser
2015-02-26 13:31:26 +00:00
krasimir
9c83b2e27d
use the same trick for comparison of doubles in cmp_expr_state
2015-02-26 13:10:01 +00:00
krasimir
532647c978
bugfix in the parser which caused some items to be poped in the wrong order if their probability differ with less that 0.5
2015-02-26 12:17:59 +00:00
krasimir
715ac1ca62
remove three more uses of meta productions that were left in debugging mode
2015-02-24 10:59:18 +00:00
krasimir
b92bbcfc37
parsing with meta rules is now removed since we don't use them anymore and I would gladly remove some code.
2015-02-19 11:59:51 +00:00
krasimir
e3a87c657b
fix in the parser for callbacks in the middle of a word
2015-02-18 16:14:29 +00:00
kr.angelov
9bc5349e62
change in the API for literals
...
The API in the C runtime as well as in the Haskell, Python and Java binding
is changed. Now instead of adding the literal callbacks to the concrete syntax
you need to supply them every time when you need to parse. The main reason is:
- referentially transparent API for Haskell
- when we start using memory mapped files we will not be allowed to change
anything in the grammar data structures. At that point the old API would
be impossible to use.
2014-12-16 10:21:26 +00:00
kr.angelov
64feca59c8
fix the last patch
2014-12-01 11:03:50 +00:00
kr.angelov
960ba43149
a small optimization in the PGF parser which gives me ~5% speed up
2014-12-01 10:48:15 +00:00
kr.angelov
f63e8faf00
remove pgf/parser.h
2014-11-27 11:12:14 +00:00
kr.angelov
41e217a641
fix the handling of the heuristic factor. it was broken when I added the BIND token which means that the App grammar was running about 2-3 times slower.
2014-11-26 15:56:36 +00:00
kr.angelov
e169f225cf
a simple optimization in cmp_item_prob with great effect on the parser's speed
2014-10-23 21:22:22 +00:00
kr.angelov
f8b73d593c
Prelude.CAPIT is now a built-in primitive. It still generates &| in the Haskell runtime but will be intepreted in the C runtime
2014-10-09 19:34:12 +00:00
kr.angelov
86e9acc7a7
throw away the long obsolete runtime type information in the C runtime
2014-10-09 13:44:26 +00:00
kr.angelov
6c86e7fa91
replace the hash maps in the abstract syntax with binary search tables
2014-10-09 08:38:55 +00:00
kr.angelov
b70cea2673
bugfix in the handling of BIND in the C runtime
2014-04-22 13:53:18 +00:00
kr.angelov
b0b27a1b32
now word completion from Java works. It could be made better
2014-04-17 11:00:27 +00:00
kr.angelov
1ca09448f9
now fully functional Java API for custom literals
2014-04-10 14:14:31 +00:00
kr.angelov
f99d3033aa
bugfix in the C runtime that shows up with only some grammars
2014-04-08 12:41:41 +00:00
kr.angelov
911ae42296
small fixes in the C runtime
2014-03-25 13:57:57 +00:00
kr.angelov
a1446b87fa
whitespace cleanup in the parser
2014-03-18 10:19:35 +00:00
kr.angelov
ae1512c926
compute the right word probability
2014-03-12 15:36:40 +00:00
kr.angelov
ba28cfd80e
added pgf_lookup_word_prefix which makes it possible to do simple word prediction
2014-03-07 21:24:20 +00:00
kr.angelov
0ece8f19ab
loading and unloading of languages in the C runtime and in the Python bindings
2013-12-10 15:00:52 +00:00
kr.angelov
1d2786f7da
bugfix in the debug mode for the parser
2013-11-22 12:26:49 +00:00
kr.angelov
aad6a1c4bb
bugfix in the C runtime
2013-11-21 14:31:28 +00:00
kr.angelov
5b83da5583
bugfix in the robust parser
2013-11-18 09:05:27 +00:00
kr.angelov
126d5e0278
bugfix in the C runtime which had made the parser up to 200 times slower for some sentences
2013-11-13 15:15:28 +00:00
kr.angelov
0095119ec0
added Predef.SOFT_BIND. This special token allows zero or more spaces between ordinary tokens. It is also used in the English RGL to attach the commas to the previous word.
2013-11-12 09:54:57 +00:00
kr.angelov
af4a1f628d
factor of 3 speedup for the translation without slowing down the grammar loading. The parser still seems to be 8 times slower than before I introduced the BIND. At least the Android UI now feels a lot more comfortable
2013-11-07 11:14:32 +00:00
kr.angelov
f9d044d294
fix in the parsing with pre
2013-11-06 16:25:25 +00:00
kr.angelov
2483dc7728
the content of ParseEngAbs3.probs is now merged with ParseEngAbs.probs. The later is now retrained. Once the grammar is compiled with the .probs file now it doesn't need anything more to do robust parsing. The robustness itself is controlled by the flags 'heuristic_search_factor', 'meta_prob' and 'meta_token_prob' in ParseEngAbs.gf
2013-11-06 10:21:46 +00:00
kr.angelov
981a728d13
fix the robust parser for unknown tokens
2013-11-01 13:05:17 +00:00
kr.angelov
fed16ca9e5
use linref in the C linearizer. The robust linearization is now fully supported
2013-10-30 21:43:52 +00:00
kr.angelov
042243f08a
added the linref construction in GF. The PGF version number is now bumped
2013-10-30 12:53:36 +00:00
kr.angelov
c60a019b70
another fix for 'pre'
2013-10-28 13:54:06 +00:00
kr.angelov
151f86c1e9
fix the handling of 'pre' in the C runtime
2013-10-28 12:35:37 +00:00
