 kr.angelov
|
818ea0d4d6
|
updated statistical model
|
2012-11-28 12:44:20 +00:00 |
|
|
kr.angelov
|
2db296db30
|
Pakistani is now properly capitalized in the English dictionary
|
2012-11-28 12:43:45 +00:00 |
|
|
kr.angelov
|
ee8f296089
|
translations for several words in the Bulgarian dictionary
|
2012-11-28 12:42:23 +00:00 |
|
|
kr.angelov
|
23873c8214
|
added familiar_A2 in DictEng, DictEngBul and DictEngGer
|
2012-11-28 12:40:20 +00:00 |
|
|
kr.angelov
|
0d35636348
|
bugfix for composite adjectivial phrases in Bulgarian
|
2012-11-28 12:37:24 +00:00 |
|
|
kr.angelov
|
6542f6ffdd
|
a bunch of additions in the parsing grammars and dictionaries plus an updated statistical model
|
2012-11-26 16:43:09 +00:00 |
|
|
kr.angelov
|
f8c302f9ef
|
remove the duplicated definition of PgfProductionIdx in parser.c
|
2012-11-19 14:16:31 +00:00 |
|
|
kr.angelov
|
ac2ea8a579
|
fix in ParseEngAbs3.probs
|
2012-11-19 14:14:08 +00:00 |
|
|
kr.angelov
|
6b0020f834
|
a statistical model for robustness in lib/src/english
|
2012-11-19 12:24:11 +00:00 |
|
|
kr.angelov
|
17ae9548d9
|
updated ParseEngAbs.probs with the latest statistics
|
2012-11-19 12:15:25 +00:00 |
|
|
kr.angelov
|
71b7c09ffe
|
bugfix for the building of bottom-up filter in the C runtime
|
2012-11-16 13:27:15 +00:00 |
|
|
kr.angelov
|
ba57ad3367
|
a couple of fixes and new words in DictEng and DictEngBul
|
2012-11-16 09:56:20 +00:00 |
|
|
kr.angelov
|
a3ba1991f4
|
revised heuristic in the statistical parser
|
2012-11-14 12:34:22 +00:00 |
|
|
kr.angelov
|
70c68f0527
|
bugfix in the statistical parser
|
2012-11-13 09:48:23 +00:00 |
|
|
kr.angelov
|
08ee662944
|
two simple heuristics which speed up the statistical parser more than seven times.
|
2012-11-12 22:17:40 +00:00 |
|
|
kr.angelov
|
68170d5b08
|
a simple refactoring in the statistical parser
|
2012-11-12 21:48:22 +00:00 |
|
|
kr.angelov
|
a2771552d6
|
more counters in the profiler for the statistical parser
|
2012-11-12 15:36:21 +00:00 |
|
|
kr.angelov
|
46de62c452
|
now we store the state instead of the offset for every continuation in the chart for the statistical parser
|
2012-11-12 14:04:52 +00:00 |
|
|
kr.angelov
|
9967c3ad04
|
in the statistical parser: move the outside probability from the parse items to their continuation. this makes the value slot shared between many items
|
2012-11-12 13:43:43 +00:00 |
|
|
kr.angelov
|
9d23093492
|
small refactoring in the C runtime
|
2012-11-12 13:05:35 +00:00 |
|
|
kr.angelov
|
a50c7c24b8
|
use size_t consistently as the type for constituent indices in the C runtime
|
2012-11-12 12:51:27 +00:00 |
|
|
kr.angelov
|
1e531e8237
|
implemented gu_map_count in runtime/c/gu/map.c
|
2012-11-12 12:42:19 +00:00 |
|
|
kr.angelov
|
52255664be
|
use prob_t instead of float in a few places
|
2012-10-29 08:52:56 +00:00 |
|
|
kr.angelov
|
0ad2405d69
|
forgot to add one #ifdef
|
2012-10-25 18:37:22 +00:00 |
|
|
kr.angelov
|
9721833680
|
a major refactoring in the robust parser: bottom-up filtering and garbage collection for the chart
|
2012-10-25 14:42:53 +00:00 |
|
|
kr.angelov
|
28b58b6267
|
add teyjus/simulator/builtins/builtins.h
|
2012-10-11 11:10:17 +00:00 |
|
|
kr.angelov
|
f0583bfd93
|
added the forgoten libteyjus.pc.in file in the C runtime
|
2012-10-11 04:22:38 +00:00 |
|
|
kr.angelov
|
bd08d98c7d
|
move examples/PennTreebank to /treebanks/PennTreebank
|
2012-10-01 08:52:54 +00:00 |
|
|
kr.angelov
|
00e85e55f8
|
Added as_Subj and UttAdV in the parsing grammars. Replaced plus_Prep with plus_Conj
|
2012-10-01 08:47:52 +00:00 |
|
|
kr.angelov
|
953633240e
|
typechecking and better error reporting in the training script for PennTreebank
|
2012-10-01 08:45:46 +00:00 |
|
|
kr.angelov
|
475109a40f
|
added the GF version of Talbanken which was imported by Malin
|
2012-10-01 08:33:43 +00:00 |
|
|
kr.angelov
|
6b7f1d2c6c
|
added AdvVPSlash and AdVVPSlash to VerbGer and an extended version of PPartNP which uses VPSlash in ParseEngGer. I guess the definitions so they might not be quite correct
|
2012-09-27 11:44:25 +00:00 |
|
|
kr.angelov
|
3845564625
|
added ParseEngGer.gf
|
2012-09-27 09:54:13 +00:00 |
|
|
kr.angelov
|
25e9f28fa4
|
add ApposNP and UncNeg to the Bulgarian parsing grammar
|
2012-09-27 09:31:37 +00:00 |
|
|
kr.angelov
|
933bdda844
|
use CNNumNP in the parsing grammars
|
2012-09-27 09:31:04 +00:00 |
|
|
kr.angelov
|
73823dbadc
|
remove no_RP from the parsing grammars and use EmptyRelSlash instead
|
2012-09-27 09:29:59 +00:00 |
|
|
kr.angelov
|
6084647328
|
added EmptyRelSlash in ExtraBul and ExtraGer. For Bulgarian and German the function simply inserts the default relative pronoun
|
2012-09-27 09:28:31 +00:00 |
|
|
kr.angelov
|
1b571d69ff
|
added according_to_Prep and ofter_AdA in DictEng, DictEngBul and DictEngGer
|
2012-09-27 09:11:04 +00:00 |
|
|
kr.angelov
|
e9800fa3eb
|
a few more words in DictEngBul
|
2012-09-27 09:09:49 +00:00 |
|
|
kr.angelov
|
dfc474580d
|
now in the parsing grammar ComplVV gets as additional arguments the polarity and the anteriority
|
2012-09-27 09:05:47 +00:00 |
|
|
kr.angelov
|
3f334fe321
|
an optimization in the German grammar for the dative/genitive variants
|
2012-09-26 11:11:42 +00:00 |
|
|
kr.angelov
|
e95e500b33
|
a bit of reordering in DictEngGer.gf
|
2012-09-26 09:16:17 +00:00 |
|
|
kr.angelov
|
b3f5835f8a
|
fixes in DictEngGer.gf
|
2012-09-26 08:52:18 +00:00 |
|
|
kr.angelov
|
fc89eaacca
|
260 new words in DictEngGer which are taken from the lexicon for patents
|
2012-09-26 08:26:39 +00:00 |
|
|
kr.angelov
|
8ba5e3fd64
|
fixes in the German parsing grammar and cleanup in DictEngGer.gf
|
2012-09-25 20:12:38 +00:00 |
|
|
kr.angelov
|
249d6cc2f8
|
fixes in the Bulgarian resource grammar. extensions in DictEng and DictEngBul
|
2012-09-24 09:41:14 +00:00 |
|
|
kr.angelov
|
18fe8af964
|
now the meta probability for a category is explicitly specified in the statistical model instead of computed internally. this avoids rounding errors while computing the sum of a large number of small values.
|
2012-09-24 09:37:21 +00:00 |
|
|
kr.angelov
|
bb15542a85
|
in the robust parser we don't have to care about trees which yeld empty strings. this makes the parser a lot faster
|
2012-09-24 09:30:20 +00:00 |
|
|
kr.angelov
|
f75d1374ff
|
the Haskell runtime now exports 'functionsByCat' which returns the list of all functions for a given category
|
2012-09-18 09:48:21 +00:00 |
|
|
kr.angelov
|
e98d62c42a
|
a few changes in DictEng
|
2012-09-18 09:25:22 +00:00 |
|
|
kr.angelov
|
44df7a33cf
|
the C runtime now has a type prob_t which is used only for probability values
|
2012-09-18 09:18:48 +00:00 |
|
|
kr.angelov
|
dd0e9411e1
|
catch all case in GenerateBC.genFun
|
2012-09-18 08:32:21 +00:00 |
|
|
kr.angelov
|
93220a2d87
|
The training script for the PennTreebank is updated to compute more statistics
|
2012-09-07 08:18:11 +00:00 |
|
|
kr.angelov
|
cd3cca4aa2
|
bugfix in the C parser
|
2012-09-06 14:52:19 +00:00 |
|
|
kr.angelov
|
1d8fb27656
|
updates to ParseEng and ParseEngBul
|
2012-09-06 13:27:08 +00:00 |
|
|
kr.angelov
|
e34d18310a
|
update the statistical model for ParseEng
|
2012-09-06 13:26:16 +00:00 |
|
|
kr.angelov
|
5c1183ebc8
|
extensions to the English and the Bulgarian grammars. Infinitives with polarity and anteriority
|
2012-08-31 09:31:36 +00:00 |
|
|
kr.angelov
|
37e1ced6cf
|
another fix for teyjus
|
2012-08-30 08:09:30 +00:00 |
|
|
kr.angelov
|
fe27540c49
|
the loading of PGF files was broken by the Teyjus patch. Now this is fixed
|
2012-08-30 07:41:49 +00:00 |
|
|
kr.angelov
|
9cf52d1f56
|
a few fixes in DictEng, DictEngBul and ParseEng
|
2012-08-29 11:46:58 +00:00 |
|
|
kr.angelov
|
fd5220038d
|
A basic infrastructure for generating Teyjus bytecode from the GF abstract syntax
|
2012-08-29 11:43:02 +00:00 |
|
|
kr.angelov
|
20d3862b17
|
fixes in DictEng and extensions in DictEngBul
|
2012-08-09 12:09:21 +00:00 |
|
|
kr.angelov
|
3ef3108314
|
fix some warnings in teyjus
|
2012-08-03 12:30:01 +00:00 |
|
|
kr.angelov
|
b34232c6dc
|
the first draft of GF.Compile.Instructions
|
2012-08-01 09:21:59 +00:00 |
|
|
kr.angelov
|
2844742afd
|
An initial import of the teyjus source code in the C runtime for GF. The two runtime are still not connected but the source code compiles.
|
2012-07-31 15:16:04 +00:00 |
|
|
kr.angelov
|
696a34bc22
|
DictEngBul is extended to 5000 words and there are few more functions in ParseEngBul
|
2012-06-29 09:57:02 +00:00 |
|
|
kr.angelov
|
852d923786
|
added VPI category and AdvVPSlash to the Bulgarian RGL
|
2012-06-26 09:54:06 +00:00 |
|
|
kr.angelov
|
70dc5d2c21
|
bugfix in GenRP in ExtraEng
|
2012-06-21 09:00:46 +00:00 |
|
|
kr.angelov
|
87401fcc4d
|
added few missing functions in the Bulgarian RGL
|
2012-06-21 09:00:21 +00:00 |
|
|
kr.angelov
|
78bd39ace7
|
various extensions to DictEngBul
|
2012-06-21 08:59:37 +00:00 |
|
|
kr.angelov
|
87818d4404
|
extensions to DictEng and addition of parallel DictEngBul which is partly imported from Apertium
|
2012-06-20 09:06:59 +00:00 |
|
|
kr.angelov
|
5b22d474ef
|
fix UncNeg in ParseEng
|
2012-06-13 06:10:51 +00:00 |
|
|
kr.angelov
|
3a352a953f
|
Use a separated tag for meta productions in the robust parser. This cleans up the code a lot
|
2012-06-13 05:49:30 +00:00 |
|
|
kr.angelov
|
7549a4876d
|
now there is a limit of 2000000 items in the chart of the robust parser. This prevents from explosion in the memory size but it will also prevent us from parsing some sentences.
|
2012-06-12 11:30:01 +00:00 |
|
|
kr.angelov
|
8f53e010d1
|
improved script for training from PennTreebank
|
2012-06-12 10:05:17 +00:00 |
|
|
kr.angelov
|
9775c1002b
|
fix the path in DictEngGer.gf
|
2012-06-12 10:04:15 +00:00 |
|
|
kr.angelov
|
99d117534d
|
more in DictEng
|
2012-06-12 09:58:01 +00:00 |
|
|
kr.angelov
|
b765b0c054
|
now the robust parser is purely top-down and the meta rules compete on a fair basis with the grammar rules
|
2012-06-12 09:29:51 +00:00 |
|
|
kr.angelov
|
5a27861038
|
several updates to DictEng and ParseEng
|
2012-05-25 07:33:40 +00:00 |
|
|
kr.angelov
|
cab4602b62
|
the viterbi probability for the epsilon categories is now updated properly
|
2012-05-25 07:30:35 +00:00 |
|
|
kr.angelov
|
ec00a42ee6
|
fixed an alignment bug in libgu which leads to crash on some platforms
|
2012-05-25 07:29:29 +00:00 |
|
|
kr.angelov
|
bd102d54d9
|
fixed memory leak in the pgf-translate tool
|
2012-05-25 07:28:18 +00:00 |
|
|
kr.angelov
|
fdcd0de336
|
added pgf-service.c in the robust parser'
|
2012-05-25 07:27:29 +00:00 |
|
|
kr.angelov
|
3549db552e
|
a bunch of new things in DictEng and ParseEng a simple fix for the VPI category in ExtraEng
|
2012-05-21 15:06:41 +00:00 |
|
|
kr.angelov
|
bd8046f23d
|
another attempt to port the robust parser to MacOS
|
2012-05-16 15:18:44 +00:00 |
|
|
kr.angelov
|
e342e5e7d6
|
small fix for compiling the robust parser on MacOS
|
2012-05-16 12:18:50 +00:00 |
|
|
kr.angelov
|
4aca965109
|
a new unbiased statistical parser. it is still far from perfect use it on your own risk.
|
2012-05-08 12:13:28 +00:00 |
|
|
kr.angelov
|
38ed063405
|
fixes in DictEng and ParseEng
|
2012-05-07 21:40:14 +00:00 |
|
|
kr.angelov
|
ed6a53609b
|
yet another fix for parsing literals
|
2012-04-18 15:50:55 +00:00 |
|
|
kr.angelov
|
c6c54f8815
|
some fixes in the robust parser and a new API for literals
|
2012-04-12 06:55:25 +00:00 |
|
|
kr.angelov
|
18185aba97
|
insert spaces in the hyphenated words in DictEng. Various small fixes in DictEng and ParseEng
|
2012-04-10 09:15:37 +00:00 |
|
|
kr.angelov
|
99cc07ad67
|
simple cleanup in the robust parser
|
2012-04-02 19:01:18 +00:00 |
|
|
kr.angelov
|
2bf3f22fac
|
libpgf: a new implementation for literals which also allows custom literals. the same mechanism is now used for the metavariables
|
2012-03-12 14:25:51 +00:00 |
|
|
kr.angelov
|
c1b2246fa9
|
libpgf: added simple lexer
|
2012-03-09 09:14:44 +00:00 |
|
|
kr.angelov
|
1da464a4cc
|
libpgf: implementation for built in literal categories
|
2012-03-07 16:39:29 +00:00 |
|
|
kr.angelov
|
bf81c0f77f
|
libpgf: simple fix in the parser debugger
|
2012-03-07 12:23:07 +00:00 |
|
|
kr.angelov
|
e871330665
|
libpgf: two APIs - one for finding all parse results and another for finding the best parse result
|
2012-03-07 11:00:17 +00:00 |
|
|
kr.angelov
|
791a1a17b0
|
libpgf: now all concrete functions and categories are explicitly linked to their abstract counter parts
|
2012-03-05 12:59:31 +00:00 |
|
|
kr.angelov
|
fdf6dd7798
|
libpgf: preliminary version for the statistical ranking. we use naive statistical model with random weight for the meta variables.
|
2012-03-02 19:25:01 +00:00 |
|
|
kr.angelov
|
79c7141973
|
libpgf: simple optimization in the implementation for heaps
|
2012-03-02 17:43:46 +00:00 |
|