Commit Graph

3918 Commits

Author SHA1 Message Date
hallgren
5cf0ab4110 PGFService: make the restriction on parallelism more effective
Restrict the entire request handler instead of just the call to the C parser.
2015-02-25 16:08:21 +00:00
krasimir
0764d518c7 fix some out-of-date code for the PGF_RESULT_DEBUG mode in the parser 2015-02-26 13:31:26 +00:00
krasimir
42908bd95f use the same trick for comparison of doubles in cmp_expr_state 2015-02-26 13:10:01 +00:00
krasimir
dfd69d5d4b bugfix in the parser which caused some items to be poped in the wrong order if their probability differ with less that 0.5 2015-02-26 12:17:59 +00:00
hallgren
83c5269a84 Wide Coverage Translation Demo: better resizing behavior on startup and when pasting text 2015-02-25 15:36:25 +00:00
hallgren
7631527395 PGFService.hs: text lexer: slightly smarter decapitalization of the first word of a sentence
Keep the first letter in upper case if it is followed by more upper case
letters.

(Also remove some left over debugging output.)
2015-02-24 15:04:06 +00:00
krasimir
79a7ae6553 remove three more uses of meta productions that were left in debugging mode 2015-02-24 10:59:18 +00:00
krasimir
8664dd80ed implemented escape characters when reading/printing an abstract expression with string literals 2015-02-24 09:35:12 +00:00
krasimir
d71d59cdc0 fix in thai_page2.xml 2015-02-20 15:31:32 +00:00
hallgren
8d0bb75121 gftransate.js: increase sentence length limit for "fast" language to 500
Currently Bul, Chi, Eng and Swe are marked as "fast" in the documentation.
2015-02-20 14:34:37 +00:00
krasimir
205403be76 set the keyboard pages for Thai in Translator.java 2015-02-20 14:26:30 +00:00
krasimir
cefd6440fe experimental Thai keyboard (still not tested) 2015-02-20 14:08:12 +00:00
joel.hinz
186cc84d37 translation app for iOS, replicating some of the functionality of the Android app. Compiles and works on iPad retina but may crash and has known issues. 2015-02-20 07:47:43 +00:00
krasimir
3ee931f905 added option -plus-as-bind which treats (+) as a bind when used with runtime variables 2015-02-20 13:26:12 +00:00
krasimir
9c6c6b6346 remove the meta prob flags 2015-02-20 13:00:51 +00:00
hallgren
913e56081a gftranslate.js: reduce sentense length limit to 200 (URL encoded) characters
This limit might still be to high to avoid excessive time/space use in the
parser for certain languages in the wide coverage translation grammar.
2015-02-20 12:31:41 +00:00
hallgren
b72b8dcef8 PGF Service: limit the number of parallel calls to the C run-time parse function to 4 by default
The limit can be changed with the -j flag
2015-02-20 12:29:44 +00:00
krasimir
87c0368281 silence some warnings in GNU Lightning for i386_64 2015-02-19 12:53:52 +00:00
krasimir
2f35aadc6f parsing with meta rules is now removed since we don't use them anymore and I would gladly remove some code. 2015-02-19 11:59:51 +00:00
krasimir
4ff6728993 fix the missmatch between Int and CInt in the Haskell binding to the word alignment API. This was causing problems on 64-bit machines 2015-02-18 17:05:35 +00:00
krasimir
f3f47c7761 fix the callback for unknown words as well 2015-02-18 16:47:33 +00:00
krasimir
4ed41214e5 fix in the callback for names in the Haskell binding 2015-02-18 16:41:13 +00:00
krasimir
27a12ca9fb fix in the parser for callbacks in the middle of a word 2015-02-18 16:14:29 +00:00
hallgren
4bd568d8bf Wide Coverage Translation Demo: use App14.pgf, remove spaces from Chi, Jpn and Tha output 2015-02-18 13:58:49 +00:00
aarne
36b1c421a6 removing spaces from App output of Chi,Jpn,Tha to get nicer speech output (and of course follow the target lang conventions) 2015-02-17 18:11:20 +00:00
aarne
98cbd993ac App14 with Tha up and running! 2015-02-17 17:18:51 +00:00
hallgren
1dbe588497 Restore compatibility with ghc-7.4
With ghc-7.4 'import M hiding (x)' causes an error if M does not export x...
2015-02-16 15:41:38 +00:00
hallgren
2ce3e954fd Changes for compatibility with ghc-7.10-rc2
2 modules: Name clashes caused by Applicative-Monad change in Prelude
2 modules: Ambiguities caused by Foldable/Traversable in Prelude
2 modules: Backwards incompatible changes in time-1.5 for defaultTimeLocale
9 modules: {-# LANGUAGE FlexibleContexts #-} (because GHC checks inferred types
           now, in addition to explicitly given type signatures)

Also silenced warnings about tab characters in source files.
2015-02-16 15:05:06 +00:00
hallgren
0058a79a8f PGF.Haskell: adding operators for selections from tables 2015-02-12 16:09:33 +00:00
hallgren
43a873b53f Translating linearization functions to Haskell: more simplifications
+ Some additional simplifying rewrites.
+ Use an intermediate representation for Haskell types, for separation of
  concerns and cleaner code.
+ Pretty printer layout tuning
+ Code cleanup.
2015-02-12 16:05:48 +00:00
hallgren
686f570660 Translating linearization functions to Haskell: simplify the generated Haskell code
Introduced an intermediate representation for the generated Haskell expressions.
This allows pretty printing concerns to be separated from conversion concerns,
and makes it easy to apply some simplifying rewrites to the generated
expressions, e.g.

	[x] ++ [y]    ==> [x,y]
	pure f <*> x  ==> f <$> x
	f <$> pure x  ==> pure (f x)
	join (pure x) ==> x
2015-02-11 23:50:19 +00:00
kr.angelov
e6f51d7ec6 fix in the JIT compiler that would hopefully make it work on iOS 2015-02-11 13:07:06 +00:00
hallgren
ad8b6429ec Translating linearization functions to Haskell: support for variants
By adding the flag -haskell=variants to the command line, GF will now generate
linearization functions in Haskell that support variants. Variants are
represented as lists in Haskell.

Variants inside pre { ... } expressions are still ignored.

TODO: apply some monad laws to generate more compact code (using an
intermediate representation of the generated Haskell code, instead of
pretty printing directly from the GF code).
2015-02-09 16:24:33 +00:00
kr.angelov
1baba82fe5 added hasLinearization in the Haskell binding 2015-01-30 13:05:57 +00:00
aarne
3c13e1abb0 added Japanese to Android app. Had to change target=android-20 to 21. 2015-01-21 22:20:25 +00:00
hallgren
28803ba322 Wide Coverage Translation Demo: use App13.pgf, adding support for Japanese
Japanese uses the same lexer as Chinese, i.e. every character is a separate
token.
2015-01-21 16:02:37 +00:00
hallgren
ed502c490c PGF2: fixes for named entity callback function 2015-01-21 13:54:48 +00:00
hallgren
ec8b4fec93 PGF2 & PGFService: enable callbacks for named entities and chunks in the c-parse & c-translate web requests 2015-01-21 10:41:12 +00:00
kr.angelov
60cb95258b fix conflicts 2015-01-21 08:35:20 +00:00
kr.angelov
07eb8fec26 added functionType in the Haskell API to the C runtime 2015-01-20 14:45:02 +00:00
hallgren
7577de236f PGF2 & PGFService: work in progress on callbacks for nerc & chunks 2015-01-20 14:57:52 +00:00
hallgren
4372b47d2a PGF2: introduced some type synonyms to make type signature more readable 2015-01-20 12:57:33 +00:00
hallgren
240ba80209 Translating linearization functions to Haskell: move a common record type to PGF.Haskell
Move the Haskell representation of the common linearization type {s:T} to the
shared module PGF.Haskell, so that the same overloaded projection function
proj_s can be used for all concrete syntaxes.
2015-01-19 12:43:32 +00:00
hallgren
0b114195aa Translating linearization functions to Haskell: better treatment of special tokens
Common code has been lifted out from the generated Haskell modules to
an auxiliary module PGF.Haskell, which is currently included in the
regular PGF library, although it is independent of it and probably belongs
in a separate library.

The type Str used by linearization functions is now based on a token
type Tok, which is defined in PGF.Haskell.

PGF.Haskell.Tok is similar to the type GF.Data.Str.Tok, but it has
constructors for the special tokens BIND, SOFT_BIND and CAPIT, and there is
a function

	fromStr :: Str -> String

that computes the effects of these special tokens.
2015-01-14 14:35:39 +00:00
hallgren
4348ae40d2 Translating linearization functions to Haskell: add support for pre {...}
STILL TODO:

	- variants
	- better treatment of special tokens BIND, SOFT_BIND & CAPIT.
2015-01-08 17:52:45 +00:00
hallgren
6db2845375 Translating linearization functions to Haskell: use qualified names to avoid name clashes
All languages in the Phasebook can now be converted to compilable Haskell
code.

STILL TODO:
  
  	- variants
  	- pre { ... }
2015-01-07 16:13:28 +00:00
hallgren
51a233b2f1 Translating linearization functions to Haskell: significant code size reductions
+ Instead of including lists of parameter values generated by GF, generate
  code to enumerate parameter values (in the same order as GF). This seems
  to give a factor of 2-3 code size reduction in the Phrasebook (e.g.
  from 84MB to 25MB for Hin, from 338MB to 154MB for Fre).

+ Deduplicate table entries, i.e. convert "table [..,E,..,E,..,E,..]" into
  "let x = E in table [..,x,..,x,..,x,..]". This gives even more significant
  code size reduction in some cases, e.g. from 569MB to 15MB for
  PhrasebookFin.

All phrasebook languages can now be converted to compilable Haskell code,
except PhrasebookPes, which still has the name clash problem.
2015-01-06 19:57:24 +00:00
hallgren
1f60646f41 More work on translating linearization functions to Haskell
Many Phrasebook languages can now be converted to compilable Haskell code.
Some languages (Fre, Hin, Snd, Urd) generate too much Haskell code to be
practically useful (e.g. 338MB for Fre). One language (Fin) took too long
to convert to Haskell. One language (Pes) has problems with name clashes in
the generated Haskell code.

STILL TODO:

  	- variants
  	- pre { ... }
  	- reduce code duplication for large tables
	- generate qualified names to avoid name clashes
2015-01-06 16:48:03 +00:00
kr.angelov
6acf100ab1 fix the reference counting for callbacks in Python 2015-01-05 11:24:00 +00:00
kr.angelov
328d4df660 API for word alignment in the C runtime and in the Haskell binding 2014-12-29 10:59:20 +00:00