Commit Graph
100 Commits
Author SHA1 Message Date
hallgren 2721f7358f Spring cleaning
Nothing major...
2014-04-28 13:56:20 +00:00
hallgren b5fd7d88c1 Simple Translator: remove extra space before punctuation 2014-04-28 12:07:41 +00:00
hallgren e891e99725 Simple Translator: remove punctuation before wide coverage translation
This gives better translations with Translate11.pgf and is how the Wide
Coverage Demo App works.
2014-04-24 11:28:15 +00:00
hallgren e3ff79d49c Some work on the download page and release notes for the next release of GF
Please add things you think are imporant to mention to the release notes!
2014-04-22 14:45:37 +00:00
hallgren 570a417615 gf.cabal: require base>=4.3
This makes it explicit that we no longer support compiling GF with GHC 6.12
(which is 4 years old and comes with base-4.2).
2014-04-22 14:38:33 +00:00
hallgren a2c25e88d4 gf.cabal: add GF builtin default -K64M. Enable parallel compile with ghc>=7.8
+ Programs compiled with ghc<7.8 have a default built-in 8M stack size limit.
  With ghc>7.0 this built-in default can be changed, so it is now 64M, allowing
  GF to work with larger grammars without manually increasing the stack size.
  (But if GF is compiled with ghc 6.12, the built-in default is still 8M.)

+ Enable ghc>=7.8 support for compiling modules in parallel, to speed up the
  compilation of GF.
2014-04-22 12:43:24 +00:00
hallgren 0e21bcde22 Wide Coverage Demo web app: more client side caching
For example, you can now reorder paragraphs by cut & paste, without causing
anything to be sent to the server for re-translation.

Also some color & layout tweaks.
2014-04-17 13:48:00 +00:00
hallgren 5cb5cdd31e Wide Coverage Demo App: show a word-for-word translation if all else fails 2014-04-16 15:37:09 +00:00
hallgren dd96c8a2c6 PGF web API: adding the command c-wordforword
It has the same parameters and result format as c-translate, but it does
the translation word for word. (To be used as a last resort).
2014-04-16 15:24:23 +00:00
hallgren 7fc3627c6e PGF web API: add commands lookupmorpho and c-lookupmorpho 2014-04-15 14:50:38 +00:00
hallgren 2ea326a83e gftranslate.js: switch to Translate11.pgf 2014-04-11 20:08:12 +00:00
hallgren ba825ae81a Wide Coverage Demo App (gftranslate.js): use a special lexer for Chinese. 2014-04-11 18:39:12 +00:00
hallgren 6601a3972e Simple Translator: use colors to indicate translation quality 2014-04-11 15:00:15 +00:00
hallgren ea389df58e Simple Translator: Use PGF API for (un)lexing instead of separate server calls
Also,

  + Get up to 10 (instead of 5) alternative robust translations.
  + Remove duplicates from list of alternative robust translations
  + Add GF Cloud logo to the simple translator page
2014-04-11 12:10:05 +00:00
hallgren 4008a2b111 PGF web service: disable caching of parse results
Caching parse results uses a lot of memory, even if they expire after
2 minutes, so it won't scale up to many simultaneous users.

But some excessive memory use seems to be caused by space leaks in
(the Haskell binding to) the C run-time system, and these should be fixed.
For example, flushing the PGF cache does not release the memory allocated
by the C run-time system when loading a PGF file.
2014-04-10 15:55:33 +00:00
hallgren 5f75baf56a Wide Coverage Demo App: it now works ok without server caching of parse results 2014-04-10 15:50:34 +00:00
hallgren d19e6d9d07 Wide Coverage Demo App: new buttons: to switch colors on/off & to try Google Translate 2014-04-09 20:51:31 +00:00
hallgren 402cccc3ca localstorage.js: bug fix
If you stored "false" and tried to retrieve it with a default value, you got
the default value instead of "false".
2014-04-09 20:49:53 +00:00
hallgren 96f36d8f0a Wide Coverage Demo App: use PGF service lexing/unlexing
It is now enough to make one server call per translation, and the results can
cached by the browser.
2014-04-09 19:49:45 +00:00
hallgren d1da0e06de PGF web service: add unlexers and enable client side caching
Most PGF web API commands that produce linearizations now accept an
unlexer parameter. Possible values are "text", "code" and "mixed".

The web service now include Date and Last-Modified headers in the HTTP,
responses. This means that browsers can treat responses as static content and
cache them, so it becomes less critical to cache parse results in the server.

Also did some cleanup in PGFService.hs, e.g. removed a couple of functions
that can now be imported from PGF.Lexing instead.
2014-04-09 17:51:25 +00:00
hallgren 50ea3d265c Change the type of PGF.Lexing.bindTok to [String] -> [String]
The old type was [String] -> String. This function was only used
in GF.Text.Lexing.stringOp, which now uses (unwords . bindTok) instead,
with no change in behaviour.
2014-04-09 17:39:21 +00:00
hallgren 677d849840 Unlexers: move capitalization of first word from GF.Text.Lexing to PGF.Lexing
The capitalization of the first word was done in GF.Text.Lexing.stringOp,
but is now done in the functions unlexText and unlexMixed in PGF.Lexing.
These functions are only used in stringOp and in PGFService (where the change
is needed), so the subtle change in behaviour should not cause any bugs.
2014-04-09 17:26:23 +00:00
hallgren 04a6260eea PGF Service: a bit more clever lexer=text
Only change the first word to lowercase if the original input is not found in
the grammar's morphology. This allows parsing of sentenses starting with "I" in 
English, nouns in German and proper names in other languages, but it can make
the wrong choice for multi-words.
2014-04-09 14:13:18 +00:00
hallgren 4479bb81b7 Wide Coverage Demo App updates
+ Add Clear button.
+ Use localstorage to remember entered text between visits.
+ Add a link to it on the GF cloud service start page.
2014-04-09 13:59:35 +00:00
hallgren 622ab0b44c Fix build problem in PGFService.hs when configured without c-runtime 2014-04-08 20:45:54 +00:00
hallgren 29efca71cb PGF web service: add lexer support
The PGF web API commands parse, translate, translategroup, c-parse and
c-translate now support a lexer parameter. Possible values are "text",
"code" and "mixed".

This is used in the Wide Coverage Demo App to save one server roundtrip.
2014-04-08 16:10:33 +00:00
hallgren 9cac98a356 Move basic lexing functions from GF.Text.Lexing to the new module PGF.Lexing
They are thus part of the PGF Run-Time Library, making it possible to add
lexing functionality in PGF service in a natural way.
2014-04-08 14:07:49 +00:00
hallgren 936f3d8297 Wide Coverage Demo App: remove quality markers ("+" or "*") before unlexing
Otherwise the first word won't be capitalized.
2014-04-08 13:46:57 +00:00
hallgren 64623d4881 Wide Coverage Demo App: restore swap button 2014-04-08 11:32:54 +00:00
hallgren c8a8883df8 demos/translation.t2t: more space for the demo app 2014-04-07 13:01:00 +00:00
hallgren 5868118ea9 Wide Coverage Demo Web App: add segmentation
Click on sentences to choose among alternative translations.
2014-04-07 12:51:13 +00:00
hallgren a0adabb3a1 Wide coverage demo app: show an error message even if the error string is empty 2014-04-06 23:44:31 +00:00
hallgren ea67953012 Wide coverage demo app: show an error message if linearization fails 2014-04-06 23:13:18 +00:00
hallgren 5344d42536 gftranslate.js: use Translate10.pgf 2014-04-06 19:59:04 +00:00
hallgren d6d056034b TypeCheck.RConcrete: contravariance in table types 2014-04-04 16:50:12 +00:00
hallgren c58649bd29 Removed imports of TypeCheck.Concrete, fixed problems in TypeCheck.RConcrete
+ The current type checker for concrete syntax is in
  GF.Compile.TypeCheck.RConcrete, but GF.Compile.TypeCheck.Concrete was
  still imported in GFI.

+ Fixed a bug that allowed Ints n as a subtype of Ints m, regardless of
  m and n. It now requires n<=m. Note: the type checker still allows Int
  as a subtype of Ints m, regardless of m.

+ Fixed a potential efficiency problem with large record types, by reducing
  the number of recursive calls from |R|*|S| to |R| when checking if R<=S.

+ Fixed a misleading comment: "alpha g t u" checks that u is a subtype of t,
  the other way around. Similarly, "checkIfEqLType gr g t u trm" checks that
  u is a subtype of t, not the other way around, and not that t is equal to u.
2014-04-04 13:51:07 +00:00
hallgren 8986541160 Wide Coverage Translation Web App: redesigned user interface
The layout now resembles a certain other online translation service.
The translation is started automatically after text is entered, no need to
press the Translate button.
2014-04-04 12:52:41 +00:00
hallgren 5cc0f39520 gftranslate.js: increase length limit from 200 to 500 encoded bytes 2014-04-04 12:39:36 +00:00
hallgren 6cbd1a753c Simple Translator: show more alternatives when using GF robust translation
The maximum number of alternatives shown has been increased from 2 to 5
although it can slow down translation.
2014-04-04 12:35:20 +00:00
hallgren 9ceadecf89 Slightly more informative error messages in translation web apps 2014-04-04 12:33:31 +00:00
hallgren 4018073bdb PGFService.hs: adapt to changes in the Haskell binding to the C run-time system
Parse errors used to cause crashes, but they are now handled and reported.
2014-04-04 12:28:00 +00:00
hallgren 23c9a042cb Bug fix for pattern macros in pre{}
This bug was introduced sometime between 2013-08-21 and 2013-11-01 and caused
the function convertTerm in GF.Compile.GeneratePMCFG to encounter a EPatt where
it expected Strs. I fixed it by applying the function getPatts (from the old
partial evaluator) to the pattern.
2014-04-01 10:54:41 +00:00
hallgren 57b2a2bc2c Wide coverage demo web app: green translations from Phrasebook.pgf
The demo web app now show translations from Phrasebook.pgf in green before
translations from Translate8.pgf, which can be yellow or red.
2014-03-31 15:28:14 +00:00
hallgren 98d9946ac4 Wide coverage demo web app improvements
+ Adapt language selection menus to the languages supported by the grammar.
+ Add translation quality colors to the links to alternative translations.
+ Also added more country codes in langcode.js (needed for speech synthesis
  voice selection).
2014-03-28 14:56:56 +00:00
hallgren 201283c055 Wide coverage demo web app: translation quality feedback with colors
The background color of the translation changes to pale green, yellow or red
according to the quality.

Two types of quality indications are supported:

+ From the grammar: linearizations starting with "+" or "*" are assumed to
  be of high or low quality, respectively.
+ From the parse tree: if the root function is "?", then the translation is
  assumed to be of low quality.

If no quality indications are detected, medium quality is assumed.
2014-03-27 15:11:02 +00:00
hallgren 9bfed99790 support.js & gftranslate.js: improved handling of server errors
In support.js, the functions http_get_json, ajax_http_get_json
and ajax_http_post_json now calls the supplied error callback if the server
returns invalid JSON (e.g. because of a crash).

The function gftranslate.translate in gftranslate.js returns
a JSON value containing an error message (since it doesn't have an error
callback).

This should result in fewer situations where "nothing happens" and the user
doesn't know if it is beacuse the server is slow, or if there was an error.
2014-03-27 15:02:27 +00:00
hallgren b061d59044 gf -server: add a command to manually flush the PGF cache
This can be used if the cloud service seems slow, but it would probably
be better to automatically expire unused PGFs from the cache after some time.
2014-03-19 16:15:05 +00:00
hallgren 22b68ad30c Wide coverage demo web app: add speech synthesis
Works in Safari and Chrome.
2014-03-17 15:59:43 +00:00
hallgren 825e0cf7cf doc/Logos/gf1.svg: variant of the GF logo with a subtle shadow 2014-03-13 14:50:30 +00:00
hallgren f6c7094955 GF home page: news item about Digital Grammars 2014-03-13 13:15:43 +00:00
hallgren ca8dd1e8cc support.js: For better error handling, always use AJAX, don't fallback to JSONP
Cross-origin requests are allowed by the PGF service and are supported in
all modern browsers.
See http://en.wikipedia.org/wiki/Cross-origin_resource_sharing
2014-03-12 14:07:35 +00:00
hallgren f092e7fe5b haskell-bind: comment out superfluous imports 2014-03-12 13:21:47 +00:00
hallgren ccc43b28f0 WebSetup.hs: use the correct --gf-lib-path when building example grammars
The directory should be the same as the one used in Setup.hs when building
the RGL, where a build directory supplied by cabal is used. This is usually
dist/build/rgl, but it could be different, e.g. when using a cabal sandbox.
2014-03-11 13:46:10 +00:00
hallgren 052ead6797 C run-time: protect from Windows specific #includes with #ifdef __MINGW32__ 2014-03-06 13:42:27 +00:00
hallgren d04ffb60ac Fix the fix 2014-03-04 14:35:14 +00:00
hallgren 108e04b7bd Fixes for GHC 7.6 compatibility 2014-03-04 14:28:44 +00:00
hallgren 76ba865c2e Three fixes for compatibility with GHC 7.8
Two of the fixes were in the custom version of the binary package. We
should get rid of it at some point, to reduce the maintenance burden.

The third fix was to add new version constraints for happy and alex in
gf.cabal. New versions of them are needed because of the ugly, low-level,
GHC-specific code they produce need to be different for GHC 7.8.

More fixes might be needed for -server mode, but the cgi package is not
compatible with GHC 7.8 at the moment, so it will have to wait.
2014-03-04 14:13:41 +00:00
hallgren 9b1b42bd22 Wide coverage demo web app: add link to Google Translate 2014-02-19 14:17:49 +00:00
hallgren 62dcb1e14a src/www/js/langcode.js: functions for converting language codes
factored out from src/www/translator/translator.js
2014-02-19 14:15:14 +00:00
hallgren 29ac39c5cc demos/translation.t2t: the translation app needs more space 2014-02-13 12:02:10 +00:00
hallgren 2a130d7565 Wide coverage demo: show up to 10 translations
The translations are loaded one at a time so you don't have to wait for all
10 to see the first one.
2014-02-12 16:08:18 +00:00
hallgren 44ee072331 Work on web api & apps based on the C run-time system
+ PGFService.hs: add command c-grammar, include probability in parse results
+ js/gftranslation.js: add start position and limit parameters, return more
  info to applications
+ Simple Translator: show two wide coverage translations
+ Wide coverage demo: show parse tree and probability (intended as grammar
  debugging aids)
2014-02-11 16:43:29 +00:00
hallgren 23dc22cea4 GFServer.hs: avoid intertwined log messages from parallel requests 2014-02-11 14:22:12 +00:00
hallgren c9af5d11c0 Restore compatibility with GHC 7.0 2014-02-10 21:44:34 +00:00
hallgren 60980d1617 Fix broken C runtime support in gf.cabal and PGFService.hs
Also add PGF service command c-flush to explicitly flush cached parse results
from memory.
2014-02-10 16:04:40 +00:00
hallgren 1bcb2d06e3 haskell-bind: fix broken pgf-shell
The type CId was replaced with String, so can't use show and read anymore.
2014-02-10 12:59:31 +00:00
hallgren 1d69feac38 gf.cabal: pgf-shell needs containers now 2014-02-07 16:42:51 +00:00
hallgren 680bf64454 pgf-shell: show the names of the concrete syntaxes after loading the grammar 2014-02-07 16:40:13 +00:00
hallgren 9a422839cc pgf-shell: enable -rtsopts, call performGC between commands
This is to make it easier to find the cause of space leaks.
2014-02-07 13:57:13 +00:00
hallgren edeb094cf5 PGF.Optimize: one more fix for unsafeFreeze 2014-02-04 13:47:37 +00:00
hallgren 9df56550cd PGF.Optimize: import unsafeFreeze from Data.Array.Unsafe for compatibility with ghc-7.8 2014-02-04 13:44:07 +00:00
hallgren 3900d3cfbb GFC.hs: encodeFile_ is not available in the standard Data.Binary 2014-02-04 13:36:16 +00:00
hallgren c97e5f4086 gf.cabal: add default-language to silence warning 2014-02-03 12:24:42 +00:00
hallgren dbd68b987e gf-server.cabal: flag to include C run-time support in the FastCGI PGF service 2014-01-31 16:11:41 +00:00
hallgren 0386334f49 Wide coverage translation web demo improvements
+ Add lexing/unlexing
+ Add a button to swap input and output
+ Minor layout adjustments
2014-01-28 16:51:21 +00:00
hallgren 06bb2a1dad PGFService.hs: fix a type ambiguity when compiling without C run-time support 2014-01-27 16:46:48 +00:00
hallgren b3251f402d PGFService.hs: add a cache for C run-time parse result and a start offset parameter
Cached parse results are discarded when they have been unused for 10 minutes.
2014-01-27 16:32:09 +00:00
hallgren 88341987a8 gf -server: fix a redirection bug...
...that was introduced with the recent changes to the handling of the
current working directory
2014-01-27 16:30:03 +00:00
hallgren daa48610b5 gf -server: allow multiple PGF service requests to be handled in parallel
PGF service requests are stateless and can run in parallel, but some other
requests handled by the server are not and might even change the current
working directory temporarily, and this affects all threads, so it is
important that the PGF service requests access PGF files by absolute paths.
2014-01-24 13:49:02 +00:00
hallgren 77150e0b27 gf.cabal: bump version number to 3.5.12 2014-01-21 16:30:01 +00:00
hallgren 0a185814fd demos/translation: add preliminary online demo,
but it is not ready to be widely advertised yet, e.g. lexing and unlexing is
missing.
2014-01-23 14:56:52 +00:00
hallgren 14ce8509f8 simple translator: Replaced 'GF Robust Parser' with 'GF Wide Coverage Translation'
The interface to the wide coverage translation is in js/gftranslate.js
and it assumes that the grammar is installed on the cloud server
as /robust/Translate8.pgf.

The list of supported languages is hardwired in gftranslate.js, since
there is no support for obtaining this info from the C run-time system
at the moment.
2014-01-21 16:47:23 +00:00
hallgren 9d71ffc831 Optionally include C run-time support
If the C run-time library is compiled and installed on your system, you can now
do 'cabal configure -fc-runtime' to get the following extras:
  + The haskell binding to the C run-time library will be included in the
    PGF library (so you can import it in Haskell applications).
    Documentation on the new modules will be included when you run
    'cabal haddock'.
  + The new command 'pgf-shell', implemented on top of haskell binding to
    the C run-time system.
  + Three new commands in the web API: c-parse, c-linearize and
    c-translate. Their interfaces are similar to the corresponding commands
    without the "c-" prefix, but they should be considered preliminary.
2014-01-20 17:06:11 +00:00
hallgren 801a20d27a haskell-bind: add an export list to hide internal stuff
It also makes the generated haddock documentaiton more structured.
2014-01-20 14:40:38 +00:00
hallgren d2d57a9210 haskell-bind.cabal: relax overly restrictive version bounds and add cc-options 2014-01-17 14:18:20 +00:00
hallgren 045eeeb280 haskell-bind: add a cabal file and examples/pgf-shell.hs
* The haskell-bind.cabal file makes it easy to build the haskell binding and
  use it in ghci.
* pgf-shell.hs is a simple example of how to use the haskell binding.
2014-01-17 14:02:35 +00:00
hallgren b0a40d82b9 haskell-bind: change the type of CRuntimeFFI.getConcr
Change the type of getConcr from

    getConcr :: PGF -> Language -> Concr

to

    getConcr :: PGF -> Language -> Maybe Concr

This is to prevent programs from crashing later if you try to select a
concrete syntax that is not present in a grammar.
2014-01-17 13:58:48 +00:00
hallgren 41e3b28bfa examples/Phasebook/Makefile2: Lativan is fixed, include it again 2014-01-13 14:18:32 +00:00
hallgren ac33c8a851 examples/Phasebook/Makefile2: Lativan is broken, omit it for now
Also rewrote Makefile2 in a more compact way.
2014-01-10 13:26:42 +00:00
hallgren ed3d30e3d1 Check file datestamp before creating PGF file when compiling grammars
When running a command like

	gf -make L_1.gf ... L_n.gf

gf now avoids recreating the target PGF file if it already exists and is
up-to-date. 

gf still reads all required .gfo files, so significant additional speed
improvements are still possible. This could be done by reading .gfo files
more lazily...
2014-01-09 17:30:24 +00:00
hallgren 768d9de668 Check file datestamps before unioning PGF files (fix)
Also check that no additional output formats have been selected.
2014-01-09 15:14:35 +00:00
hallgren ed1e662dea Check file datestamps before unioning PGF files
When running a command like

 gf -make -name=T L_1.pgf ... L_n.pgf

gf now checks if T.pgf exists and is up-to-date before reading and computing
the union of the L_i.pgf files.

The name (T) of the target PGF file has to be given explicitly for this to work,
since otherwise the name is not known until the union has been computed.
If the functions for reading PGF files and computing the union were lazier,
this would not be necessary...
2014-01-09 14:18:21 +00:00
hallgren d8d78b6187 Release note updates and some other minor things 2013-12-18 16:22:59 +00:00
hallgren 52494afc50 PGFtoPython: add Python translations for nonExist, BIND and SOFT_BIND
Someone who is familiar with the Python translation should check this.
2013-12-18 15:31:39 +00:00
hallgren 4012fa15dd GF home page: use the SVG version of the GF logo in browsers that support it
It looks better on high resolution screens.
2013-12-18 12:01:15 +00:00
hallgren e9e919e6e3 src/server/gf-server.cabal: compile it as a common library + executables 2013-12-17 15:55:14 +00:00
hallgren 9c43c2bae3 examples/phrasebook/Makefile2: Finnish needs a larger stack 2013-12-17 15:13:44 +00:00
hallgren e0481e3b8a Add backward compatibility for reading old PGF files
Some backwards incompatible changes were made to the PGF file format after
the release of GF 3.5. This patch adds a module for reading PGF files in the
old format.

This means that old PGF files on the grammaticalframework.org server will
continue to work after we install the latest version of GF.
2013-12-17 13:27:37 +00:00
hallgren 2b16962835 GF.Compile.ReadFiles: reduced code duplication 2013-12-16 14:04:57 +00:00
hallgren 8455435b75 examples/phrasebook/Makefile2: remove duplicated rule 2013-12-16 13:41:29 +00:00