1
0
forked from GitHub/gf-core
Commit Graph

308 Commits

Author SHA1 Message Date
hallgren
40c431ec6c PGF Service: add an option to return syntax trees in JSON format
The parse/translate/c-parse/c-translate commands now recognize the option
jsontree=true to augment the returned JSON structure with a field called
"jsontree" next to the field "tree", or "jsontrees" next to "trees",
containing the the returned syntax tree in JSON format (the same format
returned by the abstrjson command, similar to the format returned in the
"brackets" field).
2015-03-24 17:37:43 +00:00
hallgren
374da1f0b0 PGF service: add commands c-abstrtree and c-parsetree
These are the C runtime variants of the absttree and parsetree commands. They
work in the same way, but support fewer rendering options.
2015-03-19 14:17:45 +00:00
hallgren
5cf0ab4110 PGFService: make the restriction on parallelism more effective
Restrict the entire request handler instead of just the call to the C parser.
2015-02-25 16:08:21 +00:00
hallgren
7631527395 PGFService.hs: text lexer: slightly smarter decapitalization of the first word of a sentence
Keep the first letter in upper case if it is followed by more upper case
letters.

(Also remove some left over debugging output.)
2015-02-24 15:04:06 +00:00
hallgren
b72b8dcef8 PGF Service: limit the number of parallel calls to the C run-time parse function to 4 by default
The limit can be changed with the -j flag
2015-02-20 12:29:44 +00:00
hallgren
2ce3e954fd Changes for compatibility with ghc-7.10-rc2
2 modules: Name clashes caused by Applicative-Monad change in Prelude
2 modules: Ambiguities caused by Foldable/Traversable in Prelude
2 modules: Backwards incompatible changes in time-1.5 for defaultTimeLocale
9 modules: {-# LANGUAGE FlexibleContexts #-} (because GHC checks inferred types
           now, in addition to explicitly given type signatures)

Also silenced warnings about tab characters in source files.
2015-02-16 15:05:06 +00:00
hallgren
ec8b4fec93 PGF2 & PGFService: enable callbacks for named entities and chunks in the c-parse & c-translate web requests 2015-01-21 10:41:12 +00:00
hallgren
7577de236f PGF2 & PGFService: work in progress on callbacks for nerc & chunks 2015-01-20 14:57:52 +00:00
hallgren
fe8daf7f94 gf.cabal, gf-server.cabal: add version bounds on network & httpd-shed
* httpd-shed-0.4 does not specify an upper bound on network, but it fails
  to build against network>=2.6. This is fixed in httpd-shed-0.4.0.2.
* With network-2.6, the Network.URI modules is moved to a separate package,
  so for the time being GF requires network>=2.3 && <2.6. This is compatible
  with the four most recent versions of the Haskell Platform.
2014-09-03 15:40:26 +00:00
hallgren
0e87a88f4b src/server: refactoring to isolate dependencies on the cgi/fastcgi packages
* Introducing the module CGI, re-exporting a subset of the cgi package. It
  might complete replace the cgi package in the future.
* Introducing the module CGIUtils, containing functions from FastCGIUtils that
  have nothing to do with fastcgi.

Some low level hackery with unsafePerformIO and global variables was left
in FastCGIUtils, but it is actually not used, neither for gf -server nor
exec/pgf-fcgi.hs.
2014-09-02 12:27:47 +00:00
hallgren
ea6aa6dc02 PGF service: expire PGFs from the cache when they have been unused for 24 hours
...to keep memory use down on the server.
2014-07-21 15:53:38 +00:00
hallgren
47e185679f src/server/gf-server.cabal: update dependencies 2014-07-15 12:00:51 +00:00
john.j.camilleri
d59749c914 PGF Web Service: include entire completion in full mode
When using full=yes in the web service 'complete' command,
you now get an additional field 'seq' with the longest possible completion.
So, given:

  lin
    f1 = ss "the" ;
    f2 = ss ("the red house" | "the real deal") ;

and trying to complete on input "th", you get:

    [
        {
            "from": "TestCnc",
            "brackets": {
                "cat": "_",
                "fid": 0,
                "index": 0,
                "fun": "_",
                "children": []
            },
            "text": "th",
            "completions": [
                {
                    "token": "the",
                    "funs": [
                        {
                            "fun": "f1",
                            "hyps": [],
                            "cat": "C",
                            "seq": "the"
                        },
                        {
                            "fun": "f2",
                            "hyps": [],
                            "cat": "C",
                            "seq": "the red house"
                        },
                        {
                            "fun": "f2",
                            "hyps": [],
                            "cat": "C",
                            "seq": "the real deal"
                        }
                    ]
                }
            ]
        }
    ]
2014-07-15 09:33:22 +00:00
john
747287055f PGF web service: Return additional completion info with 'full' flag
BETA! The 'complete' command now has a new flag 'full' which when set
returns additional info about completions.

Without 'full' flag (default):

[
    {
        "from": "PhrasebookEng",
        "brackets": {
            "cat": "_",
            "fid": 0,
            "index": 0,
            "fun": "_",
            "children": [
                {
                    "token": "the"
                }
            ]
        },
        "text": "su",
        "completions": [
            "supermarket",
            "suspect"
        ]
    }
]

With full=true or full=yes:

[
    {
        "from": "PhrasebookEng",
        "brackets": {
            "cat": "_",
            "fid": 0,
            "index": 0,
            "fun": "_",
            "children": [
                {
                    "token": "the"
                }
            ]
        },
        "text": "su",
        "completions": [
            {
                "token": "supermarket",
                "funs": [
                    {
                        "fid": 421,
                        "fun": "Supermarket",
                        "hyps": [],
                        "cat": "PlaceKind"
                    }
                ]
            },
            {
                "token": "suspect",
                "funs": [
                    {
                        "fid": 445,
                        "fun": "Suspect",
                        "hyps": [],
                        "cat": "Property"
                    }
                ]
            }
        ]
    }
]
2014-07-11 09:25:26 +00:00
hallgren
78ccefc4f4 PGF Service: return error code 404 (not found) instead of 500 (internal server error) for missing grammars 2014-06-20 17:28:02 +00:00
hallgren
fae535b4e8 PGF Service, command=grammar: include the last modified time of the grammar in the output 2014-06-20 10:34:51 +00:00
hallgren
3c110cbe8d gf -server: include list of loaded PGFs in version info 2014-05-12 13:45:36 +00:00
hallgren
2a774e946b PGF web api, command c-wordforword: re-enable parsing of individual words
The space leak in the Haskell binding to the C parser has been fixed.
2014-05-08 11:38:41 +00:00
hallgren
d37c25990e PGF web api, command c-wordforword: don't repeat parsing/morpho analysis for each target language
Also omit attemts to parse individual words for now, to avoid space leaks in
the Haskell binding to the C parser.
2014-04-29 15:51:46 +00:00
hallgren
05c70fdc41 Spring cleaning
Nothing major...
2014-04-28 13:56:20 +00:00
hallgren
5c79b1d8bd PGF web API: adding the command c-wordforword
It has the same parameters and result format as c-translate, but it does
the translation word for word. (To be used as a last resort).
2014-04-16 15:24:23 +00:00
hallgren
210f28ca4b PGF web API: add commands lookupmorpho and c-lookupmorpho 2014-04-15 14:50:38 +00:00
hallgren
923107506f PGF web service: disable caching of parse results
Caching parse results uses a lot of memory, even if they expire after
2 minutes, so it won't scale up to many simultaneous users.

But some excessive memory use seems to be caused by space leaks in
(the Haskell binding to) the C run-time system, and these should be fixed.
For example, flushing the PGF cache does not release the memory allocated
by the C run-time system when loading a PGF file.
2014-04-10 15:55:33 +00:00
hallgren
43e61a1e3b PGF web service: add unlexers and enable client side caching
Most PGF web API commands that produce linearizations now accept an
unlexer parameter. Possible values are "text", "code" and "mixed".

The web service now include Date and Last-Modified headers in the HTTP,
responses. This means that browsers can treat responses as static content and
cache them, so it becomes less critical to cache parse results in the server.

Also did some cleanup in PGFService.hs, e.g. removed a couple of functions
that can now be imported from PGF.Lexing instead.
2014-04-09 17:51:25 +00:00
hallgren
1602eab5b6 PGF Service: a bit more clever lexer=text
Only change the first word to lowercase if the original input is not found in
the grammar's morphology. This allows parsing of sentenses starting with "I" in 
English, nouns in German and proper names in other languages, but it can make
the wrong choice for multi-words.
2014-04-09 14:13:18 +00:00
hallgren
5d4b849b14 Fix build problem in PGFService.hs when configured without c-runtime 2014-04-08 20:45:54 +00:00
hallgren
0341fd035f PGF web service: add lexer support
The PGF web API commands parse, translate, translategroup, c-parse and
c-translate now support a lexer parameter. Possible values are "text",
"code" and "mixed".

This is used in the Wide Coverage Demo App to save one server roundtrip.
2014-04-08 16:10:33 +00:00
hallgren
57ebfa4416 PGFService.hs: adapt to changes in the Haskell binding to the C run-time system
Parse errors used to cause crashes, but they are now handled and reported.
2014-04-04 12:28:00 +00:00
hallgren
6cc8557e6b gf -server: add a command to manually flush the PGF cache
This can be used if the cloud service seems slow, but it would probably
be better to automatically expire unused PGFs from the cache after some time.
2014-03-19 16:15:05 +00:00
john.j.camilleri
466ef94d59 Fix typo in PGFService: "incomlete" 2014-03-12 12:45:32 +00:00
hallgren
15bf881e45 Work on web api & apps based on the C run-time system
+ PGFService.hs: add command c-grammar, include probability in parse results
+ js/gftranslation.js: add start position and limit parameters, return more
  info to applications
+ Simple Translator: show two wide coverage translations
+ Wide coverage demo: show parse tree and probability (intended as grammar
  debugging aids)
2014-02-11 16:43:29 +00:00
hallgren
ac3fae334e Fix broken C runtime support in gf.cabal and PGFService.hs
Also add PGF service command c-flush to explicitly flush cached parse results
from memory.
2014-02-10 16:04:40 +00:00
hallgren
82fd8c9f94 gf-server.cabal: flag to include C run-time support in the FastCGI PGF service 2014-01-31 16:11:41 +00:00
hallgren
4e6db5d621 PGFService.hs: fix a type ambiguity when compiling without C run-time support 2014-01-27 16:46:48 +00:00
hallgren
ca72fa94eb PGFService.hs: add a cache for C run-time parse result and a start offset parameter
Cached parse results are discarded when they have been unused for 10 minutes.
2014-01-27 16:32:09 +00:00
hallgren
b8f3914209 Optionally include C run-time support
If the C run-time library is compiled and installed on your system, you can now
do 'cabal configure -fc-runtime' to get the following extras:
  + The haskell binding to the C run-time library will be included in the
    PGF library (so you can import it in Haskell applications).
    Documentation on the new modules will be included when you run
    'cabal haddock'.
  + The new command 'pgf-shell', implemented on top of haskell binding to
    the C run-time system.
  + Three new commands in the web API: c-parse, c-linearize and
    c-translate. Their interfaces are similar to the corresponding commands
    without the "c-" prefix, but they should be considered preliminary.
2014-01-20 17:06:11 +00:00
hallgren
c6d2b30c6b src/server/gf-server.cabal: compile it as a common library + executables 2013-12-17 15:55:14 +00:00
kr.angelov
620e880155 linref is now used by the linearizer. The visible change is that the 'l' command in the shell now can linearize discontinuous phrases 2013-10-30 14:42:29 +00:00
hallgren
ad0e67530b Functions merge trees into tries in the GF Shell and the PGF web service
* In the shell, the new command tt (to_trie) merges a list of trees into a
  trie and prints it in a readable way, where unique subtrees are marked with
  a "*" and alternative subtrees are marked with numbers.
* In the PGF web service, adding the parameter trie=yes to the parse and
  translate commands augments the JSON output with a trie.

Example to try in the shell:

	Phrasebook> p -lang=Eng "your son waits for you" | tt
2013-10-24 17:29:02 +00:00
hallgren
fb61fc4562 Cloud & PGF service: use Content-Type application/json for JSON output
This is in accordance with RFC 4627. http://tools.ietf.org/html/rfc4627
Use application/javascript for JSONP output.
2013-07-29 16:05:54 +00:00
hallgren
eb2b5a65d6 Allow cross origin requests to GF cloud & PGF services
By adding a header

	Access-Control-Allow-Origin: *

to the HTTP responses, web browsers are informed that it is OK to call the
services from web pages hosted on other sites.

This is apparently supported in most modern browsers, so it should no longer
be necessary to resort to JSONP.

See https://developer.mozilla.org/en-US/docs/HTTP/Access_control_CORS
2013-07-25 16:04:43 +00:00
hallgren
656f934945 gf-server.cabal: same changes as in gf.cabal for ghc-7.6 compatibility 2013-07-15 17:07:51 +00:00
hallgren
550db69caa Fix UTF-8 decoding problem in gf -server
The package network-2.4.1.1 thoughlessly introduced a backward incompatible
change to the function Network.URI.unEscapeString, see

    f2168b1f89

This also affects the function Network.Shed.Httpd.queryToArguments, which is
used in GFServer.hs.

To remain compatible with older and newer versions of the network package,
we need to stay clear of queryToArguments and unEscapeString. A replacement
function has been added to server/URLEncoding.hs.
2013-06-13 08:23:48 +00:00
hallgren
74ea0c9cf4 PGF service: apply the token binding operator &+ to all linearizations
The &+ operator is now consistently eliminated from the output of commmands
that produce linearizations. Before, only the commands translate and
translategroup did this.
2013-04-24 18:32:53 +00:00
hallgren
525b983b4c PGF web API: command=abstrtree not supports the nocat and nofun options 2013-03-28 15:57:17 +00:00
hallgren
ca09371166 PGF web API: add comand linearizeTable
The new command linearizeTable outputs linearizations as tables containing all
variants and forms, labelled by parameters
2013-03-27 16:31:52 +00:00
hallgren
45b632d530 PGF service: support all parse tree visualziation options in the web API
The new options are

  * noleaves, nofun, nocat (booleans, false by default)
  * nodefont, leaffont, nodecolor, leafcolor, nodeedgestyle, leafedgestyle
    (strings, empty by default)
2013-03-26 14:17:24 +00:00
hallgren
6045ddae19 PGF web API: add 'to' parameter to the 'alignment' command
Word alignment diagrams can now be restricted to a subset of the languages
supported by the grammar.
2013-03-13 15:56:03 +00:00
hallgren
e2bb437943 PGF web API: generalize the 'to' parameter to accept a list of languages
Some commands (linearize, linearizeAll, random, generate, translate and
translategroup) by default produce output in all languages supported by the
grammar and the 'to' parameter could be used to restrict output to a single
language. Now you can restrict the output to a list of languages. Languages
should be separated by spaces.

Also removed an unnecessary LANGUAGE pragma and reduced code verbosity.
2013-03-13 15:25:28 +00:00
Sergei Trofimovich
05e5895134 ghc-7.6: allow directory-1.2
Get rid of old-time depend (and ClockTime in favour of UTCTime).
time-compat helps to retain backward compatibility with directory-1.1
and lower.
2013-03-09 21:38:43 +00:00