This prevents HUGE space leak and makes compiling a PGF a LOT faster
For example, an application grammar moved from taking over 50GB
of ram and taking 5 minutes (most of which is spent on garbage colelction)
to taking 1.2 seconds and using 42mb of memory
The price we pay is that the "variable #n is out of scope" error is now
lazy and will happen when we try to evaluate the term instead of
happening when the function returns and allowing the caller to chose how
to handle the error.
I don't think this should matter in practice, since it's very rare;
at least Inari has never encountered it.
Bul ✓
Cat ✗
Chi ✓
Dan ✓
Dut ✓
Eng ✓
Est ✓
Fin ✓
Fre ✗
Ger ✓
Hin ✓
Ita ✗
Jpn ✓
Lav ✓
Nor ✓
Pol ✓
Ron ✓
Snd ✗
Spa ✓
Swe ✓
Tha ✓
Urd ✓
Passed 18 | Failed 4 | Total 22
Bul ✗
Cat ✗
Chi ✓
Dan ✓
Dut ✓
Eng ✓
Est ✓
Fin ✗
Fre ✗
Ger ✓
Hin ✓
Ita ✗
Jpn ✓
Lav ✓
Nor ✓
Pol ✓
Ron ✓
Snd ✗
Spa ✗
Swe ✓
Tha ✓
Urd ✓
Passed 15 | Failed 7 | Total 22
Still contains some hardcoded values, missing cases.
I notice now that LPGF and Canonical GF are almost identical, so maybe we don't need a new LPGF format,
just a linearization-only runtime which works on canonical grammars.
The argument for keeping LGPF is that it would be optimized for size and speed.
The implementation was meant to lift from SIO to IO,
but instead it was just the identity function,
which means that `fail = id . fail` and we have an infinite loop.
The GF shell no longer has `put_tree -typecheck` option, and typechecking is done automatically when parsing.
The metavariable thing is a bit unclear: you don't get it when parsing "dim the light", or "switch on the fan, but you do get it when you `gt` after adding `switchOn` and `switchOff`.
```
> p "switch on the fan"
CAction fan (switchOff fan) (DKindOne fan)
> gt
CAction light dim (DKindOne light)
CAction ?3 (switchOff ?3) (DKindOne ?3)
CAction ?3 (switchOn ?3) (DKindOne ?3)
```
My hypothesis is that you don't get metavariable when parsing e.g. "dim the light", because even though `light` is suppressed in `CAction`, it still appears in `DKindOne`, so it gets to contribute to the whole tree with its string.
Update pgf2 metadata for uploading to Hackage
Even though this isn't able to package the C runtime itself into the Haskell package, I think these changes are still worth merging into master.
GF allows more characters in its types, as long as they are inside
single quotes. E.g. 'VP/Object' is a valid name for a GF category,
but not for a Haskell data type.
The userLangauge is the name of the concrete syntax that has a languageCode
that matches the user's preferred language, as reported by the web browser.
If no matching language code is found, the PGF service now sets userLanguage
to the concrete syntax for English (e.g. FoodsEng) if present, and defaults
to the first concrete syntax (e.g. FoodsAfr) only if English is not present
in the grammar.
A preferred grammar is selected when a user visits the Minibar for the
first time. (Like before, Minibar remembers the selected grammar for future
visits.)
A preferred list of grammars can be specified in config.js in the .../minibar
directory on the server, e.g. like this:
preferred_grammars=["/grammars/Foods.pgf","/grammars/ResourceDemo.pgf"]
The first available grammar from the list is used.
When you press the "i" or "More info" button for a grammar, the info now
includes a link that can be used by anyone to open this specific grammar in
the minibar.
The documentation is taken from a file called Grammar.pgf_info, located
next to the Grammar.pgf file on the server.
The first line of the documentation is displayed below the menu bar in
the minibar. The rest of the documentation is displayed when you press
the "More info" button (or the "i" button).
The documentation can contain HTML markup. Blank lines are treated as
paragraph breaks.
It was only needed for compatibility with directory<1.2, but
directory>=1.2 has been shipped with ghc since ghc-7.6.
Note: time-compat-1.9.* (the current version) is a completely different
package, that does not provide the needed function toUTCTime, which
was provided in time-compat-0.1.*.
grammar.js contained only one langauge: FoodsEng. Now it contains
FoodsEng and FoodsIta, similar to what's shown in the gf-tutorial.
The grammar comes from gf-contrib/tutorial/foods.
After fixing trivial problem in translator.html, the demo works, but
the example grammar contains only one langauge, so it is not so
interesting. translator.html is located in src/runtime/javascript.
The editor.html in the same location also works.
e.g. if C is a fun and not a cat in the abstract syntax.
Discarding bad lincats prevents GF from generating malformed PGFs that
are rejected by the C run-time system.
I also added code to reject bad lincats with an error, but I left it
commented out since it seems a bit pedantic compared to GF's otherwise
rather sloppy grammar checking.
some object labels must be preceded by ".", to not be in conflict with GF records (which are stored as json objects)
plus some minor bugfixes and cleaning
The problem was that the RGL was both built and copied to the destdir
during the build step, which caused it to be deleted before the install
step. It is now copied to destdir during the install step.
+ Abstract syntax now is converted directly from the Grammar and not via PGF,
so you can use `gf -batch -no-pmcfg -f canonical_gf ...`, to export to
canonical_gf while skipping PMCFG and PGF file generation completely.
+ Flags that are normally copied to PGF files are now included in the
caninical_gf output as well (in particular the startcat flag).
This output format converts a GF grammar to a "canonical" GF grammar. A
canonical GF grammar consists of
- one self-contained module for the abstract syntax
- one self-contained module per concrete syntax
The concrete syntax modules contain param, lincat and lin definitions,
everything else has been eliminated by the partial evaluator, including
references to resource library modules and functors. Record types
and tables are retained.
The -output-format canonical_gf option writes canonical GF grammars to a
subdirectory "canonical/". The canonical GF grammars are written as
normal GF ".gf" source files, which can be compiled with GF in the normal way.
The translation to canonical form goes via an AST for canonical GF grammars,
defined in GF.Grammar.Canonical. This is a simple, self-contained format that
doesn't cover everyting in GF (e.g. omitting dependent types and HOAS), but it
is complete enough to translate the Foods and Phrasebook grammars found in
gf-contrib. The AST is based on the GF grammar "GFCanonical" presented here:
https://github.com/GrammaticalFramework/gf-core/issues/30#issuecomment-453556553
The translation of concrete syntax to canonical form is based on the
previously existing translation of concrete syntax to Haskell, implemented
in module GF.Compile.ConcreteToHaskell. This module could now be reimplemented
and simplified significantly by going via the canonical format. Perhaps exports
to other output formats could benefit by going via the canonical format too.
There is also the possibility of completing the GFCanonical grammar
mentioned above and using GF itself to convert canonical GF grammars to
other formats...
when debbuging labels, I find it useful to have comments saying what's
the original sentence (lazy, I know) and the original tree (depending
on the treebank, the trees can be similar).
I know this is not the goal exactly, but UDv2 treebanks
(http://universaldependencies.org/format.html) should always have a
'text =' comment, and a 'sent_id =' comment (which would be easy to
implement too, but not that useful).
The raw HTML was invalid, and this way we use the common website template
for a uniform look without any duplication.
It seems gf-refman.html was once generated from txt2tags, although I have
been unable to find this original .t2t file.
I also tried to re-generate txt2tags from HTML but was not able to.
However I was able to convert HTML to Markdown using Pandoc and I think
the result is pretty good, so I think we should use this.
The original gf-refman.html can be obtained from git history, e.g.:
a7e43d872f/doc/gf-refman.html
Traditionally, GF_LIB_PATH points to something like
`.../share/ghc-8.0.2-x86_64/gf-3.9/lib`
and if you want prelude and alltenses and present, you add a
`--# -path=.:present`
compiler pragma to the top of your .gf file
But if you are developing some kind of application grammar
library or contrib of your own, you might find yourself
repeating your library path at the top of all your .gf files.
After painstakingly maintaining the same library path at the
top of all your .gf files, you might say, let's factor this
out into GF_LIB_PATH.
Then you might then find to your surprise that GF_LIB_PATH
doesn't accept the usual colon:separated:path notation
familiar from, say, unix PATH and MANPATH.
This patch allows you to define
`GF_LIB_PATH=gf-3.9.lib:$HOME/gf-contrib/whatever/lib`
in a more natural way.
If you are an RGL hacker and have your own version of the
RGL tree sitting somewhere, you should be able to have both
paths in the GF_LIB_PATH, for added convenience. This minor
convenience will probably lead to obscure bugs and great
frustration when you find that your changes are mysteriously
not being picked up by GF; so keep this in mind and use it
cautiously.
This caution should probably sit in the documentation
somewhere. A subsequent commit will do that.
If you use zsh, you can do this to quickly build up a big
GF_LIB_PATH:
% gf_lib_path=( $HOME/src/GF/lib/src/{api,abstract,common,english,api/libraryBrowser,prelude,..} )
% typeset -xT GF_LIB_PATH gf_lib_path
All changes have been reflected in the gf-contrib repository:
https://github.com/GrammaticalFramework/gf-contrib
Now, for WebSetup to build the example grammars, one must have gf-contrib
cloned in the same top-level directory as GF. When this isn't the case,
WebSetup displays a notice without failing.
Added a few new features:
-o OldGrammar.pgf: if combined with -f , -c or -b , only test the specified trees+functions.
--show-context FId: shows contexts for the particular concrete category
--show-cats interacts with --debug by showing also concrete categories
* In GHC 8.4.1, the operator <> has become a method of the Semigroup class
and is exported from the Prelude. This is unfortunate, since <> is also
exported from the standard library module Text.PrettyPrint, so in any
module that defines a pretty printer, there is likely to be an ambiguity.
This affects ~18 modules in GF. Solution:
import Prelude hiding (<>)
This works also in older versions of GHC, since GHC does't complain if
you hide something that doesn't exists.
* In GHC 8.4.1, Semigroup has become a superclass of Monoid. This means
that anywhere you define an instance of the Monoid class you also have to
define an instance in the Semigroup class.
This affects Data.Binary.Builder in GF. Solution: conditionally define
a Semigroup instance if compiling with base>=4.11 (ghc>=8.4.1)
The editor doesn't show delete buttons on grammars published by other
users, but it was too picky when deciding which grammars you own. Now
it should be possible to delete grammars from the device/browser
you published it from, even if you don't have a private copy of it any more.
On a related note, there seems to be problem with the way unique grammars
names are created and maintained, causing published grammars to be duplicated
in some cases. This needs to be overhauled.
This was a problem in Safari (an other similar browsers I presume), but
not in Firefox: hovering over the grammar comment (shown below the grammar
name when you edit a grammar) didn't reveal the button to edit it, thus
preventing you from adding a comment. It was till possible by selecting the
"Enable editing on touch devices." at the bottom of the screen, but most
people probably didn't notice that it is possible to add a comment.
The configuration uses Nix to install dependencies. I first tried using
just Cabal, but compiling all the dependencies takes too long for the
free tier of Travis. Nix doesn't compile Hackage dependencies because
it has a "binary cache" with compiled versions of almost all
Hackage packages.
I do not run the GF test suite, because it seems to be in a
failing state. This can be enabled later.
All of this will only be enabled once the Travis service is set up by
someone in the GF team with the permissions to do so.
Build results will then be automatically visible on all new commits and
pull requests.
Cabal>=1.20 allows control over parallelism when compiling grammars from
Setup.hs and WebSetup.hs.
base>=4.6 allows conditional compilation with CPP to be eliminated from
a few modules.
base-4.6 corresponds to GHC 7.6.3, which is what you get in
Debian 8 (aka jessie, aka oldstable) from 2015.
CncLabels are read from a file in the same directory as the PGF file and
with the same name as the concrete syntax + extension .labels,
e.g. ResourceDemo.pgf would use labels from
ResouceDemo.labels (abslabels)
ResouceDemoEng.labels (clclabels)
* Just skip the Java binding if there were errors compiling it.
* Install the Python binding in the expected place on Mac OS X when creating
an OS X installer package. (i.e. under /Library/Python/2.7/site-packages)
Also updated the 3.8 download page and release notes to reflect the above.
The function listCaches returns an empty list instead of the contents
of the cached C run-time system PGFs when compiled without C-run-time support
This gave rise to an ambiguity in GF.Server.
Remove ghc-7.4 from the list of tested/recommended versions of GHC
(because of Hackage dependency problems).
Recommend version 7.10.3 of the Haskell Platform. (GHC 8.0.1 works too,
but I haven't tested the corresponding Haskell Platform packages.)
+ The PGF service now reads and caches dependency label configuration files.
+ The grammar info returned by command=grammar has a new boolean field
'hasDependencyLabels' to indicate if dependency labels were found for
the grammar. Also, command=deptree will now fail if no labels are present.
+ The minibar only shows word dependency trees if labels are present.
+ Also changed the type of getDepLabels from [String] -> Labels to
String -> Labels, since all uses were in the form "getDepLabels . lines".
This was done by introducing an intermediate representation for the LaTeX
pictures produced by the LaTeX renderer and providing a new backend that
outputs SVG instead of LaTeX.
This was done to obtain speech synthesis for more languages in the Numerals
grammar, which uses neither language flags nor the standard naming for
concrete syntax.
This is the web interface to PGF.graphvizDependencyTree.
Accepted parameters: format=<output_format>, to=<lang>, tree=<tree>.
Accepted output formats: latex, conll, malt_tab, malt_input, png, gif, svg, gv
Also, label information is taken from <path>.labels if present, where
<path>.pgf is the path to the PGF file.
They seemed very difficult to use without the documentation, since their
types are not particularly self documenting.
Also documented what the (Bool,Bool) argument to PGF.graphvizAbstractTree
is for.
* Automatically update the translations when the set of selected
application grammars is changed.
* Skip application grammars that do not support the currently selected
source & target languages.
Add missing initalization after a new application grammar has been selected.
Add an "X" close button in the upper left corner of the grammar selection popup.
When selecting which application grammars to use for translation, after
pressing the "Grammars..." button, it is now possible to change the order
of the selected grammars by dragging them up and down in the list.
gfse/editor.js had its own list with 34 languages, while js/langcode.js
only had 30 languages. The missing languages have been added to
js/langcode.js and all apps now use that list.
There is now a new button "Grammars..." which show a list where users can
select which application grammars to use for translation, in addition to
the wide coverage grammar. Application grammars can give higher quality
translations in the domain they cover.
TODO: make it possible control the order of the selected application grammars.
This was fixed by adding "&nodep=true" in the minibar when it requests a
parse tree from the server. The reason dependency information got included
in the first place was that all rendering options are on by default in
command=parsetree requests in PGF service API.
This means that you now need to have a Java SDK installed to build
a Debian package.
The bin/build-binary-dist.sh (which builds bintar packages and OS X installer
packages) now skips the Python binding if python is not installed. Likewise
it skips the Java binding if a Java SDK is not installed.
dh_shlibdeps has a problem finding which package some of the Haskell
libraries come from, even though dpkg-query -S has no problem finding them.
But the gf executable is statically linked against the Haskell libraries, so
it will work even if these shared libraries aren't installed. But there will
be a problem if other shared libraries are missing (.e.g. libtinfo, libgmp),
so we need a better solution than simply ignoring all dependency problems...
See also http://stackoverflow.com/questions/11238134/dpkg-shlibdeps-error-no-dependency-information-found-for
This patch fixes a problem introduced last year when the GF shell was
refactored to allow more commands to be treated uniformly and be part
of pipes. The cc command was one of those commands, but unfortunately this
introduced a parsing problem, e.g.
> cc "last"
constant not found: last
> cc "last"++"year"
command not parsed: cc "last"++"year"
This happened because the generic command line parser in
GF.Command.{Abstract,Parse} assumes that all commands have an argument of
type PGF.Expr. Commands that expect other types of arguments have to
use PGF.showExpr combined with other conversion to the argument type they
expect. The cc command excpets a GF.Grammar.Term, and unfortunately not
all terms survice the roundtrip through PGF.Expr, in part because of
an additional hack to allow strings to be roundtripped through PGF.Expr
without adding superfluous double quotes.
To solve the problem, this patch
+ makes room for arguments of type Term in the Argument type in
GF.Command.Abstract.
+ makes a special case for the cc command in GF.Command.Parse, by
calling the partial parser 'runPartial pTerm' recently added in
GF.Grammar.Lexer and GF.Grammar.Parser. Care was taken so that
that "|" and ";" can be used both inside terms and as separators between
commands in the shell, e.g. things like the following now work:
> cc ("a"|"b") | ps -lexcode
variants { "a" ; "b" }
+ introduces a type CommandArgument that replaces [Expr] as the
type of values passed between commands in pipes. It has room for
values of type [Expr], [String] and Term, thus eliminating the need
to roundtrip through the Expr type all the time.
The hack to avoid adding superfluous quotes when strings are
roundtripped through Expr has been left in place for now,
but can probably be removed.
Lexer.x: Change the parser monad type P to allow the remaining input to
be returned after a partial parse. Add function
runPartial :: P t -> String -> Either (Posn, String) (String, t)
Parser.y: Add a partial parser pTerm for nonterminal Exp1.
Re-export runPartial.
The automake package contains aclocal, which is needed when running
autoreconf -i.
I never needed to install automake explicitly on Linux, presumably because it
got installed anyway because of some other dependency, but apparently you
can't take this for granted.
Because of the "heurisitc factor", the results returned by the parser might
not be in strict probability order. To compensate, a sorting pass has been
added to find and show the translation with the highest probability among the
10 first translations.
However, this means that the translation demo now immediately has to ask for
10 translations of every segment. Before, it initially asked for only one
translation of every segment, and then 9 more translations for a segment
when/if the user clicked on it. This change can slow down translation
noticeably (e.g. from 15s 30s to load an example with fiction in English).
"File/Open In Wide Coverage Translation Demo" stopped working when support
for using different grammars in the Wide Coverage Translation Demo was added
On September 2.
This is implemented as a simple post-processing step after partial evaluation
to try compute pre{...} tokens in token sequences. Nothing is done to deal
with intervening free variants.
This was done in response to a query from René T on the gf-dev mailing list.
The matchPrefix function is used in str2strings and the partial evaluator to
compute pre{...} tokens. But unlike the description in the GF book and
the implementation in the run-time system, matchPrefix looked at
the concatenation(!) of all following tokens and not just the next token
when deciding how to compute a pre{...} token.
This is a backwards incompatible change, but it is subtle and probably won't
cause any problems. In particular, the example grammars are unaffected.
Updated version number in gf.cabal and update debian/changelog
News item on home page (index.html)
Some last minute edits on the download page
Move the new release notes and download page in place.
This makes the output from PGF.showExpr (and other Haskell code that uses
the Prelude.show function to show strings) parsable as GF source code in
more cases.
This is a workaround for the problem that GHC's implementation of the show
function uses numeric escapes for printable non-ASCII characters, e.g.
show "dålig" = "d\229lig"...
This should work nicely now that GF itself can detect the number of processors
in the system.
Previously there was some code in WebSetup.hs that relied on Cabal to do this,
but it was commented out because it requires Cabal>=1.20, which is fairly new
and it would complicate the installation instructions if it was required to
compile GF...
* "gf -make -j=n" uses n parallel threads.
* "gf -make -j" adapts to the number of processors in the system.
This mimics how "cabal build -j" and "ghc --make -j" works.
Support for this is implemented in the new module GF.System.Concurrency and
it depends on the function Control.Concurrent.setNumCapabilities, which is
only available in GHC>=7.6 (base>=4.6). GF can still be compiled with
GHC<7.6, but then you have to use +RTS -N -RTS to take advantage of
multicore processors.
To detect the number of processors in the system, the code depends on a
foreign import of a C function in the GHC run-time system.
Apparently Foreign.unsafePerformIO is not just a re-export of
System.IO.Unsafe.unsafePerformIO (or vise versa), it is a different function,
so you get an ambiguity if you import both.
This reduces the amount of duplicated code from 2400 to 490. No code from
data-binary-ieee754 is duplicated.
The module is called PGF.Data.Binary instead of Data.Binary. It is not
in use yet.
These changes are inspired by the gf -cshell implementation of these commands.
The output of the linearize command has been changed to remove superfluous
blank lines and commas, and deliver the result as a list of strings instead of
a single multi-line string. This makes it possible to use -all and pipe the
results to the parse command. This also means that with -treebank -all,
the language tag will be repeated for each result from the same language.
The parse command, when trying to parse with more than one language, would
"forget" other results after a failed parse, and thus not send all
successful parses through the pipe. For example, if English is not the first
language in the grammar,
p "hello" | l
would output nothing, instead of translations of "hello" to all languages,
forcing the user to write
p -lang=Eng "hello" | l
instead, to get the expected result. The cause of this behaviour was in the
function fromParse, which was rather messy, so I assume it is not intentional,
but the result of a programming mistake at some point.
The fromParse function has now been refactored from a big recursive function
into
fromParse opts = foldr (joinPiped . fromParse1 opts) void
where the helper functions fromParse1 deals with a single parse result and
joinPiped combines multiple parse results.
Even though the -lang flag was handled in the implementation, it was not
documented, and GF.Command.Interpreter rejects undocumented flags:
option not interpreted: lang
This must be a fairly old bug, so it suggests that the vp command isn't used
much...
pg supports only the -funs, -cats and -langs output modes.
ai IDENTIFIER shows info about a category or a function. ai can not type check
and refine metavariables in expressions.
showType :: Type -> String
categories :: PGF -> [Cat]
But both are implemented as quick hacks: categories is implemented by listing
all functions and taking the target categories from their types. showType uses
ppType copied & modified from PGF.Type, and needs a ppExpr, which is currently
implemented by wrapping showExpr...
TODO: need something correpsonding to PGF.categoryContext.
Options -all and -list use PGF2.linearizeAll, which lists all variants, but
not all forms...
Also, there is no attempt to be compatible with the output from the Haskell
run-rime shell, which produces superfluous blank lines (-all) or
commas (-list), and mixes tagged and untagged lines (-treebank -all).
The print_history command was among the commands implemented in an ad-hoc
way instead of being handled by the command line interpreter, which means
it could not be used in a pipe, as in the example in the help info.
The refactoring in the previous patch made this old bug easy to fix.
Also fixed a bug in the "empty" command, introduced when moving the PGF from
CommandEnv to GFEnv.
TODO: fix the undocumented eh command. A comment in the help info for
print_history, and some commented out old code, suggest that eh means
"execute_history", but at present it does nothing...
+ Generalize the CommandInfo type by parameterizing it on the monad
instead of just the environment.
+ Generalize the commands defined in
GF.Command.{Commands,Commands2,CommonCommands,SourceCommands,HelpCommand}
to work in any monad that supports the needed operations.
+ Liberate GF.Command.Interpreter from the IO monad.
Also, move the current PGF from CommandEnv to GFEnv in
GF.Interactive, making the command interpreter even more generic.
+ Use a state monad to maintain the state of the interpreter in
GF.{Interactive,Interactive2}.
These commands are now implemented as regular commands (i.e. using the
CommandInfo data type) in the new module GF.Command.SourceCommands.
The list of commands exported from GF.Command.Commmands now called pgfCommands
instead of allCommands.
The list allCommands of all commands is now assembled
from sourceCommands, pgfCommands, commonCommands and helpCommand in
GF.Interactive.
Created module GF.Command.CommonCommands with ~250 lines of code for commands
that do not depend on the type of PGF in the environemnt, either because they
don't use the PGF or because they are just documented here and implemented
elsewhere.
TODO: further refactoring so that documentation and implementation of
*all* commands can be kept together.
Some C run-time functionality is now available in the GF shell, by starting
GF with 'gf -cshell' or 'gf -crun'. Only limited functionality is available
when running the shell in these modes:
- You can only import .pgf files, not source files.
- The -retain flag can not be used and the commands that require it to work
are not available.
- Only 18 of the 40 commands available in the usual shell have been
implemented. The 'linearize' and 'parse' commands are the only ones
that call the C run-time system, and they support only a limited set of
options and flags. Use the 'help' commmands for details.
- A new command 'generate_all', that calls PGF2.generateAll, has been added.
Unfortuntaly, using it causes 'segmentation fault'.
This is implemented by adding two new modules: GF.Command.Commands2 and
GF.Interactive2. They are copied and modified versions of GF.Command.Commands
and GF.Interactive, respectively. Code for unimplemented commands and other
code that has not been adapted to the C run-time system has been left in
place, but commented out, pending further work.
+ Move type CommandInfo from GF.Command.Commands to a new module
GF.Commands.CommandInfo and make it independent of the PGF type.
+ Make the module GF.Command.Interpreter independent of the PGF type and
eliminate the import of GF.Command.Commands.
+ Move the implementation of the "help" command to its own module
GF.Command.Help
By adding unlexer=none (or unlexer=id) in requests that output linearizations
(e.g. command=linearize, command=translate), you can leave &+ uninterpreted
instead of gluing the adjacent tokens. This means that the output is left in
a format that can be parsed in a subsequent request.
To implement this consistently, the function linearizeAndBind was replaced
with the function linearizedAndUnlex (but there are a couple of requests
that do not call this function...)
Note that this applies to the Haskell run-time requests only. The C run-time
request (c-linearize, c-translate) always applies the &+ token and the
c-parse request can parse input containing glued tokens.
Becacuse of the new special tokens added to the Symbol type, .gfo and .pgf
files produced with the current version of GF can not always be used with
older versions of GF and the PGF run-time system.
The PGF version number was increased from (2,0) to (2,1). GF can still
read version (2,0) and (1,0), so old PGF files continue to work.
The GFO version was increased from "GF03" to "GF04".
The minibar lets the user choose any category as the start category, but this
can lead to run-time errors for grammars compiled with -optimize-pgf.
The problem can also be observed in the GF shell. For example,
"linearize Fish" (which means that the start category is Kind) fails for
some languages when Foods.pgf is compiled with -optimize-pgf.
When using make -j to compile examples/app or examples/phrasebook, since
the dependencies are not completely specified in the Makefiles, it can
happen that the same file is compiled at the same by more than one process,
resulting in an error when they try to write the same .gfo.tmp file. Adding a
random number to the temporary file name avoids this problem.
linearizeAll function will take an expression and an optional
n keyword argument and return an iterator to the n-variant
linearizations of the expression.
The examples directory contains two modules for using Python bindings in GF.
The gf_utils module provides various Python functions to perform parsing
and linearization using the C runtime. Additionally, the translation_pipeline
module is an replica of the current pipeline used in the Wide-coverage
Web translation pipeline.
The length limit test previously used the URL-encoded UTF-8 representation
of the source sentense. This was needed because of a fixed size buffer in C.
Now that the server is in Haskell, the only reason the length is limited
is to avoid excessive time and space use in the parser, so it is better to
count source characters. This also avoids being too restrictive with
non-European languages.
For further separation of pretty printing concerns from conversion concerns,
the Haskell AST and pretty printer has been moved to its own module,
GF.Haskell, also allowing it to be reused in other places where Haskell
code is generated.
The parse/translate/c-parse/c-translate commands now recognize the option
jsontree=true to augment the returned JSON structure with a field called
"jsontree" next to the field "tree", or "jsontrees" next to "trees",
containing the the returned syntax tree in JSON format (the same format
returned by the abstrjson command, similar to the format returned in the
"brackets" field).
Too see inflection tables, click on a translated sentense, then click on a
blue function name in the textual representation of the abstract syntax tree.
You can now click on a tree to toggle between the abstract syntax tree and
the parse tree.
Also, the implementation now uses the new C run-time requests, e.g.
App14.pgf?command=c-abstrtree&tree=...
so no need to install AppEng.pgf on the server any more.
The trees are drawn by using the Haskell run-time request
AppEng.pgf?command=abstrtree&tree=...
Thus AppEng.pgf has to be installed on the server and in sync with App14.pgf.
This is to avoid loading the huge App14.pgf in the Haskell run-time system, and
is enough since we only need the abstract syntax.
For improved performance, request only one translation from the server
initially. When the user clicks on a translated sentense, request 9 more
translations.
If the enumaration of table parameter values fails during the static
traversal phase, try again in the dynamic computation phase, when the values
of bound variables are known.
This is necessary to properly deal with generic table construction in opers,
like the ones found in prelude/Coordination.gf, e.g.
consTable : (P : PType) -> ... = \P ... -> {s1 = table P {...} ; ... }
GF.Compile.Optimize.mkLinReference can fail and cause this error because
the helper function inside it applies msum to a list that might be empty
(if there is a record type that does not contain a field of type Str).
This means that it can return mzero::Err, i.e.
Bad "error (no reason given)"
which can slip through the top level test that only catches Bad "no string".
Because the prompt included the name of the abstract syntax, the loading
of the PGF was forced even if -retain was used. Even worse,
if an error occured while loading the PGF, it was repeated and caught
every time the prompt was printed, creating an infite loop. The solution
is to not print the name of the abstract syntax when the grammar is
imported with -retain, which is the way things were before anyway.
The commands available in the shell after import -retain are now a superset
of the commands available after import without -retain.
The PGF is created lazily, so there should be no performance penalty if
the PGF isn't needed. If there are errors, they won't be reported until a
command that uses the PGF is entered.
2 modules: Name clashes caused by Applicative-Monad change in Prelude
2 modules: Ambiguities caused by Foldable/Traversable in Prelude
2 modules: Backwards incompatible changes in time-1.5 for defaultTimeLocale
9 modules: {-# LANGUAGE FlexibleContexts #-} (because GHC checks inferred types
now, in addition to explicitly given type signatures)
Also silenced warnings about tab characters in source files.
+ Some additional simplifying rewrites.
+ Use an intermediate representation for Haskell types, for separation of
concerns and cleaner code.
+ Pretty printer layout tuning
+ Code cleanup.
Introduced an intermediate representation for the generated Haskell expressions.
This allows pretty printing concerns to be separated from conversion concerns,
and makes it easy to apply some simplifying rewrites to the generated
expressions, e.g.
[x] ++ [y] ==> [x,y]
pure f <*> x ==> f <$> x
f <$> pure x ==> pure (f x)
join (pure x) ==> x
By adding the flag -haskell=variants to the command line, GF will now generate
linearization functions in Haskell that support variants. Variants are
represented as lists in Haskell.
Variants inside pre { ... } expressions are still ignored.
TODO: apply some monad laws to generate more compact code (using an
intermediate representation of the generated Haskell code, instead of
pretty printing directly from the GF code).
Move the Haskell representation of the common linearization type {s:T} to the
shared module PGF.Haskell, so that the same overloaded projection function
proj_s can be used for all concrete syntaxes.
Common code has been lifted out from the generated Haskell modules to
an auxiliary module PGF.Haskell, which is currently included in the
regular PGF library, although it is independent of it and probably belongs
in a separate library.
The type Str used by linearization functions is now based on a token
type Tok, which is defined in PGF.Haskell.
PGF.Haskell.Tok is similar to the type GF.Data.Str.Tok, but it has
constructors for the special tokens BIND, SOFT_BIND and CAPIT, and there is
a function
fromStr :: Str -> String
that computes the effects of these special tokens.
+ Instead of including lists of parameter values generated by GF, generate
code to enumerate parameter values (in the same order as GF). This seems
to give a factor of 2-3 code size reduction in the Phrasebook (e.g.
from 84MB to 25MB for Hin, from 338MB to 154MB for Fre).
+ Deduplicate table entries, i.e. convert "table [..,E,..,E,..,E,..]" into
"let x = E in table [..,x,..,x,..,x,..]". This gives even more significant
code size reduction in some cases, e.g. from 569MB to 15MB for
PhrasebookFin.
All phrasebook languages can now be converted to compilable Haskell code,
except PhrasebookPes, which still has the name clash problem.
Many Phrasebook languages can now be converted to compilable Haskell code.
Some languages (Fre, Hin, Snd, Urd) generate too much Haskell code to be
practically useful (e.g. 338MB for Fre). One language (Fin) took too long
to convert to Haskell. One language (Pes) has problems with name clashes in
the generated Haskell code.
STILL TODO:
- variants
- pre { ... }
- reduce code duplication for large tables
- generate qualified names to avoid name clashes
The API in the C runtime as well as in the Haskell, Python and Java binding
is changed. Now instead of adding the literal callbacks to the concrete syntax
you need to supply them every time when you need to parse. The main reason is:
- referentially transparent API for Haskell
- when we start using memory mapped files we will not be allowed to change
anything in the grammar data structures. At that point the old API would
be impossible to use.
When mkPresent (or another preprocessor) is used, error messages from GF will
show the file name _gf_preproc.tmp instead of the name of the file where
the error occurred. By compiling Alltenses first, the real file name will
appear in error messages (unless the error only happens then mkPresent is
used).
The translation is currently good enough to translate all concrete syntaxes
of the Foods and Letter grammars, and some concrete syntaxes of the Phrasebook
grammar (e.g. PhrasebookEng & PhrasebookSpa works, but there are problems with
e.g. PhrasebookSwe and PhrasebookChi)
This functionality is enabled by running
gf -make -output-format=haskell -haskell=concrete ...
TODO:
- variants
- pre { ... }
- eta expansion of linearization functions
- record subtyping can still cause type errors in the Haskell code
in some cases
- reduce code large tables
It was used only in cases where a lock field needed to be added to a
run-time variable, like e.g. in examples/phrasebook/SentencesTha.gf:
lin
PGreetingMale g = mkText (lin Text g) (lin Text (ss "ครับ")) | g ;
PGreetingFemale g = mkText (lin Text g) (lin Text (ss "ค่ะ")) | g ;
But lock fields are only meaningful during type checking and can safely be
ignored in later passes.
Renamed appIOE to tryIOE (it is analogous to 'try' in the standard libraries).
Removed unused IOE operations & documented the remaining ones.
Removed/simplified superfluous uses of IOE operations.
This replaces the hardwired ANSI escape codes that were accidentally included
in a previous patch.
This adds a dependency on terminfo, but this should be unproblematic, since
haskeline already depends on the same underlying C library.
The color highlighting is omitted on Windows.
Included renamings:
SourceGrammar -> Grammar
SourceModule -> Module
SourceModInfo -> ModuleInfo
emptySourceGrammar -> emptyGrammar
Also introduces a type synonym (which might be good to turn into a newtype):
type ModuleName = Ident
The reason is to make types like the following more self documenting:
type Module = (ModuleName,ModuleInfo)
type QIdent = (ModuleName,Ident)
IOE used to be a monad with extra error handling built on top of the IO monad,
But the IO monad already supports error handling, so this construction was a
superfluous.
The new 'instance ErrorMonad IOE' is defined to preserve the previous error
handling behaviour, i.e. the function 'handle' only catches errors thrown with
'raise' (or 'fail') and not other errors in the IO monad.
* The following modules are no longer used and have been removed completely:
GF.Compile.Compute.ConcreteLazy
GF.Compile.Compute.ConcreteStrict
GF.Compile.Refresh
* The STM monad has been commented out. It was only used in
GF.Compile.SubExpOpt, where could be replaced with a plain State monad,
since no error handling was needed. One of the functions was hardwired to
the Err monad, but did in fact not use error handling, so it was turned
into a pure function.
* The function errVal has been renamed to fromErr (since it is analogous to
fromMaybe).
* Replaced 'fail' with 'raise' and 'return ()' with 'done' in a few places.
* Some additional old code that was already commented out has been removed.
The module src/compiler/GF.hs now serves as a prelimiary compiler API. It just
exports a selection of functions and types from the compiler.
Haddock documentation can be generated with
cabal haddock --hyperlink-source
Also bumbed the version number to 3.6.10.
loadConcr, unloadConcr and addLiteral modify the Concr structure as a side
effect. This means that other functions with a Concr argument (e.g. parse
and linearize) are no longer pure.
Possible solutions:
1. Don't try to hide the imperative nature of the C run-time system: remove
all uses of unsafePerformIO and let all functions operate in the IO monad.
2. Don't export functions with side effects. Perhaps the desired functionality
of loadConcr, unloadConcr and addLiteral can be folded into readPGF.
The Concr structures can then treaded as immutable after after the
readPGF function returns...
(1) introduces the module GF.Infra.Concurreny with lifted concurrency
operators (to reduce uses of liftIO) and some additional concurrency
utilities, e.g. a function for sequential logging that is used in
both GF.CompileInParallel and GFServer.
(2) avoids leaving broken .gfo files behind if compilation is aborted.
* httpd-shed-0.4 does not specify an upper bound on network, but it fails
to build against network>=2.6. This is fixed in httpd-shed-0.4.0.2.
* With network-2.6, the Network.URI modules is moved to a separate package,
so for the time being GF requires network>=2.3 && <2.6. This is compatible
with the four most recent versions of the Haskell Platform.
* Introducing the module CGI, re-exporting a subset of the cgi package. It
might complete replace the cgi package in the future.
* Introducing the module CGIUtils, containing functions from FastCGIUtils that
have nothing to do with fastcgi.
Some low level hackery with unsafePerformIO and global variables was left
in FastCGIUtils, but it is actually not used, neither for gf -server nor
exec/pgf-fcgi.hs.
This makefile just calls GF once and lets GF figure out in which order to
compile things. It uses the -j flag to enable parallel compilation and
specifies an explicit -path, overriding the -path flags in the source files.
This allows all needed modules to be found automatically and ensures that
that alltenses is consistently used everywhere. But for some reason, this
doesn't work...
The script bin/build-binary-dist.sh has been updated to build either a plain
.tar.gz package or OS X Installer package (.pkg).
Note that bin/build-binary-dist.sh is designed to build and include the
C run-time system in the binary package. If the C run-time system fails to
build, no binary package will be created.
On my laptop these changes speed up the full build of the RGL and example
grammars with 'cabal build' from ~95s to ~43s and the zero build from ~18s
to ~5s.
The main change is the introduction of the module GF.CompileInParallel that
replaces GF.Compile and the function GF.Compile.ReadFiles.getAllFiles. At
present, it is activated with the new -j flag, and it is only used when
combined with --make or --batch. In addition, to get parallel computations,
you need to add GHC run-time flags, e.g., +RTS -N -A20M -RTS, to the command
line.
The Setup.hs script has been modified to pass the appropriate flags to GF
for parallel compilation when compiling the RGL and example grammars, but you
need a recent version of Cabal for this to work (probably >=1.20).
Some additonal refactoring were made during this work. A new monad is used to
avoid warnings/error messages from different modules to be intertwined when
compiling in parallel, so some functios that were hardiwred to the IO or IOE
monads have been lifted to work in arbitrary monads that are instances in
the appropriate classes.
I prefer small functions with descriptive names over large monilithic chunks
of code, so I grouped the compiler passes called from compileSourceModule
into funcitons named frontend, middle and backend. This also makes decisions
about which passes to run clearly visible up front.
Also made some small changes in GF.Compile.
In particular, the function compileOne has been moved to the new module
GF.CompileOne and its type has been changed from
compileOne :: ... -> CompileEnv -> FilePath -> IOE CompileEnv
to
compileOne :: ... -> SourceGrammar -> FilePath -> IOE OneCompiledModule
making it more suitable for use in a parallel compiler.
The RGL is now compile with only three calls to GF (prelude, present,
alltenses). This also makes even more parallelism available to GF for speeding
up full builds of the RGL.
The def rules are now compiled to byte code by the compiler and then to
native code by the JIT compiler in the runtime. Not all constructions
are implemented yet. The partial implementation is now in the repository
but it is not activated by default since this requires changes in the
PGF format. I will enable it only after it is complete.
This speedup is obtained by instead of generating 211 shell commands to compile the RGL one file at a time, generate 7 shell commands, passing a number of
related files to GF in one go.
In addition to cutting down the zero build time, this opens up for speeding up
full builds of the RGL, by adding more parallelism in GF's grammar compilation
machinery.
(Note: gf.cabal already takes advantage of GHC's parallel build option to
speed up the compilation of GF itself, if GHC>=7.8 is used.)
GF.Text.Pretty provides the class Pretty and overloaded versions of the pretty
printing combinators in Text.PrettyPrint, allowing pretty printable values to
be used directly instead of first having to convert them to Doc with functions
like text, int, char and ppIdent. Some modules have been converted to use
GF.Text.Pretty, but not all. Precedences could be added to simplify the pretty
printers for terms and patterns.
GF.Infra.Location contains the types Location and L, factored out from
GF.Grammar.Grammar, and the class HasSourcePath. This allowed the import
of GF.Grammar.Grammar to be removed from GF.Infra.CheckM, making it more
like a pure library module.
Detected when building today:
Building gf-3.6...
Preprocessing library gf-3.6...
src/runtime/haskell/PGF/Internal.hs:17:8:
Could not find module ‘Data.Binary.IEEE754’
It is a member of the hidden package ‘data-binary-ieee754-0.4.4’.
Perhaps you need to add ‘data-binary-ieee754’ to the build-depends in your .cabal file.
When using full=yes in the web service 'complete' command,
you now get an additional field 'seq' with the longest possible completion.
So, given:
lin
f1 = ss "the" ;
f2 = ss ("the red house" | "the real deal") ;
and trying to complete on input "th", you get:
[
{
"from": "TestCnc",
"brackets": {
"cat": "_",
"fid": 0,
"index": 0,
"fun": "_",
"children": []
},
"text": "th",
"completions": [
{
"token": "the",
"funs": [
{
"fun": "f1",
"hyps": [],
"cat": "C",
"seq": "the"
},
{
"fun": "f2",
"hyps": [],
"cat": "C",
"seq": "the red house"
},
{
"fun": "f2",
"hyps": [],
"cat": "C",
"seq": "the real deal"
}
]
}
]
}
]
The wide coverage demo apps now shows a "please wait" message while the grammar
is loading, and a red error message if the grammar is not found on the server.
The Simple Translation Tool also show red error messages if the grammar is not
found.
The configuration script takes significantly longer to run than the compilation
of all the C code, which is a hint that is in need of a major simplification.
PGF exports the public, stable API.
PGF.Internal exports additional things needed in the GF compiler & shell,
including the nonstardard version of Data.Binary.
From the documentation: the throwIO variant should be used in preference to
throw to raise an exception within the IO monad because it guarantees ordering
with respect to other IO operations, whereas throw does not.
Also removed some unused imports.
(table { p_i => t_i } ! x).l ==> table { p_i => t_i.l } ! x
This was used in the old partial evaluator and can significantly reduce term
sizes in some cases.
Eta expansion is applied between partial evaluation and PMCFG generation.
The buggy version generated type incorrect terms, but PMCFG generation
apparently worked anyway.
The logs are simply dumped there in raw form, but at least having a link
should make them Google-indexed. One day we'll put something there with
an interface.
Also improve behaviour for languages that are missing in Phrasebook (Chinese).
Also update the initial set of languages in the menus to match Translate11.pgf.
1. Like pgf-translate, it now shows one result at a time, press Enter to get
more results.
2. You can load a new grammar with the command 'i <path-to-pgf>'
This means that the new translation is available instantly when switching
target languages. It can also reduce space leaks problems in the server
somewhat by avoiding repeated parsing of the source text.
+ Programs compiled with ghc<7.8 have a default built-in 8M stack size limit.
With ghc>7.0 this built-in default can be changed, so it is now 64M, allowing
GF to work with larger grammars without manually increasing the stack size.
(But if GF is compiled with ghc 6.12, the built-in default is still 8M.)
+ Enable ghc>=7.8 support for compiling modules in parallel, to speed up the
compilation of GF.
For example, you can now reorder paragraphs by cut & paste, without causing
anything to be sent to the server for re-translation.
Also some color & layout tweaks.
Also,
+ Get up to 10 (instead of 5) alternative robust translations.
+ Remove duplicates from list of alternative robust translations
+ Add GF Cloud logo to the simple translator page
Caching parse results uses a lot of memory, even if they expire after
2 minutes, so it won't scale up to many simultaneous users.
But some excessive memory use seems to be caused by space leaks in
(the Haskell binding to) the C run-time system, and these should be fixed.
For example, flushing the PGF cache does not release the memory allocated
by the C run-time system when loading a PGF file.
Most PGF web API commands that produce linearizations now accept an
unlexer parameter. Possible values are "text", "code" and "mixed".
The web service now include Date and Last-Modified headers in the HTTP,
responses. This means that browsers can treat responses as static content and
cache them, so it becomes less critical to cache parse results in the server.
Also did some cleanup in PGFService.hs, e.g. removed a couple of functions
that can now be imported from PGF.Lexing instead.
The old type was [String] -> String. This function was only used
in GF.Text.Lexing.stringOp, which now uses (unwords . bindTok) instead,
with no change in behaviour.
The capitalization of the first word was done in GF.Text.Lexing.stringOp,
but is now done in the functions unlexText and unlexMixed in PGF.Lexing.
These functions are only used in stringOp and in PGFService (where the change
is needed), so the subtle change in behaviour should not cause any bugs.
Only change the first word to lowercase if the original input is not found in
the grammar's morphology. This allows parsing of sentenses starting with "I" in
English, nouns in German and proper names in other languages, but it can make
the wrong choice for multi-words.
The PGF web API commands parse, translate, translategroup, c-parse and
c-translate now support a lexer parameter. Possible values are "text",
"code" and "mixed".
This is used in the Wide Coverage Demo App to save one server roundtrip.
+ The current type checker for concrete syntax is in
GF.Compile.TypeCheck.RConcrete, but GF.Compile.TypeCheck.Concrete was
still imported in GFI.
+ Fixed a bug that allowed Ints n as a subtype of Ints m, regardless of
m and n. It now requires n<=m. Note: the type checker still allows Int
as a subtype of Ints m, regardless of m.
+ Fixed a potential efficiency problem with large record types, by reducing
the number of recursive calls from |R|*|S| to |R| when checking if R<=S.
+ Fixed a misleading comment: "alpha g t u" checks that u is a subtype of t,
the other way around. Similarly, "checkIfEqLType gr g t u trm" checks that
u is a subtype of t, not the other way around, and not that t is equal to u.
The layout now resembles a certain other online translation service.
The translation is started automatically after text is entered, no need to
press the Translate button.
This bug was introduced sometime between 2013-08-21 and 2013-11-01 and caused
the function convertTerm in GF.Compile.GeneratePMCFG to encounter a EPatt where
it expected Strs. I fixed it by applying the function getPatts (from the old
partial evaluator) to the pattern.
+ Adapt language selection menus to the languages supported by the grammar.
+ Add translation quality colors to the links to alternative translations.
+ Also added more country codes in langcode.js (needed for speech synthesis
voice selection).
The background color of the translation changes to pale green, yellow or red
according to the quality.
Two types of quality indications are supported:
+ From the grammar: linearizations starting with "+" or "*" are assumed to
be of high or low quality, respectively.
+ From the parse tree: if the root function is "?", then the translation is
assumed to be of low quality.
If no quality indications are detected, medium quality is assumed.
In support.js, the functions http_get_json, ajax_http_get_json
and ajax_http_post_json now calls the supplied error callback if the server
returns invalid JSON (e.g. because of a crash).
The function gftranslate.translate in gftranslate.js returns
a JSON value containing an error message (since it doesn't have an error
callback).
This should result in fewer situations where "nothing happens" and the user
doesn't know if it is beacuse the server is slow, or if there was an error.
The directory should be the same as the one used in Setup.hs when building
the RGL, where a build directory supplied by cabal is used. This is usually
dist/build/rgl, but it could be different, e.g. when using a cabal sandbox.
Two of the fixes were in the custom version of the binary package. We
should get rid of it at some point, to reduce the maintenance burden.
The third fix was to add new version constraints for happy and alex in
gf.cabal. New versions of them are needed because of the ugly, low-level,
GHC-specific code they produce need to be different for GHC 7.8.
More fixes might be needed for -server mode, but the cgi package is not
compatible with GHC 7.8 at the moment, so it will have to wait.
+ PGFService.hs: add command c-grammar, include probability in parse results
+ js/gftranslation.js: add start position and limit parameters, return more
info to applications
+ Simple Translator: show two wide coverage translations
+ Wide coverage demo: show parse tree and probability (intended as grammar
debugging aids)
PGF service requests are stateless and can run in parallel, but some other
requests handled by the server are not and might even change the current
working directory temporarily, and this affects all threads, so it is
important that the PGF service requests access PGF files by absolute paths.
The interface to the wide coverage translation is in js/gftranslate.js
and it assumes that the grammar is installed on the cloud server
as /robust/Translate8.pgf.
The list of supported languages is hardwired in gftranslate.js, since
there is no support for obtaining this info from the C run-time system
at the moment.
If the C run-time library is compiled and installed on your system, you can now
do 'cabal configure -fc-runtime' to get the following extras:
+ The haskell binding to the C run-time library will be included in the
PGF library (so you can import it in Haskell applications).
Documentation on the new modules will be included when you run
'cabal haddock'.
+ The new command 'pgf-shell', implemented on top of haskell binding to
the C run-time system.
+ Three new commands in the web API: c-parse, c-linearize and
c-translate. Their interfaces are similar to the corresponding commands
without the "c-" prefix, but they should be considered preliminary.
* The haskell-bind.cabal file makes it easy to build the haskell binding and
use it in ghci.
* pgf-shell.hs is a simple example of how to use the haskell binding.
Change the type of getConcr from
getConcr :: PGF -> Language -> Concr
to
getConcr :: PGF -> Language -> Maybe Concr
This is to prevent programs from crashing later if you try to select a
concrete syntax that is not present in a grammar.
When running a command like
gf -make L_1.gf ... L_n.gf
gf now avoids recreating the target PGF file if it already exists and is
up-to-date.
gf still reads all required .gfo files, so significant additional speed
improvements are still possible. This could be done by reading .gfo files
more lazily...
When running a command like
gf -make -name=T L_1.pgf ... L_n.pgf
gf now checks if T.pgf exists and is up-to-date before reading and computing
the union of the L_i.pgf files.
The name (T) of the target PGF file has to be given explicitly for this to work,
since otherwise the name is not known until the union has been computed.
If the functions for reading PGF files and computing the union were lazier,
this would not be necessary...
Some backwards incompatible changes were made to the PGF file format after
the release of GF 3.5. This patch adds a module for reading PGF files in the
old format.
This means that old PGF files on the grammaticalframework.org server will
continue to work after we install the latest version of GF.
There were two differences between the current output and the old gold file:
1. The trees are no longer generated with increasing depth
2. The meaning of the -depth flag has changed: for example,
"gt -cat=Nat -depth=1" used to generate only "zero",
now you also get "succ zero".
* The old way: a user hook in Setup.hs
* The new way: specify it in gf.cabal
* The test suite is now called gf-tests, and it runs testsuite/run.hs.
* You can run it manually with 'runhaskell testsuite/run.hs'. It also runs,
together with rgl-tests, when you do 'cabal test'
* Currently only 9 of 34 tests pass. Many failures have silly causes:
- Error messages that look slightly different
- Same output but in a different order
- Absolute paths in output
This means that the -old-comp and -new-comp flags are not recognized anymore.
The only functional difference is that printnames were still normalized with
the old partial evaluator. Now that is done with the new partial evaluator.
This was a fairly simple change thanks to previous work on making the Ident
type abstract and the fact that PGF.CId already uses UTF-8-encoded
ByteStrings.
One potential pitfall is that Data.ByteString.UTF8 uses the same type for
ByteStrings as Data.ByteString. I renamed ident2bs to ident2utf8 and
bsCId to utf8CId, to make it clearer that they work with UTF-8-encoded
ByteStrings.
Since both the compiler input and identifiers are now UTF-8-encoded
ByteStrings, the lexer now creates identifiers without copying any characters.
**END OF DESCRIPTION***
Place the long patch description above the ***END OF DESCRIPTION*** marker.
The first line of this file will be the patch name.
This patch contains the following changes:
M ./src/compiler/GF/Compile/CheckGrammar.hs -3 +3
M ./src/compiler/GF/Compile/GrammarToPGF.hs -2 +2
M ./src/compiler/GF/Grammar/Binary.hs -5 +1
M ./src/compiler/GF/Grammar/Lexer.x -11 +13
M ./src/compiler/GF/Infra/Ident.hs -19 +36
M ./src/runtime/haskell/PGF.hs -1 +1
M ./src/runtime/haskell/PGF/CId.hs -2 +3
1. The default encoding is changed from Latin-1 to UTF-8.
2. Alternate encodings should be specified as "--# -coding=enc", the old
"flags coding=enc" declarations have no effect but are still checked for
consistency.
3. A transitional warning is generated for files that contain non-ASCII
characters without specifying a character encoding:
"Warning: default encoding has changed from Latin-1 to UTF-8"
4. Conversion to Unicode is now done *before* lexing. This makes it possible
to allow arbitrary Unicode characters in identifiers. But identifiers are
still stored as ByteStrings, so they are limited to Latin-1 characters
for now.
5. Lexer.hs is no longer part of the repository. We now generate the lexer
from Lexer.x with alex>=3. Some workarounds for bugs in alex-3.0 were
needed. These bugs might already be fixed in newer versions of alex, but
we should be compatible with what is shipped in the Haskell Platform.
Makefile2 specificies the dependencies accurately enough that you can use
make's support for parallel compilation. Run
make -f Makefile2 -j
to utilize all the cores in your computer to significantly speed up the
creation of Phrasebook.pgf.
+ Eliminated vairous ad-hoc coersion functions between specific monads
(IO, Err, IOE, Check) in favor of more general lifting functions
(liftIO, liftErr).
+ Generalized many basic monadic operations from specific monads to
arbitrary monads in the appropriate class (MonadIO and/or ErrorMonad),
thereby completely eliminating the need for lifting functions in lots
of places.
This can be considered a small step forward towards a cleaner
compiler API and more malleable compiler code in general.
1. No temporary files are created.
2. The output of a system command is read lazily, making it feasible to
process large or even infinite output, e.g. the following works as
expected:
? "yes" | ? "head -5" | ps -lextext
The system_pipe (aka "?") command creates a temporary file _tmpi containing
the input of the system command. It *both* appends _tmpi as an extra argument
to the system command line *and* adds an input redirection "< _tmpi". (It
also uses and output redirection "> _tmpo" to captures the output of the
command.)
With this patch, the _tmpi argument is no longer appended to the command line.
This allows system_pipe to work with pure filters, such as the "tr" commands,
but it will no longer work with commands that require an input file name.
(It is possible to use write_file instead...)
TODO: it would also be fairly easy to eliminate the creation of the _tmpi and
_tmpo files altogether.
Add proper type checking of course-of-values tables:
+ Make sure that all subterms have the same type.
+ Resolve overloaded operators.
Note though that the GF book states in C.4.12 that the "course-of-values
table [...] format is not recommended for GF source code, since the
ordering of parameter values is not specified and therefore a
compiler-internal decision."
The CF parser in GF.Grammar.CF assigns function names to the rules, but they
are not always unique, causing rules to be dropped in the follwing CF->GF
conversion. So a pass has been added before the CF->GF conversion, to make
sure that function names are unique.
A comment says "rules have an amazingly easy parser", but the parser looks
like quick hack. It is very sloppy and silently ignores many errors, e.g.
- Explicitly given function names should end with '.', but if the do not, the
last character in the function name is silently dropped.
- Everything following a ';' is silently dropped.
Trailing spaces caused the command line parse to be ambiguous, and
ambiguous parses were rejected by function readCommandLine, causing
the cryptic error message "command not parsed".
The only use of PGF.Tree outside the PGF library was in GF.Command.Commands,
and it was eliminated by using PGF.Expr directly instead.
PGF.Paraphrase still uses PGF.Tree.
This module should not be part of the public PGF library API, and it was only
used in GF.CompileToAPI, so the code was moved there. The module defined
constFuncs and syntaxFuncs, but only syntaxFuncs was used.
+ References to modules under src/compiler have been eliminated from the PGF
library (under src/runtime/haskell). Only two functions had to be moved (from
GF.Data.Utilities to PGF.Utilities) to make this possible, other apparent
dependencies turned out to be vacuous.
+ In gf.cabal, the GF executable no longer directly depends on the PGF library
source directory, but only on the exposed library modules. This means that
there is less duplication in gf.cabal and that the 30 modules in the
PGF library will no longer be compiled twice while building GF.
To make this possible, additional PGF library modules have been exposed, even
though they should probably be considered for internal use only. They could
be collected in a PGF.Internal module, or marked as "unstable", to make
this explicit.
+ Also, by using the -fwarn-unused-imports flag, ~220 redundant imports were
found and removed, reducing the total number of imports by ~15%.
The standard binary package has improved efficiency and error handling [1], so
in the long run we should consider switching to it. At the moment, using it is
possible but not recommended, since it results in incomatible PGF files.
The modified modules from the binary package have been moved from
src/runtime/haskell to src/binary.
[1] http://lennartkolmodin.blogspot.se/2013/03/binary-07.html
* In the shell, the new command tt (to_trie) merges a list of trees into a
trie and prints it in a readable way, where unique subtrees are marked with
a "*" and alternative subtrees are marked with numbers.
* In the PGF web service, adding the parameter trie=yes to the parse and
translate commands augments the JSON output with a trie.
Example to try in the shell:
Phrasebook> p -lang=Eng "your son waits for you" | tt
This adds a simple voice translator Android app that uses
the JNI bindings to the PGF C runtime.
Caveats:
- Since the C runtime doesn't compile for Android right now,
I've bundled an old copy, along with its Java bindings.
That should be removed once the C runtime compiels for Android
again.
- Adding an automated build would be nice.
- Replacing the grammar requires editing a Java file, that should
really be more dynamic.
+ Split the list of languages into several columns if space permits
(in supported browsers, currently Firefox).
+ Limit the size of the GF logo to 75% of width the window. Looks better on
small screens.
The following are the outcomes:
- Predef.nonExist is fully supported by both the Haskell and the C runtimes
- Predef.BIND is now an internal compiler defined token. For now
it behaves just as usual for the Haskell runtime, i.e. it generates &+.
However, the special treatment will let us to handle it properly in
the C runtime.
- This required a major change in the PGF format since both
nonExist and BIND may appear inside 'pre' and this was not supported
before.
The fact that identifiers are represented as ByteStrings is now an internal
implentation detail in module GF.Infra.Ident. Conversion between ByteString
and identifiers is only needed in the lexer and the Binary instances.
Most of the explicit uses of ByteStrings were eliminated by using identS,
identS = identC . BS.pack
which was found in GF.Grammar.CF and moved to GF.Infra.Ident. The function
prefixIdent :: String -> Ident -> Ident
allowed one additional import of ByteString to be eliminated. The functions
isArgIdent :: Ident -> Bool
getArgIndex :: Ident -> Maybe Int
were needed to eliminate explicit pattern matching on Ident from two modules.
Before, they were silently converted to linear patterns.
Nonlinear patterns in MorphoCat.gf, ParadigmsGre.gf and ParadigmsFin.gf have
been make linear by renaming pattern variables.
The refresh pass does not correctly keep track of the scope of local variables
and can convert things like \x->(\x->x) x into \x1->(\x2->x2) x2. Fortunately,
it appears that the refresh pass is not needed anymore, so it has been removed.
The function GF.Grammar.PatternMatch.isInConstantForm returned False for all
tables, causing matchPattern to fail, claiming that "variables occur in" the
term if it contains tables.
This problem is several years old, confirmed present in GF 3.2.10 (Oct 2010).
When a browser requests a URL that refers to a directory, web server usually
redirect the browser to the same URL with a trailing '/' added, if one was not
already present. This is to prevent relative links in the returned document
from being interpreted relative to the parent directory instead of the current
document. This type of redirection was missing in gf -server.
By adding a header
Access-Control-Allow-Origin: *
to the HTTP responses, web browsers are informed that it is OK to call the
services from web pages hosted on other sites.
This is apparently supported in most modern browsers, so it should no longer
be necessary to resort to JSONP.
See https://developer.mozilla.org/en-US/docs/HTTP/Access_control_CORS
The package network-2.4.1.1 thoughlessly introduced a backward incompatible
change to the function Network.URI.unEscapeString, see
f2168b1f89
This also affects the function Network.Shed.Httpd.queryToArguments, which is
used in GFServer.hs.
To remain compatible with older and newer versions of the network package,
we need to stay clear of queryToArguments and unEscapeString. A replacement
function has been added to server/URLEncoding.hs.
The &+ operator is now consistently eliminated from the output of commmands
that produce linearizations. Before, only the commands translate and
translategroup did this.
GF does not show up if you google for machine translation. This meta tag might
help, but in general I think it is better for SEO to mention relevant keyword
in the body text than to rely on meta tags.
+ The syntax editor is now accessible from the Simple Translaton Tool.
+ The minibar now automatically provides access to the syntax editor (provided
the necessary JavaScript files and style sheets have been loaded).
+ Preparations for making the syntax editor accessible from the grammar editor.
Also in Commands.hs: be explicit about things imported from the PGF library
that are not in the public API.
Also a couple of haddock documentation fixes.
When compiling a grammar containing characters that are not supported in the
current locale, warning messages could cause GF fail with
hPutChar: invalid argument (Invalid or incomplete multibyte or wide character)
With this quick fix, warning messages that can not be displayed are silently
truncated instead, and compilation continues.
The new options are
* noleaves, nofun, nocat (booleans, false by default)
* nodefont, leaffont, nodecolor, leafcolor, nodeedgestyle, leafedgestyle
(strings, empty by default)
Some commands (linearize, linearizeAll, random, generate, translate and
translategroup) by default produce output in all languages supported by the
grammar and the 'to' parameter could be used to restrict output to a single
language. Now you can restrict the output to a list of languages. Languages
should be separated by spaces.
Also removed an unnecessary LANGUAGE pragma and reduced code verbosity.
This should prevent errors like
Internal error in Compute.ConcreteNew:
Applying Predef.drop: Expected a value of type String, got VFV [VString "gewandt",VString "gewendet"]
Fixes the following build failure:
src/runtime/haskell/Data/Binary/IEEE754.lhs:256:17:
Could not deduce (Num a) arising from a use of `mask'
from the context (Bits a)
bound by the type signature for
clamp :: Bits a => BitCount -> a -> a
The sequence operator (x+y) was implemented by splitting the string to be
matched at all positions and trying to match the parts against the two
subpatterns. To reduce the number of splits, we now estimate the minimum and
maximum length of the string that the subpatterns could match. For common
cases, where one of the subpatterns is a string of known length, like
in (x+"y") or (x + ("a"|"o"|"u"|"e")+"y"), only one split will be tried.
GF produced slightly different PGF files on 64-bit systems and 32-bit systems.
This could cause problems when a PGF was produced on a 32-bit system and used
on a 64-bit system.
To fix this, the GF compiler and the Haskell PGF run-time library now reads
and writes PGF files like the 32-bit version even when compiled on a 64-bit
system.
Note: the Haskell type Int is still used internally in GF, which could be
32 bits or 64 bits...
The reported number of recorded changes since the last tagged version was off
by one (because it counted the change that set the tag).
Also added a note that -K32M is not needed when using new-comp. But -old-comp
is still available as a configuration option, so I'm keeping -K32M for now.
The parser works on raw byte sequences read from source files. If parsing
succeeds the raw byte sequences are converted to proper Unicode characters
in a later phase. But the parser calls the function buildAnyTree, which can
fail and generate error messages containing source code fragments, which might
then containing raw byte sequences. To render these error messages correctly,
they need to be converted in accordance with the coding flag in the source
file. This is now done for UTF-8-encoded source files, but should ideally also
be done for other character encodings. (Latin-1-encoded files never suffered
from this problem, since raw bytes are proper Unicode characters in this case.)
+ Instead of "Internal error in ...", you now get a proper error message with
a source location and a function name.
+ Also added some missing error value propagation in the partial evaluator.
+ Also some other minor cleanup and error handling fixes.
Also a bug fix when switching to editor, although this still messes up
when using the letters grammar.
Also updated readme with options, and some style improvements.
This makes it possible to download PGF files from servers where the PGF service
is installed.
I am also considering making commmand=download the default instead of
command=grammar.
- added a paradigm for alternative plurals
- fixed usage of alternative plurals in Numerals
- added a distinction between numerals and posessive pronouns
- verb omission and appropriate cases in some uses of to be/to have
- fixed declination of personal names
- added a gender parameter to pronouns (similar to Polish RG)
- vowel insertion before short adjective suffix
- don't generate short forms for relative adjectives (might need lexicon update)
- minor Phrasebook fixes
It failed to delay table selection when the selector contains a run-time
variable, causing "gf: Prelude.(!!): index too large" instead.
Also:
+ Show better source locations on unexpected errors, to aid bug hunting.
+ Removed unused SourceGrammar argument to value2term.
The work done by the partial evaluator is now divied in two stages:
- A static "term traversal" stage that happens only once per term and uses
only statically known information. In particular, the values of lambda bound
variables are unknown during this stage. Some tables are transformed to
reduce the cost of pattern matching.
- A dynamic "function application" stage, where function bodies can be
evaluated repeatedly with different arguments, without the term traversal
overhead and without recomputing statically known information.
Also the treatment of predefined functions has been reworked to take advantage
of the staging and better handle partial applications.
* Evaluate operators once, not every time they are looked up
* Remember the list of parameter values instead of recomputing it from the
pattern type every time a table selection is made.
* Quick fix for partial application of some predefined functions.
* -new-comp (the new partial evaluator) is now chosen by default when you run
cabal install (or cabal configure). To revert to using the old partial
evaluator by default, use "cabal install -f-new-comp" (or
"cabal configure -f-new-comp").
* Regardless of the configured default, you can choose which partial evaluator
to use when you invoke gf by using the -new-comp or -old-comp command line
option.
* The cc command in the GF shell uses the chosen partial evaluator by default,
but you can override this by using "cc -new" or "cc -old".
The plan is that these flags will be romeved in a future version.
- separate tree edit buttons from option buttons
- fix bug when wrapping on freshly imported ast
- add interface for import & export of ast
- cleaner internal implementation of Editor.add_refinement
- small style updates
When at a non-leaf node, refinements with identical type signatures
are highlighting and can re placed without destroying the children.
If not, the refinement is greyed and the user is asked to clear
the current subtree first if they wish to replace it.
This aspect of the UI should be polished, but at least it is obvious.
Also, some substantial optimizations can still be made to cache
the processed type signatures (which determine what can be replaced
in-place)
After pressing the "i" button to view grammar info, there is now a
"More info" button to get more detailed information about all categories and
functions in the grammar.
Using
%!encoding:utf-8
in txt2tags files ensures that the generated HTML includes an appropriate
charset declaration. This way we avoid relying on browser defaults, which can
be wrong.
To make definitions that span multiple lines more readable in guided mode,
preserve line breaks and indentation when lin and oper definitions are
displayed and edited in guided mode.
Bug fix: moving the cursor by clicking and selecting text by dragging did
not work in textareas and input fields, because those mouse event were
intercepted by the sort-by-drag-and-drop handler.
instead of GET, to avoid the Apache httpd URI length limitation.
This means that gfse can no longer use JSONP when parsing grammars, so it
has to be located on the same host as the cloud service.
Also clean away trailing white space in support.js.
The pretty printer produced
mkDet pre {"a"; "an" / vowel} Sg
which is not accepted by the parser. The parser assigns pre { ... }, to
prededence level 4, and this is now reflected in the pretty printer, so
it prints
mkDet (pre {"a"; "an" / vowel}) Sg
(This caused a problem in GFSE since it parsers pretty printed grammars...)
This is done by including a lin_action when starting the syntax editor from
the minibar.
Also: added some minibar options in the syntax editor for consistency...
Note that some of the graphviz functions have backwards incompatible changes
that might also affect other clients of the PGF run-time library.
Also added graphvizDefaults and export it together with GraphvizOptions from
the PGF run-time library.
+ Transfer current astract syntax tree when activating the syntax editor
from the minibar.
+ Add options to start the minibar with given input. You need to include
the current grammar url, the current language and the input, like this:
initial_grammar: "/grammars/Foods.pgf",
initial: { from: "FoodsEng", input:["that","fish","is","fresh"]}
+ Fix a style sheet clash.
Created the folder js and moved minibar/support.js and minibar/pgf_online.js
there, to emphasize their status as app independent support libraries.
There are probably more files that should be moved here.
The cloud apps have been updated, externally hosted apps that link directly
to our server need to be updated too.
The browse command used to have a required parameter id=... and it returned
info on the given identifier only. Now, if format=json, the id=... parameter
can be omitted to get info on all identifiers at the same time. The returned
JSON structure in this case is
{cats:{...},funs:{...}}
where the inner objects contain one field per category and function,
respectively, in the same format as when you request info on one category or
function.
GF.Compile.Compute.ConcreteNew + two new modules contain a new
partial evaluator intended to solve some performance problems with the old
partial evalutator in GF.Compile.Compute.ConcreteLazy. It has been around for
a while, but is now complete enough to compile the RGL and the Phrasebook.
The old partial evaluator is still used by default. The new one can be activated
in two ways:
- by using the command line option -new-comp when invoking GF.
- by using cabal configure -fnew-comp to make -new-comp the default. In this
case you can also use the command line option -old-comp to revert to the old
partial evaluator.
In the GF shell, the cc command uses the old evaluator regardless of -new-comp
for now, but you can use "cc -new ..." to invoke the new evaluator.
With -new-comp, computations happen in GF.Compile.GeneratePMCFG instead of
GF.Compile.Optimize. This is implemented by testing the flag optNewComp in
both modules, to omit calls to the old partial evaluator from GF.Compile.Optimize
and add calls to the new partial evaluator in GF.Compile.GeneratePMCFG.
This also means that -new-comp effectively implies -noexpand.
In GF.Compile.CheckGrammar, there is a check that restricted inheritance is used
correctly. However, when -noexpand is used, this check causes unexpected errors,
so it has been converted to generate warnings, for now.
-new-comp no longer enables the new type checker in
GF.Compile.Typeckeck.ConcreteNew.
The GF version number has been bumped to 3.3.10-darcs
There was 55 lines of rather repetitive code with calls to 6 compiler passes.
They have been replaced with 19 lines that call the 6 compiler passes
plus 26 lines of helper functions.
The output from commands is represented as ([Expr],String), where the [Expr] is
used when data is piped between commands and the String is used for the final
output. The String can represent the same list of trees as the [Expr] and/or
contain diagnostic information.
Sometimes the data that is piped between commands is not a list of trees, but
e.g. a string or a list of strings. In those cases, functions like fromStrings
and toStrings are used to encode the data as a [Expr].
This patch introduces a newtype for CommandOutput and collects the functions
dealing with command output in one place to make it clearer what is going on.
It also makes it easier to change to a more direct representation of piped
data, and make pipes more "type safe", if desired.
- Alternative variants in AScared.
- The order of adv. modifiers: first time, then place (PSeeYouPlaceDate).
- Fixed: the object valence in QWhatAge; Day.point linearization.
- PSeeYouPlaceDate is now less formal.
- Open/Closed and SeeYou functions are ok now, except that the word order in Open/Closed is a bit clumsy. TODO: "<what> <when> is open/closed" (currently not supported by the RG).
Fixed QWhatName, QWhatAge, HowMuchCost and ItCost, however, the solutions have to be reviewed.
Introduced a Lav-specific type for Language (affected also NPNationality etc.) to support a different realization of ASpeak.
Introduced alternative realizations for AHasName and ASpeak (the most typical/simple variant is the default one).
+ You can click on a column header to switch to column view (i.e. the
corresponding abstract/concrete syntax tab).
+ The categories and functions in the abstract syntax can be edited directly
in the matrix view.
+ In the PGF web service API, the commands parse, translate and translategroup
now accept a limit parameter. It limits the number of parse trees returned.
If no limit is given, the behaviour is backwards compatible, i.e. no limit
is applied.
+ In minibar, the limit is now set to 25 by default.
This change is based on a suggestion by Kaarel Kaljurand.
Most of the pages on the GF web site have an exemplary simple design,
with just one column of text. This make them adapt exceptionally
well to screens of different sizes. In particular, they should be
easy to read even on smartphones.
However, smartphone browsers like Mobile Safari and the default
Android Browser assume that pages do *not* adapt well to small
screens, so by default they emulate a big screen, forcing the user
to zoom in to a part of the page to be able to read it. By adding
the meta tag
<meta name = "viewport" content = "width = device-width">
the big screen emulation can be turned off, allowing pages to be
formatted to fit the actual screen size and text to be displayed
at a readable size.
+ Added "Symbolic" to the list of resource modules that can be opened.
+ Clicking on the name of an opened resource module now displays the list
operations it provides. (Implemented using show_operations in the GF shell.)
+ Added a GF version info link to the cloud service start page.
This is a simple change in GF.Grammar.Lookup.allOpers, which is used only in
the implementation of the show_operations command in the shell.
This is useful when importing a concrete syntax (like LexiconEng) as a resource.
However, the types don't always look as nice as I hoped...
+ The restrictions on arbitrary IO when GF is running in restricted mode is now
enforced in the types.
+ This hopefully also solves an intermittent problem when accessing the GF
shell through the web API provided by gf -server. This was visible in the
Simple Translation Tool and probably caused by some low-level bug in the
GHC IO libraries.
The SIO monad is a restriction of the IO monad with two purposes:
+ Access to arbitrary IO operations can be turned off by setting the environment
variable GF_RESTRICTED. There is a limited set of IO operations that are
considered safe and always allowed.
+ It allows output to stdout to be captured. This can be used in gf -server
mode, where output of GF shell commands is made part of HTTP responses
returned to clients.
The dependency on PGFEnv has been moved from the list to the exec function of
the commands in the list. This means that the help command no longer needs
to generate a new list of commands and that the state of the shell
(type GF.Command.Interpreter.CommandEnv) no longer needs to contain the list
of commands.
+ More restrictive limits on which file paths can be downloaded and removed.
+ Add more extensions to the list of file types that may be removed. In
particular, allow documents created by simple translation tool to be removed.
AHasAge, AHasChildren, AHasRoom, AHasTable, ALike - "almost" done: phrases (cases) are grammatically correct, but the word order has to be changed (TODO in the resource grammar).
+ The instructions have been modernized and simplified slightly and should now
work again. (I have not verified it on Windows.)
Using 'cabal' instead of 'runghc Setup.hs' avoids some problems.
+ Also, download/index.html has simpler installation instructions sufficient
for many users, so mention it in the main README file.
+ Also adjusted the main Makefile.
+ show a cloud icon next to the document name if it is stored in the cloud
+ in addition to the name of the current document, remember if it is stored in
the cloud, so that the right document is loaded next time you open the
translator.
Introduced the function
parallelCheck :: [Check a] -> Check [a]
that runs independent checks in parallel, potentially allowing faster grammar
compilation on multi-core computers, if you run gf with +RTS -N.
However, on my dual core laptop, this seems to slow down compilation somewhat
even though CPU utilization goes up as high as 170% at times.
(This is with GF compiled with GHC 7.0.4.)
In GF.Compile.CheckGrammar, use a new topological sorting function that
groups independent judgements, allowing them all to be checked before
continuing or reporting errors.
Reimplemented it with the new function accumulateError.
Also keeping the formatting of errors and warnings unchanged for now, to avoid
potentially causing problems in the GF Eclipse Plugin.
Using accumulated errors in the Check monad.
TODO: some errors are still not accumulated, but thanks to checkMapRecover
at least one error per judgement is reported.
In addition to warnings, the Check monad in GF.Infra.CheckM can now accumulate
errors. There are two new functions
checkAccumError: Message -> Check ()
accumulateError :: (a -> Check a) -> a -> Check a
The former (with the same type as checkWarn) is used to report an accumulated
(nonfatal) error. The latter converts fatal errors into accumulated errors.
Accumulated errors are reported as regular errors by runCheck.
Also, the Check monad type has been made abstract.
This turns error messages like
gf: too few bytes. Failed reading at byte position 1
gf: /some/path/somefile.gfo: too few bytes. Failed reading at byte position 1
but a better fix would be to ignore bad .gfo files and compile from source.
The problem is the way this decision is made in
GF.Compile.ReadFiles.selectFormat...
Also added feedback to source & target language menus to indicate which
languages are supported by the selected translation method. For Apertium, the
source language menu shows all possible source languages, and the target menu
shows the possible target languages for the chosen source language.
The Setup.hs script now queries darcs to create more detailed version info
to include in the startup message.
Note thought that with distributed version control systems like darcs,
the only way to uniquely identify a version is by the set of patches included.
Since the patches are not totally ordered, just looking at the last patch is
not enough.
For official releases, we tag the current set of patches so we can refer to
it by name (e.g. RELEASE-3.3.3).
Note though that the unlexer does the wrong thing with initial words that are
supposed to be capitalized, e.g. "I am ready.", "Spanish wine is good.", so
these sentenses are not translated at the moment.
* Update Setup.hs to build Japanese by default.
* News item about Japanese on the home page.
* Add Japanese to the supported RGL imports in the grammar editor (gfse).
As a temporary workaround, alex is no longer invoked automatically when
building with cabal. Developers who want to modify the lexer need to run
alex on Lexer.x manually and record the modified Lexer.hs.
src/compiler/GF/Grammar/lexer/Lexer.x -- hidden from cabal
src/compiler/GF/Grammar/Lexer.hs -- update it manually
Instead of showing the name of a function in the abstract syntax, linearize it
and show the result. For functions with argument, e.g. That : Kind -> Item,
the function is applied to the right number of placeholder arguments: 'That ?'.
If the linearization fails, the name of the function is shown anyway.
The grammar extension is now done with a regular HTML form, so you can use the
TAB key to move between the fields and press ENTER to submit the extension when
you are done.
TODO: more immediate error feedback
The Haskell Platform page for Mac states that Xcode 3.2 or later is ok, but
there seems to be a problem with Xcode 3.2.2.
Also added some other small clarifications.
The current intput is now represented as an array of words instead of as a
string.
(This is the kind of change is scary to do in a dynamically type language
like JavaScript. In a statically typed language like Haskell you can do it
with confidence, since you know the compiler can help you catch all mistakes...)
If you leave the minibar and later return, the previous input will be
restored. One input string per grammar is rememebered, so you can also switch
back and forth between grammars without losing the input.
This is implemented using localStorage, i.e. the data is stored locally on
the user's device.
Instead of trying to reconstruct the abstract tree from the bracketed string,
use the node numbers (the field called "fid") to identify which node in the
abstact syntax tree a word in the bracketed string corresponds to.
Abstract syntax trees are represented as strings in the web API. To make them
easier to manipulate in JavaScript, the new function converts them to JSON.
To support structural editing, the nodes are numbered in the same way as in
the bracketed string created when linearizing an abstract syntax tree.
Example: "Pred (That Fish) Fresh" is converted to
{fun:"Pred",fid:3,
children:[{fun:"That",fid:1,
children:[{fun:"Fish",fid:0}]},
{fun:"Fresh",fid:2}]}
Removing "ghc-options: -O2" from gf.cabal has the following advantages:
+ Compiling GF is faster (time drops from 182s to 142s on my laptop)
+ Compiling the RGL is faster (time drops from 159s to 155s on my laptop)
+ Without the hardwired optimization level, the 'cabal configure' options
--enable-optimization=<n> and --disable-optimization work as expected
(so if you still want -O2, use --enable-optimization=2)
+ GF can be compiled with ghc-7.2.2 and ghc-7.4.1 (-O2 triggers a bug in
these versions of ghc, it seems. Another workaround, discovered by
Sergei Trofimovich, is to use -O0 in Data.Binary.)
An apparent bug in ghc-7.2.2 causes the type Value to be exported from PGF.Data.
Workaround: restrict the imports from PGF.Data in GF.Command.Abstract and
GF.Compile.GeneratePMCFG to avoid the clash with locally defined type Value.
(ghc-7.0.4 and ghc-7.4.1 appear to be free from this bug.)
Fixes the following error:
src/runtime/haskell/PGF/Expr.hs:111:14:
Ambiguous occurrence `foldl'
It could refer to either `List.foldl',
imported from `Data.List' at src/runtime/haskell/PGF/Expr.hs:27:1-24
(and originally defined in `GHC.List')
or `Map.foldl',
imported from `Data.Map' at src/runtime/haskell/PGF/Expr.hs:28:1-40
You can also add grammars from several directories on the same server of course.
The included minibar configuration file adds the user's own grammars from the
grammar editor.
+ Function div_id was defined in both editor.js and support.js. Solution:
moved the more general version from editor.js to support.js.
+ Function div_class was defined in both editor.js and support.js. Solution:
resolv incompatibility with an argument type test and put the generalized
function in support.js.
Also brought index.txt and index.html back in sync. (It looks like someone
updated index.html instead of updating index.txt and regenerating index.html.)
Check that the reconstructed abstract syntax tree is type correct before
enabling word-for-word replacement. Show an error message if word replacement
nevertheless fails.
To enable customized error handling, the methods in the pgf_online objects and
the AJAX server call functions in support.js accept an error callback function
as an extra argument, in addition to the callback (continuation) for normal
results.
Turn off the highlighting that indicates that word-for-word replacement is
enabled when it is no longer available (e.g if a word is deleted and there no
longer is a complete parse).
It seems that in Gecko browsers, the effect of "with(x) stmts" reaches inside
the bodies of local functions in stmts, but in Webkit browers it does not.
+ Make room for function names in the BracketedString data structure.
+ Fill in function names when linearizing an abstract syntax tree to a
BracketedString.
+ Fill in wildCId when it is not obvious what the function is.
+ Function bracketedLinearize: for compatibility with the other linearization
functions, return Leaf "" instead of error "cannot linearize".
+ Export flattenBracketedString from module PGF.
+ PGFServce: make function names available in the JSON representation of
BracketedString.
To get JSON output, add the parameter format=json. The JSON structure contains
the output from the function PGF.browse.
The default is format=html for backwards compatibility.
"Wiki" is a word with very low information scent: there is no way to know
what type of informaiton is hiding behind the Wiki link, e.g., that there is
a bug tracker (or an FAQ), so not many users will click on it.
+ The flag fastcgi is removed: building the fastcgi is the main purpose of
this cabal file again.
+ pgf-http superseded by gf-server, but is still available as an option
+ Require cgi>=3001.7.3 instead of cgi>=3001.8.0.
Also moved the translation quiz from demos/TransQuiz to src/www/TransQuiz so
that it will be installed by 'cabal install' along with the other files that
are installed for use by gf -server mode.
+ Preserve the startcat flag.
+ Preserve judgement order when possible (GF's parser does not preserve order
and does not record exact source locations, only line numbers)
This is an experimental feature. It requires server support for parsing and is
thus not available while offline, unlike most other editing functionality.
Grammars can now inherit from several other grammars.
Hovering over a catagory name in a function type in the abstract syntax
shows where the category was inherited from.
Two methods used the global variable p instead of declaring a local variable p.
This caused the function p (that constructs HTML <p> elements) in support.js
to suddenly be redefined as a number.
The minibar is now displayed inside the editor, intead of on a separate page.
This makes the context clearer and makes it more obiovous how to return to the
editor.
If there is an error in the grammar, the error message is shown below the
grammar instead.
Also: GFServer.hs now returns compiler output in a JSON structure instead of
as a HTML page.
The link to the minibar (or compiler errors) are now shown below the grammar
on the same page. If you go to the minibar, you only have to press the back
button once to get back to the editor.
Also some output formatting changes in GFServer.hs.
The grammar that the user is currently working is now the one shown initially
in minibar, instead of the first grammar in alphabetical order.
Also GFServer.hs now removes absolute paths to the grammar files on the server in error messages from GF returned to to gfse.
There is a new way to create a new grammar based on an existing grammar:
by extending it. (It is also possible to clone an existing grammar, a before.)
A Russian dictionary generated from a wordlist created by the FreeLing
project. The accompanying converter can be used to convert other wordlists in
EAGLES format to GF grammars.
+ Avoid looping if it is not possible to create a new server directory.
+ Work on FastCGI support using the direct-fastcgi package (commented out
for now because of buggy behavior).
Instead of showing nothing, show disabled buttons and some expanatory text to
provide guidance for users getting started with example-based grammar writing.
+ Changing version numbers and dates here and there.
+ Simplify build-binary-dist.sh since pgf-http need not be built anymore.
+ Use--gf-lib-path to make the sample grammars for minibar compile even if GF
is not installed.
+ Preparations to support more target languages. Tested with Swedish.
+ Don't enable example-based editing if there is less that two concrete
languages in the grammar.
+ Don't show the "By example" button until the required lincat has been defined.
+ Remember the chosen language for examples.
Use the command "dist/build/gf/gf" instead of "gf" to compile the sample
grammars for the minibar, to avoid failing if gf is installed in a directory
which is not in the $PATH.
"gf -server" mode now contains everything needed to run the minibar and
the grammar editor (including example-based grammar writing).
The Setup.hs script installs the required files where gf -server can find them.
These files have been moved to a new directory: src/www.
The separate server program pgf-http is now obsolete.
"gf -server" mode now includes PGF service and the services to support
example-based grammar writing. (But gf -server is not quite ready to replace
pgf-http yet...)
Also bumped GF version number to 3.2.10-darcs
The editor needs to keep track of both the raw term and the nice term returned
by exprToAPI. (Manually constructed linearization rules will now have the
raw term and can not be tested.)
Also replace metavariables in generalized terms with the apropriate parameter
from the linearization rule.
Also fix communication problems caused by inconsistent use of show/read vs
showExpr/readExpr.
This reverts the previous change. Not preprocessing opers turns out to make a
difference in what needs to be mentioned in restricted inheritance/imports.
This also allows the parameter rec to be removed from function computeTermOpt.
(The change is made in GF.Compile.Compute.ConcreteLazy, but not in
GF.Compile.Compute.ConcreteStrict.)
This patch adds GF.Compile.Compute.ConcreteLazy, which replaces the Err monad
with the Identity monad. While the Err monad makes the interpreter
(hyper)strict, the Identity monad let's the interpreter inherit Haskell's
laziness. This can give big speedups: from 50s to 1s in one example,
from ~4 minutes to ~2 minutes for the RGL.
This is still experimental and might be buggy, so it is off by default.
You can turn it on by configuring with the -fcclazy flag, e.g.
cabal configure -fcclazy
Let me know if anything breaks.
It is needed by greatestResource (and similar functions, presumably).
So keep both the list and the finite map of modules. This slows down some
things, but the compilation of PhrasebookFin.pgf benefits from it.
To be continued...
This speeds up the compilation of PhrasebookFin.pgf by 12%, mosly by speeding
up calls to lookupModule in calls from lookupParamValues, in calls
from allParamValues.
The invariant "modules are stored in dependency order" is no longer respected!
But the type MGrammar is now abstract, making it easier to maintain this or
other invariants in the future.
+ Avoids some code duplication by combinging lookupModule and lookupIdentInfo.
+ Also removed lookupIdentInfo from export list, since it is not used anywhere
else.
This was a bug in my workaround for a bug in the httpd-shed package. It
made it impossible to use the glue token "&+" for Turkish input in the minibar,
for example.
concrete language whose name ends with LaTeX.
This change also avoids duplicating output and, in addition to the linearize
command, applies the transfer also when using the linearizeAll command.
If you press Enter, the current word will be accepted, even if there are no
matching completions.
(You can now use names of people when constructing sentences in the Letter
grammar, for example.)
For lin and lincats with empty RHSs, there was nothing to point to to make the
edit button (the "%" button) appear in browsers based on Webkit
(Safari, Chrome).
This file exports the same PGF runtime interface as pgf_online.js but calls
the hs2js translation of the PGF runtime library (which is not included
in darcs).
This is just to make it easier to tell who has the latest version at the
summer school. There has been some bug fixes in gf itself and some significant
additions to the RGL since 3.2 was released.
Also explicitly require Alex 2.x. The recently released Alex 3.0 has backwards
incompatible changes and does not work. GF/Grammar/Lexer.x should probably
be modified to support both Alex 2.x and Alex 3.0...
Two smaller objects have been factored out from the Minibar object:
Input and Translations. These have been placed in two separate files:
minibar_input.js and minibar_translations.js. Some common auxiliary functions
have also been moved to a separate file: minibar_support.js
pgf_online.js has been simplified and generalized to support the full
GF Web Service API. The changes are backwards incompatible, unfortunately.
The documentation and minibar.js have been updated accordingly.
The editor now calls the GF server to check the syntax of expressions that
are part of concrete syntax (except for parameter types).
This is currently done by using the cc command of the GF shell, which puts
some unnecessary restricitons on operation definitions...
Factor out functions that interface to the old cloud service (upload.cgi)
from editor.js into cloud.js. Merge changes from editor2.js into editor.js and
factor out functions that interface to gf -server into clouds2.js.
Cloud merging: when sharing grammars between two devices that previously each
had separate grammar directoies on the server, clean up one of the directories
and make it a symbolic link to the other.
This lets the user access the same set of grammars from multiple devices.
Sharing grammars between multiple users is possible but discouraged at the
moment. There is no version handling, so concurrent editing of the same grammar
by different users might result in one user overwriting changes made by
another user. (The same goes for cuncurrent editing on multiple devices by
a single user, of course.)
pgf-service now appears to works with Apache mod_fastcgi (under Mac OS X) in
addition to lighttpd.
One difference appears to be that while lighttpd passes the name of the PGF
file in the SCRIPT_FILENAME variable, Apache passes it in PATH_TRANSLATED.
But perhaps this depends on exactly how the fastcgi service is configured.
The following is what I added to the Apache configuration file to test this:
<Directory /Library/WebServer/Documents/fcgi-bin>
SetHandler fastcgi-script
Options +ExecCGI
</Directory>
Action pgf-service /fcgi-bin/pgf-service
AddHandler pgf-service .pgf
* Use cabal install -ffastcgi to compile the fastcgi server.
* Use cabal install -fcontent to compile the content service (off by default,
currently broken)
* Use both flag to compile both.
This quick fix should make Ctrl-C in the GF shell behave more like it does in
other shells: even if no command is running, Ctrl-C now just gives you a new
prompt instead of terminating the shell.
Work in progress on making the online grammar editor use the new "gf -server"
mode instead of the old upload.cgi script. A first benefit is that the
editor now calls the server to check the syntax of lin & lincat definitions.
(But the links to minibar/gfshell/quiz are not available at the moment.)
The command "gf -server" now starts a simple HTTP server on port 41295,
providing a simple web API to the GF compiler. It currently support the
follwing operations:
* creating new temporary directories for grammar uploads,
* uploading grammars files for use in the GF shell,
* executing GF shell commands, and
* accessing static files.
This means that GF now depends on some additional networking related packages,
but they should be available and easy to install on all platforms. There is
also a new configuration flag "server" in gf.cabal, so GF will be compiled
without support for server mode if the extra packages are unavailable.
Note that running gf -server while connected to the internet can be a security
risk. To prevent unauthorized access to the rest of the system, it is
advisable to run the server in GF_RESTRICTED mode and as a user with suitably
restricted file permissions.
The GF shell welcome message will now include something like
This is GF version 3.2.1-darcs.
Built on darwin/i386 with ghc-7.0, flags: interrupt
where a list of enabled configuration flags are included on the last line.
This is implemented in GF.Infra.BuildInfo by consulting the CPP macros
defined by the respective option in gf.cabal. For this to work,
GF.Infra.BuildInfo obviously has to be updated when new flags are added to
gf.cabal or if the CPP macros are renamed.
(Also, I suspect that if you reconfigure with different flags and rebuild GF
without cleaning first, the BuildInfo module will not be recompiled,
resulting in a misleading welcome message...)
On touch-based devices, the on-screen keyboard kept popping up after every word,
which was very annoying if you were entering a sentence by tapping on the
magnets.
When adding a new concrete syntax to a grammar, the currently open concrete
syntax will be copied. If the abstract syntax is currently open, the new
concrete syntax will start out empty.
By setting the environment variable GF_RESTRICTED before starting GF, the shell
will be run in restricted mode. This will prevent the GF shell from starting
arbitrary system commands (most uses of System.Cmd.system are blocked) and
writing arbitrary files (most commands that use writeFile et al are blocked).
Restricted mode is intended minimize the potential security risks involved
in allowing public access to the GF shell over the internet. It should be used
in conjuction with system level protection mechanisms (e.g. file permissions)
to make sure that a publicly acessible GF shell does not give access to parts
of the system that should not be publicly accessible.
I changed the version number to 3.2.1-darcs.
I think the version number of the current version from darcs should always
include the "-darcs" suffix (or "-current" perhaps?). The suffix should be
removed only momentarily when making an official release.
This means that the relative grammar directory location in the default
configuration will result in AJAX requests rather than the less efficient
JSONP requests.
For GF 3.10, the Core and RGL repositories had already been separated, however
the binary packages still included the RGL. `GF-3.10` is a tag that was created
in both repositories ([gf-core](https://github.com/GrammaticalFramework/gf-core/releases/tag/GF-3.10) and [gf-rgl](https://github.com/GrammaticalFramework/gf-rgl/releases/tag/GF-3.10)) to indicate which versions of each went into the binaries.
pick' i = return $ \ vs -> maybe (err i vs) ok (pick i vs)
err i vs = bug $ "Stack problem: "++showIdent x++": "
++unwords (map showIdent (local env))
++" => "++show (i,length vs)
resource env (m,c) =
where e = fail $ "Not found: "++render m++"."++showIdent c
extR t vv =
(VRecType rs1, VRecType rs2) ->
case intersect (map fst rs1) (map fst rs2) of
[] -> VRecType (rs1 ++ rs2)
ls -> error $ "clash"<+>show ls
(v1,v2) -> error $ "not records" $$ show v1 $$ show v2
where
error explain = ppbug $ "The term" <+> t
<+> "is not reducible" $$ explain
glue env (v1,v2) = glu v1 v2
ppL loc (hang "unsupported token gluing:" 4
(Glue (vt v1) (vt v2)))
strsFromValue :: Value -> Err [Str]
_ -> fail ("cannot get Str from value " ++ show t)
match loc cs v =
case value2term loc [] v of
Left i -> bad ("variable #"++show i++" is out of scope")
Right t -> err bad return (matchPattern cs t)
where
bad = fail . ("In pattern matching: "++)
inlinePattMacro p =
VPatt p' -> inlinePattMacro p'
_ -> ppbug $ hang "Expected pattern macro:" 4
linPattVars p =
if null dups
then return pvs
else fail.render $ hang "Pattern is not linear:" 4 (ppPatt Unqualified 0 p)
---------------------------------------------
Compile.Compute.Abstract.hs
---------------------------------------------
PGF.Linearize.hs
bracketedLinearize :: PGF -> Language -> Tree -> [BracketedString]
cnc = lookMap (error "no lang") lang (concretes pgf)
---------------------------------------------
PGF.TypeCheck.hs
ppTcError :: TcError -> Doc
ppTcError (UnknownCat cat) = text "Category" <+> ppCId cat <+> text "is not in scope"
ppTcError (UnknownFun fun) = text "Function" <+> ppCId fun <+> text "is not in scope"
ppTcError (WrongCatArgs xs ty cat m n) = text "Category" <+> ppCId cat <+> text "should have" <+> int m <+> text "argument(s), but has been given" <+> int n $$
text "In the type:" <+> ppType 0 xs ty
ppTcError (TypeMismatch xs e ty1 ty2) = text "Couldn't match expected type" <+> ppType 0 xs ty1 $$
text " against inferred type" <+> ppType 0 xs ty2 $$
text "In the expression:" <+> ppExpr 0 xs e
ppTcError (NotFunType xs e ty) = text "A function type is expected for the expression" <+> ppExpr 0 xs e <+> text "instead of type" <+> ppType 0 xs ty
ppTcError (CannotInferType xs e) = text "Cannot infer the type of expression" <+> ppExpr 0 xs e
ppTcError (UnresolvedMetaVars xs e ms) = text "Meta variable(s)" <+> fsep (List.map ppMeta ms) <+> text "should be resolved" $$
text "in the expression:" <+> ppExpr 0 xs e
ppTcError (UnexpectedImplArg xs e) = braces (ppExpr 0 xs e) <+> text "is implicit argument but not implicit argument is expected here"
ppTcError (UnsolvableGoal xs metaid ty)= text "The goal:" <+> ppMeta metaid <+> colon <+> ppType 0 xs ty $$
<I>Makes a case for using GF in controlled language implementation, illustrated by Attempto Controlled English ported to French, German, and Swedish.</I>
</P>
<P>
A. Ranta. Grammars as Software Libraries.
In Y. Bertot, G. Huet, J-J. Lévy, and G. Plotkin (eds.),
<I>Relates GF not only with XML but also with definite clause grammars</I>.
</P>
<P>
P. Mäenpää and A. Ranta.
The type theory and type checker of GF.
<I>Colloquium on Principles, Logics, and Implementations of High-Level Programming Languages, Workshop on Logical Frameworks and Meta-languages, Paris, 28 September 1999</I>.
//An overview of GF for developers and users of translation systems.//
D. Dannélls and J. Camilleri.
Verb Morphology of Hebrew and Maltese - Towards an Open Source Type Theoretical Resource Grammar in GF.
//Proceedings of the Language Resources (LRs) and Human Language Technologies (HLT) for Semitic Languages Status, Updates, and Prospects, LREC-2010 Workshop//,
//Makes a case for using GF in controlled language implementation, illustrated by Attempto Controlled English ported to French, German, and Swedish.//
//Makes a case for using GF in controlled language implementation, illustrated by Attempto Controlled English ported to French, German, and Swedish; longer version in 2010.//
A. Ranta. Grammars as Software Libraries.
In Y. Bertot, G. Huet, J-J. Lévy, and G. Plotkin (eds.),
In Y. Bertot, G. Huet, J-J. Lévy, and G. Plotkin (eds.),
//Relates GF not only with XML but also with definite clause grammars//.
P. Mäenpää and A. Ranta.
P. Mäenpää and A. Ranta.
The type theory and type checker of GF.
//Colloquium on Principles, Logics, and Implementations of High-Level Programming Languages, Workshop on Logical Frameworks and Meta-languages, Paris, 28 September 1999//.
Again after the installation check that the tools are available from the terminal.
If they are not then probably you have to update the current search path in your system.
</P>
<P>
It is also a good idea to have either readline, editline or haskeline installed.
This are libraries for user friendly command line editing. On Linux, without some of this
libraries, the command line editor is very basic. Actually the only key for editing
that you can use is backspace. On Windows you get much more user friendly editor
by default but with it you cannot use the GF specific tab completion. In any case if you
plan to use GF for continuous development then it is recomended to install
some of these libraries. The GF configuration script checks the libraries
in the following order:
</P>
<UL>
<LI>haskeline
<LI>readline
<LI>editline
</UL>
<P>
the first that is found will be used in the compilation. The libraries are also written
in Haskell and could be found on Hackage: <AHREF="http://hackage.haskell.org/packages/archive/pkg-list.html">http://hackage.haskell.org/packages/archive/pkg-list.html</A>. If you want to check whether,
you already have some of these you can use the following command:
</P>
<PRE>
$ ghc-pkg list
</PRE>
<P>
which shows the list of all installed libraries.
</P>
<P>
Haskeline is the easiest to install because it is a pure Haskell library but currently
with this editor GF doesn't provide word completion. With editline we provide word completion
but the library is harder to install because it is a Haskell binding to a
library with the same name written in C. If you do not have the C library you will have to install
it first. Unfortunately editline does not have good support for Unicode. This will be a problem
if you tend to work on non-Latin language. Finaly readline supports both word completion
and Unicode. Currently this is the best supported library.
</P>
<P>
Before to get the GF sources you also need Darcs, version 2 or later.
Darcs is a decentralized revision control system,
see <AHREF="http://darcs.net/">http://darcs.net/</A> for more information. There are precompiled packages for many platforms available at
<AHREF="http://darcs.net/DarcsWiki/CategoryBinaries">http://darcs.net/DarcsWiki/CategoryBinaries</A>. There is also source code if you want to compile it yourself. Darcs is
also written in Haskell and so you can use GHC to compile it.
</P>
<ANAME="toc2"></A>
<H1>Getting the sources</H1>
<P>
Once you have all tools in place you can get the GF sources. If you just want to compile and use GF
then it is enough to have read-only access. It is also possible to make changes in the sources but if
you want these changes to be applied back to the main sources you will have to send the changes to us.
If you plan to work continuously on GF then you should consider to get read-write access.
</P>
<ANAME="toc3"></A>
<H2>Read-only access</H2>
<ANAME="toc4"></A>
<H3>Getting a fresh copy for read-only access</H3>
<P>
Anyone can get the latest development version of GF by running (all on one line):
</P>
<PRE>
$ darcs get --lazy --set-scripts-executable http://code.haskell.org/gf/
</PRE>
<P></P>
<P>
This will create a directory called <CODE>gf</CODE> in the current
directory.
</P>
<ANAME="toc5"></A>
<H3>Updating your copy</H3>
<P>
To get all new patches from the main repo:
</P>
<PRE>
$ darcs pull -a
</PRE>
<P>
This can be done anywhere in your local repository, i.e. in the <CODE>gf</CODE>
directory, or any of its subdirectories.
Without <CODE>-a</CODE>, you can choose which patches you want to get.
</P>
<ANAME="toc6"></A>
<H3>Recording local changes</H3>
<P>
Since every copy is a repository, you can have local version control
of your changes.
</P>
<P>
If you have added files, you first need to tell your local repository to
keep them under revision control:
</P>
<PRE>
$ darcs add file1 file2 ...
</PRE>
<P></P>
<P>
To record changes, use:
</P>
<PRE>
$ darcs record
</PRE>
<P></P>
<P>
This creates a patch against the previous version and stores it in your
local repository. You can record any number of changes before
pushing them to the main repo. In fact, you don't have to push them at
all if you want to keep the changes only in your local repo.
</P>
<P>
If you think there are too many questions about what to record, you
can use the <CODE>-a</CODE> flag to <CODE>record</CODE>. Or answer <CODE>a</CODE> to the first
question. Both of these record all the changes you have in your local
repository.
</P>
<ANAME="toc7"></A>
<H3>Submitting patches</H3>
<P>
If you are using read-only access, send your patches by email to
someone with write-access. First record your changes in your local
repository, as described above. You can send any number of recorded
patches as one patch bundle. You create the patch bundle with:
</P>
<PRE>
$ darcs send -o mypatch.patch
$ gzip mypatch.patch
</PRE>
<P></P>
<P>
(where <CODE>mypatch</CODE> is hopefully replaced by a slightly more
descriptive name). Since some e-mail setups change text attachments
(most likely by changing the newline characters) you need to send
the patch in some compressed format, such as GZIP, BZIP2 or ZIP.
</P>
<P>
Send it as an e-mail attachment. If you have
sendmail or something equivalent installed, it is possible to send the
patch directly from darcs. If so, replace <CODE>-o mypatch.patch</CODE> with
<CODE>--to=EMAIL</CODE> where <CODE>EMAIL</CODE> is the address to send it to.
</P>
<ANAME="toc8"></A>
<H2>Read-write access</H2>
<P>
If you have a user account on code.haskell.org, you can get read-write access over SSH
to the GF repository.
To get an account, <AHREF="http://community.haskell.org/admin/account_request.html">fill out this form</A>.
Once you have an account, ask <<AHREF="mailto:aarne@chalmers.se">aarne@chalmers.se</A>> to add you to the <CODE>GF</CODE> project.
</P>
<ANAME="toc9"></A>
<H3>Getting a fresh copy</H3>
<P>
Get your copy with (all on one line),
replacing <CODE>bringert</CODE> with your own username on code.haskell.org:
</P>
<PRE>
$ darcs get --lazy --set-scripts-executable bringert@code.haskell.org:/srv/code/gf
</PRE>
<P></P>
<P>
The option <CODE>--lazy</CODE> means that darcs defers downloading all the
history for the repository. This saves space, bandwidth and CPU time,
and most people don't need the full history of all changes in the
past.
</P>
<ANAME="toc10"></A>
<H3>Getting other people's changes?</H3>
<P>
Get all new patches from the main repo:
</P>
<PRE>
$ darcs pull -a
</PRE>
<P></P>
<P>
Without <CODE>-a</CODE>, you can choose which patches you want to get.
</P>
<ANAME="toc11"></A>
<H3>Commit your changes</H3>
<P>
There are two steps to commiting a change to the main repo. First you
have to record the changes that you want to commit, then you push them
to the main repo. For instructions on recording your changes locally,
see "Recording local changes" above. Then you can push the patch(es) to
the main repo. If you are using ssh-access, all you need to do is:
</P>
<PRE>
$ darcs push
</PRE>
<P></P>
<P>
If you use the <CODE>-a</CODE> flag to push, all local patches which are not in
the main repo are pushed.
</P>
<ANAME="toc12"></A>
<H3>Apply a patch from someone else</H3>
<P>
Use:
</P>
<PRE>
$ darcs apply < mypatch.patch
</PRE>
<P></P>
<P>
This applies the patch to your local repository. To commit it to the
main repo, use <CODE>darcs push</CODE>.
</P>
<ANAME="toc13"></A>
<H2>Further information about Darcs</H2>
<P>
For more info about what you can do with darcs, see <AHREF="http://darcs.net/manual/">http://darcs.net/manual/</A>
</P>
<ANAME="toc14"></A>
<H1>Compilation from sources</H1>
<P>
The build system of GF is based on Cabal (see <AHREF="http://www.haskell.org/cabal/">http://www.haskell.org/cabal/</A> for more information).
Cabal is installed by default together with the GHC compiler. This is actually a library which could
be used from Haskell to compile projects written in Haskell. The entry point is a script
called Setup.hs which is placed in the top directory of every project managed with Cabal.
The three main steps that are needed for compilation are much like what you do in a project
written in C, you have: configure, build and install.
</P>
<ANAME="toc15"></A>
<H2>Configure</H2>
<P>
During the configuration phase Cabal will check that you have all necessary tools and libraries
needed for GF. The configuration is started by the command:
</P>
<PRE>
$ runghc Setup.hs configure
</PRE>
<P>
The command <CODE>`runghc`</CODE> comes with the GHC compiler and is batch interpreter which executes
the specified script without the need to compile it advance. Setup.hs is our compilation driver
which is based on Cabal. If you don't see any error message from the above command then
you have everything that is needed for GF. You can also add the option <CODE>`-v`</CODE> to see
more details about the configuration.
</P>
<ANAME="toc16"></A>
<H2>Build</H2>
<P>
The build phase does two things. First it builds the GF compiler from the Haskell sources
and after that it builds the GF Resource Grammar Library using the already build compiler.
The simplest command is:
</P>
<PRE>
$ runghc Setup.hs build
</PRE>
<P>
Again you can add the option <CODE>`-v`</CODE> if you want to see more details.
</P>
<P>
Sometimes you just want to work on the GF compiler and don't want to recompile the resource
library after each change. In this case use this extended command:
</P>
<PRE>
$ runghc Setup.hs build rgl-none
</PRE>
<P>
The resource library could also be compiled in two modes: with present tense only and
with all tenses. By default it is compiled with all tenses. If you want to use
the library with only present tense you can compile it in this special mode with
the command:
</P>
<PRE>
$ runghc Setup.hs build present
</PRE>
<P>
Before to use this command make sure that the script lib/src/mkPresent has executable
permissions on Linux.
</P>
<P>
You could also control which languages you want to be recompiled by adding the option
<CODE>`langs=list`</CODE>. For example the following command will compile only the English and the Swedish
language:
</P>
<PRE>
$ runghc Setup.hs build langs=Eng,Swe
</PRE>
<P></P>
<ANAME="toc17"></A>
<H2>Install</H2>
<P>
After you have compiled GF you can install the binaries to make the system usable.
On Linux you will need root privileges to do this. Use the command:
</P>
<PRE>
$ su
</PRE>
<P>
and enter the root password. This step should be skipped on Windows.
</P>
<P>
The installation itself is started with the command:
</P>
<PRE>
$ runghc Setup.hs install
</PRE>
<P>
This command installs the GF compiler in the default place for executable
files in your system. For example on Linux this is usualy /usr/local/bin and on
Windows this is c:\Program Files\Haskell\bin. If you want to install in some
other place then use the <CODE>`--prefix`</CODE> option during the configuration phase.
</P>
<P>
The compiled GF Resource Grammar Library will be installed in /usr/local/share/gf-3.1/lib
on Linux and in c:\Program Files\Haskell\gf-3.1\lib on Windows. Again the location could
be changed using the <CODE>`--prefix`</CODE> option.
</P>
<ANAME="toc18"></A>
<H2>Clean</H2>
<P>
Sometimes you want to clean up the compilation and start again from clean
sources. Use the clean command for this purpose:
</P>
<PRE>
$ runghc Setup.hs clean
</PRE>
<P></P>
<ANAME="toc19"></A>
<H2>SDist</H2>
<P>
You can use the command:
</P>
<PRE>
$ runghc Setup.hs sdist
</PRE>
<P>
to prepare archive with all source codes needed to compile GF.
</P>
<ANAME="toc20"></A>
<H1>Compilation with make</H1>
<P>
If you feel more comfortable with Makefiles then there is a thin Makefile
wrapper arround Cabal for you. If you just type:
</P>
<PRE>
$ make
</PRE>
<P>
the configuration phase will be run automatically if needed and after that
the sources will be compiled. If you don't want to compile the resource library
every time then you can use:
</P>
<PRE>
$ make gf
</PRE>
<P>
For installation use:
</P>
<PRE>
$ make install
</PRE>
<P>
For cleaning:
</P>
<PRE>
$ make clean
</PRE>
<P>
and to build source distribution archive run:
</P>
<PRE>
$ make sdist
</PRE>
<P></P>
<ANAME="toc21"></A>
<H1>Running the testsuite</H1>
<P>
GF has testsuite. It is run with the following command:
</P>
<PRE>
$ runghc Setup.hs test
</PRE>
<P>
The testsuite architecture for GF is very simple but still very flexible.
GF by itself is an interpreter and could execute commands in batch mode.
This is everything that we need to organize a testsuite. The root of the
testsuite is the testsuite/ directory. It contains subdirectories which
(where ``mypatch`` is hopefully replaced by a slightly more
descriptive name). Since some e-mail setups change text attachments
(most likely by changing the newline characters) you need to send
the patch in some compressed format, such as GZIP, BZIP2 or ZIP.
Send it as an e-mail attachment. If you have
sendmail or something equivalent installed, it is possible to send the
patch directly from darcs. If so, replace ``-o mypatch.patch`` with
``--to=EMAIL`` where ``EMAIL`` is the address to send it to.
== Read-write access ==
If you have a user account on code.haskell.org, you can get read-write access over SSH
to the GF repository.
To get an account, [fill out this form http://community.haskell.org/admin/account_request.html].
Once you have an account, ask <aarne@chalmers.se> to add you to the ``GF`` project.
=== Getting a fresh copy ===
Get your copy with (all on one line),
replacing ``bringert`` with your own username on code.haskell.org:
Then you can get the latest updates by running the following:
```
$ darcs get --lazy --set-scripts-executable bringert@code.haskell.org:/srv/code/gf
$ git pull upstream master
```
The option ``--lazy`` means that darcs defers downloading all the
history for the repository. This saves space, bandwidth and CPU time,
and most people don't need the full history of all changes in the
past.
+ **Recording local changes —**
See Git tutorial on how to [record and push your changes https://git-scm.com/book/en/v2/Git-Basics-Recording-Changes-to-the-Repository] to your fork.
+ **Pull request —**
When you want to contribute your changes to the main gf-core repository,
[create a pull request https://docs.github.com/en/github/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-a-pull-request]
from your fork.
=== Getting other people's changes? ===
Get all new patches from the main repo:
If you want to contribute to the RGL as well, do the same process for the RGL repository.
== Compilation from source ==
By now you should have installed Stack and Haskeline, and cloned the Git repository on your own computer, in a directory called ``gf-core``.
=== Primary recommendation: use Stack ===
Open a terminal, go to the top directory (``gf-core``), and type the following command.
```
$ darcs pull -a
$ stack install
```
Without ``-a``, you can choose which patches you want to get.
It will install GF and all necessary tools and libraries to do that.
=== Alternative: use Cabal ===
You can also install GF using Cabal, if you prefer Cabal to Stack. In that case, you may need to install some prerequisites yourself.
=== Commit your changes ===
There are two steps to commiting a change to the main repo. First you
have to record the changes that you want to commit, then you push them
to the main repo. For instructions on recording your changes locally,
see "Recording local changes" above. Then you can push the patch(es) to
the main repo. If you are using ssh-access, all you need to do is:
The actual installation process is similar to Stack: open a terminal, go to the top directory (``gf-core``), and type the following command.
```
$ darcs push
$ cabal install
```
If you use the ``-a`` flag to push, all local patches which are not in
the main repo are pushed.
//The old (potentially outdated) instructions for Cabal are moved to a [separate page ../doc/gf-developers-old-cabal.html]. If you run into trouble with ``cabal install``, you may want to take a look.//
== Compiling GF with C runtime system support ==
The C runtime system is a separate implementation of the PGF runtime services.
It makes it possible to work with very large, ambiguous grammars, using
probabilistic models to obtain probable parses. The C runtime system might
also be easier to use than the Haskell runtime system on certain platforms,
e.g. Android and iOS.
To install the C runtime system, go to the ``src/runtime/c`` directory.
- **On Linux and Mac OS —**
You should have autoconf, automake, libtool and make.
If you are missing some of them, follow the
instructions in the [INSTALL https://github.com/GrammaticalFramework/gf-core/blob/master/src/runtime/c/INSTALL] file.
Once you have the required libraries, the easiest way to install the C runtime is to use the ``install.sh`` script. Just type
``$ bash install.sh``
This will install the C header files and libraries need to write C programs
that use PGF grammars.
% If this doesn't work for you, follow the manual instructions in the [INSTALL https://github.com/GrammaticalFramework/gf-core/blob/master/src/runtime/c/INSTALL] file under your operating system.
- **On other operating systems —** Follow the instructions in the
[INSTALL https://github.com/GrammaticalFramework/gf-core/blob/master/src/runtime/c/INSTALL] file under your operating system.
=== Apply a patch from someone else ===
Depending on what you want to do with the C runtime, you can follow one or more of the following steps.
Use:
=== Use the C runtime from another programming language ===[bindings]
% **If you just want to use the C runtime from Python, Java, or Haskell, you don't need to change your GF installation.**
- **What —**
This is the most common use case for the C runtime: compile
your GF grammars into PGF with the standard GF executable,
and manipulate the PGFs from another programming language,
using the bindings to the C runtime.
- **How —**
The Python, Java and Haskell bindings are found in the
respecively. Compile them by following the instructions
in the ``INSTALL`` or ``README`` files in those directories.
The Python library can also be installed from PyPI using ``pip install pgf``.
//If you are on Mac and get an error about ``clang`` version, you can try some of [these solutions https://stackoverflow.com/questions/63972113/big-sur-clang-invalid-version-error-due-to-macosx-deployment-target]—but be careful before removing any existing installations.//
=== Use GF shell with C runtime support ===
- **What —**
If you want to use the GF shell with C runtime functionalities, then you need to (re)compile GF with special flags.
The GF shell can be started with ``gf -cshell`` or ``gf -crun`` to use
the C run-time system instead of the Haskell run-time system.
Only limited functionality is available when running the shell in these
modes (use the ``help`` command in the shell for details).
(Re)compiling your GF with these flags will also give you
Haskell bindings to the C runtime, as a library called ``PGF2``,
but if you want Python or Java bindings, you need to do [the previous step #bindings].
% ``PGF2``: a module to import in Haskell programs, providing a binding to the C run-time system.
- **How —**
If you use cabal, run the following command:
```
$ darcs apply < mypatch.patch
cabal install -fc-runtime
```
This applies the patch to your local repository. To commit it to the
main repo, use ``darcs push``.
from the top directory (``gf-core``).
== Further information about Darcs ==
If you use stack, uncomment the following lines in the ``stack.yaml`` file:
For more info about what you can do with darcs, see http://darcs.net/manual/
= Compilation from sources =
The build system of GF is based on Cabal (see http://www.haskell.org/cabal/ for more information).
Cabal is installed by default together with the GHC compiler. This is actually a library which could
be used from Haskell to compile projects written in Haskell. The entry point is a script
called Setup.hs which is placed in the top directory of every project managed with Cabal.
The three main steps that are needed for compilation are much like what you do in a project
written in C, you have: configure, build and install.
== Configure ==
During the configuration phase Cabal will check that you have all necessary tools and libraries
needed for GF. The configuration is started by the command:
```
$ runghc Setup.hs configure
flags:
gf:
c-runtime: true
extra-lib-dirs:
- /usr/local/lib
```
The command ```runghc``` comes with the GHC compiler and is batch interpreter which executes
the specified script without the need to compile it advance. Setup.hs is our compilation driver
which is based on Cabal. If you don't see any error message from the above command then
you have everything that is needed for GF. You can also add the option ```-v``` to see
more details about the configuration.
and then run ``stack install`` from the top directory (``gf-core``).
== Build ==
The build phase does two things. First it builds the GF compiler from the Haskell sources
and after that it builds the GF Resource Grammar Library using the already build compiler.
The simplest command is:
```
$ runghc Setup.hs build
```
Again you can add the option ```-v``` if you want to see more details.
//If you get an "``error while loading shared libraries``" when trying to run GF with C runtime, remember to declare your ``LD_LIBRARY_PATH``.//
//Add ``export LD_LIBRARY_PATH="/usr/local/lib"`` to either your ``.bashrc`` or ``.profile``. You should now be able to start GF with C runtime.//
Sometimes you just want to work on the GF compiler and don't want to recompile the resource
library after each change. In this case use this extended command:
```
$ runghc Setup.hs build rgl-none
```
The resource library could also be compiled in two modes: with present tense only and
with all tenses. By default it is compiled with all tenses. If you want to use
the library with only present tense you can compile it in this special mode with
the command:
```
$ runghc Setup.hs build present
```
Before to use this command make sure that the script lib/src/mkPresent has executable
permissions on Linux.
You could also control which languages you want to be recompiled by adding the option
```langs=list```. For example the following command will compile only the English and the Swedish
language:
=== Use GF server mode with C runtime ===
- **What —**
With this feature, ``gf -server`` mode is extended with new requests to call the C run-time
system, e.g. ``c-parse``, ``c-linearize`` and ``c-translate``.
- **How —**
If you use cabal, run the following command:
```
$ runghc Setup.hs build langs=Eng,Swe
cabal install -fc-runtime -fserver
```
from the top directory.
If you use stack, add the following lines in the ``stack.yaml`` file:
```
flags:
gf:
c-runtime: true
server: true
extra-lib-dirs:
- /usr/local/lib
```
== Install ==
and then run ``stack install``, also from the top directory.
After you have compiled GF you can install the binaries to make the system usable.
On Linux you will need root privileges to do this. Use the command:
```
$ su
```
and enter the root password. This step should be skipped on Windows.
The installation itself is started with the command:
```
$ runghc Setup.hs install
```
This command installs the GF compiler in the default place for executable
files in your system. For example on Linux this is usualy /usr/local/bin and on
Windows this is c:\Program Files\Haskell\bin. If you want to install in some
other place then use the ```--prefix``` option during the configuration phase.
The compiled GF Resource Grammar Library will be installed in /usr/local/share/gf-3.1/lib
on Linux and in c:\Program Files\Haskell\gf-3.1\lib on Windows. Again the location could
be changed using the ```--prefix``` option.
== Compilation of RGL ==
== Clean ==
As of 2018-07-26, the RGL is distributed separately from the GF compiler and runtimes.
Sometimes you want to clean up the compilation and start again from clean
sources. Use the clean command for this purpose:
```
$ runghc Setup.hs clean
```
To get the source, follow the previous instructions on [how to clone a repository with Git #getting-source].
== SDist ==
After cloning the RGL, you should have a directory named ``gf-rgl`` on your computer.
You can use the command:
```
$ runghc Setup.hs sdist
```
to prepare archive with all source codes needed to compile GF.
= Compilation with make =
If you feel more comfortable with Makefiles then there is a thin Makefile
wrapper arround Cabal for you. If you just type:
```
$ make
```
the configuration phase will be run automatically if needed and after that
the sources will be compiled. If you don't want to compile the resource library
every time then you can use:
```
$ make gf
```
For installation use:
=== Simple ===
To install the RGL, you can use the following commands from within the ``gf-rgl`` repository:
```
$ make install
```
For cleaning:
There is also ``make build``, ``make copy`` and ``make clean`` which do what you might expect.
=== Advanced ===
For advanced build options, call the Haskell build script directly:
```
$ make clean
$ runghc Setup.hs ...
```
and to build source distribution archive run:
For more details see the [README https://github.com/GrammaticalFramework/gf-rgl/blob/master/README.md].
=== Haskell-free ===
If you do not have Haskell installed, you can use the simple build script ``Setup.sh``
(or ``Setup.bat`` for Windows).
== Creating binary distribution packages ==
The binaries are generated with Github Actions. More details can be viewed here:
searches the subdirectories of the testsuite/ directory for files with extension
.gfs and when it finds one it is executed with the GF interpreter.
The output of the script is stored in file with extension .out and is compared
with the content of the corresponding file with extension .gold, if there is one.
If the contents are identical the command reports that the test was passed successfully.
Otherwise the test had failed.
This is everything that we need to organize a testsuite. The root of the
testsuite is the ``testsuite/`` directory. It contains subdirectories
which themselves contain GF batch files (with extension ``.gfs``).
The above command searches the subdirectories of the ``testsuite/`` directory
for files with extension ``.gfs`` and when it finds one, it is executed with
the GF interpreter. The output of the script is stored in file with extension ``.out``
and is compared with the content of the corresponding file with extension ``.gold``, if there is one.
Every time when you make some changes to GF that have to be tested, instead of
writing the commands by hand in the GF shell, add them to one .gfs file in the testsuite
and run the test. In this way you can use the same test later and we will be sure
that we will not incidentaly break your code later.
Every time when you make some changes to GF that have to be tested,
instead of writing the commands by hand in the GF shell, add them to one ``.gfs``
file in the testsuite subdirectory where its ``.gf`` file resides and run the test.
In this way you can use the same test later and we will be sure that we will not
accidentally break your code later.
**Test Outcome - Passed:** If the contents of the files with the ``.out`` extension
are identical to their correspondingly-named files with the extension ``.gold``,
the command will report that the tests passed successfully, e.g.
If you don't want to run the whole testsuite you can write the path to the subdirectory
in which you are interested. For example:
```
$ runghc Setup.hs test testsuite/compiler
Running 1 test suites...
Test suite gf-tests: RUNNING...
Test suite gf-tests: PASS
1 of 1 test suites (1 of 1 test cases) passed.
```
will run only the testsuite for the compiler.
**Test Outcome - Failed:** If there is a contents mismatch between the files
with the ``.out`` extension and their corresponding files with the extension ``.gold``,
the test diagnostics will show a fail and the areas that failed. e.g.
```
testsuite/compiler/compute/Records.gfs: OK
testsuite/compiler/compute/Variants.gfs: FAIL
testsuite/compiler/params/params.gfs: OK
Test suite gf-tests: FAIL
0 of 1 test suites (0 of 1 test cases) passed.
```
The fail results overview is available in gf-tests.html which shows 4 columns:
+ __Results__ - only areas that fail will appear. (Note: There are 3 failures in the gf-tests.html which are labelled as (expected). These failures should be ignored.)
+ __Input__ - which is the test written in the .gfs file
+ __Gold__ - the expected output from running the test set out in the .gfs file. This column refers to the contents from the .gold extension files.
+ __Output__ - This column refers to the contents from the .out extension files which are generated as test output.
After fixing the areas which fail, rerun the test command. Repeat the entire process of fix-and-test until the test suite passes before submitting a pull request to include your changes.
The GF [YouTube channel](https://www.youtube.com/channel/UCZ96DechSUVcXAhtOId9VVA) keeps a playlist of [all GF videos](https://www.youtube.com/playlist?list=PLrgqBB5thLeT15fUtJ8_Dtk8ppdtH90MK), and more specific playlists for narrower topics.
If you make a video about GF, let us know and we'll add it to the suitable playlist(s)!
- [General introduction to GF](#general-introduction-to-gf)
These videos show how to install GF on your computer (Mac or Windows), and how to play with simple grammars in a [Jupyter notebook](https://github.com/GrammaticalFramework/gf-binder) (any platform, hosted at [mybinder.org](https://mybinder.org)).
These videos show incremental improvements to a [miniature version of the resource grammar](https://github.com/inariksit/comp-syntax-2020/tree/master/lab2/grammar/dummy#readme).
They assume some prior knowledge of GF, roughly lessons 1-3 from the [GF tutorial](http://www.grammaticalframework.org/doc/tutorial/gf-tutorial.html).
- [RGL Synopsis](../lib/doc/synopsis/index.html). Documentation of the Resource Grammar Library, including the syntax API and lexical paradigms for each language.
- [Shell Reference](gf-shell-reference.html). Describes the commands available in the interactive GF shell.
Also summarizes how to run GF as a batch compiler.
- [Developers Guide](gf-developers/html). Detailed information about building and developing GF.
Linux binary, with precompiled grammar libraries. To install:
<pre>
tar xvfz GF-2.8-i386-pc-linux-gnu.tgz
cd GF-2.8-i386-pc-linux-gnu
./configure
make install
</pre>
You may need <tt>sudo make install</tt>; if you cannot write in
<tt>/usr/local/</tt>, you can change the target directory
by <tt>./configure --prefix=MYDIR</tt>.
<li><ahref="download/GF-2.8-i386-apple-darwin8.9.1.tgz"><tt>GF-2.8-i386-apple-darwin8.9.1.tgz</tt></a>, Mac OS X binary (Intel), with precompiled grammar libraries. Requires Readline from
<ahref="http://www.macports.org/">MacPorts</a>. To install, see above item.
<li><ahref="download/GF-2.8-windows-cygwin.tgz"><tt>GF-2.8-windows-cygwin.tgz</tt></a>, Windows Cygwin binary. Requires
<ahref="http://www.cygwin.com/">Cygwin</a>. To install, see above item.
Don't forget that you can always get the current sources from the
<ahref="http://www.cs.chalmers.se/Cs/Research/Language-technology/darcs/GF/doc/darcs.html">GF Darcs Repository</a>. First time you do
<pre>
darcs get --partial --set-scripts-executable http://www.cs.chalmers.se/Cs/Research/Language-technology/darcs/GF/
</pre>
and subsequent times
<pre>
darcs pull -a
</pre>
Then compile and install as described in
<ahref="src/INSTALL"><tt>GF/src/INSTALL</tt></a>.
</body>
</html>
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
