Commit Graph

162 Commits

Author SHA1 Message Date
hallgren
bbc13e9f0c Faster regular expression pattern matching in the grammar compiler.
The sequence operator (x+y) was implemented by splitting the string to be
matched at all positions and trying to match the parts against the two
subpatterns. To reduce the number of splits, we now estimate the minimum and
maximum length of the string that the subpatterns could match. For common
cases, where one of the subpatterns is a string of known length, like
in (x+"y") or (x + ("a"|"o"|"u"|"e")+"y"), only one split will be tried.
2013-02-27 20:59:43 +00:00
kr.angelov
55203110bb now the beam size for the statistical parser can be configured by using the flag beam_size in the top-level concrete module 2013-02-12 10:53:13 +00:00
hallgren
09fb4cdef0 Better error message for unsupported token gluing
Instead of "Internal error in ...", you now get a proper error message with
a source location and a function name.
2013-01-29 16:25:03 +00:00
hallgren
79795cb0e7 Fix a bug with record extension
Add a conversion rule for ({ l1 = e } ** x).l2 in PMCFG generation. (A rule
for the symmetric case (x ** { l1 = e }).l2 was added some time ago.)
2013-01-29 14:59:16 +00:00
hallgren
764b649959 Better error message for Predef.error
+ Instead of "Internal error in ...", you now get a proper error message with
  a source location and a function name.
+ Also added some missing error value propagation in the partial evaluator.
+ Also some other minor cleanup and error handling fixes.
2013-01-28 16:12:56 +00:00
aarne
6af9575a68 improved error message for overloading in case the given signature looks the same as one of the expected ones: it shows full records rather than just lock fields. 2013-01-28 14:00:23 +00:00
hallgren
3712b6988e partial evaluator: fix token glueing bug
"a"+("b"++"c") was simplified to "bb"++"c" instead of "ab"++c.
2013-01-11 15:14:42 +00:00
hallgren
368cd7ffbe bug fix in the new partial evaluator
It can leave wildcard tables in their origial form, but it easy to handle
them in the unfactor function in GeneratePMCFG.
2012-12-20 16:41:43 +00:00
hallgren
f73825ddf1 partial evaluator bug fix
It failed to delay table selection when the selector contains a run-time
variable, causing "gf: Prelude.(!!): index too large" instead.

Also:
  + Show better source locations on unexpected errors, to aid bug hunting.
  + Removed unused SourceGrammar argument to value2term.
2012-12-19 23:12:37 +00:00
hallgren
4aa3638549 GF.Grammar.Lookup: new function lookupResDefLoc
It's like lookupResDef but it includes a source location in the output.
2012-12-19 23:08:56 +00:00
hallgren
3755ea673a partial evaluator bug fix
Int was missing from the list of predefined canonical constants.
2012-12-18 13:03:20 +00:00
kr.angelov
8aefd1e072 The first prototype for exhaustive generation in the C runtime. The trees are always listed in decreasing probability order. There is also an API for generation from Python 2012-12-14 15:32:49 +00:00
hallgren
79711380a2 Add language extension for ghc<7.4
FlexibleInstances does not imply TypeSynonymInstances, apparently.
2012-12-14 14:21:46 +00:00
hallgren
950832dbba More work on the new partial evaluator
The work done by the partial evaluator is now divied in two stages:
 - A static "term traversal" stage that happens only once per term and uses
   only statically known information. In particular, the values of lambda bound
   variables are unknown during this stage. Some tables are transformed to
   reduce the cost of pattern matching.
 - A dynamic "function application" stage, where function bodies can be
   evaluated repeatedly with different arguments, without the term traversal
   overhead and without recomputing statically known information.

Also the treatment of predefined functions has been reworked to take advantage
of the staging and better handle partial applications.
2012-12-14 14:00:21 +00:00
hallgren
f39466f787 partial evaluator work
* Evaluate operators once, not every time they are looked up
* Remember the list of parameter values instead of recomputing it from the
  pattern type every time a table selection is made.
* Quick fix for partial application of some predefined functions.
2012-12-11 15:37:41 +00:00
hallgren
ab97deae57 Compute.ConcreteNew: add missing case for variant functions
Also adding a test case in the test suite for this.
2012-12-10 13:25:32 +00:00
hallgren
7d0f649f29 Compute.ConcreteNew: bug fix for indirectly defined pattern macros
More changes are probably needed to make pattern macros first class values.
Also includes minor changes related to variants and error messages.
2012-12-06 16:44:03 +00:00
aarne
d34adff894 produce error message instead of failure of irrefutable pattern Ok ty_C in GrammarToPGF, to help find compilation errors; the ones I've found are because an inherited abstract excludes something that the inherited concrete does not exclude. 2012-12-02 19:40:45 +00:00
hallgren
cf00c8bd0b new-comp: rewrite f (x|y) into (f x|f y)
With this change, all languages in molto/mgl/mixture except German and Polish
can be compiled.
2012-11-16 13:47:10 +00:00
hallgren
b6f392b4e1 Adding a new experimental partial evalutator
GF.Compile.Compute.ConcreteNew + two new modules contain a new
partial evaluator intended to solve some performance problems with the old
partial evalutator in GF.Compile.Compute.ConcreteLazy. It has been around for
a while, but is now complete enough to compile the RGL and the Phrasebook.

The old partial evaluator is still used by default. The new one can be activated
in two ways:

  - by using the command line option -new-comp when invoking GF.
  - by using cabal configure -fnew-comp to make -new-comp the default. In this
    case you can also use the command line option -old-comp to revert to the old
    partial evaluator.

In the GF shell, the cc command uses the old evaluator regardless of -new-comp
for now, but you can use "cc -new ..." to invoke the new evaluator.

With -new-comp, computations happen in GF.Compile.GeneratePMCFG instead of
GF.Compile.Optimize. This is implemented by testing the flag optNewComp in
both modules, to omit calls to the old partial evaluator from GF.Compile.Optimize
and add calls to the new partial evaluator in GF.Compile.GeneratePMCFG.
This also means that -new-comp effectively implies -noexpand.

In GF.Compile.CheckGrammar, there is a check that restricted inheritance is used
correctly. However, when -noexpand is used, this check causes unexpected errors,
so it has been converted to generate warnings, for now.

-new-comp no longer enables the new type checker in
GF.Compile.Typeckeck.ConcreteNew.

The GF version number has been bumped to 3.3.10-darcs
2012-11-13 14:09:15 +00:00
hallgren
c2b7288411 Eliminate warnings about deprecated use of catch and try
This is also needed for compatibility with GHC 7.6.
2012-11-08 15:53:46 +00:00
hallgren
b841664a63 Compute.ConcreteNew: support variants
Also add a missing check for Predef values in apply.
2012-10-24 17:49:20 +00:00
hallgren
eed724271f GeneratePMCFG: prefix messages about "impossible" errors with 'Internal error:'
Just to make them easier to spot when wading through thousands of lines of
warnings...
2012-10-24 17:08:52 +00:00
hallgren
d0e1187b10 Refactor compileSourceModule
There was 55 lines of rather repetitive code with calls to 6 compiler passes.
They have been replaced with 19 lines that call the 6 compiler passes
plus 26 lines of helper functions.
2012-10-19 20:14:11 +00:00
hallgren
885aaca6de Consistenly use SourceGrammar instead of [SourceModule] when calling compiler passes 2012-10-19 19:56:00 +00:00
hallgren
1d6cbf8189 Use NOINLINE for build info and darcs version info
... to avoid unnecessary recompilation of other modules.
2012-10-18 20:01:22 +00:00
kr.angelov
dd0e9411e1 catch all case in GenerateBC.genFun 2012-09-18 08:32:21 +00:00
hallgren
95b372b281 Add type info to "Warning: ignoring lock fields in resolving..." 2012-09-06 13:15:48 +00:00
kr.angelov
37e1ced6cf another fix for teyjus 2012-08-30 08:09:30 +00:00
peter.ljunglof
f78505d88e Use nub' instead of nub in some places, remove some unused nub imports 2012-08-29 21:48:34 +00:00
peter.ljunglof
87260365c7 Added an O(n log n) version of nub
The new nub is called nub', and it replaces the old sortNub which was 
not lazy and did not retain the order between the elements.
2012-08-29 21:45:10 +00:00
peter.ljunglof
7ad4616d9c Emacs only recognizes utf-8, not UTF-8, in file headers 2012-08-29 21:40:49 +00:00
kr.angelov
fd5220038d A basic infrastructure for generating Teyjus bytecode from the GF abstract syntax 2012-08-29 11:43:02 +00:00
kr.angelov
b34232c6dc the first draft of GF.Compile.Instructions 2012-08-01 09:21:59 +00:00
peter.ljunglof
2e8d9d3829 Minor changes in Python export 2012-07-02 21:11:20 +00:00
peter.ljunglof
a82a63cc69 Fixed minor bug in prolog export 2012-06-28 07:01:29 +00:00
peter.ljunglof
035b7731e5 major changes to the prolog export 2012-06-27 23:29:05 +00:00
peter.ljunglof
871eb6eabc cleanup in the python export 2012-06-27 23:27:01 +00:00
hallgren
8444821759 Experiment with parallel grammar checks
Introduced the function

	parallelCheck :: [Check a] -> Check [a]

that runs independent checks in parallel, potentially allowing faster grammar
compilation on multi-core computers, if you run gf with +RTS -N.

However, on my dual core laptop, this seems to slow down compilation somewhat
even though CPU utilization goes up as high as 170% at times.
(This is with GF compiled with GHC 7.0.4.)
2012-06-26 17:01:15 +00:00
hallgren
5b577baf02 Report many type errors instead of stopping after the first one
In GF.Compile.CheckGrammar, use a new topological sorting function that
groups independent judgements, allowing them all to be checked before
continuing or reporting errors.
2012-06-26 14:46:18 +00:00
hallgren
abb1aedd3d GF.Compile.Rename: report many errors instead of stopping after the first one
Using accumulated errors in the Check monad.

TODO: some errors are still not accumulated, but thanks to checkMapRecover
at least one error per judgement is reported.
2012-06-25 14:15:02 +00:00
hallgren
ca4091599f Check monad: support for accumulated errors
In addition to warnings, the Check monad in GF.Infra.CheckM can now accumulate
errors. There are two new functions

	checkAccumError: Message -> Check ()
	accumulateError :: (a -> Check a) -> a -> Check a

The former (with the same type as checkWarn) is used to report an accumulated
(nonfatal) error. The latter converts fatal errors into accumulated errors.

Accumulated errors are reported as regular errors by runCheck.

Also, the Check monad type has been made abstract.
2012-06-25 14:01:58 +00:00
peter.ljunglof
a1c16e36b0 Export PGF in Python format 2012-06-25 14:16:24 +00:00
aarne
7f047a0ab5 checkMapRecover: find undefined idents in all jments in Rename 2012-06-25 11:14:52 +00:00
aarne
d59f54ba41 missing case in partial evaluation of + fixed 2012-05-15 11:48:30 +00:00
hallgren
8195c36f6f Workaround for bug in ghc-7.2.2
An apparent bug in ghc-7.2.2 causes the type Value to be exported from PGF.Data.
Workaround: restrict the imports from PGF.Data in GF.Command.Abstract and
GF.Compile.GeneratePMCFG to avoid the clash with locally defined type Value.
(ghc-7.0.4 and ghc-7.4.1 appear to be free from this bug.)
2012-03-26 23:56:56 +00:00
aarne
8b07c2607d bug fix in AppPredefined: don't compare values that contain variables. this should be checked even more generally. 2012-02-28 15:13:32 +00:00
kr.angelov
df0fe4363d buf fix in the compiler for HOAS 2012-02-28 08:10:47 +00:00
aarne
a777481313 the Predef function eqVal to compare equality of parameter values 2012-02-24 10:00:27 +00:00
aarne
8936c6d38e sorted the list of funs and cats in GrammatToPGF to get the predef categories in proper place and get gr, ai, gt work properly 2012-01-14 17:06:54 +00:00