Commit Graph

30 Commits

Author SHA1 Message Date
kr.angelov
4efb2d5a22 Prelude.CAPIT is now a built-in primitive. It still generates &| in the Haskell runtime but will be intepreted in the C runtime 2014-10-09 19:34:12 +00:00
kr.angelov
03b067782c a partial support for def rules in the C runtime
The def rules are now compiled to byte code by the compiler and then to
native code by the JIT compiler in the runtime. Not all constructions
are implemented yet. The partial implementation is now in the repository
but it is not activated by default since this requires changes in the
PGF format. I will enable it only after it is complete.
2014-08-11 10:59:10 +00:00
hallgren
cbe5e8ab18 Convert from Text.PrettyPrint to GF.Text.Pretty
All compiler modules now use GF.Text.Pretty instead of Text.PrettyPrint
2014-07-28 11:58:00 +00:00
hallgren
f739841497 PGF library: expose only PGF and PGF.Internal instead of all modules
PGF exports the public, stable API.
PGF.Internal exports additional things needed in the GF compiler & shell,
including the nonstardard version of Data.Binary.
2014-06-12 14:43:18 +00:00
kr.angelov
5f3b35a5f9 refactor the compilation of CFG and EBNF grammars. Now they are parsed by using GF.Grammar.Parser just like the ordinary GF grammars. Furthermore now GF.Speech.CFG is moved to GF.Grammar.CFG. The new module is used by both the speech conversion utils and by the compiler for CFG grammars. The parser for CFG now consumes a lot less memory and can be used with grammars with more than 4 000 000 productions. 2014-03-21 21:25:05 +00:00
kr.angelov
c119627dfc fix the export for CFG grammars with BIND, SOFT_BIND and nonExist 2013-11-22 13:36:14 +00:00
kr.angelov
475f213c99 the content of ParseEngAbs3.probs is now merged with ParseEngAbs.probs. The later is now retrained. Once the grammar is compiled with the .probs file now it doesn't need anything more to do robust parsing. The robustness itself is controlled by the flags 'heuristic_search_factor', 'meta_prob' and 'meta_token_prob' in ParseEngAbs.gf 2013-11-06 10:21:46 +00:00
hallgren
decd7122de Eliminate mutual dependencies between the GF compiler and the PGF library
+ References to modules under src/compiler have been eliminated from the PGF
  library (under src/runtime/haskell). Only two functions had to be moved (from
  GF.Data.Utilities to PGF.Utilities) to make this possible, other apparent
  dependencies turned out to be vacuous.

+ In gf.cabal, the GF executable no longer directly depends on the PGF library
  source directory, but only on the exposed library modules. This means that
  there is less duplication in gf.cabal and that the 30 modules in the
  PGF library will no longer be compiled twice while building GF.

  To make this possible, additional PGF library modules have been exposed, even
  though they should probably be considered for internal use only. They could
  be collected in a PGF.Internal module, or marked as "unstable", to make
  this explicit.

+ Also, by using the -fwarn-unused-imports flag, ~220 redundant imports were
  found and removed, reducing the total number of imports by ~15%.
2013-11-05 13:11:10 +00:00
kr.angelov
efa4bc4d62 a major refactoring in the C and the Haskell runtimes. Note incompatible change in the PGF format!!!
The following are the outcomes:

   - Predef.nonExist is fully supported by both the Haskell and the C runtimes

   - Predef.BIND is now an internal compiler defined token. For now
     it behaves just as usual for the Haskell runtime, i.e. it generates &+.
     However, the special treatment will let us to handle it properly in 
     the C runtime.

   - This required a major change in the PGF format since both 
     nonExist and BIND may appear inside 'pre' and this was not supported
     before.
2013-09-27 15:09:48 +00:00
hallgren
38fe30c610 Make Ident abstract; imports of Data.ByteString.Char8 down from 29 to 16 modules
Most of the explicit uses of ByteStrings were eliminated by using identS,

	identS = identC . BS.pack 

which was found in GF.Grammar.CF and moved to GF.Infra.Ident. The function

	prefixIdent :: String -> Ident -> Ident

allowed one additional import of ByteString to be eliminated. The functions

	isArgIdent :: Ident -> Bool
	getArgIndex :: Ident -> Maybe Int

were needed to eliminate explicit pattern matching on Ident from two modules.
2013-09-19 18:23:47 +00:00
peter.ljunglof
87260365c7 Added an O(n log n) version of nub
The new nub is called nub', and it replaces the old sortNub which was 
not lazy and did not retain the order between the elements.
2012-08-29 21:45:10 +00:00
kr.angelov
fd5220038d A basic infrastructure for generating Teyjus bytecode from the GF abstract syntax 2012-08-29 11:43:02 +00:00
kr.angelov
54019c75ae merge GF.Infra.Modules and GF.Grammar.Grammar. This is a preparation for the separate PGF building 2011-11-02 13:57:11 +00:00
krasimir
1f1a0d7ada PGFToCFG should use the full production set because the lexicon is now split out 2010-12-14 21:25:43 +00:00
krasimir
a9218ff45c simple refactoring in PGF.Macros and related 2010-12-06 14:19:51 +00:00
krasimir
be9ad26aea refactor the API for random generation again. Now PGF contains probabilities in the abstract syntax 2010-10-02 13:03:57 +00:00
krasimir
9aa7c88c5a native representation for HOAS in PMCFG and incremental type checking of the parse forest 2010-08-09 10:10:08 +00:00
krasimir
e92151caf8 redesign the open-literals API 2010-07-01 08:51:59 +00:00
krasimir
97a9ddb4a7 rename isLiteralFCat -> isPredefFId, fcat(String|Int|Float) -> fid(String|Int|Float) 2010-06-30 07:40:22 +00:00
krasimir
00a69cab3b forgot to fix the import in GF.Speech.CFG to get the Token type 2010-06-29 12:58:35 +00:00
krasimir
43736135ca introduce the type synonym Token=String in the PGF API 2010-06-29 09:00:41 +00:00
aarne
cfd3afd2d8 case for pre in CF generation (ignores alternatives so far) 2010-06-03 21:43:35 +00:00
krasimir
ce99935812 now for every category we store, in PGF, the list of functions for it in source-code order. The order matters for the termination of the exhaustive generation with dependent types. 2010-03-24 11:11:55 +00:00
krasimir
3685595ece cleanup the code of the PGF interpreter and polish the binary serialization to match the preliminary specification 2010-01-27 09:39:14 +00:00
krasimir
a039808141 PGF is now real synchronous PMCFG 2010-01-17 21:35:36 +00:00
krasimir
362f333ebd now the linearization is completely based on PMCFG 2010-01-17 17:05:21 +00:00
krasimir
11218cdbca store the label names in PMCFG 2010-01-05 08:35:33 +00:00
krasimir
f841b1ada5 printnames are now kept as String instead of Term in PGF 2010-01-05 07:05:41 +00:00
krasimir
76debee2c1 remove the old parsing code and the -erasing=on flag 2009-12-14 10:54:22 +00:00
krasimir
c92f9d1c0c reorganize the directories under src, and rescue the JavaScript interpreter from deprecated 2009-12-13 18:50:29 +00:00