1
0
forked from GitHub/gf-core

Experimental: parallel batch compilation of grammars

On my laptop these changes speed up the full build of the RGL and example
grammars with 'cabal build' from ~95s to ~43s and the zero build from ~18s
to ~5s.

The main change is the introduction of the module GF.CompileInParallel that
replaces GF.Compile and the function GF.Compile.ReadFiles.getAllFiles. At
present, it is activated with the new -j flag, and it is only used when
combined with --make or --batch. In addition, to get parallel computations,
you need to add GHC run-time flags, e.g., +RTS -N -A20M -RTS, to the command
line.

The Setup.hs script has been modified to pass the appropriate flags to GF
for parallel compilation when compiling the RGL and example grammars, but you
need a recent version of Cabal for this to work (probably >=1.20).

Some additonal refactoring were made during this work. A new monad is used to
avoid warnings/error messages from different modules to be intertwined when
compiling in parallel, so some functios that were hardiwred to the IO or IOE
monads have been lifted to work in arbitrary monads that are instances in
the appropriate classes.
This commit is contained in:
hallgren
2014-08-25 09:56:00 +00:00
parent 9253d54b7e
commit d84c5ef171
11 changed files with 420 additions and 178 deletions

View File

@@ -172,8 +172,8 @@ data Flags = Flags {
optTagsOnly :: Bool,
optHeuristicFactor :: Maybe Double,
optMetaProb :: Maybe Double,
optMetaToknProb :: Maybe Double{-,
optNewComp :: Bool-}
optMetaToknProb :: Maybe Double,
optJobs :: Maybe (Maybe String)
}
deriving (Show)
@@ -284,7 +284,8 @@ defaultFlags = Flags {
optTagsOnly = False,
optHeuristicFactor = Nothing,
optMetaProb = Nothing,
optMetaToknProb = Nothing
optMetaToknProb = Nothing,
optJobs = Nothing
}
-- | Option descriptions
@@ -297,6 +298,7 @@ optDescr =
Option ['v'] ["verbose"] (OptArg verbosity "N") "Set verbosity (default 1). -v alone is the same as -v 2.",
Option ['q','s'] ["quiet"] (NoArg (verbosity (Just "0"))) "Quiet, same as -v 0.",
Option [] ["batch"] (NoArg (mode ModeCompiler)) "Run in batch compiler mode.",
Option ['j'] ["jobs"] (OptArg jobs "N") "Compile N modules in parallel with -batch (default 1).",
Option [] ["interactive"] (NoArg (mode ModeInteractive)) "Run in interactive mode (default).",
Option [] ["run"] (NoArg (mode ModeRun)) "Run in interactive mode, showing output only (no other messages).",
Option [] ["server"] (OptArg modeServer "port") $
@@ -387,6 +389,7 @@ optDescr =
ms = mode . ModeServer
readPort p = maybe err ms (readMaybe p)
where err = fail $ "Bad server port: "++p
jobs mv = set $ \ o -> o { optJobs = Just mv }
verbosity mv = case mv of
Nothing -> set $ \o -> o { optVerbosity = Verbose }
Just v -> case readMaybe v >>= toEnumBounded of