Fix Issue 41: "," in context-free rules causes parsing failure

The CF parser in GF.Grammar.CF assigns function names to the rules, but they
are not always unique, causing rules to be dropped in the follwing CF->GF
conversion. So a pass has been added before the CF->GF conversion, to make
sure that function names are unique.

A comment says "rules have an amazingly easy parser", but the parser looks
like quick hack. It is very sloppy and silently ignores many errors, e.g.
- Explicitly given function names should end with '.', but if the do not, the
  last character in the function name is silently dropped.
- Everything following a ';' is silently dropped.
This commit is contained in:
hallgren
2013-11-11 16:50:43 +00:00
parent d1314dcbcd
commit 2da15f558e

View File

@@ -23,12 +23,13 @@ import GF.Infra.UseIO
import GF.Data.Operations import GF.Data.Operations
import GF.Data.Utilities (nub') import GF.Data.Utilities (nub')
import qualified Data.Set as S
import Data.Char import Data.Char
import Data.List import Data.List
--import System.FilePath --import System.FilePath
getCF :: FilePath -> String -> Err SourceGrammar getCF :: FilePath -> String -> Err SourceGrammar
getCF fpath = fmap (cf2gf fpath) . pCF getCF fpath = fmap (cf2gf fpath . uniqueFuns) . pCF
--------------------- ---------------------
-- the parser ------- -- the parser -------
@@ -77,6 +78,21 @@ type CFItem = Either CFCat String
type CFCat = String type CFCat = String
type CFFun = String type CFFun = String
--------------------------------
-- make function names unique --
--------------------------------
uniqueFuns :: CF -> CF
uniqueFuns = snd . mapAccumL uniqueFun S.empty
where
uniqueFun funs (L l (fun,rule)) = (S.insert fun' funs,L l (fun',rule))
where
fun' = head [fun'|suffix<-"":map show ([2..]::[Int]),
let fun'=fun++suffix,
not (fun' `S.member` funs)]
-------------------------- --------------------------
-- the compiler ---------- -- the compiler ----------
-------------------------- --------------------------