-- -
- --This is a hands-on introduction to grammar writing in GF. -
--Main ingredients of GF: -
--Prerequisites: -
-- -
- --Lesson 1: a multilingual "Hello World" grammar. English, Finnish, Italian. -
--Lesson 2: a larger grammar for the domain of food. English and Italian. -
--Lesson 3: parameters - morphology and agreement. -
--Lesson 4: using the resource grammar library. -
--Lesson 5: semantics - dependent types, variable bindings, -and semantic definitions. -
--Lesson 6: implementing formal languages. -
--Lesson 7: embedded grammar applications. -
-- -
- --You can chop this tutorial into a set of slides by the command -
-- htmls gf-tutorial.html --
-where the program htmls is distributed with GF (see below), in
-
-The slides will appear as a set of files beginning with 01-gf-tutorial.htmls.
-
-Internal links will not work in the slide format, except for those in the -upper left corner of each slide, and the links behind the "Contents" link. -
-- -
- --Goals: -
-- -
- --We use the term GF for three different things: -
--The GF system is an implementation -of the GF programming language, which in turn is built on the ideas of the -GF theory. -
--The focus of this tutorial is on using the GF programming language. -
--At the same time, we learn the way of thinking in the GF theory. -
--We make the grammars run on a computer by -using the GF system. -
-- -
- --A GF program is called a grammar. -
--A grammar defines a language. -
--From this definition, language processing components can be derived: -
--In general, a GF grammar is multilingual: -
-- -
- --Open-source free software, downloaded via the GF Homepage: -
- --There you find -
--Many examples in this tutorial are -online. -
--Normally you don't have to compile GF yourself. -But, if you do want to compile GF from source follow the -instructions in the Developers Guide. -
-- -
- -
-Type gf in the Unix (or Cygwin) shell:
-
- % gf --
-You will see GF's welcome message and the prompt >.
-The command
-
- > help --
-will give you a list of available commands. -
--As a common convention, we will use -
-% as a prompt that marks system commands
-> as a prompt that marks GF commands
--Thus you should not type these prompts, but only the characters that -follow them. -
-- -
- --Like most programming language tutorials, we start with a -program that prints "Hello World" on the terminal. -
--Extra features: -
-- -
- --A GF program, in general, is a multilingual grammar. Its main parts -are -
--The abstract syntax defines what meanings -can be expressed in the grammar -
-- -
--GF code for the abstract syntax: -
-
- -- a "Hello World" grammar
- abstract Hello = {
-
- flags startcat = Greeting ;
-
- cat Greeting ; Recipient ;
-
- fun
- Hello : Recipient -> Greeting ;
- World, Mum, Friends : Recipient ;
- }
-
--The code has the following parts: -
-Hello
-Greeting is the
- default start category for parsing and generation
- - -
--English concrete syntax (mapping from meanings to strings): -
-
- concrete HelloEng of Hello = {
-
- lincat Greeting, Recipient = {s : Str} ;
-
- lin
- Hello recip = {s = "hello" ++ recip.s} ;
- World = {s = "world"} ;
- Mum = {s = "mum"} ;
- Friends = {s = "friends"} ;
- }
-
--The major parts of this code are: -
-Hello, itself named HelloEng
-Greeting and Recipient are records with a string s
-
-Notice the concatenation ++ and the record projection ..
-
- -
--Finnish and an Italian concrete syntaxes: -
-
- concrete HelloFin of Hello = {
- lincat Greeting, Recipient = {s : Str} ;
- lin
- Hello recip = {s = "terve" ++ recip.s} ;
- World = {s = "maailma"} ;
- Mum = {s = "äiti"} ;
- Friends = {s = "ystävät"} ;
- }
-
- concrete HelloIta of Hello = {
- lincat Greeting, Recipient = {s : Str} ;
- lin
- Hello recip = {s = "ciao" ++ recip.s} ;
- World = {s = "mondo"} ;
- Mum = {s = "mamma"} ;
- Friends = {s = "amici"} ;
- }
-
-
-- -
- -
-In order to compile the grammar in GF,
-we create four files, one for each module, named Modulename.gf:
-
- Hello.gf HelloEng.gf HelloFin.gf HelloIta.gf --
-The first GF command: import a grammar. -
-- > import HelloEng.gf --
-All commands also have short names; here: -
-- > i HelloEng.gf --
-The GF system will compile your grammar -into an internal representation and show the CPU time was consumed, followed -by a new prompt: -
-- > i HelloEng.gf - - compiling Hello.gf... wrote file Hello.gfo 8 msec - - compiling HelloEng.gf... wrote file HelloEng.gfo 12 msec - - 12 msec - > -- -
- -
-
-You can use GF for parsing (parse = p)
-
- > parse "hello world" - Hello World --
-Parsing takes a string into an abstract syntax tree. -
--The notation for trees is that of function application: -
-- function argument1 ... argumentn --
-Parentheses are only needed for grouping. -
--Parsing something that is not in grammar will fail: -
-- > parse "hello dad" - Unknown words: dad - - > parse "world hello" - no tree found -- -
- -
-
-You can also use GF for linearization (linearize = l).
-It takes trees into strings:
-
- > linearize Hello World - hello world --
-Translation: pipe linearization to parsing: -
-- > import HelloEng.gf - > import HelloIta.gf - - > parse -lang=HelloEng "hello mum" | linearize -lang=HelloIta - ciao mamma --
-Default of the language flag (-lang): the last-imported concrete syntax.
-
-Multilingual generation: -
-- > parse -lang=HelloEng "hello friends" | linearize - terve ystävät - ciao amici - hello friends --
-Linearization is by default to all available languages. -
-- -
- -Hello.gf and some of the
-concrete syntaxes by five new recipients and one new greeting
-form.
-
-Hello grammars, for example, leave out
-some line, omit a variable in a lin rule, or change the name
-in one occurrence
-of a variable. Inspect the error messages generated by GF.
-- -
- -
-You can use the gf program in a Unix pipe.
-
- % echo "l Hello World" | gf HelloEng.gf HelloFin.gf HelloIta.gf --
-You can also write a script, a file containing the lines -
-- import HelloEng.gf - import HelloFin.gf - import HelloIta.gf - linearize Hello World -- -
- -
- -
-If we name this script hello.gfs, we can do
-
- $ gf --run <hello.gfs - - ciao mondo - terve maailma - hello world --
-The option --run removes prompts, CPU time, and other messages.
-
-See Lesson 7, for stand-alone programs that don't need the GF system to run. -
--Exercise. (For Unix hackers.) Write a GF application that reads -an English string from the standard input and writes an Italian -translation to the output. -
-- -
- --Some more functions that will be covered: -
-- -
- --Application programs, using techniques from Lesson 7: -
-- -
- --Goals: -
-- -
- --Phrases usable for speaking about food: -
-Phrase
-Phrase can be built by assigning a Quality to an Item
- (e.g. this cheese is Italian)
-Item is build from a Kind by prefixing this or that
- (e.g. this wine)
-Kind is either atomic (e.g. cheese), or formed
- qualifying a given Kind with a Quality (e.g. Italian cheese)
-Quality is either atomic (e.g. Italian,
- or built by modifying a given Quality with the word very (e.g. very warm)
--Abstract syntax: -
-
- abstract Food = {
-
- flags startcat = Phrase ;
-
- cat
- Phrase ; Item ; Kind ; Quality ;
-
- fun
- Is : Item -> Quality -> Phrase ;
- This, That : Kind -> Item ;
- QKind : Quality -> Kind -> Kind ;
- Wine, Cheese, Fish : Kind ;
- Very : Quality -> Quality ;
- Fresh, Warm, Italian, Expensive, Delicious, Boring : Quality ;
- }
-
-
-Example Phrase
-
- Is (This (QKind Delicious (QKind Italian Wine))) (Very (Very Expensive)) - this delicious Italian wine is very very expensive -- -
- -
- -
- concrete FoodEng of Food = {
-
- lincat
- Phrase, Item, Kind, Quality = {s : Str} ;
-
- lin
- Is item quality = {s = item.s ++ "is" ++ quality.s} ;
- This kind = {s = "this" ++ kind.s} ;
- That kind = {s = "that" ++ kind.s} ;
- QKind quality kind = {s = quality.s ++ kind.s} ;
- Wine = {s = "wine"} ;
- Cheese = {s = "cheese"} ;
- Fish = {s = "fish"} ;
- Very quality = {s = "very" ++ quality.s} ;
- Fresh = {s = "fresh"} ;
- Warm = {s = "warm"} ;
- Italian = {s = "Italian"} ;
- Expensive = {s = "expensive"} ;
- Delicious = {s = "delicious"} ;
- Boring = {s = "boring"} ;
- }
-
-
-- -
--Test the grammar for parsing: -
-- > import FoodEng.gf - > parse "this delicious wine is very very Italian" - Is (This (QKind Delicious Wine)) (Very (Very Italian)) --
-Parse in other categories setting the cat flag:
-
- p -cat=Kind "very Italian wine" - QKind (Very Italian) Wine -- -
- -
- -Food grammar by ten new food kinds and
-qualities, and run the parser with new kinds of examples.
-
-- -
- -
-Random generation (generate_random = gr): build
-build a random tree in accordance with an abstract syntax:
-
- > generate_random - Is (This (QKind Italian Fish)) Fresh --
-By using a pipe, random generation can be fed into linearization: -
-- > generate_random | linearize - this Italian fish is fresh --
-Use the number flag to generate several trees:
-
- > gr -number=4 | l - that wine is boring - that fresh cheese is fresh - that cheese is very boring - this cheese is Italian -- -
- -
-
-To generate all phrases that a grammar can produce,
-use generate_trees = gt.
-
- > generate_trees | l - that cheese is very Italian - that cheese is very boring - that cheese is very delicious - ... - this wine is fresh - this wine is warm --
-The default depth is 3; the depth can be
-set by using the depth flag:
-
- > generate_trees -depth=2 | l --
-What options a command has can be seen by the help = h command:
-
- > help gr - > help gt -- -
- -
- -gt generated all
-trees in your grammar, it would never terminate. Why?
-
-wc to count lines.
-- -
- -
-Put the tracing option -tr to each command whose output you
-want to see:
-
- > gr -tr | l -tr | p - - Is (This Cheese) Boring - this cheese is boring - Is (This Cheese) Boring --
-Useful for test purposes: the pipe above can show -if a grammar is ambiguous, i.e. -contains strings that can be parsed in more than one way. -
-
-Exercise. Extend the Food grammar so that it produces ambiguous
-strings, and try out the ambiguity test.
-
- -
- -
-To save the outputs into a file, pipe it to the write_file = wf command,
-
- > gr -number=10 | linearize | write_file -file=exx.tmp --
-To read a file to GF, use the read_file = rf command,
-
- > read_file -file=exx.tmp -lines | parse --
-The flag -lines tells GF to read each line of the file separately.
-
-Files with examples can be used for regression testing -of grammars - the most systematic way to do this is by -treebanks; see here. -
-- -
- --Parentheses give a linear representation of trees, -useful for the computer. -
-
-Human eye may prefer to see a visualization: visualize_tree = vt:
-
- > parse "this delicious cheese is very Italian" | visualize_tree --
-The tree is generated in postscript (.ps) file. The -view option is used for
-telling what command to use to view the file. Its default is "open", which works
-on Mac OS X. On Ubuntu Linux, one can write
-
- > parse "this delicious cheese is very Italian" | visualize_tree -view="eog" -- -
-
-
-This command uses the program Graphviz, which you -might not have, but which are freely available on the web. -
-
-You can save the temporary file _grph.dot,
-which the command vt produces.
-
-Then you can process this file with the dot
-program (from the Graphviz package).
-
- % dot -Tpng _grph.dot > mytree.png --
-You can also visualize parse trees, which show categories and words instead of
-function symbols. The command is visualize_parse = vp:
-
- > parse "this delicious cheese is very Italian" | visualize_parse -- -
-
-
- -
- -
-You can give a system command without leaving GF:
-! followed by a Unix command,
-
- > ! dot -Tpng grphtmp.dot > mytree.png - > ! open mytree.png --
-A system command may also receive its argument from
-a GF pipes. It then uses the symbol ?:
-
- > generate_trees -depth=4 | ? wc -l --
-This command example returns the number of generated trees. -
-
-Exercise.
-Measure how many trees the grammar FoodEng gives with depths 4 and 5,
-respectively. Use the Unix word count command wc to count lines, and
-a system pipe from a GF command into a Unix command.
-
- -
- --Just (?) replace English words with their dictionary equivalents: -
-
- concrete FoodIta of Food = {
-
- lincat
- Phrase, Item, Kind, Quality = {s : Str} ;
-
- lin
- Is item quality = {s = item.s ++ "è" ++ quality.s} ;
- This kind = {s = "questo" ++ kind.s} ;
- That kind = {s = "quel" ++ kind.s} ;
- QKind quality kind = {s = kind.s ++ quality.s} ;
- Wine = {s = "vino"} ;
- Cheese = {s = "formaggio"} ;
- Fish = {s = "pesce"} ;
- Very quality = {s = "molto" ++ quality.s} ;
- Fresh = {s = "fresco"} ;
- Warm = {s = "caldo"} ;
- Italian = {s = "italiano"} ;
- Expensive = {s = "caro"} ;
- Delicious = {s = "delizioso"} ;
- Boring = {s = "noioso"} ;
- }
-
-
-- -
--Not just replacing words: -
--The order of a quality and the kind it modifies is changed in -
-
- QKind quality kind = {s = kind.s ++ quality.s} ;
-
-
-Thus Italian says vino italiano for Italian wine.
-
-(Some Italian adjectives -are put before the noun. This distinction can be controlled by parameters, -which are introduced in Lesson 3.) -
-
-Multilingual grammars have yet another visualization option:
-word alignment, which shows what words correspond to each other.
-Technically, this means words that have the same smallest spanning subtrees
-in abstract syntax. The command is align_words = aw:
-
- > parse "this delicious cheese is very Italian" | align_words -- -
-
-
- -
- -Food for some other language.
-You will probably end up with grammatically incorrect
-linearizations - but don't
-worry about this yet.
-
-Food for German, Swedish, or some
-other language, test with random or exhaustive generation what constructs
-come out incorrect, and prepare a list of those ones that cannot be helped
-with the currently available fragment of GF. You can return to your list
-after having worked out Lesson 3.
-- -
- --Semantically indistinguishable ways of expressing a thing. -
--The variants construct of GF expresses free variation. For example, -
-
- lin Delicious = {s = "delicious" | "exquisit" | "tasty"} ;
-
-
-By default, the linearize command
-shows only the first variant from such lists; to see them
-all, use the option -all:
-
- > p "this exquisit wine is delicious" | l -all - this delicious wine is delicious - this delicious wine is exquisit - ... -- -
- -
--An equivalent notation for variants is -
-
- lin Delicious = {s = variants {"delicious" ; "exquisit" ; "tasty"}} ;
-
--This notation also allows the limiting case: an empty variant list, -
-
- variants {}
-
--It can be used e.g. if a word lacks a certain inflection form. -
--Free variation works for all types in concrete syntax; all terms in -a variant list must be of the same type. -
-- -
- --Multilingual treebank: a set of trees with their -linearizations in different languages: -
-- > gr -number=2 | l -treebank - - Is (That Cheese) (Very Boring) - quel formaggio è molto noioso - that cheese is very boring - - Is (That Cheese) Fresh - quel formaggio è fresco - that cheese is fresh -- -
- -
- -
-translation_quiz = tq:
-generate random sentences, display them in one language, and check the user's
-answer given in another language.
-
- > translation_quiz -from=FoodEng -to=FoodIta
-
- Welcome to GF Translation Quiz.
- The quiz is over when you have done at least 10 examples
- with at least 75 % success.
- You can interrupt the quiz by entering a line consisting of a dot ('.').
-
- this fish is warm
- questo pesce è caldo
- > Yes.
- Score 1/1
-
- this cheese is Italian
- questo formaggio è noioso
- > No, not questo formaggio è noioso, but
- questo formaggio è italiano
-
- Score 1/2
- this fish is expensive
-
-
-- -
- -
-The grammar FoodEng can be written in a BNF format as follows:
-
- Is. Phrase ::= Item "is" Quality ; - That. Item ::= "that" Kind ; - This. Item ::= "this" Kind ; - QKind. Kind ::= Quality Kind ; - Cheese. Kind ::= "cheese" ; - Fish. Kind ::= "fish" ; - Wine. Kind ::= "wine" ; - Italian. Quality ::= "Italian" ; - Boring. Quality ::= "boring" ; - Delicious. Quality ::= "delicious" ; - Expensive. Quality ::= "expensive" ; - Fresh. Quality ::= "fresh" ; - Very. Quality ::= "very" Quality ; - Warm. Quality ::= "warm" ; --
-GF can convert BNF grammars into GF.
-BNF files are recognized by the file name suffix .cf (for context-free):
-
- > import food.cf --
-The compiler creates separate abstract and concrete modules internally. -
-- -
- --Separating concrete and abstract syntax allows -three deviations from context-free grammar: -
-
-Exercise. Define the non-context-free
-copy language {x x | x <- (a|b)*} in GF.
-
- -
- --GF uses suffixes to recognize different file formats: -
-.gf
-.gfo
--Importing generates target from source: -
-- > i FoodEng.gf - - compiling Food.gf... wrote file Food.gfo 16 msec - - compiling FoodEng.gf... wrote file FoodEng.gfo 20 msec --
-The .gfo format (="GF Object") is precompiled GF, which is
-faster to load than source GF (.gf).
-
-When reading a module, GF decides whether
-to use an existing .gfo file or to generate
-a new one, by looking at modification times.
-
- -
-
-Exercise. What happens when you import FoodEng.gf for
-a second time? Try this in different situations:
-
empty (e), which clears the memory
- of GF.
-FoodEng.gf, be it only an added space.
-Food.gf.
-- -
- --The golden rule of functional programmin: -
--Whenever you find yourself programming by copy-and-paste, write a function instead. -
-
-Functions in concrete syntax are defined using the keyword oper (for
-operation), distinct from fun for the sake of clarity.
-
-Example: -
-
- oper ss : Str -> {s : Str} = \x -> {s = x} ;
-
--The operation can be applied to an argument, and GF will -compute the value: -
-
- ss "boy" ===> {s = "boy"}
-
-
-The symbol ===> will be used for computation.
-
- -
--Notice the lambda abstraction form -
-\x -> t
--This is read: -
--For lambda abstraction with multiple arguments, we have the shorthand -
-- \x,y -> t === \x -> \y -> t --
-Linearization rules actually use syntactic -sugar for abstraction: -
-- lin f x = t === lin f = \x -> t -- -
- -
- -
-The resource module type is used to package
-oper definitions into reusable resources.
-
- resource StringOper = {
- oper
- SS : Type = {s : Str} ;
- ss : Str -> SS = \x -> {s = x} ;
- cc : SS -> SS -> SS = \x,y -> ss (x.s ++ y.s) ;
- prefix : Str -> SS -> SS = \p,x -> ss (p ++ x.s) ;
- }
-
-
-- -
- -
-Any number of resource modules can be
-opened in a concrete syntax.
-
- concrete FoodEng of Food = open StringOper in {
-
- lincat
- S, Item, Kind, Quality = SS ;
-
- lin
- Is item quality = cc item (prefix "is" quality) ;
- This k = prefix "this" k ;
- That k = prefix "that" k ;
- QKind k q = cc k q ;
- Wine = ss "wine" ;
- Cheese = ss "cheese" ;
- Fish = ss "fish" ;
- Very = prefix "very" ;
- Fresh = ss "fresh" ;
- Warm = ss "warm" ;
- Italian = ss "Italian" ;
- Expensive = ss "expensive" ;
- Delicious = ss "delicious" ;
- Boring = ss "boring" ;
- }
-
-
-- -
- --The rule -
-- lin This k = prefix "this" k ; --
-can be written more concisely -
-- lin This = prefix "this" ; --
-Part of the art in functional programming: -decide the order of arguments in a function, -so that partial application can be used as much as possible. -
-
-For instance, prefix is typically applied to
-linearization variables with constant strings. Hence we
-put the Str argument before the SS argument.
-
-Exercise. Define an operation infix analogous to prefix,
-such that it allows you to write
-
- lin Is = infix "is" ; -- -
- -
- -
-Import with the flag -retain,
-
- > import -retain StringOper.gf --
-Compute the value with compute_concrete = cc,
-
- > compute_concrete prefix "in" (ss "addition")
- {s : Str = "in" ++ "addition"}
-
-
-- -
- --A new module can extend an old one: -
-
- abstract Morefood = Food ** {
- cat
- Question ;
- fun
- QIs : Item -> Quality -> Question ;
- Pizza : Kind ;
- }
-
--Parallel to the abstract syntax, extensions can -be built for concrete syntaxes: -
-
- concrete MorefoodEng of Morefood = FoodEng ** {
- lincat
- Question = {s : Str} ;
- lin
- QIs item quality = {s = "is" ++ item.s ++ quality.s} ;
- Pizza = {s = "pizza"} ;
- }
-
--The effect of extension: all of the contents of the extended -and extending modules are put together. -
--In other words: the new module inherits the contents of the old module. -
-- -
--Simultaneous extension and opening: -
-
- concrete MorefoodIta of Morefood = FoodIta ** open StringOper in {
- lincat
- Question = SS ;
- lin
- QIs item quality = ss (item.s ++ "è" ++ quality.s) ;
- Pizza = ss "pizza" ;
- }
-
--Resource modules can extend other resource modules - thus it is -possible to build resource hierarchies. -
-- -
- --Extend several grammars at the same time: -
-
- abstract Foodmarket = Food, Fruit, Mushroom ** {
- fun
- FruitKind : Fruit -> Kind ;
- MushroomKind : Mushroom -> Kind ;
- }
-
--where -
-
- abstract Fruit = {
- cat Fruit ;
- fun Apple, Peach : Fruit ;
- }
-
- abstract Mushroom = {
- cat Mushroom ;
- fun Cep, Agaric : Mushroom ;
- }
-
-
-
-Exercise. Refactor Food by taking apart Wine into a special
-Drink module.
-
- -
- --Goals: -
--It is possible to skip this chapter and go directly -to the next, since the use of the GF Resource Grammar library -makes it unnecessary to use parameters: they -could be left to library implementors. -
-- -
- --Plural forms are needed in things like -
-Different languages have different types of inflection and agreement. -
--In a multilingual grammar, -we want to ignore such distinctions in abstract syntax. -
--Exercise. Make a list of the possible forms that nouns, -adjectives, and verbs can have in some languages that you know. -
-- -
- --We define the parameter type of number in English by -a new form of judgement: -
-- param Number = Sg | Pl ; --
-This judgement defines the parameter type Number by listing
-its two constructors, Sg and Pl
-(singular and plural).
-
-We give Kind a linearization type that has a table depending on number:
-
- lincat Kind = {s : Number => Str} ;
-
-
-The table type Number => Str is similar a function type
-(Number -> Str).
-
-Difference: the argument must be a parameter type. Then -the argument-value pairs can be listed in a finite table. -
-- -
--Here is a table: -
-
- lin Cheese = {
- s = table {
- Sg => "cheese" ;
- Pl => "cheeses"
- }
- } ;
-
-
-The table has branches, with a pattern on the
-left of the arrow => and a value on the right.
-
-The application of a table is done by the selection operator !.
-
-It which is computed by pattern matching: return -the value from the first branch whose pattern matches the -argument. For instance, -
-
- table {Sg => "cheese" ; Pl => "cheeses"} ! Pl
- ===> "cheeses"
-
-
-- -
--Case expressions are syntactic sugar: -
-
- case e of {...} === table {...} ! e
-
--Since they are familiar to Haskell and ML programmers, they can come out handy -when writing GF programs. -
-- -
--Constructors can take arguments from other parameter types. -
--Example: forms of English verbs (except be): -
-- param VerbForm = VPresent Number | VPast | VPastPart | VPresPart ; --
-Fact expressed: only present tense has number variation. -
--Example table: the forms of the verb drink: -
-
- table {
- VPresent Sg => "drinks" ;
- VPresent Pl => "drink" ;
- VPast => "drank" ;
- VPastPart => "drunk" ;
- VPresPart => "drinking"
- }
-
-
-
-Exercise. In an earlier exercise (previous section),
-you made a list of the possible
-forms that nouns, adjectives, and verbs can have in some languages that
-you know. Now take some of the results and implement them by
-using parameter type definitions and tables. Write them into a resource
-module, which you can test by using the command compute_concrete.
-
- -
- --A morphological paradigm is a formula telling how a class of -words is inflected. -
--From the GF point of view, a paradigm is a function that takes -a lemma (also known as a dictionary form, or a citation form) and -returns an inflection table. -
--The following operation defines the regular noun paradigm of English: -
-
- oper regNoun : Str -> {s : Number => Str} = \dog -> {
- s = table {
- Sg => dog ;
- Pl => dog + "s"
- }
- } ;
-
-
-The gluing operator + glues strings to one token:
-
- (regNoun "cheese").s ! Pl ===> "cheese" + "s" ===> "cheeses" -- -
- -
--A more complex example: regular verbs, -
-
- oper regVerb : Str -> {s : VerbForm => Str} = \talk -> {
- s = table {
- VPresent Sg => talk + "s" ;
- VPresent Pl => talk ;
- VPresPart => talk + "ing" ;
- _ => talk + "ed"
- }
- } ;
-
-
-The catch-all case for the past tense and the past participle
-uses a wild card pattern _.
-
- -
- -regNoun paradigm does not
-apply in English, and implement some alternative paradigms.
-
-- -
- --Purpose: a more radical -variation between languages -than just the use of different words and word orders. -
-
-We add to the grammar Food two rules for forming plural items:
-
- fun These, Those : Kind -> Item ; --
-We also add a noun which in Italian has the feminine case: -
-- fun Pizza : Kind ; --
-This will force us to deal with gender- -
-- -
- --In English, the phrase-forming rule -
-- fun Is : Item -> Quality -> Phrase ; --
-is affected by the number because of subject-verb agreement: -the verb of a sentence must be inflected in the number of the subject, -
-- Is (This Pizza) Warm ===> "this pizza is warm" - Is (These Pizza) Warm ===> "these pizzas are warm" --
-It is the copula (the verb be) that is affected: -
-
- oper copula : Number -> Str = \n ->
- case n of {
- Sg => "is" ;
- Pl => "are"
- } ;
-
-
-The subject Item must have such a number to provide to the copula:
-
- lincat Item = {s : Str ; n : Number} ;
-
--Now we can write -
-
- lin Is item qual = {s = item.s ++ copula item.n ++ qual.s} ;
-
-
-- -
- -
-How does an Item subject receive its number? The rules
-
- fun This, These : Kind -> Item ; --
-add determiners, either this or these, which -require different this pizza vs. -these pizzas. -
-
-Thus Kind must have both singular and plural forms:
-
- lincat Kind = {s : Number => Str} ;
-
--We can write -
-
- lin This kind = {
- s = "this" ++ kind.s ! Sg ;
- n = Sg
- } ;
-
- lin These kind = {
- s = "these" ++ kind.s ! Pl ;
- n = Pl
- } ;
-
-
-- -
--To avoid copy-and-paste, we can factor out the pattern of determination, -
-
- oper det :
- Str -> Number -> {s : Number => Str} -> {s : Str ; n : Number} =
- \det,n,kind -> {
- s = det ++ kind.s ! n ;
- n = n
- } ;
-
--Now we can write -
-- lin This = det Sg "this" ; - lin These = det Pl "these" ; --
-In a more lexicalized grammar, determiners would be a category: -
-
- lincat Det = {s : Str ; n : Number} ;
- fun Det : Det -> Kind -> Item ;
- lin Det det kind = {
- s = det.s ++ kind.s ! det.n ;
- n = det.n
- } ;
-
-
-- -
- -
-Kinds have number as a parametric feature: both singular and plural
-can be formed,
-
- lincat Kind = {s : Number => Str} ;
-
-
-Items have number as an inherent feature: they are inherently either
-singular or plural,
-
- lincat Item = {s : Str ; n : Number} ;
-
-
-Italian Kind will have parametric number and inherent gender:
-
- lincat Kind = {s : Number => Str ; g : Gender} ;
-
-
-- -
--Questions to ask when designing parameters: -
--Dictionaries give good advice: -
-For words, inherent features are usually given as lexical information. -
--For combinations, they are inherited from some part of the construction -(typically the one called the head). Italian modification: -
-
- lin QKind qual kind =
- let gen = kind.g in {
- s = table {n => kind.s ! n ++ qual.s ! gen ! n} ;
- g = gen
- } ;
-
--Notice -
-let expression)
-n
-- -
- -
-We use some string operations from the library Prelude are used.
-
- concrete FoodsEng of Foods = open Prelude in {
-
- lincat
- S, Quality = SS ;
- Kind = {s : Number => Str} ;
- Item = {s : Str ; n : Number} ;
-
- lin
- Is item quality = ss (item.s ++ copula item.n ++ quality.s) ;
- This = det Sg "this" ;
- That = det Sg "that" ;
- These = det Pl "these" ;
- Those = det Pl "those" ;
- QKind quality kind = {s = table {n => quality.s ++ kind.s ! n}} ;
- Wine = regNoun "wine" ;
- Cheese = regNoun "cheese" ;
- Fish = noun "fish" "fish" ;
- Pizza = regNoun "pizza" ;
- Very = prefixSS "very" ;
- Fresh = ss "fresh" ;
- Warm = ss "warm" ;
- Italian = ss "Italian" ;
- Expensive = ss "expensive" ;
- Delicious = ss "delicious" ;
- Boring = ss "boring" ;
-
-
-- -
-
- param
- Number = Sg | Pl ;
-
- oper
- det : Number -> Str -> {s : Number => Str} -> {s : Str ; n : Number} =
- \n,d,cn -> {
- s = d ++ cn.s ! n ;
- n = n
- } ;
- noun : Str -> Str -> {s : Number => Str} =
- \man,men -> {s = table {
- Sg => man ;
- Pl => men
- }
- } ;
- regNoun : Str -> {s : Number => Str} =
- \car -> noun car (car + "s") ;
- copula : Number -> Str =
- \n -> case n of {
- Sg => "is" ;
- Pl => "are"
- } ;
- }
-
-
-- -
- --Let us extend the English noun paradigms so that we can -deal with all nouns, not just the regular ones. The goal is to -provide a morphology module that makes it easy to -add words to a lexicon. -
-- -
- --We perform data abstraction from the type -of nouns by writing a a worst-case function: -
-
- oper Noun : Type = {s : Number => Str} ;
-
- oper mkNoun : Str -> Str -> Noun = \x,y -> {
- s = table {
- Sg => x ;
- Pl => y
- }
- } ;
-
- oper regNoun : Str -> Noun = \x -> mkNoun x (x + "s") ;
-
--Then we can define -
-- lincat N = Noun ; - lin Mouse = mkNoun "mouse" "mice" ; - lin House = regNoun "house" ; --
-where the underlying types are not seen. -
-- -
--We are free to change the undelying definitions, e.g. -add case (nominative or genitive) to noun inflection: -
-
- param Case = Nom | Gen ;
-
- oper Noun : Type = {s : Number => Case => Str} ;
-
--Now we have to redefine the worst-case function -
-
- oper mkNoun : Str -> Str -> Noun = \x,y -> {
- s = table {
- Sg => table {
- Nom => x ;
- Gen => x + "'s"
- } ;
- Pl => table {
- Nom => y ;
- Gen => y + case last y of {
- "s" => "'" ;
- _ => "'s"
- }
- }
- } ;
-
--But up from this level, we can retain the old definitions -
-- lin Mouse = mkNoun "mouse" "mice" ; - oper regNoun : Str -> Noun = \x -> mkNoun x (x + "s") ; -- -
- -
-
-In the last definition of mkNoun, we used a case expression
-on the last character of the plural, as well as the Prelude
-operation
-
- last : Str -> Str ; --
-returning the string consisting of the last character. -
--The case expression uses pattern matching over strings, which -is supported in GF, alongside with pattern matching over -parameters. -
-- -
- --The regular dog-dogs paradigm has -predictable variations: -
--We could provide alternative paradigms: -
-- noun_y : Str -> Noun = \fly -> mkNoun fly (init fly + "ies") ; - noun_s : Str -> Noun = \bus -> mkNoun bus (bus + "es") ; --
-(The Prelude function init drops the last character of a token.)
-
-Drawbacks: -
-- -
--Better solution: a smart paradigm: -
-
- regNoun : Str -> Noun = \w ->
- let
- ws : Str = case w of {
- _ + ("a" | "e" | "i" | "o") + "o" => w + "s" ; -- bamboo
- _ + ("s" | "x" | "sh" | "o") => w + "es" ; -- bus, hero
- _ + "z" => w + "zes" ;-- quiz
- _ + ("a" | "e" | "o" | "u") + "y" => w + "s" ; -- boy
- x + "y" => x + "ies" ;-- fly
- _ => w + "s" -- car
- }
- in
- mkNoun w ws
-
--GF has regular expression patterns: -
-| Q
-+ Q
-
-The patterns are ordered in such a way that, for instance,
-the suffix "oo" prevents bamboo from matching the suffix
-"o".
-
- -
- -regNoun so that the analysis needed to build s-forms
-is factored out as a separate oper, which is shared with
-regVerb.
-
-- -
- --In Lesson 5, dependent function types need a notation -that binds a variable to the argument type, as in -
-- switchOff : (k : Kind) -> Action k --
-Function types without variables are actually a shorthand: -
-- PredVP : NP -> VP -> S --
-means -
-- PredVP : (x : NP) -> (y : VP) -> S --
-or any other naming of the variables. -
-- -
--Sometimes variables shorten the code, since they can share a type: -
-- octuple : (x,y,z,u,v,w,s,t : Str) -> Str --
-If a bound variable is not used, it can be replaced by a wildcard: -
-- octuple : (_,_,_,_,_,_,_,_ : Str) -> Str --
-A good practice is to indicate the number of arguments: -
-- octuple : (x1,_,_,_,_,_,_,x8 : Str) -> Str --
-For inflection paradigms, it is handy to use heuristic variable names, -looking like the expected forms: -
-- mkNoun : (mouse,mice : Str) -> Noun -- -
- -
- -
-In librarues, it is useful to group type signatures separately from
-definitions. It is possible to divide an oper judgement,
-
- oper regNoun : Str -> Noun ; - oper regNoun s = mkNoun s (s + "s") ; --
-and put the parts in different places. -
-
-With the interface and instance module types
-(see here): the parts can even be put to different files.
-
- -
- --Overloading: different functions can be given the same name, as e.g. in C++. -
--The compiler performs overload resolution, which works as long as the -functions have different types. -
-
-In GF, the functions must be grouped together in overload groups.
-
-Example: different ways to define nouns in English: -
-
- oper mkN : overload {
- mkN : (dog : Str) -> Noun ; -- regular nouns
- mkN : (mouse,mice : Str) -> Noun ; -- irregular nouns
- }
-
--Cf. dictionaries: if the -word is regular, just one form is needed. If it is irregular, -more forms are given. -
--The definition can be given separately, or at the same time, as the types: -
-
- oper mkN = overload {
- mkN : (dog : Str) -> Noun = regNoun ;
- mkN : (mouse,mice : Str) -> Noun = mkNoun ;
- }
-
--Exercise. Design a system of English verb paradigms presented by -an overload group. -
-- -
- -
-The command morpho_analyse = ma
-can be used to read a text and return for each word its analyses
-(in the current grammar):
-
- > read_file bible.txt | morpho_analyse --
-The command morpho_quiz = mq generates inflection exercises.
-
- % gf -path=alltenses:prelude $GF_LIB_PATH/alltenses/IrregFre.gfo - - > morpho_quiz -cat=V - - Welcome to GF Morphology Quiz. - ... - - réapparaître : VFin VCondit Pl P2 - réapparaitriez - > No, not réapparaitriez, but - réapparaîtriez - Score 0/1 --
-To create a list for later use, use the command morpho_list = ml
-
- > morpho_list -number=25 -cat=V | write_file exx.txt -- -
- -
- --Parameters include not only number but also gender. -
-
- concrete FoodsIta of Foods = open Prelude in {
-
- param
- Number = Sg | Pl ;
- Gender = Masc | Fem ;
-
--Qualities are inflected for gender and number, whereas kinds -have a parametric number and an inherent gender. -Items have an inherent number and gender. -
-
- lincat
- Phr = SS ;
- Quality = {s : Gender => Number => Str} ;
- Kind = {s : Number => Str ; g : Gender} ;
- Item = {s : Str ; g : Gender ; n : Number} ;
-
-
-- -
--A Quality is an adjective, with one form for each gender-number combination. -
-
- oper
- adjective : (_,_,_,_ : Str) -> {s : Gender => Number => Str} =
- \nero,nera,neri,nere -> {
- s = table {
- Masc => table {
- Sg => nero ;
- Pl => neri
- } ;
- Fem => table {
- Sg => nera ;
- Pl => nere
- }
- }
- } ;
-
--Regular adjectives work by adding endings to the stem. -
-
- regAdj : Str -> {s : Gender => Number => Str} = \nero ->
- let ner = init nero
- in adjective nero (ner + "a") (ner + "i") (ner + "e") ;
-
-
-- -
--For noun inflection, we are happy to give the two forms and the gender -explicitly: -
-
- noun : Str -> Str -> Gender -> {s : Number => Str ; g : Gender} =
- \vino,vini,g -> {
- s = table {
- Sg => vino ;
- Pl => vini
- } ;
- g = g
- } ;
-
--We need only number variation for the copula. -
-
- copula : Number -> Str =
- \n -> case n of {
- Sg => "è" ;
- Pl => "sono"
- } ;
-
-
-- -
--Determination is more complex than in English, because of gender: -
-
- det : Number -> Str -> Str -> {s : Number => Str ; g : Gender} ->
- {s : Str ; g : Gender ; n : Number} =
- \n,m,f,cn -> {
- s = case cn.g of {Masc => m ; Fem => f} ++ cn.s ! n ;
- g = cn.g ;
- n = n
- } ;
-
-
-- -
--The complete set of linearization rules: -
-
- lin
- Is item quality =
- ss (item.s ++ copula item.n ++ quality.s ! item.g ! item.n) ;
- This = det Sg "questo" "questa" ;
- That = det Sg "quel" "quella" ;
- These = det Pl "questi" "queste" ;
- Those = det Pl "quei" "quelle" ;
- QKind quality kind = {
- s = \\n => kind.s ! n ++ quality.s ! kind.g ! n ;
- g = kind.g
- } ;
- Wine = noun "vino" "vini" Masc ;
- Cheese = noun "formaggio" "formaggi" Masc ;
- Fish = noun "pesce" "pesci" Masc ;
- Pizza = noun "pizza" "pizze" Fem ;
- Very qual = {s = \\g,n => "molto" ++ qual.s ! g ! n} ;
- Fresh = adjective "fresco" "fresca" "freschi" "fresche" ;
- Warm = regAdj "caldo" ;
- Italian = regAdj "italiano" ;
- Expensive = regAdj "caro" ;
- Delicious = regAdj "delizioso" ;
- Boring = regAdj "noioso" ;
- }
-
-
-- -
- -Foods grammars.
-
-Food for a language of your choice,
-now aiming for complete grammatical correctness by the use of parameters.
-
-FoodsIta. You can do this by printing the grammar in the context-free format
-(print_grammar -printer=bnf) and counting the lines.
-- -
- --A linearization record may contain more strings than one, and those -strings can be put apart in linearization. -
--Example: English particle -verbs, (switch off). The object can appear between: -
--he switched it off -
--The verb switch off is called a -discontinuous constituents. -
--We can define transitive verbs and their combinations as follows: -
-
- lincat TV = {s : Number => Str ; part : Str} ;
-
- fun AppTV : Item -> TV -> Item -> Phrase ;
-
- lin AppTV subj tv obj =
- {s = subj.s ++ tv.s ! subj.n ++ obj.s ++ tv.part} ;
-
-
-
-Exercise. Define the language a^n b^n c^n in GF, i.e.
-any number of a's followed by the same number of b's and
-the same number of c's. This language is not context-free,
-but can be defined in GF by using discontinuous constituents.
-
- -
- --Tokens are created in the following ways: -
-"foo"
-t + s
-init, tail, tk, dp
--Since tokens must be known at compile time, -the above operations may not be applied to run-time variables -(i.e. variables that stand for function arguments in linearization rules). -
--Hence it is not legal to write -
-
- cat Noun ;
- fun Plural : Noun -> Noun ;
- lin Plural n = {s = n.s + "s"} ;
-
-
-because n is a run-time variable. Also
-
- lin Plural n = {s = (regNoun n).s ! Pl} ;
-
-
-is incorrect with regNoun as defined here, because the run-time
-variable is eventually sent to string pattern matching and gluing.
-
- -
--How to write tokens together without a space? -
-
- lin Question p = {s = p + "?"} ;
-
--is incorrect. -
--The way to go is to use an unlexer that creates correct spacing -after linearization. -
-
-Correspondingly, a lexer that e.g. analyses "warm?" into
-to tokens is needed before parsing.
-This topic will be covered in here.
-
- -
- -
-The symbol ** is used for both record types and record objects.
-
- lincat TV = Verb ** {c : Case} ;
-
- lin Follow = regVerb "folgen" ** {c = Dative} ;
-
-
-TV becomes a subtype of Verb.
-
-If T is a subtype of R, an object of T can be used whenever -an object of R is required. -
--Covariance: a function returning a record T as value can -also be used to return a value of a supertype R. -
--Contravariance: a function taking an R as argument -can also be applied to any object of a subtype T. -
-- -
--Product types and tuples are syntactic sugar for record types and records: -
-
- T1 * ... * Tn === {p1 : T1 ; ... ; pn : Tn}
- <t1, ..., tn> === {p1 = T1 ; ... ; pn = Tn}
-
-
-Thus the labels p1, p2,... are hard-coded.
-
- -
--English indefinite article: -
-
- oper artIndef : Str =
- pre {"a" ; "an" / strs {"a" ; "e" ; "i" ; "o"}} ;
-
--Thus -
-- artIndef ++ "cheese" ---> "a" ++ "cheese" - artIndef ++ "apple" ---> "an" ++ "apple" -- -
- -
- --Goals: -
-- -
- --The current 16 resource languages (GF version 3.2, December 2010) are -
-Bulgarian
-Catalan
-Danish
-Dutch
-English
-Finnish
-French
-German
-Italian
-Norwegian
-Polish
-Ron, Romanian
-Russian
-Spanish
-Swedish
-Urdu
-
-The first three letters (Eng etc) are used in grammar module names
-(ISO 639-3 standard).
-
- -
- --Semantic grammars (up to now in this tutorial): -a grammar defines a system of meanings (abstract syntax) and -tells how they are expressed(concrete syntax). -
--Resource grammars (as usual in linguistic tradition): -a grammar specifies the grammatically correct combinations of words, -whatever their meanings are. -
--With resource grammars, we can achieve a -wider coverage than with semantic grammars. -
-- -
- --A resource grammar has two kinds of categories and two kinds of rules: -
--GE makes no formal distinction between these two kinds. -
--But it is a good discipline to follow. -
-- -
- --Two kinds of lexical categories: -
-- Conj ; -- conjunction e.g. "and" - Det ; -- determiner e.g. "this" -- -
- N ; -- noun e.g. "pizza" - A ; -- adjective e.g. "good" - V ; -- verb e.g. "sleep" --
- -
- -
-Closed classes: module Syntax. In the Foods grammar, we need
-
- this_Det, that_Det, these_Det, those_Det : Det ; - very_AdA : AdA ; --
-Naming convention: word followed by the category (so we can -distinguish the quantifier that from the conjunction that). -
-
-Open classes have no objects in Syntax. Words are
-built as they are needed in applications: if we have
-
- fun Wine : Kind ; --
-we will define -
-- lin Wine = mkN "wine" ; --
-where we use mkN from ParadigmsEng:
-
- -
- --Alternative concrete syntax for -
-- fun Wine : Kind ; --
-is to provide a resource lexicon, which contains definitions such as -
-- oper wine_N : N = mkN "wine" ; --
-so that we can write -
-- lin Wine = wine_N ; --
-Advantages: -
-- -
- -
-In Foods, we need just four phrasal categories:
-
- Cl ; -- clause e.g. "this pizza is good" - NP ; -- noun phrase e.g. "this pizza" - CN ; -- common noun e.g. "warm pizza" - AP ; -- adjectival phrase e.g. "very warm" --
-Clauses are similar to sentences (S), but without a
-fixed tense and mood; see here for how they relate.
-
-Common nouns are made into noun phrases by adding determiners. -
-- -
- --We need the following combinations: -
-- mkCl : NP -> AP -> Cl ; -- e.g. "this pizza is very warm" - mkNP : Det -> CN -> NP ; -- e.g. "this pizza" - mkCN : AP -> CN -> CN ; -- e.g. "warm pizza" - mkAP : AdA -> AP -> AP ; -- e.g. "very warm" --
-We also need lexical insertion, to form phrases from single words: -
-- mkCN : N -> NP ; - mkAP : A -> AP ; --
-Naming convention: to construct a C, use a function mkC.
-
-Heavy overloading: the current library
-(version 1.2) has 23 operations named mkNP!
-
- -
- --The sentence -
- mkCl - (mkNP these_Det - (mkCN (mkAP very_AdA (mkAP warm_A)) (mkCN pizza_CN))) - (mkAP italian_AP) --
-The task now: to define the concrete syntax of Foods so that
-this syntactic tree gives the value of linearizing the semantic tree
-
- Is (These (QKind (Very Warm) Pizza)) Italian -- -
- -
- --Language-specific and language-independent parts - roughly, -
-SyntaxL has the same types and
- functions for all languages L
-ParadigmsL has partly
- different types and functions
- for different languages L
--Full API documentation on-line: the resource synopsis, -
-
-grammaticalframework.org/lib/doc/synopsis.html
-
- -
- -| Category | -Explanation | -Example | -|
|---|---|---|---|
Cl |
-clause (sentence), with all tenses | -she looks at this | -|
AP |
-adjectival phrase | -very warm | -|
CN |
-common noun (without determiner) | -red house | -|
NP |
-noun phrase (subject or object) | -the red house | -|
AdA |
-adjective-modifying adverb, | -very | -|
Det |
-determiner | -these | -|
A |
-one-place adjective | -warm | -|
N |
-common noun | -house | -|
- -
- -| Function | -Type | -Example | -|
|---|---|---|---|
mkCl |
-NP -> AP -> Cl |
-John is very old | -|
mkNP |
-Det -> CN -> NP |
-these old man | -|
mkCN |
-N -> CN |
-house | -|
mkCN |
-AP -> CN -> CN |
-very big blue house | -|
mkAP |
-A -> AP |
-old | -|
mkAP |
-AdA -> AP -> AP |
-very very old | -|
- -
- -| Function | -Type | -In English | -|
|---|---|---|---|
this_Det |
-Det |
-this | -|
that_Det |
-Det |
-that | -|
these_Det |
-Det |
-this | -|
those_Det |
-Det |
-that | -|
very_AdA |
-AdA |
-very | -|
- -
- -
-From ParadigmsEng:
-
| Function | -Type | -|
|---|---|---|
mkN |
-(dog : Str) -> N |
-|
mkN |
-(man,men : Str) -> N |
-|
mkA |
-(cold : Str) -> A |
-|
-From ParadigmsIta:
-
| Function | -Type | -|
|---|---|---|
mkN |
-(vino : Str) -> N |
-|
mkA |
-(caro : Str) -> A |
-|
- -
- -
-From ParadigmsGer:
-
| Function | -Type | -|
|---|---|---|
Gender |
-Type |
-|
masculine |
-Gender |
-|
feminine |
-Gender |
-|
neuter |
-Gender |
-|
mkN |
-(Stufe : Str) -> N |
-|
mkN |
-(Bild,Bilder : Str) -> Gender -> N |
-|
mkA |
-(klein : Str) -> A |
-|
mkA |
-(gut,besser,beste : Str) -> A |
-|
-From ParadigmsFin:
-
| Function | -Type | -|
|---|---|---|
mkN |
-(talo : Str) -> N |
-|
mkA |
-(hieno : Str) -> A |
-|
- -
- --1. Try out the morphological paradigms in different languages. Do -as follows: -
-- > i -path=alltenses -retain alltenses/ParadigmsGer.gfo - > cc -table mkN "Farbe" - > cc -table mkA "gut" "besser" "beste" -- -
- -
- -
-We assume the abstract syntax Foods from Lesson 3.
-
-We don't need to think about inflection and agreement, but just pick -functions from the resource grammar library. -
--We need a path with -
-.
-../foods, in which Foods.gf resides.
-present, which is relative to the
- environment variable GF_LIB_PATH
--Thus the beginning of the module is -
-
- --# -path=.:../foods:present
-
- concrete FoodsEng of Foods = open SyntaxEng,ParadigmsEng in {
-
-
-- -
- -
-As linearization types, we use clauses for Phrase, noun phrases
-for Item, common nouns for Kind, and adjectival phrases for Quality.
-
- lincat - Phrase = Cl ; - Item = NP ; - Kind = CN ; - Quality = AP ; --
-Now the combination rules we need almost write themselves automatically: -
-- lin - Is item quality = mkCl item quality ; - This kind = mkNP this_Det kind ; - That kind = mkNP that_Det kind ; - These kind = mkNP these_Det kind ; - Those kind = mkNP those_Det kind ; - QKind quality kind = mkCN quality kind ; - Very quality = mkAP very_AdA quality ; -- -
- -
- --We use resource paradigms and lexical insertion rules. -
--The two-place noun paradigm is needed only once, for -fish - everythins else is regular. -
-- Wine = mkCN (mkN "wine") ; - Pizza = mkCN (mkN "pizza") ; - Cheese = mkCN (mkN "cheese") ; - Fish = mkCN (mkN "fish" "fish") ; - Fresh = mkAP (mkA "fresh") ; - Warm = mkAP (mkA "warm") ; - Italian = mkAP (mkA "Italian") ; - Expensive = mkAP (mkA "expensive") ; - Delicious = mkAP (mkA "delicious") ; - Boring = mkAP (mkA "boring") ; - } -- -
- -
- -
-1. Compile the grammar FoodsEng and generate
-and parse some sentences.
-
-2. Write a concrete syntax of Foods for Italian
-or some other language included in the resource library. You can
-compare the results with the hand-written
-grammars presented earlier in this tutorial.
-
- -
- -
-If you write a concrete syntax of Foods for some other
-language, much of the code will look exactly the same
-as for English. This is because
-
Syntax API is the same for all languages (because
- all languages in the resource package do implement the same
- syntactic structures)
--But lexical rules are more language-dependent. -
--Thus, to port a grammar to a new language, you -
--Can we avoid this programming by copy-and-paste? -
-- -
- --Functors familiar from the functional programming languages ML and OCaml, -also known as parametrized modules. -
-
-In GF, a functor is a module that opens one or more interfaces.
-
-An interface is a module similar to a resource, but it only
-contains the types of opers, not (necessarily) their definitions.
-
-Syntax for functors: add the keyword incomplete. We will use the header
-
- incomplete concrete FoodsI of Foods = open Syntax, LexFoods in --
-where -
-- interface Syntax -- the resource grammar interface - interface LexFoods -- the domain lexicon interface --
-When we moreover have -
-- instance SyntaxEng of Syntax -- the English resource grammar - instance LexFoodsEng of LexFoods -- the English domain lexicon --
-we can write a functor instantiation, -
-- concrete FoodsGer of Foods = FoodsI with - (Syntax = SyntaxGer), - (LexFoods = LexFoodsGer) ; -- -
- -
- -
- --# -path=.:../foods
-
- incomplete concrete FoodsI of Foods = open Syntax, LexFoods in {
- lincat
- Phrase = Cl ;
- Item = NP ;
- Kind = CN ;
- Quality = AP ;
- lin
- Is item quality = mkCl item quality ;
- This kind = mkNP this_Det kind ;
- That kind = mkNP that_Det kind ;
- These kind = mkNP these_Det kind ;
- Those kind = mkNP those_Det kind ;
- QKind quality kind = mkCN quality kind ;
- Very quality = mkAP very_AdA quality ;
-
- Wine = mkCN wine_N ;
- Pizza = mkCN pizza_N ;
- Cheese = mkCN cheese_N ;
- Fish = mkCN fish_N ;
- Fresh = mkAP fresh_A ;
- Warm = mkAP warm_A ;
- Italian = mkAP italian_A ;
- Expensive = mkAP expensive_A ;
- Delicious = mkAP delicious_A ;
- Boring = mkAP boring_A ;
- }
-
-
-- -
- -
- interface LexFoods = open Syntax in {
- oper
- wine_N : N ;
- pizza_N : N ;
- cheese_N : N ;
- fish_N : N ;
- fresh_A : A ;
- warm_A : A ;
- italian_A : A ;
- expensive_A : A ;
- delicious_A : A ;
- boring_A : A ;
- }
-
-
-- -
- -
- instance LexFoodsGer of LexFoods = open SyntaxGer, ParadigmsGer in {
- oper
- wine_N = mkN "Wein" ;
- pizza_N = mkN "Pizza" "Pizzen" feminine ;
- cheese_N = mkN "Käse" "Käsen" masculine ;
- fish_N = mkN "Fisch" ;
- fresh_A = mkA "frisch" ;
- warm_A = mkA "warm" "wärmer" "wärmste" ;
- italian_A = mkA "italienisch" ;
- expensive_A = mkA "teuer" ;
- delicious_A = mkA "köstlich" ;
- boring_A = mkA "langweilig" ;
- }
-
-
-- -
- -- --# -path=.:../foods:present - - concrete FoodsGer of Foods = FoodsI with - (Syntax = SyntaxGer), - (LexFoods = LexFoodsGer) ; -- -
- -
- --Just two modules are needed: -
--The functor instantiation is completely mechanical to write. -
--The domain lexicon instance requires some knowledge of the words of the -language: -
-- -
- --Lexicon instance -
-
- instance LexFoodsFin of LexFoods = open SyntaxFin, ParadigmsFin in {
- oper
- wine_N = mkN "viini" ;
- pizza_N = mkN "pizza" ;
- cheese_N = mkN "juusto" ;
- fish_N = mkN "kala" ;
- fresh_A = mkA "tuore" ;
- warm_A = mkA "lämmin" ;
- italian_A = mkA "italialainen" ;
- expensive_A = mkA "kallis" ;
- delicious_A = mkA "herkullinen" ;
- boring_A = mkA "tylsä" ;
- }
-
--Functor instantiation -
-- --# -path=.:../foods:present - - concrete FoodsFin of Foods = FoodsI with - (Syntax = SyntaxFin), - (LexFoods = LexFoodsFin) ; -- -
- -
- --This can be seen as a design pattern for multilingual grammars: -
-- concrete DomainL* - - instance LexDomainL instance SyntaxL* - - incomplete concrete DomainI - / | \ - interface LexDomain abstract Domain interface Syntax* --
-Modules marked with * are either given in the library, or trivial.
-
-Of the hand-written modules, only LexDomainL is language-dependent.
-
- -
- -
-1. Compile and test FoodsGer.
-
-2. Refactor FoodsEng into a functor instantiation.
-
-3. Instantiate the functor FoodsI to some language of
-your choice.
-
-4. Design a small grammar that can be used for controlling -an MP3 player. The grammar should be able to recognize commands such -as play this song, with the following variations: -
--The implementation goes in the following phases: -
-- -
- -
-Problem: a functor only works when all languages use the resource Syntax
-in the same way.
-
-Example (contrived): assume that English has
-no word for Pizza, but has to use the paraphrase Italian pie.
-This is no longer a noun N, but a complex phrase
-in the category CN.
-
-Possible solution: change interface the LexFoods with
-
- oper pizza_CN : CN ; --
-Problem with this solution: -
-- -
- --A module may inherit just a selection of names. -
-
-Example: the FoodMarket example "Rsecarchitecture:
-
- abstract Foodmarket = Food, Fruit [Peach], Mushroom - [Agaric] --
-Here, from Fruit we include Peach only, and from Mushroom
-we exclude Agaric.
-
-A concrete syntax of Foodmarket must make the analogous restrictions.
-
- -
- -
-The English instantiation inherits the functor
-implementation except for the constant Pizza. This constant
-is defined in the body instead:
-
- --# -path=.:../foods:present
-
- concrete FoodsEng of Foods = FoodsI - [Pizza] with
- (Syntax = SyntaxEng),
- (LexFoods = LexFoodsEng) **
- open SyntaxEng, ParadigmsEng in {
-
- lin Pizza = mkCN (mkA "Italian") (mkN "pie") ;
- }
-
-
-- -
- --Abstract syntax modules can be used as interfaces, -and concrete syntaxes as their instances. -
--The following correspondencies are then applied: -
-- cat C <---> oper C : Type - - fun f : A <---> oper f : A - - lincat C = T <---> oper C : Type = T - - lin f = t <---> oper f : A = t -- -
- -
- -
-1. Find resource grammar terms for the following
-English phrases (in the category Phr). You can first try to
-build the terms manually.
-
-every man loves a woman -
--this grammar speaks more than ten languages -
--which languages aren't in the grammar -
--which languages did you want to speak -
--Then translate the phrases to other languages. -
-- -
- -
-In Foods grammars, we have used the path
-
- --# -path=.:../foods --
-The library subdirectory present is a restricted version
-of the resource, with only present tense of verbs and sentences.
-
-By just changing the path, we get all tenses: -
-- --# -path=.:../foods:alltenses --
-Now we can see all the tenses of phrases, by using the -all flag
-in linearization:
-
- > gr | l -all - This wine is delicious - Is this wine delicious - This wine isn't delicious - Isn't this wine delicious - This wine is not delicious - Is this wine not delicious - This wine has been delicious - Has this wine been delicious - This wine hasn't been delicious - Hasn't this wine been delicious - This wine has not been delicious - Has this wine not been delicious - This wine was delicious - Was this wine delicious - This wine wasn't delicious - Wasn't this wine delicious - This wine was not delicious - Was this wine not delicious - This wine had been delicious - Had this wine been delicious - This wine hadn't been delicious - Hadn't this wine been delicious - This wine had not been delicious - Had this wine not been delicious - This wine will be delicious - Will this wine be delicious - This wine won't be delicious - Won't this wine be delicious - This wine will not be delicious - Will this wine not be delicious - This wine will have been delicious - Will this wine have been delicious - This wine won't have been delicious - Won't this wine have been delicious - This wine will not have been delicious - Will this wine not have been delicious - This wine would be delicious - Would this wine be delicious - This wine wouldn't be delicious - Wouldn't this wine be delicious - This wine would not be delicious - Would this wine not be delicious - This wine would have been delicious - Would this wine have been delicious - This wine wouldn't have been delicious - Wouldn't this wine have been delicious - This wine would not have been delicious - Would this wine not have been delicious --
-We also see -
--The list is even longer in languages that have more -tenses and moods, e.g. the Romance languages. -
-- -
- --Goals: -
-- -
- --Problem: to express conditions of semantic well-formedness. -
--Example: a voice command system for a "smart house" wants to -eliminate meaningless commands. -
--Thus we want to restrict particular actions to -particular devices - we can dim a light, but we cannot -dim a fan. -
--The following example is borrowed from the -Regulus Book (Rayner & al. 2006). -
--A simple example is a "smart house" system, which -defines voice commands for household appliances. -
-- -
- --Ontology: -
--Abstract syntax formalizing this: -
-- cat - Command ; - Kind ; - Device Kind ; -- argument type Kind - Action Kind ; - fun - CAction : (k : Kind) -> Action k -> Device k -> Command ; --
-Device and Action are both dependent types.
-
- -
- -
-Assume the kinds light and fan,
-
- light, fan : Kind ; - dim : Action light ; --
-Given a kind, k, you can form the device the k. -
-- DKindOne : (k : Kind) -> Device k ; -- the light --
-Now we can form the syntax tree -
-- CAction light dim (DKindOne light) --
-but we cannot form the trees -
-- CAction light dim (DKindOne fan) - CAction fan dim (DKindOne light) - CAction fan dim (DKindOne fan) -- -
- -
- --Concrete syntax does not know if a category is a dependent type. -
-
- lincat Action = {s : Str} ;
- lin CAction _ act dev = {s = act.s ++ dev.s} ;
-
-
-Notice that the Kind argument is suppressed in linearization.
-
-Parsing with dependent types is performed in two phases: -
-
-By just doing the first phase, the kind argument is not found:
-
- > parse "dim the light" - CAction ? dim (DKindOne light) --
-Moreover, type-incorrect commands are not rejected: -
-- > parse "dim the fan" - CAction ? dim (DKindOne fan) --
-The term ? is a metavariable, returned by the parser
-for any subtree that is suppressed by a linearization rule.
-These are the same kind of metavariables as were used here
-to mark incomplete parts of trees in the syntax editor.
-
- -
- -
-Use the command put_tree = pt with the option -typecheck:
-
- > parse "dim the light" | put_tree -typecheck - CAction light dim (DKindOne light) --
-The typecheck process may fail, in which case an error message
-is shown and no tree is returned:
-
- > parse "dim the fan" | put_tree -typecheck - - Error in tree UCommand (CAction ? 0 dim (DKindOne fan)) : - (? 0 <> fan) (? 0 <> light) -- -
- -
- --Sometimes an action can be performed on all kinds of devices. -
-
-This is represented as a function that takes a Kind as an argument
-and produce an Action for that Kind:
-
- fun switchOn, switchOff : (k : Kind) -> Action k ; --
-Functions of this kind are called polymorphic. -
--We can use this kind of polymorphism in concrete syntax as well, -to express Haskell-type library functions: -
-- oper const :(a,b : Type) -> a -> b -> a = - \_,_,c,_ -> c ; - - oper flip : (a,b,c : Type) -> (a -> b ->c) -> b -> a -> c = - \_,_,_,f,x,y -> f y x ; -- -
- -
- -
-1. Write an abstract syntax module with above contents
-and an appropriate English concrete syntax. Try to parse the commands
-dim the light and dim the fan, with and without solve filtering.
-
-2. Perform random and exhaustive generation, with and without
-solve filtering.
-
-3. Add some device kinds and actions to the grammar. -
-- -
- --Curry-Howard isomorphism = propositions as types principle: -a proposition is a type of proofs (= proof objects). -
--Example: define the less than proposition for natural numbers, -
-- cat Nat ; - fun Zero : Nat ; - fun Succ : Nat -> Nat ; --
-Define inductively what it means for a number x to be less than -a number y: -
-Zero is less than Succ y for any y.
-Succ x is less than Succ y.
-
-Expressing these axioms in type theory
-with a dependent type Less x y and two functions constructing
-its objects:
-
- cat Less Nat Nat ; - fun lessZ : (y : Nat) -> Less Zero (Succ y) ; - fun lessS : (x,y : Nat) -> Less x y -> Less (Succ x) (Succ y) ; --
-Example: the fact that 2 is less that 4 has the proof object -
-- lessS (Succ Zero) (Succ (Succ (Succ Zero))) - (lessS Zero (Succ (Succ Zero)) (lessZ (Succ Zero))) - : Less (Succ (Succ Zero)) (Succ (Succ (Succ (Succ Zero)))) -- -
- -
- --Idea: to be semantically well-formed, the abstract syntax of a document -must contain a proof of some property, -although the proof is not shown in the concrete document. -
--Example: documents describing flight connections: -
--To fly from Gothenburg to Prague, first take LH3043 to Frankfurt, then OK0537 to Prague. -
--The well-formedness of this text is partly expressible by dependent typing: -
-- cat - City ; - Flight City City ; - fun - Gothenburg, Frankfurt, Prague : City ; - LH3043 : Flight Gothenburg Frankfurt ; - OK0537 : Flight Frankfurt Prague ; --
-To extend the conditions to flight connections, we introduce a category -of proofs that a change is possible: -
-- cat IsPossible (x,y,z : City)(Flight x y)(Flight y z) ; --
-A legal connection is formed by the function -
-- fun Connect : (x,y,z : City) -> - (u : Flight x y) -> (v : Flight y z) -> - IsPossible x y z u v -> Flight x z ; -- -
- -
- --Above, all Actions were either of -
--To make this scale up for new Kinds, we can refine this to -restricted polymorphism: defined for Kinds of a certain class -
--The notion of class uses the Curry-Howard isomorphism as follows: -
-- -
- --We modify the smart house grammar: -
-- cat - Switchable Kind ; - Dimmable Kind ; - fun - switchable_light : Switchable light ; - switchable_fan : Switchable fan ; - dimmable_light : Dimmable light ; - - switchOn : (k : Kind) -> Switchable k -> Action k ; - dim : (k : Kind) -> Dimmable k -> Action k ; --
-Classes for new actions can be added incrementally. -
-- -
- --Mathematical notation and programming languages have -expressions that bind variables. -
--Example: universal quantifier formula -
-- (All x)B(x) --
-The variable x has a binding (All x), and
-occurs bound in the body B(x).
-
-Examples from informal mathematical language: -
-- for all x, x is equal to x - - the function that for any numbers x and y returns the maximum of x+y - and x*y - - Let x be a natural number. Assume that x is even. Then x + 3 is odd. -- -
- -
- --Abstract syntax can use functions as arguments: -
-- cat Ind ; Prop ; - fun All : (Ind -> Prop) -> Prop --
-where Ind is the type of individuals and Prop,
-the type of propositions.
-
-Let us add an equality predicate -
-- fun Eq : Ind -> Ind -> Prop --
-Now we can form the tree -
-- All (\x -> Eq x x) --
-which we want to relate to the ordinary notation -
-- (All x)(x = x) --
-In higher-order abstract syntax (HOAS), all variable bindings are -expressed using higher-order syntactic constructors. -
-- -
- --HOAS has proved to be useful in the semantics and computer implementation of -variable-binding expressions. -
--How do we relate HOAS to the concrete syntax? -
--In GF, we write -
-
- fun All : (Ind -> Prop) -> Prop
- lin All B = {s = "(" ++ "All" ++ B.$0 ++ ")" ++ B.s}
-
-
-General rule: if an argument type of a fun function is
-a function type A -> C, the linearization type of
-this argument is the linearization type of C
-together with a new field $0 : Str.
-
-The argument B thus has the linearization type
-
- {s : Str ; $0 : Str},
-
-
-If there are more bindings, we add $1, $2, etc.
-
- -
- --To make sense of linearization, syntax trees must be -eta-expanded: for any function of type -
-- A -> B --
-an eta-expanded syntax tree has the form -
-- \x -> b --
-where b : B under the assumption x : A.
-
-Given the linearization rule -
-
- lin Eq a b = {s = "(" ++ a.s ++ "=" ++ b.s ++ ")"}
-
--the linearization of the tree -
-- \x -> Eq x x --
-is the record -
-
- {$0 = "x", s = ["( x = x )"]}
-
--Then we can compute the linearization of the formula, -
-
- All (\x -> Eq x x) --> {s = "[( All x ) ( x = x )]"}.
-
-
-The linearization of the variable x is,
-"automagically", the string "x".
-
- -
- --GF can treat any one-word string as a variable symbol. -
-- > p -cat=Prop "( All x ) ( x = x )" - All (\x -> Eq x x) --
-Variables must be bound if they are used: -
-- > p -cat=Prop "( All x ) ( x = y )" - no tree found -- -
- -
- --1. Write an abstract syntax of the whole -predicate calculus, with the -connectives "and", "or", "implies", and "not", and the -quantifiers "exists" and "for all". Use higher-order functions -to guarantee that unbounded variables do not occur. -
--2. Write a concrete syntax for your favourite -notation of predicate calculus. Use Latex as target language -if you want nice output. You can also try producing boolean -expressions of some programming language. Use as many parenthesis as you need to -guarantee non-ambiguity. -
-- -
- -
-The fun judgements of GF are declarations of functions, giving their types.
-
-Can we compute fun functions?
-
-Mostly we are not interested, since functions are seen as constructors, -i.e. data forms - as usual with -
-- fun Zero : Nat ; - fun Succ : Nat -> Nat ; --
-But it is also possible to give semantic definitions to functions.
-The key word is def:
-
- fun one : Nat ; - def one = Succ Zero ; - - fun twice : Nat -> Nat ; - def twice x = plus x x ; - - fun plus : Nat -> Nat -> Nat ; - def - plus x Zero = x ; - plus x (Succ y) = Succ (Sum x y) ; -- -
- -
- --Computation: follow a chain of definition until no definition -can be applied, -
-- plus one one --> - plus (Succ Zero) (Succ Zero) --> - Succ (plus (Succ Zero) Zero) --> - Succ (Succ Zero) --
-Computation in GF is performed with the put_term command and the
-compute transformation, e.g.
-
- > parse -tr "1 + 1" | put_term -transform=compute -tr | l - plus one one - Succ (Succ Zero) - s(s(0)) -- -
- -
- --Two trees are definitionally equal if they compute into the same tree. -
--Definitional equality does not guarantee sameness of linearization: -
-- plus one one ===> 1 + 1 - Succ (Succ Zero) ===> s(s(0)) --
-The main use of this concept is in type checking: sameness of types. -
--Thus e.g. the following types are equal -
-- Less Zero one - Less Zero (Succ Zero)) --
-so that an object of one also is an object of the other. -
-- -
- -
-The judgement form data tells that a category has
-certain functions as constructors:
-
- data Nat = Succ | Zero ; --
-The type signatures of constructors are given separately, -
-- fun Zero : Nat ; - fun Succ : Nat -> Nat ; --
-There is also a shorthand: -
-- data Succ : Nat -> Nat ; === fun Succ : Nat -> Nat ; - data Nat = Succ ; --
-Notice: in def definitions, identifier patterns not
-marked as data will be treated as variables.
-
- -
- --1. Implement an interpreter of a small functional programming -language with natural numbers, lists, pairs, lambdas, etc. Use higher-order -abstract syntax with semantic definitions. As concrete syntax, use -your favourite programming language. -
-
-2. There is no termination checking for def definitions.
-Construct an examples that makes type checking loop.
-Type checking can be invoked with put_term -transform=solve.
-
- -
- --Goals: -
-- -
- --We construct a calculator with addition, subtraction, multiplication, and -division of integers. -
-
- abstract Calculator = {
-
- cat Exp ;
-
- fun
- EPlus, EMinus, ETimes, EDiv : Exp -> Exp -> Exp ;
- EInt : Int -> Exp ;
- }
-
-
-The category Int is a built-in category of
-integers. Its syntax trees integer literals, i.e.
-sequences of digits:
-
- 5457455814608954681 : Int --
-These are the only objects of type Int:
-grammars are not allowed to declare functions with Int as value type.
-
- -
- --We begin with a -concrete syntax that always uses parentheses around binary -operator applications: -
-
- concrete CalculatorP of Calculator = {
-
- lincat
- Exp = SS ;
- lin
- EPlus = infix "+" ;
- EMinus = infix "-" ;
- ETimes = infix "*" ;
- EDiv = infix "/" ;
- EInt i = i ;
-
- oper
- infix : Str -> SS -> SS -> SS = \f,x,y ->
- ss ("(" ++ x.s ++ f ++ y.s ++ ")") ;
- }
-
--Now we have -
-- > linearize EPlus (EInt 2) (ETimes (EInt 3) (EInt 4)) - ( 2 + ( 3 * 4 ) ) --
-First problems: -
-- -
- --The input of parsing in GF is not just a string, but a list of -tokens, returned by a lexer. -
--The default lexer in GF returns chunks separated by spaces: -
-- "(12 + (3 * 4))" ===> "(12", "+", "(3". "*". "4))" --
-The proper way would be -
-
- "(", "12", "+", "(", "3", "*", "4", ")", ")"
-
-
-Moreover, the tokens "12", "3", and "4" should be recognized as
-integer literals - they cannot be found in the grammar.
-
- -
-
-Lexers are invoked by flags to the command put_string = ps.
-
- > put_string -lexcode "(2 + (3 * 4))" - ( 2 + ( 3 * 4 ) ) --
-This can be piped into a parser, as usual: -
-- > ps -lexcode "(2 + (3 * 4))" | parse - EPlus (EInt 2) (ETimes (EInt 3) (EInt 4)) --
-In linearization, we use a corresponding unlexer: -
-- > linearize EPlus (EInt 2) (ETimes (EInt 3) (EInt 4)) | ps -unlexcode - (2 + (3 * 4)) -- -
- -
- -| lexer | -unlexer | -description | -|
|---|---|---|---|
chars |
-unchars |
-each character is a token | -|
lexcode |
-unlexcode |
-program code conventions (uses Haskell's lex) | -|
lexmixed |
-unlexmixed |
-like text, but between $ signs like code | -|
lextext |
-unlextext |
-with conventions on punctuation and capitals | -|
words |
-unwords |
-(default) tokens separated by space characters | -|
- -
- --Arithmetic expressions should be unambiguous. If we write -
-- 2 + 3 * 4 --
-it should be parsed as one, but not both, of -
-- EPlus (EInt 2) (ETimes (EInt 3) (EInt 4)) - ETimes (EPlus (EInt 2) (EInt 3)) (EInt 4) --
-We choose the former tree, because -multiplication has higher precedence than addition. -
--To express the latter tree, we have to use parentheses: -
-- (2 + 3) * 4 --
-The usual precedence rules: -
-1 + 2 + 3 means the same as (1 + 2) + 3.
-- -
- --Precedence can be made into an inherent feature of expressions: -
-
- oper
- Prec : PType = Ints 2 ;
- TermPrec : Type = {s : Str ; p : Prec} ;
-
- mkPrec : Prec -> Str -> TermPrec = \p,s -> {s = s ; p = p} ;
-
- lincat
- Exp = TermPrec ;
-
-
-Notice Ints 2: a parameter type, whose values are the integers
-0,1,2.
-
-Using precedence levels: compare the inherent precedence of an -expression with the expected precedence. -
--This idea is encoded in the operation -
-
- oper usePrec : TermPrec -> Prec -> Str = \x,p ->
- case lessPrec x.p p of {
- True => "(" x.s ")" ;
- False => x.s
- } ;
-
-
-(We use lessPrec from lib/prelude/Formal.)
-
- -
- --We can define left-associative infix expressions: -
-- infixl : Prec -> Str -> (_,_ : TermPrec) -> TermPrec = \p,f,x,y -> - mkPrec p (usePrec x p ++ f ++ usePrec y (nextPrec p)) ; --
-Constant-like expressions (the highest level): -
-- constant : Str -> TermPrec = mkPrec 2 ; --
-All these operations can be found in lib/prelude/Formal,
-which has 5 levels.
-
-Now we can write the whole concrete syntax of Calculator compactly:
-
- concrete CalculatorC of Calculator = open Formal, Prelude in {
-
- flags lexer = codelit ; unlexer = code ; startcat = Exp ;
-
- lincat Exp = TermPrec ;
-
- lin
- EPlus = infixl 0 "+" ;
- EMinus = infixl 0 "-" ;
- ETimes = infixl 1 "*" ;
- EDiv = infixl 1 "/" ;
- EInt i = constant i.s ;
- }
-
-
-- -
- -
-1. Define non-associative and right-associative infix operations
-analogous to infixl.
-
-2. Add a constructor that puts parentheses around expressions
-to raise their precedence, but that is eliminated by a def definition.
-Test parsing with and without a pipe to pt -transform=compute.
-
- -
- --Translate arithmetic (infix) to JVM (postfix): -
-- 2 + 3 * 4 - - ===> - - iconst 2 : iconst 3 ; iconst 4 ; imul ; iadd --
-Just give linearization rules for JVM: -
-
- lin
- EPlus = postfix "iadd" ;
- EMinus = postfix "isub" ;
- ETimes = postfix "imul" ;
- EDiv = postfix "idiv" ;
- EInt i = ss ("iconst" ++ i.s) ;
- oper
- postfix : Str -> SS -> SS -> SS = \op,x,y ->
- ss (x.s ++ ";" ++ y.s ++ ";" ++ op) ;
-
-
-- -
- --A straight code programming language, with -initializations and assignments: -
-- int x = 2 + 3 ; - int y = x + 1 ; - x = x + 9 * y ; --
-We define programs by the following constructors: -
-- fun - PEmpty : Prog ; - PInit : Exp -> (Var -> Prog) -> Prog ; - PAss : Var -> Exp -> Prog -> Prog ; --
-PInit uses higher-order abstract syntax for making the
-initialized variable available in the continuation of the program.
-
-The abstract syntax tree for the above code is -
-- PInit (EPlus (EInt 2) (EInt 3)) (\x -> - PInit (EPlus (EVar x) (EInt 1)) (\y -> - PAss x (EPlus (EVar x) (ETimes (EInt 9) (EVar y))) - PEmpty)) --
-No uninitialized variables are allowed - there are no constructors for Var!
-But we do have the rule
-
- fun EVar : Var -> Exp ; --
-The rest of the grammar is just the same as for arithmetic expressions
-here. The best way to implement it is perhaps by writing a
-module that extends the expression module. The most natural start category
-of the extension is Prog.
-
- -
- --1. Define a C-like concrete syntax of the straight-code language. -
-
-2. Extend the straight-code language to expressions of type float.
-To guarantee type safety, you can define a category Typ of types, and
-make Exp and Var dependent on Typ. Basic floating point expressions
-can be formed from literal of the built-in GF type Float. The arithmetic
-operations should be made polymorphic (as here).
-
-3. Extend JVM generation to the straight-code language, using -two more instructions -
-iload x, which loads the value of the variable x
-istore x which stores a value to the variable x
--Thus the code for the example in the previous section is -
-- iconst 2 ; iconst 3 ; iadd ; istore x ; - iload x ; iconst 1 ; iadd ; istore y ; - iload x ; iconst 9 ; iload y ; imul ; iadd ; istore x ; -- -
-4. If you made the exercise of adding floating point numbers to
-the language, you can now cash out the main advantage of type checking
-for code generation: selecting type-correct JVM instructions. The floating
-point instructions are precisely the same as the integer one, except that
-the prefix is f instead of i, and that fconst takes floating
-point literals as arguments.
-
- -
- --Goals: -
-- -
- --GF grammars can be used as parts of programs written in other programming -languages, to be called host languages. -This facility is based on several components: -
-- -
- --The portable format is called PGF, "Portable Grammar Format". -
-
-This format is produced by using GF as batch compiler, with the option -make,
-from the operative system shell:
-
- % gf -make SOURCE.gf --
-PGF is the recommended format in -which final grammar products are distributed, because they -are stripped from superfluous information and can be started and applied -faster than sets of separate modules. -
--Application programmers have never any need to read or modify PGF files. -
--PGF thus plays the same role as machine code in -general-purpose programming (or bytecode in Java). -
-- -
- --The Haskell API contains (among other things) the following types and functions: -
-- readPGF :: FilePath -> IO PGF - - linearize :: PGF -> Language -> Tree -> String - parse :: PGF -> Language -> Category -> String -> [Tree] - - linearizeAll :: PGF -> Tree -> [String] - linearizeAllLang :: PGF -> Tree -> [(Language,String)] - - parseAll :: PGF -> Category -> String -> [[Tree]] - parseAllLang :: PGF -> Category -> String -> [(Language,[Tree])] - - languages :: PGF -> [Language] - categories :: PGF -> [Category] - startCat :: PGF -> Category --
-This is the only module that needs to be imported in the Haskell application.
-It is available as a part of the GF distribution, in the file
-src/PGF.hs.
-
- -
- --Let us first build a stand-alone translator, which can translate -in any multilingual grammar between any languages in the grammar. -
-- module Main where - - import PGF - import System (getArgs) - - main :: IO () - main = do - file:_ <- getArgs - gr <- readPGF file - interact (translate gr) - - translate :: PGF -> String -> String - translate gr s = case parseAllLang gr (startCat gr) s of - (lg,t:_):_ -> unlines [linearize gr l t | l <- languages gr, l /= lg] - _ -> "NO PARSE" --
-To run the translator, first compile it by -
-- % ghc -make -o trans Translator.hs --
-For this, you need the Haskell compiler GHC. -
-- -
- -
-Then produce a PGF file. For instance, the Food grammar set can be
-compiled as follows:
-
- % gf -make FoodEng.gf FoodIta.gf --
-This produces the file Food.pgf (its name comes from the abstract syntax).
-
-The Haskell library function interact makes the trans program work
-like a Unix filter, which reads from standard input and writes to standard
-output. Therefore it can be a part of a pipe and read and write files.
-The simplest way to translate is to echo input to the program:
-
- % echo "this wine is delicious" | ./trans Food.pgf - questo vino è delizioso --
-The result is given in all languages except the input language. -
-- -
- -
-To avoid starting the translator over and over again:
-change interact in the main function to loop, defined as
-follows:
-
- loop :: (String -> String) -> IO () - loop trans = do - s <- getLine - if s == "quit" then putStrLn "bye" else do - putStrLn $ trans s - loop trans --
-The loop keeps on translating line by line until the input line
-is quit.
-
- -
- --The next application is also a translator, but it adds a -transfer component - a function that transforms syntax trees. -
--The transfer function we use is one that computes a question into an answer. -
--The program accepts simple questions about arithmetic and answers -"yes" or "no" in the language in which the question was made: -
-- Is 123 prime? - No. - 77 est impair ? - Oui. --
-We change the pure translator by giving
-the translate function the transfer as an extra argument:
-
- translate :: (Tree -> Tree) -> PGF -> String -> String --
-Ordinary translation as a special case where
-transfer is the identity function (id in Haskell).
-
-To reply in the same language as the question: -
-- translate tr gr = case parseAllLang gr (startCat gr) s of - (lg,t:_):_ -> linearize gr lg (tr t) - _ -> "NO PARSE" -- -
- -
- --Input: abstract syntax judgements -
-
- abstract Query = {
-
- flags startcat=Question ;
-
- cat
- Answer ; Question ; Object ;
-
- fun
- Even : Object -> Question ;
- Odd : Object -> Question ;
- Prime : Object -> Question ;
- Number : Int -> Object ;
-
- Yes : Answer ;
- No : Answer ;
- }
-
-
-- -
- --To make it easy to define a transfer function, we export the -abstract syntax to a system of Haskell datatypes: -
-- % gf --output-format=haskell Query.pgf --
-It is also possible to produce the Haskell file together with PGF, by -
-- % gf -make --output-format=haskell QueryEng.gf --
-The result is a file named Query.hs, containing a
-module named Query.
-
- -
--Output: Haskell definitions -
-- module Query where - import PGF - - data GAnswer = - GYes - | GNo - - data GObject = GNumber GInt - - data GQuestion = - GPrime GObject - | GOdd GObject - | GEven GObject - - newtype GInt = GInt Integer --
-All type and constructor names are prefixed with a G to prevent clashes.
-
-The Haskell module name is the same as the abstract syntax name. -
-- -
- --Haskell's type checker guarantees that the functions are well-typed also with -respect to GF. -
-- answer :: GQuestion -> GAnswer - answer p = case p of - GOdd x -> test odd x - GEven x -> test even x - GPrime x -> test prime x - - value :: GObject -> Int - value e = case e of - GNumber (GInt i) -> fromInteger i - - test :: (Int -> Bool) -> GObject -> GAnswer - test f x = if f (value x) then GYes else GNo -- -
- -
- --The generated Haskell module also contains -
-
- class Gf a where
- gf :: a -> Tree
- fg :: Tree -> a
-
- instance Gf GQuestion where
- gf (GEven x1) = DTr [] (AC (CId "Even")) [gf x1]
- gf (GOdd x1) = DTr [] (AC (CId "Odd")) [gf x1]
- gf (GPrime x1) = DTr [] (AC (CId "Prime")) [gf x1]
- fg t =
- case t of
- DTr [] (AC (CId "Even")) [x1] -> GEven (fg x1)
- DTr [] (AC (CId "Odd")) [x1] -> GOdd (fg x1)
- DTr [] (AC (CId "Prime")) [x1] -> GPrime (fg x1)
- _ -> error ("no Question " ++ show t)
-
--For the programmer, it is enough to know: -
-G
-gf translates from Haskell objects to GF trees
-fg translates from GF trees to Haskell objects
-- -
- -- module TransferDef where - - import PGF (Tree) - import Query -- generated from GF - - transfer :: Tree -> Tree - transfer = gf . answer . fg - - answer :: GQuestion -> GAnswer - answer p = case p of - GOdd x -> test odd x - GEven x -> test even x - GPrime x -> test prime x - - value :: GObject -> Int - value e = case e of - GNumber (GInt i) -> fromInteger i - - test :: (Int -> Bool) -> GObject -> GAnswer - test f x = if f (value x) then GYes else GNo - - prime :: Int -> Bool - prime x = elem x primes where - primes = sieve [2 .. x] - sieve (p:xs) = p : sieve [ n | n <- xs, n `mod` p > 0 ] - sieve [] = [] -- -
- -
- -
-Here is the complete code in the Haskell file TransferLoop.hs.
-
- module Main where - - import PGF - import TransferDef (transfer) - - main :: IO () - main = do - gr <- readPGF "Query.pgf" - loop (translate transfer gr) - - loop :: (String -> String) -> IO () - loop trans = do - s <- getLine - if s == "quit" then putStrLn "bye" else do - putStrLn $ trans s - loop trans - - translate :: (Tree -> Tree) -> PGF -> String -> String - translate tr gr s = case parseAllLang gr (startCat gr) s of - (lg,t:_):_ -> linearize gr lg (tr t) - _ -> "NO PARSE" -- -
- -
- -
-To automate the production of the system, we write a Makefile as follows:
-
- all: - gf -make --output-format=haskell QueryEng - ghc --make -o ./math TransferLoop.hs - strip math --
-(The empty segments starting the command lines in a Makefile must be tabs.) -Now we can compile the whole system by just typing -
-- make --
-Then you can run it by typing -
-- ./math --
-Just to summarize, the source of the application consists of the following files: -
-- Makefile -- a makefile - Math.gf -- abstract syntax - Math???.gf -- concrete syntaxes - TransferDef.hs -- definition of question-to-answer function - TransferLoop.hs -- Haskell Main module -- -
- -
- -
-PGF files can be used in web servers, for which there is a Haskell library included
-in src/server/. How to build a server for tasks like translators is explained
-in the README file in that directory.
-
-One of the servers that can be readily built with the library (without any
-programming required) is fridge poetry magnets. It is an application that
-uses an incremental parser to suggest grammatically correct next words. Here
-is an example of its application to the Foods grammars.
-
-
-
- -
- --JavaScript is a programming language that has interpreters built in in most -web browsers. It is therefore usable for client side web programs, which can even -be run without access to the internet. The following figure shows a JavaScript -program compiled from GF grammars as run on an iPhone. -
-
-
-
- -
- -
-JavaScript is one of the output formats of the GF batch compiler. Thus the following
-command generates a JavaScript file from two Food grammars.
-
- % gf -make --output-format=js FoodEng.gf FoodIta.gf --
-The name of the generated file is Food.js, derived from the top-most abstract
-syntax name. This file contains the multilingual grammar as a JavaScript object.
-
- -
- -
-To perform parsing and linearization, the run-time library
-gflib.js is used. It is included in GF/lib/javascript/, together with
-some other JavaScript and HTML files; these files can be used
-as templates for building applications.
-
-An example of usage is
-translator.html,
-which is in fact initialized with
-a pointer to the Food grammar, so that it provides translation between the English
-and Italian grammars:
-
-
-
-The grammar must have the name grammar.js. The abstract syntax and start
-category names in translator.html must match the ones in the grammar.
-With these changes, the translator works for any multilingual grammar.
-
- -
- --The standard way of using GF in speech recognition is by building -grammar-based language models. -
--GF supports several formats, including -GSL, the formatused in the Nuance speech recognizer. -
-
-GSL is produced from GF by running gf with the flag
---output-format=gsl.
-
-Example: GSL generated from FoodsEng.gf.
-
- % gf -make --output-format=gsl FoodsEng.gf
- % more FoodsEng.gsl
-
- ;GSL2.0
- ; Nuance speech recognition grammar for FoodsEng
- ; Generated by GF
-
- .MAIN Phrase_cat
-
- Item_1 [("that" Kind_1) ("this" Kind_1)]
- Item_2 [("these" Kind_2) ("those" Kind_2)]
- Item_cat [Item_1 Item_2]
- Kind_1 ["cheese" "fish" "pizza" (Quality_1 Kind_1)
- "wine"]
- Kind_2 ["cheeses" "fish" "pizzas"
- (Quality_1 Kind_2) "wines"]
- Kind_cat [Kind_1 Kind_2]
- Phrase_1 [(Item_1 "is" Quality_1)
- (Item_2 "are" Quality_1)]
- Phrase_cat Phrase_1
-
- Quality_1 ["boring" "delicious" "expensive"
- "fresh" "italian" ("very" Quality_1) "warm"]
- Quality_cat Quality_1
-
-
-- -
- -
-Other formats available via the --output-format flag include:
-
| Format | -Description | -|
|---|---|---|
gsl |
-Nuance GSL speech recognition grammar | -|
jsgf |
-Java Speech Grammar Format (JSGF) | -|
jsgf_sisr_old |
-JSGF with semantic tags in SISR WD 20030401 format | -|
srgs_abnf |
-SRGS ABNF format | -|
srgs_xml |
-SRGS XML format | -|
srgs_xml_prob |
-SRGS XML format, with weights | -|
slf |
-finite automaton in the HTK SLF format | -|
slf_sub |
-finite automaton with sub-automata in HTK SLF | -|
-All currently available formats can be seen with gf --help.
-