+
+GF Resources for Swedish
+ ++ +Språkdata Seminar, Gothenburg, 1 March 2005 + +
+ +Aarne Ranta + +
+ +aarne@cs.chalmers.se +
Plan
+ +Introduction to resource grammars (pp. 1-16) + ++ +Swedish morphology and lexicon in GF + +
+ +Syntax case study: Swedish sentence structure + +
+ +Danish and Norwegian through parametrization + + + + +
Swedish morphology and lexicon
+ +Lexicon: list of words with inflection and other morphological +information (gender of nouns etc). + ++ +Paradigms: set of functions for extending the lexicon. + + + +
Parts of speech
+ +A word class is a record type, with +parametric and inherent features (parameters). +For example, nouns are the type +
+ N = {s : Number => Species => Case => Str ; g : Gender} ;
+
+where
++ param + Species = Indef | Def ; + Number = Sg | Pl ; + Case = Nom | Gen ; ++ + + + +
Defining a lexical unit
+ +Every lexical unit has a word class as its type. +The type checker of GF verifies that all and only the +relevant information of the unit is given. For instance, +an entry for the noun bil ("car") looks as follows. +
+ bil =
+ {s = table {
+ Sg => table {
+ Indef => table {Nom => "bil" ; Gen => "bils" } ;
+ Def => table {Nom => "bilen" ; Gen => "bilens" }
+ } ;
+ Pl => table {
+ Indef => table {Nom => "bilar" ; Gen => "bilars" } ;
+ Def => table {Nom => "bilarna" ; Gen => "bilarnas" }
+ }
+ } ;
+ g = Utr
+ }
+
+
+
+
+The Golden Rule of Functional Programming
+ +Whenever you find yourself programming by "copy and paste", write +a function instead. + ++ +Thus do not write +
+ gran =
+ {s = table {
+ Sg => table {
+ Indef => table {Nom => "gran" ; Gen => "grans" } ;
+ Def => table {Nom => "granen" ; Gen => "granens" }
+ } ;
+ Pl => table {
+ Indef => table {Nom => "granar" ; Gen => "granars" } ;
+ Def => table {Nom => "granarna" ; Gen => "granarnas" }
+ }
+ } ;
+ g = Utr
+ }
+
+
+
+
+
+Inflectional paradigms as functions
+ +Instead, write a paradigm that can be applied to any word +that is "inflected in the same way": +
+ decl2 : Str -> N = \bil ->
+ {s = table {
+ Sg => table {
+ Indef => table {Nom => bil + "" ; Gen => bil + "s" } ;
+ Def => table {Nom => bil + "en" ; Gen => bil + "ens" }
+ } ;
+ Pl => table {
+ Indef => table {Nom => bil + "ar" ; Gen => bil + "ars" } ;
+ Def => table {Nom => bil + "arna" ; Gen => bil + "arnas" }
+ }
+ } ;
+ g = Utr
+ }
+
+This function can be used over and over again:
++ bil = decl2 "bil" ; + gran = decl2 "gran" ; + dag = decl2 "dag" ; ++ + + + +
High-level definition of paradigms
+ +Recall: functions instead of copy-and-paste! + ++ +First define (for each word class) a worst-case function: +
+ mkN : (apa,apan,apor,aporna : Str) -> Noun =
+ {s = table {
+ Sg => table {
+ Indef => mkCase apa ;
+ Def => mkCase apan
+ } ;
+ Pl => table {
+ Indef => mkCase apor ;
+ Def => mkCase aporna
+ }
+ } ;
+ g = case last apan of {
+ "n" => Utr ;
+ _ => Neutr
+ }
+
+where we uniformly produce the genitive by
+
+ mkCase : Str -> Case => Str = \f -> table {
+ Nom => f ;
+ Gen => f + case last f of {
+ "s" | "x" => [] ;
+ _ => "s"
+ }
+ } ;
+
+
+
+
+
+High-level definition of paradigms
+ +Then define, for instance, the five declensions as follows: ++ decl1 : Str -> N = \apa -> let ap = init apa in + mkN apa (apa + "n") (ap + "or") (ap + "orna") ; + + decl2 : Str -> N = \bil -> + mkN bil (bil + "en") (bil + "ar") (bil + "arna") ; + + decl3 : Str -> N = \fil -> + mkN fil (fil + "en") (fil + "er") (fil + "erna") ; + + decl4 : Str -> N = \rike -> + mkN rike (rike + "t") (rike + "n") (rik + "ena") ; + + decl5 : Str -> N = \lik -> + mkN lik (lik + "et") lik (lik + "en") ; ++ + + + + +
What paradigms are there?
+ +Swedish nouns traditionally have 5 declensions. But each of them has +slight variations. For instance, the "2nd declension" has the following: ++ gosse - gossar -- 211 + nyckel - nycklar -- 231 + seger - segrar -- 232 + öken - öknar -- 233 + hummer - humrar -- 238 + kam - kammar -- 241 + mun - munnar -- 243 ++and many more (S. Hellberg, The Morphology of Present-Day Swedish, +Almqvist & Wiksell, Stockholm, 1978). In addition, we have at least +
+ mås - mås -- genitive form without s + sax - sax ++ + + + + + +
High-level access to paradigms
+ +The "naïve user" does not want to go through 500 noun paradigms and +pick the right one. + ++ +A much more efficient method is the one used in +dictionaries: give two (or more) forms instead of one. +Our "dictionary heuristic function" covers the following cases: +
+ flicka - flickor + kor - kor (koret) + ko - kor (kon) + ros - rosor (rosen) + bil - bilar + nyckel - nycklar + hummer - humrar + rike - riken + lik - lik (liket) + lärare - lärare (läraren) ++ + + + + + +
The definition of the dictionary heuristic
+ +
+reg2Noun : Str -> Str -> Subst = \bil,bilar ->
+ let
+ l = last bil ;
+ b = Predef.tk 2 bil ;
+ ar = Predef.dp 2 bilar
+ in
+ case ar of {
+ "or" => case l of {
+ "a" => decl1Noun bil ;
+ "r" => sLik bil ;
+ "o" => mkNoun bil (bil + "n") bilar (bilar + "na") ;
+ _ => mkNoun bil (bil + "en") bilar (bilar + "na")
+ } ;
+ "ar" => ifTok Subst (Predef.tk 2 bilar) bil
+ (decl2Noun bil)
+ (case l of {
+ "e" => decl2Noun bil ;
+ _ => mkNoun bil (bil + "n") bilar (bilar + "na")
+ }
+ ) ;
+ "er" => decl3Noun bil ;
+ "en" => ifTok Subst bil bilar (sLik bil) (sRike bil) ; -- ben-ben
+ _ => ifTok Subst bil bilar (
+ case Predef.dp 3 bil of {
+ "are" => sKikare (init bil) ;
+ _ => decl5Noun bil
+ }
+ )
+ (decl5Noun bil) --- rest case with lots of garbage
+ } ;
+
+
+
+
+When in doubt...
+ +Test in GF by generating the table +
+ > cc mk2N "öken" "öknar"
+ {s = table Number {
+ Sg => table {
+ Indef => table Case {
+ Nom => "öken" ;
+ Gen => "ökens"
+ } ;
+ Def => table Case {
+ Nom => "ökenn" ;
+ Gen => "ökenns"
+ }
+ ...
+ }
+
+Use the worst-case function if the heuristic does not work:
++ mkN "öken" "öknen" "öknar" "öknarna" ++ + + + +
The module ParadigmsSwe
+ +For main category - N, A, V - a worst-case +function and a couple of "regular" patterns. ++ mkN : (apa,apan,apor,aporna : Str) -> N ; + mk2N : (nyckel,nycklar : Str) -> N ; + + mkV : (supa,super,sup,söp,supit,supen : Str) -> V ; + regV : (tala : Str) -> V ; + mk2V : (leka,leker : Str) -> V ; + irregV : (dricka, drack, druckit : Str) -> V ; ++Construction functions for subcategorization. +
+ mkV2 : V -> Preposition -> V2 ; + dirV2 : V -> V2 ; + mkV3 : V -> Preposition -> Preposition -> V3 ; ++ + + + +
Morphology extraction
+ +Idea: search for characteristic forms of paradigms in a corpus. +
+ paradigm decl1
+ = ap+"a"
+ {ap+"a" & ap+"or" };
+
+For instance, if you find klocka and klockor, add
++ klocka_N = decl1 "klocka" ; ++to the lexicon. + +
+ +The notation for extraction and its implementation are +developed by Markus Forsberg and Harald Hammarström. + + + + + +
False positives
+ +Problem: false positives, e.g. bra - bror. + ++ +Solution: restrict stem with a regular expression +
+ paradigm decl1 [ap : char* vowel char*]
+ = ap+"a"
+ {ap+"a" & ap+"or" };
+
+In general, exclude stems shorter than 3 characters.
+
++ +It is necessary to check the results manually. + + + + +
Patterns for what?
+ +"Irregular patterns" are possible, e.g. +
+ paradigm vEI [sm:OneOrMore, t:OneOrMore]
+ = sm+"i"+t+"a"
+ {sm+"i"+t+"a" & (sm+"e"+t | sm+"i"+t+"it")} ;
+
+For rare patterns, it is more productive to build the
+corresponding part of lexicon manually.
+
+
+
+
+Current Swedish resource lexicon
+ +49,749 lemmas (1,000 manual, rest extracted), +605,453 word forms. +No subcategorization information. + ++ +Uses the + +Functional Morphology +format, which can be translated to GF, XFST, LEXC, +MySQL,... + +
+ +FM's "native" analysis engine is based on a trie +and includes compound analysis using algorithms +from G. Huet's +Zen Toolkit. +Analysis speed is 12,000 words per minute +with compound analysis, 50,000 without +(on an Intel M1.5 GHz laptop). + + + + + +