diff --git a/.authorspellings b/.authorspellings deleted file mode 100644 index e289b9482..000000000 --- a/.authorspellings +++ /dev/null @@ -1,23 +0,0 @@ --- Use darcs show authors to see the number of patches per person - -Aarne Ranta , aarne@cs.chalmers.se, aarne -Björn Bringert , bringert@cs.chalmers.se, bringert -Krasimir Angelov , kr.angelov@chalmers.se,kr.angelov@gmail.com,kr_angelov@gmail.com -Peter Ljunglöf , peb,peb@cs.chalmers.se -Thomas Hallgren , hallgren@altocumulus.org -Janna Khegai , janna,janna@cs.chalmers.se -Jordi Saludes -Ramona Enache , ra.monique@gmail.com -Hans-Joachim Daniels -Jean-Philippe Bernardy -Kevin Kofler -Harald Hammarström -Olga Caprotti , olga.caprotti@gmail.com -Adam Slaski , Adam Slaski -Moisés Salvador Meza Moreno, 'Moises Meza ' -Ali El Dada , 'ali.eldada@gmail.com' -Markus Forsberg -Server Cimen -Grégoire Détrez -Ketil Malde -John J. Camilleri , john@johnjcamilleri.com, john diff --git a/css/oldstyle.css b/css/oldstyle.css deleted file mode 100644 index fcc422975..000000000 --- a/css/oldstyle.css +++ /dev/null @@ -1,43 +0,0 @@ - -/* Found it at http://article.olduse.net/771@mit-eddie.UUCP */ -@font-face { - font-family: GlassTTYVT220; - src: local("Glass TTY VT220"), local("Glass TTY VT220 Medium"), url(/font/Glass_TTY_VT220.ttf) format("truetype"); -} - -body { - font-size: 20px; - font-family: GlassTTYVT220; - color: #e5b800; - background-color: #0a0600; -} - -h1,h2,h3,h4 { font-size: 20px; } - -img.logo, img.grow { display: none; } - -a { color: #ecc800; } -a:hover { color: #fcd800; } - -div.links { - display: inline-block; - vertical-align: top; - margin-right: 1em; -} - -table.news td { - background: inherit; - border: 0; - vertical-align: baseline; - padding: 0.25ex 0.5ex; -} - -table.news td:nth-child(1) { - /* Don't want the hyphens in dates (2012-06-29) to cause line breaks */ - white-space: nowrap; -} - -ol.languages { - column-width: 12em; - -moz-column-width: 12em; -} diff --git a/css/style0.css b/css/style0.css index 6e5500a17..3c53c5378 100644 --- a/css/style0.css +++ b/css/style0.css @@ -17,8 +17,10 @@ h1 img.nofloat { float: none; } img.right { float: right; } ol.languages { - column-width: 12em; - -moz-column-width: 12em; + display: flex; + flex-direction: column; + flex-wrap: wrap; + height: 12em; } .grow { diff --git a/demos/index-numbers.html b/demos/index-numbers.html deleted file mode 100644 index 930d6895b..000000000 --- a/demos/index-numbers.html +++ /dev/null @@ -1,43 +0,0 @@ - - - - -Portable Numeral Translator - -

Portable Numeral Translator

- -15 August 2008
-
- -

-Here is the first GF application for the iPhone. It translates -numerals from digits into number words in 15 languages. It works for -numerals in the range 1-999,999. You can use the demo online in your iPhone's -browser, or download and install it by using FileMagnet (from AppStore). -

-

-The demo should work in any web browser supporting JavaScript, but it has been -specially dimensioned for the iPhone screen. It is an application of a general -technology, namely the compilation of GF grammars to JavaScript. Thus you can -easily build similar applications from your own grammars. -

-

-The -full numeral translator, which -works in 88 language and permits arbitrary cross translations, cannot yet be ported -to the iPhone because of memory problems. -

-

-The current version has some encoding/font issues in Hebrew and Hindi. -

-

-Use the translator online -

-

-Download the translator. To install: unpack the file and move -the directory numbers to your FileMagnet uploader. -

- - - - diff --git a/doc/gf-bibliography.html b/doc/gf-bibliography.html deleted file mode 100644 index 4281f9460..000000000 --- a/doc/gf-bibliography.html +++ /dev/null @@ -1,875 +0,0 @@ - - - - - - -Grammatical Framework Bibliography - -
-

Grammatical Framework Bibliography

-Aarne Ranta
-Tue Nov 27 16:47:06 2012 -
- -

-
-

- - - -

-
-

- - -

Publications on GF

- -

-In reverse temporal order: -

-

-H. Burden and R. Heldal. -Natural Language Generation from Class Diagrams. -Proceedings of the 8th International Workshop on Model-Driven Engineering, Verification and Validation (MoDeVVa 2011), -Wellington, New Zealand, -ACM, -2011. -Draft PDF -
-Using GF to verbalize platform-independent requirement specifications as a part of a software engineering tool chain. -

-

-C. España-Bonet, R. Enache, A. Slaski, A. Ranta, L. Màrquez, and M. Gonzàlez. -Patent translation within the MOLTO project. -Proceedings of the 4th Workshop on Patent Translation, -MT Summit XIII, Xiamen, China, September 23, 2011. -PDF -
-First steps to a hybrid GF-SMT system. -

-

-Seyed M. Montazeri, Nivir Roy, and Gerardo Schneider. -From Contracts in Structured English to CL Specifications. -5th International Workshop on Formal Languages and Analysis of Contract-Oriented Software (FLACOS'11), -volume 68 of EPTCS, pages 55-69, -Málaga, Spain, 22-23 September 2011. -PDF/DOI -
-Formal analysis of contracts written in controlled English. -

-

-S. Virk, M. Humayoun, and A. Ranta. -An Open-Source Punjabi Resource Grammar. -Proceedings of RANLP-2011, Recent Advances in Natural Language Processing, -Hissar, Bulgaria, 12-14 September, 2011. -pp. 70-76. -PDF -
-Punjabi syntax, building on Humayoun and Ranta 2010. -

-

-D. Dannélls, M. Damova, R.Enache, M. Chechev. -A Framework for Improved Access to Museum Databases in the Semantic Web. -Proceedings of the Workshop on Language Technologies for Digital Humanities and Cultural Heritage, -RANLP-2011, Recent Advances in Natural Language Processing, -Hissar, Bulgaria, 12-14 September, 2011, -pp 8-15 -PDF -
-Representing and verbalizing in GF of an Cultural Heritage ontology. -

-

-A. Ranta. -Translating between Language and Logic: What Is Easy and What is Difficult? -In N. Bjørner and V. Sofronie-Stokkermans (eds.), -CADE-23. Automated Deduction, -LNCS/LNAI 6803, -pp. 5-25, -2011. -PDF at Springer -Draft PDF -
-Invited talk at the CADE conference, showing how GF can help writing natural language interfaces -to proof systems, and also exploring how to make the language richer, e.g. how to get from -"for all numbers x, x is even or x is odd" to "every number is even or odd". -

-

-A. Ranta. -Grammatical Framework: Programming with Multilingual Grammars, - CSLI Publications, Stanford, 2011. - Book web page -
-Meant as the standard reference and textbook on GF. -

-

-O. Caprotti, K. Angelov, R. Enache, T. Hallgren, A. Ranta. -The MOLTO Phrasebook. -Swedish Language Technology Conference SLTC 2010. -PDF -
-Tourist Phrasebook grammar, available in 14 European languages. -

-

-G. Détrez and R. Enache. -A Framework for Multilingual Applications on the Android Platform -Swedish Language Technology Conference SLTC 2010. -PDF -
-A description of the Java implementation of the GF runtime system, which makes it possible -to create grammar-based applications running on an Android phone.// -

-

-Normunds Gruzitis and Guntis Barzdins. -Towards a More Natural Multilingual Controlled Language Interface to OWL, -9th International Conference on Computational Semantics (IWCS), -pp. 335-339, -2011. -PDF -
-Access to ontologies in English and Latvian. -

-

-M. Humayoun and A. Ranta. -Developing Punjabi Morphology, Corpus and Lexicon. -The 24th Pacific Asia conference on Language, Information and Computation (PACLIC24), -2010. -
-Yet another South-East Asian language, spoken by 88 million people. -

-

-J. Camilleri, Gordon J. Pace, and Mike Rosner. -Playing Nomic using a Controlled Natural Language. -CNL 2010, Controlled Natural Language, -Marettimo, -2010. -
-Using GF for defining the rules of a game. -

-

-R. Enache and K. Angelov -Typeful Ontologies with Direct Multilingual Verbalization. -CNL 2010, Controlled Natural Language, -Marettimo, -2010. -
-Investigating the benefits of using GF as a language for representing ontologies. -PDF -

-

-S. Virk, M. Humayoun, and A. Ranta. -An Open Source Urdu Resource Grammar. -Proceedings of the 8th Workshop on Asian Language Resources (Coling 2010 workshop), -2010. -
-The national language of Pakistan, spoken by 60 million people; almost the same as Hindi. -

-

-A. Ranta, K. Angelov, and T. Hallgren. -Tools for multilingual grammar-based translation on the web. -Proceedings of the ACL 2010 System Demonstrations, -ACM Digital Library, -2010. -PDF -
-An overview of GF for developers and users of translation systems. -

-

-D. Dannélls and J. Camilleri. -Verb Morphology of Hebrew and Maltese - Towards an Open Source Type Theoretical Resource Grammar in GF. -Proceedings of the Language Resources (LRs) and Human Language Technologies (HLT) for Semitic Languages Status, Updates, and Prospects, LREC-2010 Workshop, -Malta, pp. 57-61. -2010. -PDF -
-A study of Semitic non-concatenative morphology from the GF point of view. -

-

-M. Humayoun and C. Raffalli. -MathNat - Mathematical Text in a Controlled Natural Language. -Special issue: Natural Language Processing and its Applications. Journal on Research in Computing Science, Volume 46. -2010. -
-Natural language interface to a proof system, implemented in GF. -

-

-D. Dannélls. -Discourse Generation from Formal Specifications Using the Grammatical Framework, GF. -Special issue: Natural Language Processing and its Applications. Journal on Research in Computing Science (RCS), -volume 46. pp. 167-178, -2010. -
-Interfacing GF with ontology, with a natural language generation perspective. -

-

-R. Enache, A. Ranta, and K. Angelov. -An Open-Source Computational Grammar of Romanian. -A. Gelbukh (ed.), CiCLING-2010, -LNCS 6008, -2010. -
-A Romance language different enough not to follow the Romance functor. -

-

-K. Angelov and A. Ranta. -Implementing Controlled Languages in GF. -N. Fuchs (ed.), CNL-2009 Controlled Natural Languages, -LNCS/LNAI 5972, -2010. -

-

-K. Angelov, B. Bringert and A. Ranta. -PGF: A Portable Run-time Format for Type-theoretical Grammars, -Journal of Logic, Language and Information, -19(2), -pp. 201-228, -2009. -SpringerLink -

-

-A. Ranta. -The GF Resource Grammar Library. -Linguistic Issues in Language Technology, -2 (2), -2009. -PDF -
-A systematic presentation of the library from the linguistic point of view. -

-

-A. Ranta. -Grammatical Framework: A Multilingual Grammar Formalism, -Language and Linguistics Compass, -Vol. 3, on-line -2009. -
-An overview of GF for readers with a general academic brackground. -

-

-A. Ranta and K. Angelov. -Implementing Controlled Languages in GF. -CNL-2009, -CEUR Workshop Proceedings, -vol. 448, -on-line, -2009. -
-Makes a case for using GF in controlled language implementation, illustrated by Attempto Controlled English ported to French, German, and Swedish; longer version in 2010. -

-

-A. Ranta. Grammars as Software Libraries. -In Y. Bertot, G. Huet, J-J. Lévy, and G. Plotkin (eds.), -From Semantics to Computer Science, -Cambridge University Press, Cambridge, -pp. 281-308, -2009. -pdf (preliminary version) -
-Grammar-libraries from the software engineering point of view, with an example application to mathematical language. -

-

-K. Angelov. -Incremental Parsing in Parallel Multiple Context-Free Grammars. -EACL 2009. -
-Describes the algorithm used in parsing with GF. -

-

-A. Ranta, B. Bringert, and K. Angelov. -The GF Grammar Development Environment. -System demo. Proceedings of EACL-2009, -2009. -PDF -
-An overview of GF from the grammarian's point of view. -

-

-B. Bringert, K. Angelov, and A. Ranta. -Grammatical Framework Web Service, -System demo. Proceedings of EACL-2009, -2009. -PDF -
-An overview of how to build web services on top of PGF using the Google Web Toolkit. -

-

-A. Ranta and K. Angelov. -Implementing Controlled Languages in GF, -To appear in the proceedings of CNL-2009, LNCS, Springer, -2009. -
-On the use of GF for controlled languages, exemplified by an implementation of Attempto Controlled English then ported to three other language. -

-

-R. Cooper and A. Ranta. -Natural Languages as Collections of Resources. -In Language in Flux: Dialogue Coordination, Language Variation, Change, -ed. by R. Cooper and R. Kempson, pp. 109-120. London: College Publications, -2008. -
-The resource grammar idea applied to language learning and evolution. -

-

-Moisés Salvador Meza Moreno and B. Bringert. -Interactive Multilingual Web Applications with Grammatical Framework. -In B. Nordström and A. Ranta (eds), -Advances in Natural Language Processing (GoTAL 2008), -LNCS/LNAI 5221, Springer, -2008. -
-Shows how GF compiled to JavaScript is used in dynamic multilingual web pages. -

-

-Peter Ljunglöf and Staffan Larsson. -A grammar formalism for specifying ISU-based dialogue systems. -In B. Nordström and A. Ranta (eds), -Advances in Natural Language Processing (GoTAL 2008), -LNCS/LNAI 5221, Springer, -2008. -pdf -
-Explains how GoDiS dialogue systems are specified by GF grammars. -

-

-K. Angelov. -Type-Theoretical Bulgarian Grammar. -In B. Nordström and A. Ranta (eds), -Advances in Natural Language Processing (GoTAL 2008), -LNCS/LNAI 5221, Springer, -2008. -
-Explains the implementation of a Bulgarian resource grammar in GF. -

-

-B. Bringert. -High-precision Domain-specific Interlingua-based Speech Translation -with Grammatical Framework. -Coling 2008 Workshop on Speech Translation for Medical and Other Safety-Critical Applications, -Manchester, UK, August 23, -2008. -
-Shows how to build spoken language translators based on GF grammars and their compilation to Nuance. -

-

-A. Ranta. -How predictable is Finnish morphology? An experiment on lexicon construction. -In J. Nivre, M. Dahllöf and B. Megyesi (eds), -Resourceful Language Technology: Festschrift in Honor of Anna Sågvall Hein, -University of Uppsala, -2008. -Available from series homepage -
-Presents an experiment on smart paradigms in Finnish. -

-

-A. Ranta. -Example-based grammar writing. -To appear in L. Borin and S. Larsson (eds), -Festschrift for Robin Cooper, -2007. -
-Presents and discusses the ideas of grammar composition and example-based grammar writing. -

-

-B. Bringert. -Rapid Development of Dialogue Systems by Grammar Compilation. -8th SIGdial Workshop on Discourse and Dialogue, -Antwerp, Belgium, September 1-2, -2007. -pdf -
-Shows how to build a web-based spoken dialogue system by generating VoiceXML and JavaScript. -

-

-A. El Dada and A. Ranta. -Implementing an Open Source Arabic Resource Grammar in GF. -In M. Mughazy (ed), -Perspectives on Arabic Linguistics XX. Papers from the Twentieth Annual Symposium on Arabic Linguistics, Kalamazoo, March 26 -John Benjamins Publishing Company. -2007. -
-An outline of the Arabic resource grammar project, focusing on linguistic aspects. -

-

-A. El Dada. -Implementation of the Arabic Numerals and their Syntax in GF. -Computational Approaches to Semitic Languages: Common Issues and Resources, - ACL-2007 Workshop, -June 28, 2007, Prague. -2007. -
-A case study with the resource grammar, focusing on the morphosyntax -and agreement of constructions with numerals. -

-

-A. Ranta. -Modular Grammar Engineering in GF. -Research on Language and Computation, -5:133-158, 2007. -Draft available as pdf. -
-Adapts library-based software engineering methods to grammar writing -and introduces the module system of GF. -

-

-A. Ranta. -The GF Grammar Compiler. -Workshop on New Directions in Type-theoretic Grammars, -Dublin, August 2007 (ESSLLI workshop). -2007. -pdf -
-Describes the compilation of GF source code to lower-level run-time formats. -

-

-M. Humayoun, H. Hammarström, and A. Ranta. -Urdu Morphology, Orthography and Lexicon Extraction. -CAASL-2: The Second Workshop on Computational Approaches to Arabic Script-based Languages, -July 21-22, 2007, LSA 2007 Linguistic Institute, Stanford University. -2007. -
-Fairly complete open-source Urdu morphology and elemenraty syntax in GF. -

-

-N. Perera and A. Ranta. -Dialogue System Localization with the GF Resource Grammar Library. -SPEECHGRAM 2007: ACL Workshop on Grammar-Based Approaches to Spoken Language Processing, -June 29, 2007, Prague. -2007. -pdf -
-An experiment in porting an in-car dialogue system from two to six languages. -

-

-B. Bringert. -Speech Recognition Grammar Compilation in Grammatical Framework -SPEECHGRAM 2007: ACL Workshop on Grammar-Based Approaches to Spoken Language Processing, -June 29, 2007, Prague. -2007. -
-Generation of speech recognition language models from GF in several formats: -GSL (Nuance), SRGS, JSGF, and HTK SLF, with embedded semantic interpretation. -

-

-A. Ranta. -Features in Abstract and Concrete Syntax. -The 2nd International Workshop on Typed Feature Structure Grammars, -Tartu, 24 May 2007 (NODALIDA workshop). -2007. -pdf -
-Explores the design choices of incorporating features in a GF-like grammar, -with comparisons to feature-based unification grammars. -

-

-O. Caprotti and M. Seppälä. -Multilingual Delivery of Online Tests in mathematics. -Proceedings of Online Educa Berlin 2006. 29 November - 1 December 2006. - Berlin, Germany. -2006. -pdf -
-This papers shows screenshots of multilingual generation in the -WebALT project, using GF and the resource grammar library. -

-

-J. Khegai. -Language engineering in Grammatical Framework (GF). -Phd thesis, Computer Science, Chalmers University of Technology, -2006. -pdf -
-Collection of articles and technical reports on multilingual authoring -and the Russian resource grammar. -

-

-B. Bringert and A. Ranta. -A Pattern for Almost Compositional Functions. -ICFP 2006, The 11th ACM SIGPLAN International Conference on Functional Programming, Portland, Oregon, September 18-20, 2006, -2006. -pdf. -
-A method of generic programming useful for compiler construction -and transfer-based translation. -

-

-M. Forsberg, H. Hammarstrom, and A. Ranta. -Morphological Lexicon Extraction from Raw Text Data. -FinTAL 2006, -Turku, August 23-25, 2006. -Springer LNCS/LNAI 4139, -pp. 488-499, -2006. -
-A method for automatical production of morphological lexica based -on inflection engines such as those of GF resource grammar library. -

-

-A. Ranta. -Type Theory and Universal Grammar. -Philosophia Scientiae, Constructivism: Mathematics, Logic, Philosophy and Linguistics, -cahier spécial 6, -pp. 115-131, -2006. -
-A philosophical study of the medieval thesis that -grammar is the same in all languages and the difference is only in words. -

-

-J. Khegai. -GF parallel resource grammars and Russian. -In proceedings of ACL2006 - (The joint conference of the International Committee on Computational - Linguistics and the Association for Computational Linguistics) (pp. 475-482), - Sydney, Australia, July 2006. -
-Gives an outline of the Russian resource grammar project. -

-

-J. Khegai. -Grammatical Framework (GF) for MT in sublanguage domains. -Proceedings of 11th Annual conference of the European Association for - Machine Translation, , Oslo. -pp. 95-104, -2005. -
-Shows how GF is used in controlled language translation. -

-

-W. Ng'ang'a. -Multilingual content development for eLearning in Africa. -eLearning Africa: 1st Pan-African Conference on ICT for Development, - Education and Training. 24-26 May 2006, Addis Ababa, Ethiopia. -2006. -pdf -
-Presents a programme for producing educational material in African languages -via multilingual generation in GF. -

-

-R. Jonson. -Generating statistical language models from interpretation grammars in dialogue system. -In Proceedings of EACL'06, Trento, Italy. -2006. -
-Uses GF grammars to generate statistical language models for speech recognition. -

-

-A. El Dada and A. Ranta. -Arabic Resource Grammar. -Arabic Language Processing (JETALA), -5-6 June 2006, IERA, Rabat, Morocco, -2006. -
-An outline of the Arabic resource grammar project, focusing on software aspects. -

-

-D. A. Burke and K. Johannisson. -Translating Formal Software Specifications to Natural Language. A Grammar-Based Approach. -In P. Blache, E. Stabler, J. Busquets and R. Moot (eds), -Logical Aspects of Computational Linguistics (LACL 2005), -Springer LNAI 3402, -pp. 51-66, -2005. -
-A paper explaining how a multilingual GF grammar is completed with -Natural Language Generation techniques to improve text quality. -

-

-B. Bringert, R. Cooper, P. Ljunglöf, A. Ranta, -Multimodal Dialogue System Grammars. -Proceedings of DIALOR'05, Ninth Workshop on the Semantics and Pragmatics of Dialogue, Nancy, France, June 9-11, 2005, -2005. -pdf. -
-Shows how mouse clicks can be integrated in GF grammars -alongside with speech input. -

-

-K. Johannisson, -Formal and Informal Software Specifications. -PhD thesis, -Computer Science, Göteborg University, -2005. -[http://www.cse.chalmers.se/~krijo/thesis/thesisA4.pdf] -
-Collection of articles in the GF-KeY project, with an introduction. -

-

-P. Ljunglöf. -Expressivity and Complexity of the Grammatical Framework. -PhD thesis, Computer Science, -Göteborg University, -2004. -pdf -
-Language-theoretical study of GF and its parsing problem. -

-

-A. Ranta. -Grammatical Framework: A Type-Theoretical Grammar Formalism. -Journal of Functional Programming, 14(2), -pp. 145-189, -2004. -Draft available as ps.gz. -
-Theoretical paper explaining the GF formalism and its implementation. -The standard reference on GF, but doesn't cover the module system. -

-

-A. Ranta. -Computational semantics in type theory. -Mathematics and Social Sciences, 165, -pp. 31-57, -2004. -Draft available as pdf. -
-// Shows how Montague-style grammars are implemented in GF and extends// -this to type-theoretical grammars for anaphoric expressions. -

-

-H. Hammarström and A. Ranta. -Cardinal Numerals Revisited in GF. -Workshop on Numerals in the World's Languages. -Dept. of Linguistics Max Planck Institute for Evolutionary Anthropology, Leipzig, -2004. -
-An overview of the numeral grammar project, covering 88 languages. -

-

-A. Ranta. -Grammatical Framework Tutorial. -In A. Beckmann and N. Preining, editors, -ESSLLI 2003 Course Material I, -Collegium Logicum, -volume V, -pp. 1-86. -Kurt Gödel Society, -Vienna, -2004. -
-A revised version of the on-line GF tutorial, v1.0. -

-

-J. Khegai and A. Ranta. -Building and Using a Russian Resource Grammar in GF. -In A. Gelbukh (ed), -Intelligent Text Processing and Computational Linguistics (CICLing-2004), -Seoul, Korea, February 2003, -Springer LNCS 945, -pp. 38-41, -2004. -ps.gz. -
-An introduction to the GF resource grammar project, with Russian as prime example. -

-

-A. Ranta and R. Cooper. -Dialogue Systems as Proof Editors. - Journal of Logic, Language and Information, 13, -pp. 225-240, -2004. -Conference version (IJCAR/ICoS-3, Siena, June 2001) available as - ps.gz. -Shows a close analogy between task-oriented dialogue systems -and metavariable-based proof editors. -

-

-J. Khegai and A. Ranta. -Building and Using a Russian Resource Grammar in GF. -In A. Gelbukh (ed), -Intelligent Text Processing and Computational Linguistics (CICLing-2004), -Seoul, Korea, February 2003, -Springer LNCS 945, -pp. 38-41, -2004. -ps.gz. -
-Explains how the GF GUI is used in syntax editing and discusses -how new grammars are created. -

-

-R. Hähnle, K. Johannisson, and A. Ranta. -An Authoring Tool for Informal and Formal Requirements Specifications. -In R. D. Kutsche and H. Weber (eds), -ETAPS/FASE-2002: Fundamental Approaches to Software Engineering, -Springer LNCS, -vol. 2306, -pp. 233--248, -2002. -pdf -
-Describes a GF-based authoring tool for object-oriented -specifications in OCL and English. Carries out in full -the work proposed in the position paper (Hähnle & Ranta 2001). -

-

-K. Johannisson and A.Ranta, -Formal Verification of Multilingual Instructions. -Proceedings of the Joint Winter Meeting 2001. -Departments of Computer Science and Computer Engineering, -Chalmers University of Technology and Göteborg University. -2001. -ps.gz. -
-Instructions for an alarm system in four languages, verified in the proof editor Alfa. -

-

-R. Hähnle and A. Ranta, -Connecting OCL with the Rest of the World. -ETAPS 2001 Workshop on Transformations in UML (WTUML), -Genova, -2001. -ps.gz. -
-A position paper explaining how GF can be used to help in object-oriented -modelling, with some examples on natural-language interaction with -OCL (Object Constraint Language). -

-

-T. Hallgren, "The Correctness of Insertion Sort", -Manuscript, Chalmers University, Göteborg, 2001. -Available in -ps -
-A seven-page text generated by GF-Alfa. -

-

-A. Ranta. "Bescherelle bricolé", -ps.gz, -2001. -
-A machine-generated book on French conjugation implemented in GF. -

-

-T. Hallgren and A. Ranta, -An Extensible Proof Text Editor. -In M. Parigot and A. Voronkov (eds), -Logic for Programming and Automated Reasoning (LPAR'2000), -LNCS/LNAI 1955, -pp. 70-84, -Springer Verlag, Heidelberg, -2000. -ps.gz. -
-Describes an interface to the proof editor Alfa written in GF. -

-

-M. Dymetman, V. Lux, and A. Ranta, -XML and multilingual document authoring: converging trends. -Proceedings of the The 18th International Conference on Computational Linguistics (COLING 2000), -pp. 243-249, -Saarbruecken, -2000. -ps.gz. -
-Relates GF not only with XML but also with definite clause grammars. -

-

-P. Mäenpää and A. Ranta. -The type theory and type checker of GF. -Colloquium on Principles, Logics, and Implementations of High-Level Programming Languages, Workshop on Logical Frameworks and Meta-languages, Paris, 28 September 1999. -1999. -ps.gz. -
-Concise theoretical presentation of GF, using the old notation prior to v0.9. -

- - -

Background for GF

- -

-In alphabetical order: -

-

-L. Magnusson. -The Implementation of ALF - a Proof Editor based on Martin-Löf's -Monomorphic Type Theory with Explicit Substitutions. -PhD Thesis, Department of Computing Science, -Chalmers University of Technology and Göteborg University, -1994. -
-Back in 1992 the most wonderful program in the world, ALF is -a model that GF directly follows: GF is Yet ALF. -

-

-P. Martin-Löf. -Intuitionistic Type Theory. -Bibliopolis, Naples, 1984. -
-A very accessible book (if you have access to it!) on type theory directly from the source. -

-

-B. Nordström, K. Petersson, and J. Smith. -Programming in Martin-Löf's Type Theory: an Introduction. -Oxford University Press, 1990. - The book is out of print, but a free version can be picked up from -www.cse.chalmers.se/Cs/Research/Logic/book/ -
-Standard reference on the subject. -

-

-A. Ranta. -Type Theoretical Grammar. -Oxford University Press, Oxford, -1994. -
-Monograph on type theory in linguistics. Includes an introduction -to type theory. Focused on semantics, in particular anaphora. A first, -very rudimentary implementation of linearization in the proof system ALF. -

-

-A. Ranta. -Syntactic categories in the language of mathematics. -In P. Dybjer, B. Nordström, and J. Smith, eds., -Types for Proofs and Programs, -pp. 162-182, -Lecture Notes in Computer Science -996, -Springer-Verlag, -Heidelberg, -1995. -ps.gz -
-Predecessor of GF: grammar defined in type theory and implemented in ALF. -

-

-A. Ranta. -Context-relative syntactic categories and the formalization of mathematical text. -In S. Berardi and M. Coppo, eds., -Types for Proofs and Programs, -pp. 231-248, -Lecture Notes in Computer Science -1158, -Springer-Verlag, -Heidelberg, -1996. -ps.gz. -
-Extending the theory of the previous paper. The implementation in ALF -eventually became so heavy that the need arose for GF. -

-

-A. Ranta. -Structures grammaticales dans le français mathématique. -Mathématiques, informatique et Sciences Humaines., -vol. 138 pp. 5-56 and 139 pp. 5-36, -1997. -
-A rather comprehensive French grammar presented in a type-theoretical style. -

-

-A. Ranta. - Syntactic calculus with dependent types. - Journal of Logic, Language and Information, vol. 4, - pp. 413-431, 1998. -
-Interprets Lambek Calculus in type theory and defines some extensions. -

- - - - diff --git a/doc/gf-editor-modes.t2t b/doc/gf-editor-modes.t2t index d9368dd32..603025834 100644 --- a/doc/gf-editor-modes.t2t +++ b/doc/gf-editor-modes.t2t @@ -13,16 +13,17 @@ welcome! ==Emacs== -[gf.el ../src/tools/gf.el] by Johan Bockgård provides syntax highlighting and -automatic indentation and lets you run the GF Shell in an emacs buffer. -See installation instructions inside. +[gf.el https://github.com/GrammaticalFramework/gf-emacs-mode] by Johan +Bockgård provides syntax highlighting and automatic indentation and +lets you run the GF Shell in an emacs buffer. See installation +instructions inside. ==Atom== [language-gf https://atom.io/packages/language-gf], by John J. Camilleri ==Eclipse== -[GF Eclipse Plugin http://www.grammaticalframework.org/eclipse/index.html], by John J. Camilleri +[GF Eclipse Plugin https://github.com/GrammaticalFramework/gf-eclipse-plugin/], by John J. Camilleri ==Gedit== diff --git a/doc/gf-reference.html b/doc/gf-reference.html deleted file mode 100644 index 330e30c7f..000000000 --- a/doc/gf-reference.html +++ /dev/null @@ -1,664 +0,0 @@ - - - - - -GF Quick Reference - -
-

GF Quick Reference

-Aarne Ranta
-April 4, 2006 -
- -

-
-

- - - -

-
-

-

-This is a quick reference on GF grammars. It aims to -cover all forms of expression available when writing -grammars. It assumes basic knowledge of GF, which -can be acquired from the -GF Tutorial. -Help on GF commands is obtained on line by the -help command (help), and help on invoking -GF with (gf -help). -

- - -

A complete example

- -

-This is a complete example of a GF grammar divided -into three modules in files. The grammar recognizes the -phrases one pizza and two pizzas. -

-

-File Order.gf: -

- -
-  abstract Order = {
-  cat 
-    Order ; 
-    Item ;
-  fun 
-    One, Two : Item -> Order ;
-    Pizza : Item ;
-  }
-
- -

-File OrderEng.gf (the top file): -

- -
-  --# -path=.:prelude
-  concrete OrderEng of Order = 
-   open Res, Prelude in {
-  flags startcat=Order ;
-  lincat 
-    Order = SS ; 
-    Item  = {s : Num => Str} ;
-  lin 
-    One it = ss ("one" ++ it.s ! Sg) ;
-    Two it = ss ("two" ++ it.s ! Pl) ;
-    Pizza  = regNoun "pizza" ;
-  }
-
- -

-File Res.gf: -

- -
-  resource Res = open Prelude in {
-  param Num = Sg | Pl ;
-  oper regNoun : Str -> {s : Num => Str} =
-    \dog -> {s = table {
-      Sg => dog ;
-      _  => dog + "s"
-      }
-    } ;
-  }
-
- -

-To use this example, do -

- -
-    % gf             -- in shell: start GF
-    > i OrderEng.gf  -- in GF: import grammar
-    > p "one pizza"  --        parse string
-    > l Two Pizza    --        linearize tree
-
- - -

Modules and files

- -

-One module per file. -File named Foo.gf contains module named -Foo. -

-

-Each module has the structure -

- -
-  moduletypename =   
-    Inherits **        -- optional
-    open Opens in      -- optional
-    { Judgements }   
-
- -

-Inherits are names of modules of the same type. -Inheritance can be restricted: -

- -
-    Mo[f,g],  -- inherit only f,g from Mo
-    Lo-[f,g]  -- inheris all but f,g from Lo
-
- -

-Opens are possible in concrete and resource. -They are names of modules of these two types, possibly -qualified: -

- -
-    (M = Mo), -- refer to f as M.f or Mo.f
-    (Lo = Lo) -- refer to f as Lo.f
-
- -

-Module types and judgements in them: -

- -
-  abstract A          -- cat, fun, def, data
-  concrete C of A     -- lincat, lin, lindef, printname
-  resource R          -- param, oper
-  
-  interface I         -- like resource, but can have
-                         oper f : T without definition
-  instance J of I     -- like resource, defines opers
-                         that I leaves undefined
-  incomplete          -- functor: concrete that opens 
-   concrete CI of A =    one or more interfaces
-    open I in ...
-  concrete CJ of A =  -- completion: concrete that
-    CI with              instantiates a functor by
-      (I = J)            instances of open interfaces
-
- -

-The forms -param, oper -may appear in concrete as well, but are then -not inherited to extensions. -

-

-All modules can moreover have flags and comments. -Comments have the forms -

- -
-  -- till the end of line
-  {- any number of lines between -}
-  --# used for compiler pragmas
-
- -

-A concrete can be opened like a resource. -It is translated as follows: -

- -
-  cat C          --->  oper C : Type = 
-  lincat C = T           T ** {lock_C : {}}
-  
-  fun f : G -> C --->  oper f : A* -> C* = \g -> 
-  lin f = t              t g ** {lock_C = <>}          
-
- -

-An abstract can be opened like an interface. -Any concrete of it then works as an instance. -

- - -

Judgements

- -
-  cat C               -- declare category C
-  cat C (x:A)(y:B x)  -- dependent category C
-  cat C A B           -- same as C (x : A)(y : B)
-  fun f : T           -- declare function f of type T
-  def f = t           -- define f as t
-  def f p q = t       -- define f by pattern matching
-  data C = f | g      -- set f,g as constructors of C
-  data f : A -> C     -- same as 
-                         fun f : A -> C; data C=f
-  
-  lincat C = T        -- define lin.type of cat C
-  lin f = t           -- define lin. of fun f
-  lin f x y = t       -- same as lin f = \x y -> t
-  lindef C = \s -> t  -- default lin. of cat C
-  printname fun f = s -- printname shown in menus
-  printname cat C = s -- printname shown in menus
-  printname f = s     -- same as printname fun f = s
-  
-  param P = C | D Q R -- define parameter type P 
-                         with constructors
-                         C : P, D : Q -> R -> P
-  oper h : T = t      -- define oper h of type T
-  oper h = t          -- omit type, if inferrable
-  
-  flags p=v           -- set value of flag p
-
- -

-Judgements are terminated by semicolons (;). -Subsequent judgments of the same form may share the -keyword: -

- -
-  cat C ; D ;         -- same as cat C ; cat D ;
-
- -

-Judgements can also share RHS: -

- -
-  fun f,g : A         -- same as fun f : A ; g : A
-
- - -

Types

- -

-Abstract syntax (in fun): -

- -
-  C                -- basic type, if cat C
-  C a b            -- basic type for dep. category
-  (x : A) -> B     -- dep. functions from A to B
-  (_ : A) -> B     -- nondep. functions from A to B
-  (p,q : A) -> B   -- same as (p : A)-> (q : A) -> B
-  A -> B           -- same as (_ : A) -> B
-  Int              -- predefined integer type
-  Float            -- predefined float type
-  String           -- predefined string type
-
- -

-Concrete syntax (in lincat): -

- -
-  Str              -- token lists
-  P                -- parameter type, if param P
-  P => B           -- table type, if P param. type
-  {s : Str ; p : P}-- record type
-  {s,t : Str}      -- same as {s : Str ; t : Str}
-  {a : A} **{b : B}-- record type extension, same as  
-                      {a : A ; b : B}
-  A * B * C        -- tuple type, same as
-                      {p1 : A ; p2 : B ; p3 : C}
-  Ints n           -- type of n first integers
-
- -

-Resource (in oper): all those of concrete, plus -

- -
-  Tok              -- tokens (subtype of Str)
-  A -> B           -- functions from A to B
-  Int              -- integers
-  Strs             -- list of prefixes (for pre)
-  PType            -- parameter type
-  Type             -- any type
-
- -

-As parameter types, one can use any finite type: -P defined in param P, -Ints n, and record types of parameter types. -

- - -

Expressions

- -

-Syntax trees = full function applications -

- -
-  f a b              -- : C if fun f : A -> B -> C
-  1977               -- : Int
-  3.14               -- : Float
-  "foo"              -- : String
-
- -

-Higher-Order Abstract syntax (HOAS): functions as arguments: -

- -
-  F a (\x -> c)      -- : C if a : A, c : C (x : B), 
-                        fun F : A -> (B -> C) -> C
-
- -

-Tokens and token lists -

- -
-  "hello"            -- : Tok, singleton Str
-  "hello" ++ "world" -- : Str
-  ["hello world"]    -- : Str, same as "hello" ++ "world"
-  "hello" + "world"  -- : Tok, computes to "helloworld"
-  []                 -- : Str, empty list
-
- -

-Parameters -

- -
-  Sg                   -- atomic constructor
-  VPres Sg P2          -- applied constructor
-  {n = Sg ; p = P3}    -- record of parameters
-
- -

-Tables -

- -
-  table {              -- by full branches
-    Sg => "mouse" ;
-    Pl => "mice"
-    }
-  table {              -- by pattern matching
-    Pl => "mice" ;
-    _  => "mouse"      -- wildcard pattern
-    }
-  table {               
-    n => regn n "cat"  -- variable pattern 
-    }
-  table Num {...}      -- table given with arg. type
-  table ["ox"; "oxen"] -- table as course of values
-  \\_ => "fish"        -- same as table {_ => "fish"} 
-  \\p,q => t           -- same as \\p => \\q => t
-  
-  t ! p                -- select p from table t
-  case e of {...}      -- same as table {...} ! e 
-
- -

-Records -

- -
-  {s = "Liz"; g = Fem} -- record in full form
-  {s,t = "et"}         -- same as {s = "et";t= "et"}
-  {s = "Liz"} **       -- record extension: same as
-    {g = Fem}             {s = "Liz" ; g = Fem}
-  
-  <a,b,c>        -- tuple, same as {p1=a;p2=b;p3=c}
-
- -

-Functions -

- -
-  \x -> t            -- lambda abstract
-  \x,y -> t          -- same as \x -> \y -> t
-  \x,_ -> t          -- binding not in t
-
- -

-Local definitions -

- -
-  let x : A = d in t -- let definition
-  let x = d in t     -- let defin, type inferred
-  let x=d ; y=e in t -- same as 
-                        let x=d in let y=e in t
-  let {...} in t     -- same as let ... in t
-  
-  t where {...}      -- same as let ... in t
-
- -

-Free variation -

- -
-  variants {x ; y}     -- both x and y possible
-  variants {}          -- nothing possible
-
- -

-Prefix-dependent choices -

- -
-  pre {"a" ; "an" / v} -- "an" before v, "a" otherw.
-  strs {"a" ; "i" ;"o"}-- list of condition prefixes
-
- -

-Typed expression -

- -
-  <t:T>                -- same as t, to help type inference
-
- -

-Accessing bound variables in lin: use fields $1, $2, $3,.... -Example: -

- -
-  fun F : (A : Set) -> (El A -> Prop) -> Prop ; 
-  lin F A B = {s = ["for all"] ++ A.s ++ B.$1 ++ B.s}
-
- - -

Pattern matching

- -

-These patterns can be used in branches of table and -case expressions. Patterns are matched in the order in -which they appear in the grammar. -

- -
-  C                 -- atomic param constructor
-  C p q             -- param constr. applied to patterns
-  x                 -- variable, matches anything
-  _                 -- wildcard, matches anything
-  "foo"             -- string
-  56                -- integer
-  {s = p ; y = q}   -- record, matches extensions too
-  <p,q>             -- tuple, same as {p1=p ; p2=q}
-  p | q             -- disjunction, binds to first match
-  x@p               -- binds x to what p matches
-  - p               -- negation
-  p + "s"           -- sequence of two string patterns
-  p*                -- repetition of a string pattern
-
- - -

Sample library functions

- -
-  -- lib/prelude/Predef.gf
-  drop   : Int -> Tok -> Tok   -- drop prefix of length
-  take   : Int -> Tok -> Tok   -- take prefix of length
-  tk     : Int -> Tok -> Tok   -- drop suffix of length
-  dp     : Int -> Tok -> Tok   -- take suffix of length
-  occur  : Tok -> Tok -> PBool -- test if substring
-  occurs : Tok -> Tok -> PBool -- test if any char occurs
-  show   : (P:Type) -> P ->Tok -- param to string
-  read   : (P:Type) -> Tok-> P -- string to param
-  toStr  : (L:Type) -> L ->Str -- find "first" string
-  
-  -- lib/prelude/Prelude.gf
-  param Bool = True | False
-  oper
-    SS  : Type                   -- the type {s : Str}
-    ss  : Str -> SS              -- construct SS
-    cc2 : (_,_ : SS) -> SS       -- concat SS's
-    optStr : Str -> Str          -- string or empty
-    strOpt : Str -> Str          -- empty or string
-    bothWays : Str -> Str -> Str -- X++Y or Y++X 
-    init : Tok -> Tok            -- all but last char
-    last : Tok -> Tok            -- last char
-    prefixSS : Str -> SS -> SS
-    postfixSS : Str -> SS -> SS
-    infixSS : Str -> SS -> SS -> SS
-    if_then_else : (A : Type) -> Bool -> A -> A -> A
-    if_then_Str : Bool -> Str -> Str -> Str
-
- - -

Flags

- -

-Flags can appear, with growing priority, -

- -
    -
  • in files, judgement flags and without dash (-) -
  • as flags to gf when invoked, with dash -
  • as flags to various GF commands, with dash -
- -

-Some common flags used in grammars: -

- -
-  startcat=cat    use this category as default 
-  
-  lexer=literals  int and string literals recognized
-  lexer=code      like program code
-  lexer=text      like text: spacing, capitals
-  lexer=textlit   text, unknowns as string lits
-  
-  unlexer=code    like program code
-  unlexer=codelit code, remove string lit quotes
-  unlexer=text    like text: punctuation, capitals
-  unlexer=textlit text, remove string lit quotes
-  unlexer=concat  remove all spaces
-  unlexer=bind    remove spaces around "&+"
-  
-  optimize=all_subs  best for almost any concrete
-  optimize=values    good for lexicon concrete
-  optimize=all       usually good for resource
-  optimize=noexpand  for resource, if =all too big
-
- -

-For the full set of values for FLAG, -use on-line h -FLAG. -

- - -

File paths

- -

-Colon-separated lists of directories searched in the -given order: -

- -
-  --# -path=.:../abstract:../common:prelude
-
- -

-This can be (in order of growing preference), as -first line in the top file, as flag to gf -when invoked, or as flag to the i command. -The prefix --# is used only in files. -

-

-If the environment variabls GF_LIB_PATH is defined, its -value is automatically prefixed to each directory to -extend the original search path. -

- - -

Alternative grammar formats

- -

-Old GF (before GF 2.0): -all judgements in any kinds of modules, -division into files uses includes. -A file Foo.gf is recognized as the old format -if it lacks a module header. -

-

-Context-free (file foo.cf). The form of rules is e.g. -

- -
-  Fun. S ::= NP "is" AP ;
-
- -

-If Fun is omitted, it is generated automatically. -Rules must be one per line. The RHS can be empty. -

-

-Extended BNF (file foo.ebnf). The form of rules is e.g. -

- -
-  S ::= (NP+ ("is" | "was") AP | V NP*) ;
-
- -

-where the RHS is a regular expression of categories -and quoted tokens: "foo", CAT, T U, T|U, T*, T+, T?, or empty. -Rule labels are generated automatically. -

-

-Probabilistic grammars (not a separate format). -You can set the probability of a function f (in its value category) by -

- -
-  --# prob f 0.009
-
- -

-These are put into a file given to GF using the probs=File flag -on command line. This file can be the grammar file itself. -

-

-Example-based grammars (file foo.gfe). Expressions of the form -

- -
-  in Cat "example string"
-
- -

-are preprocessed by using a parser given by the flag -

- -
-  --# -resource=File
-
- -

-and the result is written to foo.gf. -

- - -

References

- -

-GF Homepage -

-

-A. Ranta, Grammatical Framework: A Type-Theoretical Grammar Formalism. -The Journal of Functional Programming, vol. 14:2. 2004, pp. 145-189. -

- - - - diff --git a/download.html b/download.html deleted file mode 100644 index 5da03ea53..000000000 --- a/download.html +++ /dev/null @@ -1,54 +0,0 @@ - - - -GF Version 2.8, released 8 July 2007; updated 12 September. -
    -
  • GF-2.8.tgz, -Haskell and Java sources, grammar library sources, examples, documentation. -To compile and install: read src/INSTALL. - -
  • GF-2.8-lib.tgz, -Compiled grammar libraries. To install: -unpack this in the directory pointed by GF_LIB_PATH. - -
  • GF-2.8-i386-pc-linux-gnu.tgz, -Linux binary, with precompiled grammar libraries. To install: -
    -  tar xvfz GF-2.8-i386-pc-linux-gnu.tgz
-  cd GF-2.8-i386-pc-linux-gnu
-  ./configure
-  make install
-
    -You may need sudo make install; if you cannot write in -/usr/local/, you can change the target directory -by ./configure --prefix=MYDIR. - - -
  • GF-2.8-i386-apple-darwin8.9.1.tgz, Mac OS X binary (Intel), with precompiled grammar libraries. Requires Readline from -MacPorts. To install, see above item. - -
  • GF-2.8-windows-cygwin.tgz, Windows Cygwin binary. Requires -Cygwin. To install, see above item. - -
-The Sourceforge GF site -has currently only v 2.7 and older versions. - -

- -Don't forget that you can always get the current sources from the -GF Darcs Repository. First time you do -

-  darcs get --partial --set-scripts-executable http://www.cs.chalmers.se/Cs/Research/Language-technology/darcs/GF/
-
-and subsequent times -
-  darcs pull -a
-
-Then compile and install as described in -GF/src/INSTALL. - - - - diff --git a/download/index-3.1.6.html b/download/index-3.1.6.html deleted file mode 100644 index f351fbb1e..000000000 --- a/download/index-3.1.6.html +++ /dev/null @@ -1,93 +0,0 @@ - - - - -Grammatical Framework Download and Installation - -

Grammatical Framework Download and Installation

- -Version 3.1.6, 23 April 2010
-
- -

Latest developer code

-

-GF darcs repository -

-

Latest release

-

-GF 3.1.6 released 23 April 2010. -

- - -

-What's new? See the release notes. -

-

Previous releases

-

-Follow this link. -

-

Installation instructions

-

-The Windows package is installed by just unpacking it anywhere. -It finds the libraries relative to the .exe file. -

-

-To install a binary package for MacOS X or Linux: uncompress the executable and -put it somewhere on your path, renaming it to just gf. For instance: -

-
-    gunzip gf-bin-[version].gz
-    sudo mv gf-bin-[version] /usr/local/bin/gf
-
-

-To install the libraries (if done separately), unpack them in the place to which your -GF_LIB_PATH points. -

-
-    cd $GF_LIB_PATH
-    gtar xvfz gf-lib-[VERSION].tgz
-
-

-If this variable hasn't been defined, it is useful define it, e.g. -

-
-    export GF_LIB_PATH=/usr/local/lib/gf/
-
-

-

-To compile and install from source on Mac or Linux, do: -

-
-    tar xvfz gf-[VERSION].tgz
-    cd GF/src
-    runghc Setup configure --user
-    runghc Setup build
-    runghc Setup install
-
-

-The last command must be prefixed by sudo, if the installation is outside -user-writeable area. -

- - - - diff --git a/download/old-index.html b/download/old-index.html deleted file mode 100644 index 74dabb08c..000000000 --- a/download/old-index.html +++ /dev/null @@ -1,146 +0,0 @@ - - - - -Grammatical Framework Download and Installation - - - - -

Grammatical Framework
Download and Installation

- -

Latest developer code

-

-GF darcs repository -

- -

Latest release

- -
-
GF 3.1, 15 December 2009 -
-

-

- -

Installation instructions

- -
    -
  • Linux & Mac binaries: tar xvfz in / -
  • Windows binary: unzip in c:\ -
  • Compiled library: - tar xvfz in some DIR and point the variable GF_LIB_PATH to DIR/lib. -
  • -Source package: tar xvfz somewhere and compile as follows: -

    -

    -    tar xvfz gf-3.0.tgz
-    cd gf-3.0
-
-    # alternative 1:
-    runghc Setup configure
-    runghc Setup build
-    runghc Setup install
-
-    # alternative 2:
-    make
-    make install
-
    -
- -

Old releases

- -
-
GF 3.0, 24 June 2009. -
-

-

- -

Installation instructions

- -

-Mac binary: gunzip and put somewhere on your path. -

-Linux: tar xvfz in / -

-Windows: unzip in c:\ -

-Compiled library: tar xvfz in some DIR and point the variable GF_LIB_PATH to DIR/lib. -

-Source package: tar xvfz somewhere and compile as follows: -

-

-    tar xvfz gf-3.0.tgz
-    cd gf-3.0
-
-    # alternative 1:
-    runghc Setup configure
-    runghc Setup build
-    runghc Setup install
-
-    # alternative 2:
-    make
-    make install
-
-
- - -GF 3.0 beta3, April 2009. - -3.0 beta2 - - - - - - diff --git a/download/release-3.1.6.html b/download/release-3.1.6.html deleted file mode 100644 index e7c31458d..000000000 --- a/download/release-3.1.6.html +++ /dev/null @@ -1,87 +0,0 @@ - - - - -GF Version 3.1.6 Release Notes - -

GF Version 3.1.6 Release Notes

- -23 April 2010
-
- -

Installation

-

-The binaries now work out of the box for each platform and support -completions (file names and parsing), because readline has been -changed to haskeline. -

-

-To compile from source, GHC 6.12 is now required. But GHC is not needed -if the binary executables are used. -

-

-Binaries (.gfo and .pgf files) compiled with GF 3.1 are incompatible -with 3.1.6 and must either be removed; alternatively, the -src flag can be -used when compiling. -

-

-Notice the new place to send bug reports (see GF's welcome message)! -

-

New features

-

-Grammar language -

-
    -
  • improved support for dependent types (see SUMO, nqueens in examples) -
- -

-Shell commands and options (see help in GF for more information) -

-
    -
  • eb: example-based grammar file conversion - (see examples/animals/QuestionI.gf) -
  • vd = visualize_dependency: show dependency tree -
  • vp = visualize_parse: show parse tree -
  • gr, gt, mq, tq: use a seed tree with metavariables -
  • gr, p, tq: use -probs=FILE to bias or rank generated trees -
  • gt now tries the different alternatives in the order in which they appear in the source code -
  • dg: hide modules when printing dependency graph -
  • -output-format=lambda_prolog let us to use Lambda Prolog for exhaustive generation with dependent types -
- -

-Libraries -

-
    -
  • Urdu completed (Shafqat Virk) -
  • Turkish morphology + Zamberek's dictionary in GF (Server Cimen) -
- -

-Examples -

-
    -
  • SUMO -
  • phrasebook -
  • nqueens -
- -

-Internal -

-
    -
  • single PMCFG format for both linearization and parsing -
  • use of Haskeline for completion - more portable than Readline -
  • unicode support from Haskell 6.12 instead of home-made coding -
- -

Issues

-

-Javascript generation is not updated to the new PGF format. -GF 3.1 should still be used for building Javascript applications. -

- - - - diff --git a/download/release-3.1.6.txt b/download/release-3.1.6.t2t similarity index 100% rename from download/release-3.1.6.txt rename to download/release-3.1.6.t2t diff --git a/eclipse/images/.htaccess b/eclipse/images/.htaccess deleted file mode 100644 index 4635130a3..000000000 --- a/eclipse/images/.htaccess +++ /dev/null @@ -1,5 +0,0 @@ -# Images in this folder have moved, now hosted at GitHub -# John 2013-01-10 - -RewriteEngine On -RewriteRule ^(.*)$ https://raw.github.com/GrammaticalFramework/gf-eclipse-plugin/master/doc/images/$1 [L,R=301] \ No newline at end of file diff --git a/eclipse/index.html b/eclipse/index.html deleted file mode 100644 index 5d7405a68..000000000 --- a/eclipse/index.html +++ /dev/null @@ -1,446 +0,0 @@ - - - -GF Eclipse Plugin - - - - - - -

The GF Eclipse Plugin

-

John J. Camilleri
Updated: 22 February 2013

-

The research leading to these results has received funding from the European Union's Seventh Framework Programme (FP7/2007-2013) under grant agreement no. FP7-ICT-247914 (the MOLTO Project).

-

Introduction

-

The aim behind developing a desktop IDE for GF is to provide more powerful tools than may be possible and/or practical in a web-based environment (the GF Web IDE). In particular, the ability to resolve identifier cross-references and browse external libraries quickly during development time is one of the primary motivations for the project.

-

The choice was made to develop this desktop IDE as a plugin for the Eclipse Platform as it emerged as the most popular choice among the GF developer community. Support for the platform is vast and many tools for adapting Eclipse to domain-specific languages already exist. Unlike the zero-click Web IDE approach, using the GF Eclipse plugin requires some manual installation and configuration on the development machine.

-

License

-

The GF Eclipse Plugin is open-source under the GNU General Public License (GPL).
The licenses that cover the rest of GF are listed here. Xtext and Eclipse are released under the Eclipse Public License (EPL).

- - -

Publications

-
    -
  • Poster at EAMT 2012 in Trento, Italy
    • -
  • Paper presented at FreeRBMT 2012 in Gothenburg, Sweden.
    • -
-

Features

-
    -
  • Syntax highlighting and error detection
    • -
  • Code folding, quick block-commenting, automatic code formatting
    • -
  • Definition outlining, jump to declaration, find usage
    • -
  • Warnings for problems in module dependency hierarchy
    • -
  • Launch configurations, i.e. compilation directly from IDE
    • -
  • Use GF Shell from within Eclipse
    • -
  • Auto-completion for declared identifiers
    • -
  • Background compilation (shallow) using project builder
    • -
  • Support for Open Declaration (F3), including qualified names
    • -
  • Code generation for new languages in application grammars
    • -
  • Inline documentation for function calls, overloads
    • -
  • Proper cross-reference handling with qualified names
    • -
  • Test management and testing tool
    • -
  • External library browser
    • -
-

Release history

-

22/02/13 : (1.5.2.x) New option for specifying include/exclude build files. Progress indicators during build.

-

12/06/12 : (1.5.1.x) Support for scoping when library sources not available. Performance and implementational improvements to External Library View.

-

19/04/12 : (1.5.0.x) Removed need for .gfexternal folder. Much improved "External Libraries" view allows you to search in current scope. Memory usage improvements in scoping implementation. Validation now occurs on file open. Re-added GF Library Path field in preferences.

-

04/04/12 : (1.4.3.x) Improvements to the Test Manager. Added support for arbitrary commands in launch. Made the license information clearer.

-

22/03/12 : (1.4.2.x) Added wizard for context-free grammars (no syntax support, though). Context-menu option for toggling the GF Nature and Builder together. Test Manager now parses languages & parameters, and includes filtering and toggling options.

-

14/03/12 : (1.4.1.x) GF source files can now be anywhere in project, not necessarily at top level.

-

06/03/12 : (1.4.0.x) Test manager view. Updated launch configurations. UI improvements, new icons. Inline contextual documentation. Correct resolution of qualified identifiers. Bug fixes and some refactoring.

-

07/02/12 : (1.3.0.x) Re-written underlying GF grammar.

-

02/02/12 : (1.2.5.x) Hugely improved scoping performance. New "Clone to new language" wizard. Improved code formatting. More efficient builder. Removed library path preference. GF runtime is now automatically found.

-

23/01/12 : (1.2.4.x) Logging is now done to the Eclipse Platform Log (use "Error Log" view). Tested with Eclipse 3.6.2. Bug fixes.

-

11/01/12 : (1.2.3.x) Updated to use Xtext 2.1.0. Links to external files now moved into new "GF Library View".

-

24/11/11 : (1.2.0.x) with support for "Open Declaration" (F3) function, including for external libraries.

-

18/11/11 : Third BETA (1.1.0.x) released, overhauled to take advantage of new compiler features. Tag-based scoping with caching for better performance. Interactive console (GF Shell) for launch configs.

-

11/10/11 : Second BETA (1.0.0.x) released, with improved support for qualified names, local variable bindings, selective inheritance, module outlining, and launch configurations. Also new GF Project type.

-

01/10/11 : Initial BETA released.

- - -

Installation & updating

-

System requirements

-
    -
  1. Eclipse 3.6 or above.
    2. -
  2. GF 3.3.3 or above. The path to GF must be set within the plugin preferences (see below).
    3. -
-

Using the GF Resource Grammar Library requires that the RGL binaries are also installed on your system. If you are using the pre-compiled binaries but have separately downloaded the RGL sources to your system, you can optionally specify this path in the plugin preferences (see below).

-

Installing the plugin for the first time

-

Available Software Sites

-

In order for dependencies to be satisfied, you need to ensure that your Eclipse has the following update site URLs set:

-
    -
  • Eclipse releases (change version name to match): http://download.eclipse.org/releases/indigo
    • -
  • Xtext: http://download.eclipse.org/modeling/tmf/xtext/updates/composite/releases/
    • -
-

You can set these from Preferences → Install/Update → Available Software Sites.

-

Eclipse preferences

-

How to find the Eclipse preferences window depends on your Eclipse version and/or OS, and may be either of:

-
    -
  • Window → Preferences
    • -
  • Eclipse → Preferences
    • -
-

Installation

-
    -
  1. Inside Eclipse, go to Help → Install New Software.
    2. -
  2. Add new software site using the URL: http://www.grammaticalframework.org/eclipse/release/
    3. -
  3. Select the GF Eclipse Plugin, and make sure the Contact all update sites during install to find required software option is checked.
    4. -
  4. Click Next, accept the license agreement and install.
    5. -
  5. Accept the prompt warning that the software is unsigned.
    6. -
  6. Restart Eclipse when prompted.
    7. -
-

Settings & preferences

-
    -
  1. Add the GF perspective by clicking Window → Open Perspective → Other and choosing GF.
    2. -
  2. Open the plugin settings by going to Preferences → Grammatical Framework (image below).
    3. -
  3. The plugin will try to determine the path to your GF executable automatically, but this may need to be corrected.
    The path should include the name of the GF binary itself, e.g. /home/john/.cabal/bin/gf or C:\Users\John\GF\bin\gf.exe.
    4. -
  4. The Additional path directive setting can be used if you want to avoid writing --# -path=... directives in your source files. By default it is set to .:alltenses:prelude, but can also be left blank. This is passed to GF using the --path flag.
    5. -
  5. If you are using a pre-compiled version of the Resource Grammar Library (i.e. you didn't build it from source yourself) but you have the sources available on your system, then you can enter their path in the Library source path setting. This is optional and is only used for jumping to source definitions.
    You should enter an absolute system path, up to and including the src folder, e.g. /home/john/GF/lib/src/ or C:\Users\John\GF\lib\src\.
    6. -
  6. You can also adjust the verbosity level of the GFEP console log in the preferences window.
    7. -
-
-Plugin preferences window

Plugin preferences window

-
-

Updating the plugin

-
    -
  1. In Eclipse, go to Help → Check for updates.
    2. -
  2. Any available updates will appear in the dialog. Select them and follow the usual steps.
    3. -
-

Note: You will probably want to uncheck the Contact all update sites during install to find required software option, which is not usually needed when simply updating the plugin (it makes things go a lot slower).

-

Deprecations

-
    -
  • Any folders .gfexternal can safely deleted as of GFEP ≥ 1.5. Using Eclipse's Clean function from the Run menu will do this for you automatically.
    • -
  • If your project has a folder named (External) you can safely delete this via the Eclipse Project Explorer. It is no longer used in GFEP ≥ 1.2.3
    • -
  • If your workspace contains a log file gfep.log then you can safely delete it from your system. As of version 1.2.4, GFEP uses Eclipse's standard log file at <workspace>/.metadata/.log
    • -
-

Project settings

-

Although the plugin should take care of your project settings for you, in case anything gets messed up you can always correct them manually as follows:

-

Project nature

-

Your .project file should include the GF Project Nature as follows:

-
<natures>
-  <nature>org.grammaticalframework.eclipse.ui.natures.projectNatureID</nature>
-</natures>
-

Builder

-

Your .project file should include the GF Builder as follows:

-
<buildSpec>
-  <buildCommand>
-      <name>org.grammaticalframework.eclipse.ui.build.GFBuilderID</name>
-      <arguments>
-      </arguments>
-  </buildCommand>
-</buildSpec>
-

Specifying build files

-

As of GFEP ≥ 1.5.2, you are now able to explicitly a set of modules in your GF project which should be either included or excluded from the build process. In the inclusive case, this would normally contain your "top-level" modules. This can be done right-clicking on your project in the Project Explorer and choosing Properties → GF Build Settings. From here you simply tick which modules in your project the GF builder should compile:

-
-Project-specific build settings

Project-specific build settings

-
-

When doing this, GFEP will never attempt to compile any other files in the project. Whenever you add a new top-level module or change your module hierarchy, be sure to check these build settings again.

- - -

Getting started

-

Some steps you can follow to get a feel of of the plugin's features.

-

Video tutorial

-

A quick run through of the major features available in the GF Eclipse Plugin.

- - -

If the video doesn't appear above, you can view it at https://vimeo.com/38768382

-

The Eclipse workspace and project concepts

-

When you run Eclipse, it asks you to create or specify a workspace on your system, which is essentially the top directory that Eclipse will "see". Any code you wish to work on inside Eclipse must reside in this workspace. A workspace contains one or more projects, which may or may not be related to each other. Generally each folder under the top-level workspace directory is considered an individual project, i.e.:

-
workspace/
-    Project1/
-        file1-1.gf
-        file1-2.gf
-    Project2/
-        subdir2-1/
-            file2-1-1.gf
-            file2-1-2.gf
-        file2-1.gf
-        file2-2.gf
-

If you open an existing folder as your workspace in Eclipse, then your folders in that workspace do not automatically become Eclipse projects. To do this, you must follow the "Create new GF Project" wizard below, and give your project the exact name of the existing folder (e.g. Project1 or Project2 in the example above.)

-

GF Nature

-

The GF Nature is what makes a regular Eclipse project a GF project. If you create a new GF Project from scratch then you don't need to worry too much about them. If you already have some project set up in your Eclipse workspace, then you will need to associate the builder with your project in order for all the GF goodness to work. This will internally add the GF Builder to your project also.

-

As of 1.4.2, this can be done very easily by right-clicking on your project in the explorer and clicking Add/Remove GF Nature. If your project has the GF icon overlayed on it, then the GF nature and builder are correctly associated with it.

-

See the project settings section above if you want to mess with your .project file.

-

Metadata folders

-

GFEP will create the following metadata directories within your project:

-
    -
  • A .gfbuild directory in every folder containing GF source code, containing .gf-tags files created using the GF --tags compiler flag.
    • -
-

These can be removed manually at any time, as they will always be regenerated as necessary. You will probably want to ignore these directories in your version control system.

-

Create a new GF Project

-

If you have a folder in your workspace which you want to convert into a GF Project, then you can select it from the drop-down. This folder will then appear as an Eclipse project in your workspace.

-

New project wizard New project wizard

-

New module wizard

-

Use the GF Module wizard from File → New → Other → GF Module:

-
-New module wizard

New module wizard

-
-

You can find some small examples here. Download the files and manually add them to your Eclipse workspace to experiment with some of the plugin features.

-

Clone module wizard

-

If you want to clone an existing module into a new language, you can now do so via the Clone GF Module wizard. This is also available as a context menu option in the Project Explorer, via <Right click> → Grammatical Framework → Compile for new language:

-
-Clone GF module to new language wizard

Clone GF module to new language wizard

-
-

This will create and open a new module with the relevant replacements made. Note how in this case string constants are replaced by the empty string "". This wizard currently only works on concrete and instance modules.

-
-Newly cloned file

Newly cloned file

-
-

Workbench editor layout

-

Use Eclipse' own features to drag and drop your editor tabs to suit your needs.

-
-The editor and outline view

The editor and outline view

-
-

Outline view

-

The Outline view in the bottom-left offers a quite overview of your module structure. Clicking any of the terms will make your cursor jump to that point in the file.

-

Cross-referencing errors and warnings

-

Notice how changing a cat definition for example will produce warnings and/or errors in other the modules.

-
-Warnings and errors in the editor

Warnings and errors in the editor

-
-

Validation

-

The in-editor validation sometimes needs to be triggered/updated by a keystroke. If you can still see errors which you believe should be correct, try adding a space character to ensure the validation is re-triggered.

-

Sometimes you may get an entire file marked with errors, even though in fact there is only a single error which is causing the internal builder to fail. In such cases referring to the Problems view should help you locate the cause. This issue is particularly relevant when changes made to one module may induce errors in its descendants (e.g. renaming a parameter in a resource module). This behaviour will hopefully be improved in future versions.

-

Builder

-

For validation to work correctly, the project's Build Automatically option should be turned on. Each time you save a file (or some other change occurs in your project), the builder should get triggered which re-compiles any GF files which have changed. If the builder does not seem to get triggered, try to run a clean via Project → Clean...

-

Code formatting

-

Use the built-in code formatter to tidy your code automatically. Right-click in the editor and choose Format or use the keyboard shortcut Ctrl+Shift+F.

-

Before auto-formatting After auto-formatting

-

Syntax highlighting

-

You can change your syntax colouring styles as you wish from Preferences → Grammatical Framework → Syntax Colouring

-
-Syntax highlighting styles

Syntax highlighting styles

-
-

Launch configurations

-

Set up a launch configuration from Run → Run Configurations... → Grammatical Framework. You will see the dialog below with some fields automatically filled in for you.

-
-Launch configuration dialog

Launch configuration dialog

-
-

Run the launch config and you will see the output from the GF compiler in the console view within Eclipse.

-
-Output from the GF compiler

Output from the GF compiler

-
-

Run As... launch shortcut

-

Using the Run → Run As → Compile with GF shortcut will take the current selection or active editor and do one of two things:

-
    -
  1. If a launch configuration already exists which the current file in its Source filenames field, then that config will be launch directly.
    2. -
  2. Otherwise, a new a launch configuration window is displayed where you will need to review and confirm the default settings before continuing.
    3. -
-

Consoles

-

The plugin provides its own console view named GFEP Log, which shows log messages based on the Log Level preference (Preferences → Grammatical Framework).

-

Additionally, every time a grammar is Run a new console is opened to display the output. This console may also serve as the interactive GF Shell, depending on the Interactive Mode checkbox in your Run Configuration.

-

Eclipse does not make it obvious when multiple console windows open; click on the Display Selected Console button to switch between open consoles:

-
-Display Selected Console icon

Display Selected Console icon

-
-

If the process associated with a console window has terminated, it will be indicated in the upper-left corner as below. Such console windows can then be closed using the grey cross button:

-
-Close terminated console icon

Close terminated console icon

-
-

The Eclipse console implementation does not support command history (up arrow) or auto-completion (tab).

-

External Libraries view

-

This view is automatically populated with a list of all the external modules referenced by the current file. Double-clicking on any item provides a quick way of opening GF files which are imported into your grammar from outside your Eclipse project, for example the GF Resource Grammar Library.

-

This view also provides a text box for searching within the current module's scope, as shown in the image below. Clicking the small arrow on the right-hand side of the view provides options for case sensitive and/or regular-expression based search.

-
-External libraries view

External libraries view

-
-

Test Manager view

-

This feature works together with the launch configurations to make the process of running regression tests on your grammars quick and easy. As described in the GF Book (Section 10.5), the general development-test cycle (independent of GFEP) is as follows:

-
    -
  1. Create a file test.trees which contains a bunch of abstract syntax trees, which you want to test
    2. -
  2. Compile your grammar and linearise the trees one by one, using some command like rf -lines -tree -file=test.trees | l -table | wf -file=test.trees.out
    3. -
  3. Manually correct the output in test.trees.out and save it as your gold standard file test.trees.gold.
    4. -
  4. Each time you update your grammar, repeat step 2, then compare your new output against the gold standard using diff or some other comparison tool.
    5. -
-

The Test Manager view follows this same pattern but provides a convenient graphical interface for running your test cases and also for looking at the output, just using a few clicks:

-
-Test Manager view

Test Manager view

-
-

Note that GFEP assumes the following convention:

- - - - - - - - - - - - - - - - - - - - - -
ExtensionDescription
*.trees or *.sentencesTest input files: trees file (for testing linearisation) and sentences file (for testing parsing)
*.trees.out or *.sentences.outCorresponding output files (generated each time a test is run)
*.trees.gold or *.sentences.goldCorresponding gold standard files (manually corrected from an output file)
-

Logs

-

The GFEP writes to the Eclipse platform log. This can be accessed in two ways:

-
    -
  1. By opening the Error Log view in Eclipse
    2. -
  2. Accessing the file directly at <workspace>/.metadata/.log
    3. -
-

You can set the log verbosity from Preferences → Grammatical Framework.

- - -

Common issues and fixes

-

No identifiers are resolved and my log contains messages saying "File Syntax.gf does not exist."

-

In general, this means that something is wrong with your library path setting.

-

GFEP attempts to compile all of your files individually with GF, which is not the usual way in which you would compile your grammar. As a result, when looking for the RGL files, GF needs a little help via the path argument.

-

The normal way to do this is to specify the Library path setting from Preferences → Grammatical Framework, setting it to something like .:alltenses:prelude (this is the default setting). Note that these paths are relative to the true RGL directory, which GF should already be aware of. The value of this setting is passed to GF from GFEP using the --path flag.

-

The Library path setting is the same for all your GF projects. If you want finer-grain control over the path compiler option then you should use compiler pragmas in your source files, e.g.:

-
--# -path=.:alltenses:prelude
-

The advantage of this method is that the information is bound to the file, so you can freely compile your modules outside of GFEP without having to specify the path as a command line option to GF. The disadvantage is that you need to specify such a pragma in every single file (at least if you want to continue to use the GFEP).

-

Other strange behaviour

-

When things seem strange, the very first step is always to clean the project via Project → Clean.... I only support the latest version of the plugin, so make sure you are up to date too.

-

If problems persist, please contact me or better yet file an issue on the GitHub issue tracker (see below).

- - -

Ontology Grammar plugin

-

For information about the Ontology Grammar plugin developed by Ontotext AD, please refer here.

- - -

For developers

-

Project setup

-

In order to build/run the GFEP from source, you need to have the Xtext libraries available in your Eclipse workspace. You have two choices:

-
    -
  1. Download the Xtext libraries from http://www.eclipse.org/Xtext/download/ and add them to your existing Eclipse instance
    2. -
  2. Download a release of Eclipse packaged together with Xtext from http://xtext.itemis.com/xtext/language=en/36553/downloads
    3. -
-

In both cases make sure you get the correct version of Xtext (see below). Once you have Xtext set up, you can clone the repository locally with the following command:

-
git clone git://github.com/GrammaticalFramework/gf-eclipse-plugin.git
-

You can then add the projects under the workspace directory to your Eclipse workspace, by using the File → Import... wizard. The individual projects are:

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
NamespaceDescription
org.grammaticalframework.eclipse.uiCovers all UI aspects of the plugin. Depends on org.grammaticalframework.eclipse (but the opposite is not true).
org.grammaticalframework.eclipse.testsContains unit tests for the plugin. Not required for running the plugin.
org.grammaticalframework.eclipse.ontology-grammarGF Ontology Grammar Plugin for Eclipse, developed separately by Ontotext AD.
org.grammaticalframework.featureFeature project for the GF Eclipse Plugin. Exports org.grammaticalframework.eclipse and org.grammaticalframework.eclipse.ui.
org.grammaticalframework.feature.ontology-grammarFeature project for the GF Ontology Grammar Plugin. Exports org.grammaticalframework.eclipse.ontology-grammar.
org.grammaticalframework.updatesiteUpdate site project which exports both feature projects under the same category.
-

Extra dependencies for unit testing project

-

The unit testing project (org.grammaticalframework.eclipse.tests) has an extra requirement for the library org.eclipselabs.xtext.utils.unittesting, which you will probably get complaints about when importing your projects into Eclipse. You only need to worry about this if you want to run the unit tests; otherwise you can ignore these errors. You can get more information about this library here, or just install it in Eclipse directly using the following update site URL: http://xtext-utils.eclipselabs.org.codespot.com/git.distribution/releases/unittesting-0.9.x/

-

Xtext version

-

GFEP currently uses Xtext 2.1.0.

-

Running from source

-

Once you have set up your projects (see above), you can run/debug the GFEP by following the steps here. Note that this means you will be running two instances of Eclipse simultaneously; the first debugging the second.

-

Building plugin package

-

To package the GFEP sources as .jar files and install/update the GFEP in a separate Eclipse installation, do the following:

-
    -
  1. Open site.xml in the org.grammaticalframework.updatesite project with Eclipse's Site Manifest Editor (default).
    2. -
  2. Synchronize the feature versions with the update site project.
    3. -
  3. Click Build All and the necessary .jar files should be created/updated in the project's folder. These can then be copied, along with site.xml to any local or web directory, which becomes the "update site".
    4. -
-

artifacts.jar and contents.jar

-

These are two meta-data files created by the update site project. However there are known issues with categories sporadically not working in update sites, and it seems that if you delete these 2 files then the problems are solved. More about this can be found here.

-

Other notes

-

Some have mentioned than when opening the GFEP sources, under Linux, they needed to install the following:

-
sudo apt-get install appmenu-gtk
-sudo apt-get install gtk2-engines-pixbuf
- - -

Evaluation & feedback

-

Please try to use the plugin for developing your own GF projects and report any issues you come up against. Feedback (both negative and positive) on the following would be appreciated:

-
    -
  1. Issues encountered with installation
    2. -
  2. Incorrect support of GF syntactic constructs, module hierarchy system
    3. -
  3. Performance issues (speed, resources footprint)
    4. -
  4. General intuitiveness of the UI
    5. -
  5. Feature wish-list
    6. -
  6. Compatibilities / conflicts with other Eclipse plugins
    7. -
-

For all bug reports and feature requests, please use the GitHub Issue Tracker.

- - -

Appendix

-

Module types

-

Description of the GF module types and their icons within the GF Eclipse Plugin.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
IconDescriptionNotes
Abstract moduleabstract Foods = {...}
Concrete moduleconcrete FoodsEng of Foods = {...}
Resource moduleresource ResFoods = {...}
Interfaceinterface LexFoods = {...}
Instanceinstance LexFoodsEng of LexFoods = {...}
Functorincomplete concrete FoodsI of Foods = {...}
Functor instantiationconcrete FoodsEng of Foods = FoodsI with (...) {...}
-

Build command

-

This is the GF command run by the builder:

-
gf --v=0 --tags --output-dir=.gfbuild [--path=PATH] FILE
- - - - diff --git a/examples/READ_ME b/examples/README.md similarity index 93% rename from examples/READ_ME rename to examples/README.md index 1c86317fa..e073602ab 100644 --- a/examples/READ_ME +++ b/examples/README.md @@ -1,3 +1,5 @@ +# `/examples` + All contributions (`contrib` and `examples` folders) have now been moved to a separate git repository at: @@ -10,5 +12,5 @@ have some toy grammars to play with. Any updates to these grammars should be made at the gf-contrib repository above; changes will be copied from there to here manually. -John J. Camilleri +John J. Camilleri, 2013-09-16 diff --git a/examples/phrasebook/WordsDut.gf b/examples/phrasebook/WordsDut.gf index 2ba3f3d78..12cab7861 100644 --- a/examples/phrasebook/WordsDut.gf +++ b/examples/phrasebook/WordsDut.gf @@ -3,7 +3,7 @@ concrete WordsDut of Words = SentencesDut ** - open SyntaxDut, (P = ParadigmsDut), (I = IrregDut), (L = LexiconDut), (R=ResDut), ExtraDut, Prelude in { + open SyntaxDut, (P = ParadigmsDut), (I = IrregDut), (L = LexiconDut), (R=ResDut), (E=ExtendDut), Prelude in { lin @@ -173,7 +173,7 @@ ik ga te voet/ ik ga lopend -- miscellaneous QWhatName p = mkQS (mkQCl how_IAdv (mkCl p.name I.heten_V)) ; --hoe heet je - QWhatAge p = mkQS (mkQCl (ICompAP (mkAP L.old_A)) p.name) ; + QWhatAge p = mkQS (mkQCl (E.ICompAP (mkAP L.old_A)) p.name) ; HowMuchCost item = mkQS (mkQCl how8much_IAdv (mkCl item (P.mkV "kosten"))) ; --hoeveel kost... ItCost item price = prop (mkCl item (P.mkV2 (P.mkV "kosten")) price) ; --..item.. kost ..price.. diff --git a/framenet/index.html b/framenet/index.html deleted file mode 100644 index 33c8d48aa..000000000 --- a/framenet/index.html +++ /dev/null @@ -1,5 +0,0 @@ - - - - Redirecting... - diff --git a/gf-book/index.html b/gf-book/index.html deleted file mode 100644 index 00a0f7afe..000000000 --- a/gf-book/index.html +++ /dev/null @@ -1,177 +0,0 @@ - - - - - - -Grammatical Framework: Programming with Multilingual Grammars - -
-

Grammatical Framework: Programming with Multilingual Grammars

-Aarne Ranta
-
- -

-

-

-

-This is the web page of the book -

- -
    -
  • Aarne Ranta, - Grammatical Framework: Programming with Multilingual Grammars, - CSLI Publications, - Stanford, - 2011, - 340 pp, - ISBN-10: 1-57586-626-9 (Paper), 1-57586-627-7 (Cloth). -
- -

Publisher's information

- -

-Publisher's web page (look for "Series" -> "CSLI Studies in Computational Linguistics") -

-

- http://www-csli.stanford.edu/pubs/ -

- -

Order

- -

-Direct order ($32.50 paperback, $70 hardcover): -

-

- http://press.uchicago.edu/ucp/books/book/distributed/G/bo12469871.html -

-

-Amazon order USA ($32.50 paperback, $70 hardcover): -

-

- http://www.amazon.com/Grammatical-Framework-Programming-Multilingual-Information/dp/1575866269/ref=sr_1_2?ie=UTF8&qid=1308585697&sr=8-2 -

-

-Amazon order UK (£21 paperback, £45 hardcover): -

-

- http://www.amazon.co.uk/Grammatical-Framework-Programming-Multilingual-Computational/dp/1575866269/ref=sr_1_6?ie=UTF8&qid=1315580083&sr=8-6 -

- -

Book description

- -

-Grammars of natural languages are complex systems, and their -computer implementation requires both programming skills and -linguistic knowledge, especially when dealing with other languages -than English. This book makes such tasks accessible for a wide -range of programmers. It introduces GF (Grammatical Framework), -which is a programming language designed for writing grammars, which -may moreover address several languages in parallel. -The book shows how to write grammars in GF and use them in applications -such as tourist phrasebooks, spoken dialogue systems, and natural -language interfaces. The examples and exercises address several -languages, and the readers are guided to look at their own languages -from the computational perspective. -

-

-With an emphasis on good engineering, the book promotes modularity -and division of labour - in particular, the use of libraries. It -introduces the GF Resource Grammar Library, which currently addresses -16 languages. This number is constantly growing due to contributions -from the international GF community. The library makes it painless to -build applications and to port them to new languages. The book -introduces a wide range of such applications, which run on platforms -ranging from web servers to mobile phones. But the book also gives -guidance for those readers who want to understand the underlying -linguistics and implement resource grammars for new languages. -

-

-The book starts with a hands-on tutorial, continues with a selection -of advanced topics, and ends with a complete reference manual -of GF. Requiring very little background knowledge, it is accessible -for second-year students that have experience with computers and an -interest for languages. At the same time, its novel and advanced -material makes it interesting for senior researchers in computer science, -linguistics, and related fields. -

- -

How to cite

- -
-  @Book{ranta-2011,
-    author = {Aarne Ranta},
-    title =  {{Grammatical Framework:
-               Programming with Multilingual Grammars}},
-    publisher = {{CSLI Publications}},
-    year = {2011},
-    address = "Stanford",
-    note = "ISBN-10: 1-57586-626-9 (Paper), 1-57586-627-7 (Cloth)"
-  }
-
- -

Table of Contents

- -

-Detailed Table of Contents -

- -

Supporting material

- -

-Slides for teaching the book chapter by chapter. -

-

-Code examples. You can also download the -complete example set as a compressed tar file, -gf-book-examples.tgz. -

-

-GF Web IDE: -build grammars in the cloud, without installing GF. -

-

-Demos -

- - - -

-Other tutorials: -

- - - -

-Works mentioned in references; -more download links will be added. -

-

-Resource Grammar Library API -

-

-Sources and binaries -for GF 3.2, which exactly matches the book. -

-

-GF Home Page: http://www.grammaticalframework.org/ -

-

-Author's email: aarne'at'chalmers'dot'se -

-

-Author's home page: http://www.cse.chalmers.se/~aarne/ -

- - - - diff --git a/gf-book/index.txt b/gf-book/index.t2t similarity index 100% rename from gf-book/index.txt rename to gf-book/index.t2t diff --git a/gf.cabal b/gf.cabal index 540a54197..54661ea13 100644 --- a/gf.cabal +++ b/gf.cabal @@ -14,33 +14,30 @@ maintainer: Thomas Hallgren tested-with: GHC==7.6.3, GHC==7.8.3, GHC==7.10.3, GHC==8.0.2 data-dir: src -data-files: tools/gf.el - tools/gf.lang - tools/dark-filetypes.GF.conf - tools/light-filetypes.GF.conf - www/*.html - www/*.css - www/P/*.png - www/gfse/*.html - www/gfse/*.css - www/gfse/*.js - www/gfse/P/*.png - www/gfse/P/*.jpg - www/js/*.js - www/minibar/*.html - www/minibar/*.css - www/minibar/*.js - www/minibar/*.png - www/syntax-editor/*.html - www/syntax-editor/*.css - www/syntax-editor/*.js - www/TransQuiz/*.html - www/TransQuiz/*.css - www/TransQuiz/*.js - www/TransQuiz/*.png - www/translator/*.html - www/translator/*.css - www/translator/*.js +data-files: + www/*.html + www/*.css + www/P/*.png + www/gfse/*.html + www/gfse/*.css + www/gfse/*.js + www/gfse/P/*.png + www/gfse/P/*.jpg + www/js/*.js + www/minibar/*.html + www/minibar/*.css + www/minibar/*.js + www/minibar/*.png + www/syntax-editor/*.html + www/syntax-editor/*.css + www/syntax-editor/*.js + www/TransQuiz/*.html + www/TransQuiz/*.css + www/TransQuiz/*.js + www/TransQuiz/*.png + www/translator/*.html + www/translator/*.css + www/translator/*.js --source-repository head -- type: darcs @@ -104,9 +101,9 @@ Library --if impl(ghc>=7.8) -- ghc-options: +RTS -A20M -RTS ghc-prof-options: -fprof-auto - extensions: + extensions: - exposed-modules: + exposed-modules: PGF PGF.Internal PGF.Haskell diff --git a/index.html b/index.html index 664aa2b63..f00aaabee 100644 --- a/index.html +++ b/index.html @@ -4,9 +4,7 @@ GF - Grammatical Framework - - - + - - - - - -
- - -

Grammatical Framework

-

-Version 3.5 -
-August 2013 - -

-

-[ Demos -| Download -| Libraries -| Reference -| Tutorial -| QuickStart -| UserGroup -| Book -| GitHub -| Interactive Map -| GF Cloud Service -] -
-
-

-[ Issues -| Wiki -| People -| Publications -| QuickRefCard -| GF Shell Reference -| GF Editor Modes -| GF Eclipse Plugin -| LibTutorial -| PGF library API -| MOLTO -| SummerSchool -| Android -| Browse GF Source Code -| GF build statusNew! -] -

-
-

News

- -
- - -
2014-03-11: - A company for commercial applications of GF has been founded: - Digital Grammars. -
2013-11-25: - The default character encoding in GF grammar files will be changed - from Latin-1 to UTF-8. See - GF character encoding changes - for details. -
2013-10-18:New resource grammar language: Estonian. - See library synopsis. -
2013-09-18:New GF contributions repository, hosted on GitHub. -
2013-08-06:GF 3.5 released! - Release notes. -
2013-07-26:Started a page with RGL Documentation and Publications. -
2013-06-24:We are now running the IRC channel #gf on the Freenode network. -
2013-06-19:New resource grammar language: Maltese. - See library synopsis. -
2013-04-25:New resource grammar language: Greek. - See library synopsis. -
2013-01-31:GF 3.4 released! - Release notes. -
2012-12-10: - Resource Grammar Library - coverage map, created by Tommi Nieminen. - -
- -
- -

What is GF

-

-GF, Grammatical Framework, is a programming language for -multilingual grammar applications. It is -

-
    -
  • a special-purpose language for grammars, like - YACC, - Bison, - Happy, - BNFC, - but not restricted to programming languages -
  • a functional programming language, like - Haskell, - Lisp, - OCaml, - SML, - Scheme, - but specialized to grammar writing -
  • a development platform for natural language grammars, like - LKB, - XLE, - Regulus, - but based on functional programming and type theory -
  • a categorial grammar formalism, like - ACG, - CCG, - but specialized for multilingual grammars, -
  • a logical framework, like - Agda, - Coq, - Isabelle, - but equipped with concrete syntax in addition to logic -
  • a platform for machine translation, like - Moses, - Apertium, - but based on deep structural analysis (and usually applied for - limited fragments of language). -
- -

-Don't worry if you don't know most of the references above - but if you do know at -least one, it may help you to get a first idea of what GF is. -

-

Applications

-

-GF can be used for building -

- - -

Availability

-

-GF is open-source, licensed under GPL (the program) and -LGPL and BSD (the libraries). It -is available for -

-
    -
  • Linux -
  • Mac OS X -
  • Windows -
  • Android mobile platform (via Java; runtime) -
  • via compilation to JavaScript, almost any platform that has a web browser (runtime) -
- -

Projects

-

-GF was first created in 1998 at -Xerox Research Centre Europe, -Grenoble, in the project -Multilingual Document Authoring. At Xerox, it was used for prototypes including -a restaurant phrase book, -a database query system, -a formalization of an alarm system instructions with translations to 5 languages, and -an authoring system for medical drug descriptions. -

-

-Later projects using GF and involving third parties include, in chronological order, -

-
    -
  • GF-Alfa: - natural language interface to formal proofs -
  • Efficient: - authoring tool for business models. -
  • GF-KeY: - authoring and translation of software specifications -
  • TALK: - multilingual and multimodal spoken dialogue systems -
  • WebALT: - multilingual generation of mathematical exercises (commercial project) -
  • SALDO: - Swedish morphological dictionary based on rules developed for GF and - Functional Morphology -
  • MOLTO: - multilingual online translation -
  • REMU: - reliable multilingual digital communication. -
-

-Here is a talk -about GF at XRCE, -14 years later. - -

-Academically, GF has been used in at least ten PhD theses and resulted -in more than a hundred -scientific publications (see GF publication list). -

-

Programming in GF

-

-GF is easy to learn by following the tutorial. -You can write your first translator in 15 minutes. -

-

-GF has an interactive command interpreter, as well as a batch compiler. -Grammars can be compiled to parser and translator code in many different -formats. These components can then be embedded in applications written -in other programming languages. The formats currently supported are: -

-
    -
  • Haskell -
  • Java, in particular the Android platform -
  • JavaScript -
  • Speech recognition: HTK/ATK, Nuance, JSGF -
- -

-The GF programming language is high-level and advanced, featuring -

-
    -
  • static type checking -
  • higher-order functions -
  • dependent types -
  • pattern matching with data constructors and regular expressions -
  • module system with multiple inheritance and parametrized modules -
- -

Getting help

-

-If you need some help with GF, the first places to start are the Tutorial and Reference pages. -The printed book contains all the material in the tutorial and some extra bits, and is the recommended reference for GF. -

- -

-We run the IRC channel #gf on the Freenode network, where you are welcome to look for help with small questions or just start a general discussion. -IRC logs (in raw format) are available here. -If you have a larger question which the community may benefit from, we recommend you ask it on the mailing list. -

- -

Libraries

-

-Libraries are at the heart of modern software engineering. In natural language -applications, libraries are a way to cope with thousands of details involved in -syntax, lexicon, and inflection. The -GF resource grammar library has -support for an increasing number of languages, currently including -

-
    -
  1. Afrikaans -
  2. Amharic (partial) -
  3. Arabic (partial) -
  4. Bulgarian -
  5. Catalan -
  6. Chinese -
  7. Danish -
  8. Dutch -
  9. English -
  10. Estonian -
  11. Finnish -
  12. French -
  13. German -
  14. Greek -
  15. Hebrew (fragments) -
  16. Hindi -
  17. Interlingua -
  18. Japanese -
  19. Italian -
  20. Latin (fragments) -
  21. Latvian -
  22. Maltese -
  23. Nepali -
  24. Norwegian bokmål -
  25. Persian -
  26. Polish -
  27. Punjabi -
  28. Romanian -
  29. Russian -
  30. Sindhi -
  31. Spanish -
  32. Swahili (fragments) -
  33. Swedish -
  34. Thai -
  35. Turkish (fragments) -
  36. Urdu -
- -

-Adding a language to the resource library takes 3 to 9 -months - contributions -are welcome! You can start with the resource grammarian's tutorial. - - - - - - diff --git a/src/compiler/GF/Text/Transliterations.hs b/src/compiler/GF/Text/Transliterations.hs index 7645fc158..9b1b6e151 100644 --- a/src/compiler/GF/Text/Transliterations.hs +++ b/src/compiler/GF/Text/Transliterations.hs @@ -39,6 +39,7 @@ allTransliterations = Map.fromList [ ("amharic",transAmharic), ("ancientgreek", transAncientGreek), ("arabic", transArabic), + ("arabic_unvocalized", transArabicUnvoc), ("devanagari", transDevanagari), ("greek", transGreek), ("hebrew", transHebrew), @@ -178,6 +179,13 @@ transArabic = mkTransliteration "Arabic" allTrans allCodes where allCodes = [0x0621..0x062f] ++ [0x0630..0x063a] ++ [0x0641..0x064f] ++ [0x0650..0x0657] ++ [0x0671,0x061f] + +transArabicUnvoc :: Transliteration +transArabicUnvoc = transArabic{ + invisible_chars = ["a","u","i","v2","o","V+","V-","a:"], + printname = "unvocalized Arabic" + } + transPersian :: Transliteration transPersian = (mkTransliteration "Persian/Farsi" allTrans allCodes) {invisible_chars = ["a","u","i"]} where diff --git a/src/runtime/typescript/gflib.d.ts b/src/runtime/typescript/gflib.d.ts new file mode 100644 index 000000000..4249e66d2 --- /dev/null +++ b/src/runtime/typescript/gflib.d.ts @@ -0,0 +1,337 @@ +/** + * gflib.dt.s + * + * by John J. Camilleri + * + * TypeScript type definitions for the "original" JS GF runtime (GF:src/runtime/javascript/gflib.js) + */ + +// Note: the String prototype is extended with: +// String.prototype.tag = ""; +// String.prototype.setTag = function (tag) { this.tag = tag; }; + +/** + * A GF grammar is one abstract and multiple concretes + */ +declare class GFGrammar { + abstract: GFAbstract + concretes: {[key: string]: GFConcrete} + + constructor(abstract: GFAbstract, concretes: {[key: string]: GFConcrete}) + + translate( + input: string, + fromLang: string, + toLang: string + ): {[key: string]: {[key: string]: string}} +} + +/** + * Abstract Syntax Tree + */ +declare class Fun { + name: string + args: Fun[] + + constructor(name: string, ...args: Fun[]) + + print(): string + show(): string + getArg(i: number): Fun + setArg(i: number, c: Fun): void + isMeta(): boolean + isComplete(): boolean + isLiteral(): boolean + isString(): boolean + isInt(): boolean + isFloat(): boolean + isEqual(obj: any): boolean +} + +/** + * Abstract syntax + */ +declare class GFAbstract { + startcat: string + types: {[key: string]: Type} // key is function name + + constructor(startcat: string, types: {[key: string]: Type}) + + addType(fun: string, args: string[], cat: string): void + getArgs(fun: string): string[] + getCat(fun: string): string + annotate(tree: Fun, type: string): Fun + handleLiterals(tree: Fun, type: Type): Fun + copyTree(x: Fun): Fun + parseTree(str: string, type: string): Fun + parseTree_(tokens: string[], prec: number): Fun +} + +/** + * Type + */ +declare class Type { + args: string[] + cat: string + + constructor(args: string[], cat: string) +} + +type ApplyOrCoerce = Apply | Coerce + +/** + * Concrete syntax + */ +declare class GFConcrete { + flags: {[key: string]: string} + productions: {[key: number]: ApplyOrCoerce[]} + functions: CncFun[] + sequences: Array> + startCats: {[key: string]: {s: number, e: number}} + totalFIds: number + pproductions: {[key: number]: ApplyOrCoerce[]} + lproductions: {[key: string]: {fid: FId, fun: CncFun}} + + constructor( + flags: {[key: string]: string}, + productions: {[key: number]: ApplyOrCoerce[]}, + functions: CncFun[], + sequences: Array>, + startCats: {[key: string]: {s: number, e: number}}, + totalFIds: number + ) + + linearizeSyms(tree: Fun, tag: string): Array<{fid: FId, table: any}> + syms2toks(syms: Sym[]): string[] + linearizeAll(tree: Fun): string[] + linearize(tree: Fun): string + tagAndLinearize(tree: Fun): string[] + unlex(ts: string): string + tagIt(obj: any, tag: string): any + // showRules(): string // Uncaught TypeError: Cannot read property 'length' of undefined at gflib.js:451 + tokenize(string: string): string[] + parseString(string: string, cat: string): Fun[] + complete( + input: string, + cat: string + ): {consumed: string[], suggestions: string[]} +} + +/** + * Function ID + */ +type FId = number + +/** + * Apply + */ +declare class Apply { + id: string + fun: FId + args: PArg[] + + constructor(fun: FId, args: PArg[]) + + show(cat: string): string + isEqual(obj: any): boolean +} + +/** + * PArg + */ +declare class PArg { + fid: FId + hypos: any[] + + constructor(fid: FId, ...hypos: any[]) +} + +/** + * Coerce + */ +declare class Coerce { + id: string + arg: FId + + constructor(arg: FId) + + show(cat: string): string +} + +/** + * Const + */ +declare class Const { + id: string + lit: Fun + toks: any[] + + constructor(lit: Fun, toks: any[]) + + show(cat: string): string + isEqual(obj: any): boolean +} + +/** + * CncFun + */ +declare class CncFun { + name: string + lins: FId[] + + constructor(name: string, lins: FId[]) +} + +type Sym = SymCat | SymKS | SymKP | SymLit + +/** + * SymCat + */ +declare class SymCat { + id: string + i: number + label: number + + constructor(i: number, label: number) + + getId(): string + getArgNum(): number + show(): string +} + +/** + * SymKS + */ +declare class SymKS { + id: string + tokens: string[] + + constructor(...tokens: string[]) + + getId(): string + show(): string +} + +/** + * SymKP + */ +declare class SymKP { + id: string + tokens: string[] + alts: Alt[] + + constructor(tokens: string[], alts: Alt[]) + + getId(): string + show(): string +} + +/** + * Alt + */ +declare class Alt { + tokens: string[] + prefixes: string[] + + constructor(tokens: string[], prefixes: string[]) +} + +/** + * SymLit + */ +declare class SymLit { + id: string + i: number + label: number + + constructor(i: number, label: number) + + getId(): string + show(): string +} + +/** + * Trie + */ +declare class Trie { + value: any + items: Trie[] + + insertChain(keys, obj): void + insertChain1(keys, obj): void + lookup(key, obj): any + isEmpty(): boolean +} + +/** + * ParseState + */ +declare class ParseState { + concrete: GFConcrete + startCat: string + items: Trie + chart: Chart + + constructor(concrete: GFConcrete, startCat: string) + + next(token: string): boolean + complete(correntToken: string): Trie + extractTrees(): any[] + process( + agenda, + literalCallback: (fid: FId) => any, + tokenCallback: (tokens: string[], item: any) => any + ): void +} + +/** + * Chart + */ +declare class Chart { + active: any + actives: {[key: number]: any} + passive: any + forest: {[key: number]: ApplyOrCoerce[]} + nextId: number + offset: number + + constructor(concrete: GFConcrete) + + lookupAC(fid: FId,label) + lookupACo(offset, fid: FId, label) + + labelsAC(fid: FId) + insertAC(fid: FId, label, items): void + + lookupPC(fid: FId, label, offset) + insertPC(fid1: FId, label, offset, fid2: FId): void + shift(): void + expandForest(fid: FId): any[] +} + +/** + * ActiveItem + */ +declare class ActiveItem { + offset: number + dot: number + fun: CncFun + seq: Array + args: PArg[] + fid: FId + lbl: number + + constructor( + offset: number, + dot: number, + fun: CncFun, + seq: Array, + args: PArg[], + fid: FId, + lbl: number + ) + + isEqual(obj: any): boolean + shiftOverArg(i: number, fid: FId): ActiveItem + shiftOverTokn(): ActiveItem +} diff --git a/src/tools/gf-tools.cabal b/src/tools/gf-tools.cabal index 47ce0f01c..1f89f3a85 100644 --- a/src/tools/gf-tools.cabal +++ b/src/tools/gf-tools.cabal @@ -9,23 +9,4 @@ Executable gfdoc Executable htmls main-is: Htmls.hs - build-depends: base - - -library - hs-source-dirs: gftest - exposed-modules: Grammar - other-modules: Mu, Graph, FMap, EqRel - build-depends: base - , containers - , pgf2 - -executable gftest - hs-source-dirs: gftest - main-is: Main.hs - build-depends: base - , pgf2 - , cmdargs - , containers - , filepath - , gf-tools \ No newline at end of file + build-depends: base \ No newline at end of file diff --git a/src/tools/gf.el b/src/tools/gf.el index c8ebc9a50..594e0845e 100644 --- a/src/tools/gf.el +++ b/src/tools/gf.el @@ -1,1350 +1 @@ -;;; gf.el --- Major mode for editing GF code -*-coding: iso-8859-1;-*- - -;; Copyright (C) 2005, 2006, 2007 Johan Bockgård -;; Time-stamp: <2007-06-16 11:57:48 bojohan> - -;; Author: Johan Bockgård -;; Keywords: languages - -;; This file is free software; you can redistribute it and/or modify -;; it under the terms of the GNU General Public License as published by -;; the Free Software Foundation; either version 2, or (at your option) -;; any later version. - -;; This file is distributed in the hope that it will be useful, -;; but WITHOUT ANY WARRANTY; without even the implied warranty of -;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -;; GNU General Public License for more details. - -;; You should have received a copy of the GNU General Public License -;; along with GNU Emacs; see the file COPYING. If not, write to -;; the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, -;; Boston, MA 02110-1301, USA. - -;;; Commentary: - -;; Major mode for editing GF code, with support for running a GF -;; shell. - -;;; Usage: - -;; To use this library, put it somewhere Emacs can find it (in -;; `load-path') and add the following lines to your .emacs file. - -;; (autoload 'run-gf "gf" nil t) -;; (autoload 'gf-mode "gf" nil t) -;; (add-to-list 'auto-mode-alist '("\\.gf\\(\\|e\\|r\\|cm?\\)\\'" . gf-mode)) -;; (add-to-list 'auto-mode-alist '("\\.cf\\'" . gf-mode)) -;; (add-to-list 'auto-mode-alist '("\\.ebnf\\'" . gf-mode)) - -;;; History: - -;; 2006-10-30: -;; let a = b -;; c = d ... -;; in ... -;; indentation now works (most of the time). - -;;; Code: - -(eval-when-compile (require 'cl)) - -(defgroup gf nil - "Support for GF (Grammatical Framework)" - :group 'languages - ;; :link '(emacs-commentary-link "gf") - :link '(url-link "http://www.cs.chalmers.se/~aarne/GF/")) - -(defvar gf-mode-map - (let ((map (make-sparse-keymap))) - (define-key map "\C-c\C-l" 'gf-load-file) - (define-key map "\C-c\C-b" 'gf-display-inf-buffer) - (define-key map "\C-c\C-s" 'run-gf) - (define-key map (kbd "DEL") 'backward-delete-char-untabify) - map) - "Keymap for `gf-mode'.") - -;; Taken from haskell-mode -(defvar gf-mode-syntax-table - (let ((table (make-syntax-table))) - (modify-syntax-entry ?\ " " table) - (modify-syntax-entry ?\t " " table) - (modify-syntax-entry ?\" "\"" table) - (modify-syntax-entry ?\' "\'" table) - (modify-syntax-entry ?_ "w" table) - (modify-syntax-entry ?\( "()" table) - (modify-syntax-entry ?\) ")(" table) - (modify-syntax-entry ?\[ "(]" table) - (modify-syntax-entry ?\] ")[" table) - ;; (modify-syntax-entry ?\< "(>" table) - ;; (modify-syntax-entry ?\> ")<" table) - - (cond ((featurep 'xemacs) - ;; I don't know whether this is equivalent to the below - ;; (modulo nesting). -- fx - (modify-syntax-entry ?{ "(}5" table) - (modify-syntax-entry ?} "){8" table) - (modify-syntax-entry ?- "_ 1267" table)) - (t - ;; The following get comment syntax right, similarly to C++ - ;; In Emacs 21, the `n' indicates that they nest. - ;; The `b' annotation is actually ignored because it's only - ;; meaningful on the second char of a comment-starter, so - ;; on Emacs 20 and before we get wrong results. --Stef - (modify-syntax-entry ?\{ "(}1nb" table) - (modify-syntax-entry ?\} "){4nb" table) - (modify-syntax-entry ?- "_ 123" table))) - (modify-syntax-entry ?\n ">" table) - - (let (i lim) - (map-char-table - (lambda (k v) - (when (equal v '(1)) - ;; The current Emacs 22 codebase can pass either a char - ;; or a char range. - (if (consp k) - (setq i (car k) - lim (cdr k)) - (setq i k - lim k)) - (while (<= i lim) - (when (> i 127) - (modify-syntax-entry i "_" table)) - (setq i (1+ i))))) - (standard-syntax-table))) - - (modify-syntax-entry ?\` "$`" table) - (modify-syntax-entry ?\\ "\\" table) - (mapcar (lambda (x) - (modify-syntax-entry x "_" table)) - ;; Some of these are actually OK by default. - "!#$%&*+./:=?@^|~") - (unless (featurep 'mule) - ;; Non-ASCII syntax should be OK, at least in Emacs. - (mapcar (lambda (x) - (modify-syntax-entry x "_" table)) - (concat "¡¢£¤¥¦§¨©ª«¬­®¯°±²³´µ¶·¸¹º»¼½¾¿" - "×÷")) - (mapcar (lambda (x) - (modify-syntax-entry x "w" table)) - (concat "ÀÁÂÃÄÅÆÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖ" - "ØÙÚÛÜÝÞß" - "àáâãäåæçèéêëìíîïðñòóôõö" - "øùúûüýþÿ"))) - table) - "Syntax table used in GF mode.") - -;; Lin PType Str Strs Tok Type -;; abstract case cat concrete data def -;; flags fn fun grammar in include -;; incomplete instance interface let lin lincat -;; lindef lintype of open oper out -;; package param pattern pre printname resource -;; reuse strs table tokenizer transfer union -;; var variants where with -;; ; = { } ( ) : -> ** , [ ] - . | % ? < > @ ! * \ => ++ + _ $ / - -;; Judgements -(defvar gf-top-level-keywords - '("cat" "fun" "lincat" "lintype" "lin" "pattern" - "oper" "def" "param" "flags" "lindef" "printname" - "data" "transfer" - )) - -(defvar gf-module-keywords - '("abstract" "concrete" "resource" "instance" "interface")) - -(defvar gf-keywords - (append '("of" "let" "include" "open" "in" "where" - "with" "case" "incomplete" "table" - "variants" "pre" "strs" "overload") - gf-top-level-keywords - gf-module-keywords)) - -(defvar gf-top-level-keyword-regexp (regexp-opt gf-top-level-keywords 'words)) -(defvar gf-keyword-regexp (regexp-opt gf-keywords 'words)) - -(defvar gf-font-lock-keywords - (let ((sym "\\(\\s_\\|\\\\\\)+") - ;; (keyw gf-keyword-regexp) - (mod (concat (regexp-opt gf-module-keywords 'words) - "\\s-\\(\\w+\\)")) - (pface '(if (boundp 'font-lock-preprocessor-face) - font-lock-preprocessor-face - font-lock-builtin-face))) - `(;; Module - (,mod (1 font-lock-keyword-face) - (2 font-lock-type-face)) - ;; Keywords - (,(lambda (end) - (let (parse-sexp-lookup-properties) - (re-search-forward gf-keyword-regexp end t))) - . font-lock-keyword-face) - ;; Operators - (,sym . font-lock-variable-name-face) - ;; Pragmas - ("^--\\(#.*\\)" (1 ,pface prepend)) - ("--\\(#\\s-*\\(notpresent\\|prob\\).*\\)" (1 ,pface prepend)) - ;; GFDoc - ("^--[0-9]+\\s-*\\(.*\\)" (1 'underline prepend)) - ("^--\\([*!.]\\)" (1 'underline prepend)) - (,(lambda (end) - (let (found) - (while - (and (setq found (re-search-forward - ;; "\\$.*?\\$\\|\\*.*?\\*\\|\".*?\"" - "\\$.*?\\$" - end t)) - (not (eq (get-text-property (match-beginning 0) 'face) - 'font-lock-comment-face)))) - found)) - (0 (if (face-italic-p 'font-lock-comment-face) - '((:slant normal)) - '((:slant italic))) - prepend)) - ;; Types (?) - ;; ("[^.]\\(\\<[[:upper:]]\\w*\\)[^.]" 1 font-lock-type-face) - )) - "Keyword highlighting specification for `gf-mode'.") - -(defcustom gf-let-brace-style t - "The let...in style to use for indentaton. - -A value of t means unbraced (new) style: - - - let x = a; - y = b; in ... - -A value of nil means braced (old) style - - let { x = a; - y = b; } in ... - -Anything else means try to guess." - :type '(choice (const :tag "Unbraced" t) - (const :tag "Braced" nil) - (const :tag "Heuristic" 'heuristic)) - :group 'gf) - -;; let x = let a = f; -;; b = g; -;; in b; -;; y = d; -;; in h -(defun gf-match-let/in (let/in end) - (when gf-let-brace-style - (if (eq 'let let/in) - (and (re-search-forward "\\" end t) - (or (eq t gf-let-brace-style) - (save-excursion - (skip-syntax-forward " ") - (not (eq ?\{ (char-after)))))) - (and (re-search-forward "\\<\\(i\\)n\\>" end t) - (or (eq t gf-let-brace-style) - (save-excursion - (backward-char 2) - (skip-syntax-backward " ") - (not (eq ?\} (char-before))))))))) - -(defvar gf-font-lock-syntactic-keywords - `(;; let ... - (,(lambda (end) (gf-match-let/in 'let end)) - 1 "(") - ;; ;; open ... - ;; (,(lambda (end) - ;; (and (re-search-forward "\\" end t) - ;; (save-excursion - ;; (skip-syntax-forward " ") - ;; (not (eq ?\{ (char-after)))))) - ;; 1 "(") - ;; ... in - (,(lambda (end) (gf-match-let/in 'in end)) - 1 ")") - )) - - ;; (defvar gf-imenu-generic-expression - ;; ...) - - ;; (defvar gf-outline-regexp - ;; ...) - -;;;###autoload -(define-derived-mode gf-mode fundamental-mode "GF" - "A major mode for editing GF files." - ;; (set (make-local-variable 'imenu-generic-expression) - ;; gf-imenu-generic-expression) - ;; (set (make-local-variable 'outline-regexp) sample-outline-regexp) - (set (make-local-variable 'comment-start) "-- ") - (set (make-local-variable 'comment-start-skip) "[-{]-[ \t]*") - (set (make-local-variable 'font-lock-defaults) - '(gf-font-lock-keywords - nil nil nil nil - (font-lock-syntactic-keywords . gf-font-lock-syntactic-keywords) - )) - (set (make-local-variable 'indent-line-function) 'gf-indent-line) - (set (make-local-variable 'eldoc-documentation-function) 'gf-doc-display) - (set (make-local-variable 'beginning-of-defun-function) - 'gf-beginning-of-section) - (set (make-local-variable 'end-of-defun-function) - 'gf-end-of-section)) - -;;; Indentation -(defcustom gf-indent-basic-offset 2 - "*Number of columns to indent in GF mode." - :type 'integer - :group 'gf) - -(defcustom gf-indent-judgment-offset 2 - "*Column where judement should be indented to." - :type 'integer - :group 'gf) - -(defun gf-indent-line () - "Indent current line of GF code." - (interactive) - (save-excursion - (font-lock-fontify-syntactic-keywords-region - (point-at-bol) (point-at-bol))) - (let* ((case-fold-search nil) - (parse-sexp-lookup-properties t) - (parse-sexp-ignore-comments t) - (savep (> (current-column) (current-indentation))) - (indent (condition-case err - (max (gf-calculate-indentation) 0) - (error (message "%s" err) 0)))) - (if savep - (save-excursion (indent-line-to indent)) - (indent-line-to indent)))) - -(defun gf-beginning-of-section () - (when (re-search-backward - (concat "^\\s-*" gf-top-level-keyword-regexp) - nil 'move) - (goto-char (match-beginning 0))) - (point)) - -(defun gf-end-of-section () - (gf-forward-comment) - (when (looking-at gf-top-level-keyword-regexp) - (goto-char (match-end 0))) - (when (re-search-forward - (concat "^\\s-*" gf-top-level-keyword-regexp) - (condition-case nil - (1- (scan-lists (point) 1 1)) - (error nil)) - 'move) - (goto-char (match-beginning 0))) - (gf-backward-comment) - (point)) - -;; (defun gf-beginning-of-defun () -;; (let* ((beg (save-excursion (gf-beginning-of-section) (point))) -;; (end (save-excursion (progn (forward-line 0) (point)))) -;; (pps (parse-partial-sexp beg end))) -;; (when (nth 1 pps) (backward-up-list (nth 0 pps))) -;; (back-to-indentation) -;; (point))) - -(defun gf-beginning-of-sequence (&optional keep-going limit) - (or limit (let ((com-start (gf-in-comment-p))) - (when com-start - (save-excursion - (goto-char com-start) - (skip-chars-forward "{") - (skip-chars-forward "-") - (setq limit (point)))))) - (let* ((str "[;]") - (found-it nil) - (pps (gf-ppss)) - (depth (or (nth 0 pps) 0)) - (bol (point-at-bol)) - (lim (max (or limit (point-min)) - (if (nth 1 pps) - (1+ (nth 1 pps)) - (save-excursion - (gf-beginning-of-section) - (when (looking-at - (concat "\\s-*" gf-top-level-keyword-regexp)) - (goto-char (match-end 0)) - (gf-forward-comment)) - (point)))))) - (while (and (> (point) lim) - (setq found-it (re-search-backward str lim 'move)) - (let ((pps (gf-ppss))) - (or (/= depth (nth 0 pps)) - (nth 3 pps) - (nth 4 pps))))) - (when found-it - (when keep-going - (setq lim (max lim bol)) - (while (and (> (point) lim) - (setq found-it (re-search-backward str lim 'move)) - ;;(/= depth (nth 0 (gf-ppss))) - ))) - (when found-it (forward-char))))) - -(defun gf-in-comment-p () - (let ((pps (gf-ppss))) - (and (nth 4 pps) (nth 8 pps)))) - -(defun gf-forward-comment () (forward-comment (buffer-size))) -(defun gf-backward-comment () - (forward-comment (- (buffer-size))) - ;; (while (or (not (zerop (skip-chars-backward " \t\n"))) - ;; (let ((start (gf-in-comment))) - ;; (and start (goto-char start))))) - ) - - -(defun gf-ppss ();; (&optional from to) - (parse-partial-sexp - (save-excursion (gf-beginning-of-section)) - (point))) - -(if (fboundp 'syntax-after) - (defalias 'gf-syntax-after 'syntax-after) - (defun gf-syntax-after (pos) - "Return the raw syntax of the char after POS. -If POS is outside the buffer's accessible portion, return nil." - (unless (or (< pos (point-min)) (>= pos (point-max))) - (let ((st (if parse-sexp-lookup-properties - (get-char-property pos 'syntax-table)))) - (if (consp st) st - (aref (or st (syntax-table)) (char-after pos))))))) - -(if (fboundp 'syntax-class) - (defalias 'gf-syntax-class 'syntax-class) - (defun gf-syntax-class (syntax) - "Return the syntax class part of the syntax descriptor SYNTAX. -If SYNTAX is nil, return nil." - (and syntax (logand (car syntax) 65535)))) - -(defun gf-calculate-indentation () - "Return the column to which the current line should be indented." - (save-excursion - (forward-line 0) - (skip-chars-forward " \t") - (cond - ;; judgement - ((looking-at gf-top-level-keyword-regexp) - gf-indent-judgment-offset) - ((and gf-let-brace-style - (looking-at "in\\>")) - (if (condition-case nil - (progn (backward-up-list) - nil) - (error t)) - gf-indent-basic-offset - (gf-beginning-of-sequence) - (if (= (point) (point-min)) - 0 - (gf-forward-comment) - (+ gf-indent-basic-offset (current-column))))) - ((looking-at "[]})]") - ;; (looking-at "[]})]") - ;; (eq 5 (gf-syntax-class (gf-syntax-after (point)))) - (backward-up-list) - (gf-beginning-of-sequence) - (if (= (point) (point-min)) - 0 - (gf-forward-comment) - (+ gf-indent-basic-offset (current-column)))) - ;; heading - ((looking-at "---") - (gf-beginning-of-sequence) - (if (= (point) (point-min)) - 0 - gf-indent-judgment-offset)) - (t - (let ((opoint (point))) - (gf-backward-comment) - (cond - ((eq ?\; (char-before)) - ;; ?\, - (backward-char) - (gf-beginning-of-sequence t) - (gf-forward-comment) - (current-column)) - (;; (memq (char-before) '( ?\{ ?\[ ?\( ;; ?\< - ;; )) - (eq 4 (gf-syntax-class (gf-syntax-after (1- (point))))) - (backward-char) - ;; alt. (gf-beginning-of-sequence nil nil) - (gf-beginning-of-sequence nil (point-at-bol)) - (gf-forward-comment) - ;; alt. (+ (* 2 gf-indent-basic-offset) (current-column))) - (+ gf-indent-basic-offset (current-column))) - (t - (gf-beginning-of-sequence) - (let ((head (= (point) (point-min)))) - (gf-forward-comment) - (cond - ;; ((< opoint (point)) 0) - ((> opoint (point)) (+ gf-indent-basic-offset (current-column))) - ;; i.e. opoint == (point) - (head 0) - (t (gf-beginning-of-section) - (skip-chars-forward " \t") - (+ gf-indent-basic-offset (current-column)))))))))))) - -(defun gf-load-file () - (interactive) - (start-gf) - (comint-send-string gf-process (format "i %s\n" buffer-file-name)) - (gf-clear-lang-cache) - (gf-display-inf-buffer)) - -(defun gf-display-inf-buffer () - (interactive) - (and (get-buffer gf-process-buffer-name) - (display-buffer gf-process-buffer-name))) - -;; Inferior GF Mode ---------------- - -(defcustom gf-program-name "gf" - "*Name of GF shell invoked by `run-gf'." - :type 'file - :group 'gf) -(defvar gf-program-args nil "*Arguments passed to GF by `run-gf'.") -(defvar gf-process-buffer-name "*gf*") -(defvar gf-process) - -(require 'comint) - -(define-derived-mode inf-gf-mode comint-mode "Inf-GF" - (gf-setup-pcomplete)) - -(define-key inf-gf-mode-map "\t" 'gf-complete) - -;;;###autoload -(defun run-gf () - "Run an inferior GF process." - (interactive) - (start-gf) - (pop-to-buffer gf-process-buffer-name)) - -(defun start-gf () - (unless (comint-check-proc gf-process-buffer-name) - (with-current-buffer - (apply 'make-comint-in-buffer - "gf" gf-process-buffer-name gf-program-name - nil gf-program-args) - (setq gf-process (get-buffer-process (current-buffer))) - (set-buffer-process-coding-system 'latin-1-unix 'latin-1-unix) - (inf-gf-mode)))) - -;; (defun gf-norm-func (string) -;; (if (string-match "\\(.*\\)\\(=[^=]*\\)" string) -;; (match-string 1 string) -;; string)) - -;; (defmacro gf-pcomplete-here (&optional form stub paring form-only) -;; `(pcomplete-here ,form ,stub (or ,paring 'gf-norm-func) ,form-only)) -;; (put 'gf-pcomplete-here 'edebug-form-spec t) - -(put 'pcomplete-here 'edebug-form-spec t) - -(defun gf-setup-pcomplete () - (set (make-local-variable 'comint-prompt-regexp) "^[^>\n]*> *") - (set (make-local-variable 'pcomplete-ignore-case) nil) - (set (make-local-variable 'pcomplete-use-paring) t) - (set (make-local-variable 'pcomplete-suffix-list) '(?/ ?=)) - ;; (set (make-local-variable 'comint-dynamic-complete-functions) - ;; (add-to-list 'comint-dynamic-complete-functions 'pcomplete)) - (set (make-local-variable 'pcomplete-parse-arguments-function) - 'gf-parse-arguments) - (set (make-local-variable 'pcomplete-command-completion-function) - 'gf-complete-command) - ;; (set (make-local-variable 'pcomplete-command-name-function) - ;; 'pcomplete-erc-command-name) - (set (make-local-variable 'pcomplete-default-completion-function) - 'gf-default-completion-function) - (add-hook 'comint-input-filter-functions - 'gf-watch-for-loading - nil t)) - -(defun gf-watch-for-loading (string) - (when (string-match (concat "\\(\\`\\||\\;;\\)\\s-*" - (regexp-opt '("i" "e" "rl") 'words)) - string) - (gf-clear-lang-cache))) - -(defun gf-parse-arguments () - "Parse whitespace separated arguments in the current region." - (let ((begin (save-excursion - ;; (parse-partial-sexp begin end) - (if (re-search-backward "|\\|;;" (point-at-bol) t) - (match-end 0) - (comint-bol nil) - (point)))) - (end (point)) - begins args) - (save-excursion - (goto-char begin) - (while (< (point) end) - (skip-chars-forward " \t\n") - (setq begins (cons (point) begins)) - (let ((skip t)) - (while skip - (skip-chars-forward "^ \t\n") - (if (eq (char-before) ?\\) - (skip-chars-forward " \t\n") - (setq skip nil)))) - (setq args (cons (buffer-substring-no-properties - (car begins) (point)) - args))) - (cons (reverse args) (reverse begins))))) - -(defun gf-complete () - (interactive) - ;; (setq this-command 'pcomplete) - (pcomplete)) - -(defun gf-default-completion-function () - (pcomplete-here (pcomplete-entries))) - -(defun gf-complete-command () - (pcomplete-here (gf-complete-commands))) - -(defun gf-complete-commands () gf-short-command-names) - -;; (defun gf-complete-flagify (flags) -;; (mapcar (lambda (s) (concat s "=")) flags)) - -(defvar gf-short-command-names - '("i" "rl" "e" "sf" "s" "pg" "pm" "vg" "po" "pl" "pi" - "eh" "ph" "l" "p" "tt" "cc" "so" "t" "gr" "gt" "ma" - "ps" "pt" "st" "wt" "vt" "es" "ts" "tq" "tl" "mq" - "ml" "rf" "wf" "af" "tg" "cl" "sa" "h" "q" "!")) - -(defvar gf-long-command-names - '("import" "remove_language" "empty" "set_flags" "strip" - "print_grammar" "print_multigrammar" "visualize_graph" - "print_options" "print_languages" "print_info" - "execute_history" "print_history" "linearize" "parse" "test_tokenizer" - "compute_concrete" "show_operations" "translate" "generate_random" - "generate_trees" "morphologically_analyse" "put_string" "put_tree" - "show_tree" "wrap_tree" "visualize_tree" "editing_session" - "translation_session" "translation_quiz" "translation_list" - "morphology_quiz" "morphology_list" "read_file" "write_file" - "append_file" "transform_grammar" "convert_latex" "speak_aloud" - "help" "quit" "system_command")) - -(defun gf-complete-options (options flags &optional flags-extra-table - extra-completions) - (let ((-options (mapcar (lambda (s) (concat "-" s)) options)) - (-flags= (mapcar (lambda (s) (concat "-" s "=")) flags))) - ;; do-while - (while (progn - (cond - ((pcomplete-match "\\`-\\(\\w+\\)=\\(.*\\)" 0) - (pcomplete-here - (let ((opt (cdr (assoc (car (member - (pcomplete-match-string 1 0) - flags)) - (append flags-extra-table - gf-flags-table))))) - (if (functionp opt) - (funcall opt) - opt)) - (pcomplete-match-string 2 0))) - (t (pcomplete-here - (append - (if (functionp extra-completions) - (funcall extra-completions) - extra-completions) - -options -flags=)))) - (pcomplete-match "\\`-" 1))))) - -(defun gf-collect-results (process command function) - (let ((output-buffer " *gf-tmp*") - results) - (save-excursion - (set-buffer (get-buffer-create output-buffer)) - (erase-buffer) - (comint-redirect-send-command-to-process - command output-buffer process nil t) - ;; Wait for the process to complete - (set-buffer (process-buffer process)) - (while (null comint-redirect-completed) - (accept-process-output nil 1)) - ;; Collect the output - (set-buffer output-buffer) - (goto-char (point-min)) - ;; Skip past the command, if it was echoed - (and (looking-at command) (forward-line)) - (funcall function)))) - -;; Command Completion --------------------------------------------- - -;; i, import: i File -;; Reads a grammar from File and compiles it into a GF runtime grammar. -;; Files "include"d in File are read recursively, nubbing repetitions. -;; If a grammar with the same language name is already in the state, -;; it is overwritten - but only if compilation succeeds. -;; The grammar parser depends on the file name suffix: -;; .gf normal GF source -;; .gfc canonical GF -;; .gfr precompiled GF resource -;; .gfcm multilingual canonical GF -;; .gfe example-based grammar files (only with the -ex option) -;; .ebnf Extended BNF format -;; .cf Context-free (BNF) format -;; options: -;; -old old: parse in GF<2.0 format (not necessary) -;; -v verbose: give lots of messages -;; -s silent: don't give error messages -;; -src source: ignore precompiled gfc and gfr files -;; -retain retain operations: read resource modules (needed in comm cc) -;; -nocf don't build context-free grammar (thus no parser) -;; -nocheckcirc don't eliminate circular rules from CF -;; -cflexer build an optimized parser with separate lexer trie -;; -noemit do not emit code (default with old grammar format) -;; -o do emit code (default with new grammar format) -;; -ex preprocess .gfe files if needed -;; flags: -;; -abs set the name used for abstract syntax (with -old option) -;; -cnc set the name used for concrete syntax (with -old option) -;; -res set the name used for resource (with -old option) -;; -path use the (colon-separated) search path to find modules -;; -optimize select an optimization to override file-defined flags -;; -conversion select parsing method (values strict|nondet) -;; examples: -;; i English.gf -- ordinary import of Concrete -;; i -retain german/ParadigmsGer.gf -- import of Resource to test - -(defun pcomplete/inf-gf-mode/i () - (gf-complete-options - '("old" "v" "s" "src" "retain" "nocf" "nocheckcirc" - "cflexer" "noemit" "o" "ex") - '("abs" "cnc" "res" "path" "optimize" "conversion") - '(("conversion" . ("strict" "nondet"))) - (lambda () - (pcomplete-dirs-or-entries - (regexp-opt - '(".gf" ".gfc" ".gfr" ".gfcm" ".gfe" ".ebnf" ".cf" - ".trc")))))) - -;; * rl, remove_language: rl Language -;; Takes away the language from the state. -(defun pcomplete/inf-gf-mode/rl () - (pcomplete-here (gf-complete-lang))) - -;; e, empty: e -;; Takes away all languages and resets all global flags. -(defun pcomplete/inf-gf-mode/e ()) - -;; sf, set_flags: sf Flag* -;; The values of the Flags are set for Language. If no language -;; is specified, the flags are set globally. -;; examples: -;; sf -nocpu -- stop showing CPU time -;; sf -lang=Swe -- make Swe the default concrete -(defun pcomplete/inf-gf-mode/sf () - (message "Usage: sf Flag*") - (throw 'pcompleted nil)) - -;; s, strip: s -;; Prune the state by removing source and resource modules. -(defun pcomplete/inf-gf-mode/s ()) - -;; pg, print_grammar: pg -;; Prints the actual grammar (overridden by the -lang=X flag). -;; The -printer=X flag sets the format in which the grammar is -;; written. -;; N.B. since grammars are compiled when imported, this command -;; generally does not show the grammar in the same format as the -;; source. In particular, the -printer=latex is not supported. -;; Use the command tg -printer=latex File to print the source -;; grammar in LaTeX. -;; options: -;; -utf8 apply UTF8-encoding to the grammar -;; flags: -;; -printer -;; -lang -;; examples: -;; pg -printer=cf -- show the context-free skeleton -(defun pcomplete/inf-gf-mode/pg () - (gf-complete-options '("utf8") - '("printer" "lang"))) - ;; (while (progn - ;; (cond - ;; ((pcomplete-match "\\`-printer=\\(.*\\)" 0) - ;; (pcomplete-here gf-flag-printer-options - ;; (pcomplete-match-string 1 0))) - ;; ((pcomplete-match "\\`-\\w+=" 0) - ;; (pcomplete-here)) - ;; (t (pcomplete-here - ;; (append '("-utf8") - ;; '("-printer=" "-lang="))))) - ;; (pcomplete-match "\\`-" 1))) - -;; pm, print_multigrammar: pm -;; Prints the current multilingual grammar in .gfcm form. -;; (Automatically executes the strip command (s) before doing this.) -;; options: -;; -utf8 apply UTF8 encoding to the tokens in the grammar -;; -utf8id apply UTF8 encoding to the identifiers in the grammar -;; -graph print module dependency graph in 'dot' format -;; examples: -;; pm | wf Letter.gfcm -- print the grammar into the file Letter.gfcm -;; pm -printer=graph | wf D.dot -- then do 'dot -Tps D.dot > D.ps' -(defun pcomplete/inf-gf-mode/pm () - (gf-complete-options '("utf8" "utf8id" "graph") - '("printer" "lang") - '(("printer" . ("graph"))))) - -;; vg, visualize_graph: vg -;; Show the dependency graph of multilingual grammar via dot and gv. -(defun pcomplete/inf-gf-mode/vg ()) - -;; po, print_options: po -;; Print what modules there are in the state. Also prints those -;; flag values in the current state that differ from defaults. -(defun pcomplete/inf-gf-mode/po ()) - -;; pl, print_languages: pl -;; Prints the names of currently available languages. -(defun pcomplete/inf-gf-mode/pl ()) - -;; pi, print_info: pi Ident -;; Prints information on the identifier. -(defun pcomplete/inf-gf-mode/pi () - (message "Usage: pi Ident") - (throw 'pcompleted nil)) - -;; eh, execute_history: eh File -;; Executes commands in the file. -(defun pcomplete/inf-gf-mode/eh () - (pcomplete-here (pcomplete-entries))) - -;; ph, print_history; ph -;; Prints the commands issued during the GF session. -;; The result is readable by the eh command. -;; examples: -;; ph | wf foo.hist" -- save the history into a file -(defun pcomplete/inf-gf-mode/ph ()) - -;; -- linearization, parsing, translation, and computation - -;; l, linearize: l PattList? Tree -;; Shows all linearization forms of Tree by the actual grammar -;; (which is overridden by the -lang flag). -;; The pattern list has the form [P, ... ,Q] where P,...,Q follow GF -;; syntax for patterns. All those forms are generated that match with the -;; pattern list. Too short lists are filled with variables in the end. -;; Only the -table flag is available if a pattern list is specified. -;; HINT: see GF language specification for the syntax of Pattern and Term. -;; You can also copy and past parsing results. -;; options: -;; -table show parameters -;; -struct bracketed form -;; -record record, i.e. explicit GF concrete syntax term -;; -all show all forms and variants -;; -multi linearize to all languages (the other options don't work) -;; flags: -;; -lang linearize in this grammar -;; -number give this number of forms at most -;; -unlexer filter output through unlexer -;; examples: -;; l -lang=Swe -table -- show full inflection table in Swe -(defun pcomplete/inf-gf-mode/l () - (gf-complete-options '("table" "struct" "record" "all" "multi") - '("lang" "number" "unlexer")) - (message "Usage: l [-option*] PattList? Tree") - (throw 'pcompleted nil)) - -;; p, parse: p String -;; Shows all Trees returned for String by the actual -;; grammar (overridden by the -lang flag), in the category S (overridden -;; by the -cat flag). -;; options for batch input: -;; -lines parse each line of input separately, ignoring empty lines -;; -all as -lines, but also parse empty lines -;; options for selecting parsing method: -;; (default)parse using an overgenerating CFG -;; -cfg parse using a much less overgenerating CFG -;; -mcfg parse using an even less overgenerating MCFG -;; Note: the first time parsing with -cfg or -mcfg might take a long time -;; options that only work for the default parsing method: -;; -n non-strict: tolerates morphological errors -;; -ign ignore unknown words when parsing -;; -raw return context-free terms in raw form -;; -v verbose: give more information if parsing fails -;; flags: -;; -cat parse in this category -;; -lang parse in this grammar -;; -lexer filter input through this lexer -;; -parser use this parsing strategy -;; -number return this many results at most -;; examples: -;; p -cat=S -mcfg "jag är gammal" -- parse an S with the MCFG -;; rf examples.txt | p -lines -- parse each non-empty line of the file -(defun pcomplete/inf-gf-mode/p () - (gf-complete-options - '("lines" "all" "cfg" "mcfg" "n" "ign" "raw" "v") - '("cat" "lang" "lexer" "parser" "number")) - (message "Usage: p [-option*] String") - (throw 'pcompleted nil)) - -;; tt, test_tokenizer: tt String -;; Show the token list sent to the parser when String is parsed. -;; HINT: can be useful when debugging the parser. -;; flags: -;; -lexer use this lexer -;; examples: -;; tt -lexer=codelit "2*(x + 3)" -- a favourite lexer for program code -(defun pcomplete/inf-gf-mode/tt () - (gf-complete-options '() '("lexer")) - (message "Usage: tt [-lexer] String") - (throw 'pcompleted nil)) - -;; cc, compute_concrete: cc Term -;; Compute a term by concrete syntax definitions. Uses the topmost -;; resource module (the last in listing by command po) to resolve -;; constant names. -;; N.B. You need the flag -retain when importing the grammar, if you want -;; the oper definitions to be retained after compilation; otherwise this -;; command does not expand oper constants. -;; N.B.' The resulting Term is not a term in the sense of abstract syntax, -;; and hence not a valid input to a Tree-demanding command. -;; flags: -;; -res use another module than the topmost one -;; examples: -;; cc -res=ParadigmsFin (nLukko "hyppy") -- inflect "hyppy" with nLukko -(defun pcomplete/inf-gf-mode/cc () - (gf-complete-options '() '("res")) - (message "Usage: cc [-res] Term") - (throw 'pcompleted nil)) - -;; so, show_operations: so Type -;; Show oper operations with the given value type. Uses the topmost -;; resource module to resolve constant names. -;; N.B. You need the flag -retain when importing the grammar, if you want -;; the oper definitions to be retained after compilation; otherwise this -;; command does not find any oper constants. -;; N.B.' The value type may not be defined in a supermodule of the -;; topmost resource. In that case, use appropriate qualified name. -;; flags: -;; -res use another module than the topmost one -;; examples: -;; so -res=ParadigmsFin ResourceFin.N -- show N-paradigms in ParadigmsFin -(defun pcomplete/inf-gf-mode/so () - (gf-complete-options '() '("res")) - (message "Usage: so [-res] Type") - (throw 'pcompleted nil)) - -;; t, translate: t Lang Lang String -;; Parses String in Lang1 and linearizes the resulting Trees in Lang2. -;; flags: -;; -cat -;; -lexer -;; -parser -;; examples: -;; t Eng Swe -cat=S "every number is even or odd" -(defun pcomplete/inf-gf-mode/t () - (gf-complete-options '() '("cat" "lexer" "parser")) - (message "Usage: t [-options] Lang Lang String") - (throw 'pcompleted nil)) - -;; gr, generate_random: gr Tree? -;; Generates a random Tree of a given category. If a Tree -;; argument is given, the command completes the Tree with values to -;; the metavariables in the tree. -;; flags: -;; -cat generate in this category -;; -lang use the abstract syntax of this grammar -;; -number generate this number of trees (not impl. with Tree argument) -;; -depth use this number of search steps at most -;; examples: -;; gr -cat=Query -- generate in category Query -;; gr (PredVP ? (NegVG ?)) -- generate a random tree of this form -;; gr -cat=S -tr | l -- gererate and linearize -(defun pcomplete/inf-gf-mode/gr () - (ding) - (gf-complete-options '() '("cat" "lang" "number" "depth")) - (message "Usage: gr [-options] Tree?") - (throw 'pcompleted nil)) - -;; gt, generate_trees: gt Tree? -;; Generates all trees up to a given depth. If the depth is large, -;; a small -alts is recommended. If a Tree argument is given, the -;; command completes the Tree with values to the metavariables in -;; the tree. -;; options: -;; -metas also return trees that include metavariables -;; flags: -;; -depth generate to this depth (default 3) -;; -alts take this number of alternatives at each branch (default unlimited) -;; -cat generate in this category -;; -lang use the abstract syntax of this grammar -;; -number generate (at most) this number of trees -;; examples: -;; gt -depth=10 -cat=NP -- generate all NP's to depth 10 -;; gt (PredVP ? (NegVG ?)) -- generate all trees of this form -;; gt -cat=S -tr | l -- gererate and linearize -(defun pcomplete/inf-gf-mode/gt () - (gf-complete-options '("metas") - '("depth" "alts" "cat" "lang" "number"))) - -;; ma, morphologically_analyse: ma String -;; Runs morphological analysis on each word in String and displays -;; the results line by line. -;; options: -;; -short show analyses in bracketed words, instead of separate lines -;; flags: -;; -lang -;; examples: -;; wf Bible.txt | ma -short | wf Bible.tagged -- analyse the Bible -(defun pcomplete/inf-gf-mode/ma () - (gf-complete-options '("short") '("lang"))) - -;; -- elementary generation of Strings and Trees - -;; ps, put_string: ps String -;; Returns its argument String, like Unix echo. -;; HINT. The strength of ps comes from the possibility to receive the -;; argument from a pipeline, and altering it by the -filter flag. -;; flags: -;; -filter filter the result through this string processor -;; -length cut the string after this number of characters -;; examples: -;; gr -cat=Letter | l | ps -filter=text -- random letter as text -(defun pcomplete/inf-gf-mode/ps () - (gf-complete-options '() '("filter" "length"))) - -;; pt, put_tree: pt Tree -;; Returns its argument Tree, like a specialized Unix echo. -;; HINT. The strength of pt comes from the possibility to receive -;; the argument from a pipeline, and altering it by the -transform flag. -;; flags: -;; -transform transform the result by this term processor -;; -number generate this number of terms at most -;; examples: -;; p "zero is even" | pt -transform=solve -- solve ?'s in parse result -(defun pcomplete/inf-gf-mode/pt () - (gf-complete-options '() '("transform" "number"))) - -;; * st, show_tree: st Tree -;; Prints the tree as a string. Unlike pt, this command cannot be -;; used in a pipe to produce a tree, since its output is a string. -;; flags: -;; -printer show the tree in a special format (-printer=xml supported) -(defun pcomplete/inf-gf-mode/st ()) - -;; wt, wrap_tree: wt Fun -;; Wraps the tree as the sole argument of Fun. -;; flags: -;; -c compute the resulting new tree to normal form -(defun pcomplete/inf-gf-mode/wt () - (gf-complete-options '("c") '())) - -;; vt, visualize_tree: vt Tree -;; Shows the abstract syntax tree via dot and gv (via temporary files -;; grphtmp.dot, grphtmp.ps). -;; flags: -;; -c show categories only (no functions) -;; -f show functions only (no categories) -;; -g show as graph (sharing uses of the same function) -;; -o just generate the .dot file -;; examples: -;; p "hello world" | vt -o | wf my.dot ;; ! open -a GraphViz my.dot -;; -- This writes the parse tree into my.dot and opens the .dot file -;; -- with another application without generating .ps. -(defun pcomplete/inf-gf-mode/vt () - (gf-complete-options '("c" "f" "g" "o") '())) - -;; -- subshells - -;; es, editing_session: es -;; Opens an interactive editing session. -;; N.B. Exit from a Fudget session is to the Unix shell, not to GF. -;; options: -;; -f Fudget GUI (necessary for Unicode; only available in X Window System) - -;; ts, translation_session: ts -;; Translates input lines from any of the actual languages to all other ones. -;; To exit, type a full stop (.) alone on a line. -;; N.B. Exit from a Fudget session is to the Unix shell, not to GF. -;; HINT: Set -parser and -lexer locally in each grammar. -;; options: -;; -f Fudget GUI (necessary for Unicode; only available in X Windows) -;; -lang prepend translation results with language names -;; flags: -;; -cat the parser category -;; examples: -;; ts -cat=Numeral -lang -- translate numerals, show language names -(defun pcomplete/inf-gf-mode/ts () - (gf-complete-options '("f" "lang") '("cat"))) - -;; tq, translation_quiz: tq Lang Lang -;; Random-generates translation exercises from Lang1 to Lang2, -;; keeping score of success. -;; To interrupt, type a full stop (.) alone on a line. -;; HINT: Set -parser and -lexer locally in each grammar. -;; flags: -;; -cat -;; examples: -;; tq -cat=NP TestResourceEng TestResourceSwe -- quiz for NPs -(defun pcomplete/inf-gf-mode/tq () - (pcomplete-here (gf-complete-lang)) - (pcomplete-here (gf-complete-lang))) - -;; tl, translation_list: tl Lang Lang -;; Random-generates a list of ten translation exercises from Lang1 -;; to Lang2. The number can be changed by a flag. -;; HINT: use wf to save the exercises in a file. -;; flags: -;; -cat -;; -number -;; examples: -;; tl -cat=NP TestResourceEng TestResourceSwe -- quiz list for NPs -(defun pcomplete/inf-gf-mode/tl () - (pcomplete-here (gf-complete-lang)) - (pcomplete-here (gf-complete-lang))) - -;; mq, morphology_quiz: mq -;; Random-generates morphological exercises, -;; keeping score of success. -;; To interrupt, type a full stop (.) alone on a line. -;; HINT: use printname judgements in your grammar to -;; produce nice expressions for desired forms. -;; flags: -;; -cat -;; -lang -;; examples: -;; mq -cat=N -lang=TestResourceSwe -- quiz for Swedish nouns - -;; ml, morphology_list: ml -;; Random-generates a list of ten morphological exercises, -;; keeping score of success. The number can be changed with a flag. -;; HINT: use wf to save the exercises in a file. -;; flags: -;; -cat -;; -lang -;; -number -;; examples: -;; ml -cat=N -lang=TestResourceSwe -- quiz list for Swedish nouns -(defun pcomplete/inf-gf-mode/ml () - (gf-complete-options '() '("cat" "lang" "number"))) - -;; -- IO related commands - -;; rf, read_file: rf File -;; Returns the contents of File as a String; error if File does not exist. -(defun pcomplete/inf-gf-mode/rf () - (pcomplete-here (pcomplete-entries))) - -;; wf, write_file: wf File String -;; Writes String into File; File is created if it does not exist. -;; N.B. the command overwrites File without a warning. - -;; af, append_file: af File -;; Writes String into the end of File; File is created if it does not exist. - -;; * tg, transform_grammar: tg File -;; Reads File, parses as a grammar, -;; but instead of compiling further, prints it. -;; The environment is not changed. When parsing the grammar, the same file -;; name suffixes are supported as in the i command. -;; HINT: use this command to print the grammar in -;; another format (the -printer flag); pipe it to wf to save this format. -;; flags: -;; -printer (only -printer=latex supported currently) - -;; * cl, convert_latex: cl File -;; Reads File, which is expected to be in LaTeX form. -;; Three environments are treated in special ways: -;; \begGF - \end{verbatim}, which contains GF judgements, -;; \begTGF - \end{verbatim}, which contains a GF expression (displayed) -;; \begInTGF - \end{verbatim}, which contains a GF expressions (inlined). -;; Moreover, certain macros should be included in the file; you can -;; get those macros by applying 'tg -printer=latex foo.gf' to any grammar -;; foo.gf. Notice that the same File can be imported as a GF grammar, -;; consisting of all the judgements in \begGF environments. -;; HINT: pipe with 'wf Foo.tex' to generate a new Latex file. - -;; sa, speak_aloud: sa String -;; Uses the Flite speech generator to produce speech for String. -;; Works for American English spelling. -;; examples: -;; h | sa -- listen to the list of commands -;; gr -cat=S | l | sa -- generate a random sentence and speak it aloud - -;; h, help: h Command? -;; Displays the paragraph concerning the command from this help file. -;; Without the argument, shows the first lines of all paragraphs. -;; options -;; -all show the whole help file -;; examples: -;; h print_grammar -- show all information on the pg command - -;; q, quit: q -;; Exits GF. -;; HINT: you can use 'ph | wf history' to save your session. - -;; !, system_command: ! String -;; Issues a system command. No value is returned to GF. -;; example: -;; ! ls -(defun pcomplete/inf-gf-mode/! () - ;;(pcomplete-here (eshell-complete-commands-list)) - ) - -;; -- Flags. The availability of flags is defined separately for each command. - -;; -cat, category in which parsing is performed. -;; The default is S. - -;; -depth, the search depth in e.g. random generation. -;; The default depends on application. - -;; -filter, operation performed on a string. The default is identity. -;; -filter=identity no change -;; -filter=erase erase the text -;; -filter=take100 show the first 100 characters -;; -filter=length show the length of the string -;; -filter=text format as text (punctuation, capitalization) -;; -filter=code format as code (spacing, indentation) -(defvar gf-flag-filter-options - '("identity" "erase" "take100" "length" "text" "code")) - -;; -lang, grammar used when executing a grammar-dependent command. -;; The default is the last-imported grammar. - -(defvar gf-lang-cache 'empty) -(defun gf-clear-lang-cache () (setq gf-lang-cache 'empty)) - -(defvar gf-flag-lang-options 'gf-complete-lang) -(defun gf-complete-lang () - (if (listp gf-lang-cache) - gf-lang-cache - (setq gf-lang-cache - (gf-collect-results - gf-process "pl" - (lambda () - ;; we're at point-min - (let (result) - (while (re-search-forward "\\S-+" (point-at-eol) t) - (push (match-string 0) result)) - result)))))) - -;; -language, voice used by Festival as its --language flag in the sa command. -;; The default is system-dependent. - -;; -length, the maximum number of characters shown of a string. -;; The default is unlimited. - -;; -lexer, tokenization transforming a string into lexical units for a parser. -;; The default is words. -;; -lexer=words tokens are separated by spaces or newlines -;; -lexer=literals like words, but GF integer and string literals recognized -;; -lexer=vars like words, but "x","x_...","$...$" as vars, "?..." as meta -;; -lexer=chars each character is a token -;; -lexer=code use Haskell's lex -;; -lexer=codevars like code, but treat unknown words as variables, ?? as meta -;; -lexer=text with conventions on punctuation and capital letters -;; -lexer=codelit like code, but treat unknown words as string literals -;; -lexer=textlit like text, but treat unknown words as string literals -;; -lexer=codeC use a C-like lexer -(defvar gf-flag-lexer-options - '("words" "literals" "vars" "chars" "code" "codevars" - "text" "codelit" "textlit" "codeC")) - -;; -number, the maximum number of generated items in a list. -;; The default is unlimited. - -;; -optimize, optimization on generated code. -;; The default is share for concrete, none for resource modules. -;; Each of the flags can have the suffix _subs, which performs -;; common subexpression elimination after the main optimization. -;; Thus, -optimize=all_subs is the most aggressive one. - -;; -optimize=share share common branches in tables -;; -optimize=parametrize first try parametrize then do share with the rest -;; -optimize=values represent tables as courses-of-values -;; -optimize=all first try parametrize then do values with the rest -;; -optimize=none no optimization -(defvar gf-flag-optimize-options - '("share" "parametrize" "values" "all" "none")) - -;; -parser, parsing strategy. The default is chart. If -cfg or -mcfg are selected, only bottomup and topdown are recognized. -;; -parser=chart bottom-up chart parsing -;; -parser=bottomup a more up to date bottom-up strategy -;; -parser=topdown top-down strategy -;; -parser=old an old bottom-up chart parser -(defvar gf-flag-parser-options - '("chart" "bottomup" "topdown" "old")) - -;; -printer, format in which the grammar is printed. The default is gfc. -;; -printer=gfc GFC grammar -;; -printer=gf GF grammar -;; -printer=old old GF grammar -;; -printer=cf context-free grammar, with profiles -;; -printer=bnf context-free grammar, without profiles -;; -printer=lbnf labelled context-free grammar for BNF Converter -;; -printer=plbnf grammar for BNF Converter, with precedence levels -;; *-printer=happy source file for Happy parser generator (use lbnf!);; -printer=srg speech recognition grammar -;; -printer=haskell abstract syntax in Haskell, with transl to/from GF -;; -printer=morpho full-form lexicon, long format -;; *-printer=latex LaTeX file (for the tg command) -;; -printer=fullform full-form lexicon, short format -;; *-printer=xml XML: DTD for the pg command, object for st -;; -printer=old old GF: file readable by GF 1.2 -;; -printer=stat show some statistics of generated GFC -;; -printer=gsl Nuance GSL speech recognition grammar -;; -printer=jsgf Java Speech Grammar Format -;; -printer=slf a finite automaton in the HTK SLF format -;; -printer=slf_graphviz the same automaton as in SLF, but in Graphviz format -;; -printer=fa_graphviz a finite automaton with labelled edges -;; -printer=regular a regular grammar in a simple BNF -;; -printer=unpar a gfc grammar with parameters eliminated -(defvar gf-flag-printer-options - '("gfc" "gf" "cf" "old" "srg" "gsl" "jsgf" "slf" "slf_graphviz" - "fa_graphviz" "regular" "plbnf" "lbnf" "bnf" "haskell" "morpho" - "fullform" "opts" "words" "printnames" "stat" "unpar" "subs" - "mcfg" "cfg" "pinfo" "abstract" "gfc-haskell" "mcfg-haskell" - "cfg-haskell" "gfc-prolog" "mcfg-prolog" "cfg-prolog" "abs-skvatt" - "cfg-skvatt" "simple" "mcfg-erasing" "mcfg-old" "cfg-old" - ;;"happy" "latex" "xml" - )) - - -;; -startcat, like -cat, but used in grammars (to avoid clash with keyword cat) - -;; -transform, transformation performed on a syntax tree. The default is identity. -;; -transform=identity no change -;; -transform=compute compute by using definitions in the grammar -;; -transform=typecheck return the term only if it is type-correct -;; -transform=solve solve metavariables as derived refinements -;; -transform=context solve metavariables by unique refinements as variables -;; -transform=delete replace the term by metavariable -(defvar gf-flag-transform-options - '("identity" "compute" "typecheck" "solve" "context" "delete")) - -;; -unlexer, untokenization transforming linearization output into a string. -;; The default is unwords. -;; -unlexer=unwords space-separated token list (like unwords) -;; -unlexer=text format as text: punctuation, capitals, paragraph

-;; -unlexer=code format as code (spacing, indentation) -;; -unlexer=textlit like text, but remove string literal quotes -;; -unlexer=codelit like code, but remove string literal quotes -;; -unlexer=concat remove all spaces -;; -unlexer=bind like identity, but bind at "&+" -(defvar gf-flag-unlexer-options - '("unwords" "text" "code" "textlit" "codelit" "concat" "bind")) - -;; -- *: Commands and options marked with * are not yet implemented. - -(defvar gf-flags-table - `(("filter" . ,gf-flag-filter-options) - ("lang" . ,gf-flag-lang-options) - ("lexer" . ,gf-flag-lexer-options) - ("optimize" . ,gf-flag-optimize-options) - ("parser" . ,gf-flag-parser-options) - ("printer" . ,gf-flag-printer-options) - ("transform" . ,gf-flag-transform-options) - ("unlexer" . ,gf-flag-unlexer-options))) - -;;; gf.el ends here +(error "install updated GF mode from https://github.com/GrammaticalFramework/gf-emacs-mode") diff --git a/src/tools/gftest/EqRel.hs b/src/tools/gftest/EqRel.hs deleted file mode 100644 index 823900ae0..000000000 --- a/src/tools/gftest/EqRel.hs +++ /dev/null @@ -1,32 +0,0 @@ -module EqRel where - -import qualified Data.Map as M -import Data.List ( sort ) - -data EqRel a = Top | Classes [[a]] deriving (Eq,Ord,Show) - -(/\) :: (Ord a) => EqRel a -> EqRel a -> EqRel a -Top /\ r = r -r /\ Top = r -Classes xss /\ Classes yss = Classes $ sort $ map sort $ concat -- maybe throw away singleton lists? - [ M.elems tabXs - | xs <- xss - , let tabXs = M.fromListWith (++) - [ (tabYs M.! x, [x]) - | x <- xs ] - ] - - where - tabYs = M.fromList [ (y,representative) - | ys <- yss - , let representative = head ys - , y <- ys ] - -basic :: (Ord a) => [a] -> EqRel Int -basic xs = Classes $ sort $ map sort $ M.elems $ M.fromListWith (++) - [ (x,[i]) | (x,i) <- zip xs [0..] ] - -rep :: EqRel Int -> Int -> Int -rep Top j = 0 -rep (Classes xss) j = head [ head xs | xs <- xss, j `elem` xs ] - diff --git a/src/tools/gftest/FMap.hs b/src/tools/gftest/FMap.hs deleted file mode 100644 index f3a511706..000000000 --- a/src/tools/gftest/FMap.hs +++ /dev/null @@ -1,62 +0,0 @@ -module FMap where - --------------------------------------------------------------------------------- --- implementation - -data FMap a b = Ask a (FMap a b) (FMap a b) | Nil | Answer b - deriving ( Eq, Ord, Show ) - -toList :: FMap a b -> [([a],b)] -toList t = go [([],t)] - where - go [] = [] - go ((xs,Ask x yes no):xts) = go ((x:xs,yes):(xs,no):xts) - go ((_ ,Nil) :xts) = go xts - go ((xs,Answer z) :xts) = (reverse xs,z) : go xts - -isNil :: FMap a b -> Bool -isNil = null . toList - -nil :: FMap a b -nil = Nil - -unit :: [a] -> b -> FMap a b -unit [] y = Answer y -unit (x:xs) y = Ask x (unit xs y) Nil - -covers :: Ord a => FMap a b -> [a] -> Bool -Nil `covers` _ = False -_ `covers` [] = True -Answer _ `covers` _ = False -Ask x yes no `covers` zs@(y:ys) = - case x `compare` y of - LT -> (yes `covers` zs) || (no `covers` zs) - EQ -> yes `covers` ys - GT -> False - -ask :: a -> FMap a b -> FMap a b -> FMap a b -ask x Nil Nil = Nil -ask x s t = Ask x s t - -del :: Ord a => [a] -> FMap a b -> FMap a b -del _ Nil = Nil -del _ (Answer _) = Nil -del [] (Ask x yes no) = ask x yes (del [] no) -del (x:xs) t@(Ask y yes no) = - case x `compare` y of - LT -> del xs t - EQ -> ask y (del xs yes) (del xs no) - GT -> ask y yes (del (x:xs) no) - -add :: Ord a => [a] -> b -> FMap a b -> FMap a b -add [] y Nil = Answer y -add (x:xs) y Nil = Ask x (add xs y Nil) Nil -add xs@(_:_) y (Answer _) = add xs y Nil -add (x:xs) y t@(Ask z yes no) = - case x `compare` z of - LT -> Ask x (add xs y Nil) (del xs t) - EQ -> Ask x (add xs y yes) (del xs no) - GT -> Ask z yes (add (x:xs) y no) - --------------------------------------------------------------------------------- - diff --git a/src/tools/gftest/Grammar.hs b/src/tools/gftest/Grammar.hs deleted file mode 100644 index 0724987b2..000000000 --- a/src/tools/gftest/Grammar.hs +++ /dev/null @@ -1,1121 +0,0 @@ -{-# LANGUAGE TypeSynonymInstances, FlexibleInstances #-} - -module Grammar - ( Grammar(..), readGrammar - , Tree, top, Symbol(..), showTree - , Cat, ConcrCat(..) - , Lang, Name - - -- Categories, coercions - , ccats, ccatOf, arity - , coerces, uncoerce - , uncoerceAbsCat, mkCC - - -- Testing and comparison - , testTree, testFun - , compareTree, Comparison(..) - , treesUsingFun - - -- Contexts - , contextsFor, dummyHole - - -- FEAT - , featIth, featCard - - -- Fields - , forgets, reachableFieldsFromTop - , emptyFields, equalFields, fieldNames - - -- misc - , showConcrFun, subTree, flatten - , diffCats, hasConcrString -) where - -import Data.Either ( lefts ) -import Data.List -import qualified Data.Map as M -import Data.Maybe -import Data.Char -import qualified Data.Set as S -import qualified Mu -import qualified FMap as F -import qualified Data.Tree as T -import EqRel - -import GHC.Exts ( the ) -import Debug.Trace - -import qualified PGF2 -import qualified PGF2.Internal as I - --------------------------------------------------------------------------------- --- grammar types - --- name - -type Name = String - --- concrete category - -type Cat = PGF2.Cat -- i.e. String - -data ConcrCat = CC (Maybe Cat) I.FId -- i.e. Int - deriving ( Eq ) - -instance Show ConcrCat where - show (CC (Just cat) fid) = cat ++ "_" ++ show fid - show (CC Nothing fid) = "_" ++ show fid - -instance Ord ConcrCat where - (CC _ fid1) `compare` (CC _ fid2) = fid1 `compare` fid2 - -ccatOf :: Tree -> ConcrCat -ccatOf (App tp _) = snd (ctyp tp) - --- tree - -data RoseTree a - = App { top :: a, args :: [RoseTree a] } - deriving ( Eq, Ord ) - --- from http://hackage.haskell.org/package/containers-0.5.11.0/docs/src/Data.Tree.html#foldTree -foldTree :: (a -> [b] -> b) -> RoseTree a -> b -foldTree f = go where - go (App x ts) = f x (map go ts) - -flatten :: RoseTree a -> [a] -flatten (App tp as) = tp : concatMap flatten as - -type Tree = RoseTree Symbol -type AmbTree = RoseTree [Symbol] -- used as an intermediate category for parsing - -instance Show Tree where - show = showTree - -showTree :: Tree -> String -showTree (App a []) = show a -showTree (App f xs) = unwords (show f : map showTreeArg xs) - where showTreeArg (App a []) = show a - showTreeArg t = "(" ++ showTree t ++ ")" - -subTree :: Symbol -> Tree -> Maybe Tree -subTree symb t@(App tp tr) - | symb==tp = Just t - | otherwise = listToMaybe $ mapMaybe (subTree symb) tr - --- symbol - -type SeqId = Int - -data Symbol - = Symbol - { name :: Name - , seqs :: [SeqId] - , typ :: ([Cat], Cat) - , ctyp :: ([ConcrCat],ConcrCat) - } - deriving ( Eq, Ord ) - -instance Show Symbol where - show = name - -arity :: Symbol -> Int -arity = length . fst . ctyp - -hole :: ConcrCat -> Symbol -hole c = Symbol (show c) [] ([], "") ([],c) - -showConcrFun :: Grammar -> Symbol -> String -showConcrFun gr detCN = show detCN ++ " : " ++ args ++ show np_209 - where - (dets_cns,np_209) = ctyp detCN - args = concatMap (\x -> show x ++ " → ") dets_cns - --- grammar - -type Lang = String - -data Grammar - = Grammar - { - concrLang :: Lang - , parse :: String -> [Tree] - , readTree :: String -> Tree - , linearize :: Tree -> String - , tabularLin :: Tree -> [(String,String)] - , concrCats :: [(PGF2.Cat,I.FId,I.FId,[String])] - , coercions :: [(ConcrCat,ConcrCat)] - , contextsTab :: M.Map ConcrCat (M.Map ConcrCat [Tree -> Tree]) - , startCat :: Cat - , symbols :: [Symbol] - , lookupSymbol :: String -> [Symbol] - , functionsByCat :: Cat -> [Symbol] - , concrSeqs :: SeqId -> [Either String (Int,Int)] - , feat :: FEAT - , nonEmptyCats :: S.Set ConcrCat - , allCats :: [ConcrCat] - } - -fieldNames :: Grammar -> Cat -> [String] -fieldNames gr c = map fst . tabularLin gr $ t - where - t:_ = [ t - | f <- functionsByCat gr c - , let (_,c') = ctyp f - , c' `S.member` nonEmptyCats gr - , t <- featAll gr c' - ] - - --------------------------------------------------------------------------------- --- grammar - -readGrammar :: Lang -> FilePath -> IO Grammar -readGrammar lang file = - do pgf <- PGF2.readPGF file - return (toGrammar pgf lang) - -toGrammar :: PGF2.PGF -> Lang -> Grammar -toGrammar pgf langName = - let gr = - Grammar - { concrLang = lname - - , parse = \s -> - case PGF2.parse lang (PGF2.startCat pgf) s of - PGF2.ParseOk es_fs -> map (mkTree gr.fst) es_fs - PGF2.ParseFailed i s -> error s - PGF2.ParseIncomplete -> error "Incomplete parse" - - , readTree = \s -> - case PGF2.readExpr s of - Just t -> mkTree gr t - Nothing -> error "readTree: no parse" - - , linearize = \t -> - PGF2.linearize lang (mkExpr t) - - , tabularLin = \t -> - PGF2.tabularLinearize lang (mkExpr t) - - , startCat = - mkCat (PGF2.startCat pgf) - - , concrCats = - I.concrCategories lang - - , symbols = - [ Symbol { - name = nm, - seqs = sqs, - ctyp = (argsCC, goalCC), - typ = (map (uncoerceAbsCat gr) argsCC, goalcat) - } - | (goalcat,bg,end,_) <- I.concrCategories lang - , goalfid <- [bg..end] - , I.PApply funId pargs <- I.concrProductions lang goalfid - , let goalCC = CC (Just goalcat) goalfid - , let argsCC = [ mkCC argfid | I.PArg _ argfid <- pargs ] - , let (nm,sqs) = I.concrFunction lang funId ] - - , lookupSymbol = lookupAll (symb2table `map` symbols gr) - - , functionsByCat = \c -> - [ symb - | symb <- symbols gr - , snd (typ symb) == c - , snd (ctyp symb) `elem` nonEmptyCats gr ] - - , coercions = - [ ( mkCC cfid, CC Nothing afid ) - | afid <- [0..I.concrTotalCats lang] - , I.PCoerce cfid <- I.concrProductions lang afid ] - - , contextsTab = - M.fromList - [ (top, M.fromList (contexts gr top)) - | top <- allCats gr ] - - , concrSeqs = - map cseq2Either . I.concrSequence lang - - , feat = - mkFEAT gr - - , allCats = S.toList $ S.fromList $ - [ a | f <- symbols gr, let (args,goal) = ctyp f - , a <- goal:args - ] ++ - [ c | (cat,coe) <- coercions gr - , c <- [coe,cat] - ] - , nonEmptyCats = S.fromList - [ c - | let -- all functions, organized by result type - funs = M.fromListWith (++) $ - [ (cat,[Right f]) - | f <- symbols gr - , let (_,cat) = ctyp f - ] ++ - [ (coe,[Left cat]) - | (cat,coe) <- coercions gr - ] - - -- all categories, with their dependencies - defs = - [ if or [ arity f == 0 | Right f <- fs ] - then (c, [], \_ -> True) -- has a word - else (c, ys, h) -- no word - | c <- allCats gr - , let -- relevant functions for c - fs = fromMaybe [] (M.lookup c funs) - - -- categories we depend on - ys = S.toList $ S.fromList $ - [ cat | Right f <- fs, cat <- fst (ctyp f) ] ++ - [ cat | Left cat <- fs ] - - -- compute if we're empty, given the emptiness of others - h bs = or $ - [ and [ tab M.! a | a <- args ] - | Right f <- fs - , let (args,_) = ctyp f - ] ++ - [ tab M.! cat - | Left cat <- fs - ] - where - tab = M.fromList (ys `zip` bs) - ] - , (c,True) <- allCats gr `zip` Mu.mu False defs (allCats gr) - ] - - - - } - in gr - where - -- language - (lang,lname) = case M.lookup langName (PGF2.languages pgf) of - Just la -> (la,langName) - Nothing -> let (defName,defGr) = head $ M.assocs $ PGF2.languages pgf - msg = "no grammar found with name " ++ langName ++ - ", using " ++ defName - in trace msg (defGr,defName) - - -- categories and expressions - mkCat tp = cat where (_, cat, _) = PGF2.unType tp - - mkExpr (App n []) | not (null s) && all isDigit s = - PGF2.mkInt (read s) - where - s = show n - - mkExpr (App f xs) = - PGF2.mkApp (name f) [ mkExpr x | x <- xs ] - - mkCC fid = CC ccat fid - where ccat = case [ cat | (cat,bg,end,_) <- I.concrCategories lang - , fid `elem` [bg..end] ] of - [] -> Nothing -- means it's coercion - xs -> Just $ the xs - - -- misc - symb2table s = (s, name s) - - cseq2Either (I.SymKS tok) = Left tok - cseq2Either (I.SymCat x y) = Right (x,y) - cseq2Either x = Left (show x) - - -mkCC gr fid = CC ccat fid - where ccat = case [ cat | (cat,bg,end,_) <- concrCats gr - , fid `elem` [bg..end] ] of - [] -> Nothing -- means it's coercion - xs -> Just $ the xs - --- parsing and reading trees -mkTree :: Grammar -> PGF2.Expr -> Tree -mkTree gr = disambTree . ambTree - - where - ambTree t = -- :: PGF2.Expr -> AmbTree - case PGF2.unApp t of - Just (f,xs) -> App (lookupSymbol gr f) [ ambTree x | x <- xs ] - Nothing -> error (PGF2.showExpr [] t) - - disambTree at = -- :: AmbTree -> Tree - case foldTree reduce at of - App [x] ts -> App x [ disambTree t | t <- ts ] - App _ _ts -> error "mkTree: invalid tree" - - reduce fs as = -- :: [Symbol] -> [AmbTree] -> AmbTree - let red = [ symbol | symbol <- fs - , let argTypes = - uncoerce gr `map` fst (ctyp symbol) - , let goalTypes = - uncoerce gr `map` [ snd (ctyp s) | App [s] _ <- as ] - -- there should be only one symbol in (still ambiguous) fs - -- whose argument type matches its (already unambiguous) subtrees - , and [ intersect a r /= [] - | (a,r) <- zip argTypes goalTypes ] ] - in case red of - [x] -> App [x] as - _ -> App fs as - --- categories and coercions -ccats :: Grammar -> Cat -> [ConcrCat] -ccats gr utt = [ cc - | cc@(CC (Just cat) _) <- S.toList (nonEmptyCats gr) - , cat == utt ] - -uncoerceAbsCat :: Grammar -> ConcrCat -> Cat -uncoerceAbsCat gr c = case c of - CC (Just cat) _ -> cat - CC Nothing _ -> the [ uncoerceAbsCat gr x | x <- uncoerce gr c ] - -uncoerce :: Grammar -> ConcrCat -> [ConcrCat] -uncoerce gr c = case c of - CC Nothing _ -> lookupAll (coercions gr) c - _ -> [c] - -coerces :: Grammar -> ConcrCat -> ConcrCat -> Bool -coerces gr coe cat = (cat,coe) `elem` coercions gr - -lookupAll :: (Eq a) => [(b,a)] -> a -> [b] -lookupAll kvs key = [ v | (v,k) <- kvs, k==key ] - -singleton [x] = True -singleton xs = False - --------------------------------------------------------------------------------- --- compute categories reachable from S - -reachableCatsFromTop :: Grammar -> ConcrCat -> [ConcrCat] -reachableCatsFromTop gr top = [ c | (c,True) <- cs `zip` rs ] - where - rs = Mu.mu False defs cs - cs = S.toList (nonEmptyCats gr) - - defs = - [ if c == top - then (c, [], \_ -> True) - else (c, ys, or) - | c <- cs - , let ys = S.toList $ S.fromList $ - [ b - | f <- symbols gr - , let (as,b) = ctyp f - , all (`S.member` nonEmptyCats gr) as - , c `elem` as - ] ++ - [ b - | (a,b) <- coercions gr - , a == c - , b `S.member` nonEmptyCats gr - ] - ] - -reachableFieldsFromTop :: Grammar -> ConcrCat -> [(ConcrCat,S.Set Int)] -reachableFieldsFromTop gr top = cs `zip` rs - where - rs = Mu.mu S.empty defs cs - cs = S.toList (nonEmptyCats gr) - - defs = - [ if c == top - then (c, [], \_ -> S.fromList [0]) -- this assumes the top only has one field - else (c, ys, h) - | c <- cs - , let fs = [ Right (f,k) - | f <- symbols gr - , let (as,_) = ctyp f - , all (`S.member` nonEmptyCats gr) as - , (a,k) <- as `zip` [0..] - , c == a - ] ++ - [ Left b - | (a,b) <- coercions gr - , a == c - , b `S.member` nonEmptyCats gr - ] - - ys = S.toList $ S.fromList - [ case f of - Right (f,_) -> snd (ctyp f) - Left b -> b - | f <- fs - ] - - h rs = S.unions - [ case f of - Right (f,k) -> apply (f,k) (args M.! snd (ctyp f)) - Left b -> args M.! b - | f <- fs - ] - where - args = M.fromList (ys `zip` rs) - ] - - apply (f,k) r = - S.fromList - [ j - | (sq,i) <- seqs f `zip` [0..] - , i `S.member` r - , Right (k',j) <- concrSeqs gr sq - , k' == k - ] - --------------------------------------------------------------------------------- --- analyzing contexts - -equalFields :: Grammar -> [(ConcrCat,EqRel Int)] -equalFields gr = cs `zip` eqrels - where - eqrels = Mu.mu Top defs cs - cs = S.toList (nonEmptyCats gr) - - defs = - [ (c, depcats, h) - | c <- cs - -- fs = everything that has c as a goal category - -- there's two possibilities: - , let fs = -- 1) c is not a coercion: functions can have c as a goal category - [ Right f - | f <- symbols gr - , all (`S.member` nonEmptyCats gr) (fst (ctyp f)) - , c == snd (ctyp f) - ] ++ - -- 2) c is a coercion: here's a list of (nonempty) categories c uncoerces into - [ Left cat - | (cat,coe) <- coercions gr - , coe == c - , cat `S.member` nonEmptyCats gr - ] - - -- all the categories c depends on - depcats = S.toList $ S.fromList $ concat - [ case f of - Right f -> fst (ctyp f) -- 1) if c is not a coercion: - -- all arg cats of the functions with c as goal cat - Left cat -> [cat] -- 2) if c is a coercion: just the cats that it uncoerces into - | f <- fs - ] - - -- Function to give to mu: - -- computes the equivalence relation, given the eq.rels of its arguments - h rs = foldr (/\) Top $ [ apply f eqs - | Right f <- fs - , let eqs = map (args M.!) (fst $ ctyp f) - ] ++ - [ args M.! cat - | Left cat <- fs - ] - where - args = M.fromList (depcats `zip` rs) - ] - where - apply f eqs = - basic [ concatMap lin (concrSeqs gr sq) - | sq <- seqs f - ] - where - lin (Left str) = [ str | not (null str) ] - lin (Right (i,j)) = [ show i ++ "#" ++ show (rep (eqs !! i) j) ] - -contextsFor :: Grammar -> ConcrCat -> ConcrCat -> [Tree -> Tree] -contextsFor gr top hole = [] `fromMaybe` M.lookup hole (contextsTab gr M.! top) - -contexts :: Grammar -> ConcrCat -> [(ConcrCat,[Tree -> Tree])] -contexts gr top = - [ (c, map (path2context . reverse . snd) (F.toList paths)) - | (c, paths) <- cs `zip` pathss - ] - where - pathss = Mu.muDiff F.nil F.isNil dif uni defs cs - cs = S.toList (nonEmptyCats gr) - - -- all symbols with at least one argument, and only good arguments - goodSyms = - [ f - | f <- symbols gr - , arity f >= 1 - , snd (ctyp f) `S.member` nonEmptyCats gr - , all (`S.member` nonEmptyCats gr) (fst (ctyp f)) - ] - - -- definitions table for fixpoint iteration - fm1 `dif` fm2 = - [ d | d@(xs,_) <- F.toList fm1, not (fm2 `F.covers` xs) ] `ins` F.nil - - fm1 `uni` fm2 = - F.toList fm1 `ins` fm2 - - paths `ins` fm = - foldl collect fm - . map snd - . sort - $ [ (size p, p) | p <- paths ] - where - collect fm (str,p) - | fm `F.covers` str = fm - | otherwise = F.add str p fm - - size (_,p) = - sum [ if i == j then 1 else smallest gr t - | (f,i) <- p - , let (ts,_) = ctyp f - , (t,j) <- ts `zip` [0..] - ] - - defs = - [ if c == top - then (c, [], \_ -> F.unit [0] []) - else (c, ys, h) - | c <- cs - - -- everything that uses c in one of the two ways: - , let fs = -- 1) Functions that take c as the kth argument - [ Right (f,k) - | f <- goodSyms - , (t,k) <- fst (ctyp f) `zip` [0..] - , t == c - ] ++ - -- 2) coercions that uncoerce to c - [ Left coe - | (cat,coe) <- coercions gr - , cat == c - , coe `S.member` nonEmptyCats gr - ] - - -- goal categories for c - ys = S.toList $ S.fromList $ - [ case f of - Right (f,_) -> snd (ctyp f) -- 1) goal category of the function that uses c - Left coe -> coe -- 2) (category of the) coercion that uncoerces to c - | f <- fs - ] - - -- function to give to Mu - h ps = ([ (apply (f,k) str, (f,k):fis) - | Right (f,k) <- fs - , (str,fis) <- args M.! snd (ctyp f) - ] ++ - [ q - | Left a <- fs - , q <- args M.! a - ]) `ins` F.nil - where - args = M.fromList (ys `zip` map F.toList ps) - ] - where -- fields of B that make it to the top - apply :: (Symbol, Int) -> [Int] -> [Int] -- fields of A that make it to the top - apply (f,k) is = - S.toList $ S.fromList $ - [ y - | (sq,i) <- seqs f `zip` [0..] - , i `elem` is - , Right (x,y) <- concrSeqs gr sq - , x == k - ] - - path2context [] x = x - path2context ((f,i):fis) x = - App f - [ if j == i - then path2context fis x - else head (featAll gr t) - | (t,j) <- fst (ctyp f) `zip` [0..] - ] - -forgets :: Grammar -> ConcrCat -> [(ConcrCat,[Tree])] -forgets gr top = - filter (not . null . snd) - [ (c, [ path2context (reverse p) (head (featAll gr c)) - | (is,p) <- F.toList paths - , length is == fields c -- all indices forgotten - ] - ) - | (c, paths) <- cs `zip` pathss - ] - where - pathss = Mu.muDiff F.nil F.isNil dif uni defs cs - cs = S.toList (nonEmptyCats gr) - - -- all symbols with at least one argument, and only good arguments - goodSyms = - [ f - | f <- symbols gr - , arity f >= 1 - , snd (ctyp f) `S.member` nonEmptyCats gr - , all (`S.member` nonEmptyCats gr) (fst (ctyp f)) - ] - - fieldsTab = - M.fromList $ - [ (b, length (seqs f)) - | f <- symbols gr - , let (as,b) = ctyp f - ] - - fields a = - head $ - [ n - | c <- a : [ b | (b,a') <- coercions gr, a' == a ] - , Just n <- [M.lookup c fieldsTab] - ] ++ - error (show a ++ " has no function creating it") - - -- definitions table for fixpoint iteration - fm1 `dif` fm2 = - [ d | d@(xs,_) <- F.toList fm1, not (fm2 `F.covers` xs) ] `ins` F.nil - - fm1 `uni` fm2 = - F.toList fm1 `ins` fm2 - - paths `ins` fm = - foldl collect fm - . map snd - . sort - $ [ (size p, p) | p <- paths ] - where - collect fm (str,p) - | fm `F.covers` str = fm - | otherwise = F.add str p fm - - size (_,p) = - sum [ if i == j then 1 else smallest gr t - | (f,i) <- p - , let (ts,_) = ctyp f - , (t,j) <- ts `zip` [0..] - ] - - defs = - [ if c == top - then (c, [], \_ -> F.unit [] []) - else (c, ys, h) - | c <- cs - - -- everything that uses c in one of the two ways: - , let fs = -- 1) Functions that take c as the kth argument - [ Right (f,k) - | f <- goodSyms - , (t,k) <- fst (ctyp f) `zip` [0..] - , t == c - ] ++ - -- 2) coercions that uncoerce to c - [ Left coe - | (cat,coe) <- coercions gr - , cat == c - , coe `S.member` nonEmptyCats gr - ] - - -- goal categories for c - ys = S.toList $ S.fromList $ - [ case f of - Right (f,_) -> snd (ctyp f) - Left coe -> coe - | f <- fs - ] - - h ps = ([ (apply (f,k) str, (f,k):fis) - | Right (f,k) <- fs - , (str,fis) <- args M.! snd (ctyp f) - , length str < fields c - ] ++ - [ q - | Left a <- fs - , q@(str,_) <- args M.! a - , length str < fields c - ]) `ins` F.nil - where - args = M.fromList (ys `zip` map F.toList ps) - ] - where - apply :: (Symbol, Int) -> [Int] -> [Int] - apply (f,k) is = - [ y - | y <- [0..fields (fst (ctyp f) !! k)-1] - , y `S.notMember` used - ] - where - used = S.fromList $ - [ y - | (sq,i) <- seqs f `zip` [0..] - , i `notElem` is - , Right (x,y) <- concrSeqs gr sq - , x == k - ] - - path2context [] x = x - path2context ((f,i):fis) x = - App f - [ if j == i - then path2context fis x - else head (featAll gr t) - | (t,j) <- fst (ctyp f) `zip` [0..] - ] - ---traceLength s xs = trace (s ++ ":" ++ show (length xs)) xs - -emptyFields :: Grammar -> [(ConcrCat,S.Set Int)] -emptyFields gr = cs `zip` fields - where - cs = S.toList (nonEmptyCats gr) - fields = Mu.mu (S.fromList [0..99999]) defs cs - - defs = - [ (c, ys, h) - | c <- cs - , let fs = -- everything that has c as a goal category - [ Right f - | f <- symbols gr - , all (`S.member` nonEmptyCats gr) (fst (ctyp f)) - , c == snd (ctyp f) - ] ++ - -- 2) c is a coercion: here's a list of (nonempty) categories c uncoerces into - [ Left cat - | (cat,coe) <- coercions gr - , coe == c - , cat `S.member` nonEmptyCats gr - ] - - -- all the categories c depends on - ys = S.toList $ S.fromList $ concat - [ case f of - Right f -> fst (ctyp f) - Left cat -> [cat] - | f <- fs - ] - - -- Function to give to mu: - -- computes whether the field is empty, given the emptiness of its arguments. - -- a field in C is empty, if there's some function - -- f :: A -> B -> C - -- and it uses only empty fields from A and B. - -- we're only looking at a given C at a time, - - h :: [S.Set Int] -> S.Set Int - h vs = foldr1 S.intersection $ [ apply f emptyfields - | Right f <- fs - , let emptyfields = map (args M.!) (fst $ ctyp f) - ] ++ - [ args M.! cat - | Left cat <- fs - ] - where - args :: M.Map ConcrCat (S.Set Int) -- empty fields of each category - args = M.fromList (ys `zip` vs) - ] - where - --apply :: Symbol -- some f :: A -> B - -- -> [S.Set Int] -- for each argument type to f, which fields are empty - -- -> S.Set Int -- empty fields in B - apply f empties = - S.fromList - [ i - | (sq,i) <- seqs f `zip` [0..] - , let isEmpty s = case s of - Left str -> str == "" - Right (k,j) -> j `S.member` (empties !! k) - , all isEmpty (concrSeqs gr sq) - ] --------------------------------------------------------------------------------- --- FEAT-style generator magic - -type FEAT = [ConcrCat] -> Int -> (Integer, Integer -> [Tree]) - -smallest :: Grammar -> ConcrCat -> Int -smallest gr c = head [ n | n <- [0..], featCard gr c n > 0 ] - --- compute how many trees there are of a given size and type -featCard :: Grammar -> ConcrCat -> Int -> Integer -featCard gr c n = featCardVec gr [c] n - --- generate the i-th tree of a given size and type -featIth :: Grammar -> ConcrCat -> Int -> Integer -> Tree -featIth gr c n i = head (featIthVec gr [c] n i) - --- generate all trees (infinitely many) of a given type -featAll :: Grammar -> ConcrCat -> [Tree] -featAll gr c = [ featIth gr c n i | n <- [0..], i <- [0..featCard gr c n-1] ] - --- compute how many tree-vectors there are of a given size and type-vector -featCardVec :: Grammar -> [ConcrCat] -> Int -> Integer -featCardVec gr cs n = fst (feat gr cs n) - --- generate the i-th tree-vector of a given size and type-vector -featIthVec :: Grammar -> [ConcrCat] -> Int -> Integer -> [Tree] -featIthVec gr cs n i = snd (feat gr cs n) i - -mkFEAT :: Grammar -> FEAT -mkFEAT gr = catList - where - catList' :: FEAT - catList' [] 0 = (1, \0 -> []) - catList' [] _ = (0, error "indexing in an empty sequence") - - catList' [c] s = - parts $ - [ (n, \i -> [App f (h i)]) - | s > 0 - , f <- symbols gr - , let (xs,y) = ctyp f - , y == c - , let (n,h) = catList xs (s-1) - ] ++ - [ catList [x] s -- put (s-1) if it doesn't terminate - | s > 0 - , (x,y) <- coercions gr - , y == c - ] - - catList' (c:cs) s = - parts [ (nx*nxs, \i -> hx (i `mod` nx) ++ hxs (i `div` nx)) - | k <- [0..s] - , let (nx,hx) = catList [c] k - (nxs,hxs) = catList cs (s-k) - ] - - catList :: FEAT - catList = memoList (memoNat . catList') - where - -- all possible categories of the grammar - cats = S.toList $ S.fromList $ - [ x | f <- symbols gr - , let (xs,y) = ctyp f - , x <- y:xs ] ++ - [ z | (x,y) <- coercions gr - , z <- [x,y] ] - - memoList f = \cs -> case cs of - [] -> fNil - a:as -> fCons a as - where - fNil = f [] - fCons = (tab M.!) - tab = M.fromList [ (c, memoList (f . (c:))) | c <- cats ] - - memoNat f = (tab!!) - where - tab = [ f i | i <- [0..] ] - - parts [] = (0, error "indexing outside of a sequence") - parts ((n,h):nhs) = (n+n', \i -> if i < n then h i else h' (i-n)) - where - (n',h') = parts nhs - - --------------------------------------------------------------------------------- --- Functions used in Main - --- compare two grammars -diffCats :: Grammar -> Grammar -> [(Cat,[Int],[String],[String])] -diffCats gr1 gr2 = - [ (acat1,[difFid c1, difFid c2],labels1 \\ labels2,labels2 \\ labels1) - | c1@(acat1,_i1,_j2,labels1) <- concrCats gr1 - , c2@(acat2,_i2,_j2,labels2) <- concrCats gr2 - , difFid c1 /= difFid c2 -- different amount of concrete categories - || labels1 /= labels2 -- or the labels are different - , acat1==acat2 ] - - where - difFid (_,i,j,_) = 1 + (j-i) - - --- return a list of symbols that have a specified string, e.g. "it" in English --- grammar appears in functions CleftAdv, CleftNP, ImpersCl, DefArt, it_Pron -hasConcrString :: Grammar -> String -> [Symbol] -hasConcrString gr str = - [ symb - | symb <- symbols gr - , str `elem` concatMap (lefts . concrSeqs gr) (seqs symb) ] - --- nice printouts -type Context = String -type LinTree = ((Lang,Context),(Lang,String),(Lang,String),(Lang,String)) -data Comparison = Comparison { funTree :: String, linTree :: [LinTree] } -instance Show Comparison where - show c = unlines $ funTree c : map showLinTree (linTree c) - -dummyCCat = CC Nothing 99999999 -dummyHole = App (Symbol "∅" [] ([], "") ([], dummyCCat)) [] - -showLinTree :: LinTree -> String -showLinTree ((an,hl),(l1,t1),(l2,t2),(_l,[])) = unlines ["", an++hl, l1++t1, l2++t2] -showLinTree ((an,hl),(l1,t1),(l2,t2),(l3,t3)) = unlines ["", an++hl, l1++t1, l2++t2, l3++t3] - -compareTree :: Grammar -> Grammar -> [Grammar] -> Tree -> Comparison -compareTree gr oldgr transgr t = Comparison { - funTree = "* " ++ show t -, linTree = [ ( ("** ",hl), (langName gr,newLin), (langName oldgr, oldLin), transLin ) - | ctx <- ctxs - , let hl = show (ctx dummyHole) - , let transLin = case transgr of - [] -> ("","") - g:_ -> (langName g, linearize g (ctx t)) - , let newLin = linearize gr (ctx t) - , let oldLin = linearize oldgr (ctx t) - , newLin /= oldLin ] } - where - w = top t - c = snd (ctyp w) - cs = [ coe - | (cat,coe) <- coercions gr - , c == cat ] - ctxs = concat - [ contextsFor gr sc cat - | sc <- ccats gr (startCat gr) - , cat <- cs ] - langName gr = concrLang gr ++ "> " - -type Result = String - -testFun :: Bool -> Grammar -> [Grammar] -> Cat -> Name -> Result -testFun debug gr trans startcat funname = - let test = testTree debug gr trans - in unlines [ test t n cs - | (n,(t,cs)) <- zip [1..] testcase_ctxs ] - - where - testcase_ctxs = M.toList $ M.fromListWith (++) $ uniqueTCs++commonTCs - - uniqueTCs = [ (testcase,uniqueCtxs) - | (testcase,ctxs) <- M.elems cat_testcase_ctxs - , let uniqueCtxs = deleteFirstsBy applyHole ctxs commonCtxs - , not $ null uniqueCtxs - ] - commonTCs = [ (App newTop subtrees,ctxs) - | (coe,cats,ctxs) <- coercion_goalcats_commonCtxs - , let testcases_ctxs = catMaybes [ M.lookup cat cat_testcase_ctxs - | cat <- cats ] - , not $ null testcases_ctxs - , let fstLen (a,_) (b,_) = length (flatten a) `compare` length (flatten b) - , let (App tp subtrees,_) = -- pick smallest test case to be the representative - minimumBy fstLen testcases_ctxs - , let newTop = -- debug: put coerced contexts under a separate test case - if debug then tp { ctyp = (fst $ ctyp tp, coe)} else tp - ] - - starts = ccats gr startcat - - hl f c1 c2 = f (c1 dummyHole) == f (c2 dummyHole) --- applyHole = hl id -- TODO why doesn't this work for equality of contexts? - applyHole = hl show -- :: (Tree -> Tree) -> (Tree -> Tree) -> Bool - - funs = case lookupSymbol gr funname of - [] -> error $ "Function "++funname++" not found" - fs -> fs - - cat_testcase_ctxs = M.fromList - [ (goalcat,(testcase,ctxs)) - | testcase <- treesUsingFun gr funs - , let goalcat = ccatOf testcase -- never a coercion (coercions can't be goals) - , let ctxs = [ ctx | st <- starts - , ctx <- contextsFor gr st goalcat ] - ] :: M.Map ConcrCat (Tree,[Tree->Tree]) - goalcats = M.keys cat_testcase_ctxs - - coercion_goalcats_commonCtxs = - [ (coe,coveredGoalcats,ctxs) - | coe@(CC Nothing _) <- S.toList $ nonEmptyCats gr -- only coercions - , let coveredGoalcats = filter (coerces gr coe) goalcats - , let ctxs = [ ctx | st <- starts -- Contexts that have - , ctx <- contextsFor gr st coe -- a) hole of coercion, and are - , any (applyHole ctx) allCtxs ] -- b) relevant for the function we test - , length coveredGoalcats >= 2 -- no use if the coercion covers 0 or 1 categories - , not $ null ctxs ] - - - allCtxs = [ ctx | (_,ctxs) <- M.elems cat_testcase_ctxs - , ctx <- ctxs ] :: [Tree->Tree] - - commonCtxs = nubBy applyHole [ ctx | (_,_,ctxs) <- coercion_goalcats_commonCtxs - , ctx <- ctxs ] :: [Tree->Tree] - - -testTree :: Bool -> Grammar -> [Grammar] -> Tree -> Int -> [Tree -> Tree] -> Result -testTree debug gr tgrs t n ctxs = unlines - [ "* " ++ {- show n ++ ")" ++ -} show t - , showConcrFun gr w - , if debug then unlines $ tabularPrint gr t else "" - , unlines $ concat - [ [ "** " ++ show m ++ ") " ++ show (ctx (App (hole c) [])) - , langName gr ++ linearize gr (ctx t) - ] ++ - [ langName tgr ++ linearize tgr (ctx t) - | tgr <- tgrs ] - | (ctx,m) <- zip ctxs [1..] - ] - , "" ] - where - w = top t - c = snd (ctyp w) - langName gr = concrLang gr ++ "> " - - tabularPrint gr t = - let cseqs = [ concatMap showCSeq cseq - | cseq <- map (concrSeqs gr) (seqs $ top t) ] - tablins = tabularLin gr t :: [(String,String)] - in [ fieldname ++ ":\t" ++ lin ++ "\t" ++ s - | ((fieldname,lin),s) <- zip tablins cseqs ] - showCSeq (Left tok) = " " ++ show tok ++ " " - showCSeq (Right (i,j)) = " <" ++ show i ++ "," ++ show j ++ "> " - --------------------------------------------------------------------------------- --- Generate test trees - -treesUsingFun :: Grammar -> [Symbol] -> [Tree] -treesUsingFun gr detCNs = - [ tree - | detCN <- detCNs - , let (dets_cns,np_209) = ctyp detCN -- :: ([ConcrCat],ConcrCat) - , let bestArgs = case dets_cns of - [] -> [[]] - xs -> bestTrees detCN gr dets_cns - , tree <- App detCN `map` bestArgs ] - - -bestTrees :: Symbol -> Grammar -> [ConcrCat] -> [[Tree]] -bestTrees fun gr cats = - bestExamples fun gr $ take 200 -- change this to something else if too slow - [ featIthVec gr cats size i - | all (`S.member` nonEmptyCats gr) cats - , size <- [0..10] - , let card = featCardVec gr cats size - , i <- [0..card-1] - ] - -testsAsWellAs :: (Eq a, Eq b) => [a] -> [b] -> Bool -xs `testsAsWellAs` ys = go (xs `zip` ys) - where - go [] = - True - - go ((x,y):xys) = - and [ y' == y | (x',y') <- xys, x == x' ] && - go [ xy | xy@(x',_) <- xys, x /= x' ] - - -bestExamples :: Symbol -> Grammar -> [[Tree]] -> [[Tree]] -bestExamples fun gr vtrees = go [] vtrees_lins - where - syncategorematics = concatMap (lefts . concrSeqs gr) (seqs fun) - vtrees_lins = [ (vtree, syncategorematics ++ - concatMap (map snd . tabularLin gr) vtree) --linearise all trees at once - | vtree <- vtrees ] :: [([Tree],[String])] - - go cur [] = map fst cur - go cur (vt@(ts,lins):vts) - | any (`testsAsWellAs` lins) (map snd cur) = go cur vts - | otherwise = go' (vt:[ c | c@(_,clins) <- cur - , not (lins `testsAsWellAs` clins) ]) - vts - - go' cur vts | enough cur = map fst cur - | otherwise = go cur vts - - enough :: [([Tree],[String])] -> Bool - enough [(_,lins)] = all singleton (group $ sort lins) -- can stop earlier but let's not do that - enough _ = False - \ No newline at end of file diff --git a/src/tools/gftest/Graph.hs b/src/tools/gftest/Graph.hs deleted file mode 100644 index a440bf12d..000000000 --- a/src/tools/gftest/Graph.hs +++ /dev/null @@ -1,193 +0,0 @@ -module Graph where - -import qualified Data.Map as M -import Data.Map( Map, (!) ) -import qualified Data.Set as S -import Data.Set( Set ) -import Data.List( nub, sort, (\\) ) ---import Test.QuickCheck hiding ( generate ) - --- == almost everything in this module is inspired by King & Launchbury == - --------------------------------------------------------------------------------- --- depth-first trees - -data Tree a - = Node a [Tree a] - | Cut a - deriving ( Eq, Show ) - -type Forest a - = [Tree a] - -top :: Tree a -> a -top (Node x _) = x -top (Cut x) = x - --- pruning a possibly infinite forest -prune :: Ord a => Forest a -> Forest a -prune ts = go S.empty ts - where - go seen [] = [] - go seen (Cut x :ts) = Cut x : go seen ts - go seen (Node x vs:ts) - | x `S.member` seen = Cut x : go seen ts - | otherwise = Node x (take n ws) : drop n ws - where - n = length vs - ws = go (S.insert x seen) (vs ++ ts) - --- pre- and post-order traversals -preorder :: Tree a -> [a] -preorder t = preorderF [t] - -preorderF :: Forest a -> [a] -preorderF ts = go ts [] - where - go [] xs = xs - go (Cut x : ts) xs = go ts xs - go (Node x vs : ts) xs = x : go vs (go ts xs) - -postorder :: Tree a -> [a] -postorder t = postorderF [t] - -postorderF :: Forest a -> [a] -postorderF ts = go ts [] - where - go [] xs = xs - go (Cut x : ts) xs = go ts xs - go (Node x vs : ts) xs = go vs (x : go ts xs) - --- computing back-arrows -backs :: Ord a => Tree a -> Set a -backs t = S.fromList (go S.empty t) - where - go ups (Node x ts) = concatMap (go (S.insert x ups)) ts - go ups (Cut x) = [x | x `S.member` ups ] - --------------------------------------------------------------------------------- --- graphs - -type Graph a - = Map a [a] - -vertices :: Graph a -> [a] -vertices g = [ x | (x,_) <- M.toList g ] - -transposeG :: Ord a => Graph a -> Graph a -transposeG g = - M.fromListWith (++) $ - [ (y,[x]) | (x,ys) <- M.toList g, y <- ys ] ++ - [ (x,[]) | x <- vertices g ] - --------------------------------------------------------------------------------- --- graphs and trees - -generate :: Ord a => Graph a -> a -> Tree a -generate g x = Node x (map (generate g) (g!x)) - -dfs :: Ord a => Graph a -> [a] -> Forest a -dfs g xs = prune (map (generate g) xs) - -reach :: Ord a => Graph a -> [a] -> Graph a -reach g xs = M.fromList [ (x,g!x) | x <- preorderF (dfs g xs) ] - -dff :: Ord a => Graph a -> Forest a -dff g = dfs g (vertices g) - -preOrd :: Ord a => Graph a -> [a] -preOrd g = preorderF (dff g) - -postOrd :: Ord a => Graph a -> [a] -postOrd g = postorderF (dff g) - -scc1 :: Ord a => Graph a -> Forest a -scc1 g = reverse (dfs (transposeG g) (reverse (postOrd g))) - -scc2 :: Ord a => Graph a -> Forest a -scc2 g = dfs g (reverse (postOrd (transposeG g))) - -scc :: Ord a => Graph a -> Forest a -scc g = scc2 g - -sccs :: Ord a => Graph a -> [[a]] -sccs = map preorder . scc - --------------------------------------------------------------------------------- --- testing correctness - -{- -newtype G = G (Graph Int) deriving ( Show ) - -set :: (Ord a, Num a, Arbitrary a) => Gen [a] -set = (nub . sort . map abs) `fmap` arbitrary - -instance Arbitrary G where - arbitrary = - do xs <- set `suchThat` (not . null) - yss <- sequence [ listOf (elements xs) | x <- xs ] - return (G (M.fromList (xs `zip` yss))) - - shrink (G g) = - [ G (delNode x g) - | (x,_) <- M.toList g - ] ++ - [ G (delEdge x y g) - | (x,ys) <- M.toList g - , y <- ys - ] - where - delNode v g = - M.fromList - [ (x,filter (v/=) ys) - | (x,ys) <- M.toList g - , x /= v - ] - - delEdge v w g = - M.insert v ((g!v) \\ [w]) g - --- all vertices in a component can reach each other -prop_Scc_StronglyConnected (G g) = - whenFail (print cs) $ - and [ y `S.member` r | c <- cs, x <- c, let r = reach x, y <- c ] - where - cs = sccs g - - reach x = go S.empty [x] - where - go seen [] = seen - go seen (x:xs) - | x `S.member` seen = go seen xs - | otherwise = go (S.insert x seen) ((g!x) ++ xs) - --- vertices cannot forward-reach to other components -prop_Scc_NotConnected (G g) = - whenFail (print cs) $ - -- every vertex is somewhere - and [ or [ x `elem` c | c <- cs ] - | x <- vertices g - ] && - -- cannot foward-reach - and [ y `S.notMember` rx - | (c,d) <- pairs cs - , x <- c - , let rx = reach x - , y <- d - ] - where - cs = sccs g - - pairs (x:xs) = [ (x,y) | y <- xs ] ++ pairs xs - pairs [] = [] - - reach x = go S.empty [x] - where - go seen [] = seen - go seen (x:xs) - | x `S.member` seen = go seen xs - | otherwise = go (S.insert x seen) ((g!x) ++ xs) --} - --------------------------------------------------------------------------------- - diff --git a/src/tools/gftest/Main.hs b/src/tools/gftest/Main.hs deleted file mode 100644 index d68d78457..000000000 --- a/src/tools/gftest/Main.hs +++ /dev/null @@ -1,447 +0,0 @@ -{-# LANGUAGE DeriveDataTypeable #-} - -module Main where - -import Grammar -import EqRel - -import Control.Monad ( when ) -import Data.List ( intercalate, groupBy, sortBy, deleteFirstsBy, isInfixOf ) -import Data.Maybe ( fromMaybe, mapMaybe ) -import qualified Data.Set as S -import qualified Data.Map as M - -import System.Console.CmdArgs hiding ( name, args ) -import qualified System.Console.CmdArgs as A -import System.FilePath.Posix ( takeFileName ) -import System.IO ( stdout, hSetBuffering, BufferMode(..) ) - - -data GfTest - = GfTest - { grammar :: Maybe FilePath - -- Languages - , lang :: Lang - - -- Functions and cats - , function :: Name - , category :: Cat - , tree :: String - , start_cat :: Maybe Cat - , show_cats :: Bool - , show_funs :: Bool - , funs_of_arity :: Maybe Int - , show_coercions:: Bool - , show_contexts :: Maybe Int - , concr_string :: String - - -- Information about fields - , equal_fields :: Bool - , empty_fields :: Bool - , unused_fields :: Bool - , erased_trees :: Bool - - -- Compare to old grammar - , old_grammar :: Maybe FilePath - , only_changed_cats :: Bool - - -- Misc - , treebank :: Maybe FilePath - , count_trees :: Maybe Int - , debug :: Bool - , write_to_file :: Bool - - } deriving (Data,Typeable,Show,Eq) - -gftest = GfTest - { grammar = def &= typFile &= help "Path to the grammar (PGF) you want to test" - , lang = def &= A.typ "\"Eng Swe\"" - &= help "Concrete syntax + optional translations" - , tree = def &= A.typ "\"UseN tree_N\"" - &= A.name "t" &= help "Test the given tree" - , function = def &= A.typ "UseN" - &= A.name "f" &= help "Test the given function(s)" - , category = def &= A.typ "NP" - &= A.name "c" &= help "Test all functions with given goal category" - , start_cat = def &= A.typ "Utt" - &= A.name "s" &= help "Use the given category as start category" - , concr_string = def &= A.typ "the" &= help "Show all functions that include given string" - , show_cats = def &= help "Show all available categories" - , show_funs = def &= help "Show all available functions" - , funs_of_arity = def &= A.typ "2" &= help "Show all functions of arity 2" - , show_coercions= def &= help "Show coercions in the grammar" - , show_contexts = def &= A.typ "8410" &= help "Show contexts for a given concrete type (given as FId)" - , debug = def &= help "Show debug output" - , equal_fields = def &= A.name "q" &= help "Show fields whose strings are always identical" - , empty_fields = def &= A.name "e" &= help "Show fields whose strings are always empty" - , unused_fields = def &= help "Show fields that never make it into the top category" - , erased_trees = def &= A.name "r" &= help "Show trees that are erased" - , treebank = def &= typFile - &= A.name "b" &= help "Path to a treebank" - , count_trees = def &= A.typ "3" &= help "Number of trees of size <3>" - , old_grammar = def &= typFile - &= A.name "o" &= help "Path to an earlier version of the grammar" - , only_changed_cats = def &= help "When comparing against an earlier version of a grammar, only test functions in categories that have changed between versions" - , write_to_file = def &= help "Write the results in a file (_.org)" - } - - -main :: IO () -main = do - hSetBuffering stdout NoBuffering - - args <- cmdArgs gftest - - case grammar args of - Nothing -> putStrLn "Usage: `gftest -g [OPTIONS]'\nTo see available commands, run `gftest --help' or visit https://github.com/GrammaticalFramework/GF/blob/master/src/tools/gftest/README.md" - Just fp -> do - let (absName,grName) = (takeFileName $ stripPGF fp, stripPGF fp ++ ".pgf") --doesn't matter if the name is given with or without ".pgf" - - (langName:langTrans) = case lang args of - [] -> [ absName ++ "Eng" ] -- if no English grammar found, it will be given a default value later - langs -> [ absName ++ t | t <- words langs ] - - -- Read grammar and translations - gr <- readGrammar langName grName - grTrans <- sequence [ readGrammar lt grName | lt <- langTrans ] - - -- if language given by the user was not valid, use default language from Grammar - let langName = concrLang gr - - let startcat = startCat gr `fromMaybe` start_cat args - - testTree' t n = testTree False gr grTrans t n ctxs - where - s = top t - c = snd (ctyp s) - ctxs = concat [ contextsFor gr sc c - | sc <- ccats gr startcat ] - - output = -- Print to stdout or write to a file - if write_to_file args - then \x -> - do let fname = concat [ langName, "_", function args, category args, ".org" ] - writeFile fname x - putStrLn $ "Wrote results in " ++ fname - else putStrLn - - - intersectConcrCats cats_fields intersection = - M.fromListWith intersection - ([ (c,fields) - | (CC (Just c) _,fields) <- cats_fields - ] ++ - [ (cat,fields) - | (c@(CC Nothing _),fields) <- cats_fields - , (CC (Just cat) _,coe) <- coercions gr - , c == coe - ]) - - printStats tab = - sequence_ [ do putStrLn $ "==> " ++ c ++ ": " - putStrLn $ unlines (map (fs!!) xs) - | (c,vs) <- M.toList tab - , let fs = fieldNames gr c - , xs@(_:_) <- [ S.toList vs ] ] - ----------------------------------------------------------------------------- - -- Testing functions - - -- Test a tree - let trees = case tree args of - [] -> [] - ts -> lines ts - output $ - unlines [ testTree' (readTree gr tree) 1 | tree <- trees ] - - -- Test a function - let substrs xs = filter (/="*") $ groupBy (\a b -> a/='*' && b/='*') xs - let cats = case category args of - [] -> [] - cs -> if '*' `elem` cs - then let subs = substrs cs - in nub [ cat | (cat,_,_,_) <- concrCats gr - , all (`isInfixOf` cat) subs ] - else words cs - output $ - unlines [ testTree' t n - | cat <- cats - , (t,n) <- treesUsingFun gr (functionsByCat gr cat) `zip` [1..]] - - -- Test all functions in a category - let funs = case function args of - [] -> [] - fs -> if '*' `elem` fs - then let subs = substrs fs - in nub [ f | s <- symbols gr, let f = show s - , all (`isInfixOf` f) subs - , arity s >= 1 ] - else words fs - output $ - unlines [ testFun (debug args) gr grTrans startcat f - | f <- funs ] - ------------------------------------------------------------------------------ --- Information about the grammar - - -- Show contexts for a particular concrete category - case show_contexts args of - Nothing -> return () - Just fid -> mapM_ print - [ ctx dummyHole - | start <- ccats gr startcat - , ctx <- contextsFor gr start (mkCC gr fid) ] - - -- Show available categories - when (show_cats args) $ do - putStrLn "* Categories in the grammar:" - let concrcats = sortBy (\(_,a,_,_) (_,b,_,_) -> a `compare` b) (concrCats gr) - sequence_ [ do putStrLn cat - when (debug args) $ - putStrLn $ unwords $ - [ " Compiles to concrete" ] ++ - [ "categories " ++ show bg++"—"++show end - | bg/=end ] ++ - [ "category " ++ show bg - | bg==end ] - | (cat,bg,end,_) <- concrcats - , end >= 0] - - -- Show available functions - when (show_funs args) $ do - putStrLn "* Functions in the grammar:" - putStrLn $ unlines $ nub [ show s | s <- symbols gr ] - - -- Show coercions in the grammar - when (show_coercions args) $ do - putStrLn "* Coercions in the grammar:" - putStrLn $ unlines [ show cat++"--->"++show coe | (cat,coe) <- coercions gr ] - - case funs_of_arity args of - Nothing -> return () - Just n -> do - putStrLn $ "* Functions in the grammar of arity " ++ show n ++ ":" - putStrLn $ unlines $ nub [ show s | s <- symbols gr, arity s == n ] - - -- Show all functions that contain the given string - -- (e.g. English "it" appears in DefArt, ImpersCl, it_Pron, …) - case concr_string args of - [] -> return () - str -> do putStrLn $ "### The following functions contain the string '" ++ str ++ "':" - putStr "==> " - putStrLn $ intercalate ", " $ nub [ name s | s <- hasConcrString gr str] - - -- Show empty fields - when (empty_fields args) $ do - putStrLn "### Empty fields:" - printStats $ intersectConcrCats (emptyFields gr) S.intersection - putStrLn "" - - -- Show erased trees - when (erased_trees args) $ do - putStrLn "* Erased trees:" - sequence_ - [ do putStrLn ("** " ++ intercalate "," erasedTrees ++ " : " ++ uncoerceAbsCat gr c) - sequence_ - [ do putStrLn ("- Tree: " ++ showTree t) - putStrLn ("- Lin: " ++ s) - putStrLn $ unlines - [ "- Trans: "++linearize tgr t - | tgr <- grTrans ] - | t <- ts - , let s = linearize gr t - , let erasedSymbs = [ sym | sym <- flatten t, c==snd (ctyp sym) ] - ] - | top <- take 1 $ ccats gr startcat - , (c,ts) <- forgets gr top - , let erasedTrees = - concat [ [ showTree subtree - | sym <- flatten t - , let csym = snd (ctyp sym) - , c == csym || coerces gr c csym - , let Just subtree = subTree sym t ] - | t <- ts ] - ] - putStrLn "" - - -- Show unused fields - when (unused_fields args) $ do - - let unused = - [ (c,S.fromList notUsed) - | tp <- ccats gr startcat - , (c,is) <- reachableFieldsFromTop gr tp - , let ar = head $ - [ length (seqs f) - | f <- symbols gr, snd (ctyp f) == c ] ++ - [ length (seqs f) - | (b,a) <- coercions gr, a == c - , f <- symbols gr, snd (ctyp f) == b ] - notUsed = [ i | i <- [0..ar-1], i `notElem` is ] - , not (null notUsed) - ] - putStrLn "### Unused fields:" - printStats $ intersectConcrCats unused S.intersection - putStrLn "" - - -- Show equal fields - let tab = intersectConcrCats (equalFields gr) (/\) - when (equal_fields args) $ do - putStrLn "### Equal fields:" - sequence_ - [ putStrLn ("==> " ++ c ++ ":\n" ++ cl) - | (c,eqr) <- M.toList tab - , let fs = fieldNames gr c - , cl <- case eqr of - Top -> ["TOP"] - Classes xss -> [ unlines (map (fs!!) xs) - | xs@(_:_:_) <- xss ] - ] - putStrLn "" - - case count_trees args of - Nothing -> return () - Just n -> do let start = head $ ccats gr startcat - let i = featCard gr start n - let iTot = sum [ featCard gr start m | m <- [1..n] ] - putStr $ "There are "++show iTot++" trees up to size "++show n - putStrLn $ ", and "++show i++" of exactly size "++show n++".\nFor example: " - putStrLn $ "* " ++ show (featIth gr start n 0) - putStrLn $ "* " ++ show (featIth gr start n (i-1)) - - -------------------------------------------------------------------------------- --- Read trees from treebank. - - treebank' <- - case treebank args of - Nothing -> return [] - Just fp -> do - tb <- readFile fp - return [ readTree gr s - | s <- lines tb ] - mapM_ print treebank' - -------------------------------------------------------------------------------- --- Comparison with old grammar - - case old_grammar args of - Nothing -> return () - Just fp -> do - oldgr <- readGrammar langName (stripPGF fp ++ ".pgf") - let ogr = oldgr { concrLang = concrLang oldgr ++ "-OLD" } - difcats = diffCats ogr gr -- (acat, [#o, #n], olabels, nlabels) - - -------------------------------------------------------------------------- - -- generate statistics of the changes in the concrete categories - let ccatChangeFile = langName ++ "-ccat-diff.org" - writeFile ccatChangeFile "" - sequence_ - [ appendFile ccatChangeFile $ unlines - [ "* " ++ acat - , show o ++ " concrete categories in the old grammar," - , show n ++ " concrete categories in the new grammar." - , "** Labels only in old (" ++ show (length ol) ++ "):" - , intercalate ", " ol - , "** Labels only in new (" ++ show (length nl) ++ "):" - , intercalate ", " nl ] - | (acat, [o,n], ol, nl) <- difcats ] - when (debug args) $ - sequence_ - [ appendFile ccatChangeFile $ - unlines $ - ("* All concrete cats in the "++age++" grammar:"): - [ show cts | cts <- concrCats g ] - | (g,age) <- [(ogr,"old"),(gr,"new")] ] - - putStrLn $ "Created file " ++ ccatChangeFile - - -------------------------------------------------------------------------- - -- Print out tests for all functions in the changed cats. - -- If -f, -c or --treebank specified, use them. - - let f cat = (cat, treesUsingFun gr $ functionsByCat gr cat) - - byCat = [ f cat | cat <- cats ] -- from command line arg -c - changed = [ f cat | (cat,_,_,_) <- difcats - , only_changed_cats args ] - byFun = [ (cat, treesUsingFun gr fs) - | funName <- funs -- comes from command line arg -f - , let fs@(s:_) = lookupSymbol gr funName - , let cat = snd $ Grammar.typ s ] - fromTb = [ (cat,[tree]) | tree <- treebank' - , let (CC (Just cat) _) = ccatOf tree ] - - treesToTest = - case concat [byFun, byCat, changed, fromTb] of - [] -> [ f cat -- nothing else specified -> test all functions - | (cat,_,_,_) <- concrCats gr ] - xs -> S.toList $ S.fromList xs - - writeLinFile file grammar otherGrammar = do - writeFile file "" - putStrLn "Testing functions in… " - diff <- concat `fmap` - sequence [ do let cs = [ compareTree grammar otherGrammar grTrans t - | t <- trees ] - putStr $ cat ++ " \r" - -- prevent lazy evaluation; make printout accurate - appendFile ("/tmp/"++file) (unwords $ map show cs) - return cs - | (cat,trees) <- treesToTest ] - let relevantDiff = go [] [] diff where - go res seen [] = res - go res seen (Comparison f ls:cs) = - if null uniqLs then go res seen cs - else go (Comparison f uniqLs:res) (uniqLs++seen) cs - where uniqLs = deleteFirstsBy ctxEq ls seen - ctxEq (a,_,_,_) (b,_,_,_) = a==b - shorterTree c1 c2 = length (funTree c1) `compare` length (funTree c2) - writeFile file $ unlines - [ show comp - | comp <- sortBy shorterTree relevantDiff ] - - - writeLinFile (langName ++ "-lin-diff.org") gr ogr - putStrLn $ "Created file " ++ (langName ++ "-lin-diff.org") - - --------------------------------------------------------------------------- - -- Print statistics about the functions: e.g., in the old grammar, - -- all these 5 functions used to be in the same category: - -- [DefArt,PossPron,no_Quant,this_Quant,that_Quant] - -- but in the new grammar, they are split into two: - -- [DefArt,PossPron,no_Quant] and [this_Quant,that_Quant]. - let groupFuns grammar = -- :: Grammar -> [[Symbol]] - concat [ groupBy sameCCat $ sortBy compareCCat funs - | (cat,_,_,_) <- difcats - , let funs = functionsByCat grammar cat ] - - sortByName = sortBy (\s t -> name s `compare` name t) - writeFunFile groupedFuns file grammar = do - writeFile file "" - sequence_ [ do appendFile file "---\n" - appendFile file $ unlines - [ showConcrFun gr fun - | fun <- sortByName funs ] - | funs <- groupedFuns ] - - writeFunFile (groupFuns ogr) (langName ++ "-old-funs.org") ogr - writeFunFile (groupFuns gr) (langName ++ "-new-funs.org") gr - - putStrLn $ "Created files " ++ langName ++ "-(old|new)-funs.org" - - - where - - nub = S.toList . S.fromList - - sameCCat :: Symbol -> Symbol -> Bool - sameCCat s1 s2 = snd (ctyp s1) == snd (ctyp s2) - - compareCCat :: Symbol -> Symbol -> Ordering - compareCCat s1 s2 = snd (ctyp s1) `compare` snd (ctyp s2) - - stripPGF :: String -> String - stripPGF s = case reverse s of - 'f':'g':'p':'.':name -> reverse name - name -> s - diff --git a/src/tools/gftest/Mu.hs b/src/tools/gftest/Mu.hs deleted file mode 100644 index 4aa11e316..000000000 --- a/src/tools/gftest/Mu.hs +++ /dev/null @@ -1,113 +0,0 @@ -module Mu where - -import Data.Map( Map, (!) ) -import qualified Data.Map as M -import Data.Set( Set ) -import qualified Data.Set as S -import Graph - --------------------------------------------------------------------------------- - --- naive implementation of fixpoint computation -mu0 :: (Ord x, Eq a) => a -> [(x, [x], [a] -> a)] -> [x] -> [a] -mu0 bot defs zs = [ done!z | z <- zs ] - where - xs = [ x | (x, _, _) <- defs ] - done = iter [ bot | _ <- xs ] - - iter as - | as == as' = tab - | otherwise = iter as' - where - tab = M.fromList (xs `zip` as) - as' = [ f [ tab!y | y <- ys ] - | (_,(_, ys, f)) <- as `zip` defs - ] - --------------------------------------------------------------------------------- - --- scc-based implementation of fixpoint computation -{- - a --^ initial/bottom value (smallest element) in the fixpoint computation --> [( x, [x] --^ A single category, its arguments - , [a] -> a) --^ function that takes as its argument a list of values that we want to compute for the [x] - ] --> [x] --^ All categories that you want to see the answer for --> [a] --^ Values for the given categories --} - -mu :: (Ord x, Eq a) => a -> [(x, [x], [a] -> a)] -> [x] -> [a] -mu bot defs zs = [ vtab?z | z <- zs ] - where - ftab = M.fromList [ (x,f) | (x,_,f) <- defs ] - graph = reach (M.fromList [ (x,xs) | (x,xs,_) <- defs ]) zs - vtab = foldl compute M.empty (scc graph) - - compute vtab t = fix (-1) vtab (map (vtab ?) xs) - where - xs = S.toList (backs t) - - fix 0 vtab _ = vtab - fix n vtab as - | as' == as = vtab' - | otherwise = fix (n-1) vtab' as' - where - (_,vtab') = eval t vtab - as' = map (vtab' ?) xs - - eval (Cut x) vtab = (vtab?x, vtab) - eval (Node x ts) vtab = (a, M.insert x a vtab') - where - (as, vtab') = evalList ts vtab - a = (ftab!x) as - - evalList [] vtab = ([], vtab) - evalList (t:ts) vtab = (a:as, vtab'') - where - (a, vtab') = eval t vtab - (as,vtab'') = evalList ts vtab' - - vtab ? x = case M.lookup x vtab of - Nothing -> bot - Just a -> a - --------------------------------------------------------------------------------- - --- diff/scc-based implementation of fixpoint computation -muDiff :: (Ord x, Eq a) - => a -> (a->Bool) -> (a->a->a) -> (a->a->a) - -> [(x, [x], [a] -> a)] - -> [x] -> [a] -muDiff bot isBot diff apply defs zs = [ vtab?z | z <- zs ] - where - ftab = M.fromList [ (x,f) | (x,_,f) <- defs ] - graph = reach (M.fromList [ (x,xs) | (x,xs,_) <- defs ]) zs - vtab = foldl compute M.empty (scc graph) - - compute vtab t = fix vtab M.empty - where - xs = S.toList (backs t) - - fix dtab vtab - | all isBot ds = vtab' - | otherwise = fix (M.fromList (xs `zip` ds)) vtab' - where - dtab' = eval t dtab - vtab' = foldr (\(x,d) -> M.alter (Just . apply' d) x) vtab (M.toList dtab') - ds = map (dtab' ?) xs - - apply' d Nothing = apply d bot - apply' d (Just a) = apply d a - - eval (Cut x) tab = tab - eval (Node x ts) tab = M.insert x d tab' - where - tab' = foldl (flip eval) tab ts - d = (ftab!x) [ tab'?x | x <- map top ts ] `diff` (vtab?x) - - vtab ? x = case M.lookup x vtab of - Nothing -> bot - Just a -> a - --------------------------------------------------------------------------------- - diff --git a/src/tools/gftest/README.md b/src/tools/gftest/README.md index beecaf191..f8e90551c 100644 --- a/src/tools/gftest/README.md +++ b/src/tools/gftest/README.md @@ -1,563 +1 @@ -# gftest: Automatic systematic test case generation for GF grammars - -`gftest` is a program for automatically generating systematic test -cases for GF grammars. The basic use case is to give `gftest` a -PGF grammar, a concrete language and a function; then `gftest` generates a -representative and minimal set of example sentences for a human to look at. - -There are examples of actual generated test cases later in this -document, as well as the full list of options to give to `gftest`. - -## Table of Contents - -- [Installation](#installation) - - [Prerequisites](#prerequisites) - - [Install gftest](#install-gftest) -- [Common use cases](#common-use-cases) - - [Grammar: `-g`](#grammar--g) - - [Language: `-l`](#language--l) - - [Function(s) to test: `-f`](#functions-to-test--f) - - [Start category for context: `-s`](#start-category-for-context--s) - - [Category to test: `-c`](#category-to-test--c) - - [Tree to test: `-t`](#tree-to-test--t) - - [Compare against an old version of the grammar: `-o`](#compare-against-an-old-version-of-the-grammar--o) - - [Information about a particular string: `--concr-string`](#information-about-a-particular-string---concr-string) - - [Write into a file: `-w`](#write-into-a-file--w) -- [Less common use cases](#less-common-use-cases) - - [Empty or always identical fields: `-e`, `-q`](#empty-or-always-identical-fields--e--q) - - [Unused fields: `-u`](#unused-fields--u) - - [Erased trees: `-r`](#erased-trees--r) - - [Debug information: `-d`](#debug-intormation--d) -- [Detailed information about the grammar](#detailed-information-about-the-grammar) - - [--show-cats](#--show-cats) - - [--show-funs](#--show-funs) - - [--show-coercions](#--show-coercions) - - [--show-contexts](#--show-contexts) - - [--count-trees](#--count-trees) - - [--funs-of-arity](#--funs-of-arity) - -## Installation - -### Prerequisites - -You need the library `PGF2`. Here are instructions how to install: - -1) Install C runtime: go to the directory [GF/src/runtime/c](https://github.com/GrammaticalFramework/GF/tree/master/src/runtime/c), see -instructions in INSTALL -1) Install PGF2 in one of the two ways: - * **EITHER** Go to the directory - [GF/src/runtime/haskell-bind](https://github.com/GrammaticalFramework/GF/tree/master/src/runtime/haskell-bind), - do `cabal install` - * **OR** Go to the root directory of - [GF](https://github.com/GrammaticalFramework/GF/) and compile GF - with C-runtime system support: `cabal - install -fc-runtime`, see more information [here](http://www.grammaticalframework.org/doc/gf-developers.html#toc16). - -### Install gftest - -Go to -[GF/src/tools](https://github.com/GrammaticalFramework/GF/tree/master/src/tools), -do `cabal install`. It creates an executable `gftest`. - - -## Common use cases - -Run `gftest --help` of `gftest -?` to get the list of options. - -``` -Common flags: - -g --grammar=FILE Path to the grammar (PGF) you want to test - -l --lang="Eng Swe" Concrete syntax + optional translations - -f --function=UseN Test the given function(s) - -c --category=NP Test all functions with given goal category - -t --tree="UseN tree_N" Test the given tree - -s --start-cat=Utt Use the given category as start category - --show-cats Show all available categories - --show-funs Show all available functions - --funs-of-arity=2 Show all functions of arity 2 - --show-coercions Show coercions in the grammar - --show-contexts=8410 Show contexts for a given concrete type (given as FId) - --concr-string=the Show all functions that include given string - -q --equal-fields Show fields whose strings are always identical - -e --empty-fields Show fields whose strings are always empty - -u --unused-fields Show fields that never make it into the top category - -r --erased-trees Show trees that are erased - -o --old-grammar=ITEM Path to an earlier version of the grammar - --only-changed-cats When comparing against an earlier version of a - grammar, only test functions in categories that have - changed between versions - -b --treebank=ITEM Path to a treebank - --count-trees=3 Number of trees of depth - -d --debug Show debug output - -w --write-to-file Write the results in a file (_.org) - -? --help Display help message - -V --version Print version information -``` - -### Grammar: `-g` - -Give the PGF grammar as an argument with `-g`. If the file is not in -the same directory, you need to give the full file path. - -You can give the grammar with or without `.pgf`. - -Without a concrete syntax you can't do much, but you can see the -available categories and functions with `--show-cats` and `--show-funs` - -Examples: - -* `gftest -g Foods --show-funs` -* `gftest -g /home/inari/grammars/LangEng.pgf --show-cats` - - -### Language: `-l` - -Give a concrete language. It assumes the format `AbsNameConcName`, and you should only give the `ConcName` part. - -You can give multiple languages, in which case it will create the test cases based on the first, and show translations in the rest. - -Examples: - -* `gftest -g Phrasebook -l Swe --show-cats` -* `gftest -g Foods -l "Spa Eng" -f Pizza` - -### Function(s) to test: `-f` - -Given a grammar (`-g`) and a concrete language ( `-l`), test a function or several functions. - -Examples: - -* `gftest -g Lang -l "Dut Eng" -f UseN` -* `gftest -g Phrasebook -l Spa -f "ByTransp ByFoot"` - -You can use the wildcard `*`, if you want to match multiple functions. Examples: - -* `gftest -g Lang -l Eng -f "*hat*"` - -matches `hat_N, hate_V2, that_Quant, that_Subj, whatPl_IP` and `whatSg_IP`. - -* `gftest -g Lang -l Eng -f "*hat*u*"` - -matches `that_Quant` and `that_Subj`. - -* `gftest -g Lang -l Eng -f "*"` - -matches all functions in the grammar. (As of March 2018, takes 13 -minutes for the English resource grammar, and results in ~40k -lines. You may not want to do this for big grammars.) - -### Start category for context: `-s` - -Give a start category for contexts. Used in conjunction with `-f`, -`-c`, `-t` or `--count-trees`. If not specified, contexts are created -for the start category of the grammar. - -Example: - -* `gftest -g Lang -l "Dut Eng" -f UseN -s Adv` - -This creates a hole of `CN` in `Adv`, instead of the default start category. - -### Category to test: `-c` - -Given a grammar (`-g`) and a concrete language ( `-l`), test all functions that return a given category. - -Examples: - -* `gftest -g Phrasebook -l Fre -c Modality` -* `gftest -g Phrasebook -l Fre -c ByTransport -s Action` - - -### Tree to test: `-t` - -Given a grammar (`-g`) and a concrete language ( `-l`), test a complete tree. - -Example: - -* `gftest -g Phrasebook -l Dut -t "ByTransp Bus"` - -You can combine it with any of the other flags, e.g. put it in a -different start category: - -* `gftest -g Phrasebook -l Dut -t "ByTransp Bus" -s Action` - - -This may be useful for the following case. Say you tested `PrepNP`, -and the default NP it gave you only uses the word *car*, but you -would really want to see it for some other noun—maybe `car_N` itself -is buggy, and you want to be sure that `PrepNP` works properly. So -then you can call the following: - -* `gftest -g TestLang -l Eng -t "PrepNP with_Prep (MassNP (UseN beer_N))"` - -### Compare against an old version of the grammar: `-o` - -Give a grammar, a concrete syntax, and an old version of the same -grammar as a separate PGF file. The program generates test sentences -for all functions (if no other arguments), linearises with both -grammars, and outputs those that differ between the versions. It -writes the differences into files. - -Example: - -``` -> gftest -g TestLang -l Eng -o TestLangOld -Created file TestLangEng-ccat-diff.org -Testing functions in… - -Created file TestLangEng-lin-diff.org -Created files TestLangEng-(old|new)-funs.org -``` - -* TestLangEng-ccat-diff.org: All concrete categories that have - changed. Shows e.g. if you added or removed a parameter or a - field. - -* **TestLangEng-lin-diff.org** (usually the most relevant file): All -trees that have different linearisations in the following format. - -``` - * send_V3 - - ** UseCl (TTAnt TPres ASimul) PPos (PredVP (UsePron we_Pron) (ReflVP (Slash3V3 ∅ (UsePron it_Pron)))) - TestLangDut> we sturen onszelf ernaar - TestLangDut-OLD> we sturen zichzelf ernaar - - - ** UseCl (TTAnt TPast ASimul) PPos (PredVP (UsePron we_Pron) (ReflVP (Slash3V3 ∅ (UsePron it_Pron)))) - TestLangDut> we stuurden onszelf ernaar - TestLangDut-OLD> we stuurden zichzelf ernaar -``` - -* TestLangEng-old-funs.org and TestLangEng-new-funs.org: groups the - functions by their concrete categories. Shows difference if you have - e.g. added or removed parameters, and that has created new versions of - some functions: say you didn't have gender in nouns, but now you - have, then all functions taking nouns have suddenly a gendered - version. (This is kind of hard to read, don't worry too much if the - output doesn't make any sense.) - -#### Additional arguments to `-o` - -The default mode is to test all functions, but you can also give any -combination of `-s`, `-f`, `-c`, `--treebank`/`-b` and `--only-changed-cats`. - -With `-s`, you can change the start category in which contexts are -generated. - -With `-f` and `-c`, it tests only the specified functions and -categories. -With `-b FILEPATH` (`-b`=`--treebank`), it tests only the trees in the file. - -With `--only-changed-cats`, it only test functions in those categories -that have changed between the two versions. - -Examples: - -* `gftest -g TestLang -l Eng -o TestLangOld` tests all functions -* `gftest -g TestLang -l Eng -o TestLangOld -s S` tests all functions in start category S -* `gftest -g TestLang -l Eng -o TestLangOld --only-changed-cats` tests only changed categories. If no categories have changed (and no other arguments specified), tests everything. -* `gftest -g TestLang -l Eng -o TestLangOld -f "AdjCN AdvCN" -c Adv -b trees.txt` tests functions, `AdjCN` and `AdvCN`; same for all functions that produce an `Adv`, and all trees in trees.txt. - -### Information about a particular string: `--concr-string` - -Show all functions that introduce the string given as an argument. - -Example: - -* `gftest -g Lang -l Eng --concr-string it` - -which gives the answer `==> CleftAdv, CleftNP, DefArt, ImpersCl, it_Pron` - -(Note that you have the same feature in GF shell, command `morpho_analyse`/`ma`.) - - -### Write into a file: `-w` - -Writes the results into a file of format `_.org`, -e.g. TestLangEng-UseN.org. Recommended to open it in emacs org-mode, -so you get an overview, and you can maybe ignore some trees if you -think they are redundant. - -1) When you open the file, you see a list of generated test cases, like this: ![Instructions how to use org mode](https://raw.githubusercontent.com/inariksit/GF-testing/master/doc/instruction-1.png) -Place cursor to the left and click tab to open it. - -2) You get a list of contexts for the test case. Keep the cursor where it was if you want to open everything at the same time. Alternatively, scroll down to one of the contexts and press tab there, if you only want to open one. -![Instructions how to use org mode](https://raw.githubusercontent.com/inariksit/GF-testing/master/doc/instruction-2.png) - -3) Now you can read the linearisations. -![Instructions how to use org mode](https://raw.githubusercontent.com/inariksit/GF-testing/master/doc/instruction-3.png) - -If you want to close the test case, just press tab again, keeping the -cursor where it's been all the time (line 31 in the pictures). - -## Less common use cases - -The topics here require some more obscure GF-fu. No need to worry if -the terms are not familiar to you. - - -### Empty or always identical fields: `-e`, `-q` - -Information about the fields: always empty, or always equal to each -other. Example of empty fields: - -``` -> gftest -g Lang -l Dut -e -* Empty fields: -==> Ant: s - -==> Pol: s - -==> Temp: s - -==> Tense: s - -==> V: particle, prefix -``` - -The categories `Ant`, `Pol`, `Temp` and `Tense` are as expected empty; -there's no string to be added to the sentences, just a parameter that -*chooses* the right forms of the clause. - -`V` having empty fields `particle` and `prefix` is in this case just -an artefact of a small lexicon: we happen to have no intransitive -verbs with a particle or prefix in the core 300-word vocabulary. But a -grammarian would know that it's still relevant to keep those fields, -because in some bigger application such a verb may show up. - -On the other hand, if some other field is always empty, it might be a -hint for the grammarian to remove it altogether. - -Example of equal fields: - -``` -> gftest -g Lang -l Dut -q -* Equal fields: -==> RCl: -s Pres Simul Pos Utr Pl -s Pres Simul Pos Neutr Pl - -==> RCl: -s Pres Simul Neg Utr Pl -s Pres Simul Neg Neutr Pl - -==> RCl: -s Pres Anter Pos Utr Pl -s Pres Anter Pos Neutr Pl - -==> RCl: -s Pres Anter Neg Utr Pl -s Pres Anter Neg Neutr Pl - -==> RCl: -s Past Simul Pos Utr Pl -s Past Simul Pos Neutr Pl -… -``` - -Here we can see that in relative clauses, gender does not seem to play -any role in plural. This could be a hint for the grammarian to make a -leaner parameter type, e.g. `param RClAgr = SgAgr | PlAgr `. - - -### Unused fields: `-u` - -These fields are not empty, but they are never used in the top -category. The top category can be specified by `-s`, otherwise it is -the default start category of the grammar. - -Note that if you give a start category from very low, such as `Adv`, -you get a whole lot of categories and fields that naturally have no -way of ever making it into an adverb. So this is mostly meaningful to -use for the start category. - - -### Erased trees: `-r` - -Show trees that are erased in some function, i.e. a function `F : A -> B -> C` has arguments A and B, but doesn't use one of them in the resulting tree of type C. This is usually a bug. - -Example: - - - -``` -> gftest -g Lang -l "Dut Eng" -r - -* Erased trees: - -** RelCl (ExistNP something_NP) : RCl -- Tree: AdvS (PrepNP with_Prep (RelNP (UsePron it_Pron) (UseRCl (TTAnt TPres ASimul) PPos (RelCl (ExistNP something_NP))))) (UseCl (TTAnt TPres ASimul) PPos (ExistNP something_NP)) -- Lin: ermee is er iets -- Trans: with it, such that there is something, there is something - -** write_V2 : V2 -- Tree: AdvS (PrepNP with_Prep (PPartNP (UsePron it_Pron) write_V2)) (UseCl (TTAnt TPres ASimul) PPos (ExistNP something_NP)) -- Lin: ermee is er iets -- Trans: with it written there is something -``` - -In the first result, an argument of type `RCl` is missing in the tree constructed by `RelNP`, and in the second result, the argument `write_V2` is missing in the tree constructed by `PPartNP`. In both cases, the English linearisation contains all the arguments, but in the Dutch one they are missing. (This bug is already fixed, just showing it here to demonstrate the feature.) - -## Detailed information about the grammar - -### Debug information: `-d` - -When combined with `-f`, `-c` or `-t`, two things happen: - -1) The trees are linearised using `tabularLinearize`, which shows the -inflection table of all forms. -2) You can see traces of pruning that happens in testing functions: -contexts that are common to several concrete categories are put under -a separate test case. - -When combined with `--show-cats`, also the concrete categories are -shown. - -### --show-cats - -Shows the categories in the grammar. With `--debug`/`-d`, shows also -concrete categories. - -Example: - -``` -> gftest -g Foods -l Spa --show-cats -d - -* Categories in the grammar: -Comment - Compiles to concrete category 0 -Item - Compiles to concrete categories 1—4 -Kind - Compiles to concrete categories 5—6 -Quality - Compiles to concrete categories 7—8 -Question - Compiles to concrete category 9 -``` - -### --show-funs - -Shows the functions in the grammar. (Nothing fancy happens with other flags.) - - -### --show-coercions - -First I'll explain what *coercions* are, then why it may be -interesting to show them. Let's take a Spanish Foods grammar, and -consider the category `Quality`, e.g. `Good` and `Vegan`. -`Good` "bueno/buena/buenos/buenas" goes before the noun it modifies, -whereas `Vegan` "vegano/vegana/…" goes after, so these will become different -*concrete categories* in the PGF: `Quality_before` and -`Quality_after`. (In reality, they are something like `Quality_7` and -`Quality_8` though.) - -Now, this difference is meaningful only when the adjective is modifying -the noun: "la buena pizza" vs. "la pizza vegana". But when the -adjective is in a predicative position, they both behave the same: -"la pizza es buena" and "la pizza es vegana". For this, the grammar -creates a *coercion*: both `Quality_before` and `Quality_after` may be -treated as `Quality_whatever`. To save some redundant work, this coercion `Quality_whatever` -appears in the type of predicative function, whereas the -modification function has to be split into two different functions, -one taking `Quality_before` and other `Quality_after`. - -Now you know what coercions are, this is how it looks like in the program: - -``` -> gftest -g Foods -l Spa --show-coercions -* Coercions in the grammar: -Quality_7--->_11 -Quality_8--->_11 -``` - -(Just mentally replace 7 with `before`, 8 with `after` and 11 with `whatever`.) - -### --show-contexts - -Show contexts for a given concrete category, given as an FId -(i.e. Int). The concrete category may be a coercion or a normal -category. By combining with [`-s`](#start-category-for-context--s), -you can change the start category of the context. - -(You can get a list of all concrete categories by pairing `--show-cats` -with `--debug`: see [`--show-cats`](#--show-cats).) - -Examples: - -* First, find out some concrete categories: - -``` - > gftest -g Foods -l Spa --show-cats -d - … - Quality - Compiles to concrete categories 7—8 - … -``` - -* Then, list the contexts for some of them, say `Quality_7`: - -``` - > gftest -g Foods -l Spa --show-contexts 7 - - Pred (That (Mod ∅ Wine)) Vegan - Pred (That Wine) ∅ - Pred (These (Mod ∅ Wine)) Vegan - Pred (These Wine) ∅ - Pred (That (Mod ∅ Pizza)) Vegan - Pred (That Pizza) ∅ - Pred (These (Mod ∅ Pizza)) Vegan - Pred (These Pizza) ∅ -``` - -* Check out from [`--show-coercions`](#--show-coercions) how to find -coercions, and you can try `--show-contexts` with them: - -``` - > gftest -g Foods -l Spa --show-contexts 11 - - Pred (That Wine) ∅ - Pred (These Wine) ∅ - Pred (That Pizza) ∅ - Pred (These Pizza) ∅ -``` - -### --count-trees - -Number of trees up to given size. Gives a number how many trees, and a -couple of examples from the highest size. Examples: - -``` -> gftest -g TestLang -l Eng --count-trees 10 -There are 675312 trees up to size 10, and 624512 of exactly size 10. -For example: -* AdvS today_Adv (UseCl (TTAnt TPres ASimul) PPos (ExistNP (UsePron i_Pron))) -* UseCl (TTAnt TCond AAnter) PNeg (PredVP (SelfNP (UsePron they_Pron)) UseCopula) -``` - -This counts the number of trees in the start category. You can also -specify a category: - -``` -> gftest -g TestLang -l Eng --count-trees 4 -s Adv -There are 2409 trees up to size 4, and 2163 of exactly size 4. -For example: -* AdAdv very_AdA (PositAdvAdj young_A) -* PrepNP above_Prep (UsePron they_Pron) -``` - -### --funs-of-arity - -Show all functions of given arity (not up to). - -Example: - -``` -> gftest -g Phrasebook --funs-of-arity 3 -* Functions in the grammar of arity 3: -ADoVerbPhrasePlace -AModVerbPhrase -HowFarFromBy -QWhereModVerbPhrase -``` +Development moved to https://github.com/GrammaticalFramework/gftest