hallgren 3f57151cc3 Represent identifiers as UTF-8-encoded ByteStrings ...

This was a fairly simple change thanks to previous work on making the Ident
type abstract and the fact that PGF.CId already uses UTF-8-encoded
ByteStrings.

One potential pitfall is that Data.ByteString.UTF8 uses the same type for
ByteStrings as Data.ByteString. I renamed ident2bs to ident2utf8 and
bsCId to utf8CId, to make it clearer that they work with UTF-8-encoded
ByteStrings.

Since both the compiler input and identifiers are now UTF-8-encoded
ByteStrings, the lexer now creates identifiers without copying any characters.
**END OF DESCRIPTION***

Place the long patch description above the ***END OF DESCRIPTION*** marker.
The first line of this file will be the patch name.


This patch contains the following changes:

M ./src/compiler/GF/Compile/CheckGrammar.hs -3 +3
M ./src/compiler/GF/Compile/GrammarToPGF.hs -2 +2
M ./src/compiler/GF/Grammar/Binary.hs -5 +1
M ./src/compiler/GF/Grammar/Lexer.x -11 +13
M ./src/compiler/GF/Infra/Ident.hs -19 +36
M ./src/runtime/haskell/PGF.hs -1 +1
M ./src/runtime/haskell/PGF/CId.hs -2 +3

2013-11-26 16:12:03 +00:00

..

binary/Data

Eliminate mutual dependencies between the GF compiler and the PGF library

2013-11-05 13:11:10 +00:00

compiler

Represent identifiers as UTF-8-encoded ByteStrings

2013-11-26 16:12:03 +00:00

example-based

Eliminate mutual dependencies between the GF compiler and the PGF library

2013-11-05 13:11:10 +00:00

runtime

Represent identifiers as UTF-8-encoded ByteStrings

2013-11-26 16:12:03 +00:00

server

linref is now used by the linearizer. The visible change is that the 'l' command in the shell now can linearize discontinuous phrases

2013-10-30 14:42:29 +00:00

tools

Created a new page documenting available editor modes for GF

2011-09-20 12:41:32 +00:00

ui

fix in the GF keyboard for Android

2013-11-22 13:37:12 +00:00

www

gfse: re-enable example-based grammar construction

2013-11-14 12:54:25 +00:00