GitHub/gf-core
1
0
Fork 1
mirror of https://github.com/GrammaticalFramework/gf-core.git synced 2026-04-17 08:49:31 -06:00
Code Issues Packages Projects Releases Wiki Activity

src/www/robots.txt: robot exclusion file for the GF cloud service.

Browse Source 
To reduce potential server load caused by search bots.
This commit is contained in:
hallgren
2014-08-22 12:13:04 +00:00
parent 8dfaf2ef65
commit 6bde56bef7
1 changed files with 4 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
src/www/robots.txt Normal file
View File

@@ -0,0 +1,4 @@
User-agent: *
Disallow: /grammars
Disallow: /robust
Disallow: /*.pgf