initial robots.txt
This commit is contained in:
parent
bb93491a00
commit
9d25a7a91d
|
@ -0,0 +1,5 @@
|
||||||
|
# www.robotstxt.org/
|
||||||
|
|
||||||
|
# Allow crawling of all content
|
||||||
|
User-agent: *
|
||||||
|
Disallow:
|
Loading…
Reference in New Issue