initial robots.txt
This commit is contained in:
parent
bb93491a00
commit
9d25a7a91d
|
@ -0,0 +1,5 @@
|
|||
# www.robotstxt.org/
|
||||
|
||||
# Allow crawling of all content
|
||||
User-agent: *
|
||||
Disallow:
|
Loading…
Reference in New Issue