initial robots.txt

This commit is contained in:
rn54n 2020-11-27 15:54:22 +01:00 committed by GitHub
parent bb93491a00
commit 9d25a7a91d
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 5 additions and 0 deletions

5
src/robots.txxt Normal file
View File

@ -0,0 +1,5 @@
# www.robotstxt.org/
# Allow crawling of all content
User-agent: *
Disallow: