Files
Grav/robots.txt
Ante Drnasin 9eded2ef39 Update robots.txt (#2043)
When using "Fetch and Render as Google" in Google Search console it will report "partial" rendering due to the blocked images in /user/images directory which is blocked because of the Disallow /user/ rule. Proposing this small change as it improves google rendering of the page.
2018-05-30 11:47:21 -06:00

13 lines
218 B
Plaintext

User-agent: *
Disallow: /backup/
Disallow: /bin/
Disallow: /cache/
Disallow: /grav/
Disallow: /logs/
Disallow: /system/
Disallow: /vendor/
Disallow: /user/
Allow: /user/pages/
Allow: /user/themes/
Allow: /user/images/