Skip to content
Orino Docs

robots.txt missing

What this means

Section titled “What this means”

No robots.txt at the site root. Crawlers have no guidance on what to index and may crawl draft files, error pages, or internal directories that were never meant to be public.

How to fix it

Section titled “How to fix it”

Create robots.txt in your root directory, at the same level as index.html.

User-agent: *
Allow: /


Sitemap: https://example.com/sitemap.xml

Replace https://example.com with your actual domain. If you have paths that should not be indexed, add a Disallow rule for each one.

User-agent: *
Allow: /
Disallow: /drafts/
Disallow: /admin/


Sitemap: https://example.com/sitemap.xml

Verify the fix

Section titled “Verify the fix”
Terminal window
curl -I https://yourdomain.com/robots.txt

Confirm the response is 200 OK. Then re-run orino audit.

Section titled “Related fixes”