robots.txt missing in /public (Pages Router)
What this means
Section titled “What this means”Pages Router has no Metadata API for robots. The only way to serve /robots.txt is through the /public directory. Without it, crawlers have no guidance and may index API routes, admin paths, or staging content.
How to fix it
Section titled “How to fix it”Create public/robots.txt.
User-agent: *Allow: /
Sitemap: https://example.com/sitemap.xmlReplace https://example.com with your actual domain. Add Disallow rules for any paths that should not be indexed.
Verify the fix
Section titled “Verify the fix”curl -I https://yourdomain.com/robots.txtConfirm the response is 200 OK. Then re-run orino audit.