Skip to content
Orino Docs

ClaudeBot blocked in robots.txt

What this means

Section titled “What this means”

Your robots.txt has Disallow: / under User-agent: ClaudeBot. ClaudeBot is Anthropic’s web crawler. Blocking it prevents Anthropic from indexing your content, which reduces the likelihood of your site being surfaced in Claude’s responses.

How to fix it

Section titled “How to fix it”

Remove the ClaudeBot block from your robots.txt:

# Remove these lines
User-agent: ClaudeBot
Disallow: /

Delete those two lines to allow ClaudeBot site-wide. To exclude specific paths:

User-agent: ClaudeBot
Disallow: /private/

Verify the fix

Section titled “Verify the fix”

After deploying:

Terminal window
curl -s https://yourdomain.com/robots.txt | grep -A 3 "ClaudeBot"

No result means ClaudeBot falls back to your wildcard rules. Disallow: / means the block is still active. Re-run orino audit to confirm the check passes.

Section titled “Related fixes”