Skip to content
Orino Docs

GPTBot blocked in robots.txt (HTML project)

What this means

Section titled “What this means”

Orino found Disallow: / under User-agent: GPTBot in your project’s robots.txt file. This block prevents OpenAI’s crawler from indexing your content once the site is deployed.

This check runs during disk-only audits — when Orino scans a plain HTML project without fetching a live URL. It reads the robots.txt file directly from your project directory.

How to fix it

Section titled “How to fix it”

Open robots.txt in your project root and remove the GPTBot block:

# Remove these lines
User-agent: GPTBot
Disallow: /

To allow GPTBot site-wide, delete both lines. To restrict specific paths only:

User-agent: GPTBot
Disallow: /private/

Verify the fix

Section titled “Verify the fix”

Check the file in your project:

Terminal window
grep -A 3 "GPTBot" robots.txt

No output means GPTBot has no specific rule and falls back to your wildcard rules. Disallow: / means the block is still active. Re-run orino audit to confirm the check passes.

Section titled “Related fixes”