Four crawlers, one that decides
OpenAI publishes what each of its crawlers does. GPTBot collects content that may train future models. OAI-SearchBot, in OpenAI's words, is used to surface websites in search results in ChatGPT's search features. ChatGPT-User fetches a single page when someone asks ChatGPT to open it. OAI-AdsBot checks pages submitted as ads.
Only one of those four gates your citations, and OpenAI says so directly: sites that are opted out of OAI-SearchBot will not be shown in ChatGPT search answers. Everything else on this page follows from that one sentence.
Blocking GPTBot does not hide you from ChatGPT
This is the mistake we see most often, and it usually starts with good intentions. Someone reads an article about keeping their writing out of AI training, copies a block of robots.txt rules into their site, and the block bans anything with an AI-sounding name. GPTBot and OAI-SearchBot go out together.
Those two choices have nothing to do with each other. Refusing to feed model training is a legitimate call about your own work, and it costs you no visibility today. Refusing OAI-SearchBot removes you from the answers people read, which is the traffic you were hoping for in the first place. You can block the first and welcome the second.
Being readable is the ticket, not the prize
Once ChatGPT can read you, you are competing with every other page it found. Most pages that get cited in AI answers already rank in ordinary search, so the classic work still carries you: a clear title, a page that loads its content in HTML rather than assembling it with JavaScript, and a topic you actually cover in depth.
The last step is where most sites lose it. ChatGPT lifts short passages, not whole pages. If your answer sits in paragraph nine, after the story of how the company was founded, the model reads the top of the page, finds no answer, and quotes the competitor who put theirs in the first two sentences.
Check yours in two minutes
You do not need a tool for the first look. Type your domain followed by /robots.txt into your browser and read what comes back.
If nothing loads, you have no robots.txt, and that means nothing is blocked. That is a fine place to be. If a file does load, three things are worth finding:
- OAI-SearchBot with a Disallow line under it. That is the one closing the door on ChatGPT.
- PerplexityBot, same story, for the other big answer engine that cites live sources.
- User-agent: * followed by Disallow: / which blocks every crawler on the web, Google included. That is rarely intentional on a live site.
What to do if you find a block
Removing the line is a small edit, and in most website builders it is a settings field rather than a file you upload. The bigger job is the one underneath: making sure that when the crawler comes back, there is a page worth quoting waiting for it.
HowFindable scans your site for free and tells you which AI crawlers your robots.txt is turning away, along with the rest of the signals ChatGPT and Google read. If you want the corrected files, we generate them ready to upload, with steps for your platform.