Blocklist · OpenAI
Sites that block OAI-SearchBot
483 of the 20,803 domains we have measured, 2%, disallow oai-searchbot at the site root in robots.txt. OAI-SearchBot builds the index that decides whether the operator's assistant cites a page.
Every domain here has removed itself from OpenAI's answers. Blocking a retrieval agent withholds nothing from model training, because training uses a different token.
The list 483 domains
Page 1 of 3Next
Rank is the domain's position in the popularity list used to seed the corpus; a dash means it was scanned on request rather than seeded. Score is the site's overall AI access score, which blocking depresses but does not by itself determine.
Check a specific domain
See also the OAI-SearchBot reference, the full census, and the JSON endpoint.