Blocklist · OpenAI

Sites that block OAI-AdsBot

13 of the 29,991 domains we have measured, 0.0%, disallow oai-adsbot at the site root in robots.txt. OAI-AdsBot fetches a page live when a user asks the assistant for it.

Every domain here has removed itself from OpenAI's answers. Blocking a retrieval agent withholds nothing from model training, because training uses a different token.

The measurement

13 of 29,991 measured domains (0.0%) disallow oai-adsbot at the site root in robots.txt. Every domain named on this page was fetched directly and its robots.txt parsed against RFC 9309; none of it is inferred from a third-party index.

Measured 2026-11-21 by direct request to every domain in the corpus. Method: how this is measured. Reuse under CC BY 4.0 with attribution.

Cite as: Crawl Census, "Sites that block OAI-AdsBot", measured 2026-11-21. https://crawlcensus.com/blocked/oai-adsbot

The list 13 domains

RankDomainTLDAI access score
4,161 kompas.com .com 77
6,948 nicematin.com .com 77
13,871 ilex.by .by 59
25,911 arkansasonline.com .com 71
29,390 kompas.id .id 64
31,320 lumendatabase.org .org 51
31,499 bolasport.com .com 76
39,824 werstreamt.es .es 59
40,339 kompas.tv .tv 76
40,874 grid.id .id 64
44,807 gofeminin.de .de 74
48,942 ttnews.com .com 81
50,933 nwaonline.com .com 73

Rank is the domain's position in the popularity list used to seed the corpus; a dash means it was scanned on request rather than seeded. Score is the site's overall AI access score, which blocking depresses but does not by itself determine.

Check a specific domain

See also the OAI-AdsBot reference, the full census, and the JSON endpoint.