Blocklist · OpenAI

Sites that block GPTBot

2,937 of the 29,991 domains we have measured, 10%, disallow gptbot at the site root in robots.txt. GPTBot collects content that may be used to train a model.

Every domain here has withheld its content from OpenAI's training crawl. That is a separate decision from appearing in the operator's answers, which is governed by a different token.

The measurement

2,937 of 29,991 measured domains (10%) disallow gptbot at the site root in robots.txt. Every domain named on this page was fetched directly and its robots.txt parsed against RFC 9309; none of it is inferred from a third-party index.

Measured 2026-11-22 by direct request to every domain in the corpus. Method: how this is measured. Reuse under CC BY 4.0 with attribution.

Cite as: Crawl Census, "Sites that block GPTBot", measured 2026-11-22. https://crawlcensus.com/blocked/gptbot

The list 2,937 domains

RankDomainTLDAI access score
26 amazon.com .com 40
54 tiktok.com .com 49
58 yahoo.com .com 67
62 msn.com .com 41
155 gravatar.com .com 75
190 cnn.com .com 50
207 bbc.com .com 76
217 t-online.de .de 84
226 bbc.co.uk .uk 77
260 launchpad.net .net 52
281 nature.com .com 56
301 alibaba.com .com 81
318 amazon.co.uk .uk 40
377 amazon.de .de 40
399 aliexpress.com .com 81
412 globo.com .com 71
432 espn.com .com 48
448 goodreads.com .com 75
450 go.com .com 76
457 cnbc.com .com 54
494 amazon.co.jp .jp 40
499 aol.com .com 66
535 cookielaw.org .org 75
556 alrosa.ru .ru 68
609 amazon.fr .fr 40
617 unesco.org .org 90
623 amazon.co.za .za 40
627 dailymail.co.uk .uk 61
628 nintendo.com .com 61
629 techcrunch.com .com 86
635 amazon.ca .ca 40
656 cbsnews.com .com 88
659 amazon.in .in 40
662 cnet.com .com 79
670 wattpad.com .com 70
681 dailymail.com .com 61
699 globalsign.com .com 90
706 amazon.it .it 40
710 bandcamp.com .com 53
725 amazon.es .es 40
726 ted.com .com 70
740 mercadolivre.com.br .br 67
749 airbnb.com .com 36
779 wp.pl .pl 72
812 amazon.com.au .au 40
820 hbr.org .org 71
823 nbcnews.com .com 59
833 amazon.com.br .br 40
834 lefigaro.fr .fr 71
869 figma.com .com 86
882 latimes.com .com 83
926 genius.com .com 65
931 mercadolibre.com.ar .ar 67
932 webmd.com .com 73
941 healthline.com .com 80
942 repubblica.it .it 85
955 corriere.it .it 63
983 elmundo.es .es 84
1,017 pbs.org .org 86
1,019 theconversation.com .com 62
1,027 huffpost.com .com 85
1,032 amazon.com.mx .mx 40
1,042 spiegel.de .de 64
1,046 bidmachine.io .io 81
1,054 n-tv.de .de 88
1,056 marca.com .com 85
1,073 bfmtv.com .com 81
1,098 franceinfo.fr .fr 60
1,104 idnes.cz .cz 80
1,105 cookpad.com .com 73
1,113 skysports.com .com 88
1,125 mercadolibre.com.mx .mx 67
1,165 mercadolibre.com .com 40
1,177 pimpbunny.com .com 56
1,202 rightmove.co.uk .uk 71
1,209 dcinside.com .com 77
1,223 furaffinity.net .net 39
1,231 gazzetta.it .it 80
1,232 pandora.com .com 53
1,258 abc.net.au .au 87
1,259 jstor.org .org 58
1,262 mundodeportivo.com .com 82
1,272 vietnam.vn .vn 74
1,273 ladepeche.fr .fr 86
1,275 huffingtonpost.com .com 85
1,286 actu.fr .fr 81
1,300 f95zone.to .to 59
1,305 dw.com .com 42
1,355 iqiyi.com .com 61
1,371 sex-studentki.team .team 57
1,375 nikkansports.com .com 73
1,376 dagbladet.no .no 74
1,378 aljazeera.com .com 89
1,381 ilmeteo.it .it 87
1,394 usnews.com .com 83
1,399 zedge.net .net 75
1,409 iltalehti.fi .fi 58
1,426 crn77.com .com 51
1,430 omg10.com .com 51
1,439 toyhou.se .se 52
1,442 buzzfeed.com .com 65
1,446 ekstrabladet.dk .dk 67
1,456 eenadu.net .net 67
1,484 dogdrip.net .net 52
1,501 xnxx.health .health 58
1,505 bblaa.com .com 51
1,507 bt.dk .dk 84
1,514 255md.com .com 51
1,520 shahvani.com .com 45
1,536 theqoo.net .net 61
1,539 bef77.com .com 51
1,559 nya2.com .com 51
1,568 imgsrc.ru .ru 67
1,590 ey43.com .com 51
1,602 colliderporn.com .com 43
1,613 arabx.cam .cam 66
1,634 sex-studentki.cafe .cafe 56
1,639 darkero.com .com 67
1,677 manhwaweb.com .com 42
1,697 pornoborshch.com .com 51
1,723 lectormangass.net .net 77
1,724 xxxhindi.to .to 78
1,729 loteriadehoy.com .com 56
1,740 xn--12c1ezaww.com .com 74
1,746 mobcup.fm .fm 64
1,747 moviesda33.com .com 62
1,757 sattamatkano1.me .me 87
1,762 filmyzilla45.com .com 62
1,831 zdnet.com .com 76
1,835 wikihow.com .com 76
1,856 scientificamerican.com .com 82
1,901 ovh.com .com 89
1,933 meetup.com .com 94
1,936 euronews.com .com 89
1,984 themoviedb.org .org 74
1,989 strava.com .com 85
1,993 ign.com .com 62
2,005 flightradar24.com .com 50
2,006 amazon.eg .eg 40
2,010 mashable.com .com 61
2,012 vinted.fr .fr 68
2,085 medicalnewstoday.com .com 81
2,115 mirror.co.uk .uk 47
2,118 variety.com .com 80
2,203 ixl.com .com 69
2,207 imhentai.xxx .xxx 51
2,230 meb.gov.tr .tr 66
2,319 tukrd.com .com 47
2,324 abcnews.com .com 84
2,325 nrk.no .no 69
2,346 slate.com .com 80
2,348 nontondrama.my .my 77
2,370 umd.edu .edu 88
2,374 xvideosputaria.com .com 74
2,423 pcmag.com .com 63
2,433 express.co.uk .uk 48
2,447 sendbird.com .com 89
2,472 rockstargames.com .com 36
2,481 disney.com .com 72
2,508 yr.no .no 64
2,530 smh.com.au .au 82
2,541 mingky04.live .live 52
2,561 technologyreview.com .com 82
2,581 hollywoodreporter.com .com 79
2,607 vildey.cfd .cfd 38
2,637 filmyfly.builders .builders 64
2,638 filmyfly.racing .racing 64
2,639 filmyzilla40.com .com 62
2,640 filmyzilla43.com .com 62
2,641 moviesda32.com .com 62
2,646 moviesda31.com .com 62
2,647 moviezwap.onl .onl 52
2,691 newscientist.com .com 76
2,723 yumpu.com .com 30
2,751 hai8g.com .com 51
2,772 ko-fi.com .com 53
2,786 chicagotribune.com .com 79
2,838 mastodon.social .social 49
2,880 sueddeutsche.de .de 71
2,917 faz.net .net 39
2,969 today.com .com 62
2,971 prokerala.com .com 81
2,974 thehindu.com .com 66
2,975 imageshack.us .us 71
3,008 bgtee.com .com 51
3,015 irishtimes.com .com 86
3,043 shaalaa.com .com 65
3,054 intelbras.com.br .br 52
3,096 ydparty04.tv .tv 86
3,109 ctvnews.ca .ca 82
3,165 vinted.com .com 68
3,179 crunchbase.com .com 71
3,180 ay267.com .com 51
3,198 metro.co.uk .uk 59
3,211 cbssports.com .com 84
3,213 fool.com .com 96
3,222 loom.com .com 89
3,224 tagesschau.de .de 80
3,253 billboard.com .com 78
3,276 myanimelist.net .net 70
Page 1 of 15Next

Rank is the domain's position in the popularity list used to seed the corpus; a dash means it was scanned on request rather than seeded. Score is the site's overall AI access score, which blocking depresses but does not by itself determine.

Check a specific domain

See also the GPTBot reference, the full census, and the JSON endpoint.