Blocklist · OpenAI

Sites that block GPTBot

2,937 of the 29,991 domains we have measured, 10%, disallow gptbot at the site root in robots.txt. GPTBot collects content that may be used to train a model.

Every domain here has withheld its content from OpenAI's training crawl. That is a separate decision from appearing in the operator's answers, which is governed by a different token.

The measurement

2,937 of 29,991 measured domains (10%) disallow gptbot at the site root in robots.txt. Every domain named on this page was fetched directly and its robots.txt parsed against RFC 9309; none of it is inferred from a third-party index.

Measured 2026-11-22 by direct request to every domain in the corpus. Method: how this is measured. Reuse under CC BY 4.0 with attribution.

Cite as: Crawl Census, "Sites that block GPTBot", measured 2026-11-22. https://crawlcensus.com/blocked/gptbot

The list 2,937 domains

RankDomainTLDAI access score
3,286 flipboard.com .com 64
3,309 foursquare.com .com 92
3,320 unep.org .org 71
3,342 lifehacker.com .com 65
3,347 consumerreports.org .org 72
3,368 nzherald.co.nz .nz 56
3,414 nicovideo.jp .jp 66
3,418 deadline.com .com 81
3,433 thefreedictionary.com .com 71
3,437 tagesspiegel.de .de 74
3,499 amazon.se .se 40
3,509 dictionary.com .com 90
3,520 amazon.ae .ae 40
3,528 flightaware.com .com 63
3,542 orf.at .at 69
3,576 abc.es .es 81
3,587 br.de .de 86
3,595 politico.eu .eu 62
3,598 amazon.nl .nl 40
3,603 macomnet.ru .ru 63
3,608 case-battle.id .id 48
3,617 boston.com .com 87
3,633 ndr.de .de 79
3,642 yahoo.net .net 67
3,662 filmyzilla52.com .com 55
3,676 handelsblatt.com .com 82
3,686 moviesda34.com .com 62
3,700 moviezwap.land .land 52
3,711 rexify.com.ng .ng 59
3,712 filmyzilla46.com .com 55
3,715 xkcd.com .com 62
3,717 ansa.it .it 85
3,724 lonelyplanet.com .com 85
3,730 oregonstate.edu .edu 63
3,771 yle.fi .fi 85
3,777 chosun.com .com 74
3,796 ardmediathek.de .de 73
3,800 ebsco.com .com 65
3,808 toyota.com .com 60
3,869 pcworld.com .com 84
3,909 aol.de .de 65
3,916 news24.com .com 56
3,918 rte.ie .ie 83
3,952 gitea.com .com 69
3,963 nos.nl .nl 69
3,966 bostonglobe.com .com 83
3,974 aljazeera.net .net 89
4,061 screenrant.com .com 84
4,065 seekingalpha.com .com 91
4,070 ilsole24ore.com .com 78
4,079 vaticannews.va .va 44
4,083 stern.de .de 83
4,084 uiowa.edu .edu 70
4,108 howtogeek.com .com 84
4,145 udn.com .com 88
4,161 kompas.com .com 77
4,181 sbs.com.au .au 83
4,205 emojipedia.org .org 68
4,304 tvtropes.org .org 55
4,326 wiki.gg .gg 67
4,333 derstandard.at .at 56
4,334 blender.org .org 63
4,354 computerworld.com .com 83
4,407 20minutos.es .es 87
4,431 swr.de .de 76
4,469 hse.ru .ru 70
4,484 mdr.de .de 76
4,493 wdr.de .de 86
4,497 gazeta.pl .pl 74
4,512 amazon.com.tr .tr 40
4,532 biorxiv.org .org 44
4,551 yna.co.kr .kr 87
4,553 20minutes.fr .fr 71
4,579 srf.ch .ch 87
4,609 123rf.com .com 43
4,658 rg.ru .ru 68
4,684 msnbc.com .com 75
4,749 teacherspayteachers.com .com 69
4,753 xda-developers.com .com 83
4,791 collider.com .com 83
4,816 polygon.com .com 83
4,817 hurriyet.com.tr .tr 81
4,858 atlasobscura.com .com 73
4,866 channel4.com .com 56
4,927 iol.co.za .za 69
4,929 onlamp.com .com 80
4,965 vinted.co.uk .uk 91
4,987 dr.dk .dk 81
5,002 mercadolibre.cl .cl 67
5,037 imageshack.com .com 71
5,057 deccanherald.com .com 64
5,096 nine.com.au .au 71
5,146 everydayhealth.com .com 60
5,153 theage.com.au .au 75
5,158 mercurynews.com .com 78
5,205 lapresse.ca .ca 65
5,213 game8.co .co 76
5,223 oilprice.com .com 90
5,229 eltiempo.com .com 85
5,251 dallasnews.com .com 54
5,268 ora.com .com 94
5,272 depositphotos.com .com 63
5,274 csmonitor.com .com 76
5,277 bookshop.org .org 74
5,293 turbify.com .com 85
5,297 chinatimes.com .com 64
5,323 cbr.com .com 83
5,357 larazon.es .es 81
5,375 mercadolibre.com.co .co 67
5,395 makeuseof.com .com 84
5,413 expressen.se .se 61
5,455 gamerant.com .com 81
5,457 diigo.com .com 35
5,463 rainews.it .it 53
5,472 sudouest.fr .fr 89
5,503 bricklink.com .com 70
5,522 abc.com .com 82
5,547 amazon.ie .ie 40
5,609 rtl.fr .fr 87
5,646 baseball-reference.com .com 63
5,678 dn.se .se 60
5,684 theepochtimes.com .com 80
5,709 streamable.com .com 38
5,714 francetvinfo.fr .fr 53
5,726 chefkoch.de .de 81
5,748 ovh.us .us 89
5,749 autoscout24.de .de 86
5,752 sankei.com .com 76
5,769 degruyter.com .com 61
5,770 boingboing.net .net 79
5,775 tv2.no .no 70
5,786 cloudpath.live .live 44
5,834 motherjones.com .com 69
5,846 runescape.wiki .wiki 81
5,874 hs.fi .fi 68
5,876 dmitriy-mevedev-tor.top .top 53
5,879 cio.com .com 86
5,895 movieweb.com .com 84
5,904 windstream.net .net 53
5,936 qobuz.com .com 79
5,948 suntimes.com .com 81
5,954 tvn24.pl .pl 73
5,960 milliyet.com.tr .tr 69
5,993 eonline.com .com 81
6,019 vinted.de .de 67
6,020 giga.de .de 81
6,022 eiga.com .com 85
6,032 rtl.de .de 85
6,035 opaque.net .net 39
6,042 sporcle.com .com 87
6,062 nhk.jp .jp 64
6,064 delfi.lt .lt 83
6,088 futura-sciences.com .com 93
6,089 azcentral.com .com 74
6,111 ledauphine.com .com 82
6,125 denverpost.com .com 78
6,159 instiz.net .net 58
6,180 tf1info.fr .fr 84
6,245 raiplay.it .it 55
6,246 eurogamer.net .net 57
6,275 leprogres.fr .fr 82
6,301 iata.org .org 68
6,308 wwd.com .com 78
6,310 taylorfrancis.com .com 63
6,335 royalroad.com .com 48
6,340 vinted.it .it 68
6,342 vinted.pl .pl 86
6,352 tv2.dk .dk 74
6,353 bbcgoodfood.com .com 84
6,356 freep.com .com 73
6,361 ici.fr .fr 87
6,386 huffingtonpost.co.uk .uk 82
6,391 sex-studentki.live .live 56
6,421 chavesnamao.com.br .br 80
6,429 emol.com .com 70
6,446 mumsnet.com .com 79
6,464 ilpost.it .it 75
6,512 telegram.com .com 73
6,513 sandiegouniontribune.com .com 78
6,530 theintercept.com .com 86
6,535 literotica.com .com 85
6,544 rtvslo.si .si 65
6,549 midilibre.fr .fr 86
6,554 osf.io .io 35
6,557 sportingnews.com .com 82
6,576 o2.pl .pl 87
6,609 procyclingstats.com .com 56
6,610 is.fi .fi 68
6,702 infoworld.com .com 83
6,726 rtbf.be .be 67
6,737 altavista.com .com 53
6,759 degruyterbrill.com .com 61
6,793 svt.se .se 66
6,803 schwaebische.de .de 70
6,804 sportschau.de .de 85
6,808 chunichi.co.jp .jp 71
6,816 maxroll.gg .gg 59
6,817 funnyjunk.com .com 52
6,820 money.pl .pl 89
6,822 elcorreo.com .com 80
PreviousPage 2 of 15Next

Rank is the domain's position in the popularity list used to seed the corpus; a dash means it was scanned on request rather than seeded. Score is the site's overall AI access score, which blocking depresses but does not by itself determine.

Check a specific domain

See also the GPTBot reference, the full census, and the JSON endpoint.