Blocklist · Anthropic

Sites that block anthropic-ai

2,193 of the 29,991 domains we have measured, 7%, disallow anthropic-ai at the site root in robots.txt. anthropic-ai collects content that may be used to train a model.

Every domain here has withheld its content from Anthropic's training crawl. That is a separate decision from appearing in the operator's answers, which is governed by a different token.

The measurement

2,193 of 29,991 measured domains (7%) disallow anthropic-ai at the site root in robots.txt. Every domain named on this page was fetched directly and its robots.txt parsed against RFC 9309; none of it is inferred from a third-party index.

Measured 2026-11-21 by direct request to every domain in the corpus. Method: how this is measured. Reuse under CC BY 4.0 with attribution.

Cite as: Crawl Census, "Sites that block anthropic-ai", measured 2026-11-21. https://crawlcensus.com/blocked/anthropic-ai

The list 2,193 domains

RankDomainTLDAI access score
14,474 seiska.fi .fi 79
14,491 aerzteblatt.de .de 69
14,562 nbcchicago.com .com 62
14,576 porn-sex-site.com .com 77
14,606 cnnturk.com .com 86
14,612 mopo.de .de 79
14,661 jalalive.news .news 82
14,709 dazeddigital.com .com 78
14,733 waz.de .de 82
14,774 zebzoub.com .com 58
14,790 bokepindo13.study .study 71
14,794 sexeti.com .com 59
14,809 auto-motor-und-sport.de .de 81
14,826 rockpapershotgun.com .com 58
14,833 katmoviehd.top .top 70
14,872 mz.de .de 70
14,873 rareanimes.mov .mov 72
14,878 postheaven.net .net 54
14,902 xn--3-wxfc3gwbi.net .net 77
14,903 2osb.com .com 48
14,904 kyalsin.com .com 47
14,931 cossunex.com .com 59
14,961 townnews.com .com 75
14,973 lek-manga.net .net 68
14,976 glvietsub.net .net 82
14,982 finrezo.com .com 56
15,027 desidisini.space .space 26
15,046 drbokep.top .top 71
15,056 dindadisini.space .space 26
15,061 jnedemure.com .com 69
15,064 leadsafrica.com .com 56
15,067 rapbeh.cx .cx 35
15,068 toonstream.vip .vip 56
15,076 analdin.com .com 44
15,080 adl.org .org 87
15,146 aldat.online .online 39
15,174 pcgamesn.com .com 83
15,211 zenwriting.net .net 56
15,215 sportngin.com .com 84
15,218 statesman.com .com 51
15,230 mysanantonio.com .com 54
15,260 birminghammail.co.uk .uk 47
15,269 gamesindustry.biz .biz 57
15,291 phoronix.com .com 56
15,293 abc7news.com .com 81
15,321 coursehero.com .com 76
15,329 estrepublicain.fr .fr 82
15,336 axiombid.com .com 73
15,352 aaas.org .org 65
15,366 filmix.my .my 72
15,372 letudiant.fr .fr 81
15,391 netzwelt.de .de 84
15,401 desmoinesregister.com .com 74
15,421 mtggoldfish.com .com 63
15,458 utopia.de .de 85
15,467 betsapi.com .com 59
15,486 angkanet22.com .com 63
15,488 nagawinplus.com .com 69
15,490 vidleey.sbs .sbs 43
15,491 wetheleader.org .org 65
15,493 churchcenter.com .com 58
15,508 ppy.sh .sh 59
15,511 chocolatey.org .org 61
15,557 cincinnati.com .com 74
15,570 finna.fi .fi 54
15,571 recipetineats.com .com 82
15,607 townandcountrymag.com .com 76
15,626 moviesmod.army .army 35
15,650 asd.ink .ink 79
15,668 azoramoon.com .com 73
15,671 narto-drama.com .com 41
15,679 opendota.com .com 42
15,697 xn--3-wxfc3gwbi.com .com 77
15,700 1x1x5.com .com 55
15,710 yourstory.com .com 55
15,723 disboard.org .org 52
15,743 alexistogel.link .link 66
15,744 alexistogel.mobi .mobi 74
15,745 alexistogel.news .news 76
15,746 alexistogel.red .red 76
15,747 alexistogel.team .team 78
15,748 alexistogel.tips .tips 76
15,752 kuttymovies1.fast .fast 54
15,760 vildey.space .space 43
15,782 gamesview.shop .shop 70
15,804 moviesda.bet .bet 56
15,806 tamilprint61.art .art 55
15,830 xn--3-7wf9a3c4b3bt8f0cya.net .net 77
15,834 bokepindo13.party .party 71
15,853 1tamilmv.cards .cards 74
15,856 aagmaal.bet .bet 72
15,857 aghazno11.com .com 80
15,858 alexistogel.lol .lol 73
15,867 holoo11.com .com 84
15,868 holoo12.com .com 84
15,873 mawartotogoku.com .com 74
15,879 mkvmoviespoint.cab .cab 72
15,882 moviesda.talk .talk 56
15,885 olxcun.com .com 75
15,888 pinayflix.app .app 51
15,890 prmovies.farm .farm 70
15,898 skymovieshd.spot .spot 53
15,900 tamilprint60.art .art 55
15,909 toto12ultra.com .com 75
15,910 toto12wind.com .com 77
15,918 yallashots.top .top 72
15,944 rtl.be .be 82
16,015 blogfreely.net .net 34
16,036 palletsprojects.com .com 59
16,047 longisland.com .com 51
16,058 globalsecurity.org .org 58
16,082 t3n.de .de 78
16,103 dweb.link .link 65
16,118 healio.com .com 69
16,164 christianitytoday.com .com 89
16,209 ipfs.io .io 65
16,223 smashwords.com .com 58
16,251 nysed.gov .gov 69
16,256 runnerspace.com .com 47
16,283 positivepsychology.com .com 76
16,290 duden.de .de 71
16,317 ilmattino.it .it 62
16,365 xozilla.com .com 58
16,377 decider.com .com 79
16,423 iwm.org.uk .uk 77
16,428 e24.no .no 78
16,444 hometalk.com .com 63
16,473 skilljar.com .com 76
16,478 wnba.com .com 69
16,520 hotukdeals.com .com 78
16,564 mapgenie.io .io 59
16,607 mediapost.com .com 45
16,613 goodmorningamerica.com .com 70
16,664 lightningmaps.org .org 51
16,672 pressherald.com .com 76
16,765 scimagojr.com .com 57
16,783 taipeitimes.com .com 66
16,795 vozpopuli.com .com 87
16,796 postandcourier.com .com 77
16,803 buffalonews.com .com 80
16,814 whattoexpect.com .com 65
16,832 nltimes.nl .nl 73
16,846 porofessor.gg .gg 55
16,897 nbcbayarea.com .com 62
16,972 nbcwashington.com .com 60
16,991 u.gg .gg 42
17,039 robbreport.com .com 64
17,042 northjersey.com .com 74
17,055 official.link .link 72
17,183 semafor.com .com 72
17,189 heavyfetish.com .com 62
17,224 harpers.org .org 79
17,243 komonews.com .com 73
17,255 thegamer.com .com 83
17,266 letemps.ch .ch 74
17,285 nextech.com .com 78
17,303 businessoffashion.com .com 68
17,345 rollcall.com .com 86
17,404 poipiku.com .com 71
17,420 jsbin.com .com 53
17,432 kyoto-np.co.jp .jp 82
17,455 twincities.com .com 81
17,469 palmbeachpost.com .com 73
17,483 diariodenavarra.es .es 89
17,489 dukeupress.edu .edu 73
17,492 ina.fr .fr 76
17,522 daangn.com .com 71
17,536 seattlepi.com .com 54
17,554 abc13.com .com 79
17,596 iisd.org .org 59
17,607 philpapers.org .org 65
17,619 nst.com.my .my 42
17,634 dice.fm .fm 65
17,671 iafd.com .com 45
17,673 demilked.com .com 64
17,694 manager-magazin.de .de 63
17,721 cnn.co.jp .jp 65
17,750 6abc.com .com 79
17,785 trakt.tv .tv 58
17,796 list.ly .ly 52
17,833 novelphoenix.com .com 61
17,877 browser-update.org .org 57
17,886 agrarheute.com .com 75
17,914 jofogas.hu .hu 80
17,988 timeslive.co.za .za 80
17,993 cawsg.com .com 55
18,147 volksstimme.de .de 68
18,153 vlex.com .com 88
18,214 xxxbunker.com .com 81
18,218 globed.dev .dev 51
18,228 metafilter.com .com 51
18,266 thedirecthor.com .com 57
18,280 hoerzu.de .de 45
18,308 preyproject.com .com 93
18,309 mans.io .io 60
18,311 earthsky.org .org 85
18,338 telemundo.com .com 83
18,373 instyle.de .de 89
18,392 mawaqit.net .net 91
18,442 empireonline.com .com 62
PreviousPage 4 of 11Next

Rank is the domain's position in the popularity list used to seed the corpus; a dash means it was scanned on request rather than seeded. Score is the site's overall AI access score, which blocking depresses but does not by itself determine.

Check a specific domain

See also the anthropic-ai reference, the full census, and the JSON endpoint.