Blocklist · Meta

Sites that block Meta-ExternalAgent

2,340 of the 29,991 domains we have measured, 8%, disallow meta-externalagent at the site root in robots.txt. Meta-ExternalAgent collects content that may be used to train a model.

Every domain here has withheld its content from Meta's training crawl. That is a separate decision from appearing in the operator's answers, which is governed by a different token.

The measurement

2,340 of 29,991 measured domains (8%) disallow meta-externalagent at the site root in robots.txt. Every domain named on this page was fetched directly and its robots.txt parsed against RFC 9309; none of it is inferred from a third-party index.

Measured 2026-11-22 by direct request to every domain in the corpus. Method: how this is measured. Reuse under CC BY 4.0 with attribution.

Cite as: Crawl Census, "Sites that block Meta-ExternalAgent", measured 2026-11-22. https://crawlcensus.com/blocked/meta-externalagent

The list 2,340 domains

RankDomainTLDAI access score
13,355 marieclaire.com .com 85
13,380 flourish.studio .studio 92
13,381 bt4g.com .com 47
13,398 gq-magazine.co.uk .uk 81
13,419 vancouversun.com .com 71
13,451 skyscrapercity.com .com 43
13,532 doteasy.com .com 73
13,569 blitz.gg .gg 64
13,617 latribune.fr .fr 72
13,660 thrillist.com .com 70
13,667 dispatch.com .com 74
13,668 torontosun.com .com 71
13,696 blackfaggot.online .online 42
13,719 payoneer.com .com 79
13,772 khan.co.kr .kr 72
13,774 foodnetwork.co.uk .uk 87
13,783 creativebloq.com .com 87
13,792 sheknows.com .com 83
13,825 heraldscotland.com .com 81
13,840 writeablog.net .net 51
13,860 francebleu.fr .fr 87
13,871 ilex.by .by 59
13,873 rappler.com .com 80
13,879 tennessean.com .com 74
13,930 myminifactory.com .com 38
13,932 bokepi.com .com 75
13,937 songkick.com .com 66
13,942 dnevnik.hr .hr 55
13,987 mydealz.de .de 79
14,009 abc7chicago.com .com 81
14,034 lrt.lt .lt 84
14,053 androidcentral.com .com 83
14,065 animesuge.cz .cz 71
14,079 wccftech.com .com 80
14,138 mapy.cz .cz 52
14,153 ikiru.wtf .wtf 67
14,163 car-part.com .com 49
14,168 ncaa.com .com 85
14,337 lafeltrinelli.it .it 65
14,350 abc7ny.com .com 81
14,374 delfi.lv .lv 80
14,404 cumhuriyet.com.tr .tr 84
14,468 mediapart.fr .fr 84
14,470 gamedeveloper.com .com 83
14,474 seiska.fi .fi 79
14,491 aerzteblatt.de .de 69
14,576 porn-sex-site.com .com 77
14,612 mopo.de .de 79
14,661 jalalive.news .news 82
14,709 dazeddigital.com .com 78
14,733 waz.de .de 82
14,774 zebzoub.com .com 58
14,790 bokepindo13.study .study 71
14,794 sexeti.com .com 59
14,809 auto-motor-und-sport.de .de 81
14,826 rockpapershotgun.com .com 58
14,828 autotrader.ca .ca 81
14,833 katmoviehd.top .top 70
14,862 canada.com .com 71
14,872 mz.de .de 70
14,873 rareanimes.mov .mov 72
14,878 postheaven.net .net 54
14,902 xn--3-wxfc3gwbi.net .net 77
14,903 2osb.com .com 48
14,904 kyalsin.com .com 47
14,931 cossunex.com .com 59
14,961 townnews.com .com 75
14,973 lek-manga.net .net 68
14,976 glvietsub.net .net 82
14,982 finrezo.com .com 56
14,998 kino.de .de 84
15,027 desidisini.space .space 26
15,046 drbokep.top .top 71
15,056 dindadisini.space .space 26
15,061 jnedemure.com .com 69
15,064 leadsafrica.com .com 56
15,067 rapbeh.cx .cx 35
15,068 toonstream.vip .vip 56
15,076 analdin.com .com 44
15,080 adl.org .org 87
15,084 bestprice.gr .gr 97
15,146 aldat.online .online 39
15,149 mochahost.com .com 72
15,159 skyscanner.com .com 52
15,174 pcgamesn.com .com 83
15,211 zenwriting.net .net 56
15,215 sportngin.com .com 84
15,218 statesman.com .com 51
15,233 thieme-connect.com .com 39
15,260 birminghammail.co.uk .uk 47
15,269 gamesindustry.biz .biz 57
15,291 phoronix.com .com 56
15,293 abc7news.com .com 81
15,297 familie.de .de 84
15,311 mcmaster.com .com 69
15,321 coursehero.com .com 76
15,336 axiombid.com .com 73
15,352 aaas.org .org 65
15,366 filmix.my .my 72
15,372 letudiant.fr .fr 81
15,391 netzwelt.de .de 84
15,401 desmoinesregister.com .com 74
15,421 mtggoldfish.com .com 63
15,458 utopia.de .de 85
15,467 betsapi.com .com 59
15,486 angkanet22.com .com 63
15,488 nagawinplus.com .com 69
15,490 vidleey.sbs .sbs 43
15,491 wetheleader.org .org 65
15,493 churchcenter.com .com 58
15,508 ppy.sh .sh 59
15,509 diariovasco.com .com 82
15,511 chocolatey.org .org 61
15,557 cincinnati.com .com 74
15,570 finna.fi .fi 54
15,626 moviesmod.army .army 35
15,650 asd.ink .ink 79
15,668 azoramoon.com .com 73
15,671 narto-drama.com .com 41
15,679 opendota.com .com 42
15,686 postimees.ee .ee 66
15,697 xn--3-wxfc3gwbi.com .com 77
15,700 1x1x5.com .com 55
15,710 yourstory.com .com 55
15,723 disboard.org .org 52
15,743 alexistogel.link .link 66
15,744 alexistogel.mobi .mobi 74
15,745 alexistogel.news .news 76
15,746 alexistogel.red .red 76
15,747 alexistogel.team .team 78
15,748 alexistogel.tips .tips 76
15,752 kuttymovies1.fast .fast 54
15,760 vildey.space .space 43
15,782 gamesview.shop .shop 70
15,804 moviesda.bet .bet 56
15,806 tamilprint61.art .art 55
15,830 xn--3-7wf9a3c4b3bt8f0cya.net .net 77
15,834 bokepindo13.party .party 71
15,853 1tamilmv.cards .cards 74
15,856 aagmaal.bet .bet 72
15,857 aghazno11.com .com 80
15,858 alexistogel.lol .lol 73
15,867 holoo11.com .com 84
15,868 holoo12.com .com 84
15,873 mawartotogoku.com .com 74
15,878 mingky.live .live 52
15,879 mkvmoviespoint.cab .cab 72
15,882 moviesda.talk .talk 56
15,885 olxcun.com .com 75
15,888 pinayflix.app .app 51
15,890 prmovies.farm .farm 70
15,898 skymovieshd.spot .spot 53
15,900 tamilprint60.art .art 55
15,909 toto12ultra.com .com 75
15,910 toto12wind.com .com 77
15,918 yallashots.top .top 72
15,919 ydparty.tv .tv 69
15,944 rtl.be .be 82
16,013 bunte.de .de 87
16,015 blogfreely.net .net 34
16,036 palletsprojects.com .com 59
16,047 longisland.com .com 51
16,058 globalsecurity.org .org 58
16,082 t3n.de .de 78
16,086 nintendo.co.jp .jp 50
16,103 dweb.link .link 65
16,118 healio.com .com 69
16,134 homesandgardens.com .com 84
16,182 sermonaudio.com .com 71
16,209 ipfs.io .io 65
16,223 smashwords.com .com 58
16,254 unil.ch .ch 83
16,256 runnerspace.com .com 47
16,283 positivepsychology.com .com 76
16,290 duden.de .de 71
16,323 ard.de .de 76
16,365 xozilla.com .com 58
16,377 decider.com .com 79
16,385 spokesman.com .com 75
16,423 iwm.org.uk .uk 77
16,428 e24.no .no 78
16,444 hometalk.com .com 63
16,520 hotukdeals.com .com 78
16,564 mapgenie.io .io 59
16,607 mediapost.com .com 45
16,664 lightningmaps.org .org 51
16,672 pressherald.com .com 76
16,765 scimagojr.com .com 57
16,795 vozpopuli.com .com 87
16,796 postandcourier.com .com 77
16,803 buffalonews.com .com 80
16,814 whattoexpect.com .com 65
16,991 u.gg .gg 42
16,994 treccani.it .it 69
17,039 robbreport.com .com 64
17,042 northjersey.com .com 74
17,055 official.link .link 72
17,183 semafor.com .com 72
17,189 heavyfetish.com .com 62
17,224 harpers.org .org 79
PreviousPage 4 of 12Next

Rank is the domain's position in the popularity list used to seed the corpus; a dash means it was scanned on request rather than seeded. Score is the site's overall AI access score, which blocking depresses but does not by itself determine.

Check a specific domain

See also the Meta-ExternalAgent reference, the full census, and the JSON endpoint.