Blocklist · Meta

Sites that block Meta-ExternalAgent

2,340 of the 29,991 domains we have measured, 8%, disallow meta-externalagent at the site root in robots.txt. Meta-ExternalAgent collects content that may be used to train a model.

Every domain here has withheld its content from Meta's training crawl. That is a separate decision from appearing in the operator's answers, which is governed by a different token.

The measurement

2,340 of 29,991 measured domains (8%) disallow meta-externalagent at the site root in robots.txt. Every domain named on this page was fetched directly and its robots.txt parsed against RFC 9309; none of it is inferred from a third-party index.

Measured 2026-11-22 by direct request to every domain in the corpus. Method: how this is measured. Reuse under CC BY 4.0 with attribution.

Cite as: Crawl Census, "Sites that block Meta-ExternalAgent", measured 2026-11-22. https://crawlcensus.com/blocked/meta-externalagent

The list 2,340 domains

RankDomainTLDAI access score
25,576 giraffic.com .com 75
25,587 sfexaminer.com .com 69
25,602 behindthename.com .com 52
25,606 meteotrend.com .com 49
25,625 americanthinker.com .com 56
25,632 commonwealthfund.org .org 63
25,662 experiment.com .com 60
25,748 booru.org .org 54
25,776 zinio.com .com 76
25,796 shopltk.com .com 83
25,797 wuytl.com .com 51
25,839 delawareonline.com .com 74
25,855 deutsche-rentenversicherung.de .de 79
25,903 heraldtribune.com .com 73
25,911 arkansasonline.com .com 71
25,993 vivodemisrentas.net .net 57
26,005 people.inc .inc 64
26,108 kebirhost.com .com 79
26,114 democratandchronicle.com .com 73
26,155 attendanceondemand.com .com 60
26,174 discoverhongkong.com .com 36
26,178 atgtickets.com .com 69
26,210 kstp.com .com 83
26,240 yogajournal.com .com 77
26,249 guitarworld.com .com 84
26,261 indiaforums.com .com 59
26,284 daytondailynews.com .com 67
26,322 gimkit.com .com 42
26,342 straightdope.com .com 76
26,364 framagit.org .org 69
26,379 brisbanetimes.com.au .au 75
26,400 alternatehistory.com .com 62
26,408 xozilla.xxx .xxx 58
26,433 on.cc .cc 78
26,444 pap.pl .pl 39
26,474 mbta.com .com 64
26,505 astralane.io .io 83
26,558 tadex.id .id 63
26,567 spox.com .com 81
26,637 brigitte.de .de 79
26,672 subscan.io .io 53
26,755 interfax.com.ua .ua 75
26,799 anobii.com .com 53
26,806 golf.com .com 88
26,842 unherd.com .com 83
26,843 additudemag.com .com 93
26,845 spin.com .com 70
26,853 tallahassee.com .com 74
26,935 fd.nl .nl 73
26,949 vocus.cc .cc 78
27,001 mymrbet.com .com 75
27,047 thelocal.de .de 87
27,078 mdzol.com .com 61
27,105 unp.ac.id .id 76
27,108 marketingprofs.com .com 54
27,113 in3x.net .net 38
27,128 metapress.com .com 85
27,182 ultimahora.es .es 83
27,214 backloggd.com .com 70
27,243 themirror.com .com 47
27,248 oreillynet.com .com 94
27,256 amny.com .com 83
27,275 unenvironment.org .org 68
27,306 stylecaster.com .com 77
27,315 fourfourtwo.com .com 84
27,344 ga.de .de 80
27,357 squareblogs.net .net 57
27,361 westfalen-blatt.de .de 81
27,401 derstandard.de .de 56
27,425 carbuzz.com .com 85
27,467 wfmz.com .com 81
27,551 latostadora.com .com 65
27,608 cortexbid.com .com 72
27,657 geizhals.de .de 63
27,660 pokelike.xyz .xyz 65
27,696 staradvertiser.com .com 84
27,723 geek.com .com 38
27,756 beatstars.net .net 51
27,796 asiasociety.org .org 67
27,833 apotheken-umschau.de .de 45
27,838 mtvuutiset.fi .fi 72
27,873 royallepage.ca .ca 75
27,881 cas.org .org 89
27,906 usip.org .org 70
28,003 footballant.com .com 75
28,021 sr.ht .ht 55
28,022 direct-book.com .com 45
28,038 topspeed.com .com 81
28,047 biodiversitylibrary.org .org 50
28,074 wvdme.com .com 49
28,079 apsense.com .com 52
28,099 explained.today .today 42
28,165 cdromance.org .org 71
28,170 suedkurier.de .de 81
28,220 icecat.se .se 52
28,237 zamin.uz .uz 92
28,290 icecat.dk .dk 56
28,320 laptopmag.com .com 87
28,324 eurekamag.com .com 88
28,336 cargoagent.net .net 51
28,349 suamusica.com.br .br 77
28,361 core77.com .com 57
28,377 menuweb.menu .menu 69
28,383 lightreading.com .com 81
28,438 360cities.net .net 49
28,480 lto.de .de 72
28,488 jsr.io .io 61
28,522 distance.to .to 57
28,573 wpr.org .org 88
28,609 t-skirt.biz .biz 56
28,616 amzn.com .com 40
28,642 nachrichten.at .at 82
28,670 rapidtables.com .com 59
28,676 xforum.live .live 60
28,678 thenational.scot .scot 81
28,729 famelack.com .com 81
28,740 ctt.pt .pt 54
28,754 nekobt.to .to 54
28,767 mult-porno.team .team 56
28,775 vogue.de .de 65
28,792 uesp.net .net 55
28,794 eastbaytimes.com .com 79
28,861 deeplearning.ai .ai 95
28,868 myvolumio.org .org 38
28,895 weatheronline.co.uk .uk 47
28,930 championcross.jp .jp 62
28,935 red-dot.org .org 74
28,950 acronymfinder.com .com 65
28,996 sebokep.vip .vip 78
29,011 fraserinstitute.org .org 51
29,031 threadreaderapp.com .com 59
29,048 gq-magazin.de .de 66
29,063 golzu.com .com 54
29,139 u-cursos.cl .cl 51
29,166 freakonomics.com .com 74
29,170 spinmagazine.com .com 79
29,224 ulife.com.br .br 54
29,260 b-cd.app .app 52
29,313 qbittorrent.org .org 50
29,340 bunjang.co.kr .kr 62
29,341 geometrylitepc.io .io 70
29,390 kompas.id .id 64
29,402 mmafighting.com .com 68
29,447 artofproblemsolving.com .com 86
29,499 conjur.com.br .br 73
29,513 mindtos.com .com 57
29,526 sex-studentki.guru .guru 57
29,551 lavoro.gov.it .it 63
29,553 hankookilbo.com .com 84
29,562 therarbg.com .com 77
29,603 governing.com .com 87
29,634 westword.com .com 77
29,639 edmontonjournal.com .com 71
29,661 nn125.com .com 49
29,676 ccel.org .org 64
29,692 piliapp.com .com 65
29,700 gamessofun.com .com 50
29,701 thieme-connect.de .de 79
29,723 bookwalker.jp .jp 72
29,732 potaroo.net .net 65
29,740 zn.ua .ua 87
29,821 providencejournal.com .com 73
29,991 efhub.com .com 55
30,004 sfstandard.com .com 72
30,046 marketingweek.com .com 74
30,049 processing.org .org 57
30,052 lasprovincias.es .es 82
30,074 yugipedia.com .com 58
30,161 bostonmagazine.com .com 72
30,196 youngchampion.jp .jp 62
30,202 serializd.com .com 65
30,237 clarionledger.com .com 74
30,288 anycubic.com .com 68
30,301 qtoon.org .org 61
30,376 getxray.app .app 80
30,382 wook.pt .pt 57
30,387 bsc.es .es 37
30,393 ecode360.com .com 44
30,443 dobrzemieszkaj.pl .pl 61
30,457 fap-guru.name .name 56
30,470 turbo.cr .cr 54
30,472 bamgad7.com .com 66
30,507 abc11.com .com 81
30,517 premioibest.com .com 70
30,588 streamimdb.ru .ru 72
30,619 spectrumnews1.com .com 45
30,678 xbato.co.uk .uk 73
30,698 nswpedia.com .com 75
30,703 wickedlocal.com .com 75
30,711 hi5.com .com 43
30,712 artforum.com .com 74
30,737 diariosur.es .es 82
30,738 fedpat.com.ar .ar 69
30,810 kentonline.co.uk .uk 76
30,828 bowdoin.edu .edu 71
30,864 umms.org .org 76
30,915 sendtuna.com .com 45
30,928 bokepindoh.party .party 71
30,974 techadvisor.com .com 85
30,976 tablegame.ph .ph 39
PreviousPage 7 of 12Next

Rank is the domain's position in the popularity list used to seed the corpus; a dash means it was scanned on request rather than seeded. Score is the site's overall AI access score, which blocking depresses but does not by itself determine.

Check a specific domain

See also the Meta-ExternalAgent reference, the full census, and the JSON endpoint.