Blocklist · Meta

Sites that block Meta-ExternalFetcher

553 of the 29,991 domains we have measured, 2%, disallow meta-externalfetcher at the site root in robots.txt. Meta-ExternalFetcher fetches a page live when a user asks the assistant for it.

Every domain here has removed itself from Meta's answers. Blocking a retrieval agent withholds nothing from model training, because training uses a different token.

Meta documents that this agent may ignore robots.txt, so a Disallow here records the site's stated preference rather than an enforced block.
The measurement

553 of 29,991 measured domains (2%) disallow meta-externalfetcher at the site root in robots.txt. Every domain named on this page was fetched directly and its robots.txt parsed against RFC 9309; none of it is inferred from a third-party index.

Measured 2026-11-21 by direct request to every domain in the corpus. Method: how this is measured. Reuse under CC BY 4.0 with attribution.

Cite as: Crawl Census, "Sites that block Meta-ExternalFetcher", measured 2026-11-21. https://crawlcensus.com/blocked/meta-externalfetcher

The list 553 domains

RankDomainTLDAI access score
34,236 welib.st .st 47
34,301 stevehoffman.tv .tv 65
34,309 unionleader.com .com 63
34,445 monoskop.org .org 43
34,718 sportsyou.com .com 86
34,737 vibe.com .com 80
34,750 mailplus.co.uk .uk 44
34,822 milanofinanza.it .it 82
34,847 sydsvenskan.se .se 52
34,969 enr.com .com 82
34,973 naplesnews.com .com 73
35,062 dailypress.com .com 79
35,158 jyllands-posten.dk .dk 70
35,519 youglish.com .com 59
35,543 news-journalonline.com .com 73
35,759 houseandgarden.co.uk .uk 81
35,800 wn.de .de 80
35,828 dotdashmeredith.com .com 64
36,120 elcomercio.es .es 82
36,237 saltwire.com .com 70
36,334 ytmnd.com .com 41
36,436 liquor.com .com 77
36,539 nd-aktuell.de .de 78
36,575 dailyecho.co.uk .uk 81
36,778 cwmars.org .org 43
36,895 uokik.gov.pl .pl 85
37,613 unescap.org .org 72
37,700 rutracker.net .net 43
38,007 wired.jp .jp 78
38,114 sportico.com .com 64
38,148 news-press.com .com 73
38,327 inforum.com .com 70
38,388 goldderby.com .com 74
38,409 igt.com .com 48
38,507 bournemouthecho.co.uk .uk 79
38,519 kochbar.de .de 62
38,814 mas.to .to 31
38,876 elnortedecastilla.es .es 82
38,945 postype.com .com 81
39,303 watoday.com.au .au 75
39,349 hoy.es .es 82
39,350 thehighline.org .org 77
39,393 zakzak.co.jp .jp 72
39,824 werstreamt.es .es 59
40,317 iza.ne.jp .jp 79
40,339 kompas.tv .tv 76
40,360 noen.at .at 79
40,512 thetelegraphandargus.co.uk .uk 78
40,638 main-echo.de .de 64
40,713 dwds.de .de 64
40,796 elpasotimes.com .com 73
40,858 inspiredtaste.net .net 70
40,874 grid.id .id 64
40,876 cntraveller.in .in 81
40,904 nesn.com .com 73
40,968 kontan.co.id .id 77
41,098 gqmagazine.fr .fr 82
41,121 lancashiretelegraph.co.uk .uk 78
41,519 greenme.it .it 82
41,542 oefb.at .at 50
41,603 lci.fr .fr 84
41,647 cjonline.com .com 73
41,736 eadt.co.uk .uk 79
41,760 dailybreeze.com .com 76
41,826 defcon.org .org 66
41,872 vanityfair.fr .fr 66
41,947 vogue.co.jp .jp 65
41,958 theledger.com .com 74
41,962 okinawatimes.co.jp .jp 79
42,076 iex.nl .nl 48
42,113 glasgowtimes.co.uk .uk 79
42,160 tcpalm.com .com 73
42,614 citizen-times.com .com 74
42,651 computerwoche.de .de 85
42,682 opinautos.com .com 52
42,698 sol.no .no 74
42,862 horseracingnation.com .com 72
42,919 tcrf.net .net 29
42,981 viewbug.com .com 83
43,302 seacoastonline.com .com 69
43,587 admagazine.fr .fr 81
43,627 tvrain.ru .ru 84
43,666 tasty.co .co 79
43,983 teambelgium.net .net 59
44,026 gainesville.com .com 66
44,116 eevblog.com .com 76
44,183 barcablaugranes.com .com 68
44,255 cagesideseats.com .com 72
44,299 theprovince.com .com 71
44,807 gofeminin.de .de 74
44,874 news-leader.com .com 62
44,878 rtvnoord.nl .nl 73
44,908 mora.jp .jp 45
44,928 wgt.com .com 80
44,968 savannahnow.com .com 64
45,002 babycentre.co.uk .uk 62
45,159 lansingstatejournal.com .com 62
45,240 vogue.mx .mx 65
45,272 rtvutrecht.nl .nl 73
45,578 chugoku-np.co.jp .jp 80
45,604 beaconjournal.com .com 66
45,680 greenvilleonline.com .com 62
45,771 larioja.com .com 83
45,846 amazon.jp .jp 40
45,918 revistagq.com .com 81
46,168 bizapedia.com .com 60
46,231 registerguard.com .com 66
46,301 redcafe.net .net 64
46,342 pjstar.com .com 66
46,360 pnj.com .com 62
46,594 okazii.ro .ro 67
46,635 wikihow.life .life 76
46,805 omroepwest.nl .nl 73
46,923 alisweb.org .org 39
46,949 schulferien.org .org 51
47,173 eurogamer.de .de 57
47,285 bengo4.com .com 86
47,299 rtvdrenthe.nl .nl 73
47,526 allgaeuer-zeitung.de .de 81
47,624 lacucinaitaliana.it .it 81
47,741 salzburg24.at .at 85
47,796 topekalibraries.info .info 46
47,884 vcstar.com .com 62
48,074 grubstreet.com .com 81
48,404 lesoleil.com .com 69
48,444 vogue.com.tw .tw 81
48,513 airnav.com .com 53
48,550 argusleader.com .com 62
48,613 theoutdoorstrader.com .com 52
48,859 castro.fm .fm 67
49,059 citypopulation.de .de 54
49,067 gqitalia.it .it 64
49,126 scientific.net .net 52
49,194 mailonsunday.co.uk .uk 61
49,245 startmycar.com .com 51
49,271 eltern.de .de 72
49,279 presstelegram.com .com 80
49,343 hd.se .se 51
49,420 gds.it .it 77
49,501 sj-r.com .com 64
49,574 montgomeryadvertiser.com .com 62
49,925 aamulehti.fi .fi 66
50,054 tekstowo.pl .pl 74
50,278 burlingtonfreepress.com .com 62
50,312 mujerhoy.com .com 80
50,338 ohtuleht.ee .ee 65
50,520 womansworld.com .com 85
50,684 gq.com.mx .mx 72
50,732 vesty.co.il .il 91
50,919 japantravel.com .com 79
50,933 nwaonline.com .com 73
50,982 nrtool.st .st 54
51,018 everydaycalculation.com .com 52
PreviousPage 3 of 3

Rank is the domain's position in the popularity list used to seed the corpus; a dash means it was scanned on request rather than seeded. Score is the site's overall AI access score, which blocking depresses but does not by itself determine.

Check a specific domain

See also the Meta-ExternalFetcher reference, the full census, and the JSON endpoint.