Blocklist · Meta

Sites that block Meta-WebIndexer

402 of the 29,991 domains we have measured, 1%, disallow meta-webindexer at the site root in robots.txt. Meta-WebIndexer builds the index that decides whether the operator's assistant cites a page.

Every domain here has removed itself from Meta's answers. Blocking a retrieval agent withholds nothing from model training, because training uses a different token.

The measurement

402 of 29,991 measured domains (1%) disallow meta-webindexer at the site root in robots.txt. Every domain named on this page was fetched directly and its robots.txt parsed against RFC 9309; none of it is inferred from a third-party index.

Measured 2026-11-21 by direct request to every domain in the corpus. Method: how this is measured. Reuse under CC BY 4.0 with attribution.

Cite as: Crawl Census, "Sites that block Meta-WebIndexer", measured 2026-11-21. https://crawlcensus.com/blocked/meta-webindexer

The list 402 domains

RankDomainTLDAI access score
26 amazon.com .com 40
121 amazonvideo.com .com 55
318 amazon.co.uk .uk 40
377 amazon.de .de 40
494 amazon.co.jp .jp 40
533 wired.com .com 81
609 amazon.fr .fr 40
623 amazon.co.za .za 40
627 dailymail.co.uk .uk 61
635 amazon.ca .ca 40
659 amazon.in .in 40
662 cnet.com .com 79
671 lemonde.fr .fr 81
681 dailymail.com .com 61
682 primevideo.com .com 71
706 amazon.it .it 40
725 amazon.es .es 40
812 amazon.com.au .au 40
823 nbcnews.com .com 59
833 amazon.com.br .br 40
977 onet.pl .pl 76
1,019 theconversation.com .com 62
1,032 amazon.com.mx .mx 40
1,073 bfmtv.com .com 81
1,098 franceinfo.fr .fr 60
1,223 furaffinity.net .net 39
1,409 iltalehti.fi .fi 58
1,456 eenadu.net .net 67
1,793 newyorker.com .com 73
1,831 zdnet.com .com 76
1,835 wikihow.com .com 76
1,993 ign.com .com 62
2,006 amazon.eg .eg 40
2,010 mashable.com .com 61
2,215 techradar.com .com 87
2,421 entrepreneur.com .com 89
2,423 pcmag.com .com 63
2,585 livescience.com .com 85
2,672 hindustantimes.com .com 90
2,691 newscientist.com .com 76
2,723 yumpu.com .com 30
2,786 chicagotribune.com .com 79
2,899 vanityfair.it .it 77
2,917 faz.net .net 39
2,969 today.com .com 62
2,989 vogue.com .com 82
3,210 letour.fr .fr 73
3,276 myanimelist.net .net 70
3,331 tomshardware.com .com 87
3,342 lifehacker.com .com 65
3,347 consumerreports.org .org 72
3,499 amazon.se .se 40
3,520 amazon.ae .ae 40
3,533 vanityfair.com .com 81
3,576 abc.es .es 81
3,598 amazon.nl .nl 40
3,873 space.com .com 85
3,916 news24.com .com 56
4,161 kompas.com .com 77
4,322 livemint.com .com 86
4,512 amazon.com.tr .tr 40
5,026 pcgamer.com .com 85
5,068 gq.com .com 81
5,146 everydayhealth.com .com 60
5,158 mercurynews.com .com 78
5,229 eltiempo.com .com 85
5,413 expressen.se .se 61
5,523 architecturaldigest.com .com 83
5,547 amazon.ie .ie 40
5,678 dn.se .se 60
5,681 nationalpost.com .com 71
5,714 francetvinfo.fr .fr 53
5,846 runescape.wiki .wiki 81
5,874 hs.fi .fi 68
6,089 azcentral.com .com 74
6,125 denverpost.com .com 78
6,246 eurogamer.net .net 57
6,352 tv2.dk .dk 74
6,356 freep.com .com 73
6,361 ici.fr .fr 87
6,377 gamesradar.com .com 85
6,512 telegram.com .com 73
6,513 sandiegouniontribune.com .com 78
6,537 adda247.com .com 88
6,593 wired.co.uk .uk 64
6,610 is.fi .fi 68
6,816 maxroll.gg .gg 59
6,822 elcorreo.com .com 80
6,895 togetter.com .com 73
6,948 nicematin.com .com 77
7,042 cntraveler.com .com 82
7,127 hani.co.kr .kr 67
7,275 baltimoresun.com .com 76
7,294 fakt.pl .pl 78
7,363 bhaskar.com .com 75
7,455 amazon.sg .sg 40
7,562 nettavisen.no .no 63
7,738 pitchfork.com .com 81
7,778 livehindustan.com .com 86
7,863 refinery29.com .com 78
7,981 afr.com .com 72
8,094 nefisyemektarifleri.com .com 76
8,866 theweek.com .com 84
8,959 glamour.com .com 81
9,069 wbur.org .org 81
9,339 detroitnews.com .com 68
10,335 windowscentral.com .com 85
10,400 morgenpost.de .de 81
10,470 babycenter.com .com 62
10,495 financialpost.com .com 71
10,501 ocregister.com .com 80
10,552 self.com .com 68
10,708 vogue.co.uk .uk 79
10,729 businessinsider.com.pl .pl 79
10,826 bonappetit.com .com 82
10,885 livestrong.com .com 80
11,201 latercera.com .com 80
11,242 teenvogue.com .com 82
11,610 allure.com .com 84
11,659 abendblatt.de .de 65
12,111 techpowerup.com .com 50
12,113 bostonherald.com .com 78
12,533 indystar.com .com 73
12,542 epicurious.com .com 64
12,550 kiplinger.com .com 85
12,743 extremetech.com .com 64
13,355 marieclaire.com .com 85
13,398 gq-magazine.co.uk .uk 81
13,419 vancouversun.com .com 71
13,451 skyscrapercity.com .com 43
13,569 blitz.gg .gg 64
13,667 dispatch.com .com 74
13,668 torontosun.com .com 71
13,783 creativebloq.com .com 87
13,860 francebleu.fr .fr 87
13,871 ilex.by .by 59
13,879 tennessean.com .com 74
14,053 androidcentral.com .com 83
14,709 dazeddigital.com .com 78
14,733 waz.de .de 82
14,826 rockpapershotgun.com .com 58
14,862 canada.com .com 71
15,174 pcgamesn.com .com 83
15,233 thieme-connect.com .com 39
15,269 gamesindustry.biz .biz 57
15,401 desmoinesregister.com .com 74
15,509 diariovasco.com .com 82
15,557 cincinnati.com .com 74
16,134 homesandgardens.com .com 84
16,290 duden.de .de 71
16,564 mapgenie.io .io 59
16,664 lightningmaps.org .org 51
16,814 whattoexpect.com .com 65
17,042 northjersey.com .com 74
17,455 twincities.com .com 81
17,469 palmbeachpost.com .com 73
17,492 ina.fr .fr 76
17,835 odatv.com .com 87
17,886 agrarheute.com .com 75
17,936 cinemablend.com .com 87
18,011 genealogieonline.nl .nl 70
18,280 hoerzu.de .de 45
18,442 empireonline.com .com 62
18,489 elcolombiano.com .com 83
18,863 dailynews.com .com 76
18,883 courier-journal.com .com 74
18,913 armani.com .com 87
19,131 computerbase.de .de 89
19,429 thenewstack.io .io 69
19,563 blu-ray.com .com 69
19,860 zawya.com .com 80
19,891 musicradar.com .com 87
19,941 ottawacitizen.com .com 71
20,035 plejada.pl .pl 83
20,364 derwesten.de .de 83
20,718 itpro.com .com 83
20,847 gardeningknowhow.com .com 83
20,958 pressdemocrat.com .com 80
21,481 futureplc.com .com 86
21,566 whowhatwear.com .com 82
21,685 calgaryherald.com .com 71
21,694 rci.fm .fm 62
21,809 cyclingnews.com .com 85
21,823 desy.de .de 62
21,828 wallpaper.com .com 85
22,007 vg247.com .com 58
22,028 distrowatch.com .com 44
22,407 nzdating.com .com 68
22,620 multipasko.pl .pl 76
22,623 mcall.com .com 78
23,084 app.com .com 69
23,875 imore.com .com 81
24,014 askmen.com .com 47
24,102 knoxnews.com .com 74
24,141 cntraveller.com .com 81
24,157 whathifi.com .com 87
24,190 jacksonville.com .com 74
24,233 glamourmagazine.co.uk .uk 81
24,525 zin.ru .ru 41
24,593 vogue.in .in 64
Page 1 of 3Next

Rank is the domain's position in the popularity list used to seed the corpus; a dash means it was scanned on request rather than seeded. Score is the site's overall AI access score, which blocking depresses but does not by itself determine.

Check a specific domain

See also the Meta-WebIndexer reference, the full census, and the JSON endpoint.