Blocklist · Amazon

Sites that block Amzn-SearchBot

253 of the 29,991 domains we have measured, 0.8%, disallow amzn-searchbot at the site root in robots.txt. Amzn-SearchBot builds the index that decides whether the operator's assistant cites a page.

Every domain here has removed itself from Amazon's answers. Blocking a retrieval agent withholds nothing from model training, because training uses a different token.

The measurement

253 of 29,991 measured domains (0.8%) disallow amzn-searchbot at the site root in robots.txt. Every domain named on this page was fetched directly and its robots.txt parsed against RFC 9309; none of it is inferred from a third-party index.

Measured 2026-11-21 by direct request to every domain in the corpus. Method: how this is measured. Reuse under CC BY 4.0 with attribution.

Cite as: Crawl Census, "Sites that block Amzn-SearchBot", measured 2026-11-21. https://crawlcensus.com/blocked/amzn-searchbot

The list 253 domains

RankDomainTLDAI access score
190 cnn.com .com 50
193 theguardian.com .com 60
207 bbc.com .com 76
226 bbc.co.uk .uk 77
662 cnet.com .com 79
941 healthline.com .com 80
977 onet.pl .pl 76
1,019 theconversation.com .com 62
1,061 as.com .com 84
1,083 nypost.com .com 81
1,098 franceinfo.fr .fr 60
1,223 furaffinity.net .net 39
1,376 dagbladet.no .no 74
1,790 notion.com .com 94
1,831 zdnet.com .com 76
1,993 ign.com .com 62
2,010 mashable.com .com 61
2,085 medicalnewstoday.com .com 81
2,299 guardian.co.uk .uk 60
2,423 pcmag.com .com 63
2,786 chicagotribune.com .com 79
3,342 lifehacker.com .com 65
3,576 abc.es .es 81
4,161 kompas.com .com 77
4,866 channel4.com .com 56
4,925 cadenaser.com .com 78
5,146 everydayhealth.com .com 60
5,158 mercurynews.com .com 78
5,457 diigo.com .com 35
5,678 dn.se .se 60
5,681 nationalpost.com .com 71
5,714 francetvinfo.fr .fr 53
5,912 pagesix.com .com 80
6,089 azcentral.com .com 74
6,125 denverpost.com .com 78
6,246 eurogamer.net .net 57
6,356 freep.com .com 73
6,512 telegram.com .com 73
6,513 sandiegouniontribune.com .com 78
6,816 maxroll.gg .gg 59
6,822 elcorreo.com .com 80
6,948 nicematin.com .com 77
7,082 aktuality.sk .sk 85
7,127 hani.co.kr .kr 67
7,239 gala.de .de 76
7,275 baltimoresun.com .com 76
7,294 fakt.pl .pl 78
7,562 nettavisen.no .no 63
7,863 refinery29.com .com 78
7,981 afr.com .com 72
8,094 nefisyemektarifleri.com .com 76
8,310 psychcentral.com .com 79
8,800 sefan.ru .ru 43
8,928 nottingham.ac.uk .uk 64
9,339 detroitnews.com .com 68
9,542 findnovel.net .net 64
10,470 babycenter.com .com 62
10,495 financialpost.com .com 71
10,501 ocregister.com .com 80
10,729 businessinsider.com.pl .pl 79
10,885 livestrong.com .com 80
12,113 bostonherald.com .com 78
12,441 novelfire.net .net 64
12,533 indystar.com .com 73
12,743 extremetech.com .com 64
12,770 imginn.com .com 77
13,333 wtop.com .com 74
13,419 vancouversun.com .com 71
13,451 skyscrapercity.com .com 43
13,667 dispatch.com .com 74
13,668 torontosun.com .com 71
13,871 ilex.by .by 59
13,879 tennessean.com .com 74
14,709 dazeddigital.com .com 78
14,826 rockpapershotgun.com .com 58
14,862 canada.com .com 71
15,174 pcgamesn.com .com 83
15,233 thieme-connect.com .com 39
15,269 gamesindustry.biz .biz 57
15,401 desmoinesregister.com .com 74
15,509 diariovasco.com .com 82
15,557 cincinnati.com .com 74
16,103 dweb.link .link 65
16,209 ipfs.io .io 65
16,377 decider.com .com 79
16,564 mapgenie.io .io 59
16,661 windfinder.com .com 84
16,664 lightningmaps.org .org 51
16,814 whattoexpect.com .com 65
17,042 northjersey.com .com 74
17,455 twincities.com .com 81
17,469 palmbeachpost.com .com 73
17,492 ina.fr .fr 76
17,886 agrarheute.com .com 75
17,989 clickondetroit.com .com 86
18,280 hoerzu.de .de 45
18,309 mans.io .io 60
18,442 empireonline.com .com 62
18,489 elcolombiano.com .com 83
18,490 click2houston.com .com 85
18,522 thenews.com.pk .pk 65
18,595 trycloudflare.com .com 52
18,863 dailynews.com .com 76
18,883 courier-journal.com .com 74
19,182 clickorlando.com .com 85
19,312 pinayflix.uk .uk 46
19,852 taximaxim.ru .ru 69
19,860 zawya.com .com 80
19,941 ottawacitizen.com .com 71
20,035 plejada.pl .pl 83
20,673 catched.com .com 36
20,958 pressdemocrat.com .com 80
21,139 ledevoir.com .com 70
21,267 seher.no .no 70
21,685 calgaryherald.com .com 71
21,823 desy.de .de 62
22,007 vg247.com .com 58
22,028 distrowatch.com .com 44
22,031 news4jax.com .com 85
22,479 ksat.com .com 85
22,623 mcall.com .com 78
22,987 syakouba.com .com 56
23,084 app.com .com 69
23,157 mlnba.fun .fun 54
24,000 icecat.biz .biz 44
24,014 askmen.com .com 47
24,028 canberratimes.com.au .au 79
24,102 knoxnews.com .com 74
24,190 jacksonville.com .com 74
24,525 zin.ru .ru 41
24,892 eldiariomontanes.es .es 82
25,115 pilotonline.com .com 79
25,133 myrecipes.com .com 77
25,239 greatist.com .com 80
25,839 delawareonline.com .com 74
25,903 heraldtribune.com .com 73
25,911 arkansasonline.com .com 71
26,114 democratandchronicle.com .com 73
26,637 brigitte.de .de 79
26,853 tallahassee.com .com 74
27,256 amny.com .com 83
27,723 geek.com .com 38
27,796 asiasociety.org .org 67
28,220 icecat.se .se 52
28,290 icecat.dk .dk 56
28,794 eastbaytimes.com .com 79
29,139 u-cursos.cl .cl 51
29,390 kompas.id .id 64
29,639 edmontonjournal.com .com 71
29,701 thieme-connect.de .de 79
29,821 providencejournal.com .com 73
30,052 lasprovincias.es .es 82
30,237 clarionledger.com .com 74
30,703 wickedlocal.com .com 75
30,737 diariosur.es .es 82
30,986 commercialappeal.com .com 74
31,032 howlongtobeat.com .com 49
31,138 ideal.es .es 82
31,320 lumendatabase.org .org 51
31,352 sportsbusinessjournal.com .com 53
31,499 bolasport.com .com 76
31,549 lohud.com .com 74
31,786 thespec.com .com 64
31,850 avsforum.com .com 43
32,388 rgj.com .com 73
32,437 hoopshype.com .com 70
32,617 flo.com.tr .tr 52
32,918 laverdad.es .es 82
33,568 icecat.it .it 44
34,147 valueyourmusic.com .com 73
34,309 unionleader.com .com 63
34,387 breakingnews.ie .ie 71
34,973 naplesnews.com .com 73
35,062 dailypress.com .com 79
35,543 news-journalonline.com .com 73
35,760 montgomerycountymd.gov .gov 65
35,953 redflagdeals.com .com 44
36,120 elcomercio.es .es 82
36,237 saltwire.com .com 70
36,372 eurobilltracker.com .com 65
36,436 liquor.com .com 77
36,778 cwmars.org .org 43
37,150 taximaxim.com .com 60
37,700 rutracker.net .net 43
38,148 news-press.com .com 73
38,327 inforum.com .com 70
38,406 icecat.us .us 61
38,409 igt.com .com 48
38,814 mas.to .to 31
38,876 elnortedecastilla.es .es 82
39,349 hoy.es .es 82
39,471 vwvortex.com .com 43
39,806 mercateo.com .com 48
39,824 werstreamt.es .es 59
40,339 kompas.tv .tv 76
40,796 elpasotimes.com .com 73
40,874 grid.id .id 64
41,647 cjonline.com .com 73
41,721 canarias7.es .es 83
41,760 dailybreeze.com .com 76
Page 1 of 2Next

Rank is the domain's position in the popularity list used to seed the corpus; a dash means it was scanned on request rather than seeded. Score is the site's overall AI access score, which blocking depresses but does not by itself determine.

Check a specific domain

See also the Amzn-SearchBot reference, the full census, and the JSON endpoint.