Blocklist · DuckDuckGo

Sites that block DuckAssistBot

387 of the 20,795 domains we have measured, 2%, disallow duckassistbot at the site root in robots.txt. DuckAssistBot fetches a page live when a user asks the assistant for it.

Every domain here has removed itself from DuckDuckGo's answers. Blocking a retrieval agent withholds nothing from model training, because training uses a different token.

The measurement

387 of 20,795 measured domains (2%) disallow duckassistbot at the site root in robots.txt. Every domain named on this page was fetched directly and its robots.txt parsed against RFC 9309; none of it is inferred from a third-party index.

Measured 2026-08-23 by direct request to every domain in the corpus. Method: how this is measured. Reuse under CC BY 4.0 with attribution.

Cite as: Crawl Census, "Sites that block DuckAssistBot", measured 2026-08-23. https://crawlcensus.com/blocked/duckassistbot

The list 387 domains

RankDomainTLDAI access score
26 amazon.com .com 33
54 tiktok.com .com 41
62 msn.com .com 34
121 amazonvideo.com .com 63
157 nytimes.com .com 65
190 cnn.com .com 50
193 theguardian.com .com 60
260 launchpad.net .net 46
318 amazon.co.uk .uk 33
377 amazon.de .de 33
494 amazon.co.jp .jp 33
533 wired.com .com 74
552 usatoday.com .com 65
571 uol.com.br .br 74
609 amazon.fr .fr 33
623 amazon.co.za .za 33
627 dailymail.co.uk .uk 62
635 amazon.ca .ca 33
659 amazon.in .in 33
662 cnet.com .com 75
671 lemonde.fr .fr 72
681 dailymail.com .com 62
682 primevideo.com .com 50
706 amazon.it .it 33
725 amazon.es .es 33
763 bild.de .de 74
800 welt.de .de 65
812 amazon.com.au .au 33
815 theverge.com .com 68
833 amazon.com.br .br 33
942 repubblica.it .it 77
1,019 theconversation.com .com 52
1,027 huffpost.com .com 82
1,032 amazon.com.mx .mx 34
1,074 flashscore.com .com 67
1,177 pimpbunny.com .com 59
1,223 furaffinity.net .net 41
1,275 huffingtonpost.com .com 82
1,300 f95zone.to .to 49
1,409 iltalehti.fi .fi 50
1,442 buzzfeed.com .com 58
1,446 ekstrabladet.dk .dk 60
1,624 flashscore.mobi .mobi 63
1,640 flashscore.fr .fr 64
1,676 livescore.cz .cz 63
1,793 newyorker.com .com 71
1,831 zdnet.com .com 61
1,835 wikihow.com .com 72
1,847 arstechnica.com .com 72
1,984 themoviedb.org .org 61
1,993 ign.com .com 64
2,006 amazon.eg .eg 34
2,010 mashable.com .com 62
2,189 vox.com .com 68
2,299 guardian.co.uk .uk 60
2,325 nrk.no .no 61
2,351 diretta.it .it 64
2,423 pcmag.com .com 65
2,430 sfgate.com .com 49
2,530 smh.com.au .au 69
2,561 technologyreview.com .com 75
2,656 tmdb.org .org 66
2,659 theglobeandmail.com .com 56
2,691 newscientist.com .com 57
2,786 chicagotribune.com .com 75
2,899 vanityfair.it .it 69
2,917 faz.net .net 76
2,975 imageshack.us .us 29
2,989 vogue.com .com 78
3,116 newsbreak.com .com 84
3,198 metro.co.uk .uk 56
3,286 flipboard.com .com 60
3,342 lifehacker.com .com 63
3,368 nzherald.co.nz .nz 47
3,371 nymag.com .com 71
3,433 thefreedictionary.com .com 66
3,499 amazon.se .se 37
3,520 amazon.ae .ae 38
3,524 chron.com .com 51
3,533 vanityfair.com .com 77
3,576 abc.es .es 81
3,595 politico.eu .eu 58
3,598 amazon.nl .nl 37
3,621 nydailynews.com .com 71
3,717 ansa.it .it 80
3,808 toyota.com .com 57
3,882 radiofrance.fr .fr 80
3,963 nos.nl .nl 61
4,065 seekingalpha.com .com 79
4,070 ilsole24ore.com .com 73
4,097 amazon.pl .pl 37
4,107 thestar.com .com 60
4,161 kompas.com .com 80
4,240 amazon.sa .sa 38
4,512 amazon.com.tr .tr 37
4,592 lastampa.it .it 77
4,866 channel4.com .com 52
4,925 cadenaser.com .com 78
4,927 iol.co.za .za 65
4,987 dr.dk .dk 76
5,037 imageshack.com .com 29
5,068 gq.com .com 77
5,096 nine.com.au .au 71
5,146 everydayhealth.com .com 57
5,153 theage.com.au .au 63
5,158 mercurynews.com .com 75
5,251 dallasnews.com .com 50
5,297 chinatimes.com .com 79
5,413 expressen.se .se 58
5,416 gentoo.org .org 57
5,523 architecturaldigest.com .com 78
5,547 amazon.ie .ie 40
5,677 zepto.com .com 86
5,678 dn.se .se 63
5,722 hola.com .com 89
5,775 tv2.no .no 70
5,788 ibtimes.com .com 84
6,004 ilmessaggero.it .it 62
6,064 delfi.lt .lt 80
6,089 azcentral.com .com 69
6,125 denverpost.com .com 79
6,180 tf1info.fr .fr 84
6,184 nyt.com .com 65
6,228 sbnation.com .com 69
6,246 eurogamer.net .net 57
6,288 vulture.com .com 71
6,352 tv2.dk .dk 74
6,356 freep.com .com 73
6,361 ici.fr .fr 87
6,386 huffingtonpost.co.uk .uk 82
6,396 parismatch.com .com 74
6,473 sfchronicle.com .com 54
6,512 telegram.com .com 73
6,513 sandiegouniontribune.com .com 79
6,522 eater.com .com 67
6,557 sportingnews.com .com 83
6,568 dhnet.be .be 83
6,593 wired.co.uk .uk 69
6,726 rtbf.be .be 76
6,808 chunichi.co.jp .jp 71
6,816 maxroll.gg .gg 59
6,822 elcorreo.com .com 82
6,869 di.se .se 51
6,895 togetter.com .com 73
6,922 livescore.in .in 71
6,940 bhg.com .com 78
6,948 nicematin.com .com 78
6,988 buzzfeednews.com .com 79
7,042 cntraveler.com .com 75
7,082 aktuality.sk .sk 86
7,106 nrc.nl .nl 85
7,178 sanspo.com .com 77
7,239 gala.de .de 78
7,247 nishinippon.co.jp .jp 69
7,275 baltimoresun.com .com 77
7,372 15min.lt .lt 74
7,455 amazon.sg .sg 40
7,647 sporza.be .be 87
7,734 tf1.fr .fr 70
7,738 pitchfork.com .com 82
7,745 huffingtonpost.jp .jp 78
7,856 popsugar.com .com 75
7,863 refinery29.com .com 78
7,981 afr.com .com 80
8,094 nefisyemektarifleri.com .com 67
8,167 flashscore.com.br .br 68
8,173 leggo.it .it 62
8,207 flashscore.pl .pl 71
8,364 eredmenyek.com .com 71
8,380 veikkaus.fi .fi 67
8,398 livesport.cz .cz 71
8,435 thecut.com .com 84
8,450 complex.com .com 93
8,599 soccerway.com .com 71
8,603 rnz.co.nz .nz 52
8,704 flashscore.com.ng .ng 69
8,710 rezultati.com .com 71
8,830 portalsamorzadowy.pl .pl 69
8,897 rynekzdrowia.pl .pl 64
8,899 flashscore.co.id .id 69
8,959 glamour.com .com 81
9,007 flashscore.co .co 68
9,072 flashscore.ua .ua 70
9,094 flashscore.ro .ro 69
9,149 forth.gr .gr 60
9,196 flashscore.es .es 68
9,317 flashscore.com.ve .ve 68
9,320 flashscore.pe .pe 69
9,339 detroitnews.com .com 68
9,388 law.com .com 59
9,398 leckerschmecker.me .me 81
9,599 jsonline.com .com 73
10,289 interestingengineering.com .com 70
10,345 inews.co.uk .uk 65
10,470 babycenter.com .com 64
10,501 ocregister.com .com 76
10,552 self.com .com 65
10,678 newspapers.com .com 48
10,708 vogue.co.uk .uk 79
10,826 bonappetit.com .com 83
Page 1 of 2Next

Rank is the domain's position in the popularity list used to seed the corpus; a dash means it was scanned on request rather than seeded. Score is the site's overall AI access score, which blocking depresses but does not by itself determine.

Check a specific domain

See also the DuckAssistBot reference, the full census, and the JSON endpoint.