Blocklist · You.com

Sites that block YouBot

876 of the 29,991 domains we have measured, 3%, disallow youbot at the site root in robots.txt. YouBot builds the index that decides whether the operator's assistant cites a page.

Every domain here has removed itself from You.com's answers. Blocking a retrieval agent withholds nothing from model training, because training uses a different token.

The measurement

876 of 29,991 measured domains (3%) disallow youbot at the site root in robots.txt. Every domain named on this page was fetched directly and its robots.txt parsed against RFC 9309; none of it is inferred from a third-party index.

Measured 2026-11-22 by direct request to every domain in the corpus. Method: how this is measured. Reuse under CC BY 4.0 with attribution.

Cite as: Crawl Census, "Sites that block YouBot", measured 2026-11-22. https://crawlcensus.com/blocked/youbot

The list 876 domains

RankDomainTLDAI access score
7,109 military.com .com 84
7,127 hani.co.kr .kr 67
7,178 sanspo.com .com 77
7,239 gala.de .de 76
7,247 nishinippon.co.jp .jp 69
7,250 manchestereveningnews.co.uk .uk 47
7,275 baltimoresun.com .com 76
7,294 fakt.pl .pl 78
7,345 rpp.pe .pe 90
7,360 mathrubhumi.com .com 82
7,372 15min.lt .lt 74
7,432 nouvelobs.com .com 87
7,455 amazon.sg .sg 40
7,510 indiewire.com .com 75
7,540 freeones.com .com 71
7,556 lindependant.fr .fr 86
7,562 nettavisen.no .no 63
7,613 filmstarts.de .de 81
7,738 pitchfork.com .com 81
7,780 purepeople.com .com 85
7,821 gala.fr .fr 90
7,856 popsugar.com .com 75
7,863 refinery29.com .com 78
7,925 adorocinema.com .com 81
7,937 calciomercato.com .com 81
7,981 afr.com .com 72
8,094 nefisyemektarifleri.com .com 76
8,110 tudogostoso.com.br .br 83
8,160 javhdporn.net .net 71
8,173 leggo.it .it 62
8,207 flashscore.pl .pl 71
8,268 litmarket.ru .ru 87
8,283 americatv.com.pe .pe 89
8,287 javdock.com .com 65
8,292 walesonline.co.uk .uk 47
8,364 eredmenyek.com .com 71
8,380 veikkaus.fi .fi 67
8,398 livesport.cz .cz 71
8,435 thecut.com .com 84
8,541 91porna.com .com 88
8,599 soccerway.com .com 73
8,664 voetbalzone.nl .nl 81
8,710 rezultati.com .com 71
8,800 sefan.ru .ru 43
8,830 portalsamorzadowy.pl .pl 69
8,866 theweek.com .com 84
8,883 manhuagui.com .com 61
8,897 rynekzdrowia.pl .pl 64
8,899 flashscore.co.id .id 70
8,928 nottingham.ac.uk .uk 64
8,959 glamour.com .com 81
9,007 flashscore.co .co 69
9,072 flashscore.ua .ua 70
9,094 flashscore.ro .ro 70
9,149 forth.gr .gr 60
9,196 flashscore.es .es 69
9,285 pkok11.com .com 52
9,317 flashscore.com.ve .ve 69
9,320 flashscore.pe .pe 70
9,339 detroitnews.com .com 68
9,350 cv666.net .net 51
9,388 law.com .com 68
9,471 thewrap.com .com 87
9,542 findnovel.net .net 64
10,074 cope.es .es 86
10,279 klingai.com .com 78
10,289 interestingengineering.com .com 87
10,291 usmagazine.com .com 87
10,335 windowscentral.com .com 85
10,345 inews.co.uk .uk 67
10,400 morgenpost.de .de 81
10,432 expansion.com .com 81
10,470 babycenter.com .com 62
10,495 financialpost.com .com 71
10,501 ocregister.com .com 80
10,552 self.com .com 68
10,585 anyflip.com .com 56
10,596 uptodate.com .com 33
10,621 dailystar.co.uk .uk 47
10,680 f-droid.org .org 63
10,708 vogue.co.uk .uk 79
10,729 businessinsider.com.pl .pl 79
10,781 47news.jp .jp 67
10,826 bonappetit.com .com 82
10,885 livestrong.com .com 80
10,894 augsburger-allgemeine.de .de 81
10,934 huffingtonpost.fr .fr 81
11,130 lexpress.fr .fr 86
11,154 internshala.com .com 87
11,200 aftenposten.no .no 76
11,201 latercera.com .com 80
11,216 nola.com .com 64
11,217 businessinsider.de .de 76
11,242 teenvogue.com .com 82
11,349 posteo.de .de 67
11,390 pantone.com .com 78
11,405 svd.se .se 78
11,457 wnp.pl .pl 65
11,465 houstonchronicle.com .com 54
11,610 allure.com .com 84
11,659 abendblatt.de .de 65
11,746 fextralife.com .com 87
11,751 dailyrecord.co.uk .uk 47
11,827 timesunion.com .com 52
11,914 thehindubusinessline.com .com 65
12,058 wiwo.de .de 82
12,111 techpowerup.com .com 50
12,113 bostonherald.com .com 78
12,377 delfi.ee .ee 57
12,441 novelfire.net .net 64
12,533 indystar.com .com 73
12,542 epicurious.com .com 64
12,550 kiplinger.com .com 85
12,694 androidpolice.com .com 83
12,743 extremetech.com .com 64
12,799 curbed.com .com 73
12,841 stltoday.com .com 80
12,972 noticiasaominuto.com .com 85
13,005 simpleflying.com .com 83
13,355 marieclaire.com .com 85
13,398 gq-magazine.co.uk .uk 81
13,419 vancouversun.com .com 71
13,451 skyscrapercity.com .com 43
13,579 barcelona.cat .cat 82
13,617 latribune.fr .fr 72
13,660 thrillist.com .com 70
13,667 dispatch.com .com 74
13,668 torontosun.com .com 71
13,737 unstop.com .com 55
13,772 khan.co.kr .kr 72
13,783 creativebloq.com .com 87
13,792 sheknows.com .com 83
13,825 heraldscotland.com .com 81
13,858 flashscore.co.za .za 70
13,871 ilex.by .by 59
13,873 rappler.com .com 80
13,879 tennessean.com .com 74
13,882 vermont.gov .gov 73
14,053 androidcentral.com .com 83
14,079 wccftech.com .com 80
14,087 diepresse.com .com 80
14,090 simon.com .com 31
14,092 politiken.dk .dk 73
14,112 project-syndicate.org .org 88
14,374 delfi.lv .lv 80
14,612 mopo.de .de 79
14,733 waz.de .de 82
14,809 auto-motor-und-sport.de .de 81
14,826 rockpapershotgun.com .com 58
14,862 canada.com .com 71
14,872 mz.de .de 70
14,961 townnews.com .com 75
15,084 bestprice.gr .gr 97
15,174 pcgamesn.com .com 83
15,218 statesman.com .com 51
15,230 mysanantonio.com .com 54
15,260 birminghammail.co.uk .uk 47
15,269 gamesindustry.biz .biz 57
15,401 desmoinesregister.com .com 74
15,476 computerbild.de .de 81
15,509 diariovasco.com .com 82
15,557 cincinnati.com .com 74
15,686 postimees.ee .ee 66
16,082 t3n.de .de 78
16,103 dweb.link .link 65
16,118 healio.com .com 69
16,134 homesandgardens.com .com 84
16,164 christianitytoday.com .com 89
16,209 ipfs.io .io 65
16,223 smashwords.com .com 58
16,254 unil.ch .ch 83
16,290 duden.de .de 71
16,317 ilmattino.it .it 62
16,421 dejure.org .org 72
16,428 e24.no .no 78
16,444 hometalk.com .com 63
16,540 bundesliga.com .com 72
16,564 mapgenie.io .io 59
16,664 lightningmaps.org .org 51
16,796 postandcourier.com .com 77
16,803 buffalonews.com .com 80
16,814 whattoexpect.com .com 65
17,039 robbreport.com .com 64
17,042 northjersey.com .com 74
17,131 lrb.co.uk .uk 94
17,183 semafor.com .com 72
17,255 thegamer.com .com 83
17,266 letemps.ch .ch 74
17,455 twincities.com .com 81
17,469 palmbeachpost.com .com 73
17,492 ina.fr .fr 76
17,522 daangn.com .com 71
17,536 seattlepi.com .com 54
17,694 manager-magazin.de .de 63
17,886 agrarheute.com .com 75
17,890 directoalpaladar.com .com 92
17,936 cinemablend.com .com 87
17,988 timeslive.co.za .za 80
17,989 clickondetroit.com .com 86
18,071 flashscore.co.uk .uk 71
PreviousPage 2 of 5Next

Rank is the domain's position in the popularity list used to seed the corpus; a dash means it was scanned on request rather than seeded. Score is the site's overall AI access score, which blocking depresses but does not by itself determine.

Check a specific domain

See also the YouBot reference, the full census, and the JSON endpoint.