Blocklist · Google

Sites that block Google-CloudVertexBot

1,361 of the 29,991 domains we have measured, 5%, disallow google-cloudvertexbot at the site root in robots.txt. Google-CloudVertexBot fetches a page live when a user asks the assistant for it.

Every domain here has removed itself from Google's answers. Blocking a retrieval agent withholds nothing from model training, because training uses a different token.

The measurement

1,361 of 29,991 measured domains (5%) disallow google-cloudvertexbot at the site root in robots.txt. Every domain named on this page was fetched directly and its robots.txt parsed against RFC 9309; none of it is inferred from a third-party index.

Measured 2026-11-21 by direct request to every domain in the corpus. Method: how this is measured. Reuse under CC BY 4.0 with attribution.

Cite as: Crawl Census, "Sites that block Google-CloudVertexBot", measured 2026-11-21. https://crawlcensus.com/blocked/google-cloudvertexbot

The list 1,361 domains

RankDomainTLDAI access score
26 amazon.com .com 40
62 msn.com .com 41
121 amazonvideo.com .com 55
193 theguardian.com .com 60
207 bbc.com .com 76
226 bbc.co.uk .uk 77
260 launchpad.net .net 52
282 weather.com .com 86
318 amazon.co.uk .uk 40
377 amazon.de .de 40
494 amazon.co.jp .jp 40
533 wired.com .com 81
535 cookielaw.org .org 75
543 patreon.com .com 62
609 amazon.fr .fr 40
623 amazon.co.za .za 40
627 dailymail.co.uk .uk 61
635 amazon.ca .ca 40
659 amazon.in .in 40
671 lemonde.fr .fr 81
681 dailymail.com .com 61
682 primevideo.com .com 71
706 amazon.it .it 40
725 amazon.es .es 40
812 amazon.com.au .au 40
815 theverge.com .com 77
833 amazon.com.br .br 40
834 lefigaro.fr .fr 71
955 corriere.it .it 63
983 elmundo.es .es 84
1,019 theconversation.com .com 62
1,032 amazon.com.mx .mx 40
1,056 marca.com .com 85
1,073 bfmtv.com .com 81
1,098 franceinfo.fr .fr 60
1,177 pimpbunny.com .com 56
1,223 furaffinity.net .net 39
1,231 gazzetta.it .it 80
1,272 vietnam.vn .vn 74
1,300 f95zone.to .to 59
1,376 dagbladet.no .no 74
1,409 iltalehti.fi .fi 58
1,426 crn77.com .com 51
1,430 omg10.com .com 51
1,439 toyhou.se .se 52
1,456 eenadu.net .net 67
1,484 dogdrip.net .net 52
1,501 xnxx.health .health 58
1,505 bblaa.com .com 51
1,514 255md.com .com 51
1,520 shahvani.com .com 45
1,536 theqoo.net .net 61
1,539 bef77.com .com 51
1,559 nya2.com .com 51
1,590 ey43.com .com 51
1,639 darkero.com .com 67
1,677 manhwaweb.com .com 42
1,723 lectormangass.net .net 77
1,724 xxxhindi.to .to 78
1,729 loteriadehoy.com .com 56
1,740 xn--12c1ezaww.com .com 74
1,747 moviesda33.com .com 62
1,762 filmyzilla45.com .com 62
1,793 newyorker.com .com 73
1,984 themoviedb.org .org 74
2,006 amazon.eg .eg 40
2,207 imhentai.xxx .xxx 51
2,299 guardian.co.uk .uk 60
2,319 tukrd.com .com 47
2,325 nrk.no .no 69
2,348 nontondrama.my .my 77
2,374 xvideosputaria.com .com 74
2,530 smh.com.au .au 82
2,561 technologyreview.com .com 82
2,607 vildey.cfd .cfd 38
2,637 filmyfly.builders .builders 64
2,638 filmyfly.racing .racing 64
2,639 filmyzilla40.com .com 62
2,640 filmyzilla43.com .com 62
2,641 moviesda32.com .com 62
2,646 moviesda31.com .com 62
2,647 moviezwap.onl .onl 52
2,691 newscientist.com .com 76
2,723 yumpu.com .com 30
2,751 hai8g.com .com 51
2,772 ko-fi.com .com 53
2,838 mastodon.social .social 49
2,899 vanityfair.it .it 77
2,975 imageshack.us .us 71
2,989 vogue.com .com 82
3,008 bgtee.com .com 51
3,054 intelbras.com.br .br 52
3,180 ay267.com .com 51
3,198 metro.co.uk .uk 59
3,320 unep.org .org 71
3,368 nzherald.co.nz .nz 56
3,371 nymag.com .com 74
3,433 thefreedictionary.com .com 71
3,499 amazon.se .se 40
3,520 amazon.ae .ae 40
3,533 vanityfair.com .com 81
3,595 politico.eu .eu 62
3,598 amazon.nl .nl 40
3,608 case-battle.id .id 48
3,662 filmyzilla52.com .com 55
3,686 moviesda34.com .com 62
3,700 moviezwap.land .land 52
3,711 rexify.com.ng .ng 59
3,712 filmyzilla46.com .com 55
3,717 ansa.it .it 85
3,730 oregonstate.edu .edu 63
3,800 ebsco.com .com 65
3,869 pcworld.com .com 84
3,963 nos.nl .nl 69
4,065 seekingalpha.com .com 91
4,079 vaticannews.va .va 44
4,084 uiowa.edu .edu 70
4,181 sbs.com.au .au 83
4,326 wiki.gg .gg 67
4,333 derstandard.at .at 56
4,334 blender.org .org 63
4,354 computerworld.com .com 83
4,512 amazon.com.tr .tr 40
4,866 channel4.com .com 56
4,927 iol.co.za .za 69
5,037 imageshack.com .com 71
5,068 gq.com .com 81
5,096 nine.com.au .au 71
5,153 theage.com.au .au 75
5,274 csmonitor.com .com 76
5,413 expressen.se .se 61
5,457 diigo.com .com 35
5,523 architecturaldigest.com .com 83
5,547 amazon.ie .ie 40
5,678 dn.se .se 60
5,684 theepochtimes.com .com 80
5,714 francetvinfo.fr .fr 53
5,752 sankei.com .com 76
5,770 boingboing.net .net 79
5,786 cloudpath.live .live 44
5,874 hs.fi .fi 68
5,876 dmitriy-mevedev-tor.top .top 53
5,879 cio.com .com 86
6,035 opaque.net .net 39
6,089 azcentral.com .com 74
6,228 sbnation.com .com 69
6,288 vulture.com .com 72
6,301 iata.org .org 68
6,352 tv2.dk .dk 74
6,356 freep.com .com 73
6,361 ici.fr .fr 87
6,512 telegram.com .com 73
6,513 sandiegouniontribune.com .com 78
6,522 eater.com .com 59
6,544 rtvslo.si .si 65
6,557 sportingnews.com .com 82
6,593 wired.co.uk .uk 64
6,609 procyclingstats.com .com 56
6,610 is.fi .fi 68
6,702 infoworld.com .com 83
6,808 chunichi.co.jp .jp 71
6,948 nicematin.com .com 77
7,042 cntraveler.com .com 82
7,058 playmogo.com .com 44
7,106 nrc.nl .nl 84
7,119 rp-online.de .de 79
7,178 sanspo.com .com 77
7,206 hentaiera.com .com 57
7,239 gala.de .de 76
7,247 nishinippon.co.jp .jp 69
7,349 rae.es .es 61
7,353 hentaifox.com .com 59
7,363 bhaskar.com .com 75
7,402 bitchute.com .com 42
7,427 nami.org .org 84
7,455 amazon.sg .sg 40
7,531 theonion.com .com 84
7,562 nettavisen.no .no 63
7,589 brasil247.com .com 80
7,692 jetpunk.com .com 46
7,738 pitchfork.com .com 81
7,856 popsugar.com .com 75
7,863 refinery29.com .com 78
7,981 afr.com .com 72
8,030 oceanofpdf.com .com 59
8,094 nefisyemektarifleri.com .com 76
8,100 loudecho.ai .ai 56
8,221 tubidy.llc .llc 75
8,270 bestjavporn.com .com 66
8,378 revistaoeste.com .com 69
8,380 veikkaus.fi .fi 67
8,385 emojicombos.com .com 47
8,412 anikototv.to .to 62
8,435 thecut.com .com 84
8,436 beatport.com .com 54
8,449 sextb.net .net 62
8,511 demonicscans.org .org 57
8,568 kriesi.at .at 77
8,606 javdb.com .com 52
8,621 momvids.com .com 50
Page 1 of 7Next

Rank is the domain's position in the popularity list used to seed the corpus; a dash means it was scanned on request rather than seeded. Score is the site's overall AI access score, which blocking depresses but does not by itself determine.

Check a specific domain

See also the Google-CloudVertexBot reference, the full census, and the JSON endpoint.