Blocklist · Google

Sites that block Google-CloudVertexBot

1,361 of the 29,991 domains we have measured, 5%, disallow google-cloudvertexbot at the site root in robots.txt. Google-CloudVertexBot fetches a page live when a user asks the assistant for it.

Every domain here has removed itself from Google's answers. Blocking a retrieval agent withholds nothing from model training, because training uses a different token.

The measurement

1,361 of 29,991 measured domains (5%) disallow google-cloudvertexbot at the site root in robots.txt. Every domain named on this page was fetched directly and its robots.txt parsed against RFC 9309; none of it is inferred from a third-party index.

Measured 2026-11-21 by direct request to every domain in the corpus. Method: how this is measured. Reuse under CC BY 4.0 with attribution.

Cite as: Crawl Census, "Sites that block Google-CloudVertexBot", measured 2026-11-21. https://crawlcensus.com/blocked/google-cloudvertexbot

The list 1,361 domains

RankDomainTLDAI access score
37,566 numberempire.com .com 73
37,570 spa.gov.sa .sa 42
37,671 aerobasegroup.com .com 68
37,700 rutracker.net .net 43
37,724 mmsvibex.net .net 81
37,757 elle.de .de 89
37,762 mywebnook.com .com 60
37,807 xn--72czpfe7gxb9cveycua.com .com 74
37,874 484r.com .com 55
38,007 wired.jp .jp 78
38,024 arab89.net .net 60
38,068 bully-manga.com .com 81
38,069 kyawgyi.com .com 69
38,118 abc30.com .com 81
38,148 news-press.com .com 73
38,176 saskatchewan.ca .ca 68
38,236 datasheetq.com .com 51
38,313 wingsmob.com .com 57
38,327 inforum.com .com 70
38,357 arenapro6.com .com 53
38,409 igt.com .com 48
38,420 kinipaham.com .com 37
38,437 ersnet.org .org 81
38,455 philanthropy.com .com 75
38,502 kantotplus.com .com 79
38,581 turboquestz.com .com 56
38,603 avj-th.com .com 79
38,609 teensexmix.site .site 71
38,612 xemvl.xxx .xxx 67
38,633 pufted.com .com 35
38,648 wfmu.org .org 54
38,708 lursoft.lv .lv 63
38,711 casinotest2.com .com 75
38,713 navyteams.com .com 77
38,752 ixiporn.club .club 68
38,790 passportindex.org .org 50
38,814 mas.to .to 31
38,836 winzaa.club .club 36
38,841 gameplayup.com .com 45
38,875 bluemoon-mcfc.co.uk .uk 50
38,945 postype.com .com 81
38,971 uwm.com .com 59
39,014 nagawinhoki.com .com 75
39,059 mozillafoundation.org .org 55
39,082 1fun.plus .plus 56
39,094 ind1.bet .bet 58
39,100 latestallgames.com .com 72
39,114 vildeey.site .site 43
39,115 vivicomi21.info .info 70
39,118 xn--3-5wf2bubl3iwbf1f5e8d.com .com 71
39,206 futurezone.de .de 86
39,255 grop.net .net 57
39,303 watoday.com.au .au 75
39,393 zakzak.co.jp .jp 72
39,570 roletapixbr.com .com 51
39,597 indifferentlanguages.com .com 52
39,657 korben.info .info 77
39,692 netdoktor.de .de 79
39,703 deepsilver.com .com 70
39,712 vsbattles.com .com 68
39,713 learner.org .org 65
39,824 werstreamt.es .es 59
39,976 aifpleasurebeh.org .org 35
40,005 bobdylan.com .com 58
40,049 hpcwire.com .com 63
40,102 eurasianet.org .org 56
40,114 cgm.com .com 75
40,163 worldnomads.com .com 81
40,258 listchallenges.com .com 48
40,317 iza.ne.jp .jp 79
40,322 neatorama.com .com 55
40,360 noen.at .at 79
40,487 opendatabot.ua .ua 58
40,638 main-echo.de .de 64
40,703 mrst.one .one 52
40,740 sheetjs.com .com 59
40,796 elpasotimes.com .com 73
40,817 cancilleria.gov.co .co 60
40,859 practicalmachinist.com .com 67
40,876 cntraveller.in .in 81
40,904 nesn.com .com 73
40,912 saambaa.com .com 66
40,961 wplace.live .live 73
40,971 amlegal.com .com 76
40,973 amainhobbies.com .com 47
41,014 4archive.org .org 67
41,083 summitpost.org .org 56
41,098 gqmagazine.fr .fr 82
41,203 analdin.xxx .xxx 44
41,227 musescore.org .org 55
41,308 sunbeltrentals.com .com 79
41,317 waymarking.com .com 48
41,336 archinect.com .com 68
41,357 mamul.am .am 66
41,420 prq.to .to 42
41,459 saharareporters.com .com 53
41,465 naughtyamerica.com .com 54
41,542 oefb.at .at 50
41,577 weblly.com .com 86
41,647 cjonline.com .com 73
41,652 flowmusic.app .app 64
41,824 smiletemplates.com .com 47
41,851 pixelfed.social .social 46
41,865 thehealthy.com .com 87
41,872 vanityfair.fr .fr 66
41,947 vogue.co.jp .jp 65
41,958 theledger.com .com 74
41,987 intermountainhealthcare.org .org 63
42,010 aprs.fi .fi 41
42,099 ephotozine.com .com 64
42,160 tcpalm.com .com 73
42,192 weserv.nl .nl 55
42,222 boomi.in .in 77
42,248 architecture.com .com 59
42,289 warosu.org .org 47
42,293 tfd.com .com 69
42,336 eweka.nl .nl 75
42,353 priuschat.com .com 55
42,423 welib.org .org 49
42,505 medietall.no .no 48
42,512 tech-ties.net .net 85
42,592 erowid.org .org 40
42,607 wruser.org .org 43
42,614 citizen-times.com .com 74
42,670 wy903.com .com 53
42,698 sol.no .no 74
42,800 1000kitap.com .com 42
42,835 iodonna.it .it 86
42,842 bahamaslocal.com .com 61
42,849 hoganlovells.com .com 81
42,874 kone.gg .gg 58
42,919 tcrf.net .net 29
42,934 v2006.com .com 49
43,117 polycasino.com .com 76
43,132 jtbc.co.kr .kr 70
43,218 sermoncentral.com .com 59
43,268 candlepowerforums.com .com 61
43,281 xvix.eu .eu 75
43,302 seacoastonline.com .com 69
43,309 tonkeeper.com .com 57
43,315 stateline.org .org 69
43,414 cameradb.review .review 62
43,415 pokeheroes.com .com 57
43,455 noi.md .md 54
43,480 remoteok.com .com 76
43,544 nautiljon.com .com 62
43,587 admagazine.fr .fr 81
43,667 sweclockers.com .com 63
43,742 apsnet.org .org 50
43,802 fussballdaten.de .de 76
43,986 nn.de .de 83
44,026 gainesville.com .com 66
44,043 matters.town .town 56
44,055 270towin.com .com 72
44,057 currencyaz.com .com 55
44,151 betterbanned.com .com 71
44,155 lbbonline.com .com 70
44,183 barcablaugranes.com .com 68
44,230 mcpvp.club .club 50
44,255 cagesideseats.com .com 72
44,313 um.ac.id .id 83
44,404 umm.edu .edu 77
44,418 adoptimix.com .com 68
44,440 hitmaal.io .io 71
44,489 iha.com.tr .tr 52
44,556 tritan.gg .gg 61
44,575 flgov.com .com 54
44,635 wowinterface.com .com 58
44,689 notthebee.com .com 79
44,780 virgool.io .io 58
44,807 gofeminin.de .de 74
44,829 mobilism.org .org 48
44,874 news-leader.com .com 62
44,878 rtvnoord.nl .nl 73
44,880 santaads.com .com 73
44,904 popup-blocker.org .org 56
44,912 audima.co .co 80
44,928 wgt.com .com 80
44,968 savannahnow.com .com 64
45,003 preciso.net .net 57
45,120 cmha.ca .ca 76
45,150 samplefocus.com .com 60
45,157 durev-inc.xyz .xyz 59
45,159 lansingstatejournal.com .com 62
45,165 parkpage.in .in 41
45,216 tbcpl.lol .lol 59
45,240 vogue.mx .mx 65
45,271 deckshop.pro .pro 63
45,272 rtvutrecht.nl .nl 73
45,372 devid.info .info 57
45,461 lgbtqnation.com .com 72
45,482 iyottube.ph .ph 68
45,509 hysky.de .de 38
45,541 classiccat.net .net 50
45,571 wz.de .de 76
45,578 chugoku-np.co.jp .jp 80
45,602 kenhub.com .com 66
45,604 beaconjournal.com .com 66
45,605 soccer365.net .net 47
45,663 autoauctionhistory.com .com 46
PreviousPage 6 of 7Next

Rank is the domain's position in the popularity list used to seed the corpus; a dash means it was scanned on request rather than seeded. Score is the site's overall AI access score, which blocking depresses but does not by itself determine.

Check a specific domain

See also the Google-CloudVertexBot reference, the full census, and the JSON endpoint.