AI crawler · registry

Timpibot

Timpi · Model training · robots.txt token timpibot

honors robots.txtMeasured across 3,958 sites with a robots.txt policy record for this agent, out of 7,068 on record.
5%
of measured sites block it
206
sites blocking
3,752
sites allowing
yes
honors robots.txt

What it is definition

Crawls pages for Timpi's decentralized index, which is also used as LLM training data.

Timpibot is operated by Timpi and classified here as model training. These agents collect text and images that may end up in a training corpus. Disallowing one withholds new content from the next model. It does not withdraw anything collected before the disallow, and it does not remove the site from any answer engine.

Disallowing timpibot withholds content crawled from that point on from Timpi's training data. It does not withdraw pages collected in earlier crawls, it does not reach copies already sitting in third-party datasets, and it does not remove the site from any answer surface.

robots.txt token
timpibot
Operator
Timpi
Purpose
Model training
Honors robots.txt
yes
Documentation
https://timpi.io/

Identity on the wire exact string

User-agent string
Timpibot/1.0 (+http://timpi.io/crawler)
Verify your own edge is not refusing it
curl -A 'Timpibot/1.0 (+http://timpi.io/crawler)' https://example.com/

Replace the host with your own. A 403, a challenge page or an empty body means the edge is blocking the agent whatever robots.txt says.

How to allow it, how to block it robots.txt

Allow it
User-agent: timpibot
Allow: /

Absent any rule, the agent is already allowed, so this group only matters when your * group disallows something. Naming the agent replaces the * group for it entirely, so repeat here every disallow you still want to apply to it.

Block it
User-agent: timpibot
Disallow: /

Disallowing timpibot withholds content crawled from that point on from Timpi's training data. It does not withdraw pages collected in earlier crawls, it does not reach copies already sitting in third-party datasets, and it does not remove the site from any answer surface.

Sites that block it 60 shown

DomainAI access scoreRank
amazon.com 33 26
msn.com 34 62
amazonvideo.com 55 121
snapchat.com 63 125
nytimes.com 67 157
cnn.com 50 190
canva.com 46 239
launchpad.net 49 260
amazon.co.uk 33 318
amazon.de 33 377
businessinsider.com 84 426
amazon.co.jp 33 494
wired.com 74 533
patreon.com 60 543
usatoday.com 55 552
uol.com.br 74 571
telegraph.co.uk 45 580
amazon.fr 33 609
amazon.co.za 33 623
dailymail.co.uk 63 627
nintendo.com 59 628
amazon.ca 33 635
amazon.in 34 659
cnet.com 75 662
lemonde.fr 72 671
dailymail.com 63 681
primevideo.com 55 682
amazon.it 33 706
amazon.es 33 725
ted.com 63 726
bild.de 74 763
apnews.com 75 787
welt.de 65 800
people.com 38 801
amazon.com.au 33 812
theverge.com 68 815
amazon.com.br 33 833
investopedia.com 38 913
genius.com 60 926
healthline.com 70 941
repubblica.it 77 942
corriere.it 83 955
theatlantic.com 68 980
theconversation.com 52 1,019
huffpost.com 82 1,027
letterboxd.com 40 1,031
amazon.com.mx 33 1,032
spiegel.de 72 1,042
bfmtv.com 71 1,073
flashscore.com 67 1,074
cookpad.com 63 1,105
pimpbunny.com 56 1,177
aftonbladet.se 71 1,189
ad.nl 47 1,190
nu.nl 47 1,191
vnexpress.net 81 1,196
vg.no 73 1,208
furaffinity.net 41 1,223
pandora.com 45 1,232
hln.be 47 1,251

Compare Timpi

Timpi runs no other agent in this registry.

Common questions answered

Does blocking Timpibot remove me from the Timpi index?

No, not by itself. Timpibot collects content for training, not for the index that answers questions. Timpi operates no separate answer crawler in this registry, so there is no answer surface to lose here. Content collected before the disallow is not withdrawn.

How do I verify Timpibot requests are genuine?

Timpi publishes neither reverse DNS names nor address ranges for Timpibot, so a request carrying this user agent cannot be authenticated. The string is trivially spoofed and is routinely reused by unrelated scrapers. Treat it as a hint, rate-limit by source address, and do not grant it access you would not grant an anonymous client.

Check your own site free

See whether your robots.txt admits Timpibot today, and whether your edge agrees with it.