Don't filter the bots. Interview them.
An agent is a human, one step removed.
A customer pinged us last week, a little confused. A handful of their users had gone vertical -- big, sustained spikes in product activity. The kind of engagement curve you’d frame and put on the wall.
Turns out it was AI agents doing all the work, making it look like the individuals deployed them were super busy. These are bots those users had deliberately let loose inside the product -- not malicious, in fact very intentionally put there, part of a platform shifting from “do automations” to “be proactive.”
So despite the human users looking like they were on fire, the agents were doing the work.
Then I went and looked at our own website. PostHog reckons about 38% of our pageviews over the last 90 days are bots -- and that’s a conservative floor, sitting on top of the bots PostHog already drops before we ever see them.
An army of 267 “visitors” crawling the whole site, frozen on one Firefox version, running out of AWS datacenters. Strangely around the same time, I’d been getting Intercom messages and emails from new SaaS clearinghouses asking to list us. The traffic spike and the scraper outreach were probably the same story. I just hadn’t put them next to each other until now.
We already know bots and/or agents have overtaken humans on the internet. Expected. The question that actually matters isn’t whether it’s happening. It’s what it means for us, as builders and product owners reading data to decide what to build next.
My first instinct was wrong
My first reaction was to filter it all out and move on. Strip the noise. Get back to the humans. Maybe that was how you felt too.
But first we have to ask what value is attached to that traffic. And to answer that, we have to know what job the bot was sent to do.
If we’re talking agents, well they’re just a human abstracted away one more step. The agent does a pile of work, and at some point a human acts on it. Your product might be at arm’s length from that person now, but the activity isn’t valueless just because no person clicked the touchpad.
There are going to be entirely new jobs to be done that agents unlock -- work a human would never have done by hand. So “bot activity is just human activity, amplified” is a bad assumption. I’m betting much of it will take a different shape entirely.
Valuable job vs. vanity job
So how do you tell a valuable bot job from a vanity one? I don’t have a clean answer. But here’s where I’m landing for now:
Watch for volume swamping value. An agent that fires the same action a thousand times isn’t a thousand times more engaged. That’s the noisy bit -- the part you weight down, segment out, and look at on its own rather than letting it flood the signal.
Then do the unscalable thing: ask. Talk to the customer. What agents do you have running? What are they doing? Why? What are you hoping to get out of them? You’ll learn more in one call than in a month of staring at event counts.
Not all bots are pointed at you
There’s at least two flavors here, probably more. Worth noting, I’m using the term “bots” when it could be the traditional idea of a web scraper bot or an AI agent:
Bots working for your customer, inside your product. Think of Atlassian’s AI, Rovo. Rovo’s agents could be using your SaaS to get a job done. These can be real value -- a customer getting more out of you, just one layer removed.
Bots working for someone else, hitting your surface. AI crawlers reading your site, scrapers, eventually browser-based computer-use agents doing real work in your product on someone’s behalf.
The value splits hard inside that second group. A crawler that reads your site so ChatGPT can surface you in the right query? That might be really valuable traffic.
A scraping bot lifting your pricing and describing what you do (poorly, I might add)? That’s someone taking, not a customer arriving. It all takes the same shape in the logs, but has real opposite value. You can’t tell them apart without asking what job they’re on.
Why this isn’t academic
If you can’t separate human from agent in your engagement data:
Health scores that mean nothing
Activity rankings are built on ghosts
Usage-and-revenue forecasts ride on work no person did
We can start optimizing for the wrong user -- building toward the wrong customer because we can’t (or don’t) see who’s actually getting value.
A funny irony? I’m using AI tooling to sort the AI noise from the signal. No idea how accurate it is. Feels like the whole problem in miniature. Anyway.
So, I don’t have a neat bow to put on this one.
If you’re reading any data that touches customer or public engagement, this only gets more important: learn to split it -- human, agent, bot -- and trace where the value actually lands.
We can only keep building for the right entity on the other end if we can still tell who or what they are and what they need to get done.
Ai ai ai,
Peter