Tag: Traffic metrics
All blog posts with this tag.

- 09 Nov, 2026
Bots, AI agents and visitors: why traffic metrics disagree
A team notices a surge in requests to its documentation. Browser analytics stays flat, search performance barely moves and there are no additional commercial inquiries. Is this an increase in readers, crawlers or personal AI-agent activity? On 4 November 2026, Cloudflare announced new visibility and access controls for personal agents that may consult pages or perform actions on behalf of an individual. To the website, those actions often look like automated traffic. An HTTP request, a JavaScript pageview and a human visit are not interchangeable units. They need separate definitions before their charts can be compared. What changed with personal-agent visibility Cloudflare describes personal agents as an intermediary between someone's instruction and a website. A person might ask an agent to compare products, retrieve help information or carry out a task. The site receives automated requests even if the underlying purpose originated with a person. The announcement introduces additional visibility and control mechanisms for those agents. It does not establish that every request represents one unique human or that all customers have identical functionality across products and plans. Cloudflare's bot documentation distinguishes behaviours such as search, training, agent activity and transactions. The taxonomy is more informative than dividing all requests into just "human" and "bot". Define the units before building a report HTTP request means a request observed by a server, CDN or network layer. One page can produce multiple requests for resources. An agent may fetch pages without launching a conventional browser. Browser pageview means an event emitted under a defined tracking rule. A missing script, disabled JavaScript or a blocked request can prevent the event. Visit or session means a grouping of events according to vendor-specific rules, often involving time boundaries and continuity. Different tools do not necessarily implement identical sessions. Crawler means an automated actor retrieving content for search, indexing or other collection purposes. Its request is not proof of a person reading the page. Personal agent means software performing tasks at least sometimes on an individual's instruction. Human-directed does not mean each request should count as a human visit. Assistant referral means a detectable browser entry from a link presented in an assistant interface. When available, that is an acquisition-source question, not the same thing as server requests made by an agent. Compare observation layersSource Mainly observes Cannot establish by itselfServer or CDN logs Requests and network metadata Number of people who read a pageBot classification Estimated or verified automated behaviour The precise human intent behind every requestBrowser analytics Events received under its configuration Every request or every visitorSearch Console Google Search performance under its definitions All AI agents accessing the websiteCRM or forms Accepted inquiries and sales outcomes Total content readershipKnown assistant referrers Detectable entries from named sources Non-clicked exposure in AI answersThis is not a ranking of which tool tells "the truth." Each observes a different layer. Accurate counters can disagree when their scopes differ. Why logs increase while analytics stays stable Crawlers may not execute tracking scripts Many automated clients retrieve HTML or resources without running browser JavaScript. They can appear in infrastructure records without becoming browser analytics events. That is not necessarily a measurement error. Human-focused analytics may deliberately exclude bots, while a firewall may log or block them before requests reach the application. One action produces several requests A single page load can trigger many resource and API calls. A scraper may request dozens of URLs within seconds. Request volume is not a direct count of human pageviews. Ratios change with page design, caching and crawling behaviour. Avoid a fixed multiplier that supposedly converts all network requests into visits. Time ranges and filters differ One tool may report in local time while logs use UTC. Bot filters, consent state, URL exclusions and processing delays may vary. Align timezone, paths, HTTP statuses and filtering before claiming that one system is inaccurate. Blocking changes which layer sees the request A CDN can record a request never forwarded to the origin. Network filtering can alter application logs without removing the original attempted access. Always document where a metric is observed. Five questions for investigating a spike First, which paths changed? A documentation surge has different implications from activity on a contact form. Include response codes and error rates. Second, which actors can be identified? Review available bot categories and verification signals. A declared User-Agent is not authoritative on its own. Third, what access mode is involved? Simple HTTP retrieval, JavaScript navigation, repeated actions and transactions create different observation patterns. Fourth, are periods comparable? Align timezone, filters, paths and collection rules. Record when access policies or tracking configurations changed. Fifth, what decision will the analysis support? It may be a bot access rule, an infrastructure improvement or a decision to keep this series outside human acquisition KPIs. An automated request may still serve a person The challenge is not to relabel every bot request as human traffic. It is to recognise that some automation serves real user needs. Blocking all agents can limit a useful access route. Allowing everything can create cost and security risks. Cloudflare frames its update around visibility and policy, not as a universal recommendation to allow every agent. A B2B SaaS company may apply different policies to public documentation and authenticated endpoints. Security control remains distinct from marketing analytics. AI-assistant referrals are another dataset Our guide to measuring assistant traffic covers visits that can be associated with known referrers. It is a different question from an agent retrieving a document directly. A person can read an AI response without clicking. An agent can request a page without generating a browser session. A real click can appear as direct traffic when referrer information is unavailable. Keep separate series for observed agent requests, observed website sessions and detectable assistant referrals. Do not add them into one unexplained "AI traffic" total. Multi-site reporting needs common definitions A business with marketing, product and documentation websites should compare the same unit on each property, not automatically merge all requests. Documentation may attract substantial automated exploration. That matters for infrastructure and access policy, but it is not by itself a lead-generation metric. Governance means assigning each metric a source, owner, calculation method and acceptable use. Fewer better-defined indicators can lead to stronger decisions. What web analytics can and cannot observe Lean web analytics can support observed website visits and useful actions without claiming to inventory all network activity. Pomelo's web reporting does not replace bot-management controls or WAF telemetry at the infrastructure layer. Use infrastructure measurements for agent activity where available, and keep their definitions separate from audience reporting. Make a reusable incident note, not an emergency traffic narrative When an automated traffic spike appears, create a small incident or observation note. Include the first and last affected times, the layer where requests were measured, the relevant path patterns, observed status codes, classification evidence and any access-control changes during the period. The note should distinguish facts from hypotheses. "The CDN observed more requests to documentation" is an observation. "AI assistants sent more paying prospects" is a commercial inference requiring separate evidence. If the second statement cannot be tested, it should not be presented as the explanation of the first. For public documentation, it can be useful to measure infrastructure cost and rate limits alongside audience trends. A bot-driven increase in bandwidth or cache misses may affect operations even when human visits are unchanged. Those costs deserve an engineering response, not an acquisition headline. Also consider benign automation. Some requests may be associated with accessibility tools, user-requested fetchers or legitimate indexing. Blanket rules based on an unverified User-Agent can block useful access while missing disguised automation. A periodic review of the incident log can reveal whether a new agent category genuinely changes infrastructure needs. It can also show that the apparent change was caused by a new classification rule or a modified reporting filter rather than by a new population of visitors. Conclusion: different scopes are not automatically defects A difference between logs and analytics is an investigation prompt, not immediate proof that one platform is wrong. Personal agents make the distinctions between requests, pageviews, visits and delegated actions even more important. Start with the observation layer, period and business question. Counters should match only when they genuinely measure the same event under the same conditions. FAQ Does one AI agent request count as one visit? No. HTTP requests and analytics sessions follow different definitions. Automated activity can be useful without being a human session. Why can CDN logs exceed browser analytics traffic? Network requests and browser events are different layers. Bots, blockers and filters commonly create differences. Can a User-Agent prove that a request came from a known agent? Not reliably by itself. Use documented verification and classification mechanisms while acknowledging their limits. Can referrals from ChatGPT or Perplexity be measured? Some clicks are observable when referrer information is present. That does not measure all agent requests or non-clicked AI answers. SourcesCloudflare, See the agents, set the terms, 4 November 2026 Cloudflare, verified bot classification Cloudflare, bot scores Cloudflare, Web Analytics FAQ Google Search Central, Google crawler overview
Just the right measure, for all your websites.
Pomelo is opening gradually. Join the first teams measuring right and help shape what comes next.
