Category: Engineering
All blog posts in this category.

- 09 Nov, 2026
Bots, AI agents and visitors: why traffic metrics disagree
A team notices a surge in requests to its documentation. Browser analytics stays flat, search performance barely moves and there are no additional commercial inquiries. Is this an increase in readers, crawlers or personal AI-agent activity? On 4 November 2026, Cloudflare announced new visibility and access controls for personal agents that may consult pages or perform actions on behalf of an individual. To the website, those actions often look like automated traffic. An HTTP request, a JavaScript pageview and a human visit are not interchangeable units. They need separate definitions before their charts can be compared. What changed with personal-agent visibility Cloudflare describes personal agents as an intermediary between someone's instruction and a website. A person might ask an agent to compare products, retrieve help information or carry out a task. The site receives automated requests even if the underlying purpose originated with a person. The announcement introduces additional visibility and control mechanisms for those agents. It does not establish that every request represents one unique human or that all customers have identical functionality across products and plans. Cloudflare's bot documentation distinguishes behaviours such as search, training, agent activity and transactions. The taxonomy is more informative than dividing all requests into just "human" and "bot". Define the units before building a report HTTP request means a request observed by a server, CDN or network layer. One page can produce multiple requests for resources. An agent may fetch pages without launching a conventional browser. Browser pageview means an event emitted under a defined tracking rule. A missing script, disabled JavaScript or a blocked request can prevent the event. Visit or session means a grouping of events according to vendor-specific rules, often involving time boundaries and continuity. Different tools do not necessarily implement identical sessions. Crawler means an automated actor retrieving content for search, indexing or other collection purposes. Its request is not proof of a person reading the page. Personal agent means software performing tasks at least sometimes on an individual's instruction. Human-directed does not mean each request should count as a human visit. Assistant referral means a detectable browser entry from a link presented in an assistant interface. When available, that is an acquisition-source question, not the same thing as server requests made by an agent. Compare observation layersSource Mainly observes Cannot establish by itselfServer or CDN logs Requests and network metadata Number of people who read a pageBot classification Estimated or verified automated behaviour The precise human intent behind every requestBrowser analytics Events received under its configuration Every request or every visitorSearch Console Google Search performance under its definitions All AI agents accessing the websiteCRM or forms Accepted inquiries and sales outcomes Total content readershipKnown assistant referrers Detectable entries from named sources Non-clicked exposure in AI answersThis is not a ranking of which tool tells "the truth." Each observes a different layer. Accurate counters can disagree when their scopes differ. Why logs increase while analytics stays stable Crawlers may not execute tracking scripts Many automated clients retrieve HTML or resources without running browser JavaScript. They can appear in infrastructure records without becoming browser analytics events. That is not necessarily a measurement error. Human-focused analytics may deliberately exclude bots, while a firewall may log or block them before requests reach the application. One action produces several requests A single page load can trigger many resource and API calls. A scraper may request dozens of URLs within seconds. Request volume is not a direct count of human pageviews. Ratios change with page design, caching and crawling behaviour. Avoid a fixed multiplier that supposedly converts all network requests into visits. Time ranges and filters differ One tool may report in local time while logs use UTC. Bot filters, consent state, URL exclusions and processing delays may vary. Align timezone, paths, HTTP statuses and filtering before claiming that one system is inaccurate. Blocking changes which layer sees the request A CDN can record a request never forwarded to the origin. Network filtering can alter application logs without removing the original attempted access. Always document where a metric is observed. Five questions for investigating a spike First, which paths changed? A documentation surge has different implications from activity on a contact form. Include response codes and error rates. Second, which actors can be identified? Review available bot categories and verification signals. A declared User-Agent is not authoritative on its own. Third, what access mode is involved? Simple HTTP retrieval, JavaScript navigation, repeated actions and transactions create different observation patterns. Fourth, are periods comparable? Align timezone, filters, paths and collection rules. Record when access policies or tracking configurations changed. Fifth, what decision will the analysis support? It may be a bot access rule, an infrastructure improvement or a decision to keep this series outside human acquisition KPIs. An automated request may still serve a person The challenge is not to relabel every bot request as human traffic. It is to recognise that some automation serves real user needs. Blocking all agents can limit a useful access route. Allowing everything can create cost and security risks. Cloudflare frames its update around visibility and policy, not as a universal recommendation to allow every agent. A B2B SaaS company may apply different policies to public documentation and authenticated endpoints. Security control remains distinct from marketing analytics. AI-assistant referrals are another dataset Our guide to measuring assistant traffic covers visits that can be associated with known referrers. It is a different question from an agent retrieving a document directly. A person can read an AI response without clicking. An agent can request a page without generating a browser session. A real click can appear as direct traffic when referrer information is unavailable. Keep separate series for observed agent requests, observed website sessions and detectable assistant referrals. Do not add them into one unexplained "AI traffic" total. Multi-site reporting needs common definitions A business with marketing, product and documentation websites should compare the same unit on each property, not automatically merge all requests. Documentation may attract substantial automated exploration. That matters for infrastructure and access policy, but it is not by itself a lead-generation metric. Governance means assigning each metric a source, owner, calculation method and acceptable use. Fewer better-defined indicators can lead to stronger decisions. What web analytics can and cannot observe Lean web analytics can support observed website visits and useful actions without claiming to inventory all network activity. Pomelo's web reporting does not replace bot-management controls or WAF telemetry at the infrastructure layer. Use infrastructure measurements for agent activity where available, and keep their definitions separate from audience reporting. Make a reusable incident note, not an emergency traffic narrative When an automated traffic spike appears, create a small incident or observation note. Include the first and last affected times, the layer where requests were measured, the relevant path patterns, observed status codes, classification evidence and any access-control changes during the period. The note should distinguish facts from hypotheses. "The CDN observed more requests to documentation" is an observation. "AI assistants sent more paying prospects" is a commercial inference requiring separate evidence. If the second statement cannot be tested, it should not be presented as the explanation of the first. For public documentation, it can be useful to measure infrastructure cost and rate limits alongside audience trends. A bot-driven increase in bandwidth or cache misses may affect operations even when human visits are unchanged. Those costs deserve an engineering response, not an acquisition headline. Also consider benign automation. Some requests may be associated with accessibility tools, user-requested fetchers or legitimate indexing. Blanket rules based on an unverified User-Agent can block useful access while missing disguised automation. A periodic review of the incident log can reveal whether a new agent category genuinely changes infrastructure needs. It can also show that the apparent change was caused by a new classification rule or a modified reporting filter rather than by a new population of visitors. Conclusion: different scopes are not automatically defects A difference between logs and analytics is an investigation prompt, not immediate proof that one platform is wrong. Personal agents make the distinctions between requests, pageviews, visits and delegated actions even more important. Start with the observation layer, period and business question. Counters should match only when they genuinely measure the same event under the same conditions. FAQ Does one AI agent request count as one visit? No. HTTP requests and analytics sessions follow different definitions. Automated activity can be useful without being a human session. Why can CDN logs exceed browser analytics traffic? Network requests and browser events are different layers. Bots, blockers and filters commonly create differences. Can a User-Agent prove that a request came from a known agent? Not reliably by itself. Use documented verification and classification mechanisms while acknowledging their limits. Can referrals from ChatGPT or Perplexity be measured? Some clicks are observable when referrer information is present. That does not measure all agent requests or non-clicked AI answers. SourcesCloudflare, See the agents, set the terms, 4 November 2026 Cloudflare, verified bot classification Cloudflare, bot scores Cloudflare, Web Analytics FAQ Google Search Central, Google crawler overview

- 19 Oct, 2026
Single-page app analytics: a pre-launch QA checklist
An engineer adds a tracking script to a web application. The homepage appears in the dashboard and the installation check passes, so the task is marked complete. Yet internal routes barely appear in reporting. On other screens, each transition is recorded twice. Single-page applications can display many routes without reloading the original HTML document. A tool watching only initial page loads may miss those changes. A tool configured with both automatic history tracking and manual events may count the same view more than once. The acceptance question is therefore not "did the script load?" It is "does each meaningful screen change create the expected event, once, with the right context?" Why SPA navigation changes the measurement problem Traditional navigation usually loads a new document. A SPA router often changes visible content and sometimes the URL while keeping the same document alive. The History API supports adding and replacing history entries without a full reload. MDN documents an easy-to-miss detail: invoking "pushState" or "replaceState" does not itself fire "popstate". Back and forward navigation can fire that event, but using only "popstate" may miss internal route transitions initiated by application code. Overlapping listeners cause the opposite problem. A router hook, tag manager and automatic measurement can all consider the same transition a new page. Google's official SPA guidance recommends verifying page-view events and referrer values as users navigate. It also warns that combining methods can cause double counting. Define a meaningful page view first Not every state transition is a new page for reporting purposes. Opening a menu, changing a filter or viewing a modal may alter the interface without creating a new content screen. Write a definition that matches a decision. For a marketing website, a route displaying a distinct piece of content may count as a page view. For an authenticated product, an internal step may be better represented by a specific action event. What matters is a documented policy. A working definition might be: "Record one page view after successfully displaying a new content screen identified by its canonical path, if that change is a unit we need to report on." This is a deliberate local contract, not a universal standard. Also document intentional exclusions: UI filters, loading placeholders and private routes that should not be measured. Choose a small but representative route matrix Testing every path is rarely necessary to uncover the main classes of errors. Test distinct navigation behaviours, including an initial visit, an internal link, back, forward, redirects and query-parameter changes.Scenario Expected observationInitial direct visit One page view when collection is enabledInternal link to new content One new view for the displayed screenBrowser back Consistent return to the previous pathBrowser forward Path and previous-page context follow the definitionAutomatic redirect Final screen counted without unintended duplicatesFilter change on a list No new page view unless it constitutes a new screenModal open and close Separate event only if usefulFailed or interrupted loading Explicit policy for countingFictional sensitive query parameter Restricted value not transmittedRecord the previous path, next path, navigation method, outgoing request and reported event for each test. Six frequent failure patterns Only the initial page appears When measurement observes document load only, internal SPA routes can disappear from reporting. A multi-screen application looks like a one-page site, and engagement metrics become hard to interpret. Confirm whether the instrumented tracker is aware of the router and version actually deployed. Do not assume universal behaviour across frameworks. Every internal route is counted twice An automatic event and a manual event can describe the same screen. Inspect the network immediately after one navigation. Two outgoing requests require investigation, although they do not automatically prove two final rows without checking the full processing path. Google's pageview documentation shows how to disable automatic pageview measurement when manual tracking is used. A useful general rule is to nominate one authority for generating a page view. Referrer context is inconsistent The count may be right while the previous-page context is wrong. A documentation journey can then look as though it came from an older route or acquisition source. Inspect how the vendor defines "referrer." An HTTP Referer header and a virtual previous page reconstructed by the analytics product are not always interchangeable. URLs contain unexpected information Routes and queries may carry account IDs, search terms, email addresses or tokens. Sending the entire URL to analytics can expose information unnecessary for audience reporting. Use canonical route rules and parameter filtering, including on authenticated surfaces. Our URL filtering guide discusses practical examples. Use fictional distinctive values to verify that forbidden fields never leave the browser. Events belong to the wrong screen A conversion fired during navigation may be attached to the preceding route, a temporary URL or a reloaded confirmation. Check the order of view transitions, backend responses, outgoing events and report updates. A submit click alone does not establish a completed lead. Separate intent events from successful operations. Component lifecycle causes duplicate events Some development environments deliberately mount or replay component effects to expose lifecycle defects. A tracking effect that is not idempotent may send duplicate events. Compare a production build with development behaviour before labelling the issue a live incident. Even development-only duplicates indicate brittle instrumentation worth fixing. A practical four-stage QA method Use test accounts and synthetic input Keep the test isolated. Open browser network tools and, if the vendor offers one, a debug mode. Avoid adding invasive tracking of actual users simply to make testing easier. Execute the same journey twice Run a stable sequence, including direct entry, internal navigation, back and forward. Repeat in another browser when practical. The aim is to cover different navigation mechanics rather than accumulate arbitrary clicks. Compare three evidence layers Contrast the human action log, actual outgoing requests and final report. Differences may reflect delayed processing, filters, consent decisions or an instrumentation error. Check the documented data pipeline before declaring a missing result. Make the result actionable Log each defect with route, trigger, observed request, reporting impact and intended fix. Require a reproducible test before switching from automatic to manual route tracking. Repeat the essential cases whenever the router, form handling or analytics configuration changes. Protect data while testing SPAs can have complicated journeys without needing exhaustive collection. A compact set of page views and selected actions often answers a marketing or documentation team's questions. Do not send application account IDs or debug tokens to analytics simply because they are handy for troubleshooting. Distinguish technical logs, incident diagnostics and audience measurement. Consent and transparency obligations do not disappear because the application is a SPA. The CNIL's conditions for narrowly scoped audience measurement remain specific to purpose and implementation. How to evaluate a vendor's SPA support Ask which navigation mechanisms are detected, how duplicates are prevented, whether paths are normalised and which QA procedures are recommended. An analytics provider may require manual configuration for some routers. That is not necessarily a defect, provided it is documented and verifiable. For Pomelo as for competitors, compatibility should be tested on the actual application, deployed script version and configuration. A universal compatibility claim is not evidence that event collection works correctly in your application. Turn the QA matrix into a release safeguard A one-time manual test is useful, but its long-term value increases when it becomes part of ordinary releases. Record four journeys that matter most: direct entry to an internal route, client-side transitions, browser back and forward, and a confirmed form action. Where infrastructure allows, add browser automation that records outgoing analytics requests using synthetic accounts. Not every commit needs to verify the full live reporting pipeline. It can instead assert that a route change emits exactly one intended event with a canonical path and without a test secret in the payload. Reserve slower end-to-end checks for a controlled environment and a known release candidate. Separate evidence layers: unit tests for tracking helpers, browser tests for navigation behaviour and sampled report verification for the actual destination. When a check fails, classify it instead of adding a retry. A missing event, duplicate request and unexpected URL field are different defects. They require different corrections and owners. The benefit is consistency: changing a framework version, tag-manager rule or consent configuration can no longer silently rewrite metric definitions while dashboards keep rendering as if nothing changed. Conclusion: a page view is a tested contract Successful installation does not end when a tag loads. The evidence is agreement between a screen displayed, an event sent and a metric reported. Choose representative routes, exercise transitions, inspect payloads and retain the results. That is more convincing than a green installation indicator with no route-level verification. FAQ Should every click in a SPA send a pageview? No. A pageview should represent a meaningful content-screen transition, not every interaction. Important clicks may use distinct action events. Does popstate observe all router transitions? No. MDN explains that pushState and replaceState do not directly fire popstate. Application-driven navigation needs appropriate handling. Why are my views duplicated? Automatic and manual tracking can overlap. Inspect the outgoing requests and vendor configuration to identify the actual cause. Should full URLs always be transmitted? Not without review. Paths and query parameters can carry personal or sensitive information. Canonicalisation and filtering reduce unnecessary exposure. SourcesGoogle for Developers, measure single-page applications Google for Developers, measure pageviews MDN, working with the History API MDN, popstate event CNIL, audience measurement tools
Just the right measure, for all your websites.
Pomelo is opening gradually. Join the first teams measuring right and help shape what comes next.
