We Pointed an AI Agent at Our Own Website for a Weekend. Here Is Every Lie Our Marketing Data Was Telling Us.

Your next customer might be asking ChatGPT who to hire instead of Googling it. That is a real shift, and there is a whole industry forming around it. But before you spend a dollar improving the answer an AI gives about your company, there is a cheaper question worth asking first: are the numbers you would use to measure any of this actually true?

Ours weren't. Over the weekend of August 8–9, 2026, we pointed an AI coding agent at our own website — not to write marketing copy, but to audit the machinery underneath it. Seven distinct bugs. Five of them were not producing errors. They were producing wrong numbers, quietly, and had been for weeks.

A silent measurement bug is worse than an outage. An outage stops you. A silent bug lets you keep making decisions, confidently, on data that is fiction.

This is the whole method — what we asked the AI to do, what it found, how we verified it, and what we changed. If you run a business website and you have never audited your own measurement stack, most of this applies to you directly. Where it does, there is a "check this on your own site" note you can hand to whoever runs your website.

The scoreboard, up front

• A third of our recorded "website traffic" was our own staff working inside our internal portal.
• Our best-looking sales lead of the month — 85 pageviews, nearly 8 hours on site, a real named prospect at a real company — was one of our own reps testing a link.
• Visitors arriving from ChatGPT were being filed in the same bucket as random backlinks, despite behaving nothing like them.
• Our Google Search Console sync had never successfully written a single row. Not "wrote a few." Zero. It reported success the whole time.
• Our robots.txt was explicitly inviting Google, Bing and every AI crawler into the exact private paths the file was written to protect. Our login page was indexed and ranking around position 3.5.
• The file we publish specifically for AI crawlers to read — the one that tells an AI what our company does — did not contain our services. At all.
• Our own AI-visibility check reported 0% on its first run. The real number was 100% on those prompts. The tool was broken, in the pessimistic direction.

Every one of those systems reported healthy.

Why we used an AI coding agent and not an SEO tool

This is the part most people get wrong, so it is worth being precise.

An SEO tool tells you what is wrong with a page. It scores your titles, flags your missing alt text, and measures your load time. Those are real things and worth fixing.

What an SEO tool cannot tell you is that your Search Console sync has been throwing a database schema error into a swallowed error handler for weeks, or that a third of your pageviews are staff. Those are code problems wearing an analytics costume. Finding them means reading the tracking script, the data ingest route, the database schema, the security policies and the API client — at the same time, and holding all of it in one head at once.

That is the specific thing an AI agent with access to your codebase is unusually good at, and it is a different job from "help me with SEO." Note what it is not: it is not the chatbot in a browser tab. It is an agent running in a terminal with permission to read your repository, query your database, run your test suite, and fetch live pages. (We used Claude Code; the technique matters more than the brand.)

The framing we used all weekend was:

Here is the repository. Here is read access to the production database. Here are the API credentials. Do not trust the dashboard. Verify every claim against the source.

That last sentence did most of the work. Everything valuable this weekend came from refusing to accept a system's account of itself.

Phase 1 — Assume your analytics are lying until proven otherwise

We started here on purpose. Optimizing against a corrupted metric is worse than not optimizing at all, because it feels like progress.

A third of our traffic was us

Our first-party tracking script was loaded in the site's root layout. Sensible-looking decision. It also meant the script ran on our logged-in staff portal exactly as it ran on the public marketing site. Someone had noticed this at some point and left a comment saying internal pageviews were "harmless."

They were not. The page we open to review our traffic was outranking real marketing landing pages in our own top-pages report. Every conclusion drawn from that report was skewed by the act of reading it.

The same bug put the wrong human's name on a real session

This is the one that made us start treating measurement bugs as incidents.

We mail postcards with QR codes on them. Scanning one carries a tracking parameter that tells our system "the browser holding this link belongs to that specific contact." That association is stored in a browser cookie that outlives the click by two years.

So when one of our own reps opened a postcard's QR link to check that it worked, they permanently stamped the recipient's identity onto their own work browser. Our traffic list then reported a full day of internal CRM use as a named prospect at a named company: 85 pageviews, 7 hours 56 minutes on site.

Every number in that row was real. The name on it was not.

That is the most dangerous shape a data bug can take, because it does not look like a bug. It looks like the best lead of the month — the kind of row that gets forwarded to a salesperson, who calls a real person and references activity that person never performed.

The fix rests on two rules, both enforced on the server and not just in the JavaScript:

1. A page inside an internal path is not website traffic. Drop the event entirely.
2. A browser carrying a staff login cookie belongs to a colleague, not to a mailing recipient. Never let a tracking link attach a customer identity to it.

Why server-side, when the tracking script could just check? Because the script is a cacheable file served to browsers we do not control, and a write you refuse cannot be un-written later. Deleting bad data afterwards is a cleanup project. Never accepting it is a rule.

There is a subtlety here the agent caught and we would have missed. Our analytics platform is multi-tenant — client sites run the same tracker. A client is entitled to have a page called "/portal" that is genuinely public marketing traffic. Blanket-deleting it would have been a second silent bug, introduced by the fix for the first. So the exclusion checks whether the tracked page belongs to this deployment, using headers the browser sets and page scripts cannot forge.

Our highest-intent visitors were filed as junk

Traffic arriving from ChatGPT is neither search nor social, so every one of those sessions landed in the generic "Referral" bucket next to random backlinks.

That is a bad place for them, because the behaviour is nothing alike. One ChatGPT arrival in our records read fourteen pages over three hours after landing on our Eau Claire page. That is not a backlink click. That is someone doing vendor research who has already been told you are a candidate.

We gave AI assistants their own traffic channel and backfilled the history, because the channel is stamped when the visit is recorded — old rows would otherwise stay wrong forever.

Worth stating plainly, because it gets confused constantly: this is the opposite of an AI crawler. GPTBot and its relatives fetch your HTML on a server and never run a JavaScript tracker, so they are invisible to your analytics. Everything in this channel is a human who read an AI answer and clicked through. It is the only direct, first-party evidence you get that AI recommendations are sending you people.

Check this on your own site. Open your analytics, look at the top 20 pages, and ask two questions: is anything in that list a page only staff can see, and does your referral list have a line for AI assistants? If internal pages are in there, your entire baseline is inflated by an unknown amount — and every "we improved 12%" claim you make from it is unfalsifiable.

Phase 2 — Getting the Google plumbing actually connected

This was three separate problems stacked on top of each other, each one hiding the next.

The 403 that means two completely different things

Every Google API call was returning 403. The obvious reading is a permissions problem: go re-grant account access, check who owns what. We would have burned hours there.

A Google 403 has two entirely distinct causes:

• Permission denied — the API is on and answering, and it is telling you this account genuinely cannot read this resource. Fix: grant access.
• Service disabled — the API itself is switched off in the Google Cloud project behind your credentials. Nobody's permissions are wrong. A project-level toggle has never been flipped, and no amount of granting access will ever fix it.

They arrive with the same HTTP status code. The distinguishing signal is buried in the response body, along with the exact one-click activation link.

We built a small classifier that reads that signal properly, including a fallback that parses Google's own prose message for the cases where the structured detail is missing. The remediation text now names the right actor: "a Google Cloud owner enables it here" versus "the connected account was never granted access." Those are two different people, and telling the wrong one to act is how a problem sits unresolved for a month.

The root cause: the Analytics Admin API had simply never been enabled in our Cloud project. One toggle.

The GA4 property ID that did not exist

Our audit config held a property ID for a property that does not exist in our account. Almost certainly a stream ID pasted into the property field at some point in the past — two long numbers, two adjacent fields, no validation anywhere.

The audit had been reporting "cannot read GA4" ever since, and the error looked like a permissions problem, which is exactly why it survived so long. A wrong ID and a missing permission produce the same red box.

The response was not just to correct the value. It was to build a probe that asks Google directly: which GA4 properties can this account see, and which one's data stream points at this website? Then offer the answer as a one-click apply. The rule behind it:

Never ask a human a question the software can answer. What should remain for a person is only what the credentials genuinely forbid.

And the corollary that keeps it honest: a probe that fails degrades to "unknown", never to a wrong claim. The page says "I could not check this" rather than inventing a state.

The Search Console sync that could never have worked

We connected Search Console on August 6. The "search terms" panel stayed empty. Easy to blame on the connection being fresh — Google does take a few days.

The real cause was that the sync was writing to a database schema that our data API does not expose. Every write returned an "invalid schema" error. That call could never have succeeded, on any day, with any amount of data. It reported success while writing an empty result set, and only actually failed once there were rows to write — at which point the failure was swallowed.

The reading side had always worked, because it read the data through a different, correctly-exposed path. That mismatched pair — a read that works and a write that cannot — is exactly why nobody noticed. The dashboard was not broken. It was empty, and empty looks like "no data yet."

There was one more trap inside the fix worth knowing if you ever handle Search Console data yourself. Google returns one row per (search term, full URL), and we collapse URLs down to site-relative paths — which legitimately merges http://coulee.tech/ and https://www.coulee.tech/ into a single /. That produces two rows with an identical key, and the database rejects the entire batch. The fix merges duplicates before writing: clicks and impressions add up, and average position becomes an impression-weighted mean. A plain average would let a single-impression outlier sitting at position 90 drag a genuinely good page's number into the weeds.

Result: 253 search-term-by-page rows landed. Our first real search data.

Check this on your own site. Ask whoever runs your website one question: "When did we last confirm Search Console data is actually arriving in our reporting — not that it is connected, that rows are landing?" A connection status of "connected" is not evidence of a working pipeline. Ours said connected for days while writing nothing.

Phase 3 — Can the crawlers actually read you?

The robots.txt bug that indexed our login page

This is the one worth reading twice, because almost everyone gets the rule wrong.

Groups in a robots.txt file do not inherit from each other. A crawler obeys exactly one group — the one whose user-agent line matches it most specifically — and it never falls back to the catch-all group for extra rules.

Our file looked responsible. It had a catch-all group blocking our portal, our API and our auth paths. Then it had a friendly group listing Googlebot, Bingbot, GPTBot and the other AI crawlers, saying Allow: /.

Which meant every crawler we actually cared about was explicitly invited into the exact paths the blocklist existed to protect. Search Console confirmed it: our login page was indexed, ranking around position 3.5.

The rewrite repeats the full blocklist inside every group, adds the modern AI crawler names (OAI-SearchBot, ClaudeBot, Perplexity-User, Applebot-Extended, CCBot, Amazonbot and others), and excludes our per-recipient tracking links, because those are personalized landing pages, not content — indexing them pollutes click attribution.

We also added a noindex directive to the portal itself, because robots.txt controls crawling, not indexing. A page linked from anywhere else can be indexed without ever being fetched. Those are two separate controls, and blocking one does not accomplish the other. If you take one technical fact from this article, make it that one.

Our llms.txt described our blog and never mentioned what we sell

/llms.txt is a plain-text overview of a site, published for AI crawlers to read. Ours is generated live from our database.

The Services section was entirely missing. The file described our blog posts, our cheat sheets, our office locations and our event calendar — and never once stated what the company does.

The cause: the page is public, so it queries the database as an anonymous visitor. Our locations table had a rule permitting public reads. Our resources table had one. Our services table had none. The query returned zero rows, the generator skips empty sections, and the file shipped silently incomplete.

An AI reading that file would conclude that Coulee Tech publishes Excel shortcut guides.

Then, with the section restored, the next pass caught that every service link in it was a 404 — a path-construction bug where a category page was being emitted one level too deep. A broken link in the one file AI crawlers read to learn what you sell is a service they cannot cite.

Check this on your own site. Open yoursite.com/robots.txt in a browser. If there is more than one User-agent: group, confirm every group repeats the disallow rules — do not assume they inherit. Then search Google for site:yoursite.com and look for anything private in the results. It takes four minutes and it is the highest-value four minutes in this article.

Phase 4 — Measuring GEO, when there is no ranking report

GEO — Generative Engine Optimization — is where most of the available advice is vendor marketing. So the first thing we did was write down what is actually measurable, which is a short list.

Nobody outside OpenAI, Anthropic and Google can see why an assistant names one vendor and not another. There is no Search Console equivalent, no rank tracker, no impression count. That leaves exactly two honest feedback loops:

1. Referral traffic carrying an AI assistant as the source — which only exists in the minority of cases where the assistant links out at all, and where the human clicks.
2. Asking the assistants buyer-intent questions on a schedule and recording whether you get named.

We built the second one: a script that runs a fixed set of eight prompts — phrased the way a business owner actually types, not the way a marketer phrases a keyword — through several models, and stores every result so that "did we get named" becomes a time series rather than an anecdote.

Three design decisions in there are the entire value of the tool, and you should copy all three if you build your own:

Prompt wording is frozen; each prompt has a stable ID. If you reword a prompt, it becomes a new prompt with a new ID. Otherwise your trend line silently compares two different questions and you will draw a conclusion from the edit rather than the market.

Training mode and search mode measure different things and must never be averaged. Querying a model through an API with no web access tells you what it absorbed during training — "are we known?" — which moves on the timescale of model releases, i.e. slowly, and is nearly unactionable in a quarter. Turning on web search approximates what the real ChatGPT product does — "are we findable and citable right now?" — and that is the one that responds to things you change this month. We store which mode produced each answer, so the two series never contaminate each other.

An empty answer is missing data, not a "no." Our first run reported 0% visibility. The real answer was 100% on those prompts. Reasoning models bill their internal thinking against the response limit; with the limit set low, the model spent its entire allowance thinking and returned nothing. Our script scored that as "not mentioned."

That is the exact failure shape that makes a metric lie pessimistically while looking perfectly healthy — and pessimistic lies get acted on, because they look like a problem to fix. The script now records that case as an error in its own column and excludes it from the score entirely.

Our actual baseline

Named in 3 of 8 buyer-intent prompts. First place on the direct brand question. Second for managed IT in La Crosse. Sixth for IT support in Eau Claire. Not named at all for Fort Myers IT, AI adoption consulting, western-Wisconsin cybersecurity, accounting-software IT, or dealership IT.

The pattern is coherent, and it matches our Search Console data exactly: we are known as a Wisconsin MSP, and not as an AI consultancy, a vertical specialist, or a Florida presence — despite having an office in Fort Myers and published pages on every one of those topics.

Two things from the raw answers were more useful than the score:

• The assistant cited our manufacturing page and our Eau Claire location page directly. Those pages are doing real work, and now we know which ones.
• It described our services page as listing a market as "coming soon" that we closed in July 2026 and removed from the site. An answer engine was confidently repeating a stale snapshot of us to anyone who asked.

That last one is the entire argument for the next thing we built.

IndexNow, and why it matters more than it looks

IndexNow is a protocol that pushes a "this URL changed" notification straight to participating search engines instead of waiting to be crawled. Google has publicly said it does not use IndexNow, so expectations there should be zero.

But Bing is a participant, and ChatGPT's search reads from Bing's index. Our own analytics show ChatGPT is currently 100% of our measured AI-assistant referral traffic. So the delay between publishing a page and Bing knowing about it is, in practice, the delay before an answer engine can cite it. Crawl-and-wait takes days. An IndexNow submission is acknowledged in milliseconds.

Do this yourself, for free. Once a month, open ChatGPT (and Google's AI mode, and Perplexity) and type the five questions a customer would actually type to find a company like yours — "who should I hire for X in [your city]" — and write down whether you are named and who is. Put it in a spreadsheet with the date. That is a legitimate GEO baseline, it costs nothing but twenty minutes, and it is more than most of your competitors have.

Phase 5 — Competitive analysis against real data instead of vibes

With Search Console finally writing rows, we had 253 search-term-by-page pairs covering one week. Small, but real. Here is what honest data said:

Every single click was a brand click. Three clicks that week, all on the query "coulee tech." Non-brand clicks: zero. Average position across all 253 rows sat in the 40s.

That reframes the whole exercise. We were not losing a ranking battle with competitors — we were not in the battle. That is the profile of a site with good technical hygiene and no search authority yet. It is an uncomfortable number to publish, and it is the most useful one we have, because every future number is measured against it.

One market was carrying everything. Our Fort Myers location page pulled 74 impressions across 54 distinct search terms in one week — more than our homepage, and seven times our La Crosse page. The terms were pure commercial intent, and every one of them sat between position 14 and 43. Page two and three.

That is the highest-return cluster on the site by a distance: the demand is proven, the page is already relevant, and moving from 18 to 8 is a far smaller lift than manufacturing demand from nothing. This is the shape of finding you cannot get from a keyword tool — it comes from your own impression data, and it tells you where you are nearly winning.

We were bleeding relevance into markets we do not serve. A meaningful share of impressions came from city-modified searches in metros where we have no presence at all, ranking 50 to 99 across four states. That is not opportunity, it is dilution: our service pages carried no geographic signal whatsoever, so Google was guessing where we belong.

One national niche was sitting unclaimed. A single EDI services page pulled 31 impressions across 21 distinct terms at positions 41–79 — an under-served business-to-business niche with high deal value, on a page that already existed and had never been touched.

Then the competitive read, which came from having the agent fetch and analyze the actual page markup of the firms ranking above us. The gap list, ordered by what it costs us:

1. Reviews. The strongest competitor in our home market displays 100+ five-star Google reviews. We had three testimonials on the site and no review markup anywhere in the code. In local search, this is the single largest gap, and it is not a technical fix — it is asking customers.
2. Case studies. We have a /case-studies page and no case studies. The route exists; the asset does not.
3. Compliance proof. Competitors lead with SOC 2 Type 2 and framework alignments. For a healthcare or dealership buyer, that gap decides the shortlist before a conversation happens.
4. Named humans. Our location pages named nobody. Local search and AI assistants both reward specificity about real people at real addresses.
5. Local content volume. 3 of 61 published posts carried any local geography at all.

And the observation that made the priority order obvious: one of the firms outranking us in our own home market does it from an out-of-state headquarters, with no local address, no reviews and no testimonials — purely on the strength of systematically built location-by-service pages. If that structure alone earns page one here, it is worth running with genuine local substance behind it.

Phase 6 — What we actually shipped

Fifteen location-by-service pages across all three markets, generated from one shared template with proper structured data, driven by a single canonical facts file that also governs which pages the sitemap advertises. That coupling was deliberate: the sitemap previously advertised the same two sub-pages for every location on an assumption nothing enforced — so adding a new office would have immediately advertised two 404s to Google.

A local-SEO pass over exactly five of our sixty-one published posts. The obvious move after "only 3 of 61 posts mention a city" is to sprinkle city names across the blog. That is the wrong move, and it is worth understanding why: twenty of those posts are keyboard-shortcut and AI-tool cheat sheets with no local intent behind them. Adding a city name to a PowerPoint shortcuts page is keyword stuffing. It earns a relevance penalty, not a ranking.

The five we chose were already geographic or vertical and were stopping just short of the specificity that ranks. And the real point of that edit was not the city names in the prose — it was internal links. The fifteen new pages had almost nothing pointing at them, and a page nothing links to is a page search engines treat as unimportant. Those edits put contextual links from our most topically relevant existing content onto exactly those targets.

A 35-post editorial calendar weighted by the actual search data rather than a generic industry topic list — eight posts feeding the Fort Myers cluster because that is where the proven near-misses are, seven on verticals where AI assistants name us in none of them, six on service searches already surfacing at positions 33–91.

How to actually work with an AI agent — six rules that made the difference

This is the transferable part, and it applies whether you are auditing a website, cleaning a customer database, or automating a quoting process.

1. Point it at a real system, not at a blank page. The value here came from an agent that could read the code, query the database and fetch live pages. Asking a chatbot "how do I improve my SEO" gets you a listicle. Giving an agent your actual system gets you the seven things that are specifically wrong with yours.

2. Make "verify against the source" an explicit instruction. The single most valuable sentence of the weekend was the one refusing to let the agent believe a dashboard. Every claim got checked against the database, the raw API response, or the rendered page. AI is agreeable by default. Tell it not to be.

3. The repository is the memory. Every non-obvious decision got written into the file it affects, as a comment explaining why — not what. That is not documentation theatre. It is what stops the next session, human or AI, from re-deriving it or, worse, quietly undoing it. If you are using AI on your business, the equivalent is writing your decisions down where the work lives, not in a chat log you will never re-read.

4. Put a gate the agent cannot bypass between it and production. Every change went through our build pipeline: type checks, linting, unit tests, and a full production build before anything could be deployed. Any failure blocks it. An agent that can ship without a gate is a liability. An agent working behind one is just a fast colleague.

5. Never let a system report success on unverified state. This came up three separate times in one weekend — the sync that reported success while writing zero rows, the probe that would have said "nothing to choose from" when an API was switched off, the visibility check that scored a truncated answer as a failure. The pattern is identical every time: an absence of data was being reported as a negative finding. "Unknown" needs to be a first-class answer in anything you build.

6. Write down what is not proven. Our notes explicitly flag llms.txt as unconfirmed by any vendor — worth keeping because it costs nothing, not worth treating as a lever. Recording your uncertainty is what stops next quarter's plan from inheriting this quarter's guesses as facts.

What we have not proven

Everything above is diagnosis and infrastructure. None of it is a ranking result, and we are not going to pretend otherwise for another eight weeks.

We hold seven days of search history. Positions computed on one or two impressions are noise, not signal. The fifteen new pages shipped after that data window closed and had not been crawled yet. The AI-visibility score is a single run of eight prompts — the value of that tool starts on the second run, not the first.

What we can say is that the measurement is now trustworthy, which it demonstrably was not on Friday. Here is the scoreboard we will judge this against, in the order these should move:

1. Non-brand organic clicks above zero. Currently zero. That is the first real signal.
2. The Fort Myers cluster crossing from the 20s into the low teens.
3. Home-market impressions rising to lead rather than trail Florida.
4. Google review count per market.
5. AI visibility: named in the prompts we currently lose outright.

We will publish the follow-up either way.

The transferable lesson

If you take one thing from this: audit your measurement before you optimize anything.

We spent a weekend and found that a third of our traffic data was our own staff, our best-looking lead of the month was a rep testing a link, our search-term sync had never written a row, our robots.txt was inviting crawlers into our login page, and the file we publish for AI crawlers did not mention what we sell.

None of that appeared as an error. Every one of those systems reported healthy. That is the thing about instrumentation: when it breaks, it does not stop. It starts lying, and it keeps lying with total confidence until somebody reads the code.

The good news is that this is a weekend of work, not a quarter, and it is now genuinely accessible — an AI agent with access to your system will find in an afternoon what would previously have taken an expensive consultant a fortnight. The bad news is that nobody is going to tell you to do it, because nothing is visibly broken.

---

If you want this run on your own site, that is what our AI adoption consulting work looks like in practice: pointing the same tooling at your systems, finding what is quietly wrong, and leaving your team able to run it themselves. It is part of our broader AI services practice, and it sits alongside the managed IT and cybersecurity work we do for businesses in La Crosse, Eau Claire and Fort Myers.

Or take the four-minute version from this article and go read your own robots.txt. You may not like what you find, and that is the point.