Why we don't fabricate GEO data

Ranksify measures something inherently noisy. We ask a set of AI engines a set of prompts, on a schedule, and record what comes back. Every number on every screen — visibility percentage, share of voice, “leading on pricing” — is a statistic computed from a sample of generated answers. Not a census. A sample.

That is not a flaw in the method; it is the method. Generative engines publish no rankings and expose no log file, so sampling is the only honest instrument available. What matters is what a tool does with the uncertainty that sampling creates. Most of this category smooths it away, because a smooth number demos better. We decided not to, and that decision is written into the product in five specific places.

1. A proportion without an interval is not a measurement

“You appear in 40% of answers” means one thing when it is 2 of 5 and something entirely different when it is 400 of 1,000. Ranksify never treats those as the same claim. Proportions computed from sampled answers carry a 95% Wilson score interval — wilsonInterval(successes, n, z = 1.96) in our metrics package.

Wilson rather than the ordinary normal approximation, because the normal approximation falls apart in exactly the regime we live in: small samples, and proportions pinned near 0% or 100%. It will cheerfully hand back an interval that extends below zero or above one. Wilson stays inside the range of things that can actually happen, and stays sane on a handful of observations.

And when there are no trials at all, the function returns the interval [0, 1] — the proportion is unconstrained. Zero data is not zero percent. It means we know nothing, and the arithmetic says so out loud rather than defaulting to a confident-looking floor.

2. The evidence floor is symmetric

On the Insights attributes view, Ranksify tells you whether you are leading, at parity, or behind a named competitor on a specific attribute — pricing, onboarding speed, support quality. That verdict is decided by whether two Wilson intervals overlap. It is never decided by comparing raw ratios, because a ratio comparison calls 2-of-3 against 1-of-3 a decisive lead, and it is not one. There is a test that refuses exactly that.

Underneath sits a constant: MIN_RIVAL_RECORDS = 3. Below three extracted records a competitor is not compared at all. The reason is arithmetic rather than taste. A rival seen exactly once, naming the attribute, sits at 100% with a Wilson upper bound of 1.0 — nothing is statistically separable from that — so a single noisy row would silently veto “leading” on that attribute forever.

The part that took an adversarial review to get right: the floor is symmetric, and applies to our own side too. With one extracted answer of yours, Wilson(0, 1) runs to an upper bound of 0.79, and even a rival at 10-of-10 (lower bound 0.72) overlaps it — so every comparison chip disappears and the card reassuringly announces that no competitor beats you, on the strength of a single answer. Measured, not reasoned: at one record the chips vanish, at two they come back. Three is where a Wilson interval first stops spanning most of the range, which is why the floor is a flat three rather than a power calculation nobody would be able to explain.

When nothing clears the floor, the standing is null and the tile reads “not compared”. That is the same honest state as having no competitor records whatsoever, and it is deliberately not dressed up as parity.

3. Unknown is not zero — and “we found nothing” is not “we never asked”

This distinction produced the worst near-miss in the product’s history, so it is worth being specific.

When we extract attributes from a sampled answer, the result is stored as JSON, and there are two very different empty states. null means the record predates the extractor, or extraction never ran: we never asked. An empty array [] means the extractor ran and genuinely found no attribute in that answer: we asked, and the answer was nothing.

An earlier build counted both in the denominator. Against real data that produced a tile reading “50.0% of answers · Leading · vs Globex 0.0%” — green, confident, and wrong in both directions at once. The true rate was 100% and the true comparison did not exist, because every one of that rival’s records predated the extractor. Every brand would have sat in that regime for the first four weeks after release.

The fix is a filter, and the rule it encodes is now enforced everywhere the data flows: rows we never asked about are excluded from both denominators, and rows that came back empty are kept, because a measured nothing is real evidence. Both Wilson denominators, the rival-membership test, the extracted-record count and the compared-rival count all respect it.

4. A dash is a legitimate answer

The competitor leaderboard carries a Domain Authority column. When we do not know a competitor’s domain, or the provider has no rank for it, the cell renders an em dash. Not a zero, not a modelled estimate, not a greyed-out plausible number that a reader will quote back to their board next quarter.

The inverse matters just as much: a real rank of 0 renders as 0. A brand-new domain genuinely scoring zero is a fact about that domain, and folding it into the same bucket as “unknown” would destroy the very distinction the dash exists to protect. The rule is print the value if we have one, print the dash if we do not, and leave nothing in between for the two states to blur into.

5. When we cannot detect the market, the label says so

Share of Voice needs a denominator: who else is in this market? Ranksify widens that beyond the competitors you typed in by detecting the brands the engines themselves name. That detection is a paid model pass, so it is cached — and it can fail. A provider outage. A workspace that has hit the daily spend ceiling it set. A cold cache on a brand with nothing to detect yet.

There is a line in the codebase that reads [sov-market] detected market unavailable — tracked-only. That is what happens on failure: the view degrades to the market we can actually name — the competitors you are tracking — and the label on screen says exactly that. It does not extrapolate a market size. It does not fill the gap with a category average. A budget refusal is even logged as a budget refusal rather than a provider error, so nobody spends an afternoon debugging a model for a customer who simply reached a ceiling they chose.

What this costs, and what you get

It costs demo polish. A screen carrying dashes, “not compared” chips and interval-qualified verdicts is a harder screenshot than one where every cell has a number in it. We ship four distinct ways for a competitive card to have nothing to say — the attribute was never mentioned, extraction has not run yet, no rival cleared the evidence floor, or no rival actually beats you — in a category where the obvious product decision is to collapse all four into one reassuring empty state.

What you get in exchange is that you can act on what is there. Every number Ranksify shows is one it would defend under audit, and the ones it will not defend are visibly absent rather than quietly invented. If a competitive comparison is on your screen, at least three extracted records sit behind each side of it and the intervals do not overlap.

If that sounds like a low bar for a measurement product, we agree. It is not currently the norm in this category, which is the whole reason this post exists.