The CVE counter is a lie

Your pen test report has 47 findings. None of them describe how an attacker actually gets in. The number is comforting; the number is also a fiction. Here is what we measure instead.

Walk into any vulnerability management meeting and you'll see the same artifact: a stacked bar chart, severity colors, a 47 next to the word 'open.' Everyone nods. Everyone moves on. Nobody discusses the actual question — can someone walk through these and own the company by Friday. We rebuilt our pipeline around that question.

What the CVE counter actually measures

A CVE counter measures one thing: how many checks your scanner ran that returned a positive signal. That is not the same as risk. That is not the same as exploit feasibility. It is, at best, a tally of paint chips on a wall.

Two real findings from a real scan:

• MEDIUM: Missing Content-Security-Policy header on /admin
• MEDIUM: Reflected XSS in /search?q= parameter, sanitization bypass via charset switching

Both score MEDIUM. Both sit on the same line of the same report. Reading them in isolation, an SRE has no way to know that finding #2 lets you exfiltrate the admin's session cookie on /admin because finding #1 didn't block inline scripts. The two together are critical. Apart, they're noise.

What we measure instead — the exploit chain

Our planner builds a directed graph: nodes are findings, edges are 'finding A makes finding B more exploitable.' After every attack round our AI walks the graph and identifies chains where at least three independent findings compose into a single, demonstrable outcome — admin takeover, data exfiltration, RCE on the application server.

What ends up on page 1 of the dossier:

• One chain. With a title. "Anonymous user → admin session via CSP-relaxed XSS on /search"
• The minimum number of steps to reproduce, copy-pastable
• The exact response from the target proving each step worked, not a hypothetical
• The fix that breaks the chain at the weakest link (usually the CSP, not the XSS)

What ends up on page 14 of the dossier: the 43 other findings, grouped by topic, each with a one-line context. Available if you want to do due diligence. Not the headline.

Why this is hard

It would be easier to count. Counting is what most tools do because it scales linearly with how many checks you ran. Chain reasoning is closer to constraint satisfaction — you need a planner that can hold the state of an authenticated session, replay a vulnerability against a real fixture, and decide whether the next finding compounds the first.

That requires three things most scanners don't have: stateful exploit verification, semantic understanding of what each finding enables, and a budget to spend $0.50 of LLM time reasoning about chains rather than emitting a CSV.

The numbers that actually matter

On a recent scan (anonymized): 47 findings, 6 critical chains, 1 path to full admin takeover, 3 paths to read-only data exfiltration. The report opens with the chain. The customer fixed the CSP. Every chain in the dossier collapsed in one commit.

If we had handed them 47 findings sorted by CVSS, that customer is still working through 'remediate XSS in /search' as ticket #18 on the security backlog. Their CSP misconfiguration is sitting somewhere around ticket #34. Six months from now, neither is fixed, and the chain still works.

What you should ask your pen test provider

If your next vendor sends you a 47-page PDF with a 47-finding count on the cover, ask one question: what's the worst thing someone can do with these. If they can't answer in one sentence, they did half the work.

We answer in one sentence. We have to — that's what we built the company to do.

If your last report had 47 findings and zero chains, you have a recount problem, not a security problem. Run a real scan.