Version ranges are a coin-flip: nearly half the CVEs flagged on one repo were already fixed

We pinned a large, heavily-audited open-source project at one exact release, pulled every published security advisory for it, and computed which ones a standard version-range check - the check every dependency scanner runs: does your version fall inside the range an advisory marks as vulnerable? - says that release is exposed to. The answer was 34 advisories.

Then we opened the source for all 34. Sixteen of them - 47% - were already fixed in that release. The vulnerable code was not there. A patch had been backported - copied back into the older release line - earlier than the version the advisory records as "first patched." If we had shipped the 34-advisory list as a finding set - which is exactly what a dependency scanner, an SBOM (software bill-of-materials) tool, or a "we matched your version against the CVE database" report does - nearly half of it would have been wrong.

That is the whole post: version-range matching is triage, not a finding. For anything you are going to act on, you have to read the code.

The method

Blind LLM discovery across a million-plus lines is not reliable, and we did not pretend otherwise. Instead we anchored the audit to the published advisories and verified each one against source - a four-step loop we run the same way every time.

What the source said

The false-positive rate was not uniform - it was worse at lower severity. So you cannot even take the shortcut of "source-verify the Highs, trust the version range for the rest." The tail is where the noise concentrates.

The cause is mundane and, honestly, to the project's credit: they backport security fixes into release lines ahead of the version the advisory records as first-patched. The project is more secure than its own advisory records imply. The metadata is the thing that is wrong - and the metadata is exactly what automated scanners consume.

Two kinds of trap

The snippets below are illustrative - renamed and simplified, not the project's source - but the shape of each bug is exact. They are the two patterns that decide whether a version-range flag is a real finding or noise.

A real finding the version range got right. An allowlist meant to constrain which arguments an approved binary could run was enforced on only one operating system. On the others, the constraint silently did nothing - arbitrary arguments to the allowed binary ran with no approval prompt. The vulnerable path was right there in the pinned checkout. Confirmed.

function resolveAllowedCommand(entry, platform) {
  // argument-rule enforcement was wired up for ONE OS only
  const enforceArgRules = platform.toLowerCase().startsWith("win");

  if (!enforceArgRules) {
    return entry;          // every other OS: first path match wins,
                           // the argument allowlist is never applied
  }
  return applyArgRules(entry);   // the check that should always run
}

A trap only source can untangle. Two advisories pointed at the same shared flag-parser. One was a real bypass; one was already neutralized. The difference was a single argument at the call site: one path parsed approval and execution identically - symmetric, so the advisory was a false positive - while the other parsed them differently, letting a prefix alias of a blocked flag slip through. No version range, SBOM tool, or CVE-database match distinguishes those two. You read the call sites or you get it wrong.

// Path A — approval and execution parse flags the SAME way: safe
const okA = analyzeFlags(cmd, { allowCombined: true });
run(cmd,                       { allowCombined: true });

// Path B — approval omits the flag, against an EXACT-match blocklist
const blocked = new Set(["--run-encoded", "-r"]);
const okB = analyzeFlags(cmd /* allowCombined defaults to false */);
// but the shell accepts any unambiguous prefix: "-ru", "-run", ...
// so "-ru <payload>" is never in `blocked` -> slips through, then runs

The first bug a version range can plausibly catch, because the fix bumps a version. The second it cannot: both advisories point at the same file at the same version, and only one is real. That gap - same version, opposite truth - is the whole reason source verification is not optional.

We checked our own work

A verification audit you cannot trust is just a second opinion you also cannot trust. So three days later, with the original working files deleted, we re-ran the whole thing cold. The triage reproduced exactly - a fresh advisory pull returned the same count, a from-scratch comparator flagged the same 34, and every cited ID reconciled with zero missing and zero extra. Independent readers re-opened all 34 across five subsystems and re-verified the present-versus-backported split 34 out of 34, with zero disagreements. The judgment that survived independent review is the 16 disagreements with the raw version ranges - the calls a scanner physically cannot make.

One honest caveat

The famous, marketing-named vulnerabilities for this project - the ones with names and news coverage - did not appear in the structured advisory feed at all. Those come from journalism, and we kept them out of the verified set. Cross-checking a vulnerability name against a news story is not the same as confirming a vulnerable code path in a pinned checkout.

Takeaways

• Treat version-range matching as a filter, not a finding. It tells you what to look at, not what is true. Here it was right about half the time.
• Source-verify the whole tail (the lower-severity findings), not just the criticals. The false-positive rate climbed as severity dropped - 60% at Medium.
• Distrust the recorded "first patched" version. Projects that backport fixes - the good ones - read as vulnerable in advisory metadata long after they are safe.
• Pin the target. Findings are only reproducible against a frozen commit, and re-verification is only possible against one.
• Make the audit re-runnable. Ship the comparator and a frozen advisory snapshot so anyone can reproduce the count and re-check the split.

The uncomfortable version: most "we scanned your dependencies against the CVE database" output is a triage list wearing a findings list's clothes. The gap between those two things, on one real repo, was 16 out of 34.