Dependency research · 06

Findings that never ship

Two in five of the vulnerabilities your scanner reports are on test harnesses and build plugins that never reach production. They trigger the same escalation anyway.

10 Sep 2026 · 24 repositories · 1,203 findings · 5 ecosystems

The response costs more than the finding

A critical advisory lands. Someone is paged, a patch window opens, teams stop what they were doing, and a report goes to whoever asks for reports. That machinery runs the same way whether the vulnerable package ships to customers or only ever ran inside a test harness on a build agent.

Nothing in a normal findings list separates the two. The advisory is real, the version match is correct, the severity is accurate. What is missing is the one fact that decides how much any of it matters.

org.apache.mina:mina-core@2.1.6 — five critical advisories, test scope. Five genuine criticals against one networking library inside a Java service. It is compiled and executed when the suite runs, and it is not in the artifact that gets deployed. On a dashboard that reads version numbers and nothing else, this repository looks like one of the worst in the estate.

How we measured it

Twenty-four public repositories pinned to fixed commits, across Maven, Gradle, npm, PyPI and Go. Every dependency declares a scope in its own manifest, and we read it where the ecosystem states one. No inference, no heuristic. Where a manifest states nothing we record the dependency as undetermined and never guess.

The counting unit matters more than usual. A finding is counted once per manifest it appears in, because that is how a build gate sees it — a package present in two manifests can fail a build twice. That is a different unit from deduplicated advisory counts, and the two must never be mixed in one sentence.

How many findings are development-scoped?

479 of 1,203, or 39.8%. Just over half were runtime, and the remainder were optional, provided, or declared no scope at all.

Runtime — 610 findings, 50.7%. These ship.
Development — 479 findings, 39.8%. These do not.
Undetermined — 62 findings, 5.2%. The manifest states nothing, so neither do we.
Optional — 47, and provided — 5. Shown separately, because whether they ship depends on how the consuming project is assembled.

Why the share varies so much between estates

We have run this three times as the corpus grew, and the share fell every time. The reason is worth more to you than any single figure.

25 Aug 2026 — 17 repositories, 535 findings, 54.4%
27 Aug 2026 — 18 repositories, 604 findings, 48.3%
10 Sep 2026 — 24 repositories, 1,203 findings, 39.8%

Earlier runs were weighted toward JavaScript projects, which is where development scope concentrates. Adding Java, Go and Python repositories pulled the average down each time. Anyone quoting a single number for this is quoting their own sample. The honest claim is a range with a mechanism attached: between a third and a half, driven almost entirely by how much JavaScript is in the mix.

Dev dependency vulnerabilities by ecosystem

The variation between ecosystems is larger than the variation between estates, and it is the part you can act on immediately.

npm — 411 of 714, 57.6%. Deep build toolchains, and the clearest win available from one manifest field.
Python — 53 of 256, 20.7%. Dependency groups exist and are used more sparingly.
Gradle — 15 of 79, 19.0%, with 19 more findings where the build file declared no scope.
Maven — 0 of 73. Maven states scope reliably; these projects simply had no vulnerable test dependencies.
Go — 0 of 81, and it cannot be otherwise. go.mod records no scope, so 43 findings landed as undetermined.

A JavaScript estate gets more from this one field than from anything else it could buy. A Go estate gets nothing at all and should be told so, rather than sold a filter that suppresses zero findings. Both are one run away from being confirmed on your own code.

Critical and high findings in dev dependencies

The share is not flat across severity. It is highest in the two bands that actually interrupt people.

Critical — 22 of 61, 36.1%
High — 253 of 537, 47.1%
Medium — 175 of 469, 37.3%
Low — 29 of 99, 29.3%
Unrated — 0 of 37. No source assigned these a severity, and none were development-scoped.

More than a third of critical findings, and nearly half of highs, are on code that never reaches production. Criticals are what start the escalation. Highs are what fill the backlog someone has to justify at the end of the quarter. Both are where the cost of not reading a manifest field gets paid.

The case against this, which is real

Development dependencies execute. They run on build agents holding registry credentials, signing keys and deploy tokens, and a compromised test tool is a supply chain attack that never touches your product. Say that before someone else does.

vite@6.2.2 — known exploited, development scope, rated medium. One finding in this sample sits on the CISA Known Exploited Vulnerabilities catalogue and is development-scoped. Because it is rated medium, a severity filter would have dropped it before a scope filter got the chance. Actively exploited in the wild, running on a machine that holds signing keys, invisible to both of the filters a team is most likely to configure.

This is the argument against suppressing development findings by default, and it is why we do not. Scope belongs in the ranking, not in a filter that runs before anyone has looked.

What this does not prove

The sample is 24 pinned open-source repositories chosen for ecosystem coverage. The headline moved from 54% to 40% across three runs of this same measurement purely because composition changed, so treat any figure as a property of the sample.
Scope is not a differentiator. Most scanners read the same manifest field. The claim is about where the yield lives, not who has what.
A dependency that does not ship still runs. This is about prioritisation, not dismissal, and the known-exploited example is why that distinction is not academic.
Scope tells you what ships, not what executes. Whether the vulnerable code path is ever entered, and whether the package is in your built artifact, are two harder questions this does not answer.

The part we have not solved

Reading one manifest field returns roughly twelve times what walking an installed dependency tree returns, measured on the same corpus in the same week. The cheap technique beats the sophisticated one by an order of magnitude, and almost nobody leads with it.

What we would like argued with: should an optional or provided dependency count as shipping, given it depends entirely on how the consumer assembles the build? Is undetermined worth showing, or does an honest gap just read as a missing feature? And for Go and version-catalogue readers whose manifests say nothing at all, is there a signal we have missed?

We rank on it, we do not filter on it

depproof reads the declared scope of every dependency in every ecosystem that states one, reports what ships and what does not, and says how many it could not determine rather than assuming the answer. The suppression filter exists and is off by default, for the reason above. The number we show is what the data says, not what the gate does.

Practical next step: how to handle dev dependency findings · scanning npm projects · why zero findings is not zero risk · the same corpus, measured for maintenance

Questions people ask

How many vulnerabilities are in dev dependencies?

In our measurement of 24 pinned public repositories across five ecosystems, 479 of 1,203 gate-visible findings — two in five — were on dependencies their own manifest declares as development-only. The share varies enormously by ecosystem: 57.6% for npm, 20.7% for Python, 19.0% for Gradle, and none at all for Go, which has no scope concept to read. Treat any single number as a property of the estate it was measured on rather than a general truth, because our own figure moved from 54% to 40% across three runs as the sample grew more diverse.

Do dev dependency vulnerabilities matter?

They matter less than production ones and more than zero. A vulnerable test runner is not in the artifact you deploy, so it is not reachable by anyone attacking your running service. It does execute on a build agent that typically holds registry credentials, signing keys and deploy tokens, so the realistic threat is compromise of your build rather than exploitation of your product. That makes them a ranking problem rather than a blocking one.

What percentage of critical vulnerabilities are in dev dependencies?

In this sample, 22 of 61 critical findings — 36.1% — were development-scoped, and 253 of 537 high findings, or 47.1%. The share is highest in exactly the two severity bands that interrupt people, which is why the cost of ignoring dependency scope is paid during escalation rather than in a backlog.

Should security tools hide development findings by default?

No. In this sample one finding on the CISA Known Exploited Vulnerabilities catalogue was development-scoped and rated only medium, so a severity filter would have dropped it before a scope filter was ever consulted. A filter that runs before a human has looked will eventually hide something that mattered and nothing will report that it happened. Scope belongs in the ranking, not in a gate that silences findings nobody read.

Why do npm projects have so many dev dependency vulnerabilities?

JavaScript build toolchains are large and deep. A bundler, a test runner and a linter between them pull in thousands of transitive packages that application code never touches, and those packages carry advisories like any others. We measured 57.6% of npm findings as development-scoped, against 20.7% for Python. If npm audit output feels dominated by tooling you do not recognise, that is an accurate picture of the ecosystem rather than a problem with your project.

Does dependency scope work for Go and Gradle?

Not for Go, and only partly for Gradle. A go.mod file records no scope at all, so nothing can distinguish a test-only module from a production one and 43 of 81 Go findings in this sample landed as undetermined. Gradle states scope through testImplementation and testRuntimeOnly, but a project built on a version catalogue correctly declares no scope in the catalogue itself, which is why 19 Gradle findings were undetermined too. A tool reporting zero undetermined dependencies on either is asserting something the input does not contain.

Cite this

Two in five vulnerability findings are on dependencies that never reach production.
479 of 1,203 · 39.8% · 24 pinned public repositories · 10 Sep 2026

More than a third of critical findings are on code that never ships.
22 of 61 critical · 253 of 537 high · same sample and unit

depproof, “Findings that never ship: dependency scope as a prioritisation signal,” Dependency research 06, 10 September 2026. Figures and tables may be reproduced with attribution and a link back — no permission needed. If you re-run this against your own estate and get something different, we would rather hear it than not.