depproof

Guide · Choosing an SCA tool

SCA tool evaluation checklist: what to test before you buy

Every software composition analysis tool looks good in a demo. The differences show up on your repositories. Here is what to test during a trial, and what a good answer looks like.

Published 9 Nov 2026 · by the depproof team

In one sentence: judge an SCA tool on whether it sees your whole dependency tree and says when it cannot, whether you can check where every finding came from, and whether its results hold up as evidence.

Set up a fair trial

  • Use five to ten of your own repositories, including the awkward ones: a monorepo, a project with no lockfile, a mixed-language service.
  • Pin the commits and scan the same ones with every tool.
  • Bring both audiences: an engineer judges noise and fixes, a compliance owner judges the evidence.
  • Decide the must-haves before the first scan, and time-box it to about two weeks.

Coverage: what you actually ship

What to test What a good answer looks like
The full resolved tree, transitive dependencies included Findings name packages your manifests never mention.
Every ecosystem and build system you use Monorepos, mixed languages and older builds scan without special handling.
Honest reporting of partial coverage A repository with no lockfile is reported as partly covered, not as clean.
What ships, separated from development-only Scope ranks findings without silently hiding the development ones.
Licenses across the whole tree The same full tree as vulnerabilities, since obligations follow what you distribute.

Vulnerability data: where findings come from

What to test What a good answer looks like
A checkable source for every finding Each finding links to an advisory record you can open yourself.
Independence from NVD enrichment Library advisories still appear, with a usable severity, when the NVD has not scored the CVE.
Advisories with no CVE ID Ecosystem advisories and known-malicious packages are reported, not dropped.
Ungraded advisories Unscored is shown as unscored, never quietly treated as low.

Prioritisation: what to fix first

What to test What a good answer looks like
Signals beyond CVSS severity Known-exploited status (CISA KEV) and exploit likelihood (EPSS) sit next to severity and can drive the gate.
Whether a fix exists Each finding says whether a fixed version is available: act now, or decide.
A noise level engineers accept The findings that would fail a build are ones an engineer agrees are worth fixing.

Workflow: will engineers keep it on?

What to test What a good answer looks like
Runs where your builds run Same results in your CI system and on a laptop.
Pull-request gates on your own policy You set the threshold, and a failed gate says why in terms an engineer can act on.
Accepted risk that stays visible A waiver has an owner, a justification and an expiry, and comes back for review.

Evidence for auditors and customers

What to test What a good answer looks like
An SBOM in a standard format CycloneDX or SPDX that a customer’s own tools can read.
Results you can trace Each report records the advisory data behind it, so last quarter’s result can still be explained after new advisories change the answer.
A record of decisions, not just findings Who decided what, and when, ready for an audit or a security questionnaire.
Telling customers what does not affect you “Not affected” decisions shared in a machine-readable form such as VEX.

Data handling and fit

What to test What a good answer looks like
Where your source code and findings go You know exactly what leaves your infrastructure, and your security and legal teams accept it.
Leaving without losing your history Findings, SBOMs and decisions export in open formats you can keep.
A pricing unit that matches how you grow Next year’s cost is predictable, without restricting who uses the tool.

Why the data questions matter more now

Since April 2026, the National Vulnerability Database no longer enriches most CVEs; NIST now prioritises known-exploited vulnerabilities, federal software and software deemed critical. A tool that relies on that enrichment can show a library advisory with no score, or miss the match. For prioritisation, ask about the CISA Known Exploited Vulnerabilities catalog and FIRST’s EPSS by name. More in OSV vs CVE and CVSS vs EPSS vs KEV.

What the frameworks ask of your dependencies

NIST’s Secure Software Development Framework (SP 800-218) has three tasks a good SCA tool helps you evidence:

  • PW.4.4: “Verify that acquired commercial, open-source, and all other third-party software components comply with the requirements, as defined by the organization, throughout their life cycles.”
  • PS.3.2: “Collect, safeguard, maintain, and share provenance data for all components of each software release (e.g., in a software bill of materials [SBOM]).”
  • RV.1.1: “Gather information from software acquirers, users, and public sources on potential vulnerabilities in the software and third-party components that the software uses, and investigate all credible reports.”

The OWASP Top 10:2025 puts Software Supply Chain Failures at A03, which covers more than known vulnerabilities. The rest is in the six application security risks in your dependencies.

Scoring the trial

Mark each row meets, partly meets or does not meet, from what you saw, not what you were told. Then:

  • Must-haves are pass or fail. For most teams: full-tree coverage, honest partial-coverage reporting, checkable data sources, and a data-handling answer security accepts.
  • Compare findings, not counts. Fewer findings is only better if the tool is not seeing less.

Questions people ask

What should an SCA tool evaluation checklist cover?

Six areas: coverage of the full dependency tree, where the vulnerability data comes from, prioritisation beyond CVSS severity, workflow fit, evidence for auditors and customers, and data handling. Test each on your own repositories, not on a vendor demo.

How do you evaluate software composition analysis tools fairly?

Scan the same pinned commits with every tool. Look at the findings one tool reports and another misses, and check a sample by hand. Note whether each tool says when it could not resolve a project: fewer findings is only better if the tool is not simply seeing less.

How long should an SCA tool trial take?

About two weeks, on five to ten of your own repositories, including the awkward ones: a monorepo, a project without a lockfile, a mixed-language service.

Is a free SCA tool enough?

It can be for one team with a few repositories in one ecosystem. The gaps appear at organisation scale, which is what enterprise software composition analysis is about: consistent coverage across languages, one policy for every repository, waivers with owners and expiry dates, and evidence an auditor will accept. The checklist is the same whatever the price.

Running depproof against this list

Know exactly what you ship. And prove it. depproof reports the full resolved tree, says when a project could not be fully seen, and keeps decisions with their owner and expiry. It runs on your own infrastructure; your source and findings never leave it. Put it through the same trial: the docs show how, and pricing is per product, with every developer and repository included.

Sources

Keep reading