Ask the Whole Fleet Why
There’s a moment every Go developer has had. Something shows up in go.sum that you
didn’t ask for, or a scanner flags a package you’ve never heard of, and you type the
one command that makes it make sense:
$ go mod why -m gopkg.in/yaml.v3
# gopkg.in/yaml.v3
example.com/fleet/svc-checkout/cmd/svc-checkout
example.com/fleet/platform-httpkit
example.com/fleet/platform-telemetry
gopkg.in/yaml.v3There it is. My service imports our HTTP toolkit, which imports our telemetry library, which parses YAML. Four lines, question answered.
Now ask it about two hundred repositories.
That’s the whole problem. go mod why is an excellent tool operating at exactly the
wrong scope, and once you try to scale it you discover it has three other limits you
never noticed — plus one that Go itself introduced in 2021 and that makes the word
“direct” mean something different than you think.
I built a nine-repo fleet to work this out properly.
The fleet
Three internal libraries, one utility library, five services and tools. Small enough to draw, shaped like a real platform team’s repos.
The dependency I want to locate is gopkg.in/yaml.v3, and for a concrete reason:
versions before v3.0.1 carry
CVE-2022-28948, a HIGH-severity
denial of service where Unmarshal panics on malformed input.
But “interesting dependency” is the general case here. It could be a package with a license your legal team just reclassified, a library that got deprecated, a transitive dependency that quietly added 40MB to your images, or the one you’re trying to standardize the whole org onto. The question is the same shape every time: where did this get in, and why is it here?
Where go mod why shines
Run it in the library that actually uses YAML:
$ cd fleet/platform-telemetry && go mod why -m gopkg.in/yaml.v3
# gopkg.in/yaml.v3
example.com/fleet/platform-telemetry
gopkg.in/yaml.v3One hop — this is where it enters the fleet. And in the utility library that has nothing to do with it:
$ cd fleet/platform-logging && go mod why -m gopkg.in/yaml.v3
# gopkg.in/yaml.v3
(main module does not need module gopkg.in/yaml.v3)That’s a genuinely good answer, and note that it’s a positive statement of absence. Hold that thought — it matters more than it looks.
Limit one: a shortest path, not all paths
$ cd fleet/svc-reporting && go mod why -m gopkg.in/yaml.v3
# gopkg.in/yaml.v3
example.com/fleet/svc-reporting/cmd/svc-reporting
example.com/fleet/platform-authz
example.com/fleet/platform-telemetry
gopkg.in/yaml.v3True, and incomplete. svc-reporting reaches YAML through platform-authz and
through platform-httpkit. The docs are explicit that the command shows a shortest
path, so it structurally cannot show you the second one.
If you’re deciding which library to fix, “reporting depends on authz” is half the
picture — and the missing half is precisely the half that tells you fixing authz
alone won’t clear that repo.
Limit two: it can’t see build constraints
This one surprised me. tool-cli uses spf13/cobra, which requires
github.com/inconshreveable/mousetrap:
$ GOOS=linux go mod why -m github.com/inconshreveable/mousetrap # needed
$ GOOS=windows go mod why -m github.com/inconshreveable/mousetrap # needed
$ GOOS=darwin go mod why -m github.com/inconshreveable/mousetrap # neededIdentical on every platform. But mousetrap is Windows-only, and I ship Linux containers. Ask the package graph and the binary instead:
tool mousetrap on linux?
go mod why needed (same on every GOOS)
go list -deps 0 on linux, 1 on windows
go version -m <binary> absentgo mod why walks the module requirement graph, which is build-constraint
agnostic. go list -deps walks the package graph and gets it right. The compiled
binary is ground truth and agrees with go list.
This is not a bug — it’s answering “why is this module in my requirement graph,” which is a different question from “is this code in my binary.” But when you’re triaging a package that only compiles on a platform you don’t ship, that distinction is the entire answer.
Limit three: go.mod no longer means what you think by “direct”
Here’s the one that reorganizes the whole problem. This is every go.mod in the
fleet that mentions YAML:
platform-telemetry require gopkg.in/yaml.v3 v3.0.0
platform-httpkit gopkg.in/yaml.v3 v3.0.0 // indirect
platform-authz gopkg.in/yaml.v3 v3.0.0 // indirect
svc-checkout gopkg.in/yaml.v3 v3.0.0 // indirect
svc-inventory gopkg.in/yaml.v3 v3.0.0 // indirect
svc-notify gopkg.in/yaml.v3 v3.0.0 // indirect
svc-reporting gopkg.in/yaml.v3 v3.0.0 // indirect
tool-cli gopkg.in/yaml.v3 v3.0.0 // indirect
platform-logging absentEight manifests name it. One repository imports it.
This is module graph pruning, from Go 1.17. Before that, go.mod listed roughly
what you imported and the toolchain walked the graph for the rest. Loading that full
graph got expensive, so the toolchain now records the entire transitive set directly
in every module’s go.mod, tagged // indirect. Good decision — builds stopped
needing to fetch the world to answer questions about your own dependencies.
The side effect is that a fleet-wide grep for a package name now matches almost everything, and “is this a direct dependency” splits into three different questions with three different answers:
- imports it — an actual
importstatement in source: 1 repo - reaches it transitively: 7
- merely declares it in
go.mod: 8
The useful signal turns out to be free: go mod tidy only omits the // indirect
marker on requirements you genuinely import. So a direct requirement in go.mod is
the source-level import, and the marker separates the one from the seven without
parsing a single Go file.
Building the fleet-wide version
So I want go mod why, but across every repo, along every path, aware of build
constraints, and able to tell “imports it” from “pruning put it there.” That needs
two things: extraction at build time, and somewhere to ask graph questions.
Two SBOMs per repository, from cyclonedx-gomod:
# the requirement graph -- what the manifests declare
cyclonedx-gomod mod -json -output out/svc-checkout.mod.cdx.json fleet/svc-checkout
# the linked set -- what actually ends up in the binary
cyclonedx-gomod app -json -main cmd/svc-checkout \
-output out/svc-checkout.app.cdx.json fleet/svc-checkoutapp mode is build-constraint aware — it resolves the real package graph for a
given GOOS/GOARCH, which is exactly the thing go mod why won’t do. Load both,
label every edge with which graph produced it, and the build-constraint question
becomes answerable.
Three things that cost me time:
app mode has no -version flag. It derives the main module version from git
history and treats failure as fatal. bin mode has an override; app mode doesn’t.
Since actions/checkout defaults to fetch-depth: 1, this is the most common way
the job dies:
- uses: actions/checkout@v4
with:
fetch-depth: 0 # not optionalThe two tools disagree about paths. cyclonedx-gomod wants -main cmd/thing
and rejects a leading ./. go build wants ./cmd/thing and will otherwise
resolve cmd/thing against the standard library and fail with package cmd/thing is
not in std.
Library repos have no app graph. app mode needs a main package. That’s not a
tooling gap — a library has no linked set until something builds a binary from it.
Four of my nine repos are manifest-only, and any query treating “absent from the app
graph” as “not linked” has to exclude them or it will invent conclusions.
The trap that nearly took the whole thing down
I loaded both graphs into Neo4j and asked for paths. Everything came back as a two-node chain: every repo one hop from YAML. Confident, clean, wrong.
Because cyclonedx-gomod emits one SBOM edge per require line — and thanks to
pruning, platform-httpkit’s go.mod contains gopkg.in/yaml.v3 // indirect. So
the SBOM contains a direct edge from httpkit to yaml.v3, even though httpkit
requires only platform-telemetry and chi. That’s band three of the figure above.
The graph is dense with shortcut edges that are indistinguishable from declared ones.
Those shortcuts don’t change what’s reachable — every reachability query returned
identical results with and without them. What they destroy is path structure,
which is the entire output of a why query.
CycloneDX has no field for “was this edge declared, or computed by MVS,” so the
loader recovers it by parsing go.mod directly and tagging each edge. That only
works for first-party modules whose source you have, which is enough: those are the
repos you can act on anyway.
This is not a cyclonedx-gomod bug. The SBOM is accurate — every edge it reports genuinely is a requirement. An SBOM is a bill of materials: a faithful record of what is present. Whether an edge was hand-declared or synthesized by the module graph isn’t something the format sets out to represent. Any question about paths or ordering needs a second source, and that source is
go.mod.
Asking the whole fleet
With declared edges tagged, the flagship query is go mod why with the scope fixed:
MATCH (v:Module {name: $target})
MATCH p = (r:Repo)-[:REQUIRES {graph: 'mod', direct: true}]->(:Module)
-[:DEPENDS_ON* 0..8 {graph: 'mod', direct: true}]->(v)
WITH r, [n IN nodes(p) WHERE n:Module | n.name] AS chain
RETURN r.name AS repo, size(chain) - 1 AS hops, chain AS why
ORDER BY hops, repo;repo, hops, why
platform-telemetry, 0, [yaml.v3]
platform-authz, 1, [platform-telemetry, yaml.v3]
platform-httpkit, 1, [platform-telemetry, yaml.v3]
tool-cli, 1, [platform-telemetry, yaml.v3]
svc-checkout, 2, [platform-httpkit, platform-telemetry, yaml.v3]
svc-inventory, 2, [platform-httpkit, platform-telemetry, yaml.v3]
svc-notify, 2, [platform-authz, platform-telemetry, yaml.v3]
svc-reporting, 2, [platform-httpkit, platform-telemetry, yaml.v3]
svc-reporting, 2, [platform-authz, platform-telemetry, yaml.v3]The whole fleet, sorted by distance from the thing you’re hunting. svc-reporting
appears twice — both of its paths, the second of which go mod why will never
show you. Zero hops means “this is where it enters.”
And the absence case, because silence is not an answer:
repo, why
platform-logging, (main module does not need module gopkg.in/yaml.v3)Worth doing explicitly. In a fleet, “no rows returned” is ambiguous between this
repo is clean and this repo has never been scanned — and those demand completely
different responses. go mod why prints the positive statement for one repo; the
fleet version should too.
The two questions go mod why can’t reach
Which repos genuinely import it. Splitting declares-versus-imports is a
DECLARES edge away, parsed from the // indirect marker:
repos_naming_it, repos_reaching_it, repos_actually_importing_it
8, 7, 1A note that cost me an hour: this cannot come from the SBOM. cyclonedx-gomod’s
root component lists three dependencies for svc-checkout, where go.mod declares
nine — the root dependsOn is direct-requires-plus-replaced, not the require block.
The six missing entries are exactly the pruned transitive set, which is exactly the
part this question is about. The thing I most wanted to count is the thing the SBOM
structurally omits.
What’s required but never linked. Because both graphs are loaded, mousetrap resolves properly:
Ten components in the mod graph, nine in the app graph, and go version -m on the
built binary agrees with the nine. Where go mod why says “needed” on every
platform, the fleet graph says required by the manifest, absent from the Linux
binary — and can prove it against the shipped artifact.
One caveat that matters: an app graph is only true for the GOOS/GOARCH it was built
against. Something that’s absent on linux/amd64 is not absent for a Windows build.
Emit one app graph per target in your release matrix before trusting this.
And one it can’t even frame
Once every path is in a graph, you can ask which single repository sits on the most
of them — a question that has no go mod why equivalent at all, because it’s about
the fleet rather than about any one repo:
platform-telemetry sits on the paths of all seven affected repos. platform-authz
sits on two. If you’re standardizing a library, negotiating a license exception, or
planning a migration, that ranking is the plan.
Note what it is not: a popularity ranking. platform-logging has the most direct
dependents in the fleet — five — and no path to YAML at all. Depth drives those in
opposite directions. A module sinks deeper and more repos reach it while fewer
declare it, because everything above gets it as // indirect. Shallow utilities get
imported by everyone precisely because they’re small and dependency-free, which is
what makes them least likely to sit on a path to anything.
What it actually is
Nothing here is clever. Two cyclonedx-gomod invocations in CI, a reusable workflow
so per-repo adoption costs one file, a loader that turns CycloneDX dependencies
arrays into Cypher, and a graph database. Perhaps four hundred lines all in.
What you get is the question you already know how to ask, asked at the scope you actually operate at — every repo, every path, build constraints included, and an honest answer about which repositories really import a thing versus which ones just inherited its name in a manifest.
Which leaves the obvious follow-up: now that you know it’s in seven repositories and enters through one, where do you actually fix it? That answer is less obvious than it looks — Go’s minimal version selection means all seven can patch themselves without the library in the middle ever changing — and it’s the subject of the next post.
The code is at husobee/sbom-fleet: nine synthetic repos, the CI step, the loader, and the queries. Every number here is query output against that fleet.