Last time I built a fleet-wide go mod why — build-time SBOMs loaded into a graph, so I could ask where an interesting dependency enters a fleet and along which paths. Pointed at gopkg.in/yaml.v3 and CVE-2022-28948, it gave a clean answer:

repos_naming_it, repos_reaching_it, repos_actually_importing_it
8,               7,                 1

Seven repositories affected. One — platform-telemetry — actually contains the import statement.

Which raises the question I assumed had an obvious answer: where do I fix it?

I assumed “at the source, obviously.” I was wrong, in an interesting way.

Minimal version selection changes the answer

Go doesn’t resolve versions with a solver. It uses minimal version selection: for each module, take the highest version that anything in the build graph requires. Not the newest published — the highest one somebody actually asked for. Deterministic, boring, and it has a consequence that matters enormously here.

Since Go 1.17’s module pruning writes every transitive dependency into every go.mod as // indirect, all seven affected repos already name yaml.v3 in their own manifest. And a version you name in your own manifest is a version you can raise. If svc-checkout says v3.0.1, then v3.0.1 wins — regardless of what platform-telemetry says.

A leaf repo can upgrade a transitive dependency without the library in the middle cooperating, or even knowing about it. Here it is happening:

$ cd fleet/svc-checkout
$ go list -m gopkg.in/yaml.v3
gopkg.in/yaml.v3 v3.0.0                    # inherited from platform-telemetry

$ go get gopkg.in/yaml.v3@v3.0.1
$ go list -m gopkg.in/yaml.v3
gopkg.in/yaml.v3 v3.0.1                    # patched

$ grep yaml ../platform-telemetry/go.mod
require gopkg.in/yaml.v3 v3.0.0            # never touched

$ go build ./... && echo OK
OK

I want to be blunt rather than hedge, because if you’ve ever merged a Dependabot PR you already know this: the seven-repo fix works. Seven pull requests, each raising one // indirect line, and the CVE is gone from all seven builds. No coordination. No waiting on the platform team. No library release.

So the “naive” answer — seven places — is correct. That isn’t the problem.

The problem is that it doesn’t stay fixed

Run that play as a fleet practice and it stops being seven PRs. It’s seven PRs per advisory, times your CVE rate, times your repo count. Seven review queues, seven deploy windows, seven teams to chase. And when it lands:

  • every one of those repos now carries a hand-maintained version override that future upgrades and go mod tidy runs will churn against
  • platform-telemetry still declares v3.0.0, so the next repo anyone creates inherits the vulnerable version on day one
  • if only five of the seven land, the fleet sits in a mixed state you can’t assert compliance over

It’s a patch applied at seven leaves instead of at the root, and the root still says what it always said. The fix works; it just has no memory.

That’s the annoying case. There’s also a case where the seven-PR fix isn’t available at all.

Two shapes of remediation

Two columns. Left, labelled A version bump v3.0.0 to v3.0.1: seven repositories each with a green check mark, captioned each repo bumps its own indirect line, MVS selects the higher version, seven independent PRs, this works. Right, labelled B source change yaml.v3 to yaml.v4: the same seven repositories each with a grey cross, then platform-telemetry highlighted, captioned the import lives here, no leaf repo can patch what it does not contain.

Class A — a version bump within the same module path. v3.0.0v3.0.1. Any of the seven can do it alone. MVS does the rest.

Class B — a source change. Dropping the dependency, a major-version migration (gopkg.in/yaml.v3yaml.v4 is a different module path, not a version bump), or an API break the calling code has to absorb.

In class B, none of the seven can do anything. The import statement lives in platform-telemetry’s source, and only platform-telemetry can change it. The remediation surface collapses from seven to one.

Same CVE, same seven repos, completely different plan — and the deciding factor is a property of the fix, not of your dependency graph. Nothing in any go.mod tells you which one you’re in. That’s a judgement call about the upgrade itself, and it’s the first thing worth establishing when an advisory lands.

When it’s class B, order stops being optional

For class A, the seven PRs are independent and can land in any order. For class B they aren’t independent at all, because the fix has to propagate outward from the one repo that can make it.

That’s a topological question, and with the graph loaded it’s a query:

wave, repo_count, release_together
1,    1,          [platform-telemetry]
2,    2,          [platform-authz, platform-httpkit]

Telemetry ships first because nothing blocks it. httpkit and authz follow — and they can go concurrently, since neither depends on the other. Then the services pick up the new middle layer.

Get that order wrong and the cost is concrete rather than theoretical. Ship httpkit before telemetry and you release httpkit twice: once for whatever you thought you were fixing, again to pick up the telemetry that actually fixed it. Two of the three internal libraries here sit above telemetry, so any ordering that doesn’t start at the bottom wastes releases.

One subtlety worth flagging, since it nearly gave me a wrong answer: you cannot read this ordering off the SBOM’s requirement graph. Pruning means platform-httpkit’s go.mod lists yaml.v3 // indirect, so the SBOM contains a direct httpkit→yaml edge and every module looks one hop from the target. Reachability survives that distortion; depth does not. The waves above come from edges tagged as genuinely declared, recovered by parsing go.mod — which is the trap from the last post, showing up again in a new disguise.

So what do you actually do

Not “always fix at the source.” That’s too clean, and it’s wrong for the urgent case.

For a class-A advisory that’s actively burning — drop-in compatible, no API change — the seven PRs are fine, and they’re faster than waiting on a library release plus two waves of propagation. Ship them. Stop the bleeding.

Then fix the source, so it stays fixed. Emergency remediation and durable remediation are different jobs with different shapes, and conflating them is how you end up either sitting on a live CVE waiting for a library release, or patching the same seven leaves every quarter forever.

The graph doesn’t make that decision. What it does is make both options legible: it tells you the seven and the one, and the fact that those are different numbers is the whole point. Without it you’re picking a remediation strategy from a list of package names, which is a bit like choosing a route from a list of street names.

The fleet, the CI step, the loader and the queries are at husobee/sbom-fleet. The go get transcript above is reproducible against it — v3.0.0 really is vulnerable, v3.0.1 really is the fix, and platform-telemetry really doesn’t change.