fix(cdn): publish build artifacts in an order a reader can trust - #178
Merged
Conversation
Two ordering bugs in executeCDNBuild left windows where the CDN advertised something it had not published. `_manifest.json` went up FIRST. It is the content version pointer — consumers key freshness off its commitSha — so for the entire rest of the build it named a commit whose content, bundles and media manifest were still uploading. Objects go up one at a time: measured on staging, a full rebuild wrote the manifest at 11:01:19 and the last bundle at 11:03:04, a 105-second window (~24s on a selective build). A consumer reading in there pinned the new commitSha to pre-build bodies and, if it caches per commit, never re-read them. This is the invariant #153 protected — the manifest must not outrun the bundle — but that fix covered a build that SKIPPED the bundle, not every build that simply had not written it yet. The manifest now goes last, after every artifact it describes, so a build that dies midway leaves the old manifest pointing at the old, complete content instead of a half-written snapshot. `_media_manifest.json` was written AFTER the stale-object sweep, so it was never in uploadedPaths when the sweep ran: every full rebuild deleted it and re-uploaded it a moment later. The delivery SDK throws on any non-2xx and caches the media manifest for the lifetime of the instance, so a consumer booting inside that window stays broken until it is replaced. It now runs before the sweep, which also keeps the inverse honest — a project with no media assets uploads nothing, the path stays out of uploadedPaths, and the sweep still collects a stale manifest. Both are covered by tests that fail against the previous ordering.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #177 — the two ordering issues found while tracing that incident. Both left windows where the CDN advertised something it had not published.
1.
_manifest.jsonwas published firstIt is the content version pointer: consumers key freshness off its
commitSha. Publishing it at step 4 meant that for the entire rest of the build it named a commit whose content, bundles and media manifest were still uploading — objects go up one at a time.Measured on staging (R2
LastModified, full rebuild):105 seconds on a full rebuild, ~24 s on a selective one. A consumer reading in that window pins the new
commitShato pre-build bodies — and if it caches per commit, never re-reads them.This is the invariant #153 protected ("the manifest must not outrun the bundle"), but that fix covered a build that skipped the bundle. It did not cover every build that simply had not written it yet.
The manifest now goes last, after every artifact it describes. Side benefit: a build that dies midway leaves the old manifest pointing at the old, complete content instead of a half-written snapshot.
2.
_media_manifest.jsonwas deleted and re-uploaded on every full rebuildIt was written at step 8, after the step-7 stale-object sweep — so it was never in
uploadedPathswhen the sweep ran. Every full rebuild deleted it, then re-uploaded it ~1 s later.That window is small but not harmless: the delivery SDK throws on any non-2xx, and
MediaAccessorcaches the media manifest for the lifetime of the instance. A consumer booting inside the window stays broken until it is replaced.It now runs before the sweep. The inverse still holds — a project with no media assets uploads nothing, the path stays out of
uploadedPaths, and the sweep still collects a stale manifest.New order
Tests
Three new cases, verified to fail against the previous ordering:
publishes _manifest.json after every artifact it describes→ old:expected '_bundle/en.json' to be '_manifest.json'never deletes _media_manifest.json during a full rebuild that has media→ old:expected [...] to not include '_media_manifest.json'sweeps a stale _media_manifest.json when the project has no media assets(guards the inverse — passes both ways by design)pnpm test1098/1098 ·pnpm lint0 errors ·nuxt typecheckclean.Note
Neither of these caused the incident in #177 — that was the missing
cdn.build_completeon push-triggered builds, now fixed and verified on staging (webhook build → delivery, HTTP 200). These are the latent races found alongside it.