Skip to content

Phase 3: assets - #18

Open
patshone-gsl wants to merge 18 commits into
mainfrom
phase-3/assets
Open

Phase 3: assets#18
patshone-gsl wants to merge 18 commits into
mainfrom
phase-3/assets

Conversation

@patshone-gsl

Copy link
Copy Markdown
Member

FEATURE
Phase 3 of the GitBook → Astro Starlight migration. Moves the 500 referenced GitBook assets into src/assets/ and public/, slugified and re-encoded, and makes the page generator emit their real paths from a generated asset-map.json. Before this branch every image on the migrated site 404'd; src/assets now sits at 32.0 MiB against a 60 MB gate.

Docs

No Jira ticket — this is migration phase work, following the same branch and PR pattern as #17 (phase-2/conversion-script).

Design — docs/superpowers/specs/2026-07-30-phase-3-assets-design.md
Plan — docs/superpowers/plans/2026-07-30-phase-3-assets.md
Findings — the 2026-07-30 — Phase 3 gate: assets complete section of MIGRATION-NOTES.md

Solution overview

Current behavior

convert.mjs emitted root-absolute placeholders of the form /.gitbook/assets/<original filename> for every image. Nothing resolved them, so every image on the site 404'd. The 1,589 source assets sat only in the git-ignored source/ tree, 541 MiB of them, unoptimised and unreferenced by the build.

Approach in this PR

scripts/assets.mjs owns the physical files and emits asset-map.json; scripts/convert.mjs consumes that map exactly as it already consumes route-map.json.

That direction is the load-bearing decision. The brief described assets.mjs rewriting references in the generated pages after the fact — but convert.mjs regenerates all 204 pages from scratch and re-runs against fresh GitBook syncs right up to cutover day, so an after-the-fact rewriter would silently revert all 500 asset paths on every run, leaving a green build and a site with no images. One generator, correct on every run.

  • Copy set derived from source/ alone, never from generated output — a map derived from converted pages would need itself first. 509 references minus 9 covers that conversion drops = 500.
  • scripts/lib/asset-plan.mjs is pure — no filesystem, no image decoding — so every naming and treatment decision is unit-testable. All I/O lives in assets.mjs.
  • Slugification: lowercase, non-alphanumeric runs collapsed to single hyphens, extension preserved and lowercased. Zero collisions across all 509 references; collision handling still exists because the input is whatever GitBook syncs next.
  • Encoding: resize to a 2000px ceiling, then palette-quantise only if the resized image has ≤32,768 unique colours. The count must be taken on the resized image — an earlier draft measured at 400px, and because unique colours scale with pixel count that admitted 197 of 505 images instead of the 11 predicted, projecting 84.6 MiB against a 60 MB gate.
  • Source format is preserved: PNG→PNG quantised, JPEG→JPEG q90, WebP→WebP, so the extension always matches the content. The design had assumed all 496 stills were PNG; measured, it is 487 PNG + 7 JPEG + 2 WebP.
  • The 22.56 MiB GIF becomes a 1.12 MiB MP4 via ffmpeg and emits as <video autoplay loop muted playsinline>. Video and downloads cannot pass through astro:assets, so they land in public/media/ and public/files/.
  • Content-hash cache: a re-run reuses all 500 in 0.4s rather than ~89s, so putting assets inside npm run convert doesn't tempt an operator into running convert.mjs alone — which is the ordering trap the map exists to prevent.
  • The sweep only deletes files the script created, driven by the previous map's entries and keyed on the (destination, slug) pair. src/assets/ is not script-owned — it holds the site logo referenced from astro.config.mjs.
  • Loud failure throughout. The one sanctioned exception is assetPath's no-map fallback, so a checkout where this phase never ran still builds.

Breaking changes

None to URLs, routes or prose. npm run convert gains an assets step and now runs routes → assets → convert → sidebar.

Two behavioural notes for anyone working on the scripts:

  • EXPECTED.images moved 529 → 528. This is the only expectation adjusted in the phase, and it is single-cause: the corpus's sole video asset now renders as a <video> element rather than a markdown image. Verified three independent ways — a page-only diff of exactly 516 insertions / 516 deletions with nothing added or removed, reference sites totalling 530 before and after, and a direct on-disk count.
  • Changing MAX_WIDTH, QUANTISE_MAX_COLOURS or GIF_VIDEO_THRESHOLD requires deleting asset-map.json first, or every asset cache-hits and the new value silently never takes effect. Documented in the code.

Dependencies

  • src/assets/, public/ and asset-map.json are committed — they are the deliverable.
  • ffmpeg is required only when the source GIF's bytes change; the encoded MP4 is committed and the cache keys on content, so ffmpeg is not on the cutover critical path.
  • Phase 4 (look and feel) inherits: the 9 dropped card covers remain in source/ and are restored by putting them back through this same pipeline; the photograph among them must stay full colour. Continuation prompt at docs/superpowers/PHASE-4-CONTINUATION.md.
  • CLAUDE.md corrected in three places — its command description had assets.mjs "rewriting refs" (the rejected architecture), and two Gotchas asserted things this phase measured false.

Verification

  • npm run convert green through all four scripts; npx astro build succeeds with 205 pages and 496 optimised image variants.
  • 523 emitted references checked, 0 broken — resolved against the filesystem rather than counted as text. 523 reconciles with Phase 2's independent count of placeholder references.
  • src/assets 33,591,322 B = 32.0 MiB against a 60,000,000 gate. Largest file anywhere 1,179,445 B = 1.12 MiB, against Cloudflare's 25 MiB per-file cap.
  • 187/187 tests. Second run byte-identical; both small animated GIFs still animate (279 and 26 frames).
  • Deployed and verified live at https://opendialog-docs.opendialog.workers.dev — 8/8 images on a sampled page, 30/30 sampled sitemap routes, MP4 and CSV serving with correct content types, no-trailing-slash URLs returning 200. docs.opendialog.ai is untouched and still points at GitBook.

Review notes

Every task was reviewed individually and the branch got a final whole-branch review. Between them they found 14 real defects in the plan's never-executed draft code, four of which would have shipped silently with a green build:

  • the cover/non-cover split was computed per file, silently dropping a screenshot referenced both as a card cover and as a figure on the same page;
  • the destination clear, and later the sweep, deleted the site logo on every run;
  • the CSV download was never mapped, and the check meant to catch that was blind to markdown links;
  • a plain markdown link to an asset would 404 with a green build — latent today, closed by a new survivingAssetPaths assertion.

Two further defects were errors in the dispatch instructions rather than the code, caught by reviewers.

greenshootbot and others added 18 commits July 30, 2026 12:08
Measured five corrections to the brief: orphan deletion is not a step since
source/ is git-ignored and regenerable, all 50 extension-less files are
orphans, the 28 MB GIF is unreferenced, resizing alone cannot meet the 60 MB
gate but palette quantisation can, and the ~/* alias removes the need for
per-file relative paths.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017g54jmeHWdVo2uJ8vScwD5
Eight tasks: cover target extraction, pure naming and treatment planning,
copy-set extraction, the asset script copying verbatim, map-aware reference
emission, image encoding, video encoding, and the phase gate.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017g54jmeHWdVo2uJ8vScwD5
A missing or non-numeric byte count previously fell through the size
comparison as NaN >= threshold (false), silently copying a large GIF
into src/assets instead of throwing. Guard mirrors the existing
still-image colours check per the loud-failure constraint.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017g54jmeHWdVo2uJ8vScwD5
Removed claims of re-encoding and hash-based skip that this commit does not
implement; this task copies byte-for-byte and always re-copies.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017g54jmeHWdVo2uJ8vScwD5
convert.mjs now consumes asset-map.json to resolve every .gitbook/assets
reference to its real src/assets/, public/media/ or public/files/ location,
replacing the root-absolute placeholders that made every image 404. Without
a map (source/ has never had assets.mjs run against it) the placeholder is
kept so the build stays green; a reference missing from a map that does
exist throws.

convertFile (the {% file %} block, the corpus's one download rather than
image) is made map-aware the same way, reusing figures.mjs's NEEDS_ANGLE
predicate rather than always bracketing the link. The Step 6 on-disk
assertion is extended to resolve markdown-link destinations
([name](/files/…)) as well as quoted attributes (<video src="/media/…">),
so the download's single reference is checked too.

images drops from 529 to 528: the corpus's one video-kind asset
(Knowledge Base Demo.gif) now renders as a <video> element instead of a
markdown image, and that count was measured before the map distinguished
kinds.

npm run convert now runs assets.mjs before convert.mjs so the map exists
by the time it's needed.
- Reword the EXPECTED.images comment: it counts markdown-image occurrences
  across the corpus, not distinct assets, and states the one-occurrence
  delta caused by the sole video-kind asset explicitly rather than in a
  way that read as 500 - 1 = 499.

- Consolidate the .gitbook/assets name-unescaping rule into asset-refs.mjs
  (exported as ASSET_SRC and unescapeAssetName, with the try/catch and
  named error) and have figures.mjs import it, rather than keeping a
  second, silently driftable copy of the same regex and decode chain.

- Carry a video's alt text forward as an aria-label, guarded so an empty
  alt emits no empty attribute — the alt text was being dropped with no
  fallback for the one asset this task turns into a <video> element.
Resizes stills to a 2000px width ceiling and palette-quantises PNGs
below a colour threshold, preserving each source's own format so a
.jpg file never ends up holding PNG bytes. A content-hash cache skips
re-encoding unchanged assets, and the destination sweep is driven by
the previous run's map rather than a directory listing, so files this
script never wrote (the site logo) are never touched.

Brings src/assets from 121.4 MiB to 32.0 MiB, under the 60 MB gate.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017g54jmeHWdVo2uJ8vScwD5
A slug is not unique across destinations over time: a GIF crossing
GIF_VIDEO_THRESHOLD keeps its filename but moves between src/assets
and public/media between runs. Keying the sweep on slug alone let a
newly claimed slug in its new destination mask a stale copy left
behind in its old one, which then had no run that would ever remove
it. Legacy map entries without a destination field still resolve via
DESTINATION_FOR_KIND, since kind has always determined destination
one-to-one.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017g54jmeHWdVo2uJ8vScwD5
Knowledge Base Demo.gif was classified kind: video but copied verbatim, so
the page emitted a <video> pointing at a .gif that no browser can play. Adds
a deterministic ffmpeg encode (bitexact flags, stripped metadata) so the
same source always produces byte-identical output, and a guard in
figures.mjs that throws if a video-kind map entry ever again resolves to a
non-video container.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017g54jmeHWdVo2uJ8vScwD5
Assert no .gitbook/assets path survives conversion when the asset map is
present, route the sweep's claimed set through destinationKeyFor, correct
CLAUDE.md's stale gotchas and the assets.mjs command description, and record
the deferred findings and the corpus-count adjudication procedure.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017g54jmeHWdVo2uJ8vScwD5
Carries the look-and-feel scope with its measured specifics: the 41 repeated
sidebar parents, the 13 wrapper divs across 9 pages and 31 images, the 13/10/9
card-cover chain, the 64 dropped width attributes, and the progress-bar-message
markup decision still awaiting a human answer.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017g54jmeHWdVo2uJ8vScwD5
@patshone-gsl patshone-gsl self-assigned this Jul 30, 2026
@patshone-gsl
patshone-gsl requested a review from Copilot July 30, 2026 14:13

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review this pull request because it exceeds the maximum number of files (300). Try reducing the number of changed files and requesting a review from Copilot again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants