Skip to content

Add changelog registry reconcile and verify commands - #3741

Open
cotti wants to merge 2 commits into
changelog-registry-scrubber-ownershipfrom
changelog-registry-reconcile-cli
Open

Add changelog registry reconcile and verify commands#3741
cotti wants to merge 2 commits into
changelog-registry-scrubber-ownershipfrom
changelog-registry-reconcile-cli

Conversation

@cotti

@cotti cotti commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Why

Phase 2 of elastic/docs-eng-team#688, stacked on #3738. Once the scrubber Lambda owns the public registry.json (Phase 1), two things are still missing: a way to trigger the cutover heal (every existing public manifest is a legacy pass-through copy that needs one authoritative rebuild — and historical drift needs repairing retroactively), and a way to prove convergence before Phase 3 retires the client-side refresh.

What

  • changelog registry reconcile plans the affected groups — a single scope, or the union of groups discovered in both buckets, so orphan public groups (including manifest-only leftovers) are covered — and sends one explicit reconcile message per group to the scrubber queue: {"kind":"reconcile","version":1,"scope":"bundle"|"changelog","group":"…","correlation_id":"…"}, strictly validated through ChangelogKeys on both ends and serialized with the same source-generated context the Lambda parses with (AOT). The CLI never mutates S3 — the Lambda stays the public bucket's single writer. --dry-run prints the plan; the non-dry-run path asks for confirmation (--yes for non-interactive use). Each run stamps one correlation id and prints a ledger line (group, SQS message id, correlation id) for the cutover checkpoint. Attributes: [RequiresAuth], [CommandIntent(Intent.Destructive | Intent.Idempotent)], [MutationScope(MutationScope.Global)].
  • On a reconcile message the Lambda performs a full group heal: object-level reconcile over the union of both buckets' group listings — scrub/copy what is live in the private bucket, conditionally delete what isn't — then the group reconcile. This is what makes a lost or DLQ-expired scrub event recoverable by message (it subsumes Add changelog registry state discovery and reconciliation commands #3717's republish). Reads the (already-provisioned) PRIVATE_BUCKET_NAME Lambda env var — S3 events keep working without it since they carry their source bucket. Malformed reconcile messages are rejected to the DLQ, where the Phase 0 alarm makes the sender bug visible — redelivery can't fix a bad message.
  • changelog registry verify is the read-only sibling and the cutover gate: for each planned group it compares the public manifest against what a reconcile of the current public listing would write — same listing spec and entry rules by construction (RegistryReconciler.VerifyGroupAsync) — and reports divergence using Add changelog registry state discovery and reconciliation commands #3717's taxonomy: Missing / Stale / Corrupt / ObjectDivergent, with UnsupportedSchema reported distinctly. Non-zero exit on any divergence.
  • Manifest ETag wire-format fix: the snake_case naming policy serialized the producer-side ETag as "e_tag", while the consumer contract (ChangelogRegistryBundle) and the documented format read "etag" — every recorded ETag has been invisible to every consumer. [JsonPropertyName("etag")] aligns the writer with the documented format; old "e_tag" manifests now parse as corrupt and get rebuilt, which the cutover forces anyway via the producer mismatch.
  • docs/cli-schema.json regenerated; supplemental CLI pages added under docs/cli/changelog/registry/.

Part of elastic/docs-eng-team#688 (Phase 2).

🤖 Generated with Claude Code

Phase 2 of elastic/docs-eng-team#688. The cutover/heal tooling for the
Lambda-owned public registry:

- `changelog registry reconcile` plans groups (one scope, or the union of
  both buckets so orphan public groups are covered) and sends one versioned,
  discriminated reconcile message per group to the scrubber queue —
  {kind, version, scope, group, correlation_id}, validated through
  ChangelogKeys on both ends. The CLI never mutates S3; the Lambda stays the
  public bucket's single writer. --dry-run prints the plan; the non-dry-run
  path asks for confirmation (--yes for CI). Each run stamps one correlation
  id and prints a ledger line per group.
- On a reconcile message the Lambda performs a full group heal:
  object-level reconcile over the union of both buckets' listings (copy
  what's live, delete what isn't), then the group reconcile — recovering
  lost/DLQ-expired scrub events. Requires the new optional
  PRIVATE_BUCKET_NAME Lambda env var; malformed messages are rejected to
  the DLQ where the Phase 0 alarm surfaces them.
- `changelog registry verify` is the read-only sibling and cutover gate:
  compares each public manifest against what a reconcile would write (same
  listing spec and entry rules by construction) and reports divergence as
  missing/stale/corrupt/object-divergent, with unsupported schemas reported
  distinctly.
- Fix the manifest ETag wire format: the snake_case policy serialized the
  producer-side field as "e_tag" while consumers and the documented format
  read "etag" — recorded ETags were invisible to every consumer.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Same suppression RegistryBuilderTests carries: xUnit owns the test class
lifetime and TestDiagnosticsCollector needs no disposal in these tests.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@cotti
cotti marked this pull request as ready for review July 29, 2026 01:09
@cotti
cotti requested review from a team as code owners July 29, 2026 01:09
@cotti
cotti requested a review from technige July 29, 2026 01:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant