JITSU-181: export integration tests + zod output validation for admin config exports - #1478
Conversation
…exports Every bulker-connections / rotor-connections row is validated against its consumer contract before it reaches the response stream; a violating row is skipped-and-logged with the System error marker instead of shipping malformed. Integration tests cover the JITSU-158 incident shapes against a seeded testcontainers Postgres. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…link guard in streams export, redact backup log, relax destinationId Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
![jitsu-code-review[bot]](https://avatars.githubusercontent.com/in/3391383?s=60&v=4){kind=small}
There was a problem hiding this comment.
Reviewed the changes in , , and the new integration coverage in .
I focused on correctness, regression risk, and failure-path behavior around export row materialization/validation and per-entity error handling. I did not find actionable bugs, security issues, or user-visible regressions in this patch.
There was a problem hiding this comment.
Reviewed the changes in webapps/console/pages/api/admin/export/[name]/index.ts, webapps/console/lib/schema/export-contracts.ts, and the new integration coverage in webapps/console/tests/integration/admin-exports.test.ts.
I focused on correctness, regression risk, and failure-path behavior around export row materialization/validation and per-entity error handling. I did not find actionable bugs, security issues, or user-visible regressions in this patch.
absorbb
added
canary:console
|
🚀 |
…order (#1480) Follow-up to the config-keeper circuit breaker (#1476), from today's prod field experience. Linear: `JITSU-182`. ## Why Within hours of the breaker going live, #1478's deploy reordered JSON keys in every export row (zod `parse` emits schema keys first). Content was byte-for-byte identical after key normalization — but the breaker hashes raw row bytes, so it read 100% mass change and tripped fleet-wide on both replicas (correctly per its design, but the "serializer changes are rare" trade-off documented in the code did not survive day one). The breaker is currently disabled in prod pending this fix — all repos must survive field reorders. ## What - `hashRows` now hashes each row **canonically**: object keys sorted, ',' separator + length-prefixed keys (uniquely decodable — an encoding-collision construction from review is closed), type-tagged values, numbers by literal via `json.Number` (no float64 precision loss), decoder-normalized string escapes. - Key-order changes at any nesting depth are invisible to the breaker; real value changes still trip. Number-literal formatting changes (`1` vs `1.0`) still trip by design — single stable producer, and that failure mode is a confirmation, not a miss. - **No baseline migration needed**: seeding re-hashes the cached payload with the new function, so deploying this cannot itself trip — even onto a payload that byte-changed since the cache was written. - The whole-payload `lastRejected` short-circuit deliberately stays byte-based (it only suppresses re-parsing of a byte-identical rejected generation). Tests: reorder-immunity end-to-end (incl. clearing a held state with a reordered good payload), collision-resistance of the encoding (string/number/boundary/precision cases), escape normalization, literal sensitivity. 25 config-keeper tests green. After deploy, re-enable with `CFGKPR_BREAKER_ENABLED=true` in jitsu-cloud-infra. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
1 participant
Covers postmortem action items 4 and 5 of
JITSU-158(prevent). Linear:JITSU-181Output validation (item 5)
lib/schema/export-contracts.ts: per-variant consumer contracts (BulkerConnectionRow,BackupConnectionRow,RotorConnectionRow,RotorDestinationRow), enforced at every write site ofbulker-connections/rotor-connectionsbefore a row reaches the response stream. A violating row is skipped-and-logged with theSystem error:alerting marker — never shipped malformed. The previously-verbatim otlp and ee-api backup rows are validated too.parseLinkDatanow rejects corrupt (non-object/array) stored roots outright, so the incident shape can't collapse to{}-options in any export that uses it (incl.streams-with-destinations, where a corrupt link now drops only itself — never the stream row, which would have cut off the site's ingest routing).streamId(one legacy row) and nullabledestinationId(profile builders).Integration tests (item 4)
__tests__/integration/admin-exports.test.tsagainst seeded testcontainers Postgres, covering the incident shapes: null-datalink,{}link (both materialize destination-schema defaults,frequencycorrectly absent), wrong-typed fields (tolerant fallback ships stored fields as-is), full round-trip (stored options + materialized defaults), corrupt-root rows skipped-and-logged end-to-end in both connection exports, stream row survival with a corrupt link, and contract-level rejection of the{data_}-typo regression shape.🤖 Generated with Claude Code