flyingrobots · flyingrobots · Aug 10, 2026 · Aug 10, 2026 · Aug 10, 2026 · chatgpt-codex-connector
@@ -5,6 +5,10 @@
 > For a given content identity, Keep must return exactly the bytes named by
 > that identity—or refuse.
 
+The [authenticated reconstruction contract](docs/invariants/authenticated-reconstruction/README.md)
+defines the proof scopes, output-failure rule, receipt posture, and precise
+limits of that promise.
+
 Keep is a standalone Rust library for durable, content-addressed storage. It is
 intended to provide streaming ingestion, content-defined chunking, physical
 deduplication, exact range reads, explicit retention, integrity verification,

@@ -0,0 +1,254 @@
+# Authenticated Reconstruction Contract
+
+**Status:** Normative for every Keep operation that claims authenticated
+reconstruction. The public non-durable `ReferenceStore` implements the
+complete-object and exact-range forms. A consolidated durable logical-read
+surface is not yet implemented.
+
+The [rationale](rationale.md) records the governed decisions and rejected
+alternatives. The [requirement ledger](requirements.md) maps each law to its
+oracle and executable evidence or names the remaining gap.
+
+## Contract
+
+For a requested content identity, Keep either:
+
+1. establishes the proof scope named by the operation;
+2. emits exactly the bytes supported by that proof;
+3. returns a receipt stating the exact proposition established;
+
+or it returns a precise, evidenced content-related refusal;
+or it returns an operational failure that makes no claim about content truth.
+
+Keep never substitutes different content, promotes a narrower proof into a
+broader one, infers truth from physical existence, or silently repairs
+ambiguous state.
+
+```text
+typed content coordinate
++ admitted immutable evidence
++ declared proof scope
++ caller-owned output
+────────────────────────────
+authenticated bytes + receipt
+                     or evidenced refusal
+                     or operational failure
+```
+
+This contract refines Keep's core law:
+
+> For a given content identity, Keep must return exactly the bytes named by
+> that identity—or refuse.
+
+## Meaning of authenticated
+
+In this contract, **authenticated** means that Keep established the emitted
+bytes against the requested Keep content coordinate using the admitted
+identity law and the evidence required by the operation's proof scope.
+
+Authentication here does not establish:
+
+- authorship or provenance;
+- application meaning;
+- trustworthiness of the content;
+- caller authorization;
+- causal or publication authority outside Keep;
+- durability beyond the evidence named by the receipt.
+
+Those propositions require their own owners and evidence.
+
+## Coordinates and evidence
+
+A `BlobId` names one exact finite logical byte sequence. It is independent of
+chunking, layouts, representations, physical locations, and retention. Parsing
+a `BlobId` proves only that the coordinate is canonical and supported. It does
+not prove presence or retention.
+
+A `LayoutId` names one canonical reconstruction plan. One `BlobId` may have
+more than one admitted layout. Every admitted layout binds exactly one target
+`BlobId`.
+
+Physical paths, file names, offsets, inode values, object keys, and successful
+raw reads are evidence inputs, not stable content identity. They cannot
+authorize output without the required identity and structural verification.
+
+The following claims remain distinct:
+
+```text
+canonical coordinate
+    ≠ content present
+    ≠ content published
+    ≠ content retained
+    ≠ content durable
+    ≠ content reconstructible from this admitted view
+```
+
+## Complete-object reconstruction
+
+A successful complete-object reconstruction proves that:
+
+- every selected chunk matched its `ChunkId`;
+- the admitted storage-profile boundaries matched the layout;
+- the complete reconstructed sequence matched the requested `BlobId`;
+- the exact byte count reported by the receipt was written.
+
+The current public reference shape is:
+
+```rust
+fn reconstruct<W: std::io::Write>(
+    &self,
+    target: BlobId,
+    output: &mut W,
+) -> Result<ReconstructionReceipt, ReconstructionError>;
+```
+
+When more than one committed layout names the requested blob and the caller
+does not request an exact layout, `ReferenceStore` deterministically selects
+the lowest canonical `LayoutId`.
+
+When the caller requests an exact `LayoutId`, Keep must use that layout or
+refuse. It must not fall back to another layout, even if another layout could
+lawfully reconstruct the same `BlobId`.
+
+Different bytes or a different target `BlobId` are never lawful substitutes.
+
+## Exact-range reconstruction
+
+An exact-range operation has a deliberately narrower proof scope. Its receipt
+proves that the requested bytes came from completely authenticated overlapping
+chunks under a layout-to-target binding admitted by the selected store view.
+Structural admission of caller-supplied layout fields alone is insufficient.
+The current caller-supplied entry points calculate the canonical `LayoutId`
+and resolve that exact committed layout before planning or output.
+
+It does not prove that Keep authenticated:
+
+- unrequested chunks;
+- the complete logical blob;
+- any storage-profile boundary.
+
+A `RangeReadReceipt` must never satisfy an API that requires a complete-object
+`ReconstructionReceipt`. Complete and range operations remain separate public
+surfaces; an optional range parameter must not erase the proof distinction.
+
+## Output visibility and failure
+
+The generic reconstruction API accepts an ordinary caller-owned `Write` sink.
+An ordinary sink is not transactional. It may fail after accepting a prefix,
+and an emission-time storage failure may occur after a prefix has been
+written.
+
+Therefore:
+
+> A successful receipt authenticates the complete emitted sequence. On
+> failure, any bytes already written are uncommitted and must not be consumed
+> as authenticated output.
+
+Callers that require atomic visibility must provide a quarantined or
+transactional sink and reveal or promote its bytes only after validating the
+complete success receipt. Keep does not claim that an unsuccessful call left
+an arbitrary `Write` untouched.
+
+The current `ReferenceStore` verifies the complete realization before its
+first output write, then reverifies each immutable chunk immediately before
+emission. This prevents known unauthenticated content from being emitted; it
+does not make the caller's sink atomic.
+
+## Decisions, refusals, and operation failures
+
+A typed error does not automatically constitute evidence about content truth.
+Consumers must distinguish at least:
+
+- authenticated success;
+- an evidenced refusal supported by a complete admitted view;
+- an operational failure from which no content conclusion follows.
+
+For example, absence can be evidenced only when the admitted view and its
+indexes are complete enough to prove non-membership. A timeout, unreadable
+catalog, exhausted resource limit, cancellation, or unavailable capability
+does not prove absence.
+
+Corruption evidence establishes only the exact integrity proposition its
+evidence supports. An unreadable or internally inconsistent view may prevent
+both a positive reconstruction claim and a negative absence claim.
+
+The current `ReferenceStore` exposes boundary-specific Rust errors rather than
+a persisted refusal-receipt format. Any future durable refusal receipt must
+bind enough coordinates to replay its proposition, including:
+
+- the requested `BlobId`;
+- the admitted immutable view or generation;
+- an exact requested `LayoutId`, when present;
+- refusal classification;
+- proof stage or evidence coordinate;
+- contract version.
+
+## Receipt posture
+
+A reconstruction receipt is, by default, an ephemeral statement about one
+completed operation. It is not automatically a portable, self-contained
+cryptographic proof.
+
+A receipt may support later revalidation only when all evidence it names
+remains available and admitted under the same contract. A durable receipt must
+name the immutable store view or generation against which the result was
+established. Retaining a receipt without retaining its supporting evidence
+does not preserve the original proof.
+
+Current `ReconstructionReceipt` values bind:
+
+- target `BlobId`;
+- selected `LayoutId`;
+- exact authenticated byte count written.
+
+Current `RangeReadReceipt` values additionally bind the requested half-open
+range and explicitly carry the narrower range proof posture.
+
+## Durable reconstruction requirement
+
+A future operation claiming durable logical reconstruction must additionally:
+
+- bind reads to one admitted immutable snapshot or catalog generation;
+- prevent required supporting evidence from being garbage-collected, deleted,
+  or otherwise invalidated during the read;
+- verify the retained closure required by its declared proof scope;
+- resolve and authenticate exact immutable records;
+- preserve the selected view while successor generations publish;
+- return a receipt naming that view;
+- separate evidenced refusal from operational failure;
+- preserve the output-visibility rule above.
+
+The current durable segment, catalog, publication, and recovery surfaces do
+not yet form this consolidated high-level `BlobId`-to-writer contract.
+Retention remains planned; no current retention surface protects the evidence
+closure required by this operation. These lower-level surfaces must not be
+described as an implemented durable logical reconstruction API.
+
+## Current public evidence
+
+The non-durable `ReferenceStore` is the executable oracle for this contract.
+Its committed state is process memory; process death loses it all.
+
+Evidence anchors:
+
+- [authenticated-reconstruction requirements](requirements.md)
+- [authenticated-reconstruction rationale](rationale.md)
+- [`ReferenceStore` architecture](../../architecture/reference-store/README.md)
+- [`ReferenceStore` rationale](../../architecture/reference-store/rationale.md)
+- [exact logical byte identity](../../adr/0001-exact-logical-byte-identity.md)
+- [identity and physical-storage separation](../../adr/0002-separate-identity-from-physical-storage.md)
+- [`ReferenceStore` contract tests](../../../tests/reference_store_contract.rs)
+- [reconstruction implementation](../../../src/reference/reconstruction.rs)
+- [range-read implementation](../../../src/reference/range_read.rs)
+- [whole-object refusal laws](../../../tests/streaming_cas/refusal_laws.rs)
+- [range-read refusal laws](../../../tests/range_read_failures.rs)
+- [reconstruction receipt](../../../src/reference/reconstruction_receipt.rs)
+- [range-read receipt](../../../src/reference/range_read_receipt.rs)
+
+## Consumer rule
+
+Consumers may wrap Keep in application-specific ports, transactional output,
+identity bindings, and causal workflows. Those adapters may strengthen output
+visibility or attach additional meaning. They must not weaken Keep's proof
+scope, treat an operational failure as content evidence, or import
+application-specific semantics into Keep core.
@@ -0,0 +1,91 @@
+# Authenticated Reconstruction Rationale
+
+## Decision
+
+Keep names authenticated reconstruction as one proof-scoped operation rather
+than a generic blob lookup. A successful operation emits exact bytes and
+returns a receipt for the proposition it established. An evidenced refusal
+supports a narrower content proposition. An operational failure supports no
+content conclusion.
+
+Complete-object and exact-range operations remain separate. Complete-object
+reconstruction verifies every chunk, the registered storage-profile
+boundaries, and the complete `BlobId`. Exact-range reconstruction verifies the
+complete identities of overlapping chunks under a layout-to-target binding
+admitted by the selected store view. It verifies neither the complete blob nor
+any storage-profile boundary.
+
+The generic output boundary remains an ordinary caller-owned `Write`. A
+successful receipt authenticates the complete emitted sequence. Failure may
+leave an untrusted prefix, so consumers that require atomic visibility must
+quarantine output and publish it transactionally after receipt validation.
+
+Current `ReferenceStore` behavior is the executable oracle for non-durable
+complete-object and range forms. It is not evidence that Keep has a durable
+logical reconstruction API. A future durable form must pin one immutable view,
+retain its complete supporting evidence, and bind the view into its result.
+
+## Governed surfaces
+
+This decision governs:
+
+- logical identity versus physical realization;
+- complete-object and exact-range proof scopes;
+- receipt and refusal meaning;
+- output visibility after failure;
+- layout selection and committed layout-to-target binding;
+- the future durable read aperture and evidence-retention obligation; and
+- the public integration boundary available to consumers.
+
+It does not govern Echo semantics, causal authority, application retry law, or
+cross-store publication.
+
+## Alternatives rejected
+
+### Return `Option<Arc<[u8]>>`
+
+This collapses absence, corruption, unavailable evidence, and operational
+failure. It also requires hidden whole-object materialization and cannot carry
+proof scope or a receipt.
+
+### Use one read operation with an optional range
+
+An optional range makes it easy to pass a range receipt where complete-object
+proof is required. Separate operations and receipt types keep the narrower
+proof scope visible.
+
+### Trust any structurally admitted caller layout for range reads
+
+Structural validation proves offsets, lengths, and canonical layout shape. It
+does not prove that the target `BlobId` names the listed chunks. Range receipts
+therefore require the exact layout-to-target binding admitted by the selected
+store view.
+
+### Treat every error as an evidenced refusal
+
+Writer failure, cancellation, resource exhaustion, and unreadable evidence do
+not establish a proposition about content. Converting them into refusals would
+let infrastructure weather become false storage truth.
+
+### Promise atomic output from an ordinary `Write`
+
+An ordinary writer can accept a prefix and fail. Keep cannot roll back an
+arbitrary external sink. Transactional visibility belongs to a consumer or
+adapter that owns a quarantine and commit protocol.
+
+### Treat a receipt as a portable durable proof
+
+Current receipts state what one completed process established. They do not
+carry all supporting evidence and remain replayable only while named evidence
+is retained under the same contract.
+
+## Consequences
+
+- Consumers must preserve proof scope and outcome class.
+- Range receipts cannot satisfy complete-object requirements.
+- Caller-supplied range layouts must resolve through an admitted store view.
+- A failure after output began returns no success receipt; accepted bytes
+  remain untrusted.
+- Durable integration remains blocked on a pinned-view consumer capability and
+  evidence retention.
+- Application-specific meaning remains outside Keep core.
@@ -0,0 +1,18 @@
+# Authenticated Reconstruction Requirements
+
+This ledger maps the authenticated reconstruction contract to stable laws,
+oracles, evidence classes, and concrete witnesses. “Planned” names a required
+gap, not implementation evidence.
+
+| ID | Exact law | Oracle | Evidence type | Status | Concrete evidence |
+| --- | --- | --- | --- | --- | --- |
+| `KEEP-RECONSTRUCT-001` | A complete-object receipt is returned only after every selected chunk, registered profile boundary, complete `BlobId`, and emitted length verify. | `ReferenceStore` complete reconstruction model | Public API integration, corruption, and mutation tests | Implemented | `tests/streaming_cas/reconstruction_laws.rs`, `tests/streaming_cas/refusal_laws.rs` |
+| `KEEP-RECONSTRUCT-002` | Exact-layout reconstruction uses the requested committed `LayoutId` or refuses without substituting another layout. | Ordered committed-layout model | Public API integration tests | Implemented | `tests/streaming_cas/reconstruction_laws.rs` |
+| `KEEP-RECONSTRUCT-003` | A range receipt proves only requested bytes from authenticated overlapping chunks; it proves neither the complete blob nor any storage-profile boundary. | Minimal-overlap range model | Unit, property, and public API integration tests | Implemented | `src/reference/range_read_tests.rs`, `tests/range_read.rs`, `tests/range_read_properties.rs` |
+| `KEEP-RECONSTRUCT-004` | A range receipt names only a layout-to-target binding admitted by the selected store view. | Forged same-length target-layout fixture | Public API corruption test | Implemented | `tests/range_read_entrypoints.rs` |
+| `KEEP-RECONSTRUCT-005` | Success authenticates the complete emitted sequence; failure returns no success receipt and reports the exact accepted prefix, which remains untrusted. | Deterministic prefix-then-fail writer | Public API failure tests | Implemented | `tests/streaming_cas/refusal_laws.rs`, `tests/range_read_failures.rs` |
+| `KEEP-RECONSTRUCT-006` | Authenticated success, evidenced content refusal, and operational failure remain distinct outcomes; operational failure supports no content conclusion. | Typed outcome classification | Public API integration tests and contract inspection | Implemented for `ReferenceStore`; durable refusal receipts planned | `tests/streaming_cas/refusal_laws.rs`, `tests/range_read_failures.rs`; [Keep #22](https://github.com/flyingrobots/keep/issues/22) |
+| `KEEP-RECONSTRUCT-007` | Whole-object and range receipts bind target, exact layout, proof scope, and exact emitted coordinates without granting retention or application authority. | Receipt type inspection | Public API contract tests | Implemented | `src/reference/reconstruction_receipt.rs`, `src/reference/range_read_receipt.rs`, `tests/range_read_contract.rs` |
+| `KEEP-RECONSTRUCT-008` | Automatic layout choice is deterministic; an exact requested layout never falls back. | Canonically ordered layout set | Unit and public API integration tests | Implemented | `src/reference/store_tests.rs`, `tests/streaming_cas/reconstruction_laws.rs` |
+| `KEEP-RECONSTRUCT-009` | A durable read pins one immutable view and prevents required evidence from being garbage-collected, deleted, or invalidated through completion. | Pinned-generation and retained-closure model | Recovery, concurrency, corruption, and crash-injection tests | Planned gap | [Keep #22](https://github.com/flyingrobots/keep/issues/22), [Keep #23](https://github.com/flyingrobots/keep/issues/23) |
+| `KEEP-RECONSTRUCT-010` | Durable reconstruction names its view and returns either authenticated success, evidenced refusal, or operational failure without hidden whole-blob allocation. | Durable consumer conformance model | Public API integration, memory, recovery, and crash-injection tests | Planned gap | [Keep #22](https://github.com/flyingrobots/keep/issues/22), [Keep #23](https://github.com/flyingrobots/keep/issues/23) |