Skip to content

feat(linkable): add allocation-free postcard encode_into#180

Open
thaodt wants to merge 2 commits into
aimdb-dev:mainfrom
thaodt:feat/177-zero-alloc
Open

feat(linkable): add allocation-free postcard encode_into#180
thaodt wants to merge 2 commits into
aimdb-dev:mainfrom
thaodt:feat/177-zero-alloc

Conversation

@thaodt

@thaodt thaodt commented Jul 12, 2026

Copy link
Copy Markdown
Collaborator

Description

This PR implements the foundation proposed in #177:

  • adds a source-compatible Linkable::encode_into(&mut [u8])
  • reuses the existing aimdb_core::connector::SerializeError
  • adds an optional into-slice serializer path to the core outbound pipeline
  • reuses one bounded scratch buffer per outbound route
  • generates a true allocation-free Postcard override using postcard::to_slice
  • retains the existing owned Vec<u8> path for legacy, custom, and oversized values
  • adds generated-code drift coverage and an allocation-counting benchmark

Each outbound route allocates its scratch buffer once when its pump starts and reuses it serially across messages.

No reference crosses the boxed reader-future boundary. The reader returns only payload metadata, while pump_sink owns the scratch storage, validates the returned
length, and borrows the encoded prefix until publish().await completes. This requires no new unsafe.

When encode_into returns SerializeError::BufferTooSmall, the value is serialized once through the existing owned serializer:

BufferTooSmall -> to_bytes() -> SerializedPayload::Owned(Vec<u8>)

Other serialization errors retain the existing log-and-skip behavior.

postcard codegen

Generated postcard Linkable impls now:

  • advertise ENCODE_BUFFER_CAPACITY = Some(256)
  • use postcard::to_slice in encode_into
  • map SerializeBufferFull to LinkCodecError::BufferTooSmall
  • retain postcard::to_allocvec as the compatibility and oversized-value fallback

The 256-byte capacity is a performance budget, not a wire-size limit. Generated outbound connector chains install with_serializer_into, JSON and custom codecs remain unchanged.

Compatibility and error scope

This implementation takes the compatibility-first approach agreed in #177's comment:

  • existing Linkable::from_bytes and Linkable::to_bytes signatures continue to return String
  • existing manual Linkable implementations continue to compile through the default encode_into implementation
  • existing SerializedReader implementations continue to work through the default owned adapter
  • the new into-slice path reuses SerializeError::{BufferTooSmall, InvalidData}
  • the session-level aimdb_core::CodecError remains unchanged

Allocation claim

The measured claim is intentionally limited to the codec seam:

Generated postcard Linkable::encode_into performs no heap allocations on the successful path.

The allocation benchmark executes 10000 encodes and reports:

=== B0 Linkable Postcard encode_into ===
  Iterations  : 10000
  Allocations : 0
  Bytes       : 0

This PR does not claim that the complete outbound connector path is allocation-free. The existing boxed SerializedReader future, dynamic topic generation and connector-internal payload ownership or copies remain outside this scope.

Related Issue

This also provides the allocation-aware foundation for #178.

Checklist

  • I have read the CONTRIBUTING.md document.
  • My code follows the project's coding standards.
  • I have added tests to cover my changes.
  • All new and existing tests passed (make check).
  • I have updated the documentation accordingly.

thaodt added 2 commits July 12, 2026 15:10
  Add a caller-provided buffer API to `Linkable` while retaining the
  existing from_bytes and to_bytes signatures for compatibility.

  Introduce `LinkCodecError` and an optional into-slice serializer path in
  core. Each outbound route owns one bounded reusable scratch buffer,
  validates the returned length, and falls back to the existing `Vec`
  serializer when the buffer is too small. Legacy and custom codecs keep
  their previous behavior.

  Generate a true postcard override using `postcard::to_slice` with a
  256-byte preferred capacity. Extend codegen drift coverage for generated
  connector wiring, exact and undersized buffers, Cortex-M compilation
  and the JSON-free dependency graph.

  Add an allocation-counting benchmark covering 10000 `encode_into` calls
  with zero allocation calls and zero allocated bytes.

  This removes the codec allocation only, connector-internal payload
  copies remain outside the claim.
  Remove the newly introduced `LinkCodecError` and reuse the existing
  `aimdb_core::connector::SerializeError` across the into-slice API,
  postcard codegen, tests, benchmarks and documentation.

  Keep the legacy Linkable `from_bytes`/`to_bytes` String errors unchanged
  under the compatibility-first approach agreed in aimdb-dev#177.
@thaodt thaodt marked this pull request as ready for review July 12, 2026 15:34
@thaodt thaodt requested a review from lxsaah as a code owner July 12, 2026 15:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEAT] Zero-allocation Linkable::encode_into(&mut [u8])

1 participant