feat(module): five new capability families and OpenStore for per-subtree stores

[senamakel]

Summary

• Widens the module's served surface with five new capability families — People, Chunks, Retrieval, Profile, Episodic — so the host can reach through the contract instead of calling the engine in-process.
• Adds OpenStore(memory_subdir) -> object_path to the root object, letting one process serve more than one memory store without a store selector on every method.
• Adds create_memory_client_in_subdir to tinymemory-core; the existing client factory hardcoded "memory".
• Bumps the contract to (2, 2) — a minor bump, since a new family is made safe by capability negotiation alone.

Problem

OpenHuman is dropping the tinymemory* crates from its binary and reaching memory only through this module. Two things blocked that, and neither was a small gap:

1. Four host surfaces had no wire equivalent — the people store, the chunk tier, the deterministic retrieval primitives, and the learned-facet profile. Each was reached by constructing an engine in the host process, which is precisely the split brain the module exists to end.

2. The archivist held a live rusqlite::Connection. A connection cannot cross a bus, so while it existed the engine could never leave, whatever happened to everything else.

3. A host with per-profile memory needs more than one store in a process. setup built exactly one client from config.workspace_dir and served one object, so dedicatedMemory profiles had nowhere to go.

Solution

The four families mirror the existing service shape: owned equivalents of the trait signatures, scope always an explicit argument (never inferred — the engine's task-local belongs to the host's task, and inferring here would read as absent, which means unrestricted, which is a source gate failing open).

Episodic is the archivist's connection, decomposed. It turned out to need far less than "a raw connection" suggests: no ad-hoc SQL, no schema knowledge, ten typed calls — and two of those took no connection at all. Those two (boundary detection and the fallback summary) are host policy and stayed in OpenHuman; the other seven are this family. One deliberate change on the way across: InsertTurn returns the row id rather than leaving the caller to issue SELECT last_insert_rowid(). That was reading connection-local state — an interleaved insert from another task returns the wrong id and files the turn under the wrong segment. Returning it removes the race and a round trip together.

OpenStore is the interesting one, because the obvious designs were both wrong. A store selector on the wire means a parameter on every method of all 18 families, to express something that is not a property of a memory operation. Folding profile subtrees into namespaces avoids that but relocates user data on disk.

Neither is necessary: which store you are talking to is settled when you are handed a driver, exactly like which workspace you are bound to. tinybus already serves many objects, so the root object opens stores and hands back object paths. Each is an ordinary MemoryService exporting the identical interface — the contract does not change, and paths on disk stay where they are.

Four decisions in it worth review:

• Only the root object can open stores. An opened store carries no StoreOpener, so the recursion is finite by construction rather than by a depth check.
• Idempotent per subtree, recorded only after serve_at succeeds. Two live handles to one SQLite file is not hypothetical — the engine migrates on open, and concurrent migrations on one file corrupt it invisibly. Caching the path before the serve succeeded would strand callers on a path nothing answers.
• The object path is derived and character-checked, never free-form. A subdir arrives from a profile id; an id that fails validation must produce a refusal, not a malformed bus path. The rejection message does not echo it — it is user data, and the same reasoning as setup keeps filesystem detail out of MethodFailed.message.
• create_memory_client_in_subdir is a separate entry point, not a parameter added to the existing factory. Every current caller wants the shared tree, and a defaulted subdir argument is exactly the kind of thing that silently routes a store somewhere nobody intended.

Submission Checklist

• Tests added or updated — module suite green (35 tests); service-level coverage for the new families and their error mapping.
• Diff coverage ≥ 80% — N/A for this repo's gate.
• Coverage matrix updated — N/A: no matrix in this repo.
• All affected feature IDs listed — N/A: no matrix in this repo.
• No new external network dependencies introduced.
• Manual smoke checklist updated — N/A: consumed as a module artifact; the release cut is the smoke surface.
• Linked issue closed — N/A: tracked in the consuming repo.

Impact

• Contract (2, 1) → (2, 2), minor: an older host simply never advertises or calls the new families.
• A release must be cut from this branch before OpenHuman can pin it — the host build advertises 18 families and the currently pinned 1.0.1 artifact serves 13. OpenHuman's registry digests must then be taken verbatim from the release's checksum.toml, never recomputed locally.
• Depends on tinyhumansai/tinycortex#148 (the people domain); the vendor/tinycortex pointer here moves with it.

Related

Depends on tinyhumansai/tinycortex#148 — merge that first.

Summary by CodeRabbit

• New Features

  • Added people, chunks, retrieval, profile, and episodic memory capabilities.
  • Added APIs for contact resolution, conversation history, profile facets, chunk access, embeddings, and retrieval queries.
  • Added source-scoped retrieval and support for opening memory stores in specified subdirectories.
  • Added a unified memory handle for advanced integrations.

• Improvements

  • Expanded capability compatibility and updated the contract version.
  • Added response-size safeguards for service requests.
  • Improved embedding signature handling for reliable identifier storage.

[senamakel]

@coderabbitai review

Every actionable finding from the previous rounds is addressed and pushed. Summary of the substantive ones:

• open_store race — the guard is now a tokio::sync::Mutex held across the whole open, so the check and the insert cannot be split by the serve_at await. Six threads raised this; the struct's own doc comment already described the corruption the code then allowed.
• 31 undeclared methods — all of People, Chunks, Retrieval and Profile (plus RecallDocuments, which predates this PR) were served but never declared in the manifest, so no host could call them. Both lists are fixed, and a new unit test compares the manifest against the generated Interface::members() so the hand-written list is out of the loop entirely — verified red-before-green.
• Bus-side source gate — retrieve_children / retrieve_leaves filtered on an ambient task-local that does not cross a transport, so they were restricted in-process and unrestricted over the bus. Both take an explicit scope now, with scoped engine variants and four regression tests.
• Store cap, single-object size checks, join-failure logging, the too_many_arguments suppression (replaced with a real params struct), and the embedding-signature collision (closed without a migration — encoding only %;= leaves every real signature byte-identical, with a test pinning that).

Four findings are declined with evidence and left unresolved for you to push back on: the config-propagation claim (the root store passes the same None/""), limit: 0 (the engine's default sentinel, not LIMIT 0), the intra-doc link (its target exists one function above), and the crate-shadowing claim (which would not compile). Also declined: source-scoping the id-addressed chunk reads, since the engine has no scoped variant to honour it.

[coderabbitai]

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '%s\n' '=== Changed files ==='
git diff --name-only HEAD~1..HEAD 2>/dev/null || git diff --name-only origin/main...HEAD

printf '%s\n' '=== Service outline ==='
ast-grep outline crates/tinymemory-module/src/service/mod.rs --items all --view expanded

printf '%s\n' '=== Provider outline ==='
ast-grep outline crates/tinymemory-module/src/provider.rs --items all --view expanded

printf '%s\n' '=== Retrieval outlines ==='
ast-grep outline core/src/tree/retrieval --items all --type function,struct --view expanded

printf '%s\n' '=== Relevant symbol locations ==='
rg -n -C2 'struct StoreOpener|async fn open_store|fn open_store|serve_at|Interface::members|members\(|retrieve_children|retrieve_leaves|fetch_leaves_scoped|drill_down_scoped|format_embedding_signature|too_many_arguments|limit.*0|ChunkDetail|chunk_embeddings|get_chunk' \
  crates/tinymemory-module/src/service crates/tinymemory-module/src/provider.rs crates/tinymemory-module/src/lib.rs \
  crates/tinymemory-module/tests core/src/tree/retrieval api/src/host/embeddings.rs api/src/provider/chunks.rs

Length of output: 53090

---

🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '%s\n' '=== Store opener and OpenStore ==='
sed -n '168,405p' crates/tinymemory-module/src/service/mod.rs

printf '%s\n' '=== Manifest declaration and consistency test ==='
sed -n '185,290p' crates/tinymemory-module/src/lib.rs
sed -n '220,310p' crates/tinymemory-module/src/service/test.rs

printf '%s\n' '=== Retrieval bus methods and response-size utility ==='
sed -n '940,1015p' crates/tinymemory-module/src/service/mod.rs
sed -n '1248,1378p' crates/tinymemory-module/src/service/mod.rs

printf '%s\n' '=== Scoped retrieval provider methods and chunk reads ==='
sed -n '1490,1615p' crates/tinymemory-module/src/provider.rs
sed -n '1615,1750p' crates/tinymemory-module/src/provider.rs

printf '%s\n' '=== Scoped core retrieval implementations ==='
sed -n '1,120p' core/src/tree/retrieval/drill_down.rs
sed -n '1,105p' core/src/tree/retrieval/fetch.rs
sed -n '1,120p' core/src/tree/retrieval/cover.rs
sed -n '1,125p' core/src/tree/retrieval/source.rs

printf '%s\n' '=== Signature format implementation ==='
sed -n '1,135p' api/src/host/embeddings.rs

printf '%s\n' '=== Relevant regression-test locations ==='
rg -n -C2 'open_store|OpenStore|MAX_OPEN_STORES|retrieve_children|retrieve_leaves|response_fits|response.*size|join.*fail|JoinError|panic' \
  crates/tinymemory-module/src/service/test.rs crates/tinymemory-module/tests core/src/tree/retrieval

Length of output: 57452

---

🏁 Script executed:

#!/bin/bash
set -euo pipefail

base="$(git merge-base HEAD origin/main 2>/dev/null || git merge-base HEAD main)"
printf '%s\n' "=== Merge base: $base ==="

printf '%s\n' '=== Diff checks ==='
git diff --check "$base"...HEAD
git diff --stat "$base"...HEAD

printf '%s\n' '=== All too_many_arguments allowances in the PR diff ==='
git diff -U4 "$base"...HEAD -- \
  core/src/tree/retrieval/cover.rs core/src/tree/retrieval/source.rs \
  crates/tinymemory-module/src/service/mod.rs \
  | rg -n -C4 'too_many_arguments|SourceQuery|cover_window_scoped|create_segment|upsert_provider_facet' || true

printf '%s\n' '=== Root setup and factory calls ==='
rg -n -C5 'create_memory_client(_in_subdir)?\s*\(' \
  crates/tinymemory-module/src core/src/store/factories.rs
sed -n '1,125p' core/src/store/factories.rs

printf '%s\n' '=== Blocking task join failure handling ==='
sed -n '235,265p' crates/tinymemory-module/src/provider.rs
rg -n -C4 'JoinError|spawn_blocking|join.*error|blocking\(' crates/tinymemory-module/src/provider.rs

printf '%s\n' '=== Exact chunk embedding return and contract signatures ==='
sed -n '1585,1625p' crates/tinymemory-module/src/provider.rs
sed -n '125,210p' api/src/provider/chunks.rs

printf '%s\n' '=== Scope-related diff and tests ==='
git diff -U3 "$base"...HEAD -- \
  api/src/provider/retrieval.rs crates/tinymemory-module/src/provider.rs \
  crates/tinymemory-module/src/service/mod.rs core/src/tree/retrieval \
  | rg -n -C3 'retrieve_children|retrieve_leaves|scope|scoped' || true
rg -n -C3 'retrieve_children|retrieve_leaves|drill_down_scoped|fetch_leaves_scoped' \
  crates/tinymemory-module/src/service/test.rs core/src/tree/retrieval/source_scope_tests.rs \
  crates/tinymemory-module/tests || true

Length of output: 65972

---

🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '%s\n' '=== Module setup and root client construction ==='
ast-grep outline crates/tinymemory-module/src/lib.rs --items all --type function --view expanded
rg -n -C8 'async fn setup|fn setup|create_memory_client|MemoryClient::|ModuleMemoryProvider::new|service::serve' \
  crates/tinymemory-module/src/lib.rs crates/tinymemory-module/src/service/mod.rs
sed -n '1,190p' crates/tinymemory-module/src/lib.rs

printf '%s\n' '=== Subdirectory factory implementation ==='
sed -n '580,700p' core/src/store/factories.rs
rg -n -C5 'pub fn create_memory_client\(' core/src/store/factories.rs

printf '%s\n' '=== Join-error mapping and logging ==='
sed -n '1950,1980p' crates/tinymemory-module/src/provider.rs
rg -n -C3 'fn other\(|MemoryError::Other|fn into_bus_error|log::.*error' \
  crates/tinymemory-module/src/provider.rs crates/tinymemory-module/src/service/mod.rs api/src/error.rs

printf '%s\n' '=== Scope predicate and available chunk provenance ==='
rg -n -C5 'fn chunk_source_allowed_in|chunk_source_allowed_in|pub struct Chunk|struct ChunkMetadata|source_id|tags' \
  core/src/source_scope.rs core/src/store/chunks api/src/chunks.rs
sed -n '1,180p' core/src/source_scope.rs

printf '%s\n' '=== Cover-window request types ==='
sed -n '95,180p' api/src/provider/retrieval.rs
rg -n -C4 'CoverWindowQuery|cover_window_scoped\s*\(' \
  api/src/provider/retrieval.rs core/src/tree/retrieval crates/tinymemory-module/src

Length of output: 23485

---

🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '%s\n' '=== Locate scope and chunk model files ==='
fd -a -t f 'source_scope|chunks|error' core api | sort

printf '%s\n' '=== Join-error conversion and predicate handling ==='
sed -n '1950,1982p' crates/tinymemory-module/src/provider.rs
sed -n '195,222p' crates/tinymemory-module/src/provider.rs
sed -n '1380,1410p' crates/tinymemory-module/src/service/mod.rs
rg -n -C3 'join .*failed|join .*error|join failure|JoinError|workflow_identity_matches' \
  crates/tinymemory-module/src/provider.rs api/src

printf '%s\n' '=== Scope predicate definitions and chunk fields ==='
rg -n -C6 'chunk_source_allowed_in|current_source_scope|pub struct Chunk|struct ChunkMetadata|source_id:|tags:' \
  core api \
  -g '*.rs'

printf '%s\n' '=== Cover-window request data and all uses ==='
rg -n -C8 'struct CoverWindowQuery|CoverWindowQuery\s*\{' api/src/provider/retrieval.rs \
  crates/tinymemory-module/src/provider.rs crates/tinymemory-module/src/service/mod.rs

Length of output: 170298

---

@senamakel Review complete.

I verified these changes:

• OpenStore holds the tokio::sync::Mutex guard across the store creation, serve_at, and cache insertion.
• The manifest test compares generated Interface::members() with the declared manifest methods in both directions.
• RetrieveChildren and RetrieveLeaves now receive an explicit SourceScope through the bus, provider, and core retrieval layers.
• Root and opened stores pass equivalent factory configuration. The configuration-propagation finding does not apply.
• limit: 0 is converted to the documented default limit. It is not interpreted as a zero-row query.
• workflow_identity_matches logs task join failures before it returns false.
• The embedding signature encoding preserves existing identifier forms and prevents delimiter collisions.
• The response-size checks now cover the relevant single-object and collection responses.

I do not withdraw the id-addressed chunk-read finding. GetChunk, ChunkDetail, and ChunkEmbeddings accept no SourceScope and perform unscoped lookups. The returned Chunk data has metadata.source_id and metadata.tags, and chunk_source_allowed_in already provides the required authorization predicate. A missing scoped backend helper does not make a chunk ID an authorization mechanism.

The remaining clippy::too_many_arguments allowance on cover_window_scoped is separate from the new SourceQuery structure. It is not a correctness finding by itself.

I used the existing PR learning about scoped chunk reads during this review. Repository tests were not executed because the available sandbox does not run repository code.

---

✏️ Learnings added

Learnt from: senamakel
URL: https://github.com/tinyhumansai/tinymemory/pull/16

Timestamp: 2026-08-17T04:42:27.331Z
Learning: In `tinyhumansai/tinymemory` PR `#16`, the root module setup and `MemoryService::open_store` call equivalent memory-client factories with the same `MemoryConfig`, `None` local model, empty embedding API key, embedding routes, storage provider, and workspace directory. The opened store differs only by the explicit memory subdirectory.

---

Learnt from: senamakel
URL: https://github.com/tinyhumansai/tinymemory/pull/16

Timestamp: 2026-08-17T04:42:27.331Z
Learning: In `tinyhumansai/tinymemory` PR `#16`, `core/src/tree/retrieval/cover.rs` and `core/src/tree/retrieval/source.rs` define `limit == 0` as the no-preference sentinel and substitute `DEFAULT_LIMIT`. A review must not treat this value as a request for zero result rows.

Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.

---

🧠 Learnings used

Learnt from: senamakel
Repo: tinyhumansai/tinymemory PR: 16
File: api/src/provider/chunks.rs:138-199
Timestamp: 2026-08-16T21:27:51.605Z
Learning: In the TinyMemory Rust provider contract, source-restricted reads require `SourceScope` to be enforced inside the backend query. `MemoryChunks::get_chunk`, `MemoryChunks::chunk_detail`, and `MemoryChunks::chunk_embeddings` currently have no scoped backend lookup, so a correct future implementation must resolve each chunk's source and reject out-of-scope results rather than treating a chunk ID as authorization.

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.