feat: local assistant runtime contract

[SebastianBoehler]

Issue Spec: Local Agent Runtime Contract

Summary

Define the interface between the StudyOS app and a local or on-device assistant runtime. The app should not be locked to one model provider, and assistant output should be structured, permission-aware, and testable.

Goal

Enable local LLM experimentation while keeping StudyOS privacy-first and architecturally clean.

Scope

• Define an AssistantRuntime interface.
• Input: user text, optional context summary, available capabilities, and permission state.
• Output: typed assistant response, proposed actions, and optional generative UI component payloads.
• Provide a mock runtime for deterministic tests and demos.
• Keep backend choices replaceable: local model, platform model, remote endpoint, or no model.
• Treat LLM generation as optional and asynchronous for proactive home feed use cases.

Out Of Scope

• Choosing a final model provider.
• Routing student credentials through hosted middleware.
• Giving the model unrestricted access to local app data.
• Building full conversational memory before the core contract is proven.
• Blocking app startup or home feed rendering on model inference.

Acceptance Criteria

• The app can call one assistant interface regardless of runtime implementation.
• The mock runtime can produce deterministic action/component outputs.
• Assistant output is typed and validated before use.
• Sensitive data passed into the runtime is explicit and minimized.
• Unsupported runtime states fail with clear errors or graceful fallback UI.
• The contract can support text input first and voice input later.
• Home feed summaries can fall back to deterministic templates when the runtime is unavailable or slow.

Gates To Pass

• Architecture gate: model/provider-specific code is behind the runtime interface.
• Latency gate: assistant runtime work is never required for first paint of the home screen.
• Privacy gate: data passed to the runtime is documented by category and purpose.
• Permission gate: assistant actions require app-side permission checks before execution.
• Reliability gate: invalid or unavailable runtime output cannot crash the app.
• Test gate: mock runtime tests cover at least success, invalid output, and unavailable runtime.
• Demo gate: the app can show a local/mock assistant response without network access.

Suggested Branch

feat/local-agent-contract

[hegu-1]

For a local assistant runtime contract, I would keep the contract provider-agnostic but authority-explicit.

The app probably needs to know:

• runtime identity: provider/model/process/session;
• workspace scope: which files/projects the assistant can see;
• memory scope: what durable context it can read/write;
• tool authority: allowed actions and confirmation requirements;
• event stream: trace, tool calls, errors, and checkpoints;
• action receipts: what changed and under which grant;
• export/replay: enough structure to resume or audit a session later.

This avoids coupling StudyOS to one model while also avoiding the opposite problem: a generic assistant bridge with unclear memory and tool boundaries.

For a study/productivity OS, memory is not just chat history. It becomes the continuity layer across sessions, goals, notes, and decisions.

I have been working on a related Personal AI OS framing around local-first continuity and explicit runtime/memory/governance boundaries: https://github.com/hegu-1/personal-ai-os

[SebastianBoehler]

Thanks @hegu-1 — authority-explicit is the useful addition here. The repo already has a provider abstraction (AgentLlmProvider / AgentLlmProviderRegistry) from #20 / PR #21, plus streaming/cancellation from #37, so implementing the current issue literally would duplicate shipped work.

I suggest narrowing #51 to a runtime session contract above the provider layer:

Proposed MVP boundary

• RuntimeDescriptor: runtime/provider/model identity and supported features. Avoid exposing process IDs unless a consumer actually needs them.
• AssistantRequest: session ID, user input, explicit context references, and a snapshot of granted capabilities.
• RuntimeGrant: capability ID, allowed operation, resource scope, confirmation policy, and expiry. Enforcement stays app-side.
• AssistantEvent: typed content delta, tool proposal/status, recoverable error, checkpoint, and completion events. This should normalize the stream already delivered by feat: streaming replies, stop/cancel, thinking traces, in-app voice, GPU backend selector #37 rather than create another transport.
• AssistantResult: final response, validated feat: bounded generative UI component registry #49 component payload references, and ActionReceipts for attempted/approved/executed/rejected actions.
• Deterministic fake runtime covering success, denial, malformed event, cancellation, and unavailable runtime.

Ownership / non-goals

• feat: StudyOS data and action capability contract #54 owns StudyOS data/action capability DTOs, privacy labels, and unavailable/permission-denied states; feat: local assistant runtime contract #51 consumes them and must not redefine getSchedule, createReminder, etc.
• feat: bounded generative UI component registry #49 owns generative UI schemas and rendering.
• Stabilize assistant tool execution, mail access, and chat performance #43 owns tool-loop behavior, runtime timeout/reliability, and memory freshness.
• Durable memory policy/storage, arbitrary filesystem workspace access, audit-log persistence, and a stable cross-version replay/export format should be follow-ups. The MVP can reserve versioned IDs/checkpoint hooks without promising replay compatibility yet.

Clarifications needed before implementation

1. Is feat: local assistant runtime contract #51 meant to extend the shipped Add modular in-app LLM and tool layer #20 provider layer, or replace it? I recommend extending it to avoid a second competing runtime abstraction.
2. Must v1 support resumable/exportable sessions across app restarts, or is in-process session identity plus serializable events sufficient?
3. Are workspace grants needed for this mobile app now? If yes, which concrete resource roots exist (app documents, selected files, project IDs)? A generic filesystem scope without a real consumer would be premature.
4. Should every write/action require confirmation, or should feat: StudyOS data and action capability contract #54 capability metadata define always_confirm, confirm_once, and pre-approved read-only grants?

Proposed acceptance criteria

• Existing local and cloud providers can be adapted without changing chat UI call sites or provider-specific protocols leaking into the contract.
• Every runtime request carries a session ID and explicit, least-privilege grants; absence of a grant fails closed.
• Runtime-emitted tool/action proposals are data only and cannot execute without app-side capability validation and confirmation policy checks.
• Events use a versioned, exhaustively typed envelope with stable ordering IDs; unknown event types and malformed payloads produce explicit recoverable errors.
• Each action attempt produces a receipt containing action/capability ID, grant ID, outcome, timestamp, and a redacted summary—never secrets or full sensitive payloads.
• Cancellation and unavailable runtimes terminate cleanly without crashing or blocking first paint.
• Tests cover local/cloud adapter compatibility, grant denial, confirmation-required flow, malformed/unknown events, cancellation, unavailable runtime, and receipt redaction.
• A local deterministic demo works offline with no university credentials.

Once the first two clarifications are settled, this looks ready for a small contract-and-adapters PR; replay/export and durable memory governance can then be scoped with real storage and privacy requirements instead of speculative interfaces.