feat(plugin): add NeMo Relay run-stream dynamic plugin

[bbednarski9]

What

Adds switchyard-nemo-relay-plugin, an external NeMo Relay native dynamic plugin that drives Switchyard algorithms through libsy::Algorithm::run_stream.

The plugin:

• exports only Relay native API v2;
• uses the safe Rust continuation facade from nemo-relay-plugin, while the C ABI remains the binary boundary;
• decodes and encodes OpenAI Chat, OpenAI Responses, and Anthropic Messages exclusively through switchyard-translation;
• supports weighted, reproducible random routing and the LLM-classifier router;
• resolves every libsy CallLlm semantic target and invokes Relay with LlmContinuationV2::call() or LlmStreamContinuationV2::open_stream();
• feeds actual buffered responses, preserved provider events, and structured HTTP/non-HTTP failures back through libsy until ReturnToAgent;
• returns the SDK's explicit Passthrough(request) outcome for unmanaged streams, so provider events remain inside Relay's bounded host path; and
• preserves pre-commit retries, exactly-once trusted fallback, cancellation, post-commit late failures, credentials, and genuine routing marks.

The migration removes the plugin's 479-line generic FFI layer: no raw host table, opaque pointer, manual host-string allocation, callback-to-future bridge, stream pump, completion settlement, or release function remains in Switchyard.

Switchyard owns routing, translation, target binding, credentials, retry, and fallback policy. Relay core owns targeted HTTP(S)/JSON/SSE transport, downstream middleware execution, scopes, bounded stream forwarding, cancellation, and observability export.

The crate is a source/build unit (cdylib + rlib, publish = false). The operator-facing artifact remains a release bundle containing the shared library, manifest, schema, license notices, and digest.

Why

A selection-only decision API cannot express classifiers or future policies that transform requests, make one or more LLM calls, inspect results, and select the final response. run_stream provides that lifecycle while leaving provider dispatch under Relay's control.

Relay's raw C ABI can represent the lifecycle, but requiring each Rust policy plugin to reproduce unsafe callbacks and ownership logic is error-prone and obscures the actual Switchyard integration. NVIDIA/NeMo-Relay#594 now provides safe Rust continuations over that ABI and a host-owned streaming pass-through path. This PR consumes that general SDK surface rather than maintaining Switchyard-specific FFI glue.

This PR is stacked on and depends on #192, which preserves raw same-protocol provider events through the required Relay -> libsy -> Relay round trip. It also depends on NVIDIA/NeMo-Relay#594 and pins its SDK commit to 54452966e9c12065cf77ac3426ee81198a6658b0 until a compatible Relay SDK is published.

Related: NVIDIA/NeMo-Relay#594

How tested

• cargo test -p switchyard-nemo-relay-plugin — 9 passed.
• cargo clippy -p switchyard-nemo-relay-plugin --all-targets -- -D warnings.
• cargo fmt --all -- --check.
• uv run ruff check crates/switchyard-nemo-relay-plugin/tests/e2e.
• Real-process E2E using the final pushed Relay SDK and plugin library:
  • random router: 27 genuine decisions, 12 concurrent calls, all three wire protocols, buffered and streaming;
  • LLM classifier: buffered and streaming routes selected the expected semantic target;
  • same-protocol unknown buffered fields and raw stream events replayed exactly;
  • Switchyard's retry policy treated an HTTP 503 stream-open failure as retryable and made a fresh selection;
  • the same policy treated HTTP 400 as non-retryable and invoked trusted fallback exactly once;
  • a late transport failure after one emitted event did not retry or fall back;
  • unmanaged buffered and streaming calls passed through with zero Switchyard marks;
  • target credentials reached the provider, replaced source credentials, and were absent from logs and routing artifacts.
• Relay's safe v2 embedded-core fixture proves the same continuation surface does not depend on CLI gateway state.

The repository-wide Rust workspace run still reaches the existing macOS switchyard-py unresolved-Python-symbol link failure. Focused touched-crate validation and the cross-repository process path pass.

Checklist

• One class per file; filename = snake_case of the primary class (not applicable to the Rust crate; Python helpers follow snake_case).
• New public symbols exported from switchyard/__init__.py.__all__ if intended for downstream use (no Python public symbols added).
• Unit tests added for new components / bug fixes.
• README / --help updated if customer-facing surface changed.
• Commits signed off (Signed-off-by: Your Name <email>) per the DCO.

Notes for reviewers

flowchart LR
    A["Caller request"] --> B["Relay LLM pipeline"]
    B --> C["Safe Switchyard v2 callback"]
    C --> D["switchyard-translation decode"]
    D --> E["libsy run_stream"]
    E --> F["CallLlm"]
    F --> G["Safe continuation call / open_stream"]
    G --> H["Raw C ABI boundary"]
    H --> I["Remaining Relay LLM middleware"]
    I --> J["Relay core HTTP JSON / SSE transport"]
    J --> K["Selected provider"]
    K --> L["Response, provider stream, or typed failure"]
    L --> G
    G --> M["CallLlmRequest.respond"]
    M --> E
    E --> N["ReturnToAgent"]
    N --> O["switchyard-translation encode"]
    O --> A

    C -->|"unmanaged stream"| P["Passthrough(request)"]
    P --> I

Loading

Please focus review on:

• src/lib.rs: safe registration and complete removal of raw callback state;
• src/runtime.rs: typed continuations, boxed output stream, pre-commit retry/fallback, and post-commit failure behavior;
• the E2E reliability cases for typed stream-open failures and late errors; and
• unmanaged pass-through, where provider events do not cross the plugin ABI and no Switchyard routing marks are emitted.

Intentional constraints:

• v2-only: no Relay native API v1 compatibility mode;
• no Relay codecs, private dispatch headers, full Relay runtime dependency, Switchyard LLM client, or Switchyard service;
• no decision-only/observe-only API or synthesized decision telemetry;
• no retry or fallback after the first caller event;
• Relay supplies protocol-neutral HTTP/non-HTTP failure facts, while protocol interpretation and retry/fallback policy remain owned by Switchyard; and
• release automation remains deferred until the Git SDK pin can move to a published compatible version.