Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,12 @@ reference.
restamps `local` output with the processor's own identity and the subscribe fan-out drops any
inbound message whose identity matches that — without both halves, a `local` route would loop.
- A full route queue drops and counts; it never blocks the transport's dispatch task.
- Command verbs declare their addressing scope at registration (core 0.5.0,
`DESIGN-scoped-commands.md` D-SC-2). All four processor verbs — `get-stats`, `flush`, `pause`,
`resume` — declare `CommandScope::Both` (DESIGN.md D-TP-9): a route **is** a
`component.instances[]` entry, so a route-addressed delivery acts on that route alone while a
component-addressed one keeps its established "every route" meaning. The topic's instance token
wins over the legacy `route` body selector.
- Four-way parity: if this repo's Java/Python/TypeScript siblings exist, observable behavior should
match — same config shape, same metric names, same command verbs.
- Builders/facades are the construction path (`messaging()`, `streams()`, `events()`, `commands()`,
Expand Down
8 changes: 4 additions & 4 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,10 @@ inherits — lives in `AGENTS.md` and is shared with every agent tool. It is imp

## Local-dev notes

- **Default (committed pin):** `Cargo.toml`'s `edgecommons` dependency is a git `rev` pin, and
`Cargo.lock` is committed against that pin — a plain `cargo build`/`clone` needs only read access
to `edgecommons/edgecommons` (private; the fetch goes through the git CLI, and CI rewrites the URL
with the `EDGECOMMONS_READ_TOKEN` PAT).
- **Default (committed pin):** `Cargo.toml`'s `edgecommons` dependency is a git `rev` pin — currently
the `rust-lib/v0.5.0` release tag — and `Cargo.lock` is committed against that pin.
`edgecommons/edgecommons` is public, so a plain `cargo build`/`clone` fetches it anonymously, in CI
and locally.
- **Building against an unpushed sibling change:** add a gitignored `.cargo/config.toml` next to this
file with a `[patch]` block pointing the git URL at your local `../core/libs/rust` checkout (see the
comment above the `edgecommons` dependency in `Cargo.toml` for the exact form). This is local-dev
Expand Down
6 changes: 3 additions & 3 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

13 changes: 6 additions & 7 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -42,13 +42,12 @@ streaming-file-avro = ["streaming", "edgecommons/streaming-file-avro"]
scripting-lua = ["dep:mlua"]

[dependencies]
# The edgecommons Rust library, pinned to a commit on edgecommons/edgecommons (private). Building this
# repo standalone needs read access to that repo. The fetch goes through the git CLI (see
# .cargo/config.toml); locally it uses your gh/git credential helper, and in CI the reusable workflow
# rewrites this https URL with the EDGECOMMONS_READ_TOKEN PAT. For local development against an
# unpushed change, add a gitignored `.cargo/config.toml` `[patch]` override pointing at
# `../core/libs/rust` (the sibling checkout) instead of editing this pin — see CLAUDE.md.
edgecommons = { git = "https://github.com/edgecommons/edgecommons.git", rev = "36a70c48b65b35f77bfab70d3a73869debdfc407", default-features = false }
# The edgecommons Rust library, pinned to the `rust-lib/v0.5.0` release tag on
# edgecommons/edgecommons (a public repo — the fetch needs no credentials, in CI or locally). For
# local development against an unpushed change, add a gitignored `.cargo/config.toml` `[patch]`
# override pointing at `../core/libs/rust` (the sibling checkout) instead of editing this pin — see
# CLAUDE.md.
edgecommons = { git = "https://github.com/edgecommons/edgecommons.git", rev = "a14a3285573ef2bb6a531e1e1936c6dc40a85ef4", default-features = false }
tokio = { version = "1", features = ["rt-multi-thread", "macros", "signal", "time", "sync"] }
serde = { version = "1", features = ["derive"] }
serde_json = "1"
Expand Down
40 changes: 27 additions & 13 deletions DESIGN.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,10 +91,24 @@ Numbered `D-TP-<n>` so later sessions can cite them.
panels (`overview`: fleet totals + flush/pause/resume; `routes`: per-route counters via
`get-stats`) were straightforward to add via the library's existing `commands.register_panel` (used
identically by the protocol-adapter templates) and ride the command surface that already shipped —
so they were implemented rather than deferred. `scope: "component"`, not `"instance"`: the
processor has no console-facing UNS instance dimension (a route is internal wiring, addressed by an
optional `body.route` field on the command verbs, not a topic segment), unlike a southbound
adapter's per-device instances.
so they were implemented rather than deferred. `scope: "component"`, not `"instance"`: both panels
render fleet-wide aggregates over every route (the overview totals, the routes table). Individual
routes stay reachable through the verbs' own `Both` scope (D-TP-9), which the console derives from
`describe` rather than from a panel descriptor.
- **D-TP-9 (core 0.5.0 adoption). Every command verb declares `CommandScope::Both`.** Core 0.5.0's
scoped registration (`register(verb, scope, handler)`, core `DESIGN-scoped-commands.md` D-SC-2)
requires each verb to declare its addressing. A route **is** a `component.instances[]` entry, so
the library subscribes a per-route command inbox and an instance-addressed delivery names exactly
one route — while the established meaning of a component-addressed delivery is "every route". That
is precisely D-SC-3's dual-semantics use of `Both`, so all four verbs (`get-stats`, `flush`,
`pause`, `resume`) declare it: `None` = every route, a token = that route. `Component` would refuse
a legitimate per-route request, and `Instance` would make the fleet-wide form an error.
Resolution is `addressed_instance ?? body.route` — the topic token is authoritative (D-SC-4) and
the legacy `route` body selector is retained, unchanged, for component-addressed callers.
`get-stats` and `flush`, which never had a body selector, now honor the topic token rather than
ignoring it: silently fanning a mutating `flush` out to every route when one was addressed is
exactly the mis-targeting class D-SC-1 closes. Unknown/unconfigured route ids stay non-fatal
(they simply select nothing), matching the pre-existing `pause`/`resume` behavior.

## Config

Expand All @@ -109,15 +123,15 @@ field-by-field prose it was derived from; treat that page, not this one, as the

Beyond the library's automatic `ping` / `reload-config` / `get-configuration`:

| Verb | Body | Result |
|------|------|--------|
| `get-stats` | — | Per-route counters (`in`/`out`/`dropped`/`streamAppends`/`publishFailures`/`queueDepth`/`paused`). |
| `flush` | — | Force-closes every route's open **time** windows now; `{flushed: n}`. Count windows are unaffected. |
| `pause` | `{route?}` | Stops enqueuing to a route (or all routes when omitted); `{paused: [ids]}`. |
| `resume` | `{route?}` | The inverse of `pause`; `{resumed: [ids]}`. |
| Verb | Scope | Body | Result |
|------|-------|------|--------|
| `get-stats` | `Both` | — | Counters for the addressed route, or every route (`in`/`out`/`dropped`/`streamAppends`/`publishFailures`/`queueDepth`/`paused`). |
| `flush` | `Both` | — | Force-closes the addressed route's (or every route's) open **time** windows now; `{flushed: n}`. Count windows are unaffected. |
| `pause` | `Both` | `{route?}` | Stops enqueuing to the addressed route (or all routes when neither the topic nor the body names one); `{paused: [ids]}`. |
| `resume` | `Both` | `{route?}` | The inverse of `pause`; `{resumed: [ids]}`. |

Two edge-console panels (`overview`, `routes`; see D-TP-8) bind to these verbs. Full wire contract in
`docs/reference/messaging-interface.md`.
Every verb declares `CommandScope::Both` (D-TP-9). Two edge-console panels (`overview`, `routes`;
see D-TP-8) bind to these verbs. Full wire contract in `docs/reference/messaging-interface.md`.

## Metrics

Expand All @@ -127,7 +141,7 @@ summed across routes and emitted as interval deltas every 30s via `gg.metrics()`

## Validation

- `cargo test` (107 tests as of this remediation) — pipeline mechanics, route config parsing,
- `cargo test` (114 tests with the coverage job's feature set) — pipeline mechanics, route config parsing,
route-build decisions (target/filter/publish/script-output-topic resolution, the restamp policy —
`src/route_build.rs`), the fan-out handler + command/panel registration, the route dispatcher
(local/northbound/stream targets, restamp, failure→evt), the metric/event surface. No broker
Expand Down
15 changes: 10 additions & 5 deletions docs/how-to-guides.md
Original file line number Diff line number Diff line change
Expand Up @@ -477,17 +477,22 @@ filter to specific adapters instead when you don't need the whole fleet, e.g.

**Goal:** inspect and control a running processor from the console / any MQTT client.

The processor answers its command inbox at `ecv1/{device}/telemetry-processor/cmd/<verb>`. Send a
`cmd` envelope (`header.name` = the verb) with `header.reply_to` set to get a structured reply.
The processor answers its command inbox at `ecv1/{device}/telemetry-processor/cmd/<verb>` and, per
route, at `ecv1/{device}/telemetry-processor/{route}/cmd/<verb>`. Send a `cmd` envelope
(`header.name` = the verb) with `header.reply_to` set to get a structured reply.

Every verb below declares the scope `both`: address the component to act on **every route**, or
address one route to act on that route alone. The topic's route token wins over a `route` field in
the body.

| Verb | What it does |
|------|--------------|
| `ping` | liveness — `{status:"RUNNING", uptimeSecs}` (library built-in) |
| `reload-config` | re-fetch + re-apply the config (library built-in) |
| `get-configuration` | the redacted effective config (library built-in) |
| `get-stats` | per-route counters `{routes:[{id,in,out,dropped,streamAppends,publishFailures,queueDepth,paused}]}` |
| `flush` | force-close every route's open **time** windows now → `{flushed:n}` |
| `pause` / `resume` | stop / restart enqueuing to a route (`{route}`) or all routes (body omitted) |
| `get-stats` | counters for the addressed route, or every route: `{routes:[{id,in,out,dropped,streamAppends,publishFailures,queueDepth,paused}]}` |
| `flush` | force-close the addressed route's (or every route's) open **time** windows now → `{flushed:n}` |
| `pause` / `resume` | stop / restart enqueuing to the addressed route (or the body's `{route}`), or all routes when neither names one |

The processor also publishes, without any request: its `state` keepalive
(`ecv1/{device}/telemetry-processor/state`), a `metric/pipeline` throughput metric (when
Expand Down
40 changes: 28 additions & 12 deletions docs/reference/messaging-interface.md
Original file line number Diff line number Diff line change
Expand Up @@ -226,27 +226,43 @@ the triggering message, which continues on the source topic.

## Command verbs

The processor subscribes its own command inbox `ecv1/{device}/telemetry-processor/cmd/#` (wired
The processor subscribes its own command inbox `ecv1/{device}/telemetry-processor/cmd/#` and, for each
configured route, the per-route inbox `ecv1/{device}/telemetry-processor/{route}/cmd/#` (both wired
automatically by the library). A `cmd` request whose `header.reply_to` is set gets a structured reply
`{"ok": true, "result": …}` or `{"ok": false, "error": {"code", "message"}}`; a request without
`reply_to` is fire-and-forget.

**Built-in verbs** (library-provided, cannot be shadowed):

| Verb | Result |
|------|--------|
| `ping` | `{ "status": "RUNNING", "uptimeSecs": n }` — liveness/echo |
| `reload-config` | re-fetch + re-apply the config from the active source → `{ "reloaded": true }` |
| `get-configuration` | the current **redacted effective config** → `{ "config": … }` |
| Verb | Scope | Result |
|------|-------|--------|
| `ping` | both | `{ "status": "RUNNING", "uptimeSecs": n }` — liveness/echo |
| `reload-config` | both | re-fetch + re-apply the config from the active source → `{ "reloaded": true }` |
| `get-configuration` | both | the current **redacted effective config** → `{ "config": … }` |

**Custom verbs** (registered by the processor):

| Verb | Body | Result |
|------|------|--------|
| `get-stats` | — | `{ "routes": [ { id, in, out, dropped, streamAppends, publishFailures, queueDepth, paused } ] }` — per-route counters |
| `flush` | — | force-close every route's open **time** windows now → `{ "flushed": n }` (messages emitted). Count windows keep their count semantics. |
| `pause` | `{ "route"? }` | stop enqueuing to a route (or all routes when omitted) → `{ "paused": [ids] }` |
| `resume` | `{ "route"? }` | the inverse of `pause` → `{ "resumed": [ids] }` |
| Verb | Scope | Body | Result |
|------|-------|------|--------|
| `get-stats` | both | — | `{ "routes": [ { id, in, out, dropped, streamAppends, publishFailures, queueDepth, paused } ] }` — counters for the addressed route, or every route |
| `flush` | both | — | force-close the addressed route's (or every route's) open **time** windows now → `{ "flushed": n }` (messages emitted). Count windows keep their count semantics. |
| `pause` | both | `{ "route"? }` | stop enqueuing to a route → `{ "paused": [ids] }` |
| `resume` | both | `{ "route"? }` | the inverse of `pause` → `{ "resumed": [ids] }` |

### Verb scope and addressing

A route is a `component.instances[]` entry, so every verb above is addressable two ways and each one
declares the scope `both`:

- **Component-addressed** (`ecv1/{device}/telemetry-processor/cmd/{verb}`, no route token) means
**every route** — the fleet-wide form.
- **Route-addressed** (`ecv1/{device}/telemetry-processor/{route}/cmd/{verb}`) acts on that route
alone.

The topic's route token is authoritative: when a request is route-addressed, `pause`/`resume` ignore
a `route` field in the body. The `route` body field remains the way to target a single route over the
component-addressed topic. A request that names one route in the topic and a different instance in
`body.instance` is rejected by the library with `BAD_ARGS` before the verb runs.

> **Known limitation.** The built-in `reload-config` hot-swaps the config snapshot but the routes are
> wired once at startup, so a route topology change needs a component restart; there is no dynamic
Expand Down
Loading
Loading