Skip to content

docs: clarify middleware callback error semantics - #613

Merged
rapids-bot[bot] merged 2 commits into
NVIDIA:release/0.7from
yczhang-nv:docs/middleware-exception-contract
Jul 31, 2026
Merged

docs: clarify middleware callback error semantics#613
rapids-bot[bot] merged 2 commits into
NVIDIA:release/0.7from
yczhang-nv:docs/middleware-exception-contract

Conversation

@yczhang-nv

@yczhang-nv yczhang-nv commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

Overview

Document the fail-closed exception contract for conditional-execution guardrails and request or execution intercepts in NeMo Relay 0.7.

  • I confirm this contribution is my own work, or I have the right to submit it under this project's license.
  • I searched existing issues and open pull requests, and this does not duplicate existing work.

Details

  • Explain that callback errors, exceptions, and rejected Promise values stop the middleware chain at the failing stage and surface to the managed caller.
  • Clarify that conditional-guardrail and request-intercept failures prevent the real callback from running.
  • Clarify that an execution intercept cannot undo work from a next continuation that it already invoked.
  • Document parity between callbacks that complete directly and callbacks that complete asynchronously.
  • Update both the middleware concepts page and the v0.6-to-v0.7 migration guide.
  • Leave sanitizer exception semantics to fix: fail closed on sanitizer callback errors #607 to avoid conflicting guidance.

Validation:

  • just docs passed with 0 errors. Fern skipped the authenticated redirects check because FERN_TOKEN was unavailable.
  • Targeted pre-commit checks passed for both changed files, including the documentation link checker.
  • The all-files pre-commit run passed formatting, Ruff, type checks, documentation links, lockfile checks, Cargo formatting/clippy/check, attribution checks, and Node checks. The cargo-deny, go fmt, and go vet hooks could not run because cargo-deny and Go are not installed in the local environment.

Where should the reviewer start?

Start with the Callback Error Handling section in docs/about-nemo-relay/concepts/middleware.mdx, then compare the matching upgrade guidance in docs/reference/migration-guides.mdx.

Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)

  • Relates to: none

Summary by CodeRabbit

  • Documentation
    • Documented fail-closed behavior when middleware callbacks encounter errors, throw exceptions, or reject asynchronously.
    • Clarified that failed guards and request intercepts stop processing before the main callback runs.
    • Explained execution-intercept continuation behavior, including work already started by next.
    • Added corresponding guidance to the migration documentation.

Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
@coderabbitai

coderabbitai Bot commented Jul 31, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Enterprise

Run ID: ac8e50b0-7d65-47fe-b788-9b78be88a7e8

📥 Commits

Reviewing files that changed from the base of the PR and between d049151 and 57780bf.

📒 Files selected for processing (2)
  • docs/about-nemo-relay/concepts/middleware.mdx
  • docs/reference/migration-guides.mdx
📜 Recent review details
🧰 Additional context used
📓 Path-based instructions (8)
**/*.mdx

📄 CodeRabbit inference engine (.agents/skills/review-doc-style/SKILL.md)

MDX top-of-file SPDX comments must use {/* ... */} delimiters instead of HTML comment delimiters (Must-Fix)

In MDX files, top-of-file comments must use JSX comment delimiters ({/* to open and */} to close); do not use HTML comments for MDX SPDX headers

Files:

  • docs/reference/migration-guides.mdx
  • docs/about-nemo-relay/concepts/middleware.mdx
**/*.{md,mdx}

📄 CodeRabbit inference engine (AGENTS.md)

Update README.md, fern/, package READMEs, and binding-support notes when public behavior, package names, examples, or supported bindings change.

**/*.{md,mdx}: Prefer the documented public API, not internal shortcuts
Keep package names, repo references, and build commands current
Keep release-process and release-notes guidance in repo-maintainer docs such as RELEASING.md, not as user-facing docs pages or CHANGELOG.md
Keep stable user-facing wrappers at scripts/ root in docs and examples; only point at namespaced helper paths when documenting internal maintenance work
When detailed dynamic plugin guides exist, keep Rust native plugin examples, Python worker plugin examples, and grpc-v1 protocol details on separate pages

If links in documentation change, run just docs-linkcheck.

Files:

  • docs/reference/migration-guides.mdx
  • docs/about-nemo-relay/concepts/middleware.mdx
**/*.{md,markdown,mdx}

📄 CodeRabbit inference engine (CONTRIBUTING.md)

Add the SPDX license header to all Markdown/MDX documentation files using the HTML comment block form.

Files:

  • docs/reference/migration-guides.mdx
  • docs/about-nemo-relay/concepts/middleware.mdx
{docs,examples}/**/*

📄 CodeRabbit inference engine (.agents/skills/rename-surfaces/SKILL.md)

Update docs and examples.

Files:

  • docs/reference/migration-guides.mdx
  • docs/about-nemo-relay/concepts/middleware.mdx
**/*

📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)

**/*: Format changed files with the language-native formatter before the final lint/test pass.
If dynamic plugin behavior changed, use maintain-dynamic-plugins and include the native SDK, worker protocol, Python SDK, docs, packaging, and Codecov surfaces in the validation plan.
If code changes alter APIs, bindings, commands, paths, packaging behavior, observability/adaptive semantics, or documented best practices, update any dependent maintainer or consumer skills in the same branch.
During iteration, prefer uv run pre-commit run --files <changed files...>.
Before review or handoff, run uv run pre-commit run --all-files.

Files:

  • docs/reference/migration-guides.mdx
  • docs/about-nemo-relay/concepts/middleware.mdx
docs/**/*

📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)

If documentation examples or commands under docs/ change, run the targeted docs checks appropriate to the change.

Files:

  • docs/reference/migration-guides.mdx
  • docs/about-nemo-relay/concepts/middleware.mdx
**/*.{md,mdx,py,go,js,ts}

📄 CodeRabbit inference engine (.agents/skills/maintain-observability/SKILL.md)

Examples and documentation must use each exporter's documented flush/deregister order before shutdown.

Files:

  • docs/reference/migration-guides.mdx
  • docs/about-nemo-relay/concepts/middleware.mdx
{docs/**,README.md,CONTRIBUTING.md,RELEASING.md,SECURITY.md}

⚙️ CodeRabbit configuration file

{docs/**,README.md,CONTRIBUTING.md,RELEASING.md,SECURITY.md}: Review documentation for technical accuracy against the current API, command correctness, and consistency across language bindings.
Flag stale examples, missing SPDX headers where required, and instructions that no longer match CI or pre-commit behavior.

Files:

  • docs/reference/migration-guides.mdx
  • docs/about-nemo-relay/concepts/middleware.mdx
🔇 Additional comments (2)
docs/about-nemo-relay/concepts/middleware.mdx (1)

110-119: LGTM!

docs/reference/migration-guides.mdx (1)

54-56: LGTM!


Walkthrough

The documentation defines fail-closed behavior for conditional guardrails and request or execution intercepts. It covers synchronous and asynchronous callback failures, managed error propagation, callback prevention, and the limits of reversing work after next runs.

Changes

Middleware error handling

Layer / File(s) Summary
Document fail-closed behavior and migration guidance
docs/about-nemo-relay/concepts/middleware.mdx, docs/reference/migration-guides.mdx
The documentation states that callback failures stop processing, prevent the underlying callback when applicable, reach managed callers, and behave consistently for synchronous and asynchronous callbacks. It also documents that execution intercepts cannot undo work already started through next.

Estimated code review effort: 1 (Trivial) | ~5 minutes

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title follows Conventional Commits format, uses an allowed lowercase type, states the documentation change, and is under 72 characters.
Description check ✅ Passed The description includes all template sections, explains the documentation changes, identifies reviewer starting points, and records validation results.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added size:S PR is small Documentation documentation-related labels Jul 31, 2026
@yczhang-nv
yczhang-nv marked this pull request as ready for review July 31, 2026 21:02
@yczhang-nv
yczhang-nv requested review from a team as code owners July 31, 2026 21:02

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/about-nemo-relay/concepts/middleware.mdx`:
- Around line 109-113: Update the middleware failure semantics in the
conditional-execution guardrails and request or execution intercepts section to
distinguish callback failures from guardrail rejection: describe callback errors
as returning a failure result, and separately state that a conditional-execution
guardrail may return its documented rejection message to block execution as
normal control flow.
- Around line 114-116: Update docs/reference/migration-guides.mdx at lines 54-55
to document the continuation lifetime rule: if an execution intercept fails
after invoking next, Relay cannot undo completed downstream work, rejects
unfinished calls, and rejects calls started after the interceptor settles. No
direct change is required in docs/about-nemo-relay/concepts/middleware.mdx lines
114-116 because it already documents this behavior.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Enterprise

Run ID: addcb14c-07a8-4cb3-b611-fd9b75abebc1

📥 Commits

Reviewing files that changed from the base of the PR and between b671256 and d049151.

📒 Files selected for processing (2)
  • docs/about-nemo-relay/concepts/middleware.mdx
  • docs/reference/migration-guides.mdx
📜 Review details
🧰 Additional context used
📓 Path-based instructions (8)
**/*.mdx

📄 CodeRabbit inference engine (.agents/skills/review-doc-style/SKILL.md)

MDX top-of-file SPDX comments must use {/* ... */} delimiters instead of HTML comment delimiters (Must-Fix)

In MDX files, top-of-file comments must use JSX comment delimiters ({/* to open and */} to close); do not use HTML comments for MDX SPDX headers

Files:

  • docs/about-nemo-relay/concepts/middleware.mdx
  • docs/reference/migration-guides.mdx
**/*.{md,mdx}

📄 CodeRabbit inference engine (AGENTS.md)

Update README.md, fern/, package READMEs, and binding-support notes when public behavior, package names, examples, or supported bindings change.

**/*.{md,mdx}: Prefer the documented public API, not internal shortcuts
Keep package names, repo references, and build commands current
Keep release-process and release-notes guidance in repo-maintainer docs such as RELEASING.md, not as user-facing docs pages or CHANGELOG.md
Keep stable user-facing wrappers at scripts/ root in docs and examples; only point at namespaced helper paths when documenting internal maintenance work
When detailed dynamic plugin guides exist, keep Rust native plugin examples, Python worker plugin examples, and grpc-v1 protocol details on separate pages

If links in documentation change, run just docs-linkcheck.

Files:

  • docs/about-nemo-relay/concepts/middleware.mdx
  • docs/reference/migration-guides.mdx
**/*.{md,markdown,mdx}

📄 CodeRabbit inference engine (CONTRIBUTING.md)

Add the SPDX license header to all Markdown/MDX documentation files using the HTML comment block form.

Files:

  • docs/about-nemo-relay/concepts/middleware.mdx
  • docs/reference/migration-guides.mdx
{docs,examples}/**/*

📄 CodeRabbit inference engine (.agents/skills/rename-surfaces/SKILL.md)

Update docs and examples.

Files:

  • docs/about-nemo-relay/concepts/middleware.mdx
  • docs/reference/migration-guides.mdx
**/*

📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)

**/*: Format changed files with the language-native formatter before the final lint/test pass.
If dynamic plugin behavior changed, use maintain-dynamic-plugins and include the native SDK, worker protocol, Python SDK, docs, packaging, and Codecov surfaces in the validation plan.
If code changes alter APIs, bindings, commands, paths, packaging behavior, observability/adaptive semantics, or documented best practices, update any dependent maintainer or consumer skills in the same branch.
During iteration, prefer uv run pre-commit run --files <changed files...>.
Before review or handoff, run uv run pre-commit run --all-files.

Files:

  • docs/about-nemo-relay/concepts/middleware.mdx
  • docs/reference/migration-guides.mdx
docs/**/*

📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)

If documentation examples or commands under docs/ change, run the targeted docs checks appropriate to the change.

Files:

  • docs/about-nemo-relay/concepts/middleware.mdx
  • docs/reference/migration-guides.mdx
**/*.{md,mdx,py,go,js,ts}

📄 CodeRabbit inference engine (.agents/skills/maintain-observability/SKILL.md)

Examples and documentation must use each exporter's documented flush/deregister order before shutdown.

Files:

  • docs/about-nemo-relay/concepts/middleware.mdx
  • docs/reference/migration-guides.mdx
{docs/**,README.md,CONTRIBUTING.md,RELEASING.md,SECURITY.md}

⚙️ CodeRabbit configuration file

{docs/**,README.md,CONTRIBUTING.md,RELEASING.md,SECURITY.md}: Review documentation for technical accuracy against the current API, command correctness, and consistency across language bindings.
Flag stale examples, missing SPDX headers where required, and instructions that no longer match CI or pre-commit behavior.

Files:

  • docs/about-nemo-relay/concepts/middleware.mdx
  • docs/reference/migration-guides.mdx

Comment thread docs/about-nemo-relay/concepts/middleware.mdx Outdated
Comment thread docs/about-nemo-relay/concepts/middleware.mdx Outdated
@github-actions

Copy link
Copy Markdown

@willkill07 willkill07 added this to the 0.7 milestone Jul 31, 2026
Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
@willkill07

Copy link
Copy Markdown
Member

/merge

@rapids-bot
rapids-bot Bot merged commit d98ee9a into NVIDIA:release/0.7 Jul 31, 2026
27 checks passed
@yczhang-nv
yczhang-nv deleted the docs/middleware-exception-contract branch July 31, 2026 21:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Documentation documentation-related size:S PR is small

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants