Skip to content

feat: add SMPSerialRawTransport for Zephyr raw UART SMP#106

Merged
JPHutchins merged 1 commit into
mainfrom
feature/unencoded-serial-transport
Jun 23, 2026
Merged

feat: add SMPSerialRawTransport for Zephyr raw UART SMP#106
JPHutchins merged 1 commit into
mainfrom
feature/unencoded-serial-transport

Conversation

@JPHutchins

@JPHutchins JPHutchins commented May 21, 2026

Copy link
Copy Markdown
Collaborator

Summary

Adds SMPSerialRawTransport for Zephyr 4.4's CONFIG_MCUMGR_TRANSPORT_RAW_UART — the raw (unencoded) SMP-over-UART transport that sends each SMP message as [8-byte header][payload] bytes with no framing, base64, or CRC. Smaller code size and faster transfers than the historical "SMP over console" framing, at the cost of being unable to share the UART with shell or log output.

The existing serial.py file is split into a package so the two serial transports share connection management via a common base class:

serial/
  __init__.py    re-exports SMPSerialTransport, SMPSerialRawTransport,
                 and the Auto / BufferSize / BufferParams / FragmentationStrategy API
  common.py      _SerialTransportBase: pyserial holder, the open/retry connect loop
                 (overridable for non-serial-port byte pipes), disconnect,
                 SerialException -> SMPTransportDisconnected translation, TX drain, RX poll
  encoded.py     SMPSerialTransport: base64 + CRC + delimiter framing and its
                 Auto/BufferSize/BufferParams sizing, now on the shared base (behavior unchanged)
  unencoded.py   SMPSerialRawTransport (new): writes bytes verbatim, reads the SMP header
                 then exactly its payload, raises on overrun / oversized header length

from smpclient.transport.serial import SMPSerialTransport (and the strategy exports) continues to work, so no migration is required for existing users of the encoded transport.

Rebased onto main after the FragmentationStrategy (Auto/BufferSize/BufferParams) and camas/integration-suite work merged; encoded.py carries that sizing logic on the new shared base. connect is a normal (non-final) method so the integration suite's socket-backed QemuSocketSerialTransport can swap the byte pipe.

Integration tests

The manifest-driven SMP-server suite already described a native_sim serial_raw fixture and skipped it ("smpclient has no raw-UART transport yet"). This PR makes it live:

  • Vendors native_sim_serial_raw.exe from the pinned smp-server-fixtures release (0eae053d, sha256-verified against SHA256SUMS).
  • Drops the client_supported skip in servers.py and builds SMPSerialRawTransport for serial_raw fixtures in conftest._build_transport, so the raw transport runs the generic echo / enumeration / MCUmgr-params suite.
  • test_serial_raw.py adds a raw-specific check: after _initialize reads the server's params, the transport adopts buf_size as max_unencoded_size (the chunk size SMPClient.upload fills) — with no on-wire frame to subtract, unlike the encoded transport's buf_size - 4.

Note: there is deliberately no echo-at-max_unencoded_size round-trip. A full-buf_size echo returns MGMT_ERR_EMSGSIZE because the echo response mirrors the request and overflows the response netbuf — an echo-symmetry artifact, not a transport limit. The 384-byte request is received and parsed, so max_unencoded_size == buf_size is correct for the upload direction (large request, small response).

Related

Test plan

  • camas check (ruff + pydoclint + mypy + unit tests) — 363 passed, 14 skipped
  • camas test_integration207 passed, 91 skipped, incl. the native_sim.serial_raw fixture across echo / enumeration / MCUmgr-params and the raw-specific buf_size test
  • New raw transport unit tests (test_smp_serial_raw_transport.py) + an MTU-parametrized SMPClient.upload() chunking test over the raw transport
  • Existing SMPSerialTransport suite passes unchanged (behavior-preserving refactor onto the shared base)
  • Manual smoke test against a physical CONFIG_MCUMGR_TRANSPORT_RAW_UART=y target

🤖 Generated with Claude Code

Copilot AI review requested due to automatic review settings May 21, 2026 00:04

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new raw (unencoded) SMP-over-UART transport for Zephyr’s CONFIG_MCUMGR_TRANSPORT_RAW_UART, while refactoring the existing base64/framed serial transport into a package with shared connection management.

Changes:

  • Introduces SMPSerialRawTransport implementing Zephyr “raw UART” SMP message exchange ([8-byte header][payload]) with MTU-based send limiting.
  • Refactors existing SMPSerialTransport to share connect/disconnect/TX drain/read helpers via a _SerialTransportBase in serial/common.py.
  • Updates and expands the test suite, including a new end-to-end MTU-parametrized upload test for the raw transport.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
tests/test_smp_serial_transport.py Updates serial mocking target to the new serial.common.Serial location.
tests/test_smp_serial_raw_transport.py Adds unit tests for the new raw serial transport behavior.
tests/test_smp_client.py Adds an MTU-parametrized upload test exercising chunking with the raw transport.
tests/test_base64.py Updates internal base64 helper imports to the new serial.encoded module.
src/smpclient/transport/serial/init.py Re-exports SMPSerialTransport and SMPSerialRawTransport from the new package layout.
src/smpclient/transport/serial/common.py Adds shared serial connection management and SerialException translation utilities.
src/smpclient/transport/serial/encoded.py Moves encoded transport onto the shared serial base class; behavior largely preserved.
src/smpclient/transport/serial/unencoded.py Implements the new raw SMP serial transport.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/smpclient/transport/serial/unencoded.py
Comment thread src/smpclient/transport/serial/common.py Outdated
Comment thread src/smpclient/transport/serial/common.py Outdated
Comment thread src/smpclient/transport/serial/common.py Outdated
Comment thread tests/test_smp_client.py

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.

Comment thread src/smpclient/transport/serial/common.py
Comment thread src/smpclient/transport/serial/common.py
@JPHutchins

Copy link
Copy Markdown
Collaborator Author

This needs rebase since a lot has changed with serial transport capabilities and integration tests.

@JPHutchins

Copy link
Copy Markdown
Collaborator Author

After rebase, would be nice to add some integration tests for the MCUBoot raw transport: mcu-tools/mcuboot#2755

@JPHutchins JPHutchins force-pushed the feature/unencoded-serial-transport branch from d82c25f to 19f474f Compare June 23, 2026 00:39
Zephyr 4.4 introduced CONFIG_MCUMGR_TRANSPORT_RAW_UART -- an SMP-over-UART
transport that sends each SMP message as raw [8-byte header][payload] bytes
with no framing, base64, or CRC. Smaller code size and faster transfers than
the historical "SMP over console" framing, at the cost of being unable to
share the UART with shell or log output.

Split the single transport/serial.py into a package so the encoded and raw
transports share connection management via a common base class:

  serial/
    __init__.py    re-exports both transports and the fragmentation strategies
    common.py      _SerialTransportBase: the pyserial holder, the open/retry
                   connect loop (overridable for non-serial-port byte pipes),
                   disconnect, SerialException -> SMPTransportDisconnected
                   translation, TX drain, and the RX polling helper
    encoded.py     SMPSerialTransport: the base64 + CRC + delimiter framing, its
                   Auto/BufferSize/BufferParams sizing, and shell interleave
                   (read_serial), now on the shared base
    unencoded.py   SMPSerialRawTransport (new): writes bytes verbatim, reads the
                   SMP header then exactly its payload, and raises on overrun or
                   on a header length exceeding max_unencoded_size

The `from smpclient.transport.serial import SMPSerialTransport` path -- and the
Auto / BufferSize / BufferParams / FragmentationStrategy exports -- is
preserved, so existing users of the encoded transport need no changes.

Integration: the manifest-driven SMP-server suite already described a
native_sim serial_raw fixture and skipped it ("no raw-UART transport yet").
Vendor that fixture, drop the skip, and build SMPSerialRawTransport for it in
conftest, so the raw transport runs the generic echo / enumeration / params
suite. test_serial_raw.py adds a raw-specific check that the transport adopts
the server's buf_size as max_unencoded_size -- the chunk size SMPClient.upload
fills, with no on-wire frame to subtract (unlike the encoded transport's
buf_size - 4).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@JPHutchins JPHutchins force-pushed the feature/unencoded-serial-transport branch from 19f474f to 999fcfc Compare June 23, 2026 00:43
@JPHutchins JPHutchins linked an issue Jun 23, 2026 that may be closed by this pull request
@JPHutchins JPHutchins merged commit 8a6387d into main Jun 23, 2026
29 checks passed
@JPHutchins JPHutchins deleted the feature/unencoded-serial-transport branch June 23, 2026 00:52
@JPHutchins

Copy link
Copy Markdown
Collaborator Author

Support for: zephyrproject-rtos/zephyr#101821

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support unencoded serial

2 participants