Skip to content

feat(generator): a text/event-stream response is read as it arrives - #124

Merged
giraffesyo merged 1 commit into
canaryfrom
sse-streaming
Aug 21, 2026
Merged

feat(generator): a text/event-stream response is read as it arrives#124
giraffesyo merged 1 commit into
canaryfrom
sse-streaming

Conversation

@giraffesyo

Copy link
Copy Markdown
Member

The remaining half of #76, scoped by measurement rather than by guess.

Counting responses that offer several media types: Stripe has none, Mealie has none, GitHub has 78 (diff, patch, sarif variants), and ACTIVATE has exactly one: POST /api/openai/v1/chat/completions, offering application/json and text/event-stream. So the alternate-representation problem in your specs is one endpoint, and what it needs is not "fetch the other media type" but streaming.

What is generated

stream, err := client.ChatCompletionStream(ctx, request)
defer stream.Close()

for {
    chunk, err := stream.Next()      // typed: EventStream[ChatCompletionChunk]
    if errors.Is(err, io.EOF) {
        break
    }
    fmt.Print(chunk.Choices[0].Delta.Content)
}

The buffered method stays, so an endpoint offering both has one method for each, and the streaming one asks for text/event-stream while the other asks for JSON.

The client had to stop reading every response whole

do read the body before anything else could happen. It now shares send with doStream: send runs the request and the retry loop and returns the response with its body unread. A non-2xx comes back as an *APIError carrying the body, so a body is left open only for a response that succeeded, and a retried attempt drains rather than reads, which lets the connection be reused.

Event parsing

Follows the event stream format rather than assuming one shape: several data lines join with newlines, a blank line ends an event, : lines are the keep-alives servers send, and event/id are available through EventName() and EventID(). Two cases worth naming:

  • A [DONE] payload ends iteration instead of failing to decode. That is how OpenAI-compatible APIs close a stream, and it is a marker rather than an event.
  • A stream that ends without a trailing blank line still delivers its last event. My first parser dropped it; TestUnterminatedFinalEventIsDelivered caught that before this was reviewable.

A stream whose schema is a string hands back each event's text rather than parsing it as JSON.

Tests

  • internal/analyzer/operations_test.go: the event type comes from the text/event-stream schema, the JSON alternative still drives the buffered method, a stream with no schema is text, and an ordinary response gets no stream.
  • internal/generator/e2e_streaming_test.go: compiles and runs against httptest, covering typed chunks with a keep-alive and a [DONE], multi-line data with event and id, an unterminated final event, a 429 that is an error rather than a stream, and the buffered method still decoding JSON. One test asserts the point of the feature directly: it holds the server open mid-stream and fails if the first event has not arrived.

Verified on ACTIVATE: ChatCompletionStream(ctx, body, opts...) (*EventStream[ChatCompletionChunk], error), and the client builds and vets.

gofmt, golangci-lint, go vet ./..., and go test ./... pass.

Every response was read whole before anything was decoded, so an endpoint
that streams was reachable only as one lump once the server finished, which
for a chat completion is the opposite of the point. The ACTIVATE spec offers
exactly one such response, its OpenAI-compatible chat endpoint, and its
client could not stream it.

An operation whose success response offers text/event-stream now gets a
second method returning EventStream[T], typed to the schema the stream
declares, read event by event. The buffered method stays, so an endpoint
offering both JSON and events has one method for each.

do and doStream now share send, which runs the request and hands back the
response with its body unread. A non-2xx comes back as an APIError carrying
the body, so a body is left open only for a response that succeeded, and the
retry path drains rather than reads.

The parser follows the event stream format: several data lines join, blank
lines end an event, comments are the keep-alives servers send, and a stream
that ends without the blank line that would have dispatched its last event
still delivers it. A [DONE] payload ends iteration rather than failing to
decode, which is how OpenAI-compatible APIs close a stream.
@giraffesyo
giraffesyo merged commit 84c7655 into canary Aug 21, 2026
7 checks passed
@giraffesyo
giraffesyo deleted the sse-streaming branch August 21, 2026 20:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant