feat(generator): a text/event-stream response is read as it arrives - #124
Merged
Conversation
Every response was read whole before anything was decoded, so an endpoint that streams was reachable only as one lump once the server finished, which for a chat completion is the opposite of the point. The ACTIVATE spec offers exactly one such response, its OpenAI-compatible chat endpoint, and its client could not stream it. An operation whose success response offers text/event-stream now gets a second method returning EventStream[T], typed to the schema the stream declares, read event by event. The buffered method stays, so an endpoint offering both JSON and events has one method for each. do and doStream now share send, which runs the request and hands back the response with its body unread. A non-2xx comes back as an APIError carrying the body, so a body is left open only for a response that succeeded, and the retry path drains rather than reads. The parser follows the event stream format: several data lines join, blank lines end an event, comments are the keep-alives servers send, and a stream that ends without the blank line that would have dispatched its last event still delivers it. A [DONE] payload ends iteration rather than failing to decode, which is how OpenAI-compatible APIs close a stream.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The remaining half of #76, scoped by measurement rather than by guess.
Counting responses that offer several media types: Stripe has none, Mealie has none, GitHub has 78 (diff, patch, sarif variants), and ACTIVATE has exactly one:
POST /api/openai/v1/chat/completions, offeringapplication/jsonandtext/event-stream. So the alternate-representation problem in your specs is one endpoint, and what it needs is not "fetch the other media type" but streaming.What is generated
The buffered method stays, so an endpoint offering both has one method for each, and the streaming one asks for
text/event-streamwhile the other asks for JSON.The client had to stop reading every response whole
doread the body before anything else could happen. It now sharessendwithdoStream:sendruns the request and the retry loop and returns the response with its body unread. A non-2xx comes back as an*APIErrorcarrying the body, so a body is left open only for a response that succeeded, and a retried attempt drains rather than reads, which lets the connection be reused.Event parsing
Follows the event stream format rather than assuming one shape: several
datalines join with newlines, a blank line ends an event,:lines are the keep-alives servers send, andevent/idare available throughEventName()andEventID(). Two cases worth naming:[DONE]payload ends iteration instead of failing to decode. That is how OpenAI-compatible APIs close a stream, and it is a marker rather than an event.TestUnterminatedFinalEventIsDeliveredcaught that before this was reviewable.A stream whose schema is a string hands back each event's text rather than parsing it as JSON.
Tests
internal/analyzer/operations_test.go: the event type comes from thetext/event-streamschema, the JSON alternative still drives the buffered method, a stream with no schema is text, and an ordinary response gets no stream.internal/generator/e2e_streaming_test.go: compiles and runs againsthttptest, covering typed chunks with a keep-alive and a[DONE], multi-line data witheventandid, an unterminated final event, a 429 that is an error rather than a stream, and the buffered method still decoding JSON. One test asserts the point of the feature directly: it holds the server open mid-stream and fails if the first event has not arrived.Verified on ACTIVATE:
ChatCompletionStream(ctx, body, opts...) (*EventStream[ChatCompletionChunk], error), and the client builds and vets.gofmt,golangci-lint,go vet ./..., andgo test ./...pass.