Skip to content

chore: promote dev to main - #108

Merged
patrick-mns merged 8 commits into
mainfrom
dev
Aug 4, 2026
Merged

chore: promote dev to main#108
patrick-mns merged 8 commits into
mainfrom
dev

Conversation

@patrick-mns

Copy link
Copy Markdown
Owner

Promotes dev to main. Eight commits: the empty-reply fix for reasoning models is complete, plus a /loop session driver.

The "I wasn't able to generate a response" dead end, finished

Hit mostly on deepseek/deepseek-v4-flash via the LiteLLM gateway, always right after a large tool result. Traced from .micelio/sessions.db: a reasoning model spent its whole context window on the thinking pass and ended the turn with empty content. The previous PR capped output tokens; this one makes the loop survive the remaining cases.

  • 0e40be1 — make the agent loop resilient to transient/empty LLM responses: retry instead of surrendering.
  • a065979 — escalate the token budget instead of blindly retrying when the model reports finish_reason: length; the first retry doubles the budget so a truncated reply actually gets finished.
  • 348b335 — the empty-reply fallback now names its cause (truncation vs generic) instead of a generic apology, so the message tells you why.
  • 2c153d3 — detect dropped connections disguised as empty completions: a stream that dies mid-flight looked identical to a model that said nothing. Now it retries the stream.

Better context

  • a324ccd — inject the workspace folders into the system prompt; fall back to stale model roles when role inference breaks.

UI

  • e64f9ad — a button to add another folder to the workspace.

/loop session driver

  • ad1bb5b — a session loop driver: registry, runner, and loop-control tools (scheduled wakeup, status, stop).

patrick-mns and others added 8 commits July 31, 2026 16:18
Adds a "+" icon button next to the folder selector in GitContext so
users can add a folder to the current workspace without leaving the
selector context.
… model roles

The model had no awareness of the workspace's folders or which one was
active, only tools did. Adds a "## Workspace" section to the system
prompt listing every folder and marking the active one.

Also fixes model role selectors (chat/summarize/vision) showing
UNAVAILABLE forever when the persisted model name belonged to a
provider that's no longer active — get_model_roles now falls back to
the first catalog model that fits the role and persists the swap.
DeepSeek and other reasoning models via OpenAI-compatible endpoints were
producing empty turns (reasoning ate the whole max_tokens budget) and the
loop bailed on the first stream error with no retry.

- Raise max_tokens cap 8192 -> 16384 so reasoning content doesn't starve
  the answer.
- Retry an empty model turn up to 3x instead of once.
- Retry transient stream-open/mid-stream network errors (rate limits,
  connection resets) up to 2x with backoff before failing the whole turn,
  as long as nothing has streamed yet.
- Raise MAX_TOOL_ROUNDS 50 -> 100 for long legitimate multi-step turns.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Fixes CI formatting check broken since a324ccd.
…cation

The empty-turn retry added earlier wasn't enough: when a reasoning model
truncates deterministically (finish_reason == "length", spends its whole
budget on <think> with zero answer), repeating the identical request just
repeats the same runaway reasoning and truncates again every time.

- Provider::start_stream now takes an optional max_tokens override; both
  OpenAI-compatible and Ollama backends honor it (Ollama also widens
  num_ctx to fit).
- OpenAI-compatible streams now surface finish_reason via a new
  StreamEvent::FinishReason.
- The agent loop detects finish_reason == "length" and, on retry, doubles
  the completion budget (16384 -> 32768 -> 65536) with its own 3-attempt
  budget and a nudge telling the model to stop deliberating and answer,
  instead of sharing the generic empty-reply retry path.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The generic "I wasn't able to generate a response. Please try again or
rephrase your request." gave no signal about what actually happened,
so users had no way to tell a flaky connection from a model stuck
truncating on reasoning.

ensure_reply now takes an explicit fallback message; the agent loop
tracks whether the empty-reply retries were exhausted specifically due
to truncation (finish_reason == "length") and picks a fallback that
says so plus what to try next, instead of a one-size-fits-all apology.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Root-caused via the boost workspace's sessions.db: the failing turn's
raw response was 50+ chunks of pure reasoning_content with finish_reason
null throughout — the gateway connection closed mid-thought without ever
sending [DONE] or a finish_reason, and the SSE reader treated that
silently as a normal end-of-stream, indistinguishable from the model
choosing to say nothing.

openai_compat.rs now tracks whether the stream actually terminated
properly; if it didn't and nothing was produced, it flags
FinishReason("dropped") instead of just going quiet. The agent loop
treats that the same as the other stream-error cases (retry with
backoff) instead of running it through the model-facing empty-reply
nudge, since there's nothing to nudge — the model never got to finish.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@patrick-mns
patrick-mns merged commit 6a4bc13 into main Aug 4, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant