DUX-8223: Map Bedrock 'too many tokens, please wait' 429 to RateLimitError#17
Conversation
Bedrock returns a 429 with "Too many tokens, please wait before trying again" for token-rate throttling. The message matches the generic "too many tokens" context-length pattern, so the 429 branch was misclassifying it as ContextLengthExceededError instead of a retryable RateLimitError. Add a rate-limit pattern that matches the throttle phrasing so it is caught before the context-length check. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: Sam Boland <sam.boland@appfolio.com>
|
Jira Issue: https://appfolio.atlassian.net/browse/DUX-8223 |
|
🤖 Supernova Code Review — View trace |
![staging-supernova-dx-appf-io[bot]](https://avatars.githubusercontent.com/in/2831657?s=60&v=4){kind=small}
There was a problem hiding this comment.
Reviewed the two changed files and found no actionable issues.
lib/ruby_llm/error_middleware.rb — The fix is correct. Adding /too many tokens.*please wait/i to RATE_LIMIT_PATTERNS ensures the Bedrock throttle message is matched by rate_limited? before the 429-handler falls through to context_length_exceeded?, which would have caught it via the broader /too many tokens/i pattern. The .*please wait qualifier makes the pattern specific enough to avoid misclassifying genuine context-length errors.
spec/ruby_llm/error_middleware_spec.rb — The new test case directly exercises the bug that was fixed and clearly documents the intended disambiguation. The existing test at line 87–95 ("still maps a 400 'too many tokens' error to ContextLengthExceededError") also confirms the 400-path is unaffected, which is good coverage.
No bugs, logic errors, or other concerns found.
2 participants
Problem
Bedrock's ThrottlingException (HTTP 429) with the message "Too many tokens, please wait before trying again." was being misclassified as
ContextLengthExceededErrorinstead ofRateLimitError. This caused incorrect error handling and retry logic in production.Root Cause
The
rate_limited?pattern matching inerror_middleware.rbdid not include the specific Bedrock throttling message pattern. When the 429 response didn't match rate limit patterns (/rate limit/,/per minute/,/per hour/,/per day/), the error fell through tocontext_length_exceeded?, which includes/too many tokens/iin its patterns, causing the misclassification.Solution
Added the pattern
/too many tokens.*please wait/ito theRATE_LIMITED_PATTERNSarray to correctly identify Bedrock's token-rate throttling as a retryable rate limit error, not a context overflow error.Changes
lib/ruby_llm/error_middleware.rb: Added Bedrock throttling pattern toRATE_LIMITED_PATTERNSspec/ruby_llm/error_middleware_spec.rb: Added test case to verify the 429 response with Bedrock's specific message is correctly mapped toRateLimitErrorTesting
Added test case: "maps Bedrock's 'too many tokens, please wait' 429 throttle to RateLimitError, not context length"
Agent session: https://staging.supernova.dx.appf.io/coders/073f6ce3-0d65-47e6-9df0-695acbc8a36c