API errors and timeouts
What to do when CheapAI returns HTTP 429 or 503, takes a long time to begin a response, or ends a stream early.
HTTP 429 — Too Many Requests
Pause before retrying. If the response includes Retry-After, wait for the indicated time. Otherwise use exponential backoff with jitter. If 429s keep coming back, lower concurrency for that model.
Timeouts and slow responses
Use "stream": true for interactive requests on endpoints that support SSE. Text, reasoning and tool-call events show generation progress. A keepalive ping tells you the connection is alive. Before the response starts, client and gateway deadlines apply. A separate idle timeout applies during streaming.
The stream ended after sending a partial response
Handle partial output separately. If the client received a tool call, check whether it ran before retrying the whole request. When reporting a repeated error, include the time, model, endpoint, status code and request ID if available.