QuestionAI / MLAI / ML. models, prompts, intelligence
how do I handle rate limits and 429s gracefully?
under load I get 429s and the whole request just fails. whats a clean retry-with-backoff pattern for production?
2 answers
under load I get 429s and the whole request just fails. whats a clean retry-with-backoff pattern for production?
you are closer than you think. it is the ordering — do the setup step before the call.
this bit me too. short version: it is a timing thing. move it into the load step.