Errors, retries and safe rendering

ErrorsCopy Anchor Link

Read the status code. 400: your request is malformed or too long, so retrying will not help. 401 or 403: the key is wrong or lacks access. 429: rate limit, so wait and retry with exponential backoff (1 second, 2, 4, with some random jitter). 500 or 529: the provider is overloaded, so retry a few times and then show a clear message. Set timeouts. Log the request id from the response headers, which providers ask for in support tickets. Show users a plain message, never the raw error.

Keys stay on the serverCopy Anchor Link

An API key in browser code is a key anyone can copy. Call the provider from your backend, keep the key in an environment variable and rate-limit per user. Never commit keys to git.

Rendering model output safelyCopy Anchor Link

Models often reply in markdown. If you convert it to HTML and insert it into the page, a prompt-injected or odd answer can contain a script tag or a link that runs code. Sanitize the HTML with a library, or render markdown with a safe renderer that does not allow raw HTML. Treat the output like user input, because it can be shaped by user input. Do the same before putting model text into a database query or a shell command.

How retry with backoff worksCopy Anchor Link

1. Send the request. It fails with a retryable error such as 429 (rate limited) or an overloaded or 5xx server error.
2. Wait about 1 second plus a small random amount, then try again.
3. If it fails again, wait about 2 seconds, then about 4.
4. Stop after a fixed number of tries and show the user a plain message.

Without jitter, the waits total 1 + 2 + 4 = 7 seconds across the three retries. Jitter adds a random fraction so many clients do not retry together. Always cap the number of attempts and the longest wait.

A second example: retrying makes it worseCopy Anchor Link

A request fails with 400 because the prompt is too long. Retrying five times sends the same oversized prompt five times and fails five times. The fix is to shorten the prompt. A 401 behaves the same way: the key is wrong, and repeating the call changes nothing. Retry only errors that can pass on their own: rate limits, overload and timeouts.

What goes wrong when you render outputCopy Anchor Link

Markdown converted to HTML with raw HTML allowed can carry a script tag. Symptom: odd pop-ups or a changed page layout after one particular answer. To confirm, test your renderer with a harmless string such as an image tag with a broken source and an alert handler. If the alert fires, the renderer is unsafe.

Links can use a "javascript:" address that runs code when clicked. A safe renderer blocks that address type.

Model text placed into a database query can change the query. Pass values separately from the query text (a parameterized query) so the database treats them as data.

Try it yourselfCopy Anchor Link

Write a function that returns the wait before each of 5 attempts. Without jitter it should return 1, 2, 4, 8, 16 seconds. Add jitter and run it three times to see the lists differ. Then feed your page the harmless test string above and check that nothing runs.

Common mistakeCopy Anchor Link

Showing raw error text to users. It can reveal internal details and it is unhelpful.