Calling a model

What it isCopy Anchor Link

You send a model name, a system prompt, and a list of user and assistant messages. You get back the next assistant message and a usage count of input and output tokens. The API is stateless: to continue a conversation you send the whole history again. Streaming returns tokens as they are produced so the interface feels fast.

What to get right earlyCopy Anchor Link

Put the API key in the environment, never in code. Set a max output length. Log the request, response and token counts from the first day, because you will need them to debug and to see cost. Handle rate limits and timeouts with retries and backoff. Pin the model version in config so an upgrade is a deliberate change you can test.

Try itCopy Anchor Link

Write a 20-line script that sends one message, prints the answer and prints the token usage. Then turn on streaming.

A worked example: your first request, in wordsCopy Anchor Link

You send: a model name, a system message ("You suggest gifts in two sentences") and a user message ("Gift for a hiker, under $50"). You get back: the reply text and a usage block, for example 38 input tokens and 52 output tokens. At $3 and $15 per million tokens that call cost about $0.0009. Print the usage every time. It teaches you what things cost and where the tokens go.

Setting up safelyCopy Anchor Link

Create an API key in the provider's dashboard. Store it in an environment variable on your machine, never inside the code file. Add the file that holds it to .gitignore. If a key ever lands in a public repository, assume it is stolen and revoke it.

How one request travelsCopy Anchor Link

1. Your code builds a JSON message (a plain-text format of labeled values) holding the model name, system prompt, messages and a length cap.
2. It sends that message over HTTPS (the encrypted version of the web protocol) with your key in a header.
3. The provider checks the key, counts the input tokens and queues your request on its hardware.
4. The model writes the answer one token at a time until it finishes or hits your cap.
5. You get back the text, a stop reason (the model finished, or your length cap cut it off) and the usage counts.

What goes wrongCopy Anchor Link

The answer stops mid-sentence. Your length cap was too low. To confirm, read the stop reason in the response: it says the cap was reached.

You get a 401 error, the code for a rejected key. Often the environment variable is set in one terminal window and your script runs in another. Check whether the variable exists by printing only "set" or "missing", never the key itself.

The bill is higher than you expected. Look at the logged input tokens per call. A long system prompt or a growing history is usually the cause.

The same question gives slightly different answers on two runs. This is normal, because the model samples its words. It is not a bug.

A second example: a call that should not existCopy Anchor Link

Your app must add up a customer's cart total. A model can do this, but it is slower, costs money and can make an arithmetic slip. Three lines of ordinary code give the exact answer every time. Use a model for fuzzy language work such as summarizing or classifying. Use plain code when the rule is exact.

How to tell it is workingCopy Anchor Link

Send a very short prompt and check that the input token count is small. Then paste in a page of text and watch the count jump. Finally set the length cap to 5 and confirm the answer is cut off after a few words. If all three behave as described, you understand what you are paying for and what the cap does.

Common mistakesCopy Anchor Link

Putting the key in browser code. Forgetting a max output length. Not logging the request and the usage, which makes every later bug harder to find.

The codeCopy Anchor Link

Here is the whole request in Python, using the Anthropic SDK. Install it with pip install anthropic and set ANTHROPIC_API_KEY in your environment first. Other providers have nearly identical SDKs, and TypeScript versions exist.

from anthropic import Anthropic

client = Anthropic()  # reads ANTHROPIC_API_KEY from the environment

reply = client.messages.create(
    model="claude-sonnet-5-5",
    max_tokens=300,  # cap the answer so one runaway reply cannot cost much
    system="You suggest gifts in two sentences.",
    messages=[{"role": "user", "content": "Gift for a hiker, under $50"}],
)

print(reply.content[0].text)
print("tokens in/out:", reply.usage.input_tokens, reply.usage.output_tokens)

Run it, then change the system message and the question and watch how the answer and the token counts move.