The messages array, roles and model versions

RolesCopy Anchor Link

The system message sets standing instructions. User messages carry what the person asks. Assistant messages are the model's earlier replies. Tool messages carry tool results. Each call sends the whole list, in order, and the model writes the next assistant message. To "remember" something, include it. To "forget", leave it out.

ExampleCopy Anchor Link

Call 1 sends [system: "You are a gift expert. Answer in two sentences.", user: "Gift for a hiker?"]. The reply is added as an assistant message. Call 2 sends all three messages plus the new user message "Under $50?". Without the earlier messages the model would not know what "under $50" refers to. The cost grows too: call 2 pays for call 1's tokens again, which is why long chats get expensive and why trimming and summarizing matter.

Pin the model versionCopy Anchor Link

Model names like "claude-sonnet-5-5" can point at a moving target while dated snapshots stay fixed. Pin the exact version in config for anything in production, run your eval set before changing it, and keep a fallback. A new version can be better overall and still break your prompt.

Token costs and the windowCopy Anchor Link

Count tokens before sending: input plus the maximum output must fit in the context window. Set a max output length so one runaway answer cannot cost a lot.

A worked example: why the bill growsCopy Anchor Link

Say the first question is 100 tokens, every later question is 50 and every reply is 100. Turn 1 sends 100 tokens. Turn 2 sends the first question and its reply (200) plus the new question (50): 250. Turn 3 sends those 250, plus the second reply (100) and the new question (50): 400. By turn 10 each call carries about 1,450 tokens of history. The cost of a conversation grows faster than its length. Trim old turns or summarize them.

Where to put whatCopy Anchor Link

System message: rules that never change. User messages: this turn's request. Assistant messages: what the model said before. Do not paste secrets into any of them if you would not want them in logs.

How the model sees the listCopy Anchor Link

Behind the API, your list is turned into one long piece of text, with markers showing who said each part. The model predicts what comes next after the final marker. There is no hidden memory slot. The exact markers differ by provider.

Roles are labels the model was trained to respect, not locks. A user message can still try to talk the model out of its system message. Never rely on a system prompt alone to protect a secret or enforce a rule.

A second example: forgetting on purposeCopy Anchor Link

A user pastes a password into turn 2 by mistake. If your code drops that message from the list, later calls no longer carry it. Be clear about the limit: the provider and your own logs may still hold the copy from call 2. Removing a message from history stops future use, not past exposure.

Trade-offs when the list gets longCopy Anchor Link

Dropping the oldest turns is cheap, but the model loses facts from the start, such as the user's budget. Summarizing old turns keeps the gist but costs an extra model call and can drop details. A common middle path keeps the system message, a short summary and the last few turns.

What goes wrongCopy Anchor Link

The model forgets something you told it. Either your code trimmed it or rebuilt the list from scratch. Print the list just before each call and look for the missing line.

Two users see each other's context. The example above keeps history in one global list. A web server handling many visitors must keep one list per session. To confirm, print the length of the list for two different users.

How to tell it is workingCopy Anchor Link

Tell the model your name in turn 1. In turn 2 ask "What is my name?" with the full list: it answers. Send the same question alone: it cannot. That difference is the whole mechanism.

Common mistakeCopy Anchor Link

Using a model alias such as "latest" in production. A silent upgrade can change behavior. Pin a dated version and upgrade on purpose after testing.

The codeCopy Anchor Link

To keep a conversation going, keep a list and append to it after every turn.

history = []

def ask(text):
    history.append({"role": "user", "content": text})
    reply = client.messages.create(
        model="claude-sonnet-5-5",
        max_tokens=300,
        system="You suggest gifts in two sentences.",
        messages=history,  # the whole conversation, every time
    )
    answer = reply.content[0].text
    history.append({"role": "assistant", "content": answer})
    return answer

print(ask("Gift for a hiker, under $50"))
print(ask("What about under $20?"))  # works only because history was sent again