Reasoning and thinking models
Reasoning models spend extra tokens thinking before they answer. Use them where the problem needs it and pay for it knowingly.
Words on this page (7)
- token
- : A piece of text the model reads and writes, usually a word or part of a word. About 4 characters of English on average.
- reinforcement learning
- : Learning by trial and error: take actions, get rewards, and favor what earned reward.
- chain of thought
- : Having the model write out intermediate reasoning steps before the final answer.
- latency
- : The time you wait for a response.
- agent
- : A model run in a loop that chooses its own tools and steps until the task is done.
- reasoning model
- : A model trained to think at length, often with RL, before answering.
- logarithm
- : The power you raise a base to in order to get a number. log base 2 of 8 is 3 because 2 to the power 3 is 8.
What it is
These models are trained with to produce a long before the final answer. Providers expose a budget or effort setting that controls how much thinking happens. More thinking improves math, code and multi-step planning. It adds and cost, and can overthink simple questions.
Why it matters
Do not copy old prompting habits blindly. "Think step by step" is unnecessary for models that already think. Give a clear goal and the constraints, and let the model decide how to reach it. Start with low effort and raise it only for steps that fail. In an , a common split is high effort for planning and review, low effort for routine steps.
A worked example: when it pays off
A bug takes three files to understand. A fast model guesses a fix and often misses. A reads, forms a hypothesis, checks it against the code and adjusts, then proposes a fix. For a typo fix, the same model is overkill and ten times slower. Match effort to difficulty.
How the thinking works
Before the final answer, the model writes hidden reasoning . Some providers show a summary of them. Those tokens are usually billed as output. The effort setting limits how many it may use. Invented numbers: 2,000 thinking tokens plus a 100 token answer is 2,100 tokens, which is 21 times the answer alone. Because you pay for the thinking, the question is whether it buys a better answer.
A second example: when thinking does not help
Take "Rewrite this sentence more politely". There are no steps to plan, so extra thinking adds delay and cost with no gain. Take a question about a private fact the model never saw. More thinking cannot create missing knowledge. It can only produce a longer, more polished wrong answer. Thinking helps most when the task has several steps and the model knows the facts.
What goes wrong
The symptoms are slow replies, large bills, and answers to easy questions that arrive after long deliberation. Sometimes the final answer contradicts the reasoning that came before it. To confirm the cause, the token usage of each call (most APIs report it) and run the same 20 test questions at low and high effort. Compare accuracy and tokens. If accuracy does not rise, lower the setting.
Try it yourself: easy against hard
Write 10 questions: five easy lookups and five that need several steps, such as a small scheduling puzzle. Run each at low effort and at high effort, or on a normal model and a reasoning model. Record correctness and waiting time. Expect the easy questions to have the same accuracy but slower answers, and the hard ones to improve. That gap is what you are paying for.
Common mistake
Telling a reasoning model exactly how to think. State the goal and constraints, and let it find the route.
Ask a quick question like "capital of Japan?" and a normal model answers instantly. Ask a hard puzzle and a reasoning model quietly writes pages of notes to itself first, checks its own work, backtracks, and only then answers. It is slower and costs more, and on hard problems it is far more likely to be right.
Check yourself
You use a model that already reasons internally. What should you drop from the prompt?
Not quite. State the goal and constraints, and let the model plan the route.
Where is high thinking effort most worth paying for?
Not quite. Planning and review benefit most from extra reasoning. Routine steps do not.
A reasoning model answers a simple question in 20 seconds and the bill is large. What is a sensible fix?
Not quite. Match thinking effort to difficulty, since hidden thinking tokens are usually billed.
You got 0 of 3 right.
Go deeper
- Article
Lilian Weng · Explains why thinking at test time helps and how it is trained.
- Paper
Wei et al. · The paper that showed written reasoning improves accuracy.
- PaperDeepSeek-R1Paper
DeepSeek · Open recipe for training reasoning with verifiable rewards.
Spotted a mistake or something unclear? Report it on GitHub.