An AI reasoning model spends additional computation while solving a problem, before or between the parts of its visible answer. That extra work is usually called test-time compute or inference-time compute. It can improve results on problems that require planning, calculation, code, or several dependent decisions. It does not turn the model into a human thinker, and it does not make every answer better.
The practical question is not whether a model "reasons" in the philosophical sense. It is whether a particular inference strategy improves a measurable task enough to justify its added latency and cost.
Short version: reasoning is a compute budget, not a guarantee. Reserve it for difficult, verifiable work. A faster model usually suits simple extraction, routing, classification, and short responses. Compare both on the same test set before choosing.
What is an AI reasoning model?
A reasoning model is a language model designed or trained to use more inference-time work on a problem than a direct-response model normally would. Depending on the system, that work can include generating internal reasoning tokens, revising an approach, sampling multiple candidates, scoring candidates with a verifier, calling tools, or repeating part of a workflow after a failed check.
These mechanisms are not interchangeable. A product may expose an effort setting without revealing its internal implementation. Another may return a summary of its reasoning. A research system may search over several candidate solutions. Calling all of them "a hidden chain of thought" erases differences that matter for cost, latency, and reliability.
| Question | Direct-response model | Reasoning configuration |
|---|---|---|
| Primary goal | Produce an answer with low delay | Spend more compute on a difficult solution |
| Good first use | Extraction, classification, rewriting, simple lookup | Multi-step math, code repair, planning, constraint-heavy analysis |
| Main tradeoff | May fail on long dependency chains | Usually adds tokens, latency, and cost |
| What the user sees | Final answer | Final answer, sometimes with a provider-generated summary |
| Proof of correctness | Neither mode proves its own answer; both need task-specific checks | |
Test-time compute does not replace model scaling
The phrase "test-time scaling" describes increasing or allocating computation when the model answers, in addition to the work done during training. It provides another axis of improvement, but cannot replace pretraining, data quality, model architecture, or post-training.
A weak base model does not become universally capable because it generates a longer scratchpad. Research by Snell and colleagues found that the best allocation of test-time compute depends on the problem, its difficulty, and the base model. Their experiments studied adaptive response refinement and search guided by a process-based verifier. The result is a conditional strategy, not a law that more tokens always win.
OpenAI's early o1 research similarly reported gains from both reinforcement learning during training and more compute at test time. That work is evidence for one family of systems, not proof that every provider uses the same architecture or training recipe.
How reasoning models are trained
Most commercial training pipelines are only partly disclosed. Published pipelines illustrate individual approaches; they do not define the category.
The DeepSeek-R1 paper provides one documented example. DeepSeek-R1-Zero used large-scale reinforcement learning without supervised fine-tuning as a preliminary step. The later DeepSeek-R1 pipeline added cold-start data, further reinforcement learning, rejection sampling, and supervised fine-tuning. The two variants alone show why a fixed formula such as "SFT, then RL, then calibration" should not be presented as the definition of a reasoning model.
Training can reward correct final outcomes, productive intermediate behavior, tool use, format compliance, or some combination. It can also teach a model to spend different amounts of effort on different tasks. The exact data, reward design, search procedure, and model changes vary. Unless a provider publishes those details, the honest description is that the implementation is proprietary.
What can happen during inference
Test-time compute can be spent in several ways:
- Sequential reasoning: the model generates internal tokens before producing the visible response.
- Revision: it checks an intermediate result and tries another approach.
- Sampling and selection: the system produces multiple candidates and selects one using a score or verifier.
- Search: it explores branches of a solution instead of committing to the first path.
- Tool loops: it calls code, search, a calculator, or another service, inspects the result, and continues.
- Routing: a controller assigns more effort only when the input appears difficult.
A deployed product may combine several of these. It may also hide most of the process. The OpenAI reasoning guide, for example, says internal reasoning tokens are not returned as raw text through the API, but they still occupy context and are billed as output tokens. Optional reasoning summaries are not the raw internal trace.
Provider controls and visible output
Provider terminology changes, so treat the live documentation for your chosen model as the operational source. The documentation checked on July 17, 2026 showed these broad patterns:
| Provider pattern | Typical control | What may be returned | Important caveat |
|---|---|---|---|
| OpenAI reasoning models | Model-dependent reasoning effort | Final answer and, when requested and supported, a summary | Raw reasoning tokens are not exposed; supported effort levels vary by model |
| Claude extended thinking | Adaptive thinking or a model-dependent manual budget | Summarized thinking blocks or omitted or empty thinking blocks, depending on the model and display setting | The visible text may not represent all billed thinking tokens |
| Gemini thinking | Model-dependent thinking level or budget | Answer, optional thought summary, and signatures used for continuation | Controls and required signatures differ by model and interaction mode |
See the current provider guides for Claude extended thinking and Gemini thinking. Confirm that the target model supports a budget or effort value before carrying it over from a tutorial.
When more reasoning helps, and when it does not
Extra reasoning is most defensible when the task has several dependent steps and an answer you can check. Examples include repairing code against a test suite, reconciling constraints in a schedule, solving a math problem with a known result, or planning a tool workflow with explicit completion criteria.
It is often wasteful for tasks such as copying fields from a document, assigning a message to one of five fixed categories, translating a short label, or answering a fact that should come from a database. If the missing ingredient is current data, more internal tokens will not supply it. Give the model the data or an appropriate tool.
| Task | Default approach | Reasoning is worth testing when |
|---|---|---|
| Extract invoice fields | Fast structured-output model | Layouts are ambiguous and errors require cross-field checks |
| Route a support ticket | Fast classifier | Categories overlap and policy exceptions matter |
| Fix a failing function | Reasoning model with tests | The defect spans files or requires several hypotheses |
| Plan a migration | Reasoning model with system constraints | Steps have dependencies, rollback rules, and verification gates |
| Answer a current policy question | Retrieval plus citations | Several sources conflict and the model must reconcile them |
There are also failure modes specific to spending more compute. A model can pursue a bad premise for longer, use a tool incorrectly several times, or produce a confident but unfaithful explanation. More effort should be an evaluated setting, not a prestige feature.
How to prompt a reasoning model
Write the prompt directly. OpenAI's reasoning prompt guidance recommends simple instructions and says prompts such as "think step by step" are unnecessary. Asking for hidden chain of thought is an unreliable control and may conflict with how a provider presents reasoning.
The prompt should specify the job, available evidence, constraints, definition of success, tools, and output format:
Task
Find the cause of the three failing tests and propose the smallest safe patch.
Evidence
Use only the attached repository, test output, and API documentation.
Constraints
Preserve the public API and existing passing behavior.
Done when
Explain the root cause, identify affected files, provide the patch,
and list the tests that verify it.
Output
Return: diagnosis, patch, verification, remaining uncertainty.Use delimiters when the prompt contains source material. Add examples only when the desired classification or format is hard to infer. If the API exposes an effort or budget setting, tune it with evaluations instead of trying to control cost through prompt wording.
How to evaluate reasoning in production
Run direct-response and reasoning configurations on the same representative cases. Keep the prompt, tools, and grading rules stable. Measure the result of the task, not how convincing the explanation sounds.
- Build a fixed test set. Include common cases, difficult cases, missing data, conflicting evidence, tool failure, and requests that should be declined.
- Define a grader. Prefer executable tests, exact constraints, cited-source checks, or blinded human review over a model grading its own prose.
- Record quality. Track task success, factual correctness, tool correctness, format compliance, and serious failure rate.
- Record resources. Track input, output, and reasoning tokens where available, plus cost per successful task.
- Record latency. Measure time to first visible token, end-to-end duration, and p50/p95 latency, not one hand-picked run. Our separate guide explains TTFT, inter-token latency, and end-to-end latency.
- Choose the lowest sufficient setting. Raise effort only when the quality gain survives repeated tests and matters to the product.
Compare cost per successful task. A cheap configuration that fails half the time may cost more after retries and review. An expensive configuration that adds no measurable success is simply expensive.
Internal reasoning is not evidence
A reasoning trace or summary can help a user follow an answer, but it is not a direct reading of the model's neural state and it may not faithfully report every influence on the result. Anthropic's experiments found that reasoning models do not always state the hints they used. OpenAI's o1 system card likewise treated chain-of-thought faithfulness as an open research question.
For important decisions, validate claims against external evidence. Run the code. Recalculate the number. Check that cited text supports the conclusion. Require confirmation before a tool changes money, permissions, customer data, or production systems. A fluent rationale helps communication; it is not an audit trail by itself.
