A startup compares two closed-model rate cards before picking a vendor. Model A charges 3 dollars per million input tokens and 15 dollars per million output tokens. Model B charges 1 dollar input and 5 dollars output, a clean two-thirds discount on paper. They switch, expecting the monthly bill to shrink by roughly the same fraction. Instead it comes in higher than what Model A would have cost for the same workload the month before. Request volume did not change. What changed is that Model B is a reasoning model, and the price on the page describes one token, not one answer.

Input and output are priced separately, and neither is the whole story

Closed models bill input and output tokens at different rates, and output is almost always the more expensive of the two, often several times the input rate, because generating text takes more computation per token than reading it does. As of mid-2026, the spread across the market is wide. Budget-tier models can run under 1 dollar per million tokens for input, useful for high-volume, low-stakes work like classification or basic summarization. Flagship-tier models sit anywhere from 15 to 75 dollars per million tokens on the output side, and that gap between cheapest and most expensive is roughly two orders of magnitude. A rate card with a single attractive number on it is usually only telling you the input price, or the cheapest of several tiers, and either omission can quietly triple what a real task ends up costing.

Tiers and rate limits shape what you can actually spend

The sticker price also assumes unrestricted access, which most accounts do not have. Providers commonly gate higher usage tiers behind spending history: a new account might be capped at a modest number of requests or tokens per minute until it has billed enough over time to graduate to a looser limit. Hitting a rate limit does not change the per-token price, but it changes the shape of the bill, because throttled requests get retried, batched, or routed to a pricier fallback model to keep a product responsive. Some providers also offer discounted rates for cached or batched input, prompts reused across calls, or work that can wait in a queue rather than run immediately. None of this shows up in the headline rate. It shows up only once a team has built against the API long enough to hit the ceilings.

The reasoning tax: tokens you never see

The biggest gap between rate card and invoice comes from reasoning models. Before producing a final answer, these models generate an internal sequence of intermediate steps, working through the problem in tokens that are billed as output even though the user never reads them directly. A one-paragraph answer might sit on top of several paragraphs, or several pages, of reasoning tokens spent getting there. It is the same arithmetic as a taxi with a genuinely cheap per-mile rate: the fare still looks reasonable on the driver’s rate card, but if the route to the destination winds through half the city instead of going straight there, the total on the meter has nothing to do with how far away the destination actually was. A short question can trigger a long, winding chain of internal steps, and the bill reflects the route, not the distance.

Per-token prices have been falling for years as models get more efficient to run, and that trend is real. But cost per task does not reliably follow it downward, because the more capable reasoning models are also the ones that think longer to get a better answer, generating more tokens per query even as each individual token gets cheaper. A model that costs half as much per token but reasons three times longer than its predecessor is not a discount, it is a different, larger bill wearing a lower price tag. The most common mistake in picking a closed model is reading the rate card and stopping there, instead of running a representative task through it and reading the invoice. For a closer look at what those internal reasoning steps are actually doing before they turn into an answer, see Models That Reason: Chain of Thought. For the broader mechanics of how token-based pricing works before reasoning enters the picture, see Token Costs and Pricing.