How to Use Tokens Efficiently

A practical guide to spending fewer tokens without getting worse answers: where tokens go, how caching cuts input cost, and which writing habits shrink output.

Who this is for: anyone using AI tools or building on model APIs who wants lower cost and faster responses, not theory. A basic understanding of tokens helps; see the companion guide What Is a Token? How it's organized: where tokens go, then how to spend less on input, then how to spend less on output, then a checklist.

The one-sentence version: the cheapest token is the one you never send, the second cheapest is the one the provider caches for you, and the most expensive is the one you make the model generate twice.

Part 1: Where Tokens Go

Every request spends tokens in several places, and only some of them are the words you type:

Destination	What lands there	Who controls it
System instructions	Role, rules, constraints the app sets	The developer
Tool definitions	Names, descriptions, and schemas of every tool the model can call	The developer
Conversation history	Every earlier message and tool result in the thread	The app
Documents and retrieved text	Files, web pages, database results pasted or fetched into the prompt	You
Your message	The actual question or instruction	You
Output	The visible answer	The model, steered by you
Reasoning	Invisible thinking before the answer, billed as output [2]	The model, tunable

Two habits follow directly from this table. Tool definitions and system instructions are often the biggest fixed cost in an agent application, and they sit at the front of the prompt where caching can absorb them. Documents and history are the biggest variable cost, and they respond to editing discipline.

Part 2: Spend Less on Input

Put stable content first, changing content last

Providers cache the model's work on the unchanged beginning of your prompt, called the prefix. Reused tokens are billed at the cached-input rate, discounted up to 90 percent versus ordinary input [1]. Anything that changes between requests, such as timestamps, user-specific details, or fresh tool results, belongs at the end of the prompt so it does not break the reusable prefix [1].

A practical layout:

Position	Content	Why
First	System instructions, role, shared reference material	Rarely changes; earns cache hits
Middle	Tool definitions and schemas	Keep names, descriptions, and ordering stable [1]
Next	Conversation history, appended in order	Rewriting or summarizing earlier turns resets reuse [1]
Last	Timestamps, user details, the current question	Changes every request; keeps the prefix intact

Let caching do the arithmetic

On current OpenAI models, a cache write costs 1.25 times the normal input rate, and a cache read costs 0.1 times [1]. The write pays for itself immediately: writing a prefix once and reusing it once costs 1.35 times its ordinary input cost, against 2 times for processing it twice uncached [1]. Across ten requests that reuse the same prefix, one write plus nine reads costs 2.15 times, against 10 times without caching [1].

Pattern	Relative input cost
Same prefix, no caching, processed twice	2.0x
Cache write plus one reuse	1.35x
Cache write plus nine reuses (ten requests)	2.15x
Same prefix, no caching, processed ten times	10.0x

Two details decide whether you actually see these savings. The cacheable prefix has a minimum length (1,024 input tokens on GPT-5.6 and later models [1]), and cache entries expire: on current models the minimum lifetime is 30 minutes after the most recent write or reuse, and each reuse refreshes it at no extra charge [1]. Bursty traffic within that window is what caching rewards.

Keep the conversation append-only

In multi-turn applications, the growing history is usually the single largest input cost, and it is also the part caching saves most [1]. Append new messages instead of editing old ones. Summarizing, compacting, or truncating history changes the prefix and resets reuse from the first changed token onward [1]. If you must compact, do it deliberately and accept the one-time cache reset rather than trimming a little every turn.

Keep tool definitions stable

Changing a tool name, description, schema, or ordering changes the prefix and discards reuse from that point [1]. When a request should not use tools, disable them for that request instead of removing the definitions [1]. When only a subset should be callable, restrict which tools are enabled while keeping the supplied list identical [1].

Cut context you cannot cache

For material that changes every request, the saving comes from editing, not caching. The rules from What Is a Prompt? apply unchanged: include what changes the answer, cut the rest, and put the instruction first, the material after, and the key requirement again at the end.

For long documents, structure matters as much as length. Anthropic's long-context guidance for documents of 20,000 tokens or more: place the documents at the top of the prompt and the query at the end, which improved response quality by up to 30 percent in their tests on complex multi-document inputs [2]. For multi-document tasks, ask the model to quote the relevant passages first and reason from the quotes; this focuses attention on the content that matters and away from the rest of the document [2].

Part 3: Spend Less on Output

Output tokens usually cost more than input tokens, and reasoning tokens are billed as output even though they never appear in the answer [2]. Four habits keep this line item down.

1. Ask for the length and shape you want. Models default to generous responses. State the target: a paragraph, a table, five bullets, under 200 words. Anthropic's formatting guidance also favors telling the model what to do rather than what not to do: "write in flowing prose paragraphs" beats "do not use markdown" [2].

2. Match your prompt's style to the output you want. The formatting of the prompt influences the formatting of the response. If the answer comes back over-formatted, removing markdown and bullet structure from your own prompt reduces the volume of markdown in the output [2].

3. Steer thinking down on easy tasks. On models with adaptive thinking, the model decides how much to think based on effort settings and question complexity, and simpler questions get direct answers [2]. If a model thinks more often than the task deserves, say so: Anthropic's own sample steering prompt is "Thinking adds latency and should only be used when it will meaningfully improve answer quality. When in doubt, respond directly" [2]. Prefer lowering the effort setting or capping output with the max output parameter as the hard ceiling [2].

4. Do not regenerate what you can reuse. Asking for three variants triples output cost. Ask for one good answer, critique it in a follow-up, and let prompt caching absorb the repeated input.

Part 4: The Checklist

Technique	What it saves	Where it applies
Stable content first, dynamic last	Cache hits on the prefix	Every repeated request [1]
Append-only history	The largest reusable prefix in chat apps	Multi-turn threads [1]
Stable tool definitions	The tool block portion of the prefix	Agents and tool users [1]
Count tokens before sending	Guesswork; oversize requests	Any API workflow [1]
Documents on top, query at end	Up to 30 percent response quality on long inputs, less retry cost	Long-document tasks [2]
Quote-then-answer on long documents	Focus, fewer misses	Retrieval and review tasks [2]
Explicit output length and format	Output tokens	Every request [2]
Prompt style matching output style	Formatting overhead in output	Every request [2]
Lower effort or cap output for reasoning	Invisible reasoning tokens	Reasoning models [2]

None of these techniques degrade quality when applied with judgment; they remove waste, not substance. The exception to watch is history trimming, which trades cache reuse and context against token cost, and should be a deliberate decision rather than a habit.

Common Misconceptions

Belief	Reality
"Caching is a premium feature."	It is enabled by default on supported OpenAI models, and reused input is discounted up to 90 percent [1].
"Cache writes are a hidden fee."	A write costs 1.25 times input but pays for itself on the first reuse; write plus one read is 1.35 times versus 2 times uncached [1].
"Summarizing old messages saves money."	It can, but it rewrites the prefix and resets cache reuse from the first changed token [1].
"Removing tools between requests saves tokens."	Removing definitions breaks the prefix; disabling tool use for the request preserves reuse [1].
"Reasoning is free because it is invisible."	Reasoning tokens are billed as output [2].
"Shorter prompts are always better."	Context that changes the answer earns its tokens; indiscriminate trimming causes retries, which cost more than they save [2].

Glossary

Term	Meaning
Prefix	The unchanged beginning of a prompt that caching can reuse [1]
Cache breakpoint	The marker at the end of a cacheable prefix [1]
Cache hit / read	Reusing a stored prefix at the discounted cached-input rate [1]
Cache write	The first processing of a prefix into the cache, at a premium over input rate [1]
Append-only history	Adding new messages without editing earlier ones, preserving the prefix [1]
Adaptive thinking	Model-managed reasoning depth, driven by effort settings and question complexity [2]
Effort setting	The control that raises or lowers how much a model thinks and explores [2]
Tool definition	The name, description, and schema the model sees for each callable tool [1]

References and Helpful Links

Prompt caching (OpenAI API Docs). Cache mechanics, read and write rates, minimum cacheable length, lifetimes, and prefix-stability best practices [1].
Prompting best practices (Anthropic Docs). Long-context structuring, quote grounding, output formatting control, and adaptive thinking guidance [2].
Companion guides: What Is a Token? for what a token is and how billing categories work, and What Is a Prompt? for what belongs in the window in the first place.