# How to Use Tokens Efficiently

> Practical techniques for spending fewer tokens: prompt layout that earns cache discounts, append-only history, stable tool definitions, and output habits that cut cost without cutting quality.

**URL:** https://caywork.com/learn/docs/tips-and-guides/how-to-use-tokens-efficiently

## Content

## How to Use Tokens Efficiently

A practical guide to spending fewer tokens without getting worse answers: where tokens go, how caching cuts input cost, and which writing habits shrink output.

**Who this is for:** anyone using AI tools or building on model APIs who wants lower cost and faster responses, not theory. A basic understanding of tokens helps; see the companion guide What Is a Token?
**How it's organized:** where tokens go, then how to spend less on input, then how to spend less on output, then a checklist.

**The one-sentence version:** the cheapest token is the one you never send, the second cheapest is the one the provider caches for you, and the most expensive is the one you make the model generate twice.


## Part 1: Where Tokens Go

Every request spends tokens in several places, and only some of them are the words you type:

| Destination | What lands there | Who controls it |
|---|---|---|
| **System instructions** | Role, rules, constraints the app sets | The developer |
| **Tool definitions** | Names, descriptions, and schemas of every tool the model can call | The developer |
| **Conversation history** | Every earlier message and tool result in the thread | The app |
| **Documents and retrieved text** | Files, web pages, database results pasted or fetched into the prompt | You |
| **Your message** | The actual question or instruction | You |
| **Output** | The visible answer | The model, steered by you |
| **Reasoning** | Invisible thinking before the answer, billed as output [2] | The model, tunable |

Two habits follow directly from this table. Tool definitions and system instructions are often the biggest fixed cost in an agent application, and they sit at the front of the prompt where caching can absorb them. Documents and history are the biggest variable cost, and they respond to editing discipline.

## Part 2: Spend Less on Input

### Put stable content first, changing content last

Providers cache the model's work on the unchanged beginning of your prompt, called the prefix. Reused tokens are billed at the cached-input rate, discounted up to 90 percent versus ordinary input [1]. Anything that changes between requests, such as timestamps, user-specific details, or fresh tool results, belongs at the end of the prompt so it does not break the reusable prefix [1].

A practical layout:

| Position | Content | Why |
|---|---|---|
| First | System instructions, role, shared reference material | Rarely changes; earns cache hits |
| Middle | Tool definitions and schemas | Keep names, descriptions, and ordering stable [1] |
| Next | Conversation history, appended in order | Rewriting or summarizing earlier turns resets reuse [1] |
| Last | Timestamps, user details, the current question | Changes every request; keeps the prefix intact |

### Let caching do the arithmetic

On current OpenAI models, a cache write costs 1.25 times the normal input rate, and a cache read costs 0.1 times [1]. The write pays for itself immediately: writing a prefix once and reusing it once costs 1.35 times its ordinary input cost, against 2 times for processing it twice uncached [1]. Across ten requests that reuse the same prefix, one write plus nine reads costs 2.15 times, against 10 times without caching [1].

| Pattern | Relative input cost |
|---|---|
| Same prefix, no caching, processed twice | 2.0x |
| Cache write plus one reuse | 1.35x |
| Cache write plus nine reuses (ten requests) | 2.15x |
| Same prefix, no caching, processed ten times | 10.0x |

Two details decide whether you actually see these savings. The cacheable prefix has a minimum length (1,024 input tokens on GPT-5.6 and later models [1]), and cache entries expire: on current models the minimum lifetime is 30 minutes after the most recent write or reuse, and each reuse refreshes it at no extra charge [1]. Bursty traffic within that window is what caching rewards.

### Keep the conversation append-only

In multi-turn applications, the growing history is usually the single largest input cost, and it is also the part caching saves most [1]. Append new messages instead of editing old ones. Summarizing, compacting, or truncating history changes the prefix and resets reuse from the first changed token onward [1]. If you must compact, do it deliberately and accept the one-time cache reset rather than trimming a little every turn.

### Keep tool definitions stable

Changing a tool name, description, schema, or ordering changes the prefix and discards reuse from that point [1]. When a request should not use tools, disable them for that request instead of removing the definitions [1]. When only a subset should be callable, restrict which tools are enabled while keeping the supplied list identical [1].

### Cut context you cannot cache

For material that changes every request, the saving comes from editing, not caching. The rules from What Is a Prompt? apply unchanged: include what changes the answer, cut the rest, and put the instruction first, the material after, and the key requirement again at the end.

For long documents, structure matters as much as length. Anthropic's long-context guidance for documents of 20,000 tokens or more: place the documents at the top of the prompt and the query at the end, which improved response quality by up to 30 percent in their tests on complex multi-document inputs [2]. For multi-document tasks, ask the model to quote the relevant passages first and reason from the quotes; this focuses attention on the content that matters and away from the rest of the document [2].

## Part 3: Spend Less on Output

Output tokens usually cost more than input tokens, and reasoning tokens are billed as output even though they never appear in the answer [2]. Four habits keep this line item down.

**1. Ask for the length and shape you want.** Models default to generous responses. State the target: a paragraph, a table, five bullets, under 200 words. Anthropic's formatting guidance also favors telling the model what to do rather than what not to do: "write in flowing prose paragraphs" beats "do not use markdown" [2].

**2. Match your prompt's style to the output you want.** The formatting of the prompt influences the formatting of the response. If the answer comes back over-formatted, removing markdown and bullet structure from your own prompt reduces the volume of markdown in the output [2].

**3. Steer thinking down on easy tasks.** On models with adaptive thinking, the model decides how much to think based on effort settings and question complexity, and simpler questions get direct answers [2]. If a model thinks more often than the task deserves, say so: Anthropic's own sample steering prompt is "Thinking adds latency and should only be used when it will meaningfully improve answer quality. When in doubt, respond directly" [2]. Prefer lowering the effort setting or capping output with the max output parameter as the hard ceiling [2].

**4. Do not regenerate what you can reuse.** Asking for three variants triples output cost. Ask for one good answer, critique it in a follow-up, and let prompt caching absorb the repeated input.

## Part 4: The Checklist

| Technique | What it saves | Where it applies |
|---|---|---|
| Stable content first, dynamic last | Cache hits on the prefix | Every repeated request [1] |
| Append-only history | The largest reusable prefix in chat apps | Multi-turn threads [1] |
| Stable tool definitions | The tool block portion of the prefix | Agents and tool users [1] |
| Count tokens before sending | Guesswork; oversize requests | Any API workflow [1] |
| Documents on top, query at end | Up to 30 percent response quality on long inputs, less retry cost | Long-document tasks [2] |
| Quote-then-answer on long documents | Focus, fewer misses | Retrieval and review tasks [2] |
| Explicit output length and format | Output tokens | Every request [2] |
| Prompt style matching output style | Formatting overhead in output | Every request [2] |
| Lower effort or cap output for reasoning | Invisible reasoning tokens | Reasoning models [2] |

None of these techniques degrade quality when applied with judgment; they remove waste, not substance. The exception to watch is history trimming, which trades cache reuse and context against token cost, and should be a deliberate decision rather than a habit.

## Common Misconceptions

| Belief | Reality |
|---|---|
| "Caching is a premium feature." | It is enabled by default on supported OpenAI models, and reused input is discounted up to 90 percent [1]. |
| "Cache writes are a hidden fee." | A write costs 1.25 times input but pays for itself on the first reuse; write plus one read is 1.35 times versus 2 times uncached [1]. |
| "Summarizing old messages saves money." | It can, but it rewrites the prefix and resets cache reuse from the first changed token [1]. |
| "Removing tools between requests saves tokens." | Removing definitions breaks the prefix; disabling tool use for the request preserves reuse [1]. |
| "Reasoning is free because it is invisible." | Reasoning tokens are billed as output [2]. |
| "Shorter prompts are always better." | Context that changes the answer earns its tokens; indiscriminate trimming causes retries, which cost more than they save [2]. |

## Glossary

| Term | Meaning |
|---|---|
| **Prefix** | The unchanged beginning of a prompt that caching can reuse [1] |
| **Cache breakpoint** | The marker at the end of a cacheable prefix [1] |
| **Cache hit / read** | Reusing a stored prefix at the discounted cached-input rate [1] |
| **Cache write** | The first processing of a prefix into the cache, at a premium over input rate [1] |
| **Append-only history** | Adding new messages without editing earlier ones, preserving the prefix [1] |
| **Adaptive thinking** | Model-managed reasoning depth, driven by effort settings and question complexity [2] |
| **Effort setting** | The control that raises or lowers how much a model thinks and explores [2] |
| **Tool definition** | The name, description, and schema the model sees for each callable tool [1] |

## References and Helpful Links

- [Prompt caching](https://developers.openai.com/api/docs/guides/prompt-caching) (OpenAI API Docs). Cache mechanics, read and write rates, minimum cacheable length, lifetimes, and prefix-stability best practices [1].
- [Prompting best practices](https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/claude-prompting-best-practices) (Anthropic Docs). Long-context structuring, quote grounding, output formatting control, and adaptive thinking guidance [2].
- Companion guides: What Is a Token? for what a token is and how billing categories work, and What Is a Prompt? for what belongs in the window in the first place.