What Is a Prompt?
A plain-language explainer covering what a prompt is, what the model actually does with it, and why wording matters.
Who this is for: anyone using AI tools who wants to understand the mechanism, not just the tips. No technical background needed. How it's organized: definition, then the mechanics, then anatomy, then the parts most people never see.
The one-sentence version: a prompt is the text you send to an AI system to request a response, and it is the only control you have over what comes back.
Part 1: The Basics
What a prompt is
A prompt can be a single question, a detailed instruction, or a long message carrying background information and examples. The model reads it and generates a response.
Prompts are the primary way people interact with large language models. Clarity in equals quality out: a vague prompt tends to produce a generic answer, and a specific prompt produces a focused, useful one.
What actually happens when you press send
Understanding four mechanics explains most of what feels mysterious about AI tools.
1. The model never reads your words. Every word, space, and punctuation mark is converted into numerical tokens before the model touches it [5]. Tokens are the smallest units a model processes and may be whole words, fragments of words, or single characters, depending on the language and tokenizer [1]. In typical English, one token runs about four characters, so 100 tokens is roughly 75 words [1].
2. It predicts, it does not look things up. During training the model learned patterns, relationships between words, and common ways information is structured. Given your prompt, it produces a probability distribution over possible next tokens and samples from it, repeatedly [5]. There is no retrieval step and no reasoning about your situation unless you describe it.
3. Everything lives in one flat window. The context window is everything the model can see at once: the system prompt, the conversation so far, your current message, any retrieved documents, any tool output [5]. The model reprocesses that whole window on every response, and nothing persists outside it [5].
4. The model has no memory. It only looks like it remembers the conversation. In reality, each new prompt you send resends all the earlier prompts and replies as context [1]. When a conversation exceeds the window limit, the oldest parts are dropped and simply ignored [1]. The model is not forgetting; there was never memory in the first place, and when early instructions scroll off the top, it genuinely no longer sees them [5].
Two things that follow from this. Long conversations cost more and run slower, because the model reprocesses more data every turn [1]. And if a constraint you set 40 messages ago stops being honored, restate it rather than assuming it still applies [5].
Part 2: What a Prompt Can Contain
A well-written prompt often combines several elements:
| Element | What it does | Example |
|---|---|---|
| Instruction | The action you want performed | "Summarize...", "Compare...", "Explain..." |
| Context | Background the model needs | Audience, purpose, prior decisions |
| Examples | Sample inputs and outputs showing the style you want | One or two before/after pairs |
| Constraints | Limits on length, tone, audience, or topics | "Under 200 words, no pricing claims" |
| Output format | How the result should be presented | Bullets, table, JSON, headings |
| Input data | The material to work on | A pasted article, notes, a dataset |
You do not need all of these every time. Start with a clear instruction, then add what your task actually requires.
Where to put the important parts
Instructions placed near the end of the context tend to carry more influence, because they sit closer to where the model is generating output [5]. A carefully written instruction at the very top can lose out to a three-word instruction at the bottom [5]. For long prompts, a practical habit is to state the instruction first, paste the material, then restate the key requirement at the end.
More context is not automatically better
The most common failure is not too little context but too much: stale history, irrelevant material, and documents nobody asked for, all diluting the signal [2]. Every irrelevant token is one the model has to read, weigh, and discard, and a large enough pile of near-misses can outweigh the one passage that actually answers the question [2]. Include what changes the answer. Cut the rest.
Part 3: The Parts You Do Not See
System prompts vs. user prompts
Most AI tools involve at least two layers of input [5]:
| System prompt | User prompt | |
|---|---|---|
| Written by | The tool or developer [1] | You |
| When it applies | Injected before every conversation [5] | Each turn |
| What it sets | Role, tone, rules, constraints [5] | The specific request |
| Visible to you | Usually not | Yes |
A system prompt is a set of rules added to every chat. Tell a model "always answer in haiku form" at that level, and every answer becomes a haiku [1]. This is why a tool behaves consistently across very different requests, and why two products built on the same underlying model can feel completely different.
Worth knowing: both prompts sit in the same context window and are processed identically [5]. The model treats system instructions as more authoritative because alignment training taught it to, not because the architecture enforces it [5]. That distinction explains prompt injection: since the window has no trusted and untrusted separation, text hidden inside a document or web page the model reads can be treated as an instruction [5].
What else gets assembled
In real applications, the prompt reaching the model is rarely something a person typed. The app stitches together system instructions, retrieved documents, conversation history, tool definitions, and stored memories into one long sequence [3]. A compact way to describe it: context = Assemble(instructions, knowledge, tools, memory, state, query) [3].
This is why the field has grown a second name. Prompt engineering asks how to phrase the instruction. Context engineering asks what the model actually needs to know right now, and manages everything competing for the same limited space [2].
Settings that change the output
Some behavior comes from dials, not wording. In chat apps these are usually fixed, but they explain a lot [4]:
| Setting | Effect |
|---|---|
| Temperature | Randomness. Low (0 to 0.3) gives consistent, reproducible output; high (0.7 to 1.2) gives varied, creative output [5] |
| Top P | Nucleus sampling; another way to control determinism. Adjust this or temperature, not both [4] |
| Max length | Caps response length, which controls cost and prevents rambling [4] |
| Stop sequences | Strings that halt generation [4] |
| Frequency / presence penalty | Reduce repetition of words and phrases [4] |
This answers a common question: why does the same prompt give different answers? Unless temperature is set to 0, the model samples rather than always picking the most probable token [5]. Most consumer products run between 0.5 and 1.0 to balance consistency with natural-sounding text [5]. That variation is a design choice, not a bug.
Part 4: Why Prompts Matter
A prompt is the only control you have over what the model does. Clear instructions reduce guesswork, which means fewer rewrites and more accurate results.
Three things follow directly from the mechanics above:
- Anything you do not state, the model does not know. It has no awareness of your situation beyond what is in the window [5].
- Anything outside the window does not exist. Not the earlier conversation that scrolled off, not the file you did not paste [1][5].
- Wording is a real lever, but only within a good context. A perfectly phrased instruction surrounded by irrelevant material still fails [2].
Writing prompts well is not a special skill. It is a practical habit that improves every AI tool you use.
Common Misconceptions
| Belief | Reality |
|---|---|
| "The model remembers our conversation." | Each turn resends the whole history as context [1]. There is no memory outside the window [5]. |
| "It looks up the answer." | It predicts likely next tokens from learned patterns [5]. |
| "It understands what I mean." | It has no awareness of your situation unless you describe it [5]. |
| "The same prompt gives the same answer." | Above temperature 0, sampling produces variation [5]. |
| "More context is always better." | Past a point, extra context dilutes the signal and adds cost [2]. |
| "Bigger context windows solved this." | A larger window removes the hard ceiling, not the noise or the cost [2]. |
| "The system prompt is enforced." | It is authoritative by training convention, not architecture [5]. |
Glossary
| Term | Meaning |
|---|---|
| Prompt | The text you send to request a response |
| Token | The unit the model processes; roughly four characters of English [1] |
| Tokenization | Breaking text into tokens before processing [1] |
| Context window | Everything the model can see at once; finite, and older content scrolls off [5] |
| System prompt | Developer instructions applied to every conversation [1] |
| User prompt | What you type [1] |
| Prompt template | A reusable prompt pattern with placeholders filled in per use [5] |
| Temperature | Output randomness; low is consistent, high is varied [5] |
| Completion | The model's generated response |
| Zero-shot | Asking with no examples |
| Few-shot | Asking with a small number of examples |
| Context engineering | Deciding what fills the window, not just how the instruction is worded [2] |
| RAG | Retrieval-augmented generation; fetching documents and adding them to the prompt [3] |
| Prompt injection | Hidden instructions inside content the model reads [5] |
References and Helpful Links
- Getting started with prompts for text-based generative AI tools (Harvard University Information Technology). A practical introduction with concrete before-and-after examples.
- Effective Prompts for AI: The Essentials (MIT Sloan Educational Technology). Context, specificity, and common prompt types.
- Key concepts and considerations in generative AI (Microsoft Learn). Tokenization, context windows, how history is resent each turn, and system vs. user prompts [1].
- How LLMs Process Prompts. Tokenization, the flat context window, instruction positioning, the system/user trust hierarchy, and temperature [5].
- LLM Settings (Prompt Engineering Guide). Temperature, Top P, max length, stop sequences, and penalties [4].
- Context Assembly: Building the Prompt the Model Sees (Redis). How production apps assemble the prompt the model actually receives [3].
- Context Engineering for AI Agents. Why too much context is the more common failure, and how to budget the window [2].
- Prompt Engineering Guide. Maintained collection of prompting techniques and examples.