What Is an LLM? | AI Wisdom

What happens when you send a prompt

One request, start to finishAn LLM writes one token at a time. Streaming simply shows you this loop as it runs.

sequenceDiagram
  participant A as Your app
  participant T as Tokenizer
  participant M as Model
  A->>T: System prompt, context and question as text
  T->>M: Token IDs
  loop Until a stop token or the length limit
    M->>M: Score every possible next token
    M->>M: Sample one and append it
  end
  M-->>T: Output tokens
  T-->>A: Text

How the next token gets chosenAn illustration, not real probabilities.

flowchart LR
  A["The capital of France is"] --> B["Model scores next tokens"]
  B --> C[" Paris, most likely"]
  B --> D[" a, less likely"]
  B --> E[" located, less likely"]
  C --> F["Temperature decides how strictly to take the top choice"]
  D --> F
  E --> F

Six ideas that explain most behavior

Token

The chunk of text a model reads and writes. It is usually what you are billed for.

Context window

The most tokens a model can consider at once, covering your prompt and its reply.

Embedding

A list of numbers that captures meaning, so similar text can be found by distance.

Attention

How a transformer weighs which earlier tokens matter for the next one.

Inference

Running a trained model to get output. Every API call is an inference.

Hallucination

A confident answer that is wrong. The model predicts plausible text, and does not look anything up.

Task	What to expect
Summarize, rewrite, classify, extract	Good fit. Check with evals.
Answer from your documents	Good fit with RAG. Alone, it does not know them.
Take actions in other systems	Good fit with tools and guardrails.
Exact arithmetic or exact lookups	Use a tool. Text prediction is the wrong mechanism.
Facts after its training data	Needs retrieval or search.
Same output every time	Not guaranteed. Validate and test instead.

See it for yourself

Token Calculator

Paste text and see how it counts and what it costs.

Open the calculator

Prompt Lab

Compare prompting techniques side by side.

Open Prompt Lab

RAG Visualiser

Step through how retrieval feeds a model.

Open the visualiser

Production meaning

In engineering terms, an LLM is a probabilistic language runtime. It turns prompts, retrieved context, tool results, and system instructions into generated outputs that must be evaluated, constrained, observed, and cost-controlled.

Where it fits

LLMs power chat, classification, extraction, code generation, search grounding, RAG, agents, copilots, and workflow automation. They are strongest when paired with retrieval, structured outputs, guardrails, and telemetry.