📖 REFERENCE

AI Engineering Glossary

Concise, opinionated definitions for the terms you'll encounter in production AI systems. Click any term for the full explanation.

64 terms and counting

AI Agent

An LLM-powered system that can plan, use tools, and take actions autonomously to achieve goals.

AI Gateway

An API management layer between your application and LLM providers for routing, caching, and cost control.

Agentic Workflow

A workflow where an LLM autonomously decides steps, uses tools, and iterates until it achieves a goal.

Attention Mechanism

The core operation in transformers that computes relevance scores between all pairs of tokens.

AutoGen

Microsoft's open-source framework for building multi-agent AI applications via conversational message passing.

BM25

A keyword-based ranking function scoring documents by term frequency weighted by inverse document frequency.

Chain-of-Thought (CoT)

A prompting technique that encourages the model to reason step-by-step before giving a final answer.

Chunking

Splitting documents into smaller segments for embedding and retrieval in RAG systems.

Constrained Generation

Techniques that restrict LLM token generation to follow specific formats or formal grammars.

Content Safety

AI-powered classification layers that detect and block harmful or policy-violating content in LLM inputs and outputs.

Context Window

The maximum number of tokens an LLM can process in a single request (input + output combined).

Cosine Similarity

A metric measuring the angle between two vectors, used to compare embedding similarity.

Edge Models (SLMs)

Compact language models optimised to run on-device without cloud connectivity.

Embedding

A dense vector representation of text, images, or other data in a continuous space.

Episodic Memory

Agent memory that stores records of past interactions, enabling recall of previous tasks or conversations.

Few-Shot Learning

Providing a small number of examples in the prompt to teach the model a desired behaviour.

Fine-Tuning

Adapting a pre-trained model on domain-specific data to improve performance for a particular task.

Foundation Model

A large model pre-trained on broad data that can be adapted to many downstream tasks.

Guardrails

Input/output validation layers that prevent LLMs from generating harmful, off-topic, or incorrect responses.

HNSW (Hierarchical Navigable Small World)

The most popular approximate nearest-neighbour algorithm used in vector databases.

Hallucination

When an LLM generates plausible-sounding but factually incorrect or fabricated information.

Hallucination Detection

Techniques to identify when an LLM generates factually incorrect or unsupported claims.

Hybrid Search

Combining vector (semantic) search and keyword (BM25) search for superior retrieval quality.

Inference

Running a trained model to generate predictions or outputs from new inputs.

JSON Mode

An LLM API feature that constrains the model to output syntactically valid JSON.

Knowledge Graph

A graph database representing entities and relationships for structured, multi-hop knowledge retrieval.

LLM Evaluation (Evals)

Systematic measurement of LLM output quality using metrics, benchmarks, and automated judges.

LLM Streaming

Returning LLM tokens incrementally as they are generated instead of waiting for the full response.

LangGraph

A library for building stateful, graph-based agent workflows with persistence and human-in-the-loop support.

LangSmith

LangChain's observability platform for tracing, evaluating, and managing LLM applications.

LoRA (Low-Rank Adaptation)

A parameter-efficient fine-tuning technique that trains small adapter matrices instead of full model weights.

Long-term Memory

External persistent memory that agents can query and update across multiple sessions.

MCP (Model Context Protocol)

A standard protocol enabling AI agents to connect with tools, resources, and prompts across any provider.

Meta-Prompting

Using an LLM to generate, optimise, or select prompts for another LLM task.

Model Serving

Infrastructure for deploying and running ML models in production with APIs.

Multi-Agent System

An architecture where multiple specialised AI agents collaborate to solve complex tasks.

Offline Evaluation

Systematic evaluation of model quality using curated datasets before or outside production.

Online Evaluation

Real-time evaluation of LLM outputs in production using automated judges or user feedback signals.

OpenTelemetry for AI

Using OpenTelemetry semantic conventions to trace and instrument LLM systems for observability.

Phoenix (Arize)

An open-source LLM observability platform for trace analysis, RAG debugging, and embedding drift detection.

Planning Loop

The core agentic cycle where an LLM plans actions, executes them, observes results, and re-plans iteratively.

Prompt Engineering

The practice of designing and optimising input prompts to elicit desired outputs from LLMs.

Prompt Injection

An attack where malicious input manipulates an LLM into ignoring its instructions or leaking system prompts.

QLoRA (Quantised LoRA)

Combines 4-bit quantisation with LoRA to enable fine-tuning of large models on consumer GPUs.

Quantisation

Reducing model precision (e.g., 32-bit to 4-bit) to decrease memory usage and increase inference speed.

RAG (Retrieval-Augmented Generation)

A pattern that grounds LLM responses in retrieved external documents to reduce hallucination.

RLHF (Reinforcement Learning from Human Feedback)

A training technique that aligns LLM behaviour with human values using human preference annotations.

ReAct Prompting

A prompting paradigm that interleaves Reasoning and Acting steps, enabling LLMs to think before invoking tools.

Reciprocal Rank Fusion (RRF)

A rank-fusion algorithm that merges multiple ranked result lists into a single reliable combined ranking.

Red Teaming (LLMs)

Adversarial testing of LLM systems to systematically discover safety, security, and reliability vulnerabilities.

Reranking

Applying a cross-encoder model to rescore retrieved documents, improving RAG retrieval precision.

Semantic Kernel

Microsoft's open-source SDK for building AI-powered applications in C#, Python, and Java.

Semantic Search

Search based on meaning rather than keyword matching, using vector similarity of embeddings.

Short-term Memory (In-Context)

In-context information available to an agent within one session — limited by the model's context window.

Structured Output

Techniques for constraining LLM responses to follow a specific schema (JSON, XML, etc.).

System Prompt

Instructions provided to an LLM that define its behaviour, personality, and constraints.

Token

The smallest unit of text processed by an LLM — roughly 3/4 of a word in English.

Token Economics

The discipline of managing LLM token consumption to control costs and stay within context limits.

Tool Registry

A catalog of tools an AI agent can discover and invoke, with metadata for selection and usage.

Tool Use (Function Calling)

Enabling LLMs to invoke external functions, APIs, or tools based on user intent.

Transformer

The neural network architecture behind modern LLMs, based on self-attention mechanisms.

Tree of Thought (ToT)

A prompting strategy that explores multiple reasoning branches in parallel to find the best path to a solution.

Vector Database

A database optimised for storing and querying high-dimensional embedding vectors.

vLLM

A high-throughput, memory-efficient LLM inference engine using PagedAttention for GPU KV-cache management.