AI Engineering Glossary

A workflow where an LLM autonomously decides steps, uses tools, and iterates until it achieves a goal.

An LLM-powered system that can plan, use tools, and take actions autonomously to achieve goals.

An API management layer between your application and LLM providers for routing, caching, and cost control.

The core operation in transformers that computes relevance scores between all pairs of tokens.

Microsoft Research's original open-source multi-agent framework via conversational message passing — now in maintenance mode, superseded by Microsoft Agent Framework.

A keyword-based ranking function scoring documents by term frequency weighted by inverse document frequency.

A prompting technique that encourages the model to reason step-by-step before giving a final answer.

Splitting documents into smaller segments for embedding and retrieval in RAG systems.

Techniques that restrict LLM token generation to follow specific formats or formal grammars.

AI-powered classification layers that detect and block harmful or policy-violating content in LLM inputs and outputs.

The maximum number of tokens an LLM can process in a single request (input + output combined).

A metric measuring the angle between two vectors, used to compare embedding similarity.

Compact language models optimised to run on-device without cloud connectivity.

A dense vector representation of text, images, or other data in a continuous space.

Agent memory that stores records of past interactions, enabling recall of previous tasks or conversations.

Providing a small number of examples in the prompt to teach the model a desired behaviour.

Adapting a pre-trained model on domain-specific data to improve performance for a particular task.

A large model pre-trained on broad data that can be adapted to many downstream tasks.

Input/output validation layers that prevent LLMs from generating harmful, off-topic, or incorrect responses.

When an LLM generates plausible-sounding but factually incorrect or fabricated information.

Techniques to identify when an LLM generates factually incorrect or unsupported claims.

The most popular approximate nearest-neighbour algorithm used in vector databases.

Combining vector (semantic) search and keyword (BM25) search for superior retrieval quality.

Running a trained model to generate predictions or outputs from new inputs.

An LLM API feature that constrains the model to output syntactically valid JSON.

A graph database representing entities and relationships for structured, multi-hop knowledge retrieval.

A library for building stateful, graph-based agent workflows with persistence and human-in-the-loop support.

LangChain's observability platform for tracing, evaluating, and managing LLM applications.

Systematic measurement of LLM output quality using metrics, benchmarks, and automated judges.

Returning LLM tokens incrementally as they are generated instead of waiting for the full response.

External persistent memory that agents can query and update across multiple sessions.

A parameter-efficient fine-tuning technique that trains small adapter matrices instead of full model weights.

A standard protocol enabling AI agents to connect with tools, resources, and prompts across any provider.

Using an LLM to generate, optimise, or select prompts for another LLM task.

Infrastructure for deploying and running ML models in production with APIs.

An architecture where multiple specialised AI agents collaborate to solve complex tasks.

Systematic evaluation of model quality using curated datasets before or outside production.

Real-time evaluation of LLM outputs in production using automated judges or user feedback signals.

Using OpenTelemetry semantic conventions to trace and instrument LLM systems for observability.

An open-source LLM observability platform for trace analysis, RAG debugging, and embedding drift detection.

The core agentic cycle where an LLM plans actions, executes them, observes results, and re-plans iteratively.

The practice of designing and optimising input prompts to elicit desired outputs from LLMs.

An attack where malicious input manipulates an LLM into ignoring its instructions or leaking system prompts.

Combines 4-bit quantisation with LoRA to enable fine-tuning of large models on consumer GPUs.

Reducing model precision (e.g., 32-bit to 4-bit) to decrease memory usage and increase inference speed.

A pattern that grounds LLM responses in retrieved external documents to reduce hallucination.

A prompting paradigm that interleaves Reasoning and Acting steps, enabling LLMs to think before invoking tools.

A rank-fusion algorithm that merges multiple ranked result lists into a single reliable combined ranking.

Adversarial testing of LLM systems to systematically discover safety, security, and reliability vulnerabilities.

Applying a cross-encoder model to rescore retrieved documents, improving RAG retrieval precision.

A training technique that aligns LLM behaviour with human values using human preference annotations.

Microsoft's plugin-based SDK for building AI-powered applications in C#, Python, and Java — now in maintenance mode, superseded by Microsoft Agent Framework.

Search based on meaning rather than keyword matching, using vector similarity of embeddings.

In-context information available to an agent within one session — limited by the model's context window.

Techniques for constraining LLM responses to follow a specific schema (JSON, XML, etc.).

Instructions provided to an LLM that define its behaviour, personality, and constraints.

The smallest unit of text processed by an LLM — roughly 3/4 of a word in English.

The discipline of managing LLM token consumption to control costs and stay within context limits.

A catalog of tools an AI agent can discover and invoke, with metadata for selection and usage.

Enabling LLMs to invoke external functions, APIs, or tools based on user intent.

The neural network architecture behind modern LLMs, based on self-attention mechanisms.

A prompting strategy that explores multiple reasoning branches in parallel to find the best path to a solution.

A database optimised for storing and querying high-dimensional embedding vectors.

A high-throughput, memory-efficient LLM inference engine using PagedAttention for GPU KV-cache management.