You must check how a chatbot behaves as usage grows. What do you measure?
Why C is the answer
Latency and throughput under rising concurrent load
Scalability shows up under load, not in static file sizes.
In detail
- Scalability shows up under load.
- Test with realistic prompts.
Use case
An illustrative scenario, not a real incident.
A team raises concurrent users until latency passes its target, which shows the real limit.
See it in different ways
flowchart LR A["Users"] --> B["API"] B --> C["Batched GPU inference"] C --> D["Streamed tokens"] B --> E["Latency and error monitoring"]
mindmap
root((LLM service))
Hardware
GPU memory
Throughput
Batching
Experience
Streaming
Conversation history
Packaging
NIMflowchart LR S0["Start low"] --> S1["Add concurrent users"] --> S2["Watch latency and throughput"] --> S3["Find the limit"]
Key point: Scalability shows under load.
Why the other options are wrong
- A. The file size of the model weights on disk
- File size is fixed.
- B. The length of the system prompt in characters
- Prompt length is not load.
- D. The number of parameters in the embedding model
- Parameters are fixed.