A leader asks how to judge whether a gen AI pilot worked. What should you define?
Why C is the answer
Key metrics tied to the business goal, set before launch
Impact is measured against business outcomes, so agree on the metrics before you begin.
In detail
- Define metrics tied to the business goal before launch.
- Agree the target with the people who will judge it.
Use case
An illustrative scenario, not a real incident.
Before a pilot, a team agrees that success means answering tickets 20% faster, and then measures that.
See it in different ways
flowchart LR A["Business need"] --> B["Choose a solution"] B --> C["Integrate"] C --> D["Measure impact"] D --> E["Scale or adjust"]
sequenceDiagram participant L as Leader participant T as Team participant B as Business L->>T: Need and constraints T->>L: Metrics, set before launch T->>B: Pilot B-->>L: Measured result
flowchart LR S0["Business goal"] --> S1["Agree metrics"] --> S2["Run pilot"] --> S3["Compare results"]
Key point: Set business metrics before launch.
Why the other options are wrong
- A. The size of the context window in the chosen model
- A context window is a technical detail.
- B. The list of models the team considered and tested
- A list of models shows no outcome.
- D. The number of prompts the engineers wrote during the pilot
- Prompt count measures activity.