Answers get cut off. Which parameter do you check?
Why A is the answer
Maximum output length
Output length limits how long a response can be.
In detail
- Check the maximum output length.
- Raising the limit can raise cost.
Use case
An illustrative scenario, not a real incident.
Replies stop mid-sentence, and the maximum output length turns out to be set too low.
See it in different ways
mindmap
root((Generative AI))
Token
Unit of text
Drives cost
Embedding
Meaning as numbers
Chunking
Focused passages
Hallucination
Confident and false
Temperature
Varietyflowchart LR S0["Reply is cut off"] --> S1["Check maximum length"] --> S2["Raise it"] --> S3["Test again"]
Key point: Check the output limit first.
Why the other options are wrong
- B. Top P for the embeddings
- Top P affects sampling.
- C. The vector database engine
- The vector engine has no effect on reply length.
- D. The IAM role's permissions
- Permissions would give an error.