Many small models should share one endpoint to reduce cost. What fits?
Why C is the answer
A multi-model endpoint
Multi-model endpoints host many models behind one endpoint.
In detail
- A multi-model endpoint shares one endpoint across models.
- Models load on demand, so the first call can be slower.
Use case
An illustrative scenario, not a real incident.
A retailer has a small model per store, and hosts them all behind one multi-model endpoint.
See it in different ways
stateDiagram-v2 [*] --> Traffic Traffic --> RealTime: steady and low latency Traffic --> Serverless: sporadic, cold starts are fine Traffic --> Asynchronous: large payloads, long runs Traffic --> BatchTransform: a whole dataset
mindmap
root(("Multi-model endpoint"))
n1["Models"]
n2["Store A"]
n3["Store B"]
n4["Store C"]Key point: Multi-model endpoints share capacity.
Why the other options are wrong
- A. One batch job for each request
- A batch job per request is unsuited.
- B. A separate account for each model
- Separate accounts add cost.
- D. One large instance for each model
- One instance per model costs more.