Repeated embedding lookups are slow and costly. What pattern helps?
Why A is the answer
Cache-aside with Redis
Check Redis first, fetch on a miss, and store the result with an expiry.
In detail
- Cache-aside stores results after the first lookup.
- Set an expiry on cached values.
Use case
An illustrative scenario, not a real incident.
An app embeds the same questions again and again. Cache-aside stores each result, so the next lookup skips the model.
See it in different ways
flowchart LR
A["Request"] --> B{"In Redis?"}
B -->|Yes| C["Return cached value"]
B -->|No| D["Read the database"]
D --> E["Store in Redis"]
E --> CsequenceDiagram
participant A as App
participant R as Redis
participant S as Source
A->>R: Get the value
alt Hit
R-->>A: Value
else Miss
A->>S: Fetch or compute
A->>R: Store, with an expiry
endsequenceDiagram participant P0 as App participant P1 as Redis participant P2 as Embedding model P0->>P1: Get P1-->>P0: Miss P0->>P2: Compute P0->>P1: Set with expiry
Key point: Cache-aside avoids repeated work.
Why the other options are wrong
- B. Removing the database and keeping everything in application memory
- Application memory is lost, and is not shared between instances.
- C. Calling the embedding model twice for each request
- Calling twice adds cost.
- D. Writing all of the results to a log file and reading it back
- A log file is slow to search.