Embeddings — Animate | AI Wisdom

← Back to topic
🎞 ANIMATE · Mode 3 of 5 · +100 XP— Embedding Inference
📥Load Model
🏷️Prefix
🧬Encode
📐L2 Norm
📤Return
🎯 Next: Test →
Step 1 / 5
Speed
Load Once
Load BGE model into memory at startup💡 Analogy
Like booting a translation engine — start it once, then it serves thousands of requests.
Model loading is slow (~2s). Do it once at import — never per request.
Console Output
> model loaded: BAAI/bge-base-en-v1.5 (768d)
●embed.py
Load OnceL4–5
1from sentence_transformers import SentenceTransformer
2import numpy as np
3 
4# 1. Load model once at startup
5model = SentenceTransformer('BAAI/bge-base-en-v1.5')
6 
7def embed(texts: list[str]) -> np.ndarray:
8    # 2. BGE asymmetric: prepend "passage:" or "query:"
9    prefixed = [f"passage: {t}" for t in texts]
10 
11    # 3. Tokenize + forward + mean pool (handled inside)
12    vectors = model.encode(
13        prefixed,
14        batch_size=64,
15        normalize_embeddings=True,  # 4. L2 normalize
16        show_progress_bar=False,
17    )
18 
19    # 5. Return float32 [n, 768]
20    return vectors.astype(np.float32)
21 
22if __name__ == "__main__":
23    chunks = ["Azure Key Vault stores secrets.",
24              "Managed identities authenticate without passwords."]
25    v = embed(chunks)
26    print(v.shape, v.dtype, np.linalg.norm(v[0]))
26 lines
Timeline — 5 Steps
Load Once
Load BGE model into memory at startup💡 Analogy
Like booting a translation engine — start it once, then it serves thousands of requests.
Model loading is slow (~2s). Do it once at import — never per request.
Console Output
> model loaded: BAAI/bge-base-en-v1.5 (768d)

Load BGE model into memory at startup

Load BGE model into memory at startup

Was this page helpful?

Discussion

Load BGE model into memory at startup

Load BGE model into memory at startup

Was this page helpful?

Discussion