Tag

SLM

2 articles tagged with “SLM”

A/B-testing a-star A2A ACR adaptive-thinking agent-observability agent-patterns agent-protocols agent-security agent-skills agents AGENTS.md aggregation ai ai-gateway AI-Search aks alerting Alertmanager algorithms Anthropic Anthropic-API APE api-design api-gateway API-providers api-versioning apim APM app-router app-service appsec architecture Arize arm-templates ASP.NET-Core aspnet-core aspnetcore assertion-functions asserts async audit authentication authorization AutoGen automation azure Azure-APIM azure-content-safety azure-front-door azure-functions Azure-OpenAI azure-policy azure-storage batch-API bdd beam search Bedrock benchmarks best-practices bfs bi-encoder bias-detection bicep binary-search branching bst Bundler cache-components caching cap-theorem capability-tagging Capitalize cascading case-based-reasoning chain-of-thought charts chatbot checkpointing chunking CI ci-cd ci-optimization cicd class-decorator Claude Claude-Code CLAUDE.md clean-architecture cloud cloud-security cloudevents code-review Codex coding-agents compaction comparison compilerOptions compliance compose concurrency concurrent-rendering conditional-types consistency consistent-hashing constrained-generation constraints consumer-driven container-registry containers content-moderation content-safety context context-compression context-engineering context-window contract-testing conversational-agents Copilot cosmos-db cost-allocation cost-control cost-optimization cost-per-task cost-performance CoT coverage cqrs cross-session csharp Cursor cve cypress .d.ts dapper dashboards dast data-fetching databases dataloader datasets ddd decision-framework declaration-files decorators DeepEval DefinitelyTyped dependency-injection dependency-scanning deployment-slots deployments deprecation design-patterns devops dfs dijkstra discriminated-unions distributed-tracing distributive docker Docker-Hub documentation DORA dotnet dotnet-9 drift-detection DSPy durable-execution dynamic-discovery dynamic-programming e2e-testing edge AI ef-core ef-core-9 embeddings engineering environments episodic-memory error-budget evals evaluation event-driven event-grid event-hubs eventing examples experience-replay experimentalDecorators factuality fail-fast faithfulness feature-flags federation few-shot fiber fine-tuning FinOps flaky-tests flex-consumption frontend function-calling garak Gemini Gemma gen-ai-semantic-conventions generics GHCR git GitFlow GitHub github-actions goal-decomposition golden-signals Google-DeepMind GPT gpt-4o GPT-6 GPU grammar graph graphql Groq grounding GroupChat grpc guardrails hallucination hallucination-detection harmful-content harness-engineering hcl health-checks heap Helm hooks hotchocolate HPA http http2 http3 hybridcache iac ICL idempotency identity image-scanning in-context-learning incident-response indexes infer inference infrastructure input-validation integration integration-testing interoperability introsort invariant isolation Istio jailbreak jest joins jotai JSON JSON-mode jsonb jwt k8s key-remapping key-vault kiota knapsack knowledge-base KQL kubernetes KV cache LangChain Langfuse LangGraph LangSmith language latency least-agency legacy-code Linkerd linq Llama llama.cpp LLM LLM-as-judge LLM-monitoring LLM-security load-balancing logging long-running-agents long-term-memory lost-in-the-middle lower-bound managed-identity mapped-types materialized-views MCP memoization memory memory-consolidation merge message-queue messaging meta-prompting metadata metrics microservices microsoft minimal-apis Mistral mocking mocks Model-Context-Protocol model-routing model-selection model-serving moderation module-augmentation module-resolution modules mongodb monitoring monorepo msw MTEB mTLS multi-agent multi-stage-builds multi-tenant multimodal mvcc narrowing network-security nextjs-16 Node16 normalization nosql oauth oauth2 observability offline-evals oidc Omit on-call on-device online-evals opa open-source open-weight OpenAI OpenAI-API openapi OpenTelemetry OPRO orchestration OTEL Outlines output-format output-validation OWASP P99 paas package-exports pact pactflow page-objects PagerDuty parsing Partial partitioning paths pattern-matching performance personal-AI pgvector Phi Phoenix Pick PII PII-filtering pinecone pipeline planning playwright plugins policy-as-code polly postgresql pricing production production-monitoring progressive-disclosure project-references prompt-caching prompt-design prompt-engineering prompt-injection prompt optimisation promptfoo protobuf pulumi python Qdrant quality quality-gates quality-score quantization queues RAG RAG-evaluation RAGAS rate-limiting react react-19 react-compiler react-internals react-query react-server-components realtime reasoning reasoning-effort Record recursion recursive-types red-black red-green-refactor red teaming redis reflection regression-testing release-management replanning resource-limits Responses-API rest reusable-workflows rls rolling-updates runtime-safety saga-pattern sandboxing sast sbom scale schema schema-design schema-validation SDK sdlc search secrets-management security security-testing self-consistency self-hosting Semantic-Kernel semantic-similarity serializable server-actions serverless service-bus service-mesh sharding shortest-path sidecar-proxy signalr SKILL.md SLI SLM SLO slopsquatting snapshot-testing software-design sorting spec-driven-development speculative-decoding spies sql stability state-graph state-management streaming streams strict-mode stride string-types structured-logging structured-output stubs summarisation supertest supply-chain system-design tail-call tanstack-query task-framing TC39 tdd template-literals templates terraform test-architecture test-doubles test-pyramid testcontainers testing testing-patterns testing-strategy threat-modeling token-bucket token-budget token-economics token-limit token-masking token-tracking tokens tokens-per-second tool-calling tool-registry tool-use topological-sort ToT toxicity tracing trajectory transactions tree-of-thought trees trie trunk-based tsconfig TTFT type-guards type-narrowing type-predicates type-safe-routes type-system type-transforms types typescript unit-testing usage-metrics user-feedback utility-types validation valkey values vault vector-db vector-search vector-store vendor-neutral verification version-control versioning Vertex-AI visual-regression vitest vLLM voice-agents vulnerabilities weaviate web-api web-security WebLLM websockets window-functions Windsurf wiremock workflows working-memory workload-identity workspaces xunit yarp zero-shot zero-trust zod zustand

LLM Model Landscape

A router sends narrow requests to a quantised small model on your own servers and hard multi-step requests to a frontier API.

Small Language Models in Production

When and how to use small language models like Phi, Gemma, and Mistral in production — quantization, deployment patterns, and latency-cost trade-offs.

29 JanUpdated 22 Oct 202611 min

Read →

LLM Model Landscape

Edge models: a fine-tuned small model is quantised to a 4-bit GGUF file, downloaded once as a versioned resumable file, and run on the device with llama.cpp, MLX, ONNX Runtime or WebLLM; inputs the device cannot handle fall back to a cloud model only with consent.

Edge Models: Running Small LLMs on Phones, Laptops and Browsers

Intermediate

What fits in device memory, why bandwidth sets generation speed, which runtime to use per platform, and how to ship quantised weights with a cloud fallback that keeps the privacy promise.

09 Oct 20268 min

Read →