Tag

caching

4 articles tagged with “caching”

.d.ts A/B-testing ACR AI AI-Search APE API-gateway API-providers APM ASP.NET-Core Alertmanager Anthropic Anthropic-API Arize AutoGen Azure-APIM Azure-OpenAI Bedrock Bundler CI Capitalize Claude CoT Copilot Cosmos-DB Cursor DORA DSPy DeepEval DefinitelyTyped Docker-Hub Event-Grid FinOps GHCR GPT GPT-6 GPU Gemini Gemma GitFlow GitHub Google-DeepMind Groq GroupChat HPA Helm ICL Istio JSON JSON-mode KV cache Kubernetes LLM LLM-as-judge LLM-monitoring LLM-security LangChain LangGraph LangSmith Linkerd Llama MCP MTEB Mistral Model-Context-Protocol Node16 OIDC OPRO OTEL OWASP Omit OpenAI OpenAI-API OpenTelemetry Outlines P99 PII PII-filtering PagerDuty Partial Phi Phoenix Pick Pinecone Promptfoo Qdrant RAG RAG-evaluation RAGAS ReAct Record Responses-API SDK SLI SLM SLO Semantic-Kernel Service-Bus TC39 TTFT ToT Vertex-AI WebLLM Windsurf a-star adaptive-thinking agents aggregation ai ai-gateway aks alerting algorithms api-design api-gateway api-versioning apim app-router app-service appsec architecture arm-templates aspnet-core aspnetcore assertion-functions asserts async audit authentication authorization automation azure azure-content-safety azure-front-door azure-functions azure-openai azure-policy azure-storage batch-API bdd beam search benchmarks best-practices bfs bi-encoder bias-detection bicep binary-search branching bst cache-components caching cap-theorem capability-tagging cascading case-based-reasoning chain-of-thought charts chatbot checkpointing chunking ci-cd ci-optimization cicd class-decorator clean-architecture cloud cloud-security cloudevents code-review coding-agents comparison compilerOptions compliance compose concurrency concurrent-rendering conditional-types consistency consistent-hashing constrained-generation constraints consumer-driven container-registry containers content-moderation content-safety context context-compression context-window contract-testing conversational-agents cosmos-db cost-allocation cost-control cost-optimization cost-per-task cost-performance coverage cqrs cross-session csharp cve cypress dapper dashboards dast data-fetching databases dataloader datasets ddd decision-framework declaration-files decorators dependency-injection dependency-scanning deployment-slots deployments deprecation design-patterns devops dfs dijkstra discriminated-unions distributed-tracing distributive docker documentation dotnet dotnet-9 drift-detection dynamic-discovery dynamic-programming e2e-testing edge AI ef-core ef-core-9 embeddings engineering environments episodic-memory error-budget evals evaluation event-driven event-grid event-hubs eventing examples experience-replay experimentalDecorators factuality fail-fast faithfulness feature-flags federation few-shot fiber fine-tuning flaky-tests flex-consumption frontend function-calling garak gen-ai-semantic-conventions generics git github-actions goal-decomposition golden-signals gpt-4o grammar graph graphql grounding grpc guardrails hallucination hallucination-detection harmful-content hcl health-checks heap hooks hotchocolate http http2 http3 hybridcache iac idempotency identity image-scanning in-context-learning incident-response indexes infer inference infrastructure input-validation integration integration-testing introsort invariant isolation jailbreak jest joins jotai jsonb jwt k8s key-remapping key-vault kiota knapsack knowledge-base kubernetes language latency legacy-code linq llama.cpp load-balancing long-term-memory lost-in-the-middle lower-bound mTLS managed-identity mapped-types materialized-views memoization memory memory-consolidation merge message-queue messaging meta-prompting metadata metrics microservices microsoft minimal-apis mocking mocks model-routing model-selection moderation module-augmentation module-resolution modules mongodb monitoring msw multi-agent multi-stage-builds multi-tenant multimodal mvcc narrowing network-security nextjs-16 normalization nosql oauth oauth2 observability offline-evals oidc on-call on-device online-evals opa open-source open-weight openapi orchestration output-format output-validation owasp paas package-exports pact pactflow page-objects parsing partitioning paths pattern-matching performance personal-AI pgvector pinecone pipeline planning playwright plugins policy-as-code polly postgresql pricing production production-monitoring project-references prompt injection prompt optimisation prompt-caching prompt-design prompt-engineering prompt-injection promptfoo protobuf pulumi python quality quality-gates quality-score quantization queues rag rate-limiting react react-19 react-compiler react-internals react-query react-server-components realtime reasoning reasoning-effort recursion recursive-types red teaming red-black red-green-refactor red-teaming redis regression-testing release-management replanning resource-limits rest reusable-workflows rls rolling-updates runtime-safety saga-pattern sast sbom scale schema schema-design schema-validation sdlc search secrets-management security security-testing self-consistency self-hosting semantic-similarity serializable server-actions serverless service-bus service-mesh sharding shortest-path sidecar-proxy signalr snapshot-testing software-design sorting spies sql stability state-graph state-management streaming streams strict-mode stride string-types structured-output stubs summarisation supertest supply-chain system-design tail-call tanstack-query task-framing tdd template-literals templates terraform test-architecture test-doubles test-pyramid testcontainers testing testing-patterns testing-strategy threat-modeling token-bucket token-budget token-economics token-limit token-masking token-tracking tokens tokens-per-second tool-calling tool-registry tool-use topological-sort toxicity tracing trajectory transactions tree-of-thought trees trie trunk-based tsconfig type-guards type-narrowing type-predicates type-safe-routes type-system type-transforms types typescript unit-testing usage-metrics user-feedback utility-types vLLM validation valkey values vault vector-db vector-search vector-store vendor-neutral version-control versioning visual-regression vitest vulnerabilities weaviate web-api web-security websockets window-functions wiremock working-memory workload-identity workspaces xunit yarp zero-shot zero-trust zod zustand

AI Engineering

Twelve microservices call Azure OpenAI through one AI gateway on Azure API Management that applies token-based rate limits, semantic caching and per-team cost tracking, and fails over between regional deployments.

The AI Gateway Pattern: Why Every Production LLM Needs One

Intermediate

API Management, rate limiting, semantic caching, and cost control with Azure APIM as an AI Gateway.

21 JanUpdated 06 Aug 202610 min

Read →

AI Engineering

Where LLM tokens go: system prompt, retrieved context, history and output are measured per feature, then cut with prompt caching, compression and model routing, and checked so savings do not hurt quality.

Token Economics: Understanding and Optimizing LLM Costs

A practical guide to understanding token pricing, measuring real costs, and implementing optimization strategies — caching, prompt compression, model routing.

12 FebUpdated 18 Aug 202610 min

Read →

System Design

A multi-layer cache: requests check an in-process cache, then Redis, then the database on a miss; writes invalidate keys explicitly, TTL is only a safety net, and request coalescing prevents stampedes.

Caching in 2026: Cache-Aside, HybridCache, and the Four Failure Modes

Intermediate

Cache-aside vs write-through, .NET 9 HybridCache (L1+L2 with stampede protection), TTL design with jitter, and the four ways caches actually fail in production.

09 Dec 2025Updated 15 Aug 202613 min

Read →

Databases

Redis in a .NET app: HybridCache keeps an in-process L1 in front of Redis L2 with stampede protection and TTL jitter, atomic counters rate-limit, SET NX locks use compare-and-delete, and Streams with consumer groups carry events.

Redis 8 / Valkey in 2026: HybridCache, Streams, and Distributed Locks

Intermediate

Redis 8 SSPL/RSALv2 + Valkey LF fork, Azure Cache Enterprise, .NET 9 HybridCache (L1+L2 + stampede protection), atomic rate limit, SET NX EX + Lua compare-and-delete locks, Streams + consumer groups, sorted-set leaderboards, TTL jitter.

10 JanUpdated 02 Oct 202614 min

Read →