Articles tagged "infra"

Prompt caching explained: Anthropic, OpenAI, and Gemini in 2026

Prompt caching cuts LLM API costs up to 90%, but Anthropic, OpenAI, and Gemini implement it differently. Here's how each vendor's billing actually works.

Jun 15, 2026 · 8 min read glosprompt-cachingcost

How to route LLM prompts in 2026: cheap first, escalate on fail

Route each prompt to the cheapest model that handles it well. When quality falls short, escalate silently. Here's the pattern with working Node.js code.

Jun 14, 2026 · 4 min read infraroutingcost-optimization

How to set up OpenRouter fallback in Node.js in 2026

Add OpenRouter model fallbacks to a Node.js app: setup, the models array, response.model tracking, and four pitfalls that catch you on week two.

Jun 1, 2026 · 5 min read tutorialopenrouterinfra

Semantic caching for LLMs: 3 approaches and where each breaks

Semantic caching reduces LLM API spend by 20-70% in production. Here's how embedding-based, prompt-hash, and hybrid caching each break in practice.

May 25, 2026 · 6 min read infracachingcost

How to handle LLM rate limits: 4 production-tested patterns

Four production patterns for LLM rate limits: jitter, token pre-checks, circuit breakers, and provider failover. Backoff alone won't save you in 2026.

May 20, 2026 · 5 min read infrarate-limitsreliability

Prompt caching breaks even at 1.3 requests. Here's the math.

Prompt caching cuts LLM costs 90% on Anthropic and 50% on OpenAI, but only when your workload fits. Here's the exact break-even math per provider.

Apr 27, 2026 · 5 min read infracostprompt-caching

Build an LLM fallback chain in 10 minutes

One model going down shouldn't take your AI feature with it. Here's how to build a fallback chain using LiteLLM, OpenRouter, and LLMTest.

Apr 21, 2026 · 4 min read infrafallbackreliability