Articles tagged "nodejs"

How to cache LLM API calls in Node.js 2026: Redis and break-even math

Redis exact-match caching cuts LLM API spend at scale. Working Node.js middleware, 30-minute TTL trade-off, and when semantic search is worth the overhead.

Sep 9, 2026 · 5 min read tutorialcachingnodejs

How to add per-user LLM cost guardrails in Node.js in 2026

Add per-user LLM budget caps with Redis: soft warnings at 80%, hard stops at 100%, and automatic model downgrade. Full Node.js middleware under 60 lines.

Sep 4, 2026 · 5 min read infracostnodejs

How to build a RAG API in Node.js 2026: pgvector and cited answers

Build a production RAG API in Node.js using pgvector: text ingestion, chunked embeddings, vector search, and source-cited answers in under 80 lines.

Aug 31, 2026 · 5 min read tutorialragnodejs

How to add context management to a Node.js chat API in 2026

Add token counting, sliding window, and summarization compaction to a Node.js chat route to keep conversations coherent without blowing the context budget.

Aug 24, 2026 · 5 min read tutorialnodejscontext-window

How to queue LLM API calls in Node.js: BullMQ patterns

BullMQ turns long-running LLM API calls into background jobs with live progress events, stall detection, and dead-letter recovery in Node.js in 2026.

Aug 14, 2026 · 5 min read infranodejsbullmq

How to manage LLM context windows in production: 3 Node.js patterns

Sliding window, summarization compaction, priority truncation: three Node.js patterns for managing context in long-running LLM conversations.

Aug 5, 2026 · 5 min read infranodejscontext-window

How to add LLM circuit breakers in Node.js in 2026

Per-provider LLM circuit breakers stop cascade failures early. Closed, open, half-open state machine with auto-reset: working Node.js code.

Jul 24, 2026 · 7 min read infrareliabilitynodejs

How to add LLM observability in 2026: cost, latency, quality

Four signals every LLM call should log: tokens, cost, latency, and a quality proxy. An async Node.js middleware that adds under 2ms of overhead.

Jul 17, 2026 · 6 min read infraobservabilitymonitoring

How to test LLM prompts in CI in 2026: a 30-line Node eval script

Catch LLM prompt regressions before users do: a 30-line Node.js eval script that runs golden-set tests on every PR and fails the build when quality drops.

Jul 13, 2026 · 6 min read tutorialtestingci

How to detect stalled LLM streams: watchdog timers in Node.js

Three production patterns for detecting stalled LLM streams in Node.js: chunk watchdogs, wall-clock gates, and partial output buffering, with working code.

Jul 8, 2026 · 5 min read infrastreamingnodejs

Tool calling in Node.js in 2026: 3 guards against runaway loops

Three production guards that stop runaway LLM agent loops in Node.js: max turns, cost budgets, and return value validation with working code.

Jul 3, 2026 · 6 min read tutorialagentstool-calling

Streaming LLM responses in 2026: Next.js, FastAPI, Node

Three production patterns for streaming LLM responses in Node.js, Next.js, and FastAPI, with the mid-stream error catch most tutorials skip.

Jun 24, 2026 · 4 min read tutorialstreamingnodejs

How to route LLM prompts in 2026: cheap first, escalate on fail

Route each prompt to the cheapest model that handles it well. When quality falls short, escalate silently. Here's the pattern with working Node.js code.

Jun 14, 2026 · 4 min read infraroutingcost-optimization

How to set up OpenRouter fallback in Node.js in 2026

Add OpenRouter model fallbacks to a Node.js app: setup, the models array, response.model tracking, and four pitfalls that catch you on week two.

Jun 1, 2026 · 5 min read tutorialopenrouterinfra