How to retry LLM API calls in Node.js in 2026: three-layer pattern
Client backoff with jitter, proxy fallback, and dead-letter queue: the three Node.js layers that stop cascading LLM failures in production.
LLMTest Blog
Real-world guides on cutting LLM API costs, writing prompts that hold up, and comparing models — for solo developers, vibe coders, and indie hackers.
Client backoff with jitter, proxy fallback, and dead-letter queue: the three Node.js layers that stop cascading LLM failures in production.
Prompt injection hijacks your LLM via user input, retrieved docs, and tool results. Three attack surfaces, and the defenses that actually work in production.
Per-user LLM cost at 1,000 daily users in 2026: token math across chat, extraction, and RAG at two model tiers, with break-even math for solo devs.
An in-process LLM request queue for Node.js: concurrency cap, per-user fairness, and backpressure under 60 lines. No Redis or BullMQ needed in 2026.
Redis exact-match caching cuts LLM API spend at scale. Working Node.js middleware, 30-minute TTL trade-off, and when semantic search is worth the overhead.
An AI agent is an LLM in a tool-calling loop. How loops work, why they stall or run away, and the 3 architecture decisions that matter in production.
Add per-user LLM budget caps with Redis: soft warnings at 80%, hard stops at 100%, and automatic model downgrade. Full Node.js middleware under 60 lines.
Fine-tuning LLMs in 2026 costs $3-$30 in GPU time for open-weight models, or a 50% inference premium on GPT-4.1. Break-even math across three real scenarios.
Build a production RAG API in Node.js using pgvector: text ingestion, chunked embeddings, vector search, and source-cited answers in under 80 lines.
Why LLMs hallucinate, the three error types, prompting patterns that reduce them, and the one production eval that catches failures before your users see them.
JSON mode vs structured output: how constrained decoding works, when 3% failure rates become a real cost, and which enforcement approach your pipeline needs.
Add token counting, sliding window, and summarization compaction to a Node.js chat route to keep conversations coherent without blowing the context budget.
Per-session cost math for 1,000 daily LLM chat sessions at three model tiers. Why stateless APIs compound costs and when sliding-window compaction pays off.
Five LLMs ranked on unit test quality, edge case coverage, and cost per 1,000 tests in 2026. Claude Haiku 4.5 is the value pick for solo devs.
Best LLMs for business email writing in 2026: ranked by tone control, length discipline, and per-email API cost for solopreneurs.
BullMQ turns long-running LLM API calls into background jobs with live progress events, stall detection, and dead-letter recovery in Node.js in 2026.
GPT-4o-mini, Gemini 2.5 Flash, DeepSeek V4-Flash, and Haiku 4.5 ranked on schema errors, retry costs, and multi-turn reliability for function calling in 2026.
Batch APIs from OpenAI, Anthropic, and Google give 50% off for async workloads. Here's the break-even math for three real use cases in 2026.
The best LLMs for multilingual apps in 2026: how frontier and budget models handle French, German, Japanese, and Arabic, with the token-cost math.
Sliding window, summarization compaction, priority truncation: three Node.js patterns for managing context in long-running LLM conversations.
A system prompt is the hidden instruction block that shapes every LLM response. Here's how it works, what it costs, and the production pitfalls to avoid.
LLM agent costs in production 2026: three workload models with token math and USD totals per 1,000 runs, from a 3-call support bot to an 8-step coding loop.
Three tiers of LLM JSON output validation in 2026: schema (Zod), semantic checks, and retry with reprompt. Includes cost math at 1,000 requests per day.
Which LLMs hit 99%+ JSON schema compliance natively in 2026, which need format enforcement middleware, and the real cost of retries at scale.
Per-provider LLM circuit breakers stop cascade failures early. Closed, open, half-open state machine with auto-reset: working Node.js code.
Wire Anthropic and OpenAI into one Next.js API route: primary call, automatic provider switch on 429 or 5xx, and the real latency cost of switching mid-request.
Fine-tuning trains a model on your data to shape behavior and output format. Here's when it beats prompting, what it costs in 2026, and when to skip it.
Four signals every LLM call should log: tokens, cost, latency, and a quality proxy. An async Node.js middleware that adds under 2ms of overhead.
Embedding 1M docs costs $10 with text-embedding-3-small or $67 with the large model. BGE-M3 self-hosted breaks even at 27B tokens per month. The math.
Catch LLM prompt regressions before users do: a 30-line Node.js eval script that runs golden-set tests on every PR and fails the build when quality drops.
Function calling lets LLMs trigger your code via JSON schemas. How the loop works, when to use it over plain prompting, and the pitfalls most builders hit.
Three production patterns for detecting stalled LLM streams in Node.js: chunk watchdogs, wall-clock gates, and partial output buffering, with working code.
Embeddings convert text into vectors that capture meaning. Here's how cosine similarity works and how to pick the right model for your RAG pipeline.
Three production guards that stop runaway LLM agent loops in Node.js: max turns, cost budgets, and return value validation with working code.
Per-feature LLM cost math for 2026: four workload patterns with token budgets, caching discounts, and total USD per 1,000 requests across model tiers.
Work backward from your margin target to a token budget: break-even math for AI features in 2026, with three concrete worked examples in USD.
Which LLMs meet HIPAA, SOC 2, and data-residency requirements in 2026? Four paths compared: ZDR API, SOC 2 cloud, EU data residency, and full self-hosting.
Three production patterns for streaming LLM responses in Node.js, Next.js, and FastAPI, with the mid-stream error catch most tutorials skip.
Six models now claim 1M token contexts. MRCR v2 shows GPT-5.5 at 74% recall at 1M tokens; Gemini 3.5 Flash drops to 26%. Ranked by what they actually use.
LiteLLM has multiple routing strategies. These 4 matter in production: weighted, fallback, semantic, and budget-capped, each with production-ready YAML.
Five LLMs under $1/M input tokens ranked by throughput and quality in 2026. Gemini 2.5 Flash leads on tokens per second; DeepSeek V4 wins on output cost.
Prompt caching cuts LLM API costs up to 90%, but Anthropic, OpenAI, and Gemini implement it differently. Here's how each vendor's billing actually works.
Route each prompt to the cheapest model that handles it well. When quality falls short, escalate silently. Here's the pattern with working Node.js code.
GPT-5 costs $2.13/1k for chat, $4.50 for extraction, $11.25 for summarization. Here's the exact per-token math and where batch saves you 50%.
Claude Fable 5 review with real benchmark data: 5-3 over Opus 4.8, 3-0 vs GPT-5.5 on 12 coding and reasoning prompts. Includes subscription break-even math.
We benchmarked 4 LLMs on 6 real OCR tasks: receipts, invoices, prescriptions. GPT-5.5 wins 10/18 matchups; Haiku 4.5 crumbles on JSON formatting.
DeepSeek V3 wins 10 of 15 coding and reasoning tasks against Llama 4 Maverick. Full benchmark results, three judge excerpts, and when to pick each.
We ran 4 models through 6 RAG-specific prompts testing faithfulness, citation accuracy, and I-don't-know honesty. Opus 4.8 takes 15 of 18 head-to-heads.
Add OpenRouter model fallbacks to a Node.js app: setup, the models array, response.model tracking, and four pitfalls that catch you on week two.
We ran 12 coding, math, and data tasks through Opus 4.8, Opus 4.7, and GPT-5.5 via LLMTest. Opus 4.8 swept GPT-5.5 but split with its predecessor.
Six open-source LLMs ranked for on-prem deployment in 2026: hardware minimums, real license terms, and the performance tier you get at each level.
Semantic caching reduces LLM API spend by 20-70% in production. Here's how embedding-based, prompt-hash, and hybrid caching each break in practice.
Four LLMs, six French translation tasks tested by a judge: idioms, false cognates, literary register. Claude leads overall. Gemini 2.5 Flash is the value pick.
Mixture of Experts models run only a fraction of their parameters per token. Here's why DeepSeek and Mixtral are cheap, and when MoE gets expensive.
Prompt caching and the batch API cut a real Claude API bill from $797 to $127/month in 2026. Full worked example with exact token counts and 2026 pricing.
Four production patterns for LLM rate limits: jitter, token pre-checks, circuit breakers, and provider failover. Backoff alone won't save you in 2026.
We tested four LLMs on six real buggy diffs: Claude Opus 4.7 swept the field, Haiku 4.5 beat GPT-4o 5-0, and GPT-4o finished with zero wins in 2026.
Eight free LLMs worth actually using in 2026 — ranked by quality ceiling, real rate limits, and the exact point each stops being enough.
We ran 20 real prompts through Claude Sonnet 4.5 and GPT-5. Claude won 8 of 15 comparisons, ran 1.7x faster, and GPT-5 timed out on 5 of 20.
We ran 15 real coding tasks through Claude Opus 4.7 and GPT-5.5 via LLMTest. Claude won 10, GPT-5.5 won 2, 3 ties. Full outputs and verdict inside.
Four LLMs, six SQL tasks, one PostgreSQL schema. GPT-4o-mini led with 9 wins over Claude Sonnet 4.5, GPT-4o, and Gemini 2.5 Flash. Here's the full breakdown.
We ran 5 developer tasks through DeepSeek V4 Pro, GPT-5.5, Opus 4.7, and Llama 4. V4 Pro beats GPT-5.5 while costing 4.5x less, but latency averages 28 seconds.
Prompt caching cuts LLM costs 90% on Anthropic and 50% on OpenAI, but only when your workload fits. Here's the exact break-even math per provider.
The exact token-to-word and token-to-character conversion rates for English, code, and non-English LLM input, plus a practical counting recipe.
OpenAI's GPT-5.5 brings a 1M-token context and native computer use to the frontier, at double GPT-5.4's price. Here's what actually changed.
A 7-step framework for picking the right LLM for any job. Real constraints, real benchmarks, real routing. Stop guessing from leaderboards.
RAG has 3 moving parts: ingestion, retrieval, and generation. Here's what each does, when RAG beats fine-tuning, and when to skip it entirely.
Opus 4.7 scores higher on coding benchmarks and adds 3.75MP vision, but its new tokenizer inflates real cost by up to 35%. Here's what changed.
One model going down shouldn't take your AI feature with it. Here's how to build a fallback chain using LiteLLM, OpenRouter, and LLMTest.
Your OpenAI bill isn't just input + output tokens. Thinking tokens, JSON retries, and prompt bloat quietly triple costs. Here's how to spot each one in your own app.
The context window is your LLM's working memory per call. What 128k tokens actually fits, why usable size is smaller than advertised, and how to check yours.