Embedding costs for RAG in 2026: $0.02/M vs free self-hosted

Embedding costs for RAG don't register until they do. You spin up a vector store, index a few thousand documents, and the bill rounds to zero. Then your corpus grows to a million documents, a teammate asks about upgrading to a better model, and someone opens the monthly invoice.

That question deserves real numbers.

What you're actually paying for

Unlike chat model calls, embedding APIs only charge for input tokens. There are no output tokens: the response is a fixed-length vector regardless of input size. A 512-token document costs the same whether the resulting vector is 1,536 or 3,072 dimensions.

That makes cost forecasting straightforward: multiply your token count by the per-million rate and you have the ingestion bill.

The less obvious cost is re-embedding. Every time you switch to a better model, you re-index your entire corpus. A team that upgrades embedding models twice a year pays ingestion cost three times over: initial load plus two full sweeps. For large corpora, that is where budget conversations happen.

The API tier: prices and what you trade off going cheap

The main hosted options in 2026:

text-embedding-3-small is the practical default for most pipelines. At $0.02/M, it delivers roughly 95% of text-embedding-3-large's retrieval quality on English and major European language corpora. The 6.5x price gap between small and large is hard to justify unless you're doing high-precision multilingual retrieval or working with dense technical documents where nuance matters.

gemini-embedding-001 is worth testing if you're already in the Google ecosystem. It runs free within Gemini API rate limits and costs $0.025/M on the paid tier. The 2,048-token cap is a real constraint for legal, research, or long-form document pipelines.

One migration note: Google deprecated text-embedding-004 on January 14, 2026. If your code still references that model ID, migrate to gemini-embedding-001.

Gemini Embedding 2 (March 2026, $0.20/M for text) adds multimodal support: images at $0.45/M, audio at $6.50/M, video at $12/M. For text-only RAG it is expensive with no benefit.

Worked example: embedding 1M documents

A mid-size knowledge base: 1 million documents, 512 tokens each on average. Total: 512 million tokens.

$10.24 to embed 1M documents. That is the baseline with text-embedding-3-small. The Batch API halves it to $5.12 if you can tolerate up to 24-hour turnaround.

For ongoing ingestion at 10,000 new documents per day (same average length), you add 5.12M tokens daily. At $0.02/M that is $0.10/day ($3.07/month). At this scale, embeddings don't show up in budget conversations.

At 100,000 new documents per day: 51.2M tokens, $1.02/day, $30.70/month. Still manageable, and a reasonable point to start using the Batch API to cut that to $15.35/month.

The self-hosted tier: when BGE-M3 on a GPU starts making sense

BGE-M3 (BAAI, MIT license) is the production benchmark for open-weights embedding in 2026. At 568M parameters, it fits on a single A100. It supports 100+ languages, handles inputs up to 8,000 tokens, and runs dense, sparse, and multi-vector retrieval from a single model.

Running costs on an on-demand A10G ($0.75/hr, 24GB VRAM):

• Batched throughput: roughly 60,000 tokens per second
• Full utilization: 216M tokens per hour
• Effective cost per million tokens: $0.0035

That sounds cheap. But an A10G running 24/7 costs $540/month whether you use it or not. To break even against text-embedding-3-small at $0.02/M:

Break-even: 27 billion tokens per month.

$540 divided by $0.02/M equals 27,000 million tokens. Below that threshold, the API is cheaper. Above it, the GPU wins by a growing margin.

To put 27B tokens/month in concrete terms:

• 1 million new 1,000-token documents per day = 1B tokens/day = 30B/month. You're at the threshold.
• 100k docs/day at 1,000 tokens = 3B tokens/month. API territory. The GPU sits 89% idle and costs 180x more per token.

For most solo devs and solopreneurs: the API is the correct call. You are not building an embedding service at a billion documents per month.

For teams running high-volume ingestion pipelines or wanting to eliminate per-token vendor pricing at scale: BGE-M3 on a shared A10G pays for itself above roughly 1B tokens per day.

One honest cost the break-even math skips: the engineering time to stand up GPU inference, handle model upgrades, manage dimension mismatches between old and new vectors, and maintain uptime SLAs. Add several engineer-days to the first-year total before the self-hosted path looks cheaper.

The upgrade tax

Every time you move to a better embedding model, you re-index everything. For the same 1M-document corpus:

At 10M documents, those numbers are $102, $204, and $307, still easy to absorb. At 100M documents, a twice-yearly upgrade cycle costs $3,072 per year in embedding alone, before compute to run the pipeline and storage for the new vectors.

This is the real reason teams stick with embedding models past their prime: not technical debt, but re-indexing inertia. Factor it into your model choice early.

The decision in three questions

1. Are you processing more than 27B tokens per month? If no, use the API. Text-embedding-3-small at $0.02/M is the correct default.

2. Is your corpus mostly non-English? If yes, gemini-embedding-001 or BGE-M3 (self-hosted) outperform text-embedding-3-small at comparable or lower cost.

3. Do you have multimodal content? If yes, Gemini Embedding 2 is the only hosted option that handles images, audio, and video in a single pipeline, costing $0.20/M for text and significantly more for other modalities.

For the other half of RAG costs, which LLM to use for the generation step after retrieval, our head-to-head benchmark across four models has the numbers.

Current embedding model rates across providers are in the LLMTest pricing explorer.