Your LLM gives confident answers about your product docs, except the answers aren't real because your docs aren't in its training data. The framework-heavy tutorials take 400 lines to show what is conceptually 3 steps: chunk text, store vectors, retrieve and generate. This tutorial does it in under 80 lines of Node.js, using pg and the Anthropic SDK with no abstraction layers you would later need to fight around.
If you need to understand how the three stages of RAG fit together before writing code, start there and come back. Here the assumption is that you know what you are building.
The schema: two tables and one index
Enable pgvector, then create two tables. Separating documents from their embeddings means you can re-embed chunks with a better model later without losing the source text.
CREATE EXTENSION IF NOT EXISTS vector;
CREATE TABLE documents (
id BIGSERIAL PRIMARY KEY,
content TEXT NOT NULL,
source TEXT,
chunk_index INTEGER
);
CREATE TABLE embeddings (
id BIGSERIAL PRIMARY KEY,
doc_id BIGINT REFERENCES documents(id),
content TEXT NOT NULL,
embedding vector(1536)
);
CREATE INDEX ON embeddings USING hnsw (embedding vector_cosine_ops);
The HNSW index is the 2026 default over IVFFlat. It builds before you have any data, avoids the post-insert cluster pass that IVFFlat requires, and returns top-10 results in under 10ms at 10M rows on a standard Postgres instance. The dimension count, 1536, matches text-embedding-3-small; update it if you switch embedding models.
Ingest: chunk, embed, store
Word-based chunking is blunt but predictable. Start at 400 words with 40-word overlap and tune based on what your queries miss.
// rag.js
import pkg from 'pg';
import Anthropic from '@anthropic-ai/sdk';
const { Pool } = pkg;
const pool = new Pool({ connectionString: process.env.DATABASE_URL });
const client = new Anthropic();
function chunkText(text, size = 400, overlap = 40) {
const words = text.split(/\s+/);
const chunks = [];
for (let i = 0; i < words.length; i += size - overlap) {
chunks.push(words.slice(i, i + size).join(' '));
if (i + size >= words.length) break;
}
return chunks;
}
async function embed(text) {
const res = await fetch('https://api.openai.com/v1/embeddings', {
method: 'POST',
headers: {
Authorization: `Bearer ${process.env.OPENAI_API_KEY}`,
'Content-Type': 'application/json',
},
body: JSON.stringify({ model: 'text-embedding-3-small', input: text }),
});
const json = await res.json();
return json.data[0].embedding;
}
export async function ingestDocument(content, source) {
const chunks = chunkText(content);
for (let i = 0; i < chunks.length; i++) {
const { rows } = await pool.query(
'INSERT INTO documents (content, source, chunk_index) VALUES ($1, $2, $3) RETURNING id',
[content, source, i]
);
const embedding = await embed(chunks[i]);
await pool.query(
'INSERT INTO embeddings (doc_id, embedding, content) VALUES ($1, $2, $3)',
[rows[0].id, `[${embedding.join(',')}]`, chunks[i]]
);
}
}
text-embedding-3-small costs $0.02 per million input tokens. Indexing a 10,000-page knowledge base runs around $10. For a full breakdown of self-hosted alternatives and where the break-even math falls, see the embedding cost comparison for RAG in 2026. You can also swap the embed function for Voyage AI's API (Anthropic's embedding service) by pointing to https://api.voyageai.com/v1/embeddings and using voyage-3 as the model name.
One constraint to lock in early: the model used in embed() during ingestion and retrieval must be identical. Switching models later means re-indexing the entire corpus.
Query: retrieve, rank, synthesize
The <=> operator in pgvector returns cosine distance. Subtracting from 1 gives similarity. Pass the top-5 chunks to Claude and instruct it to cite which ones it drew from.
async function retrieve(question, k = 5) {
const vec = await embed(question);
const { rows } = await pool.query(
`SELECT e.content, d.source, 1 - (e.embedding <=> $1) AS score
FROM embeddings e JOIN documents d ON e.doc_id = d.id
ORDER BY e.embedding <=> $1 LIMIT $2`,
[`[${vec.join(',')}]`, k]
);
return rows;
}
export async function answer(question) {
const chunks = await retrieve(question);
const context = chunks
.map((c, i) => `[${i + 1}] (source: ${c.source})\n${c.content}`)
.join('\n\n');
const msg = await client.messages.create({
model: 'claude-haiku-4-5-20251001',
max_tokens: 1024,
messages: [{
role: 'user',
content: `Answer using only the context below. Cite sources as [1], [2], etc. If the context does not answer the question, say so.\n\nContext:\n${context}\n\nQuestion: ${question}`,
}],
});
return {
answer: msg.content[0].text,
sources: chunks.map(c => c.source),
};
}
claude-haiku-4-5-20251001 handles synthesis at $0.80/M input tokens, fast enough for user-facing latency. For documents where answers require reasoning across conflicting passages, upgrade to Sonnet 5. The head-to-head comparison of frontier models on RAG answer synthesis shows exactly where each model breaks down on ambiguous retrieval.
The "if the context does not answer the question, say so" instruction catches the most common RAG failure: hallucinating an answer when retrieval returns wrong chunks. It will not catch every case, but it handles the obvious ones.
Wire up the API routes
// server.js
import express from 'express';
import { ingestDocument, answer } from './rag.js';
const app = express();
app.use(express.json());
app.post('/ingest', async (req, res) => {
await ingestDocument(req.body.content, req.body.source);
res.json({ ok: true });
});
app.post('/ask', async (req, res) => {
const result = await answer(req.body.question);
res.json(result);
});
app.listen(3000);
Start it with node server.js (Node 22+ uses ESM by default; for Node 18-20, add "type": "module" to package.json). The /ingest route processes one document synchronously. For large corpora, move ingestion behind a queue and return a job ID instead of blocking the request.
What breaks at scale
Chunk boundaries cut the answer in half. The most common retrieval miss is an answer that spans two consecutive chunks. The 40-word overlap reduces this but does not eliminate it. If queries repeatedly miss things you know are in the corpus, reduce chunk size to 200 words first, then investigate whether sentence-boundary splitting helps.
Model mismatch on re-embedding. Upgrading from text-embedding-3-small to text-embedding-3-large or voyage-3 requires re-embedding the entire corpus. Vectors from different models are not comparable.
Empty retrieval still generates. When no relevant chunks exist, the model sometimes answers from its training data instead of saying it cannot help. Add a score cutoff in retrieve: if the highest-scoring chunk falls below 0.70 cosine similarity, skip generation and return a fixed "no relevant documents found" response.
Context overflow at higher k values. Returning 5 chunks at 400 words each is around 2,000 words of context, well inside any frontier model's window. At k = 20 or with larger chunks, count tokens before building the context string or you will hit the limit silently.
For multi-provider fallback on the synthesis step (swapping Claude for GPT-5 when latency spikes), or to route through a unified proxy that handles retries and observability without changes to application code, see the LLMTest proxy docs.
Set up your first RAG pipeline and start querying your own documents.