A chatbot takes your message and returns a response. An AI agent takes a task, calls tools, reads results, decides what to do next, and keeps going until it's done or you stop it. That loop is the difference.
It also explains most of what makes agents useful and surprisingly easy to break.
How the tool-calling loop works
At the core, an AI agent is a language model with tools attached and a loop wrapped around it. The loop has four steps:
- The model receives the user's task and a list of tools it can call.
- The model outputs either a final answer or a tool-call request.
- If it's a tool call, your code runs the tool and feeds the result back as a new message.
- The model reads the result and repeats.
The model never executes anything directly. It asks your code to run the tool, reads the output, and decides what comes next. Function calling is the mechanism that makes steps 2 and 3 possible; the agent loop is what happens when that mechanism repeats across multiple turns.
A concrete example: a research agent asked to "summarize analyst coverage of Stripe from last quarter." It might call a web-search tool for recent reports, fetch two of the most relevant pages, call a summarizer on each, then compose the final answer. That's four tool calls across four loop iterations. The model picked the tools and ordered the steps; your code ran all of them.
The loop terminates when the model returns a final text answer with no tool call (it signals task complete) or when your code enforces a ceiling.
When agents stall and when they run away
The two failure modes are opposite problems with the same root cause: missing exit conditions.
Stalling happens when a tool returns nothing useful and the model keeps requesting the same one, waiting for a result that isn't coming. The model is doing the right thing from its perspective; it believes the task requires that data. The problem is "try again" is the wrong response to a broken tool.
Runaway loops happen when there's no ceiling on turns. A tool returns an error. The model reads "no result" as "task incomplete" and calls the tool again. Every iteration appends another tool result to the conversation, so later turns cost more than earlier ones. The cost compounds while the agent goes nowhere. A developer shared an account of a coding agent burning $15 in 10 minutes by retrying a broken file-search tool with no turn limit, which circulated widely across developer communities in mid-2026.
Three guards against runaway loops in Node.js has the implementation for all three fixes: max-turn limits, cost caps, and return-value validation. The code is under 50 lines total.
The single most reliable protection is a turn limit set before the loop starts, not added reactively. An agent that takes more than 15 tool calls without resolving a typical task has likely hit a dead end.
Three architecture decisions every builder makes
1. Tool count and selection. Every tool you attach adds to the input token cost on every turn because tool schemas go in the system prompt. Models degrade on which-tool-to-call judgment noticeably above 15 to 20 tools. If you have a large tool library, expose only the subset relevant to the current task rather than the full catalog. This both cuts input cost and keeps tool selection accurate.
2. Loop termination policy. The model alone should not decide when the loop ends. That's a job for your infrastructure. At minimum: a hard max-turn count (15 to 20 works for most single-task agents) and a cost budget checked before the next LLM call. For agents serving multiple users in parallel, per-user limits prevent one runaway session from bleeding over; per-user LLM cost guardrails in Node.js covers the implementation pattern.
3. State between sessions. Ephemeral agents start fresh every session. This is the right default for single-task agents: simpler, easier to reason about, no persistent storage needed. If your use case requires continuity, such as a personal assistant that should remember user preferences across days, you need external storage and a retrieval step at session start. The tradeoff is added complexity and a new failure mode: stale or contradictory memory that confuses the model rather than helping it.
Most builders should start ephemeral and add persistence only when users actively complain they have to re-explain their context every session.
What agents actually cost in production
A minimal agent that completes an 8-step task on a mid-tier model costs around $0.40, driven mostly by context accumulation as tool results stack up across turns. The cost breakdown for agents in production shows the step-by-step math, including how a 100-token system prompt compounds to around 50,000 input tokens by step 8 when tool results aren't trimmed.
Context growth is why tool-output sanitization matters beyond just loop prevention. Truncating tool results before appending them cuts cost meaningfully on long sessions without hurting the model's ability to continue. A truncated result with an explicit notice ("output capped at 8,000 chars") gives the model the information it needs to decide whether to call the tool again with a narrower query. Sanitizing tool results also guards against prompt injection: embedded instructions in content retrieved from external sources can redirect the agent's behavior if they reach the model unfiltered.
Track per-call token counts and session cost in real time with the LLMTest proxy to catch runaway sessions before they complete.
FAQ
What's the difference between an AI agent and a chatbot? A chatbot takes a message and returns a response, one exchange at a time. An agent takes a task, works through multiple steps autonomously using tools, and returns a result when the task is done. The key difference is autonomous multi-step action, not just the presence of tools.
How many tools should an agent have? As few as the task requires. Tool-selection accuracy drops noticeably above 15 to 20 tools. If you have a large library, expose a task-relevant subset per session rather than everything at once. This also cuts input token cost since tool schemas go in the system prompt on every turn.
Can an agent run without human oversight? Yes, and for well-scoped tasks that's often the right call. The conditions: a clear success state the model can recognize, tools that return deterministic outputs, and hard limits on turns and cost. Open-ended tasks without a clear completion signal are where autonomous agents cause the most trouble.
What's a reasonable max-turn limit? Ten to twenty turns covers most single-task agents. A debugging agent that hasn't resolved an issue in 15 tool calls has probably hit a dead end rather than made meaningful progress. The specific number matters less than having one; pick a number you'd be comfortable seeing on an invoice and adjust from production logs.
Do agents remember things between sessions? Only if you build that. By default, every agent session starts with a blank context. Continuity across sessions requires external storage and a retrieval step at session start. Start without it; the complexity cost is real, and most single-task agents don't need it.
What's the cheapest way to prototype an agent?
Use a budget model for all tool-calling turns and escalate to a frontier model only if tool-selection accuracy is too low for your task. claude-haiku-4-5 and gpt-4o-mini handle the tool-calling mechanics correctly. The question is whether their judgment on which tool to call is precise enough. Benchmark on real inputs before paying frontier rates across the board.
Sign up for LLMTest to route your agents across providers, track per-call cost, and set per-session budgets before they become a problem.