A plain LLM answers once and stops. An agent runs a loop: it reasons about a goal, picks a tool, observes the result, and reasons again - repeating until the task is done. Scale that idea to many specialized agents coordinating under a supervisor, and you get systems that plan, act, and self-correct.
An agent is an LLM placed inside a control loop and given tools. Instead of producing a single answer, it decides what to do next - call a search API, run code, query a database - then observes the outcome and reasons about it. It keeps looping until it reaches the goal or hits a stopping condition. Multi-agent orchestration is the next step up: several agents, each with a narrow role, coordinating to solve a task no single agent handles well.
Perceive → plan/reason → act with a tool → observe the result → reflect. Repeat until the goal is met or a guardrail stops it.
Functions the model can call - search, code execution, HTTP, DB queries. Tools are how an agent affects the world beyond text.
Short-term context holds the running scratchpad; long-term memory (a vector store) persists facts and past outcomes across runs.
| Piece | What it does | Common choices |
|---|---|---|
| Reasoning policy | Decides the next thought and action each step | ReAct, Plan-and-Execute, Reflexion |
| Tool layer | Exposes callable functions with typed schemas | Function calling, MCP, code interpreter |
| Memory | Holds working state and long-term recall | Context window, vector store, scratchpad |
| Orchestrator | Routes work across agents and steps | Supervisor, pipeline, fan-out, graph |
| Guardrails | Bounds cost, loops, and unsafe actions | Step limits, budgets, approvals, validators |
Many real tasks can't be solved in one shot - they need lookups, intermediate results, and course-correction. A single prompt has no way to react to what it learns midway. Agents close that gap by turning a one-shot call into an adaptive, tool-using process.
Goals like "research this and draft a report" decompose into many actions. An agent sequences them, using each result to inform the next step.
Tools let the model do things - run code, hit APIs, update records - not just describe them. That turns a chatbot into a worker.
By observing tool outputs and errors, an agent can retry, backtrack, or ask a critic to review - recovering from mistakes a single pass would ship.
Multi-agent setups give each agent a focused role and prompt. Specialists outperform one overloaded generalist and are easier to test.
Same idea, four ways to picture it, so it clicks whoever you are.
An agent is like working through a hard problem set with scratch paper. You don't answer in one line; you try a step, check the result, cross out what failed, and keep going until the answer holds up. A multi-agent team is a study group where one person researches, one calculates, and one writes it up.
It's a while-loop around an LLM call, where each iteration can invoke a function, read its return value, and decide the next call. Orchestration is like a job queue with a coordinator: the supervisor dispatches subtasks to worker functions, then joins their results.
Think of a project manager delegating to specialists. The PM breaks a goal into tasks, hands each to the right expert, reviews what comes back, and asks for a redo when something is off, rather than doing every part alone.
A cook following a recipe and tasting as they go. Each step produces a result they check - too salty, not done - and adjust before the next step, instead of blindly plating whatever comes out first.
Two layers make an agentic system: the single-agent loop that reasons and acts, and the orchestration layer that coordinates multiple agents around a shared goal.
ReAct interleaves reasoning and acting: think → choose a tool → observe → think again. The scratchpad of thoughts and observations grows each turn until the model emits a final answer.
A supervisor decomposes the goal and routes subtasks to worker agents - sequentially, in parallel, or through a critic - then synthesizes their outputs into one result.
flowchart LR
U["👤 Goal"] --> SUP["🧭 Supervisor / planner"]
subgraph Workers["🤝 Worker agents"]
SUP --> W1["🔎 Researcher"]
SUP --> W2["💻 Coder"]
SUP --> W3["✍️ Writer"]
end
subgraph Tools["🧰 Tools"]
W1 --> T1["🌐 Search"]
W2 --> T2["⚙️ Code exec"]
W3 --> T3["🗄️ Knowledge base"]
end
subgraph Mem["🧠 Memory"]
MS["📝 Short-term context"]
ML[("📚 Long-term vectors")]
end
W1 --> MS
W2 --> MS
W3 --> ML
W1 --> SYN["🧩 Synthesize"]
W2 --> SYN
W3 --> SYN
SYN --> ANS["✅ Final result"]
messages = [
{"role": "system", "content": "Reason step by step. Call ONE tool per turn, or reply to finish."},
{"role": "user", "content": goal},
]
for step in range(MAX_STEPS): # guardrail: bounded loop
reply = client.messages.create(
model="claude-opus-4-8",
tools=TOOL_SCHEMAS,
messages=messages,
)
messages.append({"role": "assistant", "content": reply.content})
tool_calls = [b for b in reply.content if b.type == "tool_use"]
if not tool_calls: # no tool -> agent is done
return reply.text
results = []
for call in tool_calls: # tool use
output = TOOLS[call.name](**call.input)
results.append({"type": "tool_result", "tool_use_id": call.id, "content": output})
messages.append({"role": "user", "content": results})
if spend.exceeded() or looks_stuck(messages): # soft termination
return escalate_to_human(messages)
return "Step limit reached, returning best effort"Notice the two termination conditions: a hard step limit and a soft stuck/budget check. Without them an agent can loop forever or burn tokens repeating a failing action - guardrails are what make the loop safe to run in production.
Three views: a single agent's ReAct loop, a supervisor delegating to specialist workers, and a generator paired with a critic for review.
sequenceDiagram
autonumber
participant U as 👤 User
participant A as 🤖 Agent
participant T as 🧰 Tool
participant M as 🧠 Memory
U->>A: Goal
loop until FINISH or step limit
A->>A: Think - plan next action
A->>T: Act - call tool with args
T-->>A: Observation - result or error
A->>M: Append thought and observation
A->>A: Reflect - closer to the goal
end
A-->>U: Final answer
sequenceDiagram
autonumber
participant U as 👤 User
participant S as 🧭 Supervisor
participant R as 🔎 Researcher
participant C as 💻 Coder
participant W as ✍️ Writer
U->>S: Complex goal
S->>S: Decompose into subtasks
par Fan-out to specialists
S->>R: Gather sources
R-->>S: Findings
and
S->>C: Produce analysis
C-->>S: Results
end
S->>W: Draft from findings and results
W-->>S: Draft
S->>S: Synthesize and check
S-->>U: Final deliverable
sequenceDiagram
autonumber
participant U as 👤 User
participant G as ✏️ Generator
participant V as 🔬 Critic
participant T as ✅ Verifier
U->>G: Task
G->>V: Draft answer
loop up to N revisions
V->>T: Run checks - tests or rubric
T-->>V: Pass or fail with reasons
alt verifier fails
V-->>G: Feedback - fix these issues
G->>V: Revised draft
else verifier passes
V-->>U: Approved answer
end
end
Note over G,V: Stop after N rounds and return best effort
From a raw goal to a verified, delivered result - the full agentic journey in order.
A user or upstream system hands the agent a goal in natural language, along with any constraints, context, and success criteria.
The planner breaks the goal into an ordered set of steps and decides whether one agent suffices or the work should be split across specialists.
Relevant facts are pulled from short-term context and long-term memory so the agent starts each step with the state it needs.
Using ReAct, the model writes a thought and selects exactly one tool call - or decides it already has enough to finish.
The agent invokes the tool with typed arguments - search, code execution, an API, a database query - to take a concrete action.
The tool returns an observation - data, a value, or an error. This real-world feedback is appended to the scratchpad.
The agent judges whether the observation moved it toward the goal, whether to retry, backtrack, or change approach.
Step count, token budget, and policy checks run every loop. Risky actions may pause for human approval before proceeding.
For multi-agent work the supervisor fans out subtasks in parallel, chains them in a pipeline, or routes a draft to a critic.
Outputs are reviewed against tests or a rubric and merged into a single coherent result, with failing pieces sent back for revision.
The verified result goes back to the user. Traces, outcomes, and durable facts are written to long-term memory to improve future runs.
Most agents fail on control, not on reasoning. These are the failure modes that bite in production.
With no step limit or termination check, an agent retries the same failing action forever, burning tokens and time. Every loop needs a hard cap and a way to detect it is stuck and give up.
Appending every thought, tool output, and error to the scratchpad grows the prompt until it is slow, costly, and "lost in the middle." Summarize or prune old steps instead of carrying the whole history.
One bad tool result feeds the next step, which compounds the mistake across the trajectory. Without validation between steps, a single wrong observation can derail the entire run.
Splitting a simple goal into a dozen agents and subtasks adds coordination overhead, latency, and more places to fail. Reach for one agent first; add specialists only when a single loop truly cannot cope.
Giving an agent write access to prod, unrestricted shell, or a real payments API invites irreversible damage. Scope every tool to least privilege and gate risky actions behind approvals.
Judging only the final answer hides how the agent got there - wasteful paths, lucky guesses, silent side effects. Without evals over the whole trajectory you cannot tell reliable agents from fragile ones.
Score the whole trajectory, not just the last message, so you know whether an agent is reliable and affordable.
| Metric | What it tells you | Good sign |
|---|---|---|
| Task success rate | How often the agent actually reaches the goal end to end | High and stable across cases |
| Steps / tool-calls per task | How efficiently it reaches the answer | Low: few detours or wasted actions |
| Cost & latency per task | Token spend and wall-clock time for a full run | Within your budget and SLA |
| Tool-call error rate | Fraction of tool calls that fail or return bad args | Low: schemas and inputs are sound |
| Loop / termination rate | How often runs hit the step cap instead of finishing | Low: the agent knows when it is done |
| Human intervention rate | Share of tasks needing a person to unstick or approve | Low for routine work, gated for risky work |
Three representative patterns showing agentic orchestration in production-style use.
An engineering platform wants agents that resolve issues end to end - read the repo, edit code, run tests, and open a pull request.
A team needs an agent that answers deep questions by searching many sources and producing a cited briefing, not a one-shot guess.
A support org wants agents to triage tickets, take routine actions, and escalate anything sensitive to a person.
Agentic systems are moving from clever demos to reliable infrastructure - the frontier is coordination, memory, and trust at scale.
Explicit plan-and-execute and tree-search over actions make agents look several steps ahead instead of greedily reacting one turn at a time.
Tool and agent protocols like MCP and agent-to-agent messaging let agents share tools and talk across vendors and frameworks.
Richer long-term memory - episodic, semantic, and procedural - lets agents learn from past runs rather than starting cold every time.
Policy engines, sandboxes, and permissioned tools become a dedicated safety layer around every action an agent can take.
Trajectory-level evaluation - did the agent reach the goal, at what cost, with what side effects - becomes the metric teams optimize.
Approval gates and interruptible loops shift from afterthoughts to first-class controls so people stay in command of consequential steps.