An interactive guide to LLM agents
Agents,
Demystified

How LLMs use tools, carry state and complete multi-step work

LLM→tool request→result↺
Independent educational project in a personal capacity — not affiliated with or endorsed by any employer · operational scenarios, identifiers, figures and logs are fictional · public sources are cited · not investment advice
Opening

What this talk will answer

01

What is an LLM agent?

A model call embedded in software that can execute and repeat steps.

02

How does it continue?

Tool calls and results become part of the next model call.

03

Why use more than one agent?

Separate contexts support parallel work and create summary handoffs.

04

How should we evaluate and control it?

Repeat the task, measure the distribution and enforce limits in code.

01
The loop

The agent loop

The application calls the LLM, runs approved tools and adds each result to the next model call.

model call→tool request→result→next model call
01 · The loop

Copy, run, paste, repeat

You

Ask: “Why did demo portfolio’s reported exposure double overnight?”

↓
LLM

Suggests a SQL query to compare the position snapshots.

↓
You

Copy the query and run it against the database.

↓
You

Paste the returned row counts into the chat.

↓
LLM

Suggests the next check: inspect the overnight load history.

Fictional example. Who carries the investigation forward between calls? The person copies every action and result.
01 · The loop

The harness runs the round trip

User

Supplies the investigation task once.

↓
Harness

Builds the model input and calls the LLM.

↓
LLM

Returns a structured request for a tool.

↓
Harness

Validates the request, runs the tool and records the result.

↺
Harness

Calls the LLM again with the updated transcript.

For this talk: an LLM agent is a software system in which an LLM can select tools, inspect their results and continue until the task finishes or the harness stops the run.

“Agent” has broader meanings in AI; this talk focuses on tool-using LLM agents.
01 · The loop

One model call receives three main inputs

complete model input

System prompt

Developer-supplied text that frames the task and the model’s role.

Tool definitions

Function names, descriptions and argument schemas describing the operations the LLM may request.

Message transcript

The ordered user, assistant, tool-call and tool-result history from the current run.

The API may expose these as separate request fields. Together they occupy the model’s context window.
01 · The loop

The LLM returns a function name and arguments

tool definition supplied to the LLM
{
  "name": "run_query",
  "description": "Run read-only SQL",
  "parameters": {
    "sql": "string"
  }
}
structured assistant output
{
  "call_id": "call_17",
  "name": "run_query",
  "arguments": {
    "sql": "SELECT ..."
  }
}
1

LLM request

name + arguments

→
2

Validation

permissions + schema

→
3

Execution

function or adapter

→
4

Tool result

returned to the loop

01 · The loop

The harness repeats the cycle and decides when to stop

construct model input call LLM tool call? no finalresponse yes validate → execute → append call + result → repeat
  1. The LLM returns a final response.
  2. A step, time or cost budget is reached.
  3. An approval is required and withheld.
  4. A tool or policy error triggers the failure path.
  5. The user or application cancels the run.
01 · The loop

The minimal loop fits on one screen

tools = [run_query, read_job_log, ...]
messages = [user_task]

while True:
    reply = llm(messages, tools)
    messages.append(reply)

    if not reply.tool_call:
        break

    result = run_tool(reply.tool_call)
    messages.append(result)

This is the mental model for the rest of the talk: call, act, observe, repeat.

01 · The loop — now running
Fictional scenario · simulated · no live model
minimal loop, running
tools    = [menu...]messages = [user_task] while True:  reply = llm(messages, tools)  messages.append(reply)   if not reply.tool_call: break   result = run_tool(reply.tool_call)  messages.append(result)
think → act → observe ↻
ready — press Step
message transcript — the loop appends here
context estimate: 412 tokens
01 · The loop

What the harness handled during that run

annotate the trace
model request
Built from the task, transcript and tool definitions.
tool call
Validated and dispatched through the permitted adapter.
tool result
Paired with the request and added to the transcript.
next model call
Constructed from the updated transcript.
approval / stop
Applied by ordinary code around the model.

The next step was only as useful as the evidence returned by the environment.

01 · The loop

The transcript records each tool call and its result

user
task: explain why demo portfolio’s exposure doubled overnight
assistant
tool call · call_id=call_17 · name=run_query · arguments={sql: "SELECT …"}
tool
result · call_id=call_17 · output={D-1: 1,206, D-0: 2,412}
assistant
tool call · call_id=call_18 · name=read_job_log · arguments={job: "demo_snapshot_load"}
tool
result · call_id=call_18 · output={run_1: "timeout", run_2: "success"}
assistant
final response: duplicated load caused the exposure anomaly; proposed remediation awaits approval
Fictional trace. API role names and envelopes vary; the ordered call/result relationship is the same idea.
01 · The loop

The model reprocesses an expanding input

stable prefix
tool
definitions
system
prompt
user
task
tool call · result · tool call · result · …

Context window

The complete input and generated output must fit within the model’s token limit. Longer histories cost more and can make relevant details harder to retrieve.

Prompt-prefix reuse

Providers may reuse computation for an unchanged prefix. Appending new records preserves it; changing earlier content can reduce the reusable portion.

Exact matching, cache lifetimes and invalidation rules are provider- and model-specific.
02 · Feedback

Tests and compilers return evidence the loop can use

software task

Code change → test result → retry

A failing assertion identifies a concrete condition. The loop can inspect it, modify the code and rerun the same check.

Fast · machine-readable · repeatable
market interpretation

Retrieve sources → assemble evidence → judge

The loop can collect facts and citations. Whether those facts support the conclusion often still requires judgement.

Evidence-rich · less objective · higher review need

The environment determines which mistakes the loop can detect and correct.

02 · Feedback — measured capability

Newer agents complete longer software tasks in METR’s benchmark

METR chart showing 50 percent task-completion time horizon increasing across model releases
Source: METR, “Measuring AI Ability to Complete Long Tasks”, 19 Mar 2025 · CC BY 4.0
How to read “30 minutes at 50%”
30 min

Take software tasks estimated to require a skilled human about 30 minutes. At this horizon, the agent completes roughly half of them successfully.

Duration: estimated human completion time.
50%: the benchmark success threshold used to define the horizon.

Across the historical data, this 50% horizon doubled roughly every seven months.

03
Multi-agent coordination

Coordinating several agent loops

A lead assigns separate investigations to workers with their own transcripts, then combines the reports.

lead→separate workers→reports→synthesis
03 · Multi-agent

Three investigations compete for one context window

NEWS PORTFOLIO CALENDAR CONTEXT LIMIT SEARCHES, TOOL CALLS AND RESULTS ACCUMULATE TOGETHER
03 · Multi-agent

One lead receives final reports from separate workers

lead contextreceives final reports news workerlocal trace + sources portfolio workerlocal trace + sources calendar workerlocal trace + sources GREY: TASK · CLAY: FINAL REPORT WITH SOURCE REFERENCES
03 · Multi-agent — watch both versions run
Fictional scenario · illustrative data
lead context · receives final worker reports only
Task: produce a research-team briefing. Press Run.
shared draft · both workers write here at the same time
private worker contexts · raw searches and tool results stay with the worker

worker · news

own context: 0 tokens

worker · portfolio

own context: 0 tokens

worker · calendar

own context: 0 tokens
the briefing delivered to the team
—
03 · Multi-agent — evidence and cost

Anthropic’s research system improved its internal evaluation and used more tokens

+90.2%
Improvement over a single agent on Anthropic’s internal research evaluation.
Anthropic engineering write-up · vendor evaluation · 2025
~15×
Tokens used by the multi-agent research system versus normal chat; agents generally used ~4×.
Same Anthropic write-up
14
Recurring multi-agent failure modes catalogued across design, coordination and verification.
MAST · 1,600+ traces · 7 frameworks · arXiv v3
04 · In practice

Two questions determine the control boundary

1

What evidence can check the result?

A test, reconciliation service, independent source or person may provide the checker.

This determines whether the loop can detect and correct errors.

2

What happens if the action is wrong?

Consider reversibility, financial impact, regulation and the time available to intervene.

This determines permissions, approvals and deterministic limits.

04 · In practice

The surrounding system sets the level of autonomy

Workflow
What checks it?
If it is wrong
Boundary
Refactor code
Tests and compiler
Revert the branch
Agent can complete
Draft a customer update
Facts are checkable; wording is judged
Customer impact
Person reviews and sends
Execute a payment
Authorisation checks and execution receipt
Financial and regulatory impact
Policy controls and approval
04 · In practice

Developer and quant workflows offer useful verifiers

Code and backtest changes

Refactors, migrations, test generation and backtest plumbing can run against regression suites and known invariants.

verifier

Tests, compilation, benchmark outputs and invariant checks.

Data-pipeline investigations

The agent can inspect trades, compare snapshots and read job logs before proposing a remediation.

boundary

Read operations run directly; state-changing writes wait for approval.

04 · In practice

Information workflows need evidence attached to the output

Research briefing

Retrieve: overnight news, portfolio and calendar.

Produce: a concise team briefing with source links.

Review: material claims and audience relevance.

Earnings and research digestion

Retrieve: transcripts, estimates and prior guidance.

Produce: surprises, changes and cited excerpts.

Review: interpretation and analytical relevance.

The agent compresses the information. A person remains responsible for the judgement.
05 · Evaluation

Three failure modes recur across agent systems

0.910 ≈ 35%
Error accumulation: a long sequence creates more opportunities for an unchecked mistake.
Illustrative calculation, not empirical evidence
a ≠ a
Run-to-run variation: the same input can produce a different sequence of choices.
“…”
Untrusted external content: retrieved material can attempt to redirect the agent or trigger unsafe actions.
05 · Evaluation

Evaluate repeated runs under normal and broken conditions

Measure the distribution

  • Task success and forbidden-action rates
  • Human interventions per run
  • Tool calls and total tokens
  • Latency and cost at p50 / p95
  • Evidence and citation correctness

Break the environment deliberately

  • Missing, malformed or unavailable tools
  • Stale and duplicated data
  • Long transcripts
  • Hostile text in retrieved sources
  • Approval denied or task cancelled
An agent demo is one sample from a distribution.
06 · Emerging research

A second loop can search for a better harness

inspect traces + scores
→
propose harness change
→
run held-out evaluations
→
keep or reject
↺

Task-specific agent harness

system prompt · tools · state construction · parsers · retries · dispatch logic

Meta-Harness is one recent research system that searches over the code controlling what an LLM stores, retrieves and sees.

Lee et al., “Meta-Harness: End-to-End Optimization of Model Harnesses”, arXiv:2603.28052, 2026
06 · Emerging research

Self-improvement research targets several layers of the system

Output refinement

Revise the current answer.

Memory and harness

Change stored state, prompts or tools.

Agent code

Search over workflows and scaffolds.

Model weights

Train parameters from task feedback.

Research loop

Use improved systems to produce further improvements.

Current work demonstrates bounded forms under benchmarks and controls. Open-ended recursive self-improvement remains a research problem.
Closing synthesis

Four answers to take away

1What is an LLM agent?An LLM operating inside a software loop with tools.

2How does it continue?The harness records tool calls and results in the transcript.

3Why multiple agents?Separate contexts support parallel work and introduce lossy handoffs.

4How should we use them?Attach verifiers, repeat evaluations and enforce limits in code.

Built and written by Max Khor · maxkskhor.github.io · a personal educational project · all examples fictional · not affiliated with or endorsed by any employer
+
Optional deep dives
Architecture, caching and audience exercises
Optional · architecture choice

When do you need an agent at all?

Fixed workflow

  • Application code chooses the next step.
  • The model performs bounded transformations.
  • Path, cost and failure points are easier to predict.

Agent loop

  • The LLM selects a next step from what it discovers.
  • Useful when the path cannot be fixed in advance.
  • Cost and outcomes vary between runs.
If the path is known, code it directly. Use an agent when new evidence changes what should happen next.
Optional · audience exercise

Design a fictional exposure-investigation workflow

available operations
query positionsread tradescompare snapshotsread job logsdelete duplicate rowsamend risk report

Which operations become tools?

Which actions require approval?

What result verifies the fix?

discuss first
One design: read and comparison tools run directly. Deletes and report amendments wait for approval. Success requires 1,206 unique position IDs and a reconciled risk report.
Optional · serving detail

Why append-only transcripts can be cheaper

Generation-time KV cache

While generating one response, the inference engine stores key/value representations for preceding tokens so it does not recompute the full sequence for every new token.

Cross-request prompt cache

Across agent-loop calls, a provider may reuse computation for an unchanged prompt prefix. Tools, system content and earlier messages must match the provider’s rules.

tools
system
task
append new call + result
Preserve a stable prefix when practical. Compact deliberately when context pressure or relevance justifies the rewrite.
Optional · audience exercise

What should a worker report preserve?

01

Conclusion

The worker’s answer to its assigned subtask.

02

Material evidence

Facts and qualifiers that could change the conclusion.

03

Source route

Citations, document IDs or retrieval handles.

04

Uncertainty

Open questions, confidence and conflicting evidence.

Further reading

Practical agent engineering

Building effective agents — Anthropic, 2024

Workflows, agents and common implementation patterns.

anthropic.com/engineering/building-effective-agents

Effective context engineering for AI agents — Anthropic, 2025

State, retrieval, compaction and long-running tasks.

anthropic.com/engineering/effective-context-engineering-for-ai-agents

How we built our multi-agent research system — Anthropic, 2025

Lead-worker architecture, evaluation results and token costs.

anthropic.com/engineering/multi-agent-research-system
Further reading

Systems and emerging research

Context and caching

Prompt caching — OpenAI

developers.openai.com/api/docs/guides/prompt-caching

Prompt caching — Anthropic

platform.claude.com/docs/en/build-with-claude/prompt-caching

PagedAttention — Kwon et al., 2023

arxiv.org/abs/2309.06180

Harnesses and self-improvement

Meta-Harness — Lee et al., 2026

arxiv.org/abs/2603.28052

Automated Design of Agentic Systems — Hu et al., 2024

arxiv.org/abs/2408.08435

Darwin Gödel Machine — Zhang et al., 2025

arxiv.org/abs/2505.22954