The agent loop

Two ways into the same idea

Before You Start

The Core Idea

Every agent is the same loop. In this lesson you write it by hand, add spin detection, then the trace, and then put a real model in place of the scripted one.

1The loop and its step cap

The loop is four moves: send the messages; if the model asks for a tool, run it; append its result; repeat until a final answer or the step cap.

Look at the picture: the messages grow with every step. Each tool result is appended to them, and that is how the model "knows" what happened.

import json

def get_order_status(order_id):
    return {"48213": {"status": "delayed", "reason": "weather"}}.get(order_id, {"error": "not found"})

def get_refund_policy(reason):
    return {"policy": "Delays over 3 days due to weather: free shipping refund."}

TOOLS = {"get_order_status": get_order_status, "get_refund_policy": get_refund_policy}

def scripted_model(messages):
    """Stands in for the model so the loop runs here. A real model decides from the messages."""
    tool_results = [m for m in messages if m["role"] == "tool"]
    if len(tool_results) == 0:
        return {"tool": "get_order_status", "args": {"order_id": "48213"}}
    if len(tool_results) == 1:
        return {"tool": "get_refund_policy", "args": {"reason": "weather"}}
    return {"answer": "Your order 48213 is delayed by weather; you qualify for a shipping refund."}

def run_agent(question, max_steps=5):
    messages = [{"role": "user", "content": question}]
    for step in range(1, max_steps + 1):
        action = scripted_model(messages)
        if "answer" in action:
            print(f"step {step}: answer -> {action['answer']}")
            return action["answer"]
        result = TOOLS[action["tool"]](**action["args"])
        print(f"step {step}: {action['tool']}({json.dumps(action['args'])}) -> {json.dumps(result)}")
        messages.append({"role": "assistant", "content": json.dumps(action)})
        messages.append({"role": "tool", "content": json.dumps(result)})
    print(f"stopped: reached {max_steps} steps without an answer")
    return None

run_agent("Why is my order 48213 late, and do I get anything back?")

The model here is a pre-written function that decides the steps in place of a real model, so the loop runs in the browser without an API key. The loop code itself does not change when you put a real model in its place. The first step fetched the status and the delay's reason, the second used the reason to fetch the policy, and the third answered. Notice each tool result is appended to the messages; that is how the model "knows" what happened. And max_steps=5 guarantees the loop ends whatever the model does.

2Detecting a spin

A spin is the same call repeated with the same arguments, usually because the tool does not give the model what it needs. The step cap stops it eventually, but only after wasted steps.

Look at the picture: the loop counts each identical call. When the count reaches 3, more than max_repeats = 2, it stops and hands over to a person.

A search tool that finds nothing, and a model that repeats the same search. Without detection it would spin until the cap.

import json

def search(query):
    return {"results": []}                       # the tool keeps finding nothing

def stuck_model(messages):
    return {"tool": "search", "args": {"query": "refund policy weather"}}

def run_agent(model, max_steps=8, max_repeats=2):
    messages, seen = [], {}
    for step in range(1, max_steps + 1):
        action = model(messages)
        key = json.dumps(action, sort_keys=True)
        seen[key] = seen.get(key, 0) + 1
        if seen[key] > max_repeats:
            return f"stopped at step {step}: same call repeated {max_repeats} times, handing over to a person"
        result = search(**action["args"])
        print(f"step {step}: search -> {result}")
        messages.append({"role": "tool", "content": json.dumps(result)})
    return f"stopped: step cap of {max_steps} reached"

print(run_agent(stuck_model))

The loop stores each call as JSON with sorted keys and counts repeats. On the third repeat it stops and hands over to a person, instead of burning five more steps for nothing. In a real system, also return a message to the model saying "this search found nothing twice; try another phrasing or ask the user" before stopping.

3The trace

The trace records every step: what the model asked for, with which arguments, what the tool returned, and how long and how much it took. Without it you cannot diagnose any failure.

Look at the picture: one line per step with five fields. It is the first thing you read when the agent goes wrong.

4The loop with a real model

With a real model one line changes: scripted_model becomes an API call with the tool list. The reply is either text, the final answer, or a tool request with its name and arguments.

def run_agent(client, question, tools, impl, max_steps=10):
    messages = [{"role": "user", "content": question}]
    for step in range(max_steps):
        reply = client.chat.completions.create(model=MODEL, messages=messages, tools=tools)
        msg = reply.choices[0].message
        if not msg.tool_calls:
            return msg.content                              # the final answer
        messages.append(msg)
        for call in msg.tool_calls:
            args = json.loads(call.function.arguments)
            result = impl[call.function.name](**args)
            log_step(step, call.function.name, args, result, reply.usage)
            messages.append({"role": "tool", "tool_call_id": call.id, "content": json.dumps(result)})
    return "I could not finish this within the step limit; a person will follow up."

Exactly the same structure. The difference is the model may request several tools in one reply, so we loop over them, and each result is tied to its request id tool_call_id. And log_step writes the trace with the token usage of each step.

How It Works, Step by Step: one round of the agent loop

Worked example

A customer asks why their order is late and whether they get anything back. The agent needs two steps before answering: the order status, then the refund policy for the delay's reason.

Try it without code

Myth vs Reality

Try it yourself

Quick review cards

Check Your Understanding