2:46 - 5:22
The general agent loop and its middleware levers
Chapter 3 · 2:46–5:22
By the end of this chapter, you should be able to trace a general agent run from a request to a tool observation and back to the model, identify where middleware can change that run, and distinguish summarization from context offloading.
The previous chapter separated an agent into a model, the information available to it called context, and a harness, the software that organizes the model's work. This chapter follows the harness while it runs. Harrison Chase presents the smallest useful architecture as a large language model (LLM) in a loop. The harness keeps that loop moving and adds behavior around it.
The minimal loop
The general flow has six steps:
- The agent receives a user request.
- The LLM generates a response or decides that it needs a tool, an external capability or system that can perform an action or return information.
- The harness invokes the tool.
- The tool returns an observation, meaning the result supplied by the external system.
- The harness gives that observation back to the LLM as new context. Here, context means the material available to the model for its next step.
- The LLM interprets the updated context. It may call another tool, or it may return the final result.
A tool call does not automatically end the run. The observation becomes input to another model step, so the LLM can interpret what happened and decide what to do next. After that step, the LLM may call another tool again or return the result.
The following Mermaid diagram is a teaching reconstruction of this sequence, not a screenshot or a quotation from the talk:
flowchart LR
A[User request] --> B[LLM generates]
B -->|needs a tool| C[Harness invokes tool]
C --> D[Observation]
D --> B
B -->|finished| E[Return result]
"General" describes the reusable pattern. It does not mean that every harness implements the loop in the same way. Harnesses can add different policies, capabilities, and checks while preserving the request, generation, action, observation, and repeat cycle.
The source visual shows a general agent loop alongside a more detailed agent harness flow. It visibly supports a request entering the model, an optional tool action producing an observation that returns to the model, and an eventual result. Source at 3:17 (visual timestamp: 197.5s).
Middleware around the loop
Middleware is behavior inserted at a defined point in a running process. A hook is an insertion point where extra behavior runs. A wrapper surrounds a call so it can do something before or after that call. A plugin is an extension that adds behavior or a capability to the harness. In this chapter, these are ways to extend the running agent process, not replacements for the LLM or for the general loop.
The speaker describes several places where this extra behavior can run:
| Insertion point | What it can control |
|---|---|
| Before agent invocation | Prepare or check the run before the agent starts. |
| Before each model call | Adjust or check what is about to reach the model. |
| Around a model call | Run behavior before and after the model call. |
| After a model call | Inspect or process the model's result. |
| Around a tool call | Control what happens before the tool runs and after it returns. |
| After agent completion | Handle the completed run. |
These levers let a team change one part of a general harness without discarding the loop. A hook before a model call can check the active context. A wrapper around a tool call can control how a large result is handled. In both cases, the model still takes the next step in the same request-to-observation cycle.
The source visual contrasts a minimal agent loop with a more extensive harness lifecycle. The minimal loop centers on a model interacting with tools and producing a result, while the broader diagram adds lifecycle stages and wrappers around agent, model, and tool calls. Source at 3:35 (visual timestamp: 215.5s).
The named levers
The next slide gives the insertion points concrete names. It labels hooks such as before_agent and before_model, wrappers such as wrap_model_call and wrap_tool_call, and later points such as after_model and after_agent.
The visible labels explicitly show before_agent, before_model, wrap_model_call, wrap_tool_call, after_model, and after_agent around the request/model/tools/result flow. They are examples of where customization can enter the lifecycle. Source at 3:59 (visual timestamp: 239s).
Frameworks will implement these points differently. The design question stays stable: at which transition should the extra behavior run? A check before a model call is different from a wrapper that manages a tool's output. Naming the transition keeps the customization's timing visible.
Capabilities supplied by the harness
The harness also supplies capabilities that the LLM can use inside this loop. The speaker mentions:
- a sandbox, an execution environment in which the agent can perform work;
- a file system, which lets the agent read or change files;
- a sub-agent, another agent used for delegated work; and
- memory, information retained for later use.
These capabilities expand what the agent can do or what information it can use. They do not create a different core architecture. The harness still calls the model, acts when the model requests a capability, returns the resulting observation, and calls the model again.
Teacher-created example. Suppose an agent is asked to inspect a file. The harness can make file-system access available, invoke that capability when the LLM requests it, and return the result as an observation. The LLM then gets another model step. This example illustrates the timing; it is not a scenario quoted from the talk.
Summarizing an overlong context
One middleware behavior manages context before it becomes too large. Summarization reduces accumulated model messages and tool observations into a shorter representation before the next model call.
The causal sequence is:
- Model messages and tool observations accumulate.
- Before the next model call, the harness checks whether the active context is too long.
- If it is too long, the harness summarizes the accumulated context.
- The model receives the reduced context and continues the loop.
The timing matters. The harness checks first and summarizes before invoking the model. Summarization changes how the accumulated conversation or work history is represented. It does not mean that the tool call is skipped.
Context offloading is a different operation
Context offloading handles a large tool call rather than an already accumulated conversation. The harness wraps the tool call and dumps the tool's large contents elsewhere instead of keeping all of them in the model's active context. The agent can then continue without carrying the entire result in every later model input.
| Strategy | What it manages | Basic action |
|---|---|---|
| Summarization | Context accumulated before a model call | Reduce the context before invoking the model. |
| Context offloading | A large tool result | Move or dump the tool output rather than keeping all of it active. |
Both are harness behavior around the same general loop, but they act at different points. Summarization reduces the representation of context already in the run. Offloading changes where a large tool result is kept so that it does not all remain active. The source describes this at a high level. It does not specify a storage design or a retrieval protocol for the offloaded content.
The source visual organizes the base deepagents harness into execution environment, delegation, steering, and context-management components. It visibly identifies context offloading as one context-management capability. The frame does not itself show the tool-call wrapper or the content-dumping sequence described in the talk. Source at 5:05 (visual timestamp: 305.5s).
Misconceptions to avoid
- A tool call is not necessarily the end of an agent run. Its observation normally returns to the model for another step.
- A general loop is not a promise of minimal behavior. Middleware, hooks, plugins, capabilities, summarization, and offloading can add substantial behavior around the same core.
- Summarization and context offloading are not synonyms. Summarization reduces accumulated context before a model call. Offloading handles the contents of a large tool call.
Source boundary and uncertainty
The loop, insertion points, capabilities, summarization behavior, and context-offloading distinction above explain the speaker's described architecture. The Mermaid diagram and the file-inspection scenario are teaching aids, not quotations or additional video evidence. The visual explanations stay limited to what the supplied visual observations support. In particular, the last slide does not show the tool-call wrapper or content-dumping sequence.
The transcript has a noisy fragment around the transition into the core loop and renders one product name as "Quad Code". That product spelling is uncertain, so this chapter does not silently correct or rely on it. The middleware labels are described verbally in the transcript; their exact visible labels are supported here by the cited slide rather than attributed to spoken wording.
Check your understanding
An agent calls a tool and receives an observation. Does the general loop end automatically?
Answer: No. The harness returns the observation to the LLM as context for another model step. The LLM may call another tool or produce the final result.
The conversation has become too long before a model call, while one tool has returned a very large result. Which operation matches each problem?
Answer: Summarization reduces the accumulated context before the model call. Context offloading wraps the large tool call and moves or dumps its contents so they do not all stay active.
Synthesis
A general agent harness keeps a request, model step, optional tool action, observation, and next model step moving in a loop. Middleware changes what happens before, around, or after those steps. Capabilities give the loop access to execution, files, delegated agents, or retained information. Summarization and context offloading manage different context pressures. Customization can therefore add substantial control while preserving the general architecture. The next chapter asks when these local levers should become a more explicit domain-specific flow.