This chapter examines why, when an agent fails, we should not suspect only the model. The speaker names two possible causes of failure: the model may not be capable enough, or the context given to the model may be insufficient or unsuitable. In the speaker's view, the latter happens more often.
Even a bad result does not by itself show that the model is weak. The model may have had the necessary information and still failed to answer. But the harness may also have failed to gather the necessary information or pass it to the model. Separating these possibilities is the starting point for choosing the next fix.
Here, context means all the information that enters the model's context window. It is not only the initial instruction. Previous conversations, memory, results returned by tools, and intermediate decisions can also accumulate as the run continues. Therefore, if we do not check whether the harness supplied the necessary information at the necessary time, replacing the model may leave the same failure in place.
As the speaker explains it, observability is a way to inspect an agent's execution from the outside. At minimum, it should let us follow the following things.
コンテキストウィンドウに実際に何が入ったか
実行が進む間に、そのコンテキストがどう積み重なったか
どのステップが、どの順番で動いたか
どのツールが呼び出され、何を返したか
どのモデル呼び出しが、その時点の処理を進めたか
What actually entered the context window
How that context accumulated as the run progressed
Which steps ran, and in what order
Which tools were called and what they returned
Which model call moved the processing forward at that point
This connects directly to the role of the harness discussed in the previous chapter. The harness assembles context, calls the model, runs tools according to the model's response, and returns observations from external systems to the next model step. With observability, we can follow this flow afterward. In other words, we can check whether the harness supplied the right information at the right time.
As a supplement, a trajectory is a sequence of the messages or visible interaction steps that appear during an agent run. It is close to a readable conversation record, so it is useful for checking the overall flow quickly. For example, it can list a request, the model's response, a tool result, and the next response in order.
A full trace, on the other hand, is a record that can be investigated more deeply than a list. We can open individual model calls or tool calls and inspect their details at that point. We can dig down as far as needed to see what input was passed and inside which operation the problem occurred. The supplied material does not reveal the exact field names or nesting of the screen.
In a usable interface, tool calls can be shown in a collapsed form instead of opening all of them from the start. This short view is convenient for scanning a long run. However, a collapsed view alone is not complete debugging. When investigating a cause, we need to expand the relevant calls and check their inputs, results, and surrounding steps.
This display supports the point that, for quick scanning, we do not need to expand every call. However, the screen itself does not show the action used to open or hide the calls.
For example, suppose an agent receives information from an external tool and then gives a wrong final answer. Instead of first looking only at the final answer and replacing the model, inspect the trace from the beginning. Check whether the initial context contained the necessary conditions, whether the tool result entered the next model call, and whether an important detail was lost during an intermediate summary. If the necessary information was absent from the beginning, the problem may be closer to context assembly than to model capability. If the information entered correctly and the steps were also correct but the model still could not answer, it becomes more worthwhile to test insufficient model capability. This is an explanatory example, not a report about a particular product or run result.
An eval judges whether an agent's result meets a defined standard. A trace provides material for investigating what happened on the way to that result. Therefore, an evaluation score alone may not tell us what to change next. Conversely, a trace does not establish an organization's standard for what counts as good. Together, they let us turn a low-scoring run into a hypothesis and choose whether to change the harness, the context, or the model before evaluating again.
The speaker's claim in this chapter that context is the cause more often is an opinion or observation, not a result supported by a quantity. The supplied material also does not show every field of the full trace screen or its internal nesting. Therefore, we cannot say that observability always identifies one unique cause.
To understand an agent failure, we need to see the path to it, not only the result. A trajectory is the entry point for reading the overall run. A full trace is where we inspect the accumulated context, each step, tools, and model calls in detail. An eval shows what was good or bad, while observability gives clues for investigating why it happened. Together, they let us choose the next change without assuming that replacing the model is the only answer.