モデルのループだけではない、複数のフィードバックループ

This chapter explains why a loop for tuning the model alone is not enough to improve an agent in production. The talk introduces, in addition to the model loop, a loop that tests the app through real use and a loop that learns from the marketplace. Different kinds of information need a common process before they can be fed back to the agent.

First, look at the model loop

The model loop centers on aligning the agent with offline human labels. For example, the team regularly samples production data and has people label it using defined guidelines. The team then runs the agent on the same data and checks for differences from the human judgments.

This comparison helps find drift, where data conditions or system behavior change so that earlier tuning no longer fits. When a mismatch appears, the team diagnoses which part has a problem and tunes the prompt or the agent configuration. After tuning, the team evaluates again on golden data and ships only if the guardrails are satisfied.

Example: the question answered by the model loop

The main question for the model loop is, "Is this agent moving closer to the behavior that people want?" Human labels are an important reference here, but they are not treated as perfectly error-free. The way labels are created and how representative the data is can also affect the evaluation result.

A loop that tests the app through use

There is also feedback from using the product. The talk uses dogfooding, meaning that the team uses and tests its own real app, to find problems that are hard to discover in model-evaluation data alone. The audio makes this word fragmentary, but the context indicates that it means dogfooding here.

This loop can be used both before and after an official launch. Users and internal teams can react with signals such as "good" or "bad," or can write free-form comments. These reactions should not end as a record of complaints; they should become evidence for deciding which agent behavior needs to be fixed.

As a supplement, an image may pass offline evaluation but look unnatural in the app screen. Sending reactions from real use back into improvements to the agent configuration or prompt makes it possible to address problems that test data alone did not reveal.

Figure: This slide summarizes that, in addition to the model loop, there is a dogfooding loop and a marketplace loop.

Supplement: The figure directly shows the idea that evaluation should not end inside the model, but should also receive feedback from actual app use and the production marketplace. It does not show the detailed collection methods mentioned in the audio or every loop not included in the figure.

Relevant part of the talk (18:45)

A loop that learns from the production marketplace

The marketplace loop looks at actual production outcomes. For example, it tracks marketplace quality and health, including conversion—the rate at which behavior moves to the next step—from adding an item to the cart through completing an order. This means checking product results, not only how an image looks.

Overall numbers can hide a problematic group of users. The team therefore examines results by geography, device, dish type, and other segments. This way of "slicing" the results can reveal that one region is improving while another dish type is getting worse. The team uses the result to tune the configuration for each segment. The talk does not provide improvement rates or the causal effect of an experiment.

The three loops are not the same

• Model loop: Compare new human labels with agent output, detect drift, and tune the system.
• Product/dogfooding loop: Find usage problems from reactions and comments when people use the app in practice.
• Marketplace loop: Examine production quality, health, and conversion by segment, and improve the configuration.

More loops create a need for a common mechanism

The three loops have different input information and look for different failures. Human labels are information close to an explicit ground truth. App comments contain concrete problems felt by users. Production conversion is a result of the whole system, rather than a direct evaluation of one image. Even so, if every loop becomes a separate one-off script, the tuning methods and safety checks will become inconsistent.

As a supplement, a common interface—a boundary through which different mechanisms pass information in the same form—makes it easier to put each loop into the same process. First receive the feedback, organize the problem, find the relevant agent or configuration, and propose a fix. Then check it with benchmarks and guardrails before sending the change to production. The talk does not show this internal format in detail, so this is a design supplement.

The diagnoser that comes next

The next chapter introduces a diagnoser to solve this commonization problem. The diagnoser is not another image-editing agent. It receives information from several feedback loops, reflects on the system, identifies which agent or configuration should be optimized, and sends the fix to the right place. In other words, it does not only strengthen the model loop; it connects signals from different parts of the real world to the same improvement cycle.

Summary

The model loop corrects drift by using mismatches with human-labeled data. But a production agent also needs to learn from reactions to app use and from marketplace behavior. As more loops are added, a general workflow that connects feedback to diagnosis, tuning, and safety checks becomes more important than separate one-off processes.