実例：スイートポテトフライとK回目での合格

Here, we make the loop of image editing and QA (quality assurance) concrete with a sweet-potato-fries example. We follow the abstract process in order: an initial edit, QA rejection, a revision, and then a pass.

First, look at the example's flow

In the example shown in the presentation, the image is edited first. QA then examines the edited result. If the result does not meet the criteria, QA returns what should be fixed. The edit is performed again using that feedback. In this example, the second attempt passes.

The first edit is not enough by appearance alone

The first generated result for the sweet potato fries failed QA. Specifically, the portion size and plating were incorrect. In other words, the check considers not only whether the image looks attractive, but also whether the food content and the way it is presented are appropriate.

Why pixel-only comparison is not enough

Comparing only pixel differences is not enough to judge whether the amount of fries is appropriate as a dish or whether the plating on the plate looks natural. These are constraints about the image's meaning and the way the food is presented. Therefore, this QA does not use a simple image difference; it checks the content represented by the image and the quality of its presentation.

Make the second attempt with QA feedback

The initial failure is used as input for the next edit.

• QA points out the problems with the portion and plating.
• The editing stage uses that feedback to revise the image.
• The second result is sent to QA again.
• In this example, the revised result passes QA.

What does “pass at K” measure?

By “pass at K,” the presenters mean the proportion that passes at the Kth iteration. It does not mean that one particular image will necessarily pass by the Kth attempt. The presentation does not give a numeric value for K, a formula, or a specific pass rate.

This metric is used to see how many results pass when the system uses QA feedback and how many edits it allows. The presenters expect that increasing the number of iterations and applying more feedback can raise the pass rate. However, this is not a claim that the success shown by this one example is guaranteed for every image.

Figure: The “Generation Evals” slide shows the sweet-potato-fries input image, the first generated result, and the second generated result. The first result is marked QA FAIL, with problems in portion size and plating. The second is marked QA PASS and also indicates that the QA suggestion was applied.

This figure directly shows an example in which QA feedback is used in the next edit and the result improves. What the slide shows is this specific example; it does not show that every image passes after the same number of attempts. You can check the relevant part of the presentation at 14:25 in the video.

Cautions when increasing the number of iterations

Increasing the number of iterations may create more opportunities to pass. However, every additional edit increases compute use and waiting time. If the image is edited more than necessary, it may also move too far from the original image; this is over-editing. Therefore, K must be chosen by balancing the pass rate with cost, latency, and faithfulness to the original image. The presentation does not explain the specific K or the numerical tradeoffs.

What this example shows

This sweet-potato-fries example shows a mechanism in which QA is not performed just once: specific feedback about the edit is sent back into the next attempt. Success does not mean merely that the appearance changes. It means meeting semantic conditions such as portion size and plating, then passing QA within the allowed iteration range.