In the previous chapter, we saw a loop that decides how many times to enhance an image and sends the quality-check result back into the next edit. This chapter evaluates whether the edited result really became better than the input. The presenters explain that an enhancement evaluation can use pairwise comparison, which looks at the input image and output image together.
Pairwise comparison does not give a score to the output image alone. It treats the original input image and the edited or enhanced output image as one pair. It then judges the relationship: is the output better than the input? In other words, the object of evaluation is not the image by itself, but the difference created by the change.
This difference matters. Even if a single output looks polished, it may not be an improvement for the product if it changes the original dish or makes the edit look unnatural. Using the input as a reference checks not only whether the appearance changed, but also whether the change serves its purpose.
Figure: A slide defines generation evaluation for editing and enhancement tasks as a pairwise comparison between the input or original image and the output or generated image.
This frame shows a slide about the idea of pairwise comparison, not a concrete comparison of food images. It directly supports the explanation that the input and output are compared. The relevant explanation appears at 00:15:04 of the video.
Here, “better” is not a simple image-quality score decided only by the model. According to the presenters, the evaluation incorporates a definition of quality agreed with product, design, policy, and legal stakeholders. This makes the eval represent the image behavior the product wants, rather than a purely technical preference.
主な観点は、次のように分けて考えます。
The main dimensions can be considered separately:
忠実性（faithfulness）：入力画像の内容や意味を保っているかを見ます。
完全性（completeness）：元画像にある重要な内容を、出力が失っていないかを見ます。
自然さ（naturalness）：編集の結果が、不自然な加工に見えないかを見ます。
現実らしさ（realism）：画像全体が、現実の写真として納得できる状態かを見ます。
Faithfulness: Does the output preserve the content and meaning of the input image?
Completeness: Has the output lost important content that was present in the original image?
Naturalness: Does the edit avoid looking like an unnatural manipulation?
Realism: Does the image as a whole remain convincing as a real photograph?
These dimensions are related, but they do not mean the same thing. For example, an image may be made brighter and look more natural while still losing faithfulness if it adds something that was not in the original. Conversely, it may preserve the original content but still have a problem with naturalness if the composition or edit looks unnatural. Improving one dimension does not necessarily make the whole output better.
判定は yes・no・unsure で残す
Keep the judgment as yes, no, or unsure
ペア比較の結果は、発表者の説明では次の三つです。
The presenters describe three possible pairwise-comparison results:
yes：出力が入力よりよいと判断します。
no：出力が入力よりよいとは判断しません。
unsure：手元の情報や画像だけでは、自信を持って判断できません。
yes: Judge that the output is better than the input.
no: Do not judge that the output is better than the input.
unsure: The available information or images do not support a confident judgment.
unsure は、無理に yes か no に変えないための重要な結果です。評価者が確信できないケースを二値のどちらかへ押し込むと、実際には弱い根拠しかない判定が、確かな判定として扱われます。不確かさを残せば、後で人が確認する、または安全側の運用に回す、といった判断につなげられます。具体的な運用ルールは、この発表では詳しく示されていません。
unsure is important because it prevents the evaluator from forcing a choice between yes and no. If uncertain cases are pushed into one of two categories, a judgment with weak evidence can be treated as certain. Keeping the uncertainty makes it possible to send the case for later human review or to use a safety-oriented policy. This presentation does not explain the specific operating rule.
似ている評価との違い
How this differs from similar evaluations
ペア比較は、システム内の別の評価と役割が違います。
Pairwise comparison has a different role from other evaluations in the system.
Separating these stages makes it possible to diagnose a router-selection error, insufficient improvement in the edit, and a pre-publication safety problem separately. A pairwise yes is not an automatic permission to publish.
例：一つの改善が、全体の改善とは限らない
Example: one improvement is not necessarily an overall improvement
Example (for explanation): Consider an output that makes a dark area brighter while preserving the dish in the input image. If the brightness improves, the dish's content stays unchanged, and the edit looks natural, it becomes easier to judge yes. But if the brightening removes part of the dish or adds a decoration that was not in the original image, it fails another dimension. This is a teacher-created example for explaining pairwise comparison.
The key point is not to ask only whether the output looks good. The question is whether the change from input to output satisfies multiple quality conditions defined by the stakeholders and is desirable in relation to the original image.
The presenters say that the detailed implementation of pairwise evaluation is proprietary and was intentionally omitted. Therefore, the reliable points in this chapter are the comparison of input and output, the high-level quality dimensions, and the yes/no/unsure result format. The specific prompts, data format, thresholds, and additional criteria are not available from this source.
Pairwise comparison asks an editing model not merely whether it made a change, but whether it made a desirable change that serves its purpose and improves on the input. By defining quality with stakeholders and keeping uncertain judgments uncertain, the system can examine quality and safety together.