Putting ontology in the tool-result validation loop

Video segment: 16:34–17:46

The previous chapter showed how an agent can call a tool and place the tool's result back into its running context. This chapter adds the missing reliability step: check the result before allowing the loop to continue.

The speaker points to the red part of the example as the place where an ontology can enter. The ontology does not replace the language model, and it does not replace the tool. It surrounds the tool loop with domain-aware checks.

The basic idea

An LLM can propose a tool call and interpret the result. A tool can return information or perform an operation. Neither fact alone tells us that the result makes sense for the domain.

For example, a tool might return text that is well-formed but still unreasonable. It might describe an entity with the wrong type, report a relationship that violates a domain rule, or return a value that the domain does not allow. A validator needs more than the shape of the text to detect these problems. It needs expectations about the domain.

An ontology supplies those expectations. In the talk's architecture, the surrounding program does the following:

1. The agent requests a tool call.
2. The program checks the response's control information, such as stop_reason.
3. The program runs the tool when the response asks for tool use.
4. The tool returns information.
5. The information is put into a form that a validator can inspect.
6. The validator applies the domain ontology.
7. The program accepts the result, or recovers when the result is unreasonable.

The important insertion point is between receiving the tool information and accepting it as usable context or acting on it.

Click to enlarge

Source visual: The slide shows the tool running at run_tool(call) between an input gate and an output gate. It labels Pydantic at the input side and an ontology at the output side. This places ontology-backed checking after the tool has produced its result.

Why the result must be checked

It is tempting to treat a tool result as automatically trustworthy because it came from a program rather than directly from the LLM. That is an incomplete model.

The tool may have executed correctly while still returning information that does not fit the agent's intended domain state. The result can also expose a problem in the model's proposed call, in the tool's interpretation of that call, or in the surrounding data. The loop therefore needs a decision point after execution.

The validator asks a domain question:

Given what this system knows about the domain, is this result reasonable?

Here, reasonable does not mean “grammatically clear” or merely “parseable as JSON.” It means that the information is compatible with the entities, relationships, types, and constraints represented by the ontology. This is teaching context that makes the speaker's distinction explicit; the speaker describes the validator as applying ontologies about the domain to judge the LLM response.

The difference can be stated as two levels of checking:

Check	Main question
Structural or input checking	Do the request and its parameters have the expected form?
Ontology-backed result checking	Does the returned information make sense under the domain model?

The second check is valuable because a response can pass the first level and fail the second. A value can have the right data type while still violating a relationship or domain rule.

A more precise loop

The following pseudocode is a teaching reconstruction of the architecture described in the talk. It shows the decision points without claiming to reproduce the speaker's exact implementation:

context = [user_prompt]

while True:
    response = ask_model(context, tools)

    if response.stop_reason == "tool_use":
        call = extract_tool_call(response)

        # A separate input gate may check the proposed call.
        result = run_tool(call)

        checked_result = make_validator_input(result)
        decision = ontology_validator.check(checked_result)

        if decision.is_reasonable:
            context.append(tool_result(call, result))
        else:
            context.append(validation_feedback(decision))
            # The next model turn can revise the proposal.
    else:
        print(response.text)
        break

The key line is not run_tool(call) by itself. It is the check that follows it. The tool result becomes input to the validator, and the ontology gives the validator a formal reference for judging it.

What happens when validation fails?

The speaker names two recovery paths.

1. Send feedback to the LLM

The program can return information about the failure to the model. The next turn then has a chance to revise its proposal or choose a different action.

This creates a corrective loop:

model proposes
    → tool runs
    → ontology check fails
    → feedback goes to model
    → model proposes again

The model is still probabilistic. Ontology validation does not make it infallible. It gives the surrounding program a reasoned checkpoint and a way to tell the model that the current path is not working.

2. Bring in a human

The program can also put a human in the loop. This is useful when the system cannot establish that a result is reasonable, or when retrying would be inappropriate.

The ontology is therefore a guardrail, not a guarantee. A guardrail can detect a violation according to the rules it knows. It cannot prove that every possible failure has been represented in those rules. If the check fails, the system needs a policy for feedback, stopping, or escalation.

Click to enlarge

Source visual: The slide shows a tool-use stop_reason leading to a tool call and a result being added to the running context. It also shows separate input and output validation gates. The visible slide shows the validation points around the tool loop; it does not show a human handoff.

How earlier ontology ideas support this check

Earlier examples in the talk explain what the validator might use.

Inferred types

With RDFS-style domain and range information, a relationship can imply types. In the speaker's example, if Bob teaches Scooter, the relationship's domain can support the inference that Bob is a teacher, while its range can support the inference that Scooter is a student. If teachers are persons, Bob can also be inferred to be a person.

When a tool result adds a relationship, the validator can use the same assumptions to see whether the subjects and objects have compatible roles. The result is not judged only as a string. It is interpreted as a statement inside a typed domain model.

Property behavior and constraints

The talk also presents OWL property characteristics. A transitive ancestor relationship can derive a longer relationship from two shorter ones. A functional property can allow only one value, such as one father or mother value in the speaker's example. These rules can reveal consistency problems or support identity inferences.

Such rules matter in a loop because the agent may propose an action whose consequences conflict with information already in the graph. The validator can compare the new result with the domain's existing relationships and constraints before the loop treats it as accepted information.

This is the connection between the abstract ontology and the practical agent:

ontology vocabulary
    → types, relationships, and constraints
    → reasoner or validator
    → decision about a tool result
    → continue, retry, or escalate

Input checks and output checks are different

The visual presents two gates. They should not be confused.

An input gate examines what the model is asking the tool to do. It can check whether the proposed call has the expected form. An output gate examines what the tool returned. The speaker assigns ontology-based validation to this result side in the slide's architecture.

These checks solve different problems:

• An input check can catch a malformed parameter.
• An ontology check can catch a result that is structurally readable but semantically inconsistent.

For instance, a request may contain a correctly formatted identifier, yet the resulting operation may still produce a state that conflicts with the domain model. Passing the input gate is not permission to skip the output gate.

The next chapter develops this division using the speaker's phrase “Pydantic at the door, ontology at the ledger.” This chapter's narrower point is the location and purpose of the ontology: it checks the meaning of information after the tool interaction, before that information is allowed to guide the next autonomous turn.

The complete mental model

It helps to separate the roles clearly:

Component	Role in the loop
LLM	Generates a proposed response or tool call and interprets context probabilistically
Client program	Controls the loop and chooses which branch to execute
Tool	Retrieves information or performs the requested external operation
Ontology	Represents domain concepts, relationships, and applicable constraints
Validator or reasoner	Uses that domain representation to judge the result
Recovery policy	Accepts the result, asks for another model attempt, or involves a human

The agent becomes more dependable through this division of labor. The LLM supplies flexible generation and planning. The tool connects the loop to external information or action. The ontology and validator provide a domain-aware checkpoint. The surrounding program decides what to do when the checkpoint passes or fails.

Takeaway

The speaker's proposal is to put ontology-backed validation inside the agent loop, after tool output arrives and before the result is accepted. The validator turns domain knowledge into a decision point: continue when the result is reasonable, return feedback to the LLM when it is not, or bring in a human.

This does not remove probabilistic behavior from the agent. It adds formal checks around that behavior. The result is a loop in which the model can propose and explore, while the domain model helps keep the system on track.