MCP Gateway and Token-Efficient Tool Access

Chapter 3 · 3:48–5:43

An agent is useful only when it can reach the systems needed for its task. At Uber, those systems include internal APIs and hosted SaaS services. The speakers describe an MCP Gateway as one authenticated entry point for these different tool servers.

The gateway solves the access problem. It gives harnesses a common way to reach first-party and third-party capabilities. A second problem remains: if an agent can use more than 1,000 tools, showing every tool definition and every result to the model can consume a large amount of context. Uber therefore also develops caller patterns that make tool access more selective and token-efficient.

Source note: The speakers report more than 1,000 tools and over 40% fleetwide token savings after deploying several access patterns. The savings figure is an aggregate claim from the talk. It does not, by itself, specify the baseline, workload mix, or the contribution of each individual pattern.

MCP is a protocol or tool interface. It gives an agent harness a standard way to discover and invoke capabilities. It is not a model, and it does not generate the answer itself.

An MCP server exposes a particular set of operations. One server may represent an Uber internal API. Another may connect to a hosted SaaS product. The underlying systems remain different, even when they are presented through the same interface.

The MCP Gateway is the shared access layer in front of those servers. It authenticates the caller and coordinates access to the available tools. This distinction matters:

Layer	Question it answers
Model	How should the task be reasoned about or answered?
MCP interface	How can the harness discover and invoke a tool?
MCP Gateway	Which caller may reach which tool, through what governed path?
Internal API or SaaS service	What operation is actually performed?

The gateway does not turn every backend into the same product. It creates a controlled route to heterogeneous systems.

Turning existing APIs into discoverable tools

The speakers also describe automated projection. Here, projection means making capabilities from existing APIs available through the MCP tool surface without requiring every caller to design a one-off integration. The source names this capability but does not specify its implementation details.

This matters because the gateway is not useful only for systems built expressly for agents. It can sit in front of a changing collection of internal APIs and hosted SaaS MCPs. Shared discovery helps a harness find what is available. Token exchange provides the delegated credentials needed for an authorized call. Together, these mechanisms make the catalog usable as a platform rather than as a pile of unrelated integrations.

The speakers describe a path like this:

flowchart LR
    H[Agent harness] --> G[Authenticated MCP Gateway]
    G --> I[Internal API MCP servers]
    G --> S[Hosted SaaS MCP servers]
    I --> IA[Uber internal systems]
    S --> SA[External SaaS systems]

At 4:41, the slide shows one MCP Gateway connecting harnesses to first- and third-party servers. The common entry point is visible; token exchange is supported by the source discussion rather than by a visible label in this frame.

The value of the common entry point is not that all tools have identical behavior. The value is that callers do not need a separate access design for every internal API and SaaS integration. Authentication, discovery, and token exchange can be handled as shared platform concerns rather than rebuilt in each harness.

Teaching example: internal and SaaS capabilities together

Suppose an agent needs one internal service and also needs capabilities associated with Google, Slack, and Jira. These are distinct systems with distinct APIs. Without a common access layer, the harness would need separate integration and credential paths for each one.

With an MCP Gateway, the agent can use one governed tool surface while the gateway reaches the appropriate first-party or third-party server. The systems remain distinct behind the surface. The gateway standardizes the route, not the business meaning of every operation.

Tool access has a cost beyond the operation itself. Before a model can choose a tool, it may need to receive information about the available tools. Tool definitions consume model context, and tool results consume context too. This is the tool-context tax.

The tax becomes important when the catalog is large. A catalog with more than 1,000 tools creates a poor default if every request includes the full catalog:

1. The harness sends a large list of tool descriptions.
2. The model spends context on tools that are irrelevant to the current task.
3. The model chooses or invokes one tool.
4. The result is added to the context for the next reasoning step.

Only the last two steps directly advance the task. The first step may still be necessary in some designs, but sending the whole catalog repeatedly is often wasteful.

Teaching example: one repository query

Imagine an agent that can potentially use more than 1,000 tools but needs only one repository query for the current task. The efficient design does not give the model the full catalog and ask it to ignore almost everything. It selectively discloses the relevant capability and then returns the result needed for the next step.

This is the key optimization:

Give the model the capability and result it needs, rather than the entire universe of possible tools.

This does not mean hiding tools from the system forever. It means separating catalog access from task-time context. The platform can maintain a broad catalog while a particular reasoning step sees a narrow working set.

It is tempting to treat the gateway as the complete solution. It is not. Two related but separate design questions are involved:

• Access and governance: Can this harness authenticate and reach the requested server through a controlled path? The shared MCP Gateway addresses this question.
• Context efficiency: Which tool definitions and results should enter this model interaction? Caller patterns such as direct MCP, Omni MCP, and code mode address this question in the talk.

A gateway can centralize authentication without automatically minimizing context. Conversely, a caller can reduce the tools shown to a model without replacing the need for governed access to internal and SaaS systems.

The architecture therefore has two goals:

Many heterogeneous systems
            │
            ▼
   Common authenticated access       ← gateway responsibility
            │
            ▼
   Selective task-time disclosure     ← caller/context responsibility
            │
            ▼
   Fewer unnecessary tokens and turns

The exact implementation details of the named patterns are not established by the visible slide alone. What the chapter establishes is their role in the broader design: they are ways of accessing MCP capabilities while reducing unnecessary context cost.

The patterns shown in the talk

Near the end of this chapter, the speakers discuss direct MCP, Omni MCP, CLI, and code-mode patterns. The visible slide names direct MCP, Omni MCP, and code mode; the CLI is part of the source's spoken account of the access patterns.

At 5:24, the retry confirms the named direct MCP, Omni MCP, and Code Mode patterns. It does not visually establish implementation details such as a particular programming language.

Pattern named in the talk	Role in the chapter's mental model
Direct MCP	A caller can work with MCP capabilities directly.
Omni MCP	A later MCP access pattern intended to improve how capabilities are presented or used.
CLI	A command-line access pattern for working with the tool platform.
Code mode	A caller pattern associated with making tool use more efficient for the model.

These labels should not be read as different models. They are access or execution patterns around tools. The important comparison is not which label is newest. It is whether a pattern avoids putting unnecessary tool definitions and intermediate results into the model's context.

Reading the reported savings carefully

The speakers report over 40% fleetwide savings. “Fleetwide” means the claim summarizes usage across the deployed collection of callers or workloads, rather than describing one isolated request. The chapter does not provide a complete experimental baseline or a breakdown that lets us assign the savings to one pattern.

The safe conclusion is narrower and more useful:

1. Tool access was deployed at a large catalog scale.
2. Uber used multiple access patterns to control context use.
3. The speakers report a substantial aggregate reduction in token consumption.

The claim is not evidence that every task becomes 40% cheaper. Savings depend on how many tools a task needs, how often tools are described, how large results are, and which caller pattern is used.

Why this layer matters to the software factory

The broader lesson is that agents do not scale by adding models alone. A software-delivery platform must also make the surrounding capabilities usable and governable.

The MCP Gateway contributes a reusable access foundation:

• internal APIs and hosted SaaS MCPs can be reached through one authenticated gateway;
• discovery and token exchange become shared platform capabilities;
• a large tool catalog can be exposed without requiring every model request to carry the entire catalog;
• caller patterns can reduce the context spent on irrelevant tools and results.

This is the difference between “the agent has many integrations” and “the agent can use many integrations predictably.” The first describes capability count. The second describes a system that controls access and manages the cost of reasoning over that capability.

Boundary to the next chapter

The MCP Gateway makes tools reachable and helps make their use efficient. It does not provide the runtime in which a long-running agent performs repository work. The next chapter turns to that separate layer: pre-provisioned execution environments for isolated, parallel agent work.