3:48 - 5:43
An MCP gateway that makes tools usable and affordable
Chapter 3 · 03:48–05:43
By the end of this chapter, you should be able to explain why many independently configured tools create access and context overhead. You should also be able to compare the four access patterns described in the talk: direct MCP, Omni MCP, the CLI pattern, and code mode.
Chapter 2's model gateway gives engineers a common path to models. The next problem is giving those models access to the rest of the software system. An agent may need an internal API, a Slack search, or a Jira operation. The speakers describe an MCP gateway as the shared tool-access layer for that problem.
An API is not yet an agent-accessible tool
An API is an interface through which one software system requests data or an operation from another. A human-written program may already know the API's URL, request format, and credentials. An agent needs more than the API's existence. It needs a discoverable description of the operation, a way to invoke it, an authentication path, and a result it can interpret.
The talk uses MCP for the tool-integration mechanism that exposes internal APIs and external tools to agents. The transcript does not expand the acronym. As a teaching model, think of an MCP wrapper or projection as an agent-facing adapter around an API or service. It makes the capability available in a form that an agent can discover and invoke. The speakers do not describe the implementation details of this projection.
The MCP gateway is the common entry point for those integrations. It is the tool counterpart to the model gateway, but the two layers have different jobs:
| Layer | Organizes | Problem it addresses |
|---|---|---|
| Model gateway | Requests to internal and vendor models | Common model access, policy controls, attribution, and traces |
| MCP gateway | Requests to internal APIs and SaaS tools | Common tool installation, shared middleware, token exchange, and context efficiency |
The MCP gateway is therefore not another model. It standardizes how an agent reaches tools. The model gateway standardizes how a client reaches models.
Why direct access creates two kinds of overhead
The speakers describe three problems at the start of this tool journey:
- Uber had thousands of internal APIs, but none were agent-accessible out of the box.
- Its software-as-a-service tools had different authentication methods and setup procedures. The transcript calls these “SAS tools” and names Google, Slack, and Jira. This chapter uses SaaS tools for the intended category.
- Once many MCPs are exposed directly, their descriptions and responses add to the model's context. The speakers call this accumulated cost a token tax.
Here, context means the information supplied to a model while it reasons. It includes instructions, tool descriptions, and tool results. A token is a unit of text used in model processing, so a model has a finite context budget measured in tokens. Context used to describe many unrelated tools is not available for the task itself. A tool schema is the structured description of a tool's inputs and outputs, so schemas can consume context even before the agent calls a tool.
Teacher-created example: suppose an agent only needs one Jira issue and one Slack thread. If it receives descriptions for hundreds of tools, then receives verbose results from the tools it calls, those descriptions and results occupy part of its context. The tools are still useful, but the extra context is the token tax. This is an explanatory scenario, not a request reported by the speakers.
The two kinds of overhead are related but different:
| Overhead | What causes it | What a platform can improve |
|---|---|---|
| Setup overhead | Separate installation and authentication arrangements for separate tools | Provide one common entry point and shared middleware |
| Context overhead | Many tool schemas and tool responses being carried with the model's task | Consolidate discovery or change how selected calls return information |
One gateway for discovery, routing, and token exchange
The speakers describe an MCP gateway that handles common middleware for engineers. The request path can be read as follows:
agent harnesses
↓
MCP gateway: authenticate → discover → schema → route → execute
↓
MCP server → internal API or SaaS tool
An agent harness is the interface or runtime from which an agent uses tools. In this diagram, an MCP server is the tool-side endpoint that handles the request before it reaches the underlying API or SaaS tool; the source does not describe its internals. The gateway sits between the harness and those servers. The stages in the diagram mean that the gateway establishes identity, finds available capabilities, provides their schemas, routes a call to the relevant server, and executes it. The supplied slide also shows identity and authentication, observability, and rate-limit management alongside the request flow.
The slide presents the gateway as a common entry point: multiple agent harnesses feed into one gateway, which authenticates, discovers and schemas tools, routes the request, and executes it toward MCP servers, with identity, observability, and rate-limit management alongside the flow. Source at 4:17
The speakers also say that the gateway hosts the MCPs and performs token exchange. In this context, token exchange means that the gateway participates in obtaining or passing the credentials needed to reach an underlying service. A common entry point does not make authorization unnecessary. The talk does not specify the authorization protocol or the exact permission boundaries.
The gateway gives the crawler a central place to publish integrations. An automated crawler examines Uber's internal APIs and projects them into MCPs with one configuration change, according to the speakers. This does not mean that the transcript explains the crawler's exact conversion process. The same gateway handles SaaS MCPs such as Google, Slack, and Jira. It hosts them and performs token exchange, so engineers get one common way to install MCPs instead of a separate setup for every service.
Teacher-created trace: imagine an internal service that already exposes a delivery-status API. In the talk's model, the crawler could project that API into an MCP after the stated configuration change. An agent harness would then reach the MCP through the gateway, rather than learning the service's setup independently. If the agent instead needs Slack, the request follows the same gateway path to the Slack MCP. The service name and operation here are invented; the source does not specify the crawler's implementation or permission rules.
Four ways to reduce the token tax
The common gateway reduces installation friction, but it does not by itself solve context growth. The speakers describe a progression of access patterns that changes how agents discover and invoke tools.
The frame provides direct visual evidence for a comparison of MCP access patterns and a proposed gateway architecture: callers and interfaces are shown above a gateway that handles authentication, discovery, schema, routing, and execution, with management and infrastructure layers beneath. Source at 5:00
| Pattern | What the speakers describe | Effect on setup or context |
|---|---|---|
| Direct MCP | The initial pattern: an agent uses MCPs directly. | Each MCP is a separate integration, and direct exposure can add tool schemas and results to the model context. |
| Omni MCP | One installed MCP discovers and invokes any MCP available inside the gateway. | Discovery and installation are consolidated into one entry point. The underlying tools keep their different meanings. |
| CLI pattern | The gateway projects MCP capabilities into a command-line-oriented pattern. CLI means command-line interface. | The speakers say this keeps a tool response from consuming as much model context. |
| Code mode skill | An automatically installed skill creates Python scripts on the fly for selected high-token-consuming MCP use cases. | Script generation targets particular expensive uses instead of exposing every result in the same direct form. |
These patterns are easier to distinguish when followed in order.
Direct MCP is the baseline
With direct MCP, each integration is available as its own MCP. That can be straightforward when the tool inventory is small. As the inventory grows, an engineer must deal with more installations and authentication arrangements, while the model may receive more tool descriptions and results.
Omni MCP consolidates discovery
Omni MCP is the next consolidation step described by the speakers. An engineer installs one MCP. That MCP can discover and invoke other MCPs inside the gateway. The single entry point does not make a Jira operation, a Slack search, and an internal API operation semantically identical. It only centralizes how the agent discovers and invokes them.
The CLI pattern changes the access form
The speakers then describe projecting the MCPs into a CLI pattern. The stated purpose is to keep a tool response from eating as much of the model's context. This changes how the agent reaches the capability. It does not imply that the underlying API or SaaS service has changed.
Code mode targets selected expensive uses
Finally, the speakers describe a code mode skill that is autoinstalled. It creates Python scripts on the fly to optimize some of the MCP use cases that consume the most tokens. This is more targeted than placing every tool description and every raw result in the model's context. The transcript does not define the skill's interface or the exact cases it selects.
Omni MCP mainly addresses consolidated discovery and installation. The CLI pattern and code mode explicitly address how much tool information reaches the model. They are complementary strategies, not four claims that the underlying tools have become one kind of service.
Interpreting the reported savings
The speakers say that Uber now has more than 1,000 MCP tools. They also report “more than 40% fleetwide savings” from the optimization efforts. The source does not define what the savings measure, its denominator, its baseline, or its scope.
That figure should not be restated as a specific reduction in token cost, dollar cost, latency, or any other single metric. The narrower supported reading is that, as the tool inventory grew, Uber used gateway-level consolidation and several context-optimization patterns, and the speakers associate those efforts with a reported fleetwide savings figure. It is not a universal guarantee for another MCP gateway.
The MCP gateway's place in the software factory
The MCP gateway contributes two capabilities to the larger platform. First, it gives engineers a common path for installing and reaching internal APIs and SaaS tools. The crawler extends that path from existing internal APIs to agent-accessible MCPs, while shared middleware and token exchange keep the access path centralized.
Second, the gateway gives the platform several ways to control context growth. Omni MCP consolidates discovery. The CLI pattern changes the response path. Code mode generates scripts for selected high-token-consuming cases. Later skills, context retrieval, and the Cortana assistant can build on this common tool layer rather than each inventing its own integrations.
Check your understanding
Why can an agent not use every internal API simply because the API already exists?
Answer: An agent also needs a discoverable tool description, an invocation format, a credential path, and an interpretable result. The crawler's MCP projection and the gateway's common path address that access problem. The source does not specify the projection's implementation.
How do Omni MCP and code mode differ?
Answer: Omni MCP is one installed MCP that discovers and invokes other MCPs inside the gateway. Code mode is an automatically installed skill that creates Python scripts for selected high-token-consuming uses. Omni MCP centralizes discovery and invocation; code mode changes how particular uses are carried out.
Synthesis
The model gateway organizes model requests; the MCP gateway organizes tool access. Direct MCP exposes each integration separately. Omni MCP consolidates discovery, the CLI pattern reduces response context according to the speakers, and code mode scripts selected expensive uses. The reported savings remain speaker-reported and undefined in scope, but the design problem is clear: a large tool inventory needs both a common setup path and deliberate control of model context.
Source boundary: The speakers state the existence and purpose of the MCP gateway, its crawler, common hosting and token exchange, the four access patterns, the count of more than 1,000 MCP tools, and the reported “more than 40% fleetwide savings.” The transcript does not expand MCP, explain the crawler's exact conversion or authorization behavior, or define the savings metric. The request diagrams, token-tax scenarios, tables, and comparison explanations above are teaching aids grounded in the source and supplied visuals.