Google Opens Gemini API Tool Retrieval Preview for Large Catalogs
Google’s preview puts an illustrative 200-tool Gemini API case at 23,650 input and tool-use tokens, versus 164,000 static tokens, but search can erase savings for small libraries.
By Nathan Brooks
4 min read

Google’s setup guide puts an illustrative 200-tool Gemini API workload at 164,000 static input tokens versus about 23,650 input and tool-use tokens with Tool Retrieval. Google announced the feature in preview on November 19, 2026, through the Gemini Interactions API. The company says the feature lets agents keep large tool libraries on standby and load full parameter schemas only when the current task requires them.
Google says the conventional approach becomes unwieldy when agents connect to dozens or hundreds of tools. Sending every full schema inflates input-token counts, adds latency and reduces tool-selection accuracy as the candidate list grows.
In Google’s design, Tool Retrieval initially keeps only each deferred tool’s name and description in context. Gemini withholds the schema defining the function’s arguments until the model asks for it.
Google says the Gemini Interactions API became generally available in June 2026 and is its recommended interface for new projects. Tool Retrieval works exclusively through that interface, and the older generateContent method does not support it.
Server-side retrieval
In the default server-side mode, developers add a tool_search tool and set defer_loading: true on selected function tools. Frequently used functions can stay fully loaded, while specialized ones initially contribute only their names and descriptions.
When the model needs a deferred tool, one API call can contain a tool_search_call, a tool_search_result carrying the full schema and the resulting function_call. Google leaves those execution steps visible for debugging.
For a custom function, the developer’s code still executes it and returns a function_result. Google performs the server-side search, so developers need no retrieval index to build or host.
Setting defer_loading on a remote MCP server defers every tool that server exposes, so a large server does not require each tool to be listed individually. Google inserts retrieved schemas where they load rather than at the start of the prompt, preserving the earlier prefix for implicit caching.
Client-side retrieval
Client-side retrieval transfers search, ranking and filtering to the developer’s application. It adds one round trip per search, but allows tools to enter mid-conversation without upfront declaration, such as when a user connects a new integration.
Server-side and client-side retrieval cannot be combined within a request. Exactly one tool_search definition is allowed.
Token economics
Google charges no separate fee for Tool Retrieval, though the feature changes token billing rather than eliminating tool overhead. Server-side search instructions add a small, fixed input block to every request. Google’s setup guide reports early testing produced “about 26 tokens per deferred tool” for the compact name-and-description entry.
Every schema Gemini loads remains billable as input on later calls and turns. Each server-side search also launches another model call with the full context and newly loaded schemas; Google reports that prompt as tool-use tokens and charges it at the input rate.
The guide’s illustrative two-request example uses round numbers and excludes output tokens and caching. With 200 tools of 400 tokens each and two tools loaded, static loading produces 164,000 input tokens, compared with about 23,650 input and tool-use tokens using retrieval.
In a second scenario with 10 tools of the same size and a 30,000-token context, static loading totals 68,000 tokens. Retrieval totals about 92,830, because repeating a large context for search can outweigh the savings from a small tool library.
Model and operating limits
Google’s guide explicitly lists Gemini 4 Argon, Gemini 3.8 Flash and Gemini 3.6 Flash as supported models. A request that sets defer_loading on an unsupported model returns a 400 error.
Google cautions that the preview API surface may change before general availability. Retrieved tools cannot be unloaded during a conversation, and the complete tools array must be resent on every request.
Turns chained with previous_interaction_id face the same resend requirement because tool declarations are not stored between requests.
What does this change for API teams?
Google’s broader function-calling guide recommends considering retrieval once a library passes “roughly 20 to 30 tools.” Tool Retrieval now lets teams keep much larger catalogs on standby, while search frequency and retained schemas determine whether the extra capability reduces or increases token use.
For practitioners, the feature is not a blanket cost cut. It should pay off when selective loading saves more than repeated search calls consume; teams must also account for resending the tools array and retaining loaded schemas for the life of each conversation. The preview gives teams a way to test that tradeoff before Google makes the API generally available.
Source: unite.ai
More from Nathan Brooks
Show full bio
News editor covering regulation, policy and their impact on entrepreneurship, startups and venture funding at Business Bearings.
2,524 articles