LLM Integration & APIs

How to Build a Multi-Provider LLM Client for Agents

10/22/2026

An agent that depends directly on one provider’s request and response format can be awkward to adapt when you want to try another model. The useful first step is to separate the agent’s tool-dispatch logic from the provider-specific details.

This walkthrough builds that boundary around one concrete example: Gemini’s Python API. You’ll define a normalized result for assistant text and tool calls, then keep Gemini’s request formatting, response parsing, and continuation history inside an adapter.

The core loop can dispatch a tool by its name without needing to understand Gemini’s response structure.

There’s one important limit: this establishes the foundation for a multi-provider client, but does not implement a second provider or cover streaming. Gemini’s tool calls and follow-up history also have provider-specific structure, so preserving that state matters as much as extracting readable text.

Quick answer

The short version: define an internal reply containing assistant text and tool calls, with each call carrying its function name, arguments, and call ID. Put Gemini’s request construction and response parsing in an adapter, and keep the core loop responsible for dispatching local functions.

For the Python setup, install the Google GenAI SDK, configure GEMINI_API_KEY outside your source code, and initialize the client with genai.Client(). If both GOOGLE_API_KEY and GEMINI_API_KEY are set, Google documents that GOOGLE_API_KEY takes precedence.

When Gemini returns a tool call, parse it from the candidate’s content parts and map it into your internal representation. After the local function runs, have the adapter format the result as a Gemini function response and preserve the original model response in the follow-up history.

Don’t send only a flattened text transcript if the next request needs provider-specific continuation state.

Finally, smoke-test both ordinary text and a tool cycle. Check that the parsed call has the expected name, arguments, and call ID, and that the follow-up history retains the model response and tool result.

This walkthrough covers the Gemini adapter foundation, not a second provider or streaming.

Step-by-step guide

Prerequisites and starting state

Have a Python development environment ready, along with a Gemini API key. This walkthrough uses Google’s Gemini API through its Python SDK, so it does not assume that other providers share Gemini’s request, tool-call, or conversation-history formats.

Keep the agent’s local tool functions available to your application: the core loop will need to call a function after the model requests it. You do not need to change those functions to match Gemini’s response schema; the adapter will handle that translation.

Plan to provide the API key through an environment variable rather than putting it in source code. Google’s Python client recognizes GEMINI_API_KEY; if GOOGLE_API_KEY is also set, Google documents that it takes precedence.

The starting point is an agent loop that can dispatch a local function and pass its result onward. The adapter will preserve Gemini’s original response and continuation state, so a plain-text transcript alone is not a sufficient substitute for the provider-specific history.

Sources: Google AI for Developers: Get started with the Gemini API; Google AI for Developers: Function calling with the Gemini API; Google AI for Developers: Using Gemini API keys.

Define the provider-neutral result the agent loop consumes

Keep the agent loop focused on what it needs: assistant text and tool calls it can dispatch. Give each tool call a name, arguments, and call ID so the adapter and loop can track the request and its result.

from dataclasses import dataclass

@dataclass
class ToolCall:
    name: str
    arguments: dict[str, object]
    call_id: str

@dataclass
class AgentReply:
    text: str
    tool_calls: list[ToolCall]

Treat these as your application’s internal types, not Gemini’s API schema. The Gemini adapter should translate its response into this shape while keeping the original model response and any continuation data in provider-specific history; don’t flatten that history into text alone.

This boundary lets the core loop dispatch tools without depending on Gemini’s response format.

Sources: OpenAI API Reference: Responses API output and function-call items; Google AI for Developers: Get started with the Gemini API; Google AI for Developers: Function calling with the Gemini API; Google AI for Developers: Using Gemini API keys.

Install the Gemini SDK and configure credentials

Install the Google GenAI SDK in the Python environment your agent uses:

python -m pip install -U google-genai

In Bash on Linux or macOS, enter your key when prompted; it will not be displayed, and it stays in the current shell session:

read -s -p "Gemini API key: " GEMINI_API_KEY; export GEMINI_API_KEY; printf '\n'

On Windows, add GEMINI_API_KEY through the Environment Variables settings, then open a new terminal. Keep the key out of your source code; if both GOOGLE_API_KEY and GEMINI_API_KEY are set, the client gives GOOGLE_API_KEY precedence.

Initialize the client in Python:

from google import genai

client = genai.Client()

The SDK detects the configured environment variable, so you do not need to pass the key in this code.

Implement the Gemini request and response adapter

Keep Gemini’s request and response details inside one adapter. The agent loop should receive the normalized AgentReply from the previous step, while the adapter retains the original Gemini response for continuation.

class GeminiAdapter:
    def __init__(self, client, model):
        self.client = client
        self.model = model

    def generate(self, history, config=None):
        response = self.client.models.generate_content(
            model=self.model, contents=history, config=config
        )
        text, tool_calls = [], []
        for candidate in response.candidates or []:
            if not candidate.content:
                continue
            for part in candidate.content.parts or []:
                if part.text:
                    text.append(part.text)
                call = part.function_call
                if call:
                    tool_calls.append(ToolCall(
                        name=call.name,
                        arguments=dict(call.args or {}),
                        call_id=call.id,
                    ))
        return AgentReply("\n".join(text), tool_calls), response

Have the adapter build and retain history in Gemini’s content-and-parts format; don’t reduce the returned response to text before a follow-up. This parser collects ordinary text and maps each function call to the internal name, arguments, and call ID, leaving tool execution to the core loop.

Connect tool dispatch and format the follow-up

Dispatch each normalized call by name, and pass its arguments to the matching local function. Keep Gemini-specific history updates behind the adapter: it should append the original model response, then encode each result as a Gemini function-response part carrying the function name and call ID.

results = []
for call in reply.tool_calls:
    tool = tools.get(call.name)
    if tool is None:
        raise ValueError(f"Unknown tool: {call.name}")
    results.append((call, tool(**call.arguments)))

adapter.append_tool_results(history, model_response, results)
reply, model_response = adapter.generate(history)

Implement append_tool_results inside the adapter, not the agent loop. This keeps provider-specific serialization out of dispatch and preserves the model response needed for Gemini continuation; the loop only sees normalized calls and local results.

If no tool calls were returned, skip dispatch and follow your loop’s normal completion path.

Verify the adapter and tool cycle

Run a simple prompt through the configured adapter and check that it returns readable text. Use a prompt with a predictable response, then confirm reply.text is non-empty and that the adapter returns the raw Gemini response separately for continuation.

reply, model_response = adapter.generate(history)
assert reply.text.strip()
assert model_response is not None

Next, make a tool-enabled request using a tool your agent already supports. Confirm the normalized call has the expected function name, arguments, and call ID, then verify the adapter’s follow-up history contains the original model response and a Gemini function-response part with the result and matching call ID.

If the model does not return a tool call, check that the selected model supports the tool capability and that the request includes the tool configuration. Don’t treat plain-text output alone as proof that the tool cycle works; verify the follow-up state as well.

Troubleshoot credentials, tool support, and continuation state

If the client seems to use the wrong API key, check which supported environment variables are set without printing their values. Google documents that GOOGLE_API_KEY takes precedence over GEMINI_API_KEY; remove or correct the unintended variable, start a fresh process, and rerun the smoke test.

If a tool call does not appear, check whether the selected model supports the tool capability you need. Then inspect the response candidate parts: a call the adapter can dispatch should include a function name, arguments, and call ID.

If the follow-up loses context, check the history before sending the next request. It should retain Gemini’s original model response and include the function result in Gemini’s function-response format with the matching call ID; plain text alone may not preserve the continuation state.

Next steps

The key design choice is to normalize what the agent loop needs without discarding what the provider needs. I’d keep the internal tool-call representation small and explicit, while letting each adapter own its request format, response parsing, and continuation history.

Before relying on the Gemini adapter, verify a text response and a complete tool cycle. Confirm the call’s name, arguments, and ID survive parsing, then check that the follow-up includes Gemini’s original model response and a correctly associated function result.

If a tool call does not appear, check the selected model’s supported capabilities and the request configuration.

This is a useful foundation, not a finished multi-provider client. When you add another provider, give it its own adapter and validate its tool-call and continuation behavior rather than assuming Gemini’s formats carry over.

That keeps the agent loop steady while making provider-specific differences visible where they belong.