# LLM providers

Add Amazon Bedrock, Anthropic, Google Gemini, Vertex AI or any OpenAI-compatible endpoint, test it, and watch its health.

**AI Gateway › LLM providers** has three tabs:

| Tab | What it holds |
| --- | --- |
| **Providers** | The model endpoints your agents run on (`/api/llm/providers`). This page describes them. |
| **Agent bindings** | Which agent runs on which provider and model, with its fallback lists. |
| **Gateway routes** | Endpoints the Liyagent gateway itself calls for agent-less requests and model prices (`/api/llm-gateway/providers`). Agents do not run on these. |

> [!NOTE]
> To give an agent a model, add the endpoint under **Providers** and bind the agent to it. A gateway route offers Anthropic, AWS Bedrock, Google AI, OpenAI and OpenAI-compatible endpoints, but not Vertex AI, and no agent can be bound to it.

## Provider types

| Type | Use for |
| --- | --- |
| Amazon Bedrock | Claude and other models in your AWS account |
| Anthropic | Anthropic's API directly |
| Google Gemini | Gemini API keys |
| Google Vertex AI | Gemini and partner models in your GCP project |
| OpenAI-compatible | OpenAI, Azure OpenAI, OpenRouter, vLLM, llama.cpp, Ollama: anything that speaks the OpenAI API |

## Add a provider

1. Open **AI Gateway › LLM providers** and choose **+ Add provider** (or open `/providers?new=1`). Pick a type.
2. Fill in the form.

   | Field | Notes |
   | --- | --- |
   | Base URL | For OpenAI-compatible providers, for example `https://openrouter.ai/api/v1` |
   | Endpoint override (optional) | A regional or private endpoint |
   | Credential | A key, or `secret://<label>` from the secrets store |
   | Default model | Used when an agent binds to the provider without naming a model |
   | Data handling | Record the provider's data terms, for example no training on inputs |
   | Enabled | Off keeps it configured but unused |

3. Test before saving: **Test connection** calls `POST /api/llm/providers/test`. **Create provider** saves it. Once saved, the provider shows a **Gateway URL**: set it as an OpenAI SDK's base URL, with an agent token as the key, to call the provider through the gateway.
4. List its models (`GET /api/llm/providers/{name}/models`) and bind agents to them: open the **Agent bindings** tab and choose **Bind** on the agent's row.

> [!TIP]
> `POST /api/llm/providers/discover` lists the models a provider would serve from the details in the form, before it is saved.

## Health and circuits

The providers table shows **Status**, **Default model**, **Health** and **Used by**. A provider's details add its **Circuit**, and its **Leaderboard** tab ranks the callers that use it and the models they reach for.

- Health comes from scheduled probes (`GET /api/llm/provider-health`). Run one now with `POST /api/llm/providers/{name}/health/probe`. A failed probe opens the provider's circuit.
- Each backend has a circuit breaker: closed, open, or half-open with a single probe. Each breaker's state is at `GET /api/llm/breakers`.

## Next steps

- [Model chain and fallback](/docs/ai-gateway/model-chain)
- [Lab: Add a free OpenRouter model to the chain](/docs/labs/openrouter-free-model)