TICKETIQ

LLM providers Preview

Add Amazon Bedrock, Anthropic, Google Gemini, Vertex AI or any OpenAI-compatible endpoint, test it, and watch its health.

After reading this page you will be able to:
  • Add and test a provider before saving it
  • Keep its key in the secrets store
  • Read provider health and the circuit state

AI Gateway › LLM providers has three tabs:

TabWhat it holds
ProvidersThe model endpoints your agents run on (/api/llm/providers). This page describes them.
Agent bindingsWhich agent runs on which provider and model, with its fallback lists.
Gateway routesEndpoints the Liyagent gateway itself calls for agent-less requests and model prices (/api/llm-gateway/providers). Agents do not run on these.
Note

To give an agent a model, add the endpoint under Providers and bind the agent to it. A gateway route offers Anthropic, AWS Bedrock, Google AI, OpenAI and OpenAI-compatible endpoints, but not Vertex AI, and no agent can be bound to it.

Provider types

TypeUse for
Amazon BedrockClaude and other models in your AWS account
AnthropicAnthropic's API directly
Google GeminiGemini API keys
Google Vertex AIGemini and partner models in your GCP project
OpenAI-compatibleOpenAI, Azure OpenAI, OpenRouter, vLLM, llama.cpp, Ollama: anything that speaks the OpenAI API

Add a provider

  1. Open AI Gateway › LLM providers and choose + Add provider (or open /providers?new=1). Pick a type.
  2. Fill in the form.
    FieldNotes
    Base URLFor OpenAI-compatible providers, for example https://openrouter.ai/api/v1
    Endpoint override (optional)A regional or private endpoint
    CredentialA key, or secret://<label> from the secrets store
    Default modelUsed when an agent binds to the provider without naming a model
    Data handlingRecord the provider's data terms, for example no training on inputs
    EnabledOff keeps it configured but unused
  3. Test before saving: Test connection calls POST /api/llm/providers/test. Create provider saves it. Once saved, the provider shows a Gateway URL: set it as an OpenAI SDK's base URL, with an agent token as the key, to call the provider through the gateway.
  4. List its models (GET /api/llm/providers/{name}/models) and bind agents to them: open the Agent bindings tab and choose Bind on the agent's row.
Tip

POST /api/llm/providers/discover lists the models a provider would serve from the details in the form, before it is saved.

Health and circuits

The providers table shows Status, Default model, Health and Used by. A provider's details add its Circuit, and its Leaderboard tab ranks the callers that use it and the models they reach for.

  • Health comes from scheduled probes (GET /api/llm/provider-health). Run one now with POST /api/llm/providers/{name}/health/probe. A failed probe opens the provider's circuit.
  • Each backend has a circuit breaker: closed, open, or half-open with a single probe. Each breaker's state is at GET /api/llm/breakers.

Next steps