AI Gateway & LLM providers
One governed path for every model call: providers, the model chain and fallback, guardrails, budgets and metering.
Every model call in Liyagent, from a built-in Liya agent, a managed agent or a self-managed agent calling /api/gateway/v1/chat/completions, goes through the same steps:
A refusal at any step is recorded in the audit log with the reason, and nothing after it runs. The pages below cover each step you configure.
Two wire formats, one path
| Clients that speak | Point them at | Authenticate with |
|---|---|---|
| OpenAI chat completions (the OpenAI SDKs, most frameworks) | https://<instance>/api/gateway/v1 | the agent's gateway token as the API key |
| Anthropic Messages (Claude Code, the Anthropic SDKs) | https://<instance>/api/gateway/anthropic | the agent's gateway token, as ANTHROPIC_AUTH_TOKEN (a bearer) or ANTHROPIC_API_KEY |
The Anthropic endpoint translates each Messages request into the same governed call, so every step above applies and every call is one gateway.call row in the audit log. It carries text, images, tool use and tool results, client tools and tool_choice, and streams Anthropic's events when asked. The model that answers is the agent's binding, whatever model the client names, and the response says which one it was.
What it does not carry:
- Anthropic's server-side tools (web search, code execution) run on Anthropic's servers, which a governed call never reaches. They are left out of the request and named in the
X-TicketIQ-Dropped-Toolsresponse header. - Thinking blocks in the conversation are dropped, because only the model that wrote them can read them back. Sampling settings (temperature, stop sequences, the thinking budget) are the binding's to decide.
- Any other block (a PDF document, a server tool's result) is refused with a
422, rather than dropped without a guardrail reading it. - count_tokens answers an estimate, about four characters a token: the agent may run on a model whose tokenizer the gateway cannot run.
To start Claude Code this way from a terminal, see liya run claude.
In this section
- LLM providers Preview
Add Amazon Bedrock, Anthropic, Google Gemini, Vertex AI or any OpenAI-compatible endpoint, test it, and watch its health.
- Model chain and fallback
Give an agent backup models, choose exactly which failures move down the chain, and see every attempt a call made.
- Guardrails Preview
Guardrail rules are single checks on text going into and coming out of a model. Guardrail profiles bundle checks for a provider or agent.
- Create a guardrail rule Preview
Create a guardrail rule from a template or from blank, set its check and action, test it and bind it to an agent.
- Guardrail templates Preview
The starting points offered when you create a guardrail rule or a guardrail profile, and what each one turns on.
- Test a guardrail Preview
Dry-run guardrail rules over sample text, see raw detector signal, score text against deny topics and test a guardrail profile, without calling a model.
- Budgets
Cap spend per period for every agent or one agent, cap any scope in dollars or tokens, choose to refuse or fall back at the limit, and pace callers with rate limits.