# Self-improvement

Let an agent propose its own improvements on a schedule, from how it fails and what people corrected, proven on held-out cases before anyone is asked to adopt them.

An agent improves through the **optimizer**: it mines what went wrong, drafts edits to the agent's instructions and tool descriptions, and offers them only when an offline eval run shows they beat the agent as it is. **Self-improvement** runs that loop on a schedule, for the agents you switch it on for, with extra checks for a loop nobody is watching.

A cycle can only **offer** a proposal. Adopting one is always a person's decision, through the promotion gate, and becomes a new agent version you can restore away from.

## What the agent learns from

| Signal | Where it comes from |
|---|---|
| Refusals | Conversations a guardrail, policy or containment stopped |
| Tool errors | Tool calls that failed, by tool and error code |
| Thumbs down | Answers people voted unhelpful, with their labels and comment |
| Knowledge gaps | Questions the agent could not answer |
| Failed eval cases | Cases of the agent's latest offline eval run that failed |
| Write decisions | Writes the agent proposed that a person **rejected**, **edited** before approving, **cancelled** (an SAP Confirm card) or let **expire**, with the person's reason |
| Run retro | What the [retro over finished runs](#the-retro-over-finished-runs) found: the same call repeated, a failed call sent again, runs cut off by a cap, calls refused by policy |

Write decisions are the strongest evidence there is that an action was wrong. Reasons are free text, so they are kept through the input guardrails (personal data and credentials masked) and bounded. Approved writes are not kept: they are not failures.

## Corrections

Every time a person corrects an agent, the correction is logged as an event:

| Correction | Where it comes from |
|---|---|
| A corrected answer | A reviewer's corrected answer to a conversation in a review queue |
| A corrected routing | A reviewer's corrected category, team or priority on a ticket |
| A thumbs-down that says what is right | An answer voted unhelpful with a comment, from the console, the feedback link or Teams |

Each event keeps what was asked, what the agent said and what the person said it should be, with personal data redacted. Each event also proposes two things:

- **A regression case** for `corrections-<agent>`, or `corrections-triage` for a routing correction. Its expectations come from what the correction changed. Codes, versions and identifiers the corrected answer has and the wrong one lacked must appear. Ones only the wrong answer had must not. When there are no such terms, the proposal says it needs your expectation.
- **A draft knowledge article** for an answer correction, with the question as its title and the corrected answer as its resolution. It is never published on its own.

Nothing changes until you act. `GET /api/corrections` lists the open events. You need permission to read conversation content to see their text. `POST /api/corrections/{id}/apply` with `{"eval": true, "kb": true}` adds the case and creates the draft. Add `"expect"` to give the case your own expectation. Creating the draft needs `kb.approve`. `POST /api/corrections/{id}/dismiss` closes an event. Logging a correction never blocks the correction itself, and every event, apply and dismiss is audited by id.

## Which component is failing

Not every failure is fixed by editing the prompt. A question the knowledge base cannot answer is a retrieval problem. A provider timing out is the model's problem. A guardrail block is the controls doing their job. Before anything is drafted, the mined failures are attributed to a component:

| Component | What points at it | Where the fix lives |
|---|---|---|
| Retrieval | Knowledge gaps; answers voted outdated, not relevant or incomplete; eval cases failed on passages or grounding | The knowledge base: close the gaps and re-run the retrieval eval |
| Prompt | Answers voted unclear; eval cases failed on their own expectations | An instruction edit, which is what the optimizer drafts |
| Tool | Tool calls that errored; writes a person rejected or corrected; eval cases failed on tool choice or order | A tool-description edit, or the tool's server itself |
| Model | Provider errors, timeouts, budget and rate limits; answers voted wrong | The optimizer's what-if, or the binding |
| Policy | Guardrail and policy refusals, containment, answers voted unsafe | The policy, if the blocks are wrong; never the agent |

`GET /api/agents/{key}/optimizer/diagnosis?days=30` shows each component's share, the primary one, and the **targets**: the components an edit can fix (prompt and tool) that carry at least 20% of the failures. The diagnosis also comes with the mined signals and is kept on every proposal and every cycle.

What it changes is the `TICKETIQ_OPTIMIZER_DIAGNOSIS` setting:

| Mode | What happens |
|---|---|
| `advise` (default) | The diagnosis is reported and recorded. Drafting and cycles run exactly as before |
| `steer` | The draft is told the diagnosis and may only edit its targets: an instruction rewrite is dropped when only tools are to blame. When no edit can fix the failures (they are mostly retrieval, model or policy), nothing is drafted and a cycle is skipped with the advice. Proposals you write by hand are never steered |

## The retro over finished runs

Some waste fails nothing: the same search made five times, a call that failed and was sent again unchanged, a run that ran out of tool calls. Every six hours a **retro** reads the audit rows of each agent's runs that have finished (no activity for 10 minutes), each run once, and looks for:

| Finding | What it means |
|---|---|
| `repeated_call` | One tool was called 3 or more times in the same run |
| `retry_after_error` | A tool failed, and the agent called it again in the same run |
| `run_cap` | The run ended on a cap (model calls, tool calls, time, cost) |
| `halted` | A run hook stopped the run |
| `refused` | A policy refused a call the agent tried |

A retro that found anything is filed as a report, with how many runs showed each finding, up to five run ids to open, and a suggested fix. It is also recorded on the [trust ledger](/docs/govern/trust-ledger) and audited as `agent.retro.reported`.

A retro changes nothing about the agent. Its findings are the **Run retro** signal above, so the next cycle, or a person drafting a proposal, works from them, and any change still goes through the gates below.

- `GET /api/governance/retro/{agent}`: the agent's reports, newest first.
- `POST /api/governance/retro/{agent}:run` (`agents.update`): review the runs finished since the last retro now.
- `POST /api/governance/retro/{agent}/dismiss` with `{"kind", "cause", "reason"}`: a finding you expect, for example a search the agent is meant to repeat. It is no longer mined (`agent.retro.finding.dismissed`).

`TICKETIQ_RETRO_ENABLED` (default `true`) switches the six-hourly retro off. It only reads and files.

## A cycle

1. **Waits** while a proposal from an earlier cycle is still waiting on review, and during the cool-down (default 7 days; a cycle that found too little tries again a day later).
2. **Mines** the agent's failures over the window (default 14 days). Fewer than the minimum (default 5) is not yet a pattern: nothing is drafted.
3. **Drafts** edits through the governed path, as the `agent-optimizer` agent, with its own binding, budget and audit. Edits are checked as any proposal's are: size caps, only tools the agent is offered, and a tool description that reads like instructions to a model is refused outright.
4. **Evaluates on the calibrate split**: the optimizer's verdict. The draft must beat the agent as it is, with no metric and no category regressed.
5. **Evaluates on the holdout split**: the same verdict on cases the draft never saw. This is what tells a real improvement from one fitted to the cases it was shown.
6. **Checks the canaries**: items marked `"metadata": {"canary": true}` are cases no honest answer can pass. A draft that passes a canary the agent as it is fails has learned to game the grader, and is rejected whatever else it scored.
7. **Offers** the proposal if every check passed, or rejects it with the checks that failed.

Every cycle is recorded and audited as `agent.selfimprove.offered`, `.rejected`, `.skipped` or `.failed`, with `system:selfimprove` as the actor.

## Set it up

The dataset must be an `agent_turn` dataset split into `calibrate` and `holdout` cases. Add a few canaries to its holdout side.

```bash
curl -X PUT https://liyagent.example/api/agents/service-desk/self-improve \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d '{"enabled": true, "dataset": "service-desk-cases", "min_signals": 5,
       "days": 14, "cooldown_days": 7}'
```

| Action | Endpoint |
|---|---|
| Read the configuration and recent cycles | `GET /api/agents/{key}/self-improve` |
| Switch on, off, or tune | `PUT /api/agents/{key}/self-improve` |
| Run one cycle now (past the cool-down, never past the checks) | `POST /api/agents/{key}/self-improve/run` |

All three need `agents.update` and Cedar `update` on the agent. Switching on and running a cycle also need what an eval run needs: `transcripts.content.read`, `governance.write` and Cedar `invoke`.

## Review and adopt

An offered proposal appears with the agent's other optimizer proposals (`GET /api/agents/{key}/optimizer/proposals`), with its rationale, the signals it answers, and both evaluations. Adopt it as you would any proposal: make it a promotion candidate, pass the frozen eval bank and a second person's sign-off, then promote. To undo it, restore the version before it.

## What it does not do

- It never adopts a change by itself, and the loop that proposes a change cannot apply it.
- It does not write new tools or change code. Research on self-improving agents finds that tools and skills written by a model without review add no reliable gain, and that self-modifying code is still a sandboxed research technique.
- It does not use a model's judgement as the final gate: the verdict compares eval runs case by case, and canaries catch a grader being gamed.