Self-improvement

An agent improves through the optimizer: it mines what went wrong, drafts edits to the agent's instructions and tool descriptions, and offers them only when an offline eval run shows they beat the agent as it is. Self-improvement runs that loop on a schedule, for the agents you switch it on for, with extra checks for a loop nobody is watching.

A cycle can only offer a proposal. Adopting one is always a person's decision, through the promotion gate, and becomes a new agent version you can restore away from.

What the agent learns from#

Write decisions are the strongest evidence there is that an action was wrong. Reasons are free text, so they are kept through the input guardrails (personal data and credentials masked) and bounded. Approved writes are not kept: they are not failures.

Corrections#

Every time a person corrects an agent, the correction is logged as an event:

Each event keeps what was asked, what the agent said and what the person said it should be, with personal data redacted. Each event also proposes two things:

• A regression case for corrections-<agent>, or corrections-triage for a routing correction. Its expectations come from what the correction changed. Codes, versions and identifiers the corrected answer has and the wrong one lacked must appear. Ones only the wrong answer had must not. When there are no such terms, the proposal says it needs your expectation.
• A draft knowledge article for an answer correction, with the question as its title and the corrected answer as its resolution. It is never published on its own.

Nothing changes until you act. GET /api/corrections lists the open events. You need permission to read conversation content to see their text. POST /api/corrections/{id}/apply with {"eval": true, "kb": true} adds the case and creates the draft. Add "expect" to give the case your own expectation. Creating the draft needs kb.approve. POST /api/corrections/{id}/dismiss closes an event. Logging a correction never blocks the correction itself, and every event, apply and dismiss is audited by id.

Which component is failing#

Not every failure is fixed by editing the prompt. A question the knowledge base cannot answer is a retrieval problem. A provider timing out is the model's problem. A guardrail block is the controls doing their job. Before anything is drafted, the mined failures are attributed to a component:

GET /api/agents/{key}/optimizer/diagnosis?days=30 shows each component's share, the primary one, and the targets: the components an edit can fix (prompt and tool) that carry at least 20% of the failures. The diagnosis also comes with the mined signals and is kept on every proposal and every cycle.

What it changes is the TICKETIQ_OPTIMIZER_DIAGNOSIS setting:

The retro over finished runs#

Some waste fails nothing: the same search made five times, a call that failed and was sent again unchanged, a run that ran out of tool calls. Every six hours a retro reads the audit rows of each agent's runs that have finished (no activity for 10 minutes), each run once, and looks for:

A retro that found anything is filed as a report, with how many runs showed each finding, up to five run ids to open, and a suggested fix. It is also recorded on the trust ledger and audited as agent.retro.reported.

A retro changes nothing about the agent. Its findings are the Run retro signal above, so the next cycle, or a person drafting a proposal, works from them, and any change still goes through the gates below.

• GET /api/governance/retro/{agent}: the agent's reports, newest first.
• POST /api/governance/retro/{agent}:run (agents.update): review the runs finished since the last retro now.
• POST /api/governance/retro/{agent}/dismiss with {"kind", "cause", "reason"}: a finding you expect, for example a search the agent is meant to repeat. It is no longer mined (agent.retro.finding.dismissed).

TICKETIQ_RETRO_ENABLED (default true) switches the six-hourly retro off. It only reads and files.

A cycle#

1. Waits while a proposal from an earlier cycle is still waiting on review, and during the cool-down (default 7 days; a cycle that found too little tries again a day later).
2. Mines the agent's failures over the window (default 14 days). Fewer than the minimum (default 5) is not yet a pattern: nothing is drafted.
3. Drafts edits through the governed path, as the agent-optimizer agent, with its own binding, budget and audit. Edits are checked as any proposal's are: size caps, only tools the agent is offered, and a tool description that reads like instructions to a model is refused outright.
4. Evaluates on the calibrate split: the optimizer's verdict. The draft must beat the agent as it is, with no metric and no category regressed.
5. Evaluates on the holdout split: the same verdict on cases the draft never saw. This is what tells a real improvement from one fitted to the cases it was shown.
6. Checks the canaries: items marked "metadata": {"canary": true} are cases no honest answer can pass. A draft that passes a canary the agent as it is fails has learned to game the grader, and is rejected whatever else it scored.
7. Offers the proposal if every check passed, or rejects it with the checks that failed.

Every cycle is recorded and audited as agent.selfimprove.offered, .rejected, .skipped or .failed, with system:selfimprove as the actor.

Set it up#

The dataset must be an agent_turn dataset split into calibrate and holdout cases. Add a few canaries to its holdout side.

curl -X PUT https://liyagent.example/api/agents/service-desk/self-improve \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d '{"enabled": true, "dataset": "service-desk-cases", "min_signals": 5,
       "days": 14, "cooldown_days": 7}'

All three need agents.update and Cedar update on the agent. Switching on and running a cycle also need what an eval run needs: transcripts.content.read, governance.write and Cedar invoke.

Review and adopt#

An offered proposal appears with the agent's other optimizer proposals (GET /api/agents/{key}/optimizer/proposals), with its rationale, the signals it answers, and both evaluations. Adopt it as you would any proposal: make it a promotion candidate, pass the frozen eval bank and a second person's sign-off, then promote. To undo it, restore the version before it.

What it does not do#

• It never adopts a change by itself, and the loop that proposes a change cannot apply it.
• It does not write new tools or change code. Research on self-improving agents finds that tools and skills written by a model without review add no reliable gain, and that self-modifying code is still a sandboxed research technique.
• It does not use a model's judgement as the final gate: the verdict compares eval runs case by case, and canaries catch a grader being gamed.