Deletion propagation
When a source deletes a record or hides it from the connector, Liyagent withholds everything derived from it at once and purges it after a grace period.
- Know what a source deletion removes and when
- Run a reconcile for one source
- Read the deletion list and the hold keeping a record
- Keep legal holds intact through deletions
A file deleted from SharePoint, a page removed from Confluence, a Jira issue deleted, or a customer row gone from an export: none of these should keep answering questions here. Deletion propagation carries the deletion across, including records the connector's account can no longer read.
How a deletion is noticed
| Signal | Sources | When |
|---|---|---|
| Webhook delete events | Jira (jira:issue_deleted), ServiceNow (a delete business rule), through the source webhook | At once, not at the next sync |
| Change feeds | SharePoint (Graph drive delta, deleted facet), Google Drive (changes feed, removed or trashed) | Polled hourly |
| Stream tombstones | A Kafka record with a key and no value; any queue or stream message {"_deleted": true} or Debezium op: "d" | As the message is consumed |
| Reconcile | SharePoint, Confluence, Google Drive, Salesforce Knowledge, Jira, ServiceNow, and CSV or SQL sources in records mode | Every TICKETIQ_SOURCE_RECONCILE_HOURS (default 24) per source |
A reconcile lists only the ids the source has now (no downloads, no permission reads) and compares them with what Liyagent holds. A full KB sync and a full records sync do the same for themselves.
Confluence and Salesforce Knowledge have no delete feed, so their deletions are seen by the daily reconcile, not at once.
See Stream consumers for how tombstones arrive on Kafka, Kinesis and queues.
What a deletion does
At once, everything derived from the record is withheld:
| Where | What happens |
|---|---|
| Knowledge base | The point is marked deleted, so retrieval never serves it, and its passages are retracted |
| Records | The row is tombstoned: no read returns it |
| A ticket a pipeline sync opened | Its knowledge points are marked. The ticket row itself stays, noted as deleted in its source |
| Caches | Semantic-cache entries grounded in it, and the ticket's response-cache entries, are dropped |
Each deletion is written to the deletion list (one row per source, kind and ref) and audited as source.record.deleted.
After TICKETIQ_KB_DELETED_GRACE_DAYS (default 7) the purge deletes for good: the points and passages, the record rows, and the ontology objects and links of the source's system (for a ticket, its ontology object, links and lineage). The purge is audited as source.record.purged.
A record that comes back in its source is live again, and its tombstone goes.
Legal holds
Holds are read at the deletion and again at the purge. A record a hold covers (the source's tenant, or for a ticket any of its facets) is withheld and listed with the hold that keeps it, but never purged while the hold stands. This is audited as source.record.held and source.record.purge_held.
If holds cannot be read, everything is held.
The guard against mass deletion
A reconcile that would delete more than 25% of what a source holds is held back and raises a warning alert, audited as source.reconcile.held_back. A connector that suddenly lists a fraction of its repository, because a credential was narrowed or a folder moved, must not take the knowledge base with it. A listing cut short at the walk cap is not treated as a listing at all.
Fix the cause (the credential or the folder), then reconcile again.
Run and read it
| Task | Endpoint | Permission |
|---|---|---|
The deletion list: each tombstone's source, kind (kb, record, ticket), ref, signal, deleted_at, the hold keeping it (hold_id) and purged_at | GET /api/ingestion/deletions (optional source_id, limit) | sources.read |
| Reconcile one source now | POST /api/sources/{id}/reconcile | sources.write |
The deletion list is your record of what was withdrawn and when it was purged. It shows only your tenant's sources. There is no console view of it yet; use the API.
A manual reconcile returns how many ids the source lists and how many are missing, or why it was skipped (for example the listing was incomplete), and is audited as source.reconcile.
Settings
| Setting | Default | Effect |
|---|---|---|
TICKETIQ_SOURCE_RECONCILE_ENABLED | true | The hourly change-feed poll and the periodic reconcile. It reads only configured sources |
TICKETIQ_SOURCE_RECONCILE_HOURS | 24 | Hours between full reconciles of one source |
TICKETIQ_SOURCE_RECONCILE_BUDGET_S | 900 | Time budget per reconcile pass |
TICKETIQ_KB_DELETED_GRACE_DAYS | 7 | Days between withholding and purging |
Set reconcile: off in a source's config to skip it.
Limits
- A pipeline ticket whose source record is deleted keeps its ticket row; only what was derived from it is purged.
- Records are not covered by GDPR erasure or retention. See GDPR for what erasure covers.