Your own vector store (pgvector and Bedrock)

Let Liya answer from a pgvector table or an Amazon Bedrock knowledge base you already run, read at answer time and never copied in.

After reading this page you will be able to:
  • Connect a pgvector table with a read-only login over TLS
  • Connect a Bedrock knowledge base through a role in your AWS account
  • Choose who may read each record and what label it carries
  • Test the store and calibrate its scores

How it works

A vector store you run is added as a data source of type External vector store. Liya reads it when it answers, beside your other knowledge, for the tenant that owns it. Nothing is copied in, and Liya keeps no fingerprint of the records and does not check them overnight.

  • Read only. Liya never writes to the store, and the source has nothing to sync. Every pgvector query runs in a read-only transaction that is rolled back.
  • Checked before anyone sees it. Every record is mapped, checked against the reader's access, held to the surface's sensitivity limit and scanned for secrets, hidden text and embedded instructions. A record that fails any check is withheld.
  • Tested before it answers. A store answers only while its latest Test passed.
  • Audited. Every search of the store writes one kb.external_query audit record, with a SHA-256 hash of the question and never its text. Adding, changing, testing and deleting a store are audited as kb.external_store.created, .updated, .tested and .deleted.
  • Kept apart. A slow or failing store only drops its own results. After 5 failures in a row the store is paused for 60 seconds.

Two providers are available: pgvector and Bedrock Knowledge Bases.

Add a store

  1. Open Settings, then Data & retention. Under Data sources, give the source a Name and choose the type External vector store.
  2. Pick the Provider and fill in the four steps: Connection, Embeddings, Fields and access and Test. The provider sections below list the fields.
  3. Choose Connect source. The store's page opens.
  4. Make sure the store belongs to exactly one tenant: its source id must be in that tenant's sources (see Tenancy). On an instance with no tenants it belongs to the instance.
  5. On the store's page, choose Test and fix what it reports. When it says Ready, Liya answers from the store.

A credential can be a stored secret, or a value you type. A typed value is filed in Secrets for this store and never shown again. Pick the tenant the store belongs to before you type one.

Test and Calibrate open the stored credential, so they need both the sources.write and governance.write permissions. Anyone with sources.read can open the store's page and see the last result.

pgvector

What you need

  • PostgreSQL with the vector extension, and a table (or view) with an id column, a vector(n) column and a text column.
  • The embedding model the vectors were made with, reachable through an LLM provider configured in Liya. Liya embeds each question with that model, so the store and the question are on the same scale.
  • A login that can read the table and nothing more.
  • The database reachable from Liya over TLS.

Create a read-only login

Run these as an administrator of the database, with your own database, schema and table names. \password asks for the password twice and sends only its hash.

psql -d knowledge -c "CREATE ROLE liya_reader LOGIN"
psql -d knowledge -c "\password liya_reader"
psql -d knowledge -c "GRANT CONNECT ON DATABASE knowledge TO liya_reader"
psql -d knowledge -c "GRANT USAGE ON SCHEMA public TO liya_reader"
psql -d knowledge -c "GRANT SELECT ON public.documents TO liya_reader"

Liya signs in with SCRAM-SHA-256 only. A server that asks for md5, a cleartext password or no password is refused. If that happens, set password_encryption to scram-sha-256 on the server and set the login's password again.

Test checks whether the login can INSERT, UPDATE, DELETE or TRUNCATE on the table. If it can, the page says so, and the store stays Not ready until you tick I understand, keep this credential. Use a read-only login if you can.

Connection fields

FieldDefaultNotes
HostnoneThe database's DNS name
Port5432
Databasenone
Schemapublic
TablenoneA table, materialised view, partitioned table or view
Id columnidEach record's key
Vector columnembeddingA vector(n) column
Text columntextThe passage Liya quotes
TLS modeverify-fullSee the next section
Username, PasswordnoneThe read-only login
Region the store is innoneNeeded when the tenant has a data residency, and must be inside it

Liya connects only to the address it checked. Loopback, link-local and cloud metadata addresses are always refused. Private (RFC 1918) addresses are allowed unless the operator sets TICKETIQ_INTEGRATION_BLOCK_PRIVATE. Your egress policy (Access › Egress) applies too.

TLS modes

Every connection is encrypted. TLS mode decides how much of the server's certificate Liya checks.

ModeEncryptedChecks the certificate's CAChecks the certificate names the host
verify-full (default)YesYesYes
verify-caYesYesNo
requireYesNoNo

disable, allow and prefer are refused when you save. The verify modes check the certificate against the CA bundle installed on the Liya server, which holds the public CAs.

Note

Liya cannot check a certificate issued by a provider's own private CA yet, as on Amazon RDS, Aurora and Cloud SQL. Test reports that the certificate could not be verified. Until a CA option is added, require still encrypts the connection but does not prove which server you reached.

Embeddings

A pgvector table holds vectors only, so Liya always embeds the question (Liya, with this model).

FieldNotes
Embedding model<provider>/<model id>, where the provider is one configured in Liya that can embed (Bedrock Titan or Cohere, OpenAI, or an OpenAI-compatible endpoint). For example bedrock/amazon.titan-embed-text-v2:0 for a provider named bedrock
Vector sizeThe n of the vector(n) column, for example 1024
DistanceCosine (<=>), Dot product (<#>) or Euclidean (<->). Use the one your index was built for

The embedding call is held to the tenant's data residency. On pgvector 0.8 and later, Liya turns on iterative index scans, so an access filter does not leave a search short of results.

Bedrock Knowledge Bases

How Liya reaches it

You create an IAM role in your AWS account that can read the knowledge base. Liya assumes that role with AWS STS, from its own AWS roles, and sends an external id that Liya issues. The external id is the same for every Bedrock store of a tenant, and a tenant cannot choose it. This stops another customer who learns your role's ARN from having Liya assume it for them.

  • The role must be in a different AWS account from Liya's, and in the same AWS partition.
  • Before it first uses a role, and on every Test, Liya also tries the role without the external id and expects a refusal. A role that lets Liya in without the id is refused. Your CloudTrail shows each of those tries as a denied AssumeRole.
  • Liya names its sessions liya-extstore-<source id>.

1. Save the store

Fill in the form with the provider Bedrock Knowledge Bases.

FieldNotes
Knowledge base idAs Bedrock shows it, for example ABCDEFGHIJ
Region the store is inAn AWS region name, as eu-west-2. Required, and inside the tenant's data residency when it has one
Role ARN to assumearn:aws:iam::<your account id>:role/<name>. You can finish the role's trust policy after you save
Metadata attributes that hold listsComma separated. Defaults to the groups and principals attributes

Bedrock embeds the question itself, so the only embedding choice is The store does.

Choose Connect source, then open the store's page. Under the store's stats it shows the AWS principals for your role's trust policy and the external id. On an instance with tenants, the external id appears once the store belongs to exactly one of them. If the page says the AWS principal is not set on this server, ask Liya's operator for it.

2. Write the role's trust policy

Name each AWS principal the store's page shows, and require the external id. Name those roles, not your account root or Liya's account root.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": [
          "<first AWS principal from the store's page>",
          "<second AWS principal from the store's page>"
        ]
      },
      "Action": "sts:AssumeRole",
      "Condition": {
        "StringEquals": { "sts:ExternalId": "<external id from the store's page>" }
      }
    }
  ]
}

3. Give the role read access only

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": ["bedrock:Retrieve", "bedrock:GetKnowledgeBase"],
      "Resource": "arn:aws:bedrock:<region>:<your account id>:knowledge-base/<knowledge base id>"
    }
  ]
}

Bedrock cannot report what a role is allowed to do, so Test shows Read only: not known with that advice.

4. Test

Choose Test. Liya checks the role refuses it without the external id, assumes the role with the id, and reads the knowledge base's status. A knowledge base still being created or updated leaves the test unfinished. One that is FAILED, being deleted or DELETE_UNSUCCESSFUL fails the test. When the role refuses Liya, or lets it in without the external id, the error names the principals and the external id the trust policy must hold.

Each result's fields are its metadata attributes, plus text (the chunk) and source_uri (where the chunk came from: the S3 URI, web URL or document id). Text field defaults to text, which is right for Bedrock. To give citations a link, set mapping.url_field to source_uri through the API.

For operators

Liya calls STS with its own default AWS credential chain. Its roles need sts:AssumeRole on the tenants' roles. The k3s stack grants it to its ServerRole and WorkerRole for session names starting liya-extstore-, and denies it for roles in its own account.

The principals the store's page shows come from TICKETIQ_EXTSTORE_AWS_PRINCIPALS, a comma-separated list of IAM role ARNs. The k3s stack sets it to its ServerRole and WorkerRole. On another install, set it to the roles your Liya runs as. Values that are not IAM role ARNs are dropped.

Fields and access

Record fields

FieldNotes
Title fieldDefaults to title. A record without a title is titled with the first line of its text
Text fieldDefaults to text. A record with no text is withheld
Category field (optional)
Updated field (optional)When the record last changed

For pgvector these are column names. For Bedrock they are metadata attribute names.

Who may read what it returns

There is no default. You choose one of two modes.

ModeWho reads a record
Everyone in these groupsThe groups you list, or everyone in the tenant when you tick Everyone in the tenant may read it. The same rule applies to every record
Read access from fieldsThe record's own fields: a groups field, a people field and a public flag field. Set at least one

In Read access from fields:

  • A groups or people field can hold a list or a single name. For pgvector it can be a text column, an array column, or a json or jsonb column holding a list or a string.
  • The public flag makes a record readable by everyone in the tenant only when its value is true itself. For pgvector the column must be boolean, json or jsonb, and Test fails otherwise.
  • For pgvector, a field that is not a column of the table fails Test.
  • For Bedrock, values must match exactly, including case.

How names are read:

  • Prefix for people starts on email:, so alice@example.com in a record is read as email:alice@example.com. Empty the box to read people as the store holds them.
  • With Prefix for groups empty, each group a record names is read as a group of this store. To let people read those records, open the Identity map page and map their group or user to group:<name> under this store, as group:Support L1. Type a prefix only when the identity map names these groups another way.
  • A value that already starts with email: is kept as it is. A value that names its kind, as group:Support L1 or user:alice, is read under this store.

A record whose access Liya cannot read is withheld from everyone. It is never treated as public.

Sensitivity label

Every record from the store carries the Sensitivity label you pick, which starts on the instance's default. A surface whose limit is lower does not show the store's records. A passing Test notes which surfaces those are. Through the API, sensitivity.mode field reads each record's label from sensitivity.field, and a record without one gets the highest label. See Sensitivity labels.

Test and readiness

Test reports:

LineMeaning
DimensionThe store's vector size against Vector size (pgvector)
Model checkLiya embeds the text of up to 5 sampled records and compares its vectors with the stored ones. They match at a cosine of 0.98 or more, and 4 of 5 must match (all of them when only 3 or 4 came back). With fewer than 3 records, or a store that returns no vectors (Bedrock), the model is not checked
Read onlyWhether the credential can write
SampleUp to 3 records as you would see them, and how many were withheld and why

To check what someone else would see, type their user name in Also count the sample for. The test then counts how many records that person would see, without showing which.

Liya answers from a store only while its latest test passed and nothing that test checked has changed. Saving a new provider, connection, region, credential or embedding setting means you run Test again. A test the store could not finish (busy, timed out, unreachable, throttled or unavailable) changes nothing, and Liya goes by the last test that finished.

Scores and calibration

Your store's scores are not on Liya's own scale. Until you calibrate it, answers from the store are shown as related and never quoted as confident:

  • With Cosine and a model check that matched, Liya uses the scores as they are, capped below the quote bar.
  • With another distance, or when the store embeds the question (Bedrock), Liya ranks the results against each other. The best ones show as related even when they are weak.

To calibrate, add a retrieval dataset for the store's tenant (see Datasets) whose questions name the store's collection, kb_<source id>, or no collection. Then choose Calibrate on the store's page. Liya runs the questions against the store and saves the score at which an answer is related and the score at which it is confident. If there are too few right answers to quote with confidence, the store is calibrated for related answers only. The store must be Ready to calibrate.

Settings only the API takes

Create the store with POST /api/sources (type vector_store, the config nested as below), read it with GET /api/sources/{id}, test it with POST /api/sources/{id}/probe (optionally ?as_user=<name>) and calibrate it with POST /api/sources/{id}/calibrate.

SettingDefaultRange
limits.top_k_max201 to 100 records per search
limits.overfetch41 to 10
limits.timeout_ms2000100 to 10000 milliseconds per call
acl.allowed_principalsnonePeople who may read every record, in Everyone in these groups mode
mapping.url_fieldnoneThe field holding each record's link
mapping.default_categorynoneThe category of a record without one

Next steps