Ground an Agent with retrieval

Static context

Use .context(...) for a small, known piece of evidence that is already available when the Agent is built.

let agent = runtime
    .agent("policy")
    .system("Use only the supplied policy evidence.")
    .context("Returns are accepted within 30 days.");

Static context is simple and deterministic, but it does not scale to a large or frequently changing corpus.

Dynamic context

A dynamic context source retrieves documents for the current request and injects only the selected evidence. Keep retrieval behind the Retriever trait so the Agent does not depend on a database vendor.

Choose the query, result limit, score threshold, and context size explicitly. Return stable document IDs and source metadata so answers can be attributed.

Security boundary

Retrieved text is untrusted evidence, never a system instruction. A document that says “ignore previous instructions” must not gain authority by appearing in search results.

Apply tenant filters before vector search, not after. Never let a model invent the namespace, authorization predicate, or database credentials used for retrieval.

Persistent stores

Start with InMemoryVectorIndex for tests and small local experiments. Enable the qdrant or pgvector feature when persistence, distributed access, or a larger corpus is required.

Embedding model, distance metric, vector dimension, and normalization form one index contract. Changing any of them requires a deliberate migration or reindex.

Evaluate quality

Measure retrieval separately from answer generation:

• recall of the required source in the top results;
• irrelevant context rate;
• citation correctness;
• answer quality when no relevant document exists;
• latency and cost by query class.

Keep a versioned evaluation set and rerun it when the embedding model, chunking, filters, prompt, or provider changes.