Build and evaluate retrieval pipelines

Pipeline boundaries

Runifold supplies provider-neutral EmbeddingModel, VectorStore, Retriever, document, query, usage, deadline, cancellation, and capability contracts. It does not own chunking, document authorization, source-of-truth sync, reranking, or an opinionated RAG pipeline.

Keep ingestion and query paths explicit. They often need different authority, budgets, task hints, and operational schedules.

Build and query a working index

[dependencies]
runifold = "=0.9.0"
runifold-providers = { version = "=0.9.0", features = ["openai"] }
runifold-retrieval-text = "=0.9.0"

use std::sync::Arc;
 
use runifold::{
    Document, InMemoryVectorIndex, RetrievalContext, RetrievalQuery, Retriever,
};
use runifold_providers::openai::OpenAiClient;
 
let client = OpenAiClient::from_api_key(std::env::var("OPENAI_API_KEY")?)?;
let embedder = Arc::new(client.embedding_model("text-embedding-3-small")?);
let built = InMemoryVectorIndex::build(
    "product-help",
    embedder,
    vec![
        Document::new("returns", "Returns are accepted within 30 days.")?,
        Document::new("shipping", "Standard shipping takes 3–5 business days.")?,
    ],
    RetrievalContext::new(),
)
.await?;
 
let response = built
    .index
    .retrieve(
        RetrievalQuery::new("How long can I return an order?", 2)?,
        RetrievalContext::new(),
    )
    .await?;
 
for hit in response.documents {
    println!("{}: {}", hit.document.id, hit.document.text);
}

built.usage is ingestion embedding usage; response.usage belongs to the query. The in-memory index is immutable and intended for tests or small static corpora. Rebuild it when documents change, or use pgvector/Qdrant for persistent upserts and shared access.

Attach it to an Agent with .dynamic_context(4, built.index). The limit bounds how many documents can enter one model turn; it does not replace an adapter's own top-K, tenant filter, score threshold, or context-size limit.

Embedding models

OpenAI-compatible, Gemini, and Ollama clients expose embedding adapters. Requests preserve ordered batches and distinguish document from query tasks. Adapters reject blank model names, invalid vectors, and silent truncation by default while reporting attributable usage.

Record provider, model, dimensions, task mode, normalization, and index version. Never mix vectors from incompatible configurations in one search space.

Index and query

VectorRetriever composes any Embedding Model with any Vector Store. The in-memory index is a deterministic reference. Qdrant maps application document IDs to stable point IDs; pgvector uses explicit setup and parameterized queries.

Upsert stable document identity, text, source metadata, tenant namespace, and embedding version. Bound top-K at both adapter and composition layers.

Security and attribution

Retrieval is external ReadOnly authority. Grant it explicitly, apply tenant and ACL filters before similarity ranking, preserve source attribution, and label retrieved text as untrusted data. It cannot create system instructions.

Checkpointed execution persists successfully prepared context before the first model call, so resume does not repeat retrieval or silently change evidence.

Retrieval evaluation

Evaluate retrieval separately from answer generation. Stable relevance judgments can measure Precision@K, Recall@K, reciprocal rank, nDCG, usage, and host-observed latency. Preserve case order and dataset version.

Then evaluate grounded answer quality, citation correctness, and refusal behavior. A good answer score can hide poor recall; good recall can still feed an Agent hostile or irrelevant text.

Production ingestion sequence

1. authorize the source before reading content;
2. normalize and chunk deterministically while retaining source and ACL identity;
3. assign stable document/chunk IDs and an explicit index version;
4. embed bounded batches with RetrievalDocument task mode;
5. reject count or dimension mismatches before writing;
6. upsert text, attribution, tenant namespace, ACL, model, and version together;
7. publish the new index version only after validation;
8. delete superseded chunks by source identity, not similarity.

At query time, derive tenant and ACL filters from authenticated application state, create a deadline-aware RetrievalContext, embed with RetrievalQuery, search a bounded top-K, optionally rerank, and pass attributed evidence to the Agent as untrusted context.

Retrieval troubleshooting

Before troubleshooting production quality, verify the 0.9 ingestion and ranking contracts. Use runifold-retrieval-text for bounded UTF-8 loading and stable Unicode/provenance-aware chunk IDs:

use std::num::NonZeroUsize;
use runifold_retrieval_text::{
    DEFAULT_MAX_TEXT_BYTES, TextChunkPolicy, chunk_document, load_text,
    split_markdown_sections,
};
 
let source = load_text("handbook", markdown_bytes, DEFAULT_MAX_TEXT_BYTES)?;
let sections = split_markdown_sections(&source)?;
let policy = TextChunkPolicy::new(NonZeroUsize::new(800).expect("non-zero"), 80)?;
let chunks = sections
    .iter()
    .map(|section| chunk_document(section, policy))
    .collect::<Result<Vec<_>, _>>()?
    .into_iter()
    .flatten()
    .collect::<Vec<_>>();

HybridRetriever queries two independent retrievers concurrently and applies bounded weighted reciprocal-rank fusion. Wrap it in RerankingRetriever when a provider-neutral Reranker should reorder an expanded candidate set. Version the chunk policy, embedding model, fusion weights and reranker descriptor together; otherwise an index rebuild can silently invalidate evaluation data.