Build on the Provider Runtime contract

Provider responsibilities

A provider adapter has two core responsibilities: implement Model with a lossless canonical stream, and implement ProviderModel with a stable provider namespace. It must translate request content, response lifecycle, tool calls, reasoning, usage, warnings, typed errors, deadlines, cancellation, and retry safety.

It must not reimplement Agent loops, workflow recovery, budgets, capabilities, effects, or observability policy. Those layers remain provider-neutral.

Runtime composition

ProviderModelExt adds provider-qualified Agent construction, a resilient single-route builder, and ProviderRuntime. The Runtime wraps the canonical stream with same-route retry and an independent circuit breaker while remaining a Model.

Because the result is still a model boundary, it can be instrumented by OtelModel, placed behind ModelRouter, called directly, used by an Agent, or executed inside an AgentStep.

Adapter-owned safe defaults

In 0.9, each concrete adapter publishes a reviewed ProviderRuntimeProfile through ProviderModel::runtime_profile; the ordinary .runtime(model) path applies it automatically. Delivery mode, request options, retry permission, circuit policy and capability behavior therefore follow the actual protocol instead of one facade-wide guess.

Only errors marked retry-safe are eligible. Unknown, unsafe, cancellation and post-commit stream failures do not retry. Override the complete profile only through runtime_with_profile and only with deployment-specific evidence.

Choose a workload preset

use runifold::{BatchProfile, InteractiveProfile, ProviderModelExt};
use runifold_providers::openai::OpenAiClient;
 
let client = OpenAiClient::from_api_key(std::env::var("OPENAI_API_KEY")?)?;
let interactive = client
    .clone()
    .runtime_with_preset("gpt-5", InteractiveProfile)?;
let batch = client.runtime_with_preset("gpt-5", BatchProfile)?;
 
let audit = batch.capability_audit().await?;
for item in audit.review_required() {
    println!("{}: {}", item.feature, item.recommendation);
}

ProductionProfile keeps the adapter recommendation, InteractiveProfile commits streamed events promptly, and BatchProfile validates a complete response before Router commit. A capability audit is deployment evidence; it does not guess that an unknown model supports a feature.

Compatible endpoints

Protocol compatibility is not capability equivalence. An OpenAI-compatible endpoint may accept the same JSON shape while differing in streaming, structured output, usage, reasoning, error bodies, or cancellation.

Use a custom validated endpoint and stable provider identity. Keep credentials server-side. Record the exact model, endpoint family, enabled feature policy, and verified behaviors as part of deployment configuration.

Adapter acceptance

A provider is production-ready only when deterministic evidence covers request encoding, fragmented streaming, terminal completion, tool arguments, typed errors, retry safety, timeout, cancellation, truncation, credential redaction, concurrency isolation, and Runtime compatibility.

runifold-provider-testkit supplies cassette, conformance, and benchmark boundaries. Live tests complement protocol tests; they do not replace failure injection or fixed assertions.

Construct and share one runtime

use runifold::{ProductionProfile, ProviderModelExt};
use runifold_providers::openai::OpenAiClient;
 
let runtime = OpenAiClient::from_api_key(std::env::var("OPENAI_API_KEY")?)?
    .runtime_with_preset("gpt-5", ProductionProfile)?;
let health = runtime.route_health();
println!("initial routes: {health:?}");
 
let agent = runtime
    .agent("assistant")
    .system("Answer precisely and expose uncertainty.")
    .build()?;
let answer = agent.prompt_text("Why use a shared runtime?").await?;

Create ProviderRuntime once at service startup and clone it into handlers. Clones share retry and circuit-breaker state. Calling .runtime(...) for every request creates independent health state and defeats coordinated circuit protection. Prefer adapter defaults plus a standard preset; use .runtime_with_profile(...) only at an explicit reviewed override boundary.