Use OpenAI control-plane and Realtime APIs

Install the right surfaces

The normal OpenAI adapter, control plane, hosted tools, image generation, speech, and transcription all live behind the openai feature. Realtime is a separate opt-in so ordinary server applications do not pull in its transports.

cargo add runifold@0.9.0
cargo add runifold-providers@0.9.0 --features openai
# Add this only for WebSocket or browser WebRTC Realtime sessions:
cargo add runifold-providers@0.9.0 --features openai,openai-realtime

Two API surfaces

Runifold's OpenAI integration has a data-plane model surface and two specialized surfaces: OpenAiControlPlane for models, files, Batch, and ephemeral Realtime credentials; and a model-bound Realtime connector for WebSocket or browser WebRTC sessions.

These APIs are not Agent shortcuts. They expose lifecycle operations that an application or durable Workflow must coordinate explicitly.

Control plane

OpenAiClient::control_plane() shares validated endpoint, pooled transport, and credential policy. Typed inputs bound file names, upload size, file purpose, Batch endpoints, metadata, and future status values.

The library lists models, uploads bounded files, creates a Batch, reads it once, or requests cancellation. It does not poll implicitly. Persist the Batch identity and schedule explicit inspection under your own deadline, budget, and workflow recovery policy.

Realtime

OpenAiClient::realtime(model) creates a connector with typed commands and events for session updates, text, bounded audio, response creation, cancellation, transcripts, and function-call argument deltas.

The state machine requires session creation before commands, permits one active response, correlates deltas to it, bounds frames and queues, and preserves unknown events. WebRTC and WebSocket share lifecycle, cancellation, and deadline semantics.

Credential boundaries

Native server connections may use the configured long-lived credential. Browser builds reject long-lived provider secrets and require an application-controlled gateway.

Use short-lived client secrets for browser Realtime and obtain a fresh secret for every reconnect attempt. Do not place API keys in WASM, JavaScript, WebSocket query parameters, logs, or persisted session state.

Recovery semantics

There is no automatic replay. A disconnect before session creation or while idle is safe to replace. A disconnect during an active response is AmbiguousResponseInFlight: output may already have committed.

The reconnect controller automates only safe replacement attempts under a bounded policy. It never stores secrets, SDP, transcripts, commands, or model output. Application code must reconcile ambiguous responses and decide whether the user should retry, continue, or start a new session.

Use the control plane

The control plane is explicit: persist returned IDs and schedule later reads in your own workflow. It does not hide polling or retries.

use runifold::ModelCallContext;
use runifold_providers::openai::{OpenAiBatchEndpoint, OpenAiBatchRequest};
 
let control = client.control_plane();
let models = control.list_models(ModelCallContext::new()).await?;
 
let request = OpenAiBatchRequest::new("file_input", OpenAiBatchEndpoint::Responses)?
    .with_metadata("tenant", "acme")?;
let created = control.create_batch(request, ModelCallContext::new()).await?;
persist_batch_id(&created.id).await?;
 
let current = control
    .get_batch(load_batch_id().await?, ModelCallContext::new())
    .await?;

Bound the surrounding workflow with a deadline and a polling interval. Treat unknown future statuses as data to record, not as success. On timeout, retain the Batch ID so an operator or later workflow can reconcile it without submitting the input file again.

Use OpenAI-hosted tools

Hosted tools execute inside the provider, unlike Runifold Tool values that execute in your process. Construct them with typed helpers, convert them into a provider tool spec, and attach them to a ModelRequest:

use runifold::{Message, ModelRef, ModelRequest};
use runifold_providers::openai::OpenAiHostedTool;
 
let request = ModelRequest::new(
    ModelRef::new("openai", "gpt-5"),
    Message::user("Find the latest public release notes and summarize them."),
)
.provider_tool(OpenAiHostedTool::web_search().into())
.provider_tool(
    OpenAiHostedTool::file_search(["vs_product_docs"])?
        .into(),
);

Available typed constructors are web_search(), image_generation(), code_interpreter_auto(), file_search(...), and remote_mcp(label, url). file_search requires 1–100 non-blank vector-store IDs. remote_mcp accepts only HTTP(S) URLs and deliberately preserves OpenAI's approval default. Use with_option only for provider-specific options; the reserved type field cannot be overridden.

Hosted tools are not local capabilities: Runifold cannot enforce your local tool sandbox inside the provider. Restrict models, data sources, vector stores, remote MCP endpoints, spend, and user-visible approval at the application boundary.

Generate images, speech, and transcripts

Media tasks use separate provider-neutral traits instead of pretending binary outputs are chat text. The OpenAI client implements ImageGenerationModel, SpeechModel, and TranscriptionModel:

use runifold::{
    ImageFormat, ImageGenerationModel, ImageGenerationRequest, ModelCallContext,
    ModelRef, SpeechFormat, SpeechModel, SpeechRequest,
};
 
let image = client.generate_image(
    ImageGenerationRequest {
        model: ModelRef::new("openai", "gpt-image-1"),
        prompt: "A precise isometric diagram of a durable workflow".into(),
        count: 1,
        size: Some("1024x1024".into()),
        quality: Some("high".into()),
        format: ImageFormat::Png,
        transparent: false,
    },
    ModelCallContext::new(),
).await?;
 
let speech = client.synthesize_speech(
    SpeechRequest {
        model: ModelRef::new("openai", "gpt-4o-mini-tts"),
        input: "The workflow completed safely.".into(),
        voice: "alloy".into(),
        instructions: None,
        format: SpeechFormat::Mp3,
        speed: None,
    },
    ModelCallContext::new(),
).await?;

For transcription, call TranscriptionModel::transcribe with a bounded TranscriptionRequest containing the file name, media type, bytes, and optional language or prompt. Store returned bytes or remote media deliberately; do not log binary payloads, signed URLs, transcripts, or user audio by default.