Agentic AI Architecture

Open Responses: A Primer on the Open LLM API Specification

October 26, 2026 · Vikas Goyal

Every major LLM API now offers the same building blocks, but each encodes them differently. Open Responses is an open, vendor-neutral specification that standardizes how you call models, stream their output and run agentic loops, so you can write a request once and run it on any compliant provider.

Key takeaways

Open Responses is a Linux Foundation project based on the OpenAI Responses API, backed by NVIDIA, Vercel, OpenRouter, Hugging Face, Databricks, Red Hat, AWS, Ollama, vLLM and others.
Its core ideas are typed items, semantic streaming events, state machines and a built-in agentic loop.
Anthropic and Google don't offer native endpoints yet. Adopt the Open Responses shape as your internal contract now and translate at the edge.

What it is

Open Responses is an open, vendor-neutral specification for calling LLMs, streaming their output and running agentic loops. It is based on the OpenAI Responses API. Write a request once and it runs on any compliant provider, whether hosted or local.

What you get: a shared schema, an OpenAPI reference, and acceptance tests for checking that your own endpoint complies.
Who runs it: it is a Linux Foundation project (a Series of LF Projects, LLC), governed by a technical charter and a Technical Steering Committee (TSC).
Who backs it: NVIDIA, Vercel AI SDK, OpenRouter, Hugging Face, LM Studio, Databricks, Red Hat, AWS, Ollama, OpenAI, vLLM and Llama Stack.
Current version: 2026-04-24. The previous version is 2026-01-15.

Why it exists

Every major LLM API now has the same building blocks: messages, tool calls, streaming and multimodal input. Each provider encodes them differently, so switching models means rewriting glue code. Open Responses standardizes that layer.

Portability: describe inputs and outputs once, then run them on OpenAI, Anthropic, Gemini or local models.
Routing and evals: compare providers, route requests and log results through one schema.
A blueprint for providers: model labs can expose their APIs in a format that existing clients already understand.

In short, it is the Responses API without the vendor lock-in.

Core mental model

Four principles explain almost everything in the spec.

Principle	What it means
Agentic loop	A single request can involve several model steps and tool calls before it returns a result.
Items in, items out	Items are the atomic unit of context: a message, a function call, a reasoning step. The same item types work as both input and output, and each is identified by its `type` field.
Semantic events	Streaming sends meaningful transitions, such as `response.output_item.added`, `response.output_text.delta` and `response.completed`, rather than raw token chunks.
State machines	Items and responses move through defined states: `in_progress`, then `completed`, `incomplete` or `failed`. Having a state does not mean an item is stored anywhere.

Every item carries an id, a type and a status. Here is an assistant message next to a tool call, both from one weather lookup:

{ "type": "message", "id": "msg_01", "role": "assistant", "status": "completed",
  "content": [{ "type": "output_text", "text": "Let me check the weather." }] }

{ "type": "function_call", "id": "fc_01", "call_id": "call_42", "status": "completed",
  "name": "get_weather", "arguments": "{\"city\":\"Chicago\"}" }

Content is split into two unions. User content can mix text, images and other modalities. Model content is kept deliberately narrow, mainly output_text, so clients can rely on its shape.

Key request controls

A few request fields control most of a response's behaviour.

Field	What it does	When to use it
`previous_response_id`	Continues from a prior response. The model sees the prior input, then the prior output, then the new input.	Multi-turn chats and tool loops, without resending the transcript
`tool_choice`	Set to `auto` (the default), `required`, `none`, or a specific function	Force a lookup, or disable tools for model-only evals
`allowed_tools`	Limits which tools the model may call while the full `tools` list stays visible. The server must enforce the limit.	Per-tenant or per-role policies that keep prompt caches valid
`truncation`	`auto` lets the server trim older context to fit. `disabled` makes oversized requests fail instead.	Choose between graceful degradation and strict correctness
`service_tier`	A priority hint, such as standard, priority or batch	Workloads where latency matters versus cost

Reasoning items expose the model's thinking in three optional forms. Providers may support any combination of them.

content is the raw reasoning trace, and it can be streamed.
encrypted_content is an opaque payload that the client passes back to the provider unchanged.
summary is a short account that is safe to show end users.

allowed_tools deserves particular attention for enterprise work. It turns tool governance into a request-level setting, with no need to rebuild prompts or schemas.

Tools and the agentic loop

The difference between the two kinds of tool is who holds control while the tool runs.

Internally hosted tools, such as a provider's file search, run inside the provider. The loop continues without coming back to your app.
Externally hosted tools, such as functions, run in your code. The response ends with a function_call. Your app runs the function, then sends a function_call_output together with the previous_response_id. MCP tools are also hosted externally, but the provider calls those servers directly rather than handing control back to you.

The agentic loop: hosted tools versus function calls

A response that contains only a final message ends the loop. Each pass through the loop adds items to the context the model sees.

Transport, streaming and errors

Requests and responses are plain HTTP with JSON. Streaming responses use server-sent events (SSE), and every stream must end with a terminal event: response.completed, response.failed or response.incomplete.

Every streamed item follows the same lifecycle:

response.output_item.added announces the item.
response.content_part.added opens a content part.
One or more response.output_text.delta events carry the text, ending with response.output_text.done.
response.content_part.done closes the content part.
response.output_item.done closes the item, now in its final state.

There are two kinds of event. Delta events report changes to an object, and state machine events report a change in status, such as response.in_progress.

New in 2026-04-24: WebSocket mode

This is a persistent connection on the same /v1/responses resource.

Each turn starts with a response.create message, and the server sends back the same events as SSE.
A connection handles one response at a time. Clients open more connections to run work in parallel.
The server keeps the latest response state in the connection's memory. That makes continuation fast, even with store=false and zero data retention.
Connections last at most 60 minutes. Clients should handle the websocket_connection_limit_reached and previous_response_not_found errors.

Errors

Errors come back as a structured object with type, code, param and message. The main types are invalid_request (400), not_found (404), too_many_requests (429), server_error (500) and model_error (500). An error during a stream is followed by response.failed.

Extensibility, compliance and governance

The spec has a stable core and leaves room for providers to add features that aren't standard yet.

Vendor prefix: custom item types, hosted tools and stream events must be prefixed with the provider's slug, such as openai:web_search_call or acme:trace_event. Clients must be able to skip types they don't recognize.
Hosted tools need receipts: every custom tool needs a matching item type that records the call, its status and its result. That record must be detailed enough to send back in a later request without losing anything.
Compliance: an API is compliant if it implements the spec or a proper superset of it. The acceptance tests check this.
Governance: extensions that become widely adopted can be promoted into the core spec by the TSC, under the project's technical charter.

Recommendations for teams building agentic apps today

Adopt the Open Responses shape as your internal contract now, and translate at the edge. Anthropic and Google don't offer native Open Responses endpoints yet, but gateways such as Databricks already translate Open Responses requests to Claude and Gemini, with a narrower set of features.

Model the agent loop on Open Responses items, not on a vendor SDK. Use typed items such as message, function_call, function_call_output and reasoning, each with an id, type and status. Your traces, evals and memory then stay provider-neutral.
Put a thin adapter or gateway at the edge. Native providers go straight through, and Anthropic and Gemini get translated. Nothing upstream of the adapter imports a vendor SDK.
Standardize on function tools and MCP. Function calls carry over cleanly between providers. Treat provider-hosted tools, such as web search or code execution, as slug-prefixed extensions that won't port.
Pass reasoning state back unchanged. Gemini attaches an encrypted_content value to every function call, and Anthropic's thinking blocks need the same treatment. Multi-turn tool loops break if your adapter drops or alters them.
Own your conversation state. Store the item history yourself and resend it, rather than relying on server-side previous_response_id across providers. This also keeps zero-data-retention deployments working.
Write a provider conformance suite. Run the same scenarios against every model: a tool loop, a parallel tool call, streaming and an error path. It shows exactly where each adapter leaks.
Keep a flagged escape hatch for each provider. When a native feature is worth the lock-in, such as prompt caching or long context, call the native API behind a flag and document the exception.

Layer	Portable?	Owner
Agent loop, item schema, traces, evals	Yes, using the Open Responses shape	Your harness
Tool definitions (functions, MCP)	Yes	Your harness
Wire protocol and translation	No, one adapter per provider	Gateway
Hosted tools, caching, provider-only features	No	Flagged exceptions

When Anthropic or Google ship native support, you delete an adapter rather than rewrite your agents.

Getting started

Read the specification, starting with items, streaming events and tools.
Browse the OpenAPI reference for the full set of types.
Run the acceptance tests against your endpoint.
Read the governance charter if you plan to contribute.