Skip to content

Add OpenAI Responses API support with reasoning traces and streaming - #316

Open
Sixzero wants to merge 1 commit into
svilupp:mainfrom
Sixzero:response-endpoint-support
Open

Add OpenAI Responses API support with reasoning traces and streaming#316
Sixzero wants to merge 1 commit into
svilupp:mainfrom
Sixzero:response-endpoint-support

Conversation

@Sixzero

@Sixzero Sixzero commented Nov 17, 2025

Copy link
Copy Markdown
Collaborator

Summary

This PR adds support for the OpenAI Responses API, including reasoning traces and streaming, integrated with AbstractResponseSchema.

Key changes

  • Extends the existing airespond / aigenerate interface to work with the OpenAI Responses endpoint via AbstractResponseSchema
  • Introduces create_response and render_responses helpers specialized for the Responses API
  • Enables reasoning traces with configurable summary levels ("detailed", "concise", "none")
  • Adds streaming support for Responses API calls using the existing callback pattern
  • Keeps template / message / string prompts supported without extra placeholder logic

Implementation notes

  • Defaults reasoning summary to "detailed" so reasoning traces are visible by default
  • Only api_kwargs are forwarded into the request body (not all kwargs), keeping the API payload clean
  • Reuses the existing AIMessage shape and statistics reporting
  • No web search or tool-calling helpers are wired in at this stage

Usage examples

schema = OpenAIResponseSchema()

# Basic usage with reasoning traces
response = airespond(schema, "Solve 2+2*3")

# Custom reasoning configuration
response = airespond(schema, "Question";
    api_kwargs = (reasoning = Dict("summary" => "concise"),),
)

# With streaming
response = airespond(schema, "Count to 10"; streamcallback = stdout)

Testing

Existing tests pass locally. The Responses API integration follows the same patterns as other PromptingTools LLM integrations.

@Sixzero
The new Response endpoint support.
2129f73 
also the first response native codex models are added with OpenAIResponseSchema schema
@Sixzero

Sixzero commented Nov 17, 2025

Copy link
Copy Markdown
Collaborator Author

We first I guess need to approve StreamCallbacks PR. svilupp/StreamCallbacks.jl#23

@svilupp

svilupp commented Nov 18, 2025

Copy link
Copy Markdown
Owner

Thank you for the PR! Sorry but I have a crunch at work -- it will take me until the weekend to reply

Btw have you considered opening the PR in OpenAI.jl? Then we would just extend OpenAI.create_response like we do with create_chat -- it would belong in there anyway

svilupp
svilupp requested changes Nov 23, 2025
View reviewed changes

@svilupp svilupp left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the PR!

I left several structural comments on how to organize things.

In addition, could you look into how we build genuine tests to handle this new functionality? See some patterns how I tested the other ones. It would be great to keep the coverage level.

Comment thread src/PromptingTools.jl
include("llm_shared.jl")
include("llm_openai_schema_defs.jl")
include("llm_openai.jl")
include("llm_responses.jl")

@svilupp svilupp Nov 23, 2025

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For consistency, could we perhaps organize things as:
llm_openai_chat.jl and llm_openai_responses.jl and explain it at the top level in the llm_interface.jl

Comment thread src/llm_responses.jl
- `input`: The input string for the API, with replacements
- `instructions`: Optional instructions for the API, or nothing, with replacements
"""
function render_responses end

@svilupp svilupp Nov 23, 2025

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we stick to unified render function? It already has promptSchema as the first argument, so easy to extend with responsesSchema (see the signature elsewhere) ,eg,

PromptingTools.jl/src/llm_shared.jl

Line 33 in 73d7039

function render(schema::NoSchema,

Comment thread src/llm_responses.jl
# Returns
- String with placeholders replaced with values
"""
function replace_placeholders(text::AbstractString, kwargs)

@svilupp svilupp Nov 23, 2025

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't this a bit duplicative with the logic in other files?

Comment thread src/llm_responses.jl
enable_websearch::Bool = false,
model::AbstractString = MODEL_CHAT,
verbose::Bool = true,
api_key::AbstractString = get(ENV, "OPENAI_API_KEY", ""),

@svilupp svilupp Nov 23, 2025

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

doesn't this defy the consistency with other function?

I thought you or Marcel did the PR on empty api_key by default and load from ENV var if empty

Comment thread src/llm_responses.jl

See `?airespond` for detailed documentation of arguments and usage.
"""
function aigenerate(schema::AbstractResponseSchema, prompt::ALLOWED_PROMPT_TYPE; kwargs...)

@svilupp svilupp Nov 23, 2025

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good idea to handle it!

Comment thread src/user_preferences.jl
"gpt5" => "gpt-5",
"gpt5m" => "gpt-5-mini",
"gpt5n" => "gpt-5-nano",
"gpt51" => "gpt-5.1",

@svilupp svilupp Nov 23, 2025

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll do a separate PR with the new models not to block it!

svilupp pushed a commit that referenced this pull request Nov 27, 2025
@web-flow @claude
Add OpenAI Responses API support with refactored implementation
6447570 
This PR adds support for OpenAI's Responses API (/responses endpoint)
used by models like gpt-5.1-codex, addressing review feedback from PR #316:

File organization:
- Renamed llm_openai.jl to llm_openai_chat.jl for clarity
- Created llm_openai_responses.jl for Responses API implementation
- Added documentation in llm_interface.jl explaining OpenAI file structure

API consistency:
- Replaced render_responses with unified render(schema::AbstractResponseSchema, ...)
- Removed duplicative replace_placeholders function (uses render(NoSchema, ...) instead)
- Fixed API key handling to use empty string default with ENV fallback

Testing:
- Added TestEchoOpenAIResponseSchema for testing
- Created comprehensive test suite in test/llm_openai_responses.jl
- All tests pass with 1856 passed, 0 failed

Note: Uses OpenAIStream for streaming until ResponseStream is available
in StreamCallbacks.jl

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
svilupp pushed a commit that referenced this pull request Nov 27, 2025
@web-flow
Add OpenAI Responses API support with refactored implementation
642d1ac 
This PR adds support for OpenAI's Responses API (/responses endpoint)
used by models like gpt-5.1-codex, addressing review feedback from PR #316:

File organization:
- Renamed llm_openai.jl to llm_openai_chat.jl for clarity
- Created llm_openai_responses.jl for Responses API implementation
- Added documentation in llm_interface.jl explaining OpenAI file structure

API consistency:
- Replaced render_responses with unified render(schema::AbstractResponseSchema, ...)
- Removed duplicative replace_placeholders function (uses render(NoSchema, ...) instead)
- Fixed API key handling to use empty string default with ENV fallback

Testing:
- Added TestEchoOpenAIResponseSchema for testing
- Created comprehensive test suite in test/llm_openai_responses.jl
- All tests pass with 1856 passed, 0 failed

Note: Uses OpenAIStream for streaming until ResponseStream is available
in StreamCallbacks.jl
@svilupp svilupp mentioned this pull request Nov 28, 2025
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@svilupp svilupp svilupp requested changes

Requested changes must be addressed to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@Sixzero @svilupp