feat(rig-openai-responses): add image support for OpenAI reasoning models #207
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
convert_user_content()function to properly handle multimodal contentProblem
The
extract_user_text()function inrig-openai-responsesonly handledTextandToolResultcontent types, ignoringImagecontent entirely. This meant images sent to reasoning models were silently dropped.Solution
convert_user_content()function that handles all content types including imagesUserContent::Imageto async-openai'sInputImageContentformatEasyInputContent::ContentListwhen images are present (required by the OpenAI Responses API for multimodal content)base64dependency for encoding raw image bytesAffected Models
All OpenAI reasoning models now properly support images:
Test Plan
cargo test -p rig-openai-responses)