Mistral: implement native output mode for structured outputs #3662
Conversation
Mistral API supports grammar for json mode for some time now. This changes make it possible to leverage it.
Some azure tests were failing because the schema transformer should be different for different model manufacturers that deploy in Azure
|
I will fix the tests ASAP |
| assert output_object is not None | ||
| json_schema = self._map_json_schema(output_object) | ||
| return MistralResponseFormat(type='json_schema', json_schema=json_schema) | ||
| elif model_request_parameters.output_mode == 'prompted' and not model_request_parameters.function_tools: |
There was a problem hiding this comment.
I don't think we need not model_request_parameters.function_tool
| name=o.name or 'output', | ||
| schema_definition=o.json_schema, | ||
| description=o.description or UNSET, | ||
| strict=o.strict if o.strict is not None else None, |
There was a problem hiding this comment.
This is just strict=o.strict right?
| # Determine the response format based on output mode | ||
| response_format = self._get_response_format(model_request_parameters) | ||
|
|
||
| # When using native JSON schema mode, don't use tool-based output |
There was a problem hiding this comment.
I don't think we need to branch on this: if model_request_parameters.output_mode == 'native', model_request_parameters.output_tools will be empty anyway. Note that we don't have logic like this in the other models
| # Determine the response format based on output mode | ||
| response_format = self._get_response_format(model_request_parameters) | ||
|
|
||
| # When using native JSON schema mode, don't use tool-based output |
There was a problem hiding this comment.
See above; I don't think we need this. And if we do, we should dedupe it between the request/request_stream methods :)
There was a problem hiding this comment.
There are a few TODOs in the file that we should address at the same time:
# TODO: Should be able to use json_schema# TODO: Port to native "manual JSON" mode
These are related to an old approach for handling output_tools by passing the schemas to the API as user text parts, which I don't think we need anymore, so we should implement the tool, native, and prompted modes the way the other models do.
2 participants
As noted in some TODOs, Mistral has been supporting constrained json grammar for some time now. Following Mistral 3 release last week, I wanted to check how well it does. This PR integrates it into pydantic-ai:
MistralJsonSchemaTransformerfor compatibility with Mistral's APIsupports_json_object_outputtoTrue🥳pydantic_ai_slim/pydantic_ai/models/google.pygoesI've been playing around with it and it looks fine so far. Please let me know of any changes u'd like to see <3