-
Notifications
You must be signed in to change notification settings - Fork 1
Open
Description
Token streaming, simple generates, and chat completions should have an up-to-standard json output format.
See tests/test_model_api for usages and ways of testing. Most of the changes would need to be made to the api/node, api/models, and ml/validator.
Ideas
- Ensure that the output and input format args perform accordingly
- Add any relevant missing input fields to models
- Reasoning blocks can be separated into a separate field for openai responses, e.g.:
{
"choices": [{
"message": {
"content": "Ahoy there! ⚓🏴☠️ How can I assist you today?",
"reasoning": "<think>...</think>..." // Optional, separate field
}
}]
}Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels