Skip to content

Standardize API Output Formats #99

@mattjhawken

Description

@mattjhawken

Token streaming, simple generates, and chat completions should have an up-to-standard json output format.

See tests/test_model_api for usages and ways of testing. Most of the changes would need to be made to the api/node, api/models, and ml/validator.

Ideas

  • Ensure that the output and input format args perform accordingly
  • Add any relevant missing input fields to models
  • Reasoning blocks can be separated into a separate field for openai responses, e.g.:
 {
   "choices": [{
     "message": {
       "content": "Ahoy there! ⚓🏴‍☠️ How can I assist you today?",
       "reasoning": "<think>...</think>..."  // Optional, separate field
     }
   }]
}

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions