Skip to content

Task and Benchmark API #6

@sixtam

Description

@sixtam

Write documentation from a developer point of view of Task and Benchmark API.

Outline includes

  • how to write a task
    • logging
    • using the persistence layer to store results
  • how to write an evaluator
    • retrieving results (for query creation, forward link to persistence.md )
    • where to submit evaluation results (and supported MIME types)
    • what are contexts good for
  • creating & modifying contexts
    • life cycle of tasks, contexts and benchmarks
    • how to write a benchmark
  • how it s similar to a task

Be clear with concepts, reference API when appropriate.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions