Implement Generated Case Runner

[jharris1679]

Overview

Implement LLM-powered case generation that discovers improvement opportunities specific to the user's repository.

Generation Templates

Template	What LLM Looks For	Output
style-conformance	Code not matching project style	"Fix naming in X"
missing-coverage	Untested public functions	"Add tests for X"
duplication	Repeated code patterns	"Extract X to shared util"
type-safety	Weak/missing types	"Add types to X"
dependency-update	Deprecated API usage	"Update X to modern API"

Tasks

Template System

• Define template YAML schema
• Create 5 generation templates
• Template includes: analysis prompt, output schema, validation rules

Generation Pipeline

• sniff generate command
• Analyze codebase with LLM (Claude)
• LLM outputs structured case definitions
• Validate generated cases against schema
• Store in cases/generated/

Quality Assurance

• Confidence scoring for generated cases
• Filter low-confidence cases
• Human review interface before running
• Deduplication (don't generate same case twice)

Case Execution

• Generated cases use standard rubric
• Track which template generated each case
• Include detection evidence in results

Acceptance Criteria

• 5 generation templates defined and working
• LLM analyzes codebase and produces valid cases
• Generated cases include evidence and confidence
• Human review workflow before evaluation
• Cases run with standard evaluation pipeline