Skip to content

Add skill: CodeRabbit review extraction to knowledge base #89

@jordanpartridge

Description

@jordanpartridge

Problem

CodeRabbit generates valuable code review insights on every PR, but they're trapped in GitHub comments. We lose this institutional knowledge once the PR is merged.

Proposed Solution

Create a skill /coderabbit:extract that:

  1. Fetches CodeRabbit review from current or specified PR
  2. Parses review into structured findings
  3. Rates each finding by severity and confidence
  4. Adds valuable insights to knowledge base with semantic IDs

Usage

# Extract from current PR branch
/coderabbit:extract

# Extract from specific PR
/coderabbit:extract --pr=87

# Filter by severity
/coderabbit:extract --pr=87 --min-severity=major

Knowledge Entry Format

Each CodeRabbit finding becomes:

  • ID: knowledge-pr-{number}-coderabbit-{finding-number}
  • Title: Finding summary
  • Content: Full description, code snippet, proposed fix
  • Category: code-review
  • Tags: coderabbit, pr-{number}, severity level
  • Priority: Maps from severity (critical→high, major→medium, etc.)
  • Confidence: Rated 30-95% based on finding quality

Benefits

  • Searchable review history: ./know search "command injection security"
  • Learn from past reviews: Agents discover previous code review patterns
  • Filter noise: Only index high-confidence findings
  • Track patterns: See recurring issues across PRs
  • Context for agents: Future agents learn from CodeRabbit's analysis

Implementation

  1. Use gh api to fetch CodeRabbit review body
  2. Parse markdown structure (Critical/Major/Minor/Nitpick sections)
  3. Extract findings with regex/parser
  4. Rate confidence based on:
    • Severity tag (Critical=95%, Major=85%, Minor=75%, Nitpick=60%)
    • Finding type (security issues higher confidence than style)
    • Code provided (findings with diffs more valuable)
  5. Call ./know add for each finding
  6. Generate summary report

Example Output

📊 CodeRabbit Review Extraction - PR #87

Findings:
✅ 3 Critical (95% confidence) → Added to knowledge
✅ 4 Major (85% confidence) → Added to knowledge  
⏭️  10 Minor (75% confidence) → Skipped (below threshold)
⏭️  36 Nitpicks (60% confidence) → Skipped

Knowledge entries created:
- knowledge-pr-87-coderabbit-1: Delete KnowledgeIndexCommandTest.php
- knowledge-pr-87-coderabbit-2: Update composer.json metadata
- knowledge-pr-87-coderabbit-3: Command injection in TestExecutorService
...

Use: ./know search "coderabbit pr-87" to view all findings

Future Enhancements

  • Auto-run on PR merge via GitHub Actions
  • Categorize findings by file/service
  • Link to PR discussion threads
  • Track which findings were actually fixed
  • Generate "lessons learned" summaries per project

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions