ralph: #23 — Design Bootstrap Cases for common real tasks — create 5 dynamic case YAMLs#50
ralph: #23 — Design Bootstrap Cases for common real tasks — create 5 dynamic case YAMLs#50jharris1679 wants to merge 1 commit intomainfrom
Conversation
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
There was a problem hiding this comment.
Your free trial has ended. If you'd like to continue receiving code reviews, you can add a payment method here.
WalkthroughFive new bootstrap YAML case files introduce standardized coding tasks: linting error fixes, symbol renaming, code deduplication extraction, TypeScript type annotations, and deprecated API updates. Each specifies task metadata, target file templates with initial content, and evaluation rubrics using command-based validators (linter, grep, type-checker, tests) to verify task completion. Changes
Estimated code review effort🎯 2 (Simple) | ⏱️ ~12 minutes Poem
🚥 Pre-merge checks | ✅ 6✅ Passed checks (6 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Fix all issues with AI agents
In `@cases/bootstrap/boot-002-rename-symbol.yaml`:
- Around line 38-48: Add an evaluator to the rename-completeness criterion that
verifies the old symbol is removed by running ${SEARCH_COMMAND} against
"${OLD_NAME}" with a low passThreshold (e.g. 0.1); update
criteria.rename-completeness.evaluators (alongside the existing verifier for
${NEW_NAME}) to include a new evaluator entry named like "Verify old name
removed" that runs ${SEARCH_COMMAND} "${OLD_NAME}" and uses passThreshold: 0.1
so the rubric asserts absence of ${OLD_NAME}.
In `@cases/bootstrap/boot-004-add-types.yaml`:
- Line 18: The metadata field 'language' currently set to 'javascript' must be
changed to 'typescript' to match the task and evaluator; locate the YAML entry
named language (value 'javascript') in the boot-004-add-types.yaml test case and
replace its value with 'typescript' so the task metadata, tags, and type-checker
align.
🧹 Nitpick comments (1)
cases/bootstrap/boot-002-rename-symbol.yaml (1)
28-36: Template code usesnewon a plain function, which is misleading.Line 34 calls
new ${OLD_NAME}()on a function that simply returnsx * 2. Usingnewon a non-constructor function silently discards the return value and yields an empty object. For a rename exercise template, a plain function call (const result = ${OLD_NAME}(5)) would be more representative and less confusing to the agent.Suggested template fix
function ${OLD_NAME}(x) { return x * 2; } - const ${OLD_NAME}_instance = new ${OLD_NAME}(); + const ${OLD_NAME}_result = ${OLD_NAME}(5); module.exports = { ${OLD_NAME} };
| rubric: | ||
| extends: default | ||
| criteria: | ||
| rename-completeness: | ||
| weight: 50 | ||
| description: "All occurrences of ${OLD_NAME} are renamed to ${NEW_NAME}" | ||
| evaluators: | ||
| - type: command | ||
| name: "Verify new name exists" | ||
| run: ${SEARCH_COMMAND} "${NEW_NAME}" | ||
| passThreshold: 0.9 |
There was a problem hiding this comment.
Missing evaluator to verify the old symbol is removed.
The rubric only checks that ${NEW_NAME} exists but never verifies that ${OLD_NAME} is absent. An agent could pass this criterion by simply adding the new name without removing the old one. The sibling cases (boot-003, boot-005) use a search + low passThreshold (0.1) to assert pattern absence — this case should do the same.
Proposed fix: add an evaluator for old-name absence
rubric:
extends: default
criteria:
rename-completeness:
weight: 50
description: "All occurrences of ${OLD_NAME} are renamed to ${NEW_NAME}"
evaluators:
- type: command
name: "Verify new name exists"
run: ${SEARCH_COMMAND} "${NEW_NAME}"
passThreshold: 0.9
+ - type: command
+ name: "Verify old name is gone"
+ run: ${SEARCH_COMMAND} "${OLD_NAME}"
+ passThreshold: 0.1
tests:📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| rubric: | |
| extends: default | |
| criteria: | |
| rename-completeness: | |
| weight: 50 | |
| description: "All occurrences of ${OLD_NAME} are renamed to ${NEW_NAME}" | |
| evaluators: | |
| - type: command | |
| name: "Verify new name exists" | |
| run: ${SEARCH_COMMAND} "${NEW_NAME}" | |
| passThreshold: 0.9 | |
| rubric: | |
| extends: default | |
| criteria: | |
| rename-completeness: | |
| weight: 50 | |
| description: "All occurrences of ${OLD_NAME} are renamed to ${NEW_NAME}" | |
| evaluators: | |
| - type: command | |
| name: "Verify new name exists" | |
| run: ${SEARCH_COMMAND} "${NEW_NAME}" | |
| passThreshold: 0.9 | |
| - type: command | |
| name: "Verify old name is gone" | |
| run: ${SEARCH_COMMAND} "${OLD_NAME}" | |
| passThreshold: 0.1 |
🤖 Prompt for AI Agents
In `@cases/bootstrap/boot-002-rename-symbol.yaml` around lines 38 - 48, Add an
evaluator to the rename-completeness criterion that verifies the old symbol is
removed by running ${SEARCH_COMMAND} against "${OLD_NAME}" with a low
passThreshold (e.g. 0.1); update criteria.rename-completeness.evaluators
(alongside the existing verifier for ${NEW_NAME}) to include a new evaluator
entry named like "Verify old name removed" that runs ${SEARCH_COMMAND}
"${OLD_NAME}" and uses passThreshold: 0.1 so the rubric asserts absence of
${OLD_NAME}.
| All functions and variables should have proper type annotations. | ||
| source: bootstrap | ||
| category: type-safety | ||
| language: javascript |
There was a problem hiding this comment.
language should be typescript, not javascript.
The task is explicitly about adding TypeScript type annotations, the tags include typescript, and the evaluator runs a type-checker. The metadata field should match.
Fix
-language: javascript
+language: typescript📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| language: javascript | |
| language: typescript |
🤖 Prompt for AI Agents
In `@cases/bootstrap/boot-004-add-types.yaml` at line 18, The metadata field
'language' currently set to 'javascript' must be changed to 'typescript' to
match the task and evaluator; locate the YAML entry named language (value
'javascript') in the boot-004-add-types.yaml test case and replace its value
with 'typescript' so the task metadata, tags, and type-checker align.
1 participant
Issue
Closes #23
Status: ✓ verified
Build, tests, and lint all pass locally.
Summary
Automated implementation by Ralph (rlmkit + MiniMax M2.5).
Review the changes carefully — this was generated by a local model.
Summary by CodeRabbit
New Features