Skip to content

Commit 4217355

Browse files
huangmingxiahuangmingxia
committed
Migrate the test-generation-assistant into the .ai_testgen folder.
1 parent a175f00 commit 4217355

32 files changed

+3389
-0
lines changed

‎.ai_testgen/CLAUDE.md‎

Lines changed: 127 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,127 @@
1+
# CLAUDE.md
2+
3+
## MANDATORY Protocol
4+
5+
Before executing ANY user request, Claude MUST:
6+
1. Read this CLAUDE.md file first
7+
2. Read prompts/STARTUP_CHECKLIST.md
8+
3. **CRITICAL: NEVER use Task tool for agents**
9+
4. Identify the correct agent YAML config file
10+
5. Read the agent YAML config directly using Read tool
11+
6. Execute EACH step in the agent configuration manually using available tools
12+
7. Verify each step's output before proceeding to next step
13+
14+
**MANDATORY** = Required, cannot be skipped
15+
16+
## Why NOT to use Task Tool
17+
- Task tool uses general-purpose subagent that doesn't follow YAML configs exactly
18+
- Steps get skipped, simplified, or executed incorrectly
19+
- Results in incomplete or wrong outputs
20+
- Agent configurations are designed for direct execution only
21+
22+
## Supported Workflows
23+
24+
### Workflow Execution Order
25+
The workflows can be executed independently or in sequence:
26+
27+
**Complete End-to-End Flow:**
28+
1. Generate Test Cases → 2. Generate E2E Test Code → 2.1 Execute E2E Tests → 3. Submit E2E Pull Request → 4. Update Polarion → 5. Update JIRA QE Comment
29+
30+
**Dependencies:**
31+
- Workflow 2.1 requires completion of Workflow 2
32+
- Workflow 3 requires completion of Workflow 2 (and optionally 2.1)
33+
- Workflow 4 requires completion of Workflow 1
34+
- Workflow 5 can be executed after any workflow completion
35+
36+
### 1. Generate Test Cases
37+
- **Input**: Create test case for JIRA issue key (e.g., "Create test case for HIVE-2883")
38+
- **Action**: NEVER use Task tool
39+
- **Execution Steps:**
40+
1. Read `config/agents/test_case_generation.yaml` using Read tool
41+
2. Execute each step in `task.steps` section manually
42+
3. Use tools: jira-mcp-snowflake MCP, WebFetch, Read, Write
43+
4. Verify outputs match `output.files` specifications
44+
- **Agent config**: `config/agents/test_case_generation.yaml`
45+
- **Output**: Test cases in markdown format for Polarion integration
46+
47+
### 2. Generate E2E Test Code
48+
- **Input**: Request to generate E2E code for JIRA issue key (e.g., "generate E2E test case for HIVE-2883")
49+
- **Action**: NEVER use Task tool
50+
- **Execution Steps:**
51+
1. Read `config/agents/e2e_test_generation_openshift_private.yaml` using Read tool
52+
2. Execute each subtask in `task.subtasks` section sequentially
53+
3. Each subtask has its own steps and success_criteria
54+
4. Use tools: Bash, Read, Write, Edit, WebFetch
55+
5. Verify each subtask's success_criteria before proceeding
56+
- **Agent config**: `config/agents/e2e_test_generation_openshift_private.yaml`
57+
- **Output**: E2E test code integrated into openshift-tests-private repository
58+
59+
### 2.1 Execute E2E Tests
60+
- **Input**: Run the generated E2E test cases and capture their outcomes
61+
- **Action**: NEVER use Task tool
62+
- **Execution Steps:**
63+
1. Read `config/agents/test-executor.yaml` using Read tool
64+
2. Execute each subtask in `task.subtasks` section sequentially
65+
3. Each subtask has its own steps and success_criteria
66+
4. Use tools: Bash, Read, Write, Edit, WebFetch
67+
5. Capture test results and logs
68+
- **Agent config**: `config/agents/test-executor.yaml`
69+
- **Output**: Test execution results and reports
70+
71+
### 3. Submit E2E Pull Request
72+
- **Input**: Request to create PR for generated E2E tests
73+
- **Action**: NEVER use Task tool
74+
- **Tools**: Bash (git, gh CLI), Read, Write, jira-mcp-snowflake MCP
75+
- **Output**: Created pull request with E2E test code and JIRA updated
76+
77+
### 4. Update Polarion [In Progress]
78+
- **Input**: Request to update Polarion with test case information
79+
- **Action**: NEVER use Task tool
80+
- **Execution Steps:**
81+
1. Read generated test case files
82+
2. Format according to Polarion requirements
83+
3. Use Polarion API integration tools
84+
4. Upload/update test cases in Polarion system
85+
- **Tools**: Read, Write, WebFetch (for API calls)
86+
- **Output**: Updated Polarion test cases
87+
88+
### 5. Update JIRA QE Comment [In Progress]
89+
- **Input**: Request to add QE comment to update QE test status
90+
91+
## Workflow Usage Examples
92+
93+
### Example 1: Complete Flow for HIVE-2883
94+
```
95+
User: "Create test case for HIVE-2883"
96+
→ Execute Workflow 1
97+
98+
User: "Generate E2E code for HIVE-2883"
99+
→ Execute Workflow 2
100+
101+
User: "Run the E2E tests for HIVE-2883"
102+
→ Execute Workflow 2.1
103+
104+
User: "Submit PR for HIVE-2883 E2E tests"
105+
→ Execute Workflow 3
106+
107+
User: "Update Polarion with HIVE-2883 test cases"
108+
→ Execute Workflow 4
109+
110+
User: "Add QE comment to HIVE-2883"
111+
→ Execute Workflow 5
112+
```
113+
114+
### Example 2: Individual Workflows
115+
```
116+
User: "Just generate test cases for CCO-1234"
117+
→ Execute only Workflow 1
118+
119+
User: "Create E2E tests for HIVE-5678 and submit PR"
120+
→ Execute Workflow 2 → Workflow 3
121+
```
122+
123+
## Key Rules
124+
- Always read agent YAML configs directly
125+
- Execute each step manually using available tools
126+
- Verify each step before proceeding
127+
- Follow agent configurations exactly

‎.ai_testgen/GEMINI.md‎

Whitespace-only changes.

‎.ai_testgen/config/agents/e2e_test_generation_openshift_private.yaml‎

Lines changed: 110 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,110 @@
1+
agent:
2+
name: "e2e_test_generation_openshift_private"
3+
display_name: "OpenShift Tests Private E2E Generator"
4+
description: "Independent workflow: Generate E2E test code in openshift-tests-private repository based on existing test cases. Focuses on integration and end-to-end testing."
5+
6+
role:
7+
title: "OpenShift Private E2E Generator"
8+
focus: "Integration testing and end-to-end workflows"
9+
10+
task:
11+
primary_goal: "Generate E2E tests in openshift-tests-private repository"
12+
13+
subtasks:
14+
- name: "OpenShift Tests Private Repository Setup"
15+
timeout: "40s"
16+
input_requirements: ["jira_issue_key", "requirements_input"]
17+
steps:
18+
- "Extract component name from test requirements output"
19+
- "Create temp_repos directory if it doesn't exist: mkdir -p temp_repos"
20+
- "Check if openshift-tests-private repository exists: temp_repos/openshift-tests-private/"
21+
- "cd into temp_repos/openshift-tests-private/"
22+
# Replace with your own repository
23+
- "If not exists: Clone https://github.com/huangmingxia/openshift-tests-private.git to temp_repos/openshift-tests-private/"
24+
- "Check git status and branch"
25+
- "Switch to master branch: git checkout master"
26+
- "Fetch latest: git pull origin master (from your own repository)"
27+
- "Create or checkout branch: ai-case-design-{JIRA_KEY} from master branch"
28+
- "Ensure working directory is clean and ready"
29+
success_criteria:
30+
- "openshift-tests-private repository ready"
31+
- "Branch ai-case-design-{JIRA_KEY} ready for development"
32+
- "Repository is in clean state"
33+
34+
- name: "Integration E2E Test Generation"
35+
timeout: "120s"
36+
input_requirements: ["requirements_input", "test_case_input", "jira_issue_key"]
37+
steps:
38+
# - "Load test case from Polarion, filter test case by jira issue key "
39+
- "Load and apply E2E generation rules from local directory: config/rules/e2e_rules/e2e_test_case_guidelines_test_private.md"
40+
- "Analyze existing E2E tests in openshift-tests-private to learn Ginkgo/Gomega patterns"
41+
- "Focus on end-to-end workflows and multi-component integration testing"
42+
- "Extract test scenarios suitable for integration E2E validation from test_case_input"
43+
- "Generate E2E test code following openshift-tests-private patterns"
44+
- "Convert {JIRA_KEY} to lowercase and remove symbols for testCaseID (e.g., HIVE-2883 becomes hive2883)"
45+
- "Write test file in test/extended/cluster_operator/{COMPONENT}/ following Ginkgo structure"
46+
- "Use converted testCaseID for all resource naming to ensure RFC 1123 compliance"
47+
- "Add AI authorship information: '// Auto-generated by AI Test Generation Assistant (OpenShift Private)' and '// JIRA: {JIRA_KEY}'"
48+
- "Include appropriate test selectors and metadata"
49+
success_criteria:
50+
- "Integration E2E test file created in openshift-tests-private"
51+
- "Test logic focuses on end-to-end workflows"
52+
53+
- name: "OpenShift Private Validation"
54+
timeout: "60s"
55+
input_requirements: ["jira_issue_key", "requirements_input"]
56+
steps:
57+
- "Load validation commands from openshift-tests-private E2E rules"
58+
- "Run 'make all' to validate compilation and formatting"
59+
- "Check output for errors (syntax, imports, formatting)"
60+
- "If errors detected:"
61+
- " Attempt to fix common issues (missing imports, formatting via go fmt)"
62+
- " Re-run 'make all' to validate fixes"
63+
- " If still failing, log errors for manual intervention"
64+
success_criteria:
65+
- "openshift-tests-private E2E tests compile and validate successfully"
66+
67+
input:
68+
required:
69+
- name: "jira_issue_key"
70+
type: "string"
71+
description: "JIRA issue key"
72+
example: "HIVE-2883"
73+
- name: "requirements_input"
74+
type: "file"
75+
description: "Test requirements"
76+
path: "workflow_outputs/{COMPONENT}/{JIRA_KEY}/phases/test_requirements_output.yaml"
77+
- name: "test_case_input"
78+
type: "file"
79+
description: "Test case scenarios for integration E2E analysis"
80+
path: "workflow_outputs/{COMPONENT}/{JIRA_KEY}/test_cases/{JIRA_KEY}_test_case.md"
81+
82+
output:
83+
files:
84+
- name: "{JIRA_KEY}_openshift_private_e2e_results.md"
85+
path: "workflow_outputs/{COMPONENT}/{JIRA_KEY}/phases/{JIRA_KEY}_openshift_private_e2e_results.md"
86+
format: "markdown"
87+
description: "OpenShift Tests Private E2E test generation results and repository integration status"
88+
template: "config/templates/openshift_private_e2e_results_template.md"
89+
90+
tools:
91+
- name: "Git operations"
92+
description: "openshift-tests-private repository cloning, branching, and management"
93+
operations: ["git clone", "git checkout", "git merge", "git push"]
94+
- name: "File operations"
95+
description: "Read inputs, write integration E2E test files, and result documentation"
96+
operations: ["Read", "Write", "Glob", "Edit"]
97+
- name: "Build validation"
98+
description: "Run openshift-tests-private validation commands"
99+
operations: ["make all", "make verify"]
100+
101+
requires_mcp_tools: ["WebFetch", "Terminal"]
102+
103+
configuration:
104+
dynamic_variables:
105+
- "{COMPONENT}"
106+
- "{JIRA_KEY}"
107+
repository_management: true
108+
validation_required: true
109+
template_replacement: true
110+
repository_type: "openshift_tests_private"

‎.ai_testgen/config/agents/e2e_test_generation_upstream.yaml‎

Lines changed: 109 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,109 @@
1+
agent:
2+
name: "e2e_test_generation_upstream"
3+
display_name: "Upstream E2E Test Generator"
4+
description: "Independent workflow: Generate E2E test code in upstream component repository based on existing test cases. Focuses on component-specific testing."
5+
6+
role:
7+
title: "Upstream E2E Generator"
8+
9+
task:
10+
primary_goal: "Generate E2E tests in upstream component repository"
11+
12+
subtasks:
13+
- name: "Upstream Repository Setup"
14+
timeout: "40s"
15+
input_requirements: ["jira_issue_key", "requirements_input"]
16+
steps:
17+
- "Extract component name from test requirements output"
18+
- "Create temp_repos directory if it doesn't exist: mkdir -p temp_repos"
19+
- "Load E2E generation rules from local directory: config/rules/e2e_rules/e2e_test_case_guidelines.md"
20+
- "Check if component repository exists: temp_repos/{COMPONENT}/"
21+
- "cd into temp_repos/{COMPONENT}/"
22+
# Replace with your own repository
23+
- "If not exists: Clone https://github.com/huangmingxia/{COMPONENT}.git to temp_repos/{COMPONENT}/"
24+
- "Check git status and branch"
25+
- "Switch to main/master branch: git checkout main"
26+
- "Fetch latest: git pull origin main (from your own repository)"
27+
- "Create or checkout branch: ai-e2e-{JIRA_KEY} from main branch"
28+
- "Ensure working directory is clean and ready"
29+
success_criteria:
30+
- "Upstream component repository ready"
31+
- "Branch ai-e2e-{JIRA_KEY} ready for development"
32+
- "Repository is in clean state"
33+
34+
- name: "Component E2E Test Generation"
35+
timeout: "120s"
36+
input_requirements: ["requirements_input", "test_case_input", "jira_issue_key"]
37+
steps:
38+
# - "Load test case from Polarion, filter test case by jira issue key (if needed)"
39+
- "Load upstream E2E rules from local directory: config/rules/e2e_rules/e2e_test_case_guidelines.md"
40+
- "Analyze existing E2E tests in upstream repo to learn testing patterns"
41+
- "Focus on component controller behavior and CRD lifecycle testing"
42+
- "Extract test scenarios suitable for component-level E2E validation from test_case_input"
43+
- "Generate component-focused E2E test code following upstream patterns"
44+
- "Write test file in upstream repo following component test structure"
45+
- "Add AI authorship information: '// Auto-generated by AI Test Generation Assistant (Upstream)' and '// JIRA: {JIRA_KEY}'"
46+
success_criteria:
47+
- "Component E2E test file created in upstream repo"
48+
- "Test logic focuses on component behavior"
49+
50+
- name: "Upstream Validation"
51+
timeout: "60s"
52+
input_requirements: ["jira_issue_key"]
53+
steps:
54+
- "Load validation commands from upstream E2E rules"
55+
- "Run 'make verify' or component-specific E2E validation command"
56+
- "Check output for errors (syntax, imports, formatting)"
57+
- "If errors detected:"
58+
- " Attempt to fix common issues (missing imports, formatting)"
59+
- " Re-run validation command"
60+
- " If still failing, log errors for manual intervention"
61+
success_criteria:
62+
- "Upstream E2E tests compile and validate successfully"
63+
64+
input:
65+
required:
66+
- name: "jira_issue_key"
67+
type: "string"
68+
description: "JIRA issue key"
69+
example: "HIVE-2883"
70+
- name: "requirements_input"
71+
type: "file"
72+
description: "Test requirements"
73+
path: "workflow_outputs/{COMPONENT}/{JIRA_KEY}/phases/test_requirements_output.yaml"
74+
- name: "test_case_input"
75+
type: "file"
76+
description: "Test case scenarios for upstream E2E analysis"
77+
path: "workflow_outputs/{COMPONENT}/{JIRA_KEY}/test_cases/{JIRA_KEY}_test_case.md"
78+
79+
output:
80+
files:
81+
- name: "{JIRA_KEY}_upstream_e2e_results.md"
82+
path: "workflow_outputs/{COMPONENT}/{JIRA_KEY}/phases/{JIRA_KEY}_upstream_e2e_results.md"
83+
format: "markdown"
84+
description: "Upstream E2E test generation results and repository integration status"
85+
template: "config/templates/upstream_e2e_results_template.md"
86+
87+
tools:
88+
- name: "Git operations"
89+
description: "Upstream repository cloning, branching, and management"
90+
operations: ["git clone", "git checkout", "git merge", "git push"]
91+
- name: "File operations"
92+
description: "Read inputs, write upstream E2E test files, and result documentation"
93+
operations: ["Read", "Write", "Glob", "Edit"]
94+
- name: "Build validation"
95+
description: "Run upstream component E2E validation commands"
96+
operations: ["make test-e2e", "make verify"]
97+
- name: "Repository discovery"
98+
description: "Intelligent GitHub repository discovery and validation"
99+
operations: ["GitHub API search", "Repository metadata analysis", "Component mapping cache"]
100+
requires_mcp_tools: ["WebFetch", "Terminal"]
101+
102+
configuration:
103+
dynamic_variables:
104+
- "{COMPONENT}"
105+
- "{JIRA_KEY}"
106+
repository_management: true
107+
validation_required: true
108+
template_replacement: true
109+
repository_type: "upstream_component"

0 commit comments

Comments
 (0)