Skip to content

Add AI Triage Workflow and LLM Guidelines for Automated Issue Management#1789

Merged
chagong merged 2 commits intomainfrom
copilot/fix-26bb98a5-9abe-4da9-8dc3-a14df9f918db
Sep 4, 2025
Merged

Add AI Triage Workflow and LLM Guidelines for Automated Issue Management#1789
chagong merged 2 commits intomainfrom
copilot/fix-26bb98a5-9abe-4da9-8dc3-a14df9f918db

Conversation

Copy link
Contributor

Copilot AI commented Sep 3, 2025

This PR adds an AI-powered triage system to automatically label and comment on new issues in the vscode-java-test repository. The implementation is based on the proven workflow from the vscode-gradle repository, adapted specifically for Java testing scenarios.

Changes Made

.github/workflows/triage-agent.yml

  • AI Triage Workflow: Automatically triggers when new issues are opened
  • Manual Dispatch: Supports manual execution for specific issue numbers
  • Azure Function Integration: Calls an external triage service with issue data
  • Repository Configuration: Updated repoId from microsoft/vscode-gradle to microsoft/vscode-java-test
  • Error Handling: Includes comprehensive timeout and error handling for API calls

.github/llms.md

  • LLM Guidelines: Provides structured labeling rules for AI triage
  • Java Test Specific: Tailored content for Java testing functionality
  • Comprehensive Labels: Includes categories for:
    • Issue types (bug, enhancement, documentation, question)
    • Test frameworks (JUnit, TestNG)
    • Project types (Maven, Gradle)
    • Components (test-discovery, test-execution, test-debugging, test-reporting)
    • Priority and status tracking

Benefits

  1. Consistent Labeling: Automated labeling ensures consistent issue categorization
  2. Faster Triage: Reduces manual effort in initial issue processing
  3. Better Organization: Improved issue discoverability through structured labeling
  4. Enhanced User Experience: Faster response times for new issue reporters

Implementation Notes

The workflow requires the following repository secrets and variables to be configured:

  • TRIAGE_FUNCTION_LINK: Azure Function endpoint URL
  • TRIAGE_AGENT_TIMEOUT: Timeout value for triage requests
  • GITHUB_TOKEN: Automatically provided by GitHub Actions

The triage system is designed to be non-disruptive - if the external service is unavailable, issues will simply not be auto-labeled, allowing normal manual triage to continue.


💬 Share your feedback on Copilot coding agent for the chance to win a $200 gift card! Click here to start the survey.

Co-authored-by: chagong <831821+chagong@users.noreply.github.com>
Copilot AI changed the title [WIP] create a new branch, fetch https://github.com/microsoft/vscode-gradle/blob/develop/.github/workflows/triage-agent.yml and https://github.com/microsoft/vscode-gradle/blob/develop/.github/llms.md , add them into current repo, you need to update the paylo... Add AI Triage Workflow and LLM Guidelines for Automated Issue Management Sep 3, 2025
Copilot AI requested a review from chagong September 3, 2025 07:13
@chagong chagong marked this pull request as ready for review September 3, 2025 07:40
@chagong chagong merged commit e3eb3bd into main Sep 4, 2025
4 checks passed
@chagong chagong deleted the copilot/fix-26bb98a5-9abe-4da9-8dc3-a14df9f918db branch September 4, 2025 01:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants