-
Notifications
You must be signed in to change notification settings - Fork 3.2k
Add AI issue assessment workflow with github/ai-assessment-comment-labeler #1498
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 2 commits
cc60c3c
9ec4c81
705ca08
0bfc619
f68072e
5e1b41c
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,32 @@ | ||
| messages: | ||
| - role: system | ||
| content: | | ||
| You are a triage assistant for the GitHub MCP Server repository. This is a Model Context Protocol (MCP) server that connects AI tools to GitHub's platform, enabling AI agents to manage repositories, issues, pull requests, workflows, and more. | ||
|
|
||
| Your job is to analyze bug reports and assess their completeness. | ||
|
|
||
| Analyze the issue for these key elements: | ||
| 1. Clear description of the problem | ||
| 2. Affected version (from running `docker run -i --rm ghcr.io/github/github-mcp-server ./github-mcp-server --version`) | ||
| 3. Steps to reproduce the behavior | ||
| 4. Expected vs actual behavior | ||
| 5. Relevant logs (if applicable) | ||
|
|
||
| Provide ONE of these assessments: | ||
|
|
||
| ### AI Assessment: Ready for Review | ||
| Use when the bug report has most required information and can be triaged by a maintainer. | ||
|
|
||
| ### AI Assessment: Missing Details | ||
| Use when critical information is missing (no reproduction steps, no version info, unclear problem description). | ||
|
|
||
| ### AI Assessment: Unsure | ||
| Use when you cannot determine the completeness of the report. | ||
|
|
||
| After your assessment header, provide a brief explanation of your rating. | ||
| If details are missing, note which specific sections need more information. | ||
| - role: user | ||
| content: "{{input}}" | ||
| model: openai/gpt-4o-mini | ||
| modelParameters: | ||
| max_tokens: 500 |
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -0,0 +1,28 @@ | ||||||
| name: AI Issue Assessment | ||||||
|
|
||||||
| on: | ||||||
| issues: | ||||||
| types: [labeled] | ||||||
|
|
||||||
| jobs: | ||||||
| ai-issue-assessment: | ||||||
| if: github.event.label.name == 'triage-pending' | ||||||
| runs-on: ubuntu-latest | ||||||
| permissions: | ||||||
| issues: write | ||||||
| models: read | ||||||
| contents: read | ||||||
|
|
||||||
| steps: | ||||||
| - name: Checkout | ||||||
| uses: actions/checkout@v4 | ||||||
JoannaaKL marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||||||
|
|
||||||
| - name: Run AI assessment | ||||||
| uses: github/ai-assessment-comment-labeler@e3bedc38cfffa9179fe4cee8f7ecc93bffb3fee7 # v1.0.1 | ||||||
| with: | ||||||
| token: ${{ secrets.GITHUB_TOKEN }} | ||||||
| issue_number: ${{ github.event.issue.number }} | ||||||
| issue_body: ${{ github.event.issue.body }} | ||||||
| ai_review_label: 'triage-pending' | ||||||
| prompts_directory: '.github/prompts' | ||||||
| labels_to_prompts_mapping: 'bug,bug-report-review.prompt.yml' | ||||||
|
||||||
| labels_to_prompts_mapping: 'bug,bug-report-review.prompt.yml' | |
| labels_to_prompts_mapping: 'bug,bug-report-review.prompt.yml;default,default-issue-review.prompt.yml' |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@copilot but default-issue-review.prompt.yml doesn't exist, can you propose how it could look like?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Created default-issue-review.prompt.yml in 5e1b41c. It analyzes new issues to determine their type (bug, feature request, question, etc.) and assesses whether they have enough information for maintainers to act on.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@copilot run this workflow on issues that have label
bugand issues that don't have any labelThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Updated workflow to trigger on issues with
buglabel OR newly opened issues without any labels in 0bfc619.