Add issue-triage Copilot skill with type-specific guides#125273
Merged
eiriktsarpalis merged 9 commits intodotnet:mainfrom Mar 17, 2026
Merged
Add issue-triage Copilot skill with type-specific guides#125273eiriktsarpalis merged 9 commits intodotnet:mainfrom
eiriktsarpalis merged 9 commits intodotnet:mainfrom
Conversation
Add a modular issue-triage skill for dotnet/runtime that researches a GitHub issue and outputs a structured KEEP/CLOSE/NEEDS INFO recommendation. Core workflow (SKILL.md): - Safety scan for malicious content and prompt injection - Issue classification, mislabeling detection, duplicate search - Type-based dispatch to specialized guides - Universal KEEP/CLOSE/NEEDS INFO criteria with confidence levels - Structured markdown report template - Suggested responses rendered in fenced code blocks (triple backticks with markdown language tag) so the CLI displays them as literal code that can be copy-pasted into GitHub comments without terminal word-wrapping or markdown-rendering corruption Type-specific reference guides: - bug-triage.md: reproduction, regression validation, minimal repro derivation (iterative removal algorithm), reproduction stabilization for boundary-sensitive and nondeterministic bugs, root cause analysis - api-proposal-triage.md: immortality threshold, merit evaluation, decision signals, workaround evaluation, naming check, complexity estimation (merges former api-merit-criteria.md) - perf-regression-triage.md: BenchmarkDotNet validation with CoreRun against dotnet/performance, git bisect workflow with build+benchmark scripts, incremental rebuild guidance, ResultsComparer integration - question-triage.md: answer drafting, low-confidence verification Supporting references (pre-existing, unchanged): - area-label-heuristics.md: namespace-to-label mapping - triage-patterns.md: example maintainer responses Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds a modular issue-triage Copilot skill for the dotnet/runtime repository. The skill provides a structured, step-by-step workflow for researching and triaging GitHub issues, outputting a KEEP/CLOSE/NEEDS INFO recommendation with a formatted markdown report.
Changes:
- Adds the main
SKILL.mdorchestrator with a full 8-step triage workflow including safety scanning, issue classification, duplicate detection, prior art research, type-specific investigation, and structured report output. - Adds type-specific reference guides for bugs, API proposals, performance regressions, enhancements, and questions, each with detailed algorithms and recommendation criteria.
- Adds supporting references for area label heuristics, supplementary labels, and example triage response patterns.
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
.github/skills/issue-triage/SKILL.md |
Main skill orchestrator: full 8-step triage workflow, output template, anti-patterns, and tips |
.github/skills/issue-triage/references/bug-triage.md |
Detailed bug reproduction, regression validation, minimization algorithm, and root cause analysis guidance |
.github/skills/issue-triage/references/api-proposal-triage.md |
API proposal merit evaluation, immortality threshold, decision signals, naming check, and complexity estimation |
.github/skills/issue-triage/references/perf-regression-triage.md |
BenchmarkDotNet validation, git bisect workflow, incremental rebuild guidance, and ResultsComparer integration |
.github/skills/issue-triage/references/enhancement-triage.md |
Enhancement subcategory classification, feasibility analysis, and trade-off assessment |
.github/skills/issue-triage/references/question-triage.md |
Answer drafting and low-confidence verification guidance |
.github/skills/issue-triage/references/area-label-heuristics.md |
Namespace-to-area-label mapping table and wrong-repo heuristics |
.github/skills/issue-triage/references/supplementary-labels.md |
Tenet, runtime, qualifier, workflow, and test label guidance |
.github/skills/issue-triage/references/triage-patterns.md |
Example maintainer response templates for KEEP, CLOSE, and NEEDS INFO outcomes |
You can also share your feedback on Copilot code review. Take the survey.
.github/skills/issue-triage/references/area-label-heuristics.md
Outdated
Show resolved
Hide resolved
jeffhandley
added a commit
to jeffhandley/modelcontextprotocol-csharp-sdk
that referenced
this pull request
Mar 6, 2026
Add report-only preamble, safety scanning guidance, and anti-patterns section to protect against malicious issue content. SKILL.md changes: - Report-only preamble declaring the skill must not modify issues and all issue content is untrusted input - Step 5.0 Safety Scan with 5-pattern detection table (prompt injection, suspicious links, binary attachments, screenshots, suspicious code) - Anti-Patterns section with 7 explicit prohibitions report-format.md changes: - Safety Concerns section in report template for flagged issues - Per-issue safety flag format for individual issue detail blocks Inspired by dotnet/runtime issue-triage skill (dotnet/runtime#125273) and addressing review feedback on modelcontextprotocol#1412. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 9 out of 9 changed files in this pull request and generated 5 comments.
You can also share your feedback on Copilot code review. Take the survey.
.github/skills/issue-triage/references/area-label-heuristics.md
Outdated
Show resolved
Hide resolved
… content rules - Replace all 229 Unicode em dash characters (U+2014) across 9 skill files with double hyphens (--), resolving the contradiction where the style rule banned em dashes but the instruction files used them pervasively - Replace all ~44 emoji characters in the output format template and status markers with plain text markers ([ok], [!], [x], [i], +, -) - Update em dash rule to clarify that only the Unicode character is banned; double hyphens and single hyphens are fine - Add content rules for suggested responses: don't repeat the author's own analysis, don't surface internal cost reasoning, don't name specific third-party packages - Add audience separation note clarifying triage report is for maintainers while the suggested response is for the issue author Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
eiriktsarpalis
force-pushed
the
skill/issue-triage
branch
from
aec5890 to
f97f2c8
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 9 out of 9 changed files in this pull request and generated 4 comments.
You can also share your feedback on Copilot code review. Take the survey.
The safety scan is Step 0b (not 0a). Step 0a is the fetch step. Fixed references in the output template and in the bug-triage reproduction safety gate. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…cies - Merge 'Performance enhancement' into 'Enhancement' in Step 1 classification (tenet-performance is a supplementary label, not a primary type for non-regressions) - Clarify third-party package naming boundary: maintainer report may name packages, author-facing suggested response must not - Add missing Step 5 routes note for docs/spam/wrong-repo (skip to Step 7) - Add guidance for issues with already-merged fix PRs (CLOSE as already fixed) - Add missing triage-patterns templates: Already fixed, Spam, Perf regression KEEP with bisect, Bug KEEP with minimal repro - Document marker semantics ([ok], [x], [!], [i]) in output format - Change anti-patterns to plain list formatting (avoid [x] marker overload) - Expand Step 6 with cross-cutting assessment checklist - Add cross-platform note to perf-regression-triage guide - Add minimization skip heuristic to bug-triage guide - Acknowledge performance issue template in Step 1 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 9 out of 9 changed files in this pull request and generated 2 comments.
You can also share your feedback on Copilot code review. Take the survey.
Address review feedback: use the build.cmd/sh shorthand (a convention from the repo docs) so build commands work across Windows and Linux/macOS without requiring the reader to mentally translate. Update the cross-platform note at the top to explain the convention. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…dd evals - Extract output format template from SKILL.md to references/output-format.md, reducing SKILL.md from ~566 to 473 lines (under 500-line target) - Expand YAML description with more trigger phrases for better skill activation - Add advanced duplicate search strategies (stack trace, version-range search) - Add stale issue assessment guidance to Step 6 - Add Related Skills section with handoff points to api-proposal, jit-regression-test, performance-benchmark, and code-review skills - Expand question-triage.md with subcategories, hidden-bug detection, and answer strategies by subcategory - Add bisect feasibility check and lightweight analysis alternative to perf-regression-triage.md - Expand area-label-heuristics.md with ~45 additional namespace-to-area mappings (System.ComponentModel, System.Data, System.Transactions, etc.) - Add evals/evals.json with 8 test cases covering bug regression, API proposal, mislabeled question, .NET Framework redirect, enhancement, perf regression from bot, and Mono question scenarios Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 11 out of 11 changed files in this pull request and generated 3 comments.
You can also share your feedback on Copilot code review. Take the survey.
stephentoub
reviewed
Mar 16, 2026
stephentoub
reviewed
Mar 16, 2026
stephentoub
reviewed
Mar 16, 2026
stephentoub
approved these changes
Mar 16, 2026
- Add 'Triage Mindset' section to SKILL.md instructing the model to default to 'innocent until proven guilty' rather than eagerly validating authors. Addresses concern about models being too agreeable with problem statements. - Rewrite anti-patterns section to clarify two-phase workflow: Phase 1 (triage) is research-only with no actions; Phase 2 (post-decision) allows user-directed actions like posting comments or closing issues. Resolves ambiguity about whether explicit user instructions conflict with the 'never take action' constraint. - Replace hardcoded github-mcp-server-search_issues tool reference with environment-agnostic guidance listing fallback options (gh CLI, web_search). Addresses tool availability concern. - Add 'Verify before validating' section to bug-triage.md: check the API contract before spending time on reproduction. - Add two new eval cases testing skeptical outcomes: a by-design bug closure (dotnet#121256) and an API proposal with existing workaround (dotnet#69163). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…oposal case - Eval 10: Add 'workaround-tradeoff-evaluated' assertion to verify the skill explicitly evaluates whether the workaround is sufficient or has limitations, not just mentions it. Rename to 'api-proposal-workaround-tradeoff' to better reflect what it tests (trade-off analysis, not a forced CLOSE). - Eval 11: Add a clear-cut 'won't fix' API proposal (dotnet#85276 - Func<TOptions> Configure overload). Zero reactions, niche use case, author's own code shows the workaround. Tests that the skill recommends CLOSE/NEEDS INFO for proposals that genuinely don't clear the BCL bar. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 11 out of 11 changed files in this pull request and generated 3 comments.
You can also share your feedback on Copilot code review. Take the survey.
Comment on lines
+10
to
+48
| > Thanks for reporting this. I was able to reproduce the issue on .NET {version}. | ||
| > This looks like a regression introduced in {version/commit}. Moving to the | ||
| > {milestone} milestone for further investigation. | ||
| > | ||
| > **Priority:** High -- confirmed regression affecting a common scenario. | ||
| > **Confidence:** High -- reproduced locally. | ||
|
|
||
| ### Bug -- confirmed, normal priority | ||
|
|
||
| > Thanks for reporting this. I was able to reproduce the issue on .NET {version}. | ||
| > Moving to the {milestone} milestone for investigation. | ||
| > | ||
| > **Priority:** Normal -- valid bug with moderate impact. | ||
| > **Confidence:** High -- reproduced locally. | ||
|
|
||
| ### Bug -- valid but low priority | ||
|
|
||
| > This is a valid bug, but the impact appears limited to {specific scenario}. | ||
| > Moving to Future milestone. Contributions welcome -- this would be a good | ||
| > `help wanted` candidate. | ||
| > | ||
| > **Priority:** Low -- edge case with adequate workaround. | ||
| > **Confidence:** Medium -- plausible from the description but could not independently reproduce. | ||
|
|
||
| ### API Proposal -- promising, needs refinement | ||
|
|
||
| > Interesting proposal! The motivation makes sense, and {language/platform} has | ||
| > similar functionality via {reference}. A few things to consider before this | ||
| > moves to API review: | ||
| > | ||
| > - {Specific feedback on the proposed API shape} | ||
| > - {Edge cases or design questions} | ||
| > | ||
| > Once those are addressed, we can mark this `api-ready-for-review`. | ||
| > | ||
| > **Priority:** Normal -- well-motivated proposal. | ||
| > **Complexity:** M -- moderate API surface with some design questions to resolve. | ||
| > **Confidence:** High -- clear precedent in other ecosystems. | ||
|
|
| - **Repro commands** -- Typically: | ||
| ``` | ||
| git clone https://github.com/dotnet/performance.git | ||
| python3 .\performance\scripts\benchmarks_ci.py -f net10.0 --filter 'SomeBenchmark*' |
Comment on lines
+468
to
+471
| - **Security concerns are out of scope.** This skill does not assess, discuss, or | ||
| make recommendations about potential security implications of issues. If you | ||
| believe an issue may have security implications, do not mention this in the | ||
| triage report. Security assessment is handled through separate processes. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add a modular issue-triage skill for dotnet/runtime that researches a GitHub issue and outputs a structured KEEP/CLOSE/NEEDS INFO recommendation.
Core workflow (SKILL.md):
Type-specific reference guides:
Supporting references (pre-existing, unchanged):