Add shared Copilot skills and skill development tooling#14285
Draft
ronniegeraghty wants to merge 8 commits intomainfrom
Draft
Add shared Copilot skills and skill development tooling#14285ronniegeraghty wants to merge 8 commits intomainfrom
ronniegeraghty wants to merge 8 commits intomainfrom
Conversation
Migrated from Azure/azure-rest-api-specs#40811 per #14261. All shared skills prefixed with shared-. Excluded: typespec authoring skill, customizations skill. Skills added: - shared-sdk-release - shared-prepare-release-plan - shared-pipeline-troubleshooting - shared-apiview-feedback-resolution Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Collaborator
|
The following pipelines have been queued for testing: |
- Move all 4 skill directories from eng/common/ to .github/skills/ - Rename directory prefix from shared- to azsdk-common- - Update name: field in all SKILL.md frontmatter - Update all internal cross-skill references Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Add azsdk-common-markdown-token-optimizer skill - Add azsdk-common-sensei skill - Add azsdk-common-skill-authoring skill - Create eval.yaml for each skill - Add basic evaluation tasks - Create .github/workflows/skill-eval.yml for automated testing Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Fix name: fields in 3 meta-skills to use azsdk-common- prefix - Convert invalid keyword/regex grader types to text in all 7 eval.yaml files - Add evals/ directory with trigger_tests.yaml for all 7 skills - Replace workflow with spec-compliant version (compliance check + eval jobs) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The azsdk-common- prefix is reserved for skills shared across Azure SDK repos. Meta-skills (sensei, skill-authoring, markdown-token-optimizer) are internal tooling and keep their original names. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Added from azure-rest-api-specs commit 54d54fb with azsdk-common- prefix: - azsdk-common-check-package-validation - azsdk-common-generate-sdk-locally - azsdk-common-package-release-readiness - azsdk-common-report-test-bug - azsdk-common-typespec-to-sdk-workflow - azsdk-common-validate-codeowners - azsdk-common-validate-service-label - azsdk-common-verify-setup Also added README.md, INSTRUCTION-TO-SKILL-MAPPING.md, .waza.yaml, .gitignore All eval.yaml files use valid waza grader types (text, code). Name fields updated to azsdk-common- prefix. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Removed skills that praveenkuttappan flagged as not needing standalone skills: - validate-service-label (Daniel reworking labels) - verify-setup (single MCP call, fold into AGENTS.md) - validate-codeowners (wait for Daniel's codeowners project) - typespec-to-sdk-workflow (belongs in AGENTS.md) - check-package-validation (single MCP call, fold into generate-sdk-locally) - package-release-readiness (redundant with sdk-release) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
User decided this skill is not needed as a standalone skill. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
ronniegeraghty
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds shared Copilot skills to
.github/skills/per #14261.Migrated from Azure/azure-rest-api-specs#40811.
Naming Convention
azsdk-common-*prefix — shared skills distributed to other Azure SDK reposSkills (8 total)
Shared Domain Skills (5) —
azsdk-common-*prefixazsdk-common-sdk-release.github/skills/azsdk-common-sdk-release/azsdk-common-prepare-release-plan.github/skills/azsdk-common-prepare-release-plan/azsdk-common-pipeline-troubleshooting.github/skills/azsdk-common-pipeline-troubleshooting/azsdk-common-apiview-feedback-resolution.github/skills/azsdk-common-apiview-feedback-resolution/azsdk-common-generate-sdk-locally.github/skills/azsdk-common-generate-sdk-locally/Internal Meta-Skills (3) — no prefix
markdown-token-optimizer.github/skills/markdown-token-optimizer/sensei.github/skills/sensei/skill-authoring.github/skills/skill-authoring/What's Included
Each skill includes:
SKILL.mdwith waza-compliant frontmattereval.yamlevaluation config with valid waza grader types (text,code)evals/directory witheval.yaml,trigger_tests.yaml, andtasks/(basic-trigger + anti-trigger)tasks/with eval scenarios (basic, edge, negative)references/andfixtures/where applicableCI Workflow
Added
.github/workflows/skill-eval.ymlfor automated skill compliance and evaluation:waza checkon all skills, uploads compliance reportwaza runon allevals/eval.yaml, uploads results.github/skills/**)Skills Removed After Review
Per @praveenkuttappan's feedback, removed 7 skills that don't need standalone implementations:
validate-service-label— Daniel reworking labels with different approachverify-setup— Single MCP tool call, fold into AGENTS.md or generate-sdk-locallyvalidate-codeowners— Wait for Daniel's codeowners projecttypespec-to-sdk-workflow— Should be guidance in AGENTS.md, not standalone skillcheck-package-validation— Single MCP tool call, fold into generate-sdk-locallypackage-release-readiness— Redundant with sdk-release skillazsdk-common-report-test-bug— User decided not needed as standalone skillExcluded skills (not shared across repos):
Closes #14261