feat(ci): PGS project board sync workflow + bootstrap script#919
feat(ci): PGS project board sync workflow + bootstrap script#919danielmeppiel wants to merge 2 commits intomainfrom
Conversation
danielmeppiel
added
theme/governance
Wires up https://github.com/orgs/microsoft/projects/2304 (APM Roadmap) to stay in lockstep with the theme/*/area/*/type/*/priority/* label taxonomy. Single Python script does the GraphQL; one workflow fires on issues+PRs label/milestone events; a one-shot backfill script seeds existing labelled work. Tracking: #916 (PGS roadmap-management system epic). Setup gate (one time, manual): an org admin must save a fine-grained PAT with Projects: Read & Write + Issues: Read on microsoft/apm as the PROJECT_SYNC_PAT secret. The 7 issues currently labelled with theme/* have already been backfilled into the project as a smoke test. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
danielmeppiel
force-pushed
the
feat/pgs-project-sync
branch
from
d9e4534 to
6cf502f
Compare
There was a problem hiding this comment.
Pull request overview
Adds automation to keep the APM PGS GitHub Project board in sync with the repo’s new label taxonomy (Theme/Area/Kind/Priority) and milestone-derived Tier.
Changes:
- New workflow to trigger project sync on issue/PR label + milestone lifecycle events.
- New Python script to read issue/PR metadata and write ProjectV2 single-select fields via GraphQL.
- New backfill script to populate/sync existing theme-labeled issues/PRs.
Show a summary per file
| File | Description |
|---|---|
.github/workflows/project-sync.yml |
Adds event-driven workflow to run the sync script with a PAT secret. |
scripts/project/sync_item.py |
Implements GraphQL read/write logic and derives Theme/Area/Kind/Priority/Tier from labels + milestone. |
scripts/project/backfill.sh |
One-shot helper to find theme-labeled open items and invoke the sync script. |
CHANGELOG.md |
Adds an Unreleased changelog entry for the new automation. |
Copilot's findings
Comments suppressed due to low confidence (1)
scripts/project/backfill.sh:22
REPOis defined but the GraphQL search query hard-codesrepo:microsoft/apm. Using$REPOin the query would avoid drift if the script is reused or the repo name changes.
REPO="microsoft/apm"
LIMIT="${LIMIT:-100}"
PROJECT_ID="PVT_kwDOAF3p4s4BVoGw"
HERE="$(cd "$(dirname "$0")" && pwd)"
export PAGER=cat GH_PAGER=cat
echo "Fetching open issues + PRs with any theme/* label..."
ISSUES=""
for THEME in theme/portability theme/security theme/governance; do
CHUNK=$(gh api graphql -f query='
{
search(query: "repo:microsoft/apm is:open label:\"'$THEME'\"", type: ISSUE, first: '$LIMIT') {
nodes { ... on Issue { id number title } ... on PullRequest { id number title } }
- Files reviewed: 4/4 changed files
- Comments generated: 7
| print(f"Syncing {content.get('repository', {}).get('nameWithOwner')}#{content['number']}: {content['title']}") | ||
| print(f" labels: {labels}") |
There was a problem hiding this comment.
This prints content['title'] (and the labels list) directly. Issue/PR titles and labels can contain non-ASCII characters, which can raise UnicodeEncodeError on Windows/cp1252 terminals and violates the repo's ASCII-only output rule. Consider sanitizing these values for output (e.g., encode('ascii', 'backslashreplace').decode() or similar) or avoid printing user-controlled text entirely.
| # Requires: GITHUB_TOKEN env var with project + repo scopes. | ||
| # Usage: scripts/project/backfill.sh [--limit N] | ||
| set -euo pipefail |
There was a problem hiding this comment.
The header says Usage: scripts/project/backfill.sh [--limit N], but the script does not parse --limit (it only reads LIMIT from the environment). Either implement --limit argument parsing or update the usage comment to match the actual interface.
This issue also appears on line 9 of the same file.
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
| with: | ||
| sparse-checkout: scripts/project |
There was a problem hiding this comment.
This workflow uses ubuntu-latest. Elsewhere in the repo CI workflows are pinned to ubuntu-24.04 for determinism; consider pinning here as well to avoid unexpected breakages when GitHub changes the default runner image.
| ### Added | ||
| - PGS project board sync: `scripts/project/sync_item.py` and `.github/workflows/project-sync.yml` keep https://github.com/orgs/microsoft/projects/2304 in lockstep with `theme/*`, `area/*`, `type/*`, `priority/*` labels and milestones. Backfill helper at `scripts/project/backfill.sh`. (#919) | ||
|
|
There was a problem hiding this comment.
The changelog entry still ends with (#TBD). Per the repo changelog rules, entries should end with the actual PR number before merge (e.g., (#919) for this PR).
| jobs: | ||
| sync: | ||
| if: | | ||
| github.event_name == 'workflow_dispatch' || |
There was a problem hiding this comment.
The job-level if only runs when the current label set contains theme/. For unlabeled events that remove the last theme/* label, this condition will be false, so the workflow won't run and the project item won't be cleared/updated. Consider also keying off the event's label.name for labeled/unlabeled actions (e.g., run when github.event.label.name starts with theme/).
| github.event_name == 'workflow_dispatch' || | |
| github.event_name == 'workflow_dispatch' || | |
| startsWith(github.event.label.name, 'theme/') || |
| item_id = add_to_project(args.project_id, args.content_id) | ||
| print(f" item: {item_id}") |
There was a problem hiding this comment.
addProjectV2ItemById is not idempotent: if the issue/PR is already on the project, GitHub returns a GraphQL error and the script exits (via gql()), which will break sync on subsequent label/milestone events. Consider first querying for an existing ProjectV2 item for this content ID (or catching the specific error and then looking up the existing item id) and only calling addProjectV2ItemById when needed.
| def derive_field_value(labels, mapping): | ||
| for lab in labels: | ||
| if lab in mapping: | ||
| return mapping[lab] | ||
| return None | ||
|
|
||
|
|
||
| def derive_area(labels): | ||
| for lab in labels: | ||
| if lab in AREA_NAMES: | ||
| return lab.split("/", 1)[1] | ||
| return None | ||
|
|
||
|
|
There was a problem hiding this comment.
Theme/Kind/Priority/Area selection depends on the order of labels returned by GraphQL (which is not a stable precedence). If multiple matching labels exist (multi-theme and multi-area are explicitly allowed in #916), this can lead to non-deterministic field values and flapping updates. Recommend implementing an explicit precedence order (e.g., Portability < Security < Governance for Theme) and a deterministic rule for Area (e.g., choose a primary area label or sort and pick consistently).
| def derive_field_value(labels, mapping): | |
| for lab in labels: | |
| if lab in mapping: | |
| return mapping[lab] | |
| return None | |
| def derive_area(labels): | |
| for lab in labels: | |
| if lab in AREA_NAMES: | |
| return lab.split("/", 1)[1] | |
| return None | |
| def derive_field_value( | |
| labels: list[str], mapping: dict[str, str] | |
| ) -> str | None: | |
| """Return the first mapped value using explicit mapping precedence. | |
| GraphQL label ordering is not a stable source of precedence. This | |
| function instead treats the insertion order of ``mapping`` as the | |
| authoritative precedence and returns the first matching mapped value. | |
| """ | |
| label_set = set(labels) | |
| for label_name, mapped_value in mapping.items(): | |
| if label_name in label_set: | |
| return mapped_value | |
| return None | |
| def derive_area(labels: list[str]) -> str | None: | |
| """Return a deterministic Area value from the available labels. | |
| Multiple area labels may be present. To avoid flapping updates caused | |
| by GraphQL label ordering, choose the lexicographically first matching | |
| area label and map it to the single-select option name. | |
| """ | |
| matching_areas = sorted(label for label in labels if label in AREA_NAMES) | |
| if not matching_areas: | |
| return None | |
| return matching_areas[0].split("/", 1)[1] |
|
Layout direction (mirroring GitHub Public Roadmap): default view becomes Board, group-by Tier, slice-by Theme. UI steps updated in #920. The sync script needs no change -- Theme + Tier are already the two fields it writes. |
APM Review Panel VerdictDisposition: REQUEST_CHANGES (two required pre-merge fixes: shell injection hardening + CHANGELOG URL typo) Per-persona findingsPython Architect: This PR is purely procedural infrastructure automation outside 1. OO / class diagramclassDiagram
direction LR
class sync_item_py {
<<Module>>
<<IOBoundary>>
+main() int
+gql(query, variables) dict
+fetch_project_meta(project_id) dict
+fetch_content(content_id) dict
+add_to_project(project_id, content_id) str
+update_single_select(pid, iid, fid, oid) void
+clear_field(pid, iid, fid) void
+derive_tier(content) str
+derive_field_value(labels, mapping) str
+derive_area(labels) str
}
class backfill_sh {
<<Script>>
<<IOBoundary>>
}
class project_sync_yml {
<<GitHubAction>>
<<IOBoundary>>
}
class GitHubGraphQLAPI {
<<ExternalSystem>>
}
class GitHubProjectV2 {
<<ExternalSystem>>
}
class sync_item_py:::touched
class backfill_sh:::touched
class project_sync_yml:::touched
classDef touched fill:#fff3b0,stroke:#d47600
sync_item_py ..> GitHubGraphQLAPI : NET POST
sync_item_py ..> GitHubProjectV2 : NET mutates
backfill_sh ..> sync_item_py : EXEC spawns
project_sync_yml ..> sync_item_py : EXEC spawns
note for sync_item_py "Pure procedural; outside APM CLI src/"
2. Execution flow diagramflowchart TD
A[GH Event: issue or PR labeled/milestoned] --> B{job if: contains theme/ label?}
B -- No --> C[Skip]
B -- Yes --> D["[FS] actions/checkout@v4 sparse-checkout: scripts/project"]
D --> E{event_name?}
E -- workflow_dispatch --> F["[FS] echo id from inputs.content_id to GITHUB_OUTPUT"]
E -- issues --> G["[FS] echo id from github.event.issue.node_id"]
E -- pull_request_target --> H["[FS] echo id from github.event.pull_request.node_id"]
F --> I
G --> I
H --> I
I["[EXEC] python3 scripts/project/sync_item.py --content-id node_id"] --> J["[NET] gql: fetch_project_meta -- fields/options"]
J --> K["[NET] gql: fetch_content -- labels, milestone, state"]
K --> L["[NET] gql: addProjectV2ItemById mutation"]
L --> M[derive_tier / derive_field_value / derive_area]
M --> N{for each field: Theme Area Kind Priority Tier}
N -- value present --> O["[NET] gql: updateProjectV2ItemFieldValue"]
N -- value None --> P["[NET] gql: clearProjectV2ItemFieldValue"]
O --> N
P --> N
N -- all done --> Q[Exit 0]
3. Design patternsDesign patterns
Architecture concern (non-blocking): CLI Logging Expert: This PR does not touch DevX UX Expert: No APM CLI command surface changes. Contributor UX observations (non-blocking, all minor):
Supply Chain Security Expert: The primary threat surface for this PR is the GitHub Actions workflow. Finding 1 -- REQUIRED FIX (shell injection, # CURRENT -- expression directly interpolated into shell
echo "id=${{ inputs.content_id }}" >> "$GITHUB_OUTPUT"
# FIX -- pass through env var, never interpolate into shell
- name: Resolve content node ID
id: cid
env:
INPUT_CONTENT_ID: ${{ inputs.content_id }}
run: |
if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
echo "id=$INPUT_CONTENT_ID" >> "$GITHUB_OUTPUT"
elif [ "${{ github.event_name }}" = "issues" ]; then
echo "id=${{ github.event.issue.node_id }}" >> "$GITHUB_OUTPUT"
else
echo "id=${{ github.event.pull_request.node_id }}" >> "$GITHUB_OUTPUT"
fiFinding 2 -- Finding 3 -- Token scope (OK): Auth Expert: Not activated -- PR only touches OSS Growth Hacker: No conversion surface impact -- README, quickstart, templates, and CLI surface are unchanged. Positive signal: automated PGS roadmap sync is professional OSS hygiene that builds trust with potential contributors who browse the project board. At 250+ stars, the roadmap is actively viewed. The Side-channel to CEO: "Not a release story by itself, but a credibility signal. The CHANGELOG entry format is correct; just fix the URL typo." One CHANGELOG observation: the entry contains CEO arbitrationSpecialists are in full agreement on this PR. It is a narrow, well-scoped CI automation addition that belongs at this stage of the project: a growing OSS repo benefits from automated roadmap hygiene. The only blocking findings are Supply Chain Security's shell injection hardening (trivial one-liner fix, best practice regardless of current blast radius) and the CHANGELOG URL typo (cosmetic but factually incorrect). The Python Architect's Required actions before merge
Optional follow-ups
|
Points to https://github.com/orgs/microsoft/projects/2304 alongside Documentation / Quick Start / CLI Reference. Part of #919. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
What
Auto-sync workflow that mirrors the APM Roadmap project board to the live label taxonomy.
Part of epic #916.
Files
.github/workflows/project-sync.yml-- fires on issue/PRopened|labeled|unlabeled|milestoned|demilestoned|closed|reopenedevents; idempotentscripts/project/sync_item.py-- pure-stdlib Python; reads labels+milestone, writes Theme/Area/Kind/Priority/Tier project fieldsscripts/project/backfill.sh-- one-shot backfill helper; already executed locally against the 7 currently-labelled issuesTier derivation
Setup gate (org admin)
This PR is mergeable on its own; the workflow becomes operational once an org admin:
Projects: Read & Write+Issues: Readonmicrosoft/apmPROJECT_SYNC_PATUntil then, the workflow runs but the sync step fails (graceful no-op for unrelated PRs since the
if:guards ontheme/*labels).Verification
Backfill was already executed against the 7 open theme-labelled issues. Project board now shows the cohort under correct Theme/Area/Kind/Priority/Tier values. See project board.
Followup work tracked in #916