Breaking change doc tool

[ericstj]

This adds a workflow to run a script when a PR is labeled as needs-breaking-change-doc-created,

The script collects details about the PR, the breaking change issue template, and sample issues then asks an LLM to generate an issue based on that.

The issue content is then encoded into a clickable link which is then added as a comment to the PR.

I'm doing this from a dotnet branch so that I can hopefully create a PR off of it to demonstrate the workflow in action.

[Copilot]

Pull Request Overview

This PR introduces an automated workflow for creating breaking change documentation. When a PR is labeled needs-breaking-change-doc-created, a PowerShell script analyzes the PR details and uses an LLM to generate documentation content. The generated content is then encoded into a clickable GitHub issue creation link, which is posted as a comment on the PR.

Key Changes

• PowerShell script that collects PR data, analyzes it with LLM, and generates breaking change documentation
• GitHub Actions workflow that triggers on label addition to run the automation
• Configuration file for customizing LLM provider, repository settings, and other parameters

Reviewed Changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 5 comments.

File	Description
eng/breakingChanges/config.ps1	Configuration file defining LLM settings, GitHub repository paths, issue template settings, and rate limiting parameters
eng/breakingChanges/breaking-change-doc.ps1	Main PowerShell script implementing the breaking change documentation workflow with data collection, LLM analysis, and issue generation
eng/breakingChanges/README.md	Documentation explaining the automation tool's features, setup instructions, usage examples, and configuration options
.github/workflows/breaking-change-doc.yml	GitHub Actions workflow that triggers the script when a PR receives the needs-breaking-change-doc-created label

[ericstj]

This is testing the workflow - #120988 Looks like it's failing to fetch the issues from dotnet/docs. I'll see if I can get it issue read permission to the docs repo.

[ericstj]

Now seeing Error: unexpected response from the server: 403 Forbidden

Need to find out how to use models from a workflow.

[rzikm]

Looks great, I look forward to using this once the technical hurdles get ironed out.

[ericstj]

This works file even without the workflow, so it can be used by directly running the script after installing the github CLI. Worse case we'll just use that and occasionally run it.

The summary of the problems with the workflow is that our workflows in the dotnet org don't have access to github models, and that's currently not a technical problem. I can unblock that by storing a secret in the repo as a token for another LLM provider, but I don't like storing/needing-to-update a secret.

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-meta
See info in area-owners.md if you want to be subscribed.

[ericstj]

I don't expect this to work, but also trying copilot CLI provider.

[ericstj]

it looks like the changes to URL escaping broke the formatting of newlines. I'll need to see when that happened and undo it.

[ericstj]

I actually broke it when refactoring how we call the gh CLI. I also noticed some concern with passing prompts on the commandline. We should avoid this - instead passing a prompt as a file.