Skip to content

Replace CHANGELOG.md with release-notes/AGENTS.md#20974

Draft
andrross wants to merge 1 commit intoopensearch-project:mainfrom
andrross:kill-the-changelog
Draft

Replace CHANGELOG.md with release-notes/AGENTS.md#20974
andrross wants to merge 1 commit intoopensearch-project:mainfrom
andrross:kill-the-changelog

Conversation

@andrross
Copy link
Copy Markdown
Member

@andrross andrross commented Mar 23, 2026

Use an AI agent steering file to generate release notes from commit messages, PR descriptions, and PR labels.

Here is an example PR using kiro-cli with Claude Opus 4.6

This has quite a lot of overlap with the existing AI-generated release notes tooling. There are key differences in the prompt. Namely that for this repo I think it's pretty important to filter out commits that are not important to users, as opposed to the strict one-entry-per-commit policy. There are also a policies like filtering out commit/revert pairs, collapsing multiple version updates to the same dependency, etc. This version also puts explanation about judgement calls into the PR description, giving human reviews some specific things to look at.

The other main difference is that this relies on an agent to do the work of getting commits and scraping the GitHub API, as opposed to scripting it. There are some subtleties with getting the correct commits for a release given that the previous release branch overlaps with main for the duration of the release window. The agent seemed to be pretty good at figuring this out.

I would like to see if we can integrate some of these changes into the automation in the build repo and then maybe not need this. At least in the short term this seems like a pretty good improvement over the CHANGELOG!

Check List

  • Functionality includes testing.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Use an AI agent steering file to generate release notes from commit
messages, PR descriptions, and PR labels.

Signed-off-by: Andrew Ross <andrross@amazon.com>
@andrross andrross requested a review from a team as a code owner March 23, 2026 20:52
@github-actions
Copy link
Copy Markdown
Contributor

PR Code Analyzer ❗

AI-powered 'Code-Diff-Analyzer' found issues on commit 9710c7c.

PathLineSeverityDescription
release-notes/AGENTS.md55mediumThe agent prompt instructs an AI to fetch and process PR body content from arbitrary pull requests via `gh pr view ... --json title,body,labels`. PR bodies are user-controlled content. If a PR description contains adversarial instructions targeting AI agents, this creates a prompt injection vector — the agent could be manipulated into categorizing changes incorrectly, including malicious content in release notes, pushing to unintended remotes, or creating PRs with attacker-controlled content. This is a structural risk in the workflow design, not necessarily malicious intent by the author, but it is worth flagging.
release-notes/AGENTS.md120lowThe agent is instructed to identify the user's fork remote by scanning `git remote -v` for a URL that does not match `opensearch-project/OpenSearch`, then push a branch and open a PR to that remote. If the local git remote configuration has been tampered with (e.g., a malicious remote added by a prior supply-chain compromise or misconfiguration), the agent could be directed to push code to an attacker-controlled repository without explicit user confirmation of the target remote.

The table above displays the top 10 most important findings.

Total: 2 | Critical: 0 | High: 0 | Medium: 1 | Low: 1


Pull Requests Author(s): Please update your Pull Request according to the report above.

Repository Maintainer(s): You can bypass diff analyzer by adding label skip-diff-analyzer after reviewing the changes carefully, then re-run failed actions. To re-enable the analyzer, remove the label, then re-run all actions.


⚠️ Note: The Code-Diff-Analyzer helps protect against potentially harmful code patterns. Please ensure you have thoroughly reviewed the changes beforehand.

Thanks.

@andrross
Copy link
Copy Markdown
Member Author

@peterzhuamazon @gaiksaya @rishabh6788 What do you think about integrating some of the things in the prompt here into https://github.com/opensearch-project/opensearch-build/blob/main/src/llms/prompts.py ?

@andrross andrross marked this pull request as draft March 23, 2026 22:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant