tooling: changelog automatic reporting

[systay]

Description

This PR introduces a comprehensive toolkit for automating the analysis of pull requests during release cycles. The tooling enables systematic review of hundreds of PRs to identify public-facing changes like flag modifications, metric additions, API changes, and breaking changes.

What's included

• Methodology guide - Step-by-step process for analyzing large PR sets
• Specialized agent - Pre-configured for detecting flag/metric changes
• Example outputs - Sample reports and final documentation
• Automation scripts - Setup and progress monitoring utilities
• Templates - Standardized formats for consistent documentation

Problem this solves

Previously, creating comprehensive release documentation required manually reviewing hundreds of PRs, which was:

• Time-intensive and boring
• Error-prone (easy to miss important changes)
• Inconsistent (different reviewers, different formats)

The tooling was successfully used to analyze all 276 PRs from the v23 milestone, identifying critical changes like the major flag standardization effort and new feature additions.

[vitess-bot]

Review Checklist

Hello reviewers! 👋 Please follow this checklist when reviewing this Pull Request.

General

• Ensure that the Pull Request has a descriptive title.
• Ensure there is a link to an issue (except for internal cleanup and flaky test fixes), new features should have an RFC that documents use cases and test cases.

Tests

• Bug fixes should have at least one unit or end-to-end test, enhancement and new features should have a sufficient number of tests.

Documentation

• Apply the release notes (needs details) label if users need to know about this change.
• New features should be documented.
• There should be some code comments as to why things are implemented the way they are.
• There should be a comment at the top of each new or modified test to explain what the test does.

New flags

• Is this flag really necessary?
• Flag names must be clear and intuitive, use dashes (-), and have a clear help text.

If a workflow is added or modified:

• Each item in Jobs should be named in order to mark it as required.
• If the workflow needs to be marked as required, the maintainer team must be notified.

Backward compatibility

• Protobuf changes should be wire-compatible.
• Changes to _vt tables and RPCs need to be backward compatible.
• RPC changes should be compatible with vitess-operator
• If a flag is removed, then it should also be removed from vitess-operator and arewefastyet, if used there.
• vtctl command output order should be stable and awk-able.

[codecov]

Codecov Report

❌ Patch coverage is 0% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 67.51%. Comparing base (1baf02b) to head (763dcdf).
⚠️ Report is 2 commits behind head on main.

Files with missing lines	Patch %	Lines
go/tools/releases/releases.go	0.00%	3 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #18600      +/-   ##
==========================================
- Coverage   67.52%   67.51%   -0.02%     
==========================================
  Files        1607     1607              
  Lines      263338   263341       +3     
==========================================
- Hits       177812   177784      -28     
- Misses      85526    85557      +31     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[systay]

Sharing the report here. This is not the final one, so I didn't want to commit it to the PR.

Vitess v23.0.0 API Changes Report

Summary

This report documents all public-facing API changes, flag modifications, metric additions/removals, and parser enhancements that were merged into Vitess v23.0.0. Based on analysis of 276 pull requests from the v23 milestone.

Table of Contents

• Major Changes
  • Flag Standardization
  • New Flags
  • New Metrics
  • Deleted/Modified Metrics
  • New APIs
  • Parser Changes
  • Query Planning Changes
• New Features
  • MySQL 9.x Support
  • Enhanced Query Logging
  • VStream Improvements
• Breaking Changes
• Minor Changes

---

Major Changes

Flag Standardization

The most significant change in v23 is the systematic migration of CLI flags from underscore (_) to dash (-) notation. This affects over 1,000+ flags across all Vitess components.

Key Flag Migration PRs

PR	Component Focus	Flags Migrated	Description	Breaking Change
#18009	gRPC	234	gRPC authentication, TLS, keepalive flags	⚠️ Yes
#18064	All Components	394	Database connections, auth, service management	⚠️ Yes
#18095	All Components	273	Backup, MySQL config, tablet management flags	⚠️ Yes
#18280	All Components	1,170+	MEGA MIGRATION - Most comprehensive flag refactor	⚠️ Yes
#18291	vtcombo	51	vtcombo-specific flags	⚠️ Yes
#18292	vtctld/vtbackup	55	Schema management, backup flags	⚠️ Yes
#18294	vtctld	16	vtctld-specific flags	⚠️ Yes
#18296	Security/TLS	55	Critical security and TLS flags	⚠️ Yes
#18297	vtcombo	56	Buffer management, health checks	⚠️ Yes
#18298	vtcombo	69	Schema migration, query processing	⚠️ Yes

Migration Timeline: Underscore flags will be completely removed in Vitess v25. Both formats are supported in v23 for compatibility.

New Flags

Component	Flag Name	Type	Description	PR
vtgate, vttablet, vtcombo	--querylog-time-threshold	duration	Execution time threshold for query logging	#18520
vtgate, vttablet, vtcombo	--querylog-emit-on-any-condition-met	bool	Enable OR logic for query logging conditions	#18546
vtgate	--default-multi-shard-autocommit	bool	Enable multi-shard autocommit by default	#17635
vttablet	--init_tablet_type	string	Now accepts DRAINED type for maintenance	#18179
vttablet	--xtrabackup-should-drain	bool	Control tablet draining during XtraBackup	#18431
vtorc	--allow-recovery	bool	Allow VTOrc recoveries to be disabled from startup	#18005
vtorc	--grpc-enable-orca-metrics	bool	Enable ORCA metrics for gRPC load balancing	#18282
vtctldclient	--add-reference-tables	string	Add reference tables to materialize workflow	#17804
vtctldclient	--params-file	string	JSON file for bulk lookup vindex creation	#17566
vtctldclient	--verbosity-level	enum	Control output verbosity (MINIMAL/LOW/MEDIUM/HIGH)	#17799
vttablet	--skip-user-metrics	bool	Prevent metric cardinality explosion from user labels	#18085
vtteseter	--olap	bool	Enable OLAP workload mode in tests	#18042

New Metrics

VTGate

Name	Dimensions	Description	PR
TransactionsProcessed	TransactionType, ShardDistribution	Track transactions by type and shard distribution	#18171

VTTablet

Name	Dimensions	Description	PR
TabletServerState	TabletType, Keyspace, Shard	Enhanced with tablet dimensions	#18451
OnlineDDLStaleMigrationMinutes	keyspace, migration_uuid, shard	Time since last migration status update	#18417
OptimizedQueryExecutions	plan_type	Track deferred optimization query executions	#18067

VTOrc

Name	Dimensions	Description	PR
FailedRecoveries	Keyspace, Shard, RecoveryType	Failed recovery operations with shard context	#18304
PendingRecoveries	Keyspace, Shard, RecoveryType	Pending recovery operations with shard context	#18304
RecoveriesCount	Keyspace, Shard, RecoveryType	Total recovery operations with shard context	#18304
SuccessfulRecoveries	Keyspace, Shard, RecoveryType	Successful recovery operations with shard context	#18304

Deleted/Modified Metrics

Component	Metric Name	Change Type	Description	PR
vtgate	QueriesProcessed	DELETED	Deprecated metric removed	#18149
vtgate	QueriesRouted	DELETED	Deprecated metric removed	#18149
vtgate	QueriesProcessedByTable	DELETED	Deprecated metric removed	#18149
vtgate	QueriesRoutedByTable	DELETED	Deprecated metric removed	#18149
vtgate	QueryExecutionsByTable	MODIFIED	Only records successful queries now	#18584

New APIs

Component	API Name	Type	Description	PR
TabletManager	UpdateSequenceTables	gRPC	Batch update sequence tables with max values	#18172
TabletManager	GetMaxValueForSequences	gRPC	Get max values for multiple sequence tables	#18172
MySQL Protocol	ConnectWithAttributes()	API	Client connection with metadata attributes	#18548
VStream	tables_to_copy	Option	Filter specific tables during copy phase	#18184
VStream	exclude_keyspace_from_table_name	Option	Optimize table name handling in streams	#18274

Parser Changes (go/vt/sqlparser)

Feature	Description	PR
VALUE keyword	Support VALUE as alias for VALUES in INSERT/REPLACE	#18116
RETURNING clauses	Full PostgreSQL-style RETURNING syntax support	#18318
SET NAMES binary	Fixed parsing of SET NAMES binary statements	#18582
CREATE PROCEDURE	Enhanced DEFINER clause support for procedures	#18142
CREATE TABLE ... SELECT	Complete support for CREATE TABLE from SELECT	#18443
Transaction types	New BeginStmt, StartTransactionStmt constants	#18279
VITESS_SHARDS	New keyword for shard-specific OnlineDDL operations	#18331
IsReadStatement()	New method for transaction mode handling	#18173

Query Planning Changes

Change	Description	Impact	PR
Prepare statement handling	Allow prepare to succeed even when plan can't be constructed	Medium	#18126
INSERT IGNORE with NULLs	Fixed bug where rows weren't inserted when vindex columns were NULL	High	#18151
SINGLE transaction mode	Allow read-only multi-shard transactions in SINGLE mode	Medium	#18173
UNION query optimization	Improved UNION query merging and alias handling	Medium	#18289, #18484
Window function support	Allow OVER clauses in single-shard scenarios	Medium	#18103

---

New Features

MySQL 9.x Support

Vitess v23 introduces support for MySQL 9.x with new flavor detection and configuration handling.

Added in: #18399

Enhanced Query Logging

Two new flags provide MySQL-style slow query log functionality:

• --querylog-time-threshold: Filter queries by execution time
• --querylog-emit-on-any-condition-met: Use OR logic instead of AND for multiple conditions

Added in: #18520, #18546

VStream Improvements

Enhanced VStream functionality for better performance and flexibility:

• Selective table copying: tables_to_copy option reduces copy time for large databases
• Optimized table naming: exclude_keyspace_from_table_name reduces memory usage
• Performance: Up to 75% execution time reduction and 82% CPU usage improvement

Added in: #18184, #18274

---

Breaking Changes

Flag Naming Convention

• Impact: All deployment scripts, Kubernetes manifests, and configuration files
• Action Required: Update all flag references from flag_name to flag-name
• Timeline: Both formats supported in v23, underscore flags removed in v25

API Changes

• TabletManager ExecuteFetchAsDba: Now strictly rejects multi-statement queries | #18183
• Foreign Key Metadata: JSON format now includes keyspace qualification | #18537
• VReplication Behavior: UPDATE events now include both before/after images when filtered | #18319

Configuration Changes

• MySQL Environment Variables: Fixed propagation may affect timezone behavior | #18561
• Consul Auth: --consul_auth_static_file now validates non-empty credentials | #18152

---

Minor Changes

Version Updates

• Go: Upgraded to 1.25.0 | #18573
• MySQL: Default testing moved from 8.0 to 8.4 | #18569

Compatibility Improvements

• MySQL Protocol: Added connection attributes support | #18548
• PostgreSQL Compatibility: RETURNING clause support | #18318

Performance Optimizations

• VTOrc: Significantly improved emergency reparent performance (60s → <15s) | #18234
• Query Planning: Various optimizations for UNION, GROUP BY, and derived tables

Bug Fixes

• Critical Stability: Fixed multiple deadlocks, panics, and race conditions
• Data Correctness: Fixed INSERT IGNORE, UNION query handling, and VReplication issues
• Foreign Keys: Improved routing and error handling

---

Summary Statistics

• Total PRs Analyzed: 276
• Merged PRs: ~180 (estimated based on analysis)
• Flag Migrations: 1,000+ flags standardized
• New Features: MySQL 9.x support, enhanced logging, VStream improvements
• Critical Bug Fixes: 20+ stability and correctness improvements
• API Additions: 5+ new gRPC endpoints and protocol enhancements

---

Generated from analysis of all v23 milestone pull requests

[harshit-gangal]

This is really cool.