Skip to content

[BUG] BWC tests are running against too many versions #20866

@andrross

Description

@andrross

Describe the bug

The BWC tests in CI are building and testing a dead branch (3.4) in addition to the correct branches (3.5 and 2.19). The root cause is a broken heuristic in BwcVersions.getUnreleased().

The BWC infrastructure parses version constants from Version.java and uses a heuristic to determine which are "unreleased." It groups versions by minor and checks how many entries each minor has. A minor with multiple entries (e.g., 3.5.0 and 3.5.1) is considered released, with the last entry being the unreleased next patch. A minor with only a single .0 entry is considered unreleased itself.

This heuristic depended on a convention: every time a minor was released, a placeholder next-patch version was added to Version.java (e.g., V_2_3_1 after releasing 2.3.0), even if no such release was planned. Commit 69c88ea ("Remove invalid unreleased versions (#5312)") removed these placeholders, breaking the heuristic. Without them, every minor that only had a .0 release looks "unreleased," causing the algorithm to walk backwards and incorrectly mark extra minors as unreleased.

With CURRENT = 3.6.0, the algorithm computes the unreleased versions as [2.19.5, 3.4.0, 3.5.0], but the correct set is [2.19.5, 3.5.0]:

  • 2.19 — actively maintained, receiving backports ✓
  • 3.4 — released Dec 2025, essentially dead (1 commit since release) ✗
  • 3.5 — actively maintained, 20+ commits since its Feb 2026 release ✓

The same heuristic is duplicated in VersionUtils.java, so the test framework's understanding of released vs. unreleased versions is also wrong.

Related component

Build

To Reproduce

Check the build log of a successful build: https://build.ci.opensearch.org/job/gradle-check/72177/console

% grep -oE '\[[0-9]+\.[0-9]+\.[0-9]+\]' consoleText | tr -d '[]' | sort -uV
2.19.5
3.4.0
3.5.0
% ./gradlew bwcTestSnapshots -m 2>&1 | grep -oE 'v[0-9]+\.[0-9]+\.[0-9]+#' | tr -d '#v' | sort -uV
2.19.5
3.4.0
3.5.0

Expected behavior

Replace the heuristic with a deterministic rule derived from the current version:

  1. Current version M.N.0 is unreleased (on main)
  2. Previous minor M.(N-1) is unreleased (on its branch)
  3. Previous major tip is unreleased (on its maintenance branch)

This requires no placeholder versions, no configuration files, and automatically adjusts when main is bumped during the release process. It matches the project's maintenance policy: test the previous minor of the current major and the tip of the previous major.

Metadata

Metadata

Assignees

No one assigned

    Labels

    BuildBuild Tasks/Gradle Plugin, groovy scripts, build tools, Javadoc enforcement.bugSomething isn't workinguntriaged

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions