Add supported_versions.yml and CI generator#330
Draft
pawelchcki wants to merge 16 commits intomasterfrom
Draft
Conversation
Add a Docker-based code formatting pipeline that ensures consistent formatting across local development and CI environments. - Add Dockerfile.formatter with clang-format-14 and yapf (via uv), using MIRROR_REGISTRY pattern for CI registry mirrors - Add Makefile.formatter with format/lint targets that auto-detect Docker/CI environments and skip container wrapping when unnecessary - Add GitLab CI jobs: build-formatter-image, convert-formatter-image, lint - Switch bin/format.sh and bin/lint.sh to bash with pipefail for proper error propagation in find|xargs pipelines - Add broken-symlink detection with actionable error messages - Apply clang-format pointer alignment to C++ source files - Remove requirements.txt (yapf now provided by Docker image) - Add AGENTS.md with formatting instructions
Address reviewer feedback: rather than a separate include file, put all formatter definitions alongside the existing Docker image definitions in the main Makefile. - Merge all targets and variables from Makefile.formatter into Makefile - Delete Makefile.formatter - Rename DOCKER_RUN → FORMATTER_RUN (and related vars) to avoid confusion with the build toolchain's Docker usage - Keep ifdef GITLAB_CI for FORMATTER_IMAGE (needs CI_PIPELINE_ID) and TOOLCHAIN_DEPENDENCY/TEST_DEPENDENCY (paired with ifdef GITLAB_CI in build targets); use IN_DOCKER_OR_CI only for formatter execution
- Use hardcoded formatter image (from pipeline 102793200) instead of rebuilding on every push; build/convert jobs are now manual - Move git submodule and insteadOf config to global variables in common.yml, removing per-job duplication - Remove redundant git config invocation from .build-nginx-rum - Remove build-push-formatter-image target from Makefile (inlined in CI) - Simplify FORMATTER_IMAGE to local-only (CI uses hardcoded image tag)
- Use content hash of Dockerfile.formatter as image tag so the image is only rebuilt when the Dockerfile actually changes - Merge build + nydus-convert into a single job (nydus image includes docker with all credential helpers) - Move git submodule/insteadOf config to global variables in common.yml - Remove redundant git config from .build-nginx-rum - Simplify FORMATTER_IMAGE to local-only (CI uses hash-tagged image) - Remove build-push-formatter-image Makefile target (inlined in CI)
Use $(CURDIR) as both source and destination in the Docker volume mount so file paths are identical inside and outside the container.
Single source of truth for nginx, ingress-nginx, and openresty versions.
bin/generate_gitlab_ci.py reads supported_versions.yml and produces
.gitlab/build-and-test-{all,fast}.yml with stable semver-sorted output.
A check-generated-ci job in the fast pipeline validates that committed
CI files match the YAML source.
pawelchcki
force-pushed
the
supported_versions.yml
branch
from
659e1c6 to
5105235
Compare
…e helpers - One _versions() function replaces 7 separate extractors - Static CI jobs as readable string constants (ALL_HEADER, FAST_HEADER) - Composable helpers (build_matrix_job, test_matrix_job, etc.) return complete YAML fragments - generate_all/generate_fast read like a manifest of jobs - 583 → 446 lines
- Generator now builds YAML as native Python dicts/lists, serialized by ruamel.yaml with proper formatting control (flow/block styles, double-quoted strings, indented comments) - Split static jobs (formatter, lint, shellcheck, check-generated-ci, coverage rules) into hand-maintained *-static.yml files - Generated files now contain only parameterized matrix jobs
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #330 +/- ##
=======================================
Coverage 68.85% 68.85%
=======================================
Files 56 56
Lines 7471 7471
Branches 1058 1058
=======================================
Hits 5144 5144
Misses 1820 1820
Partials 507 507 🚀 New features to boost your workflow:
|
- Replace dedent/f-string rendering with Python dicts + yaml.dump() - Add FlowList, QuotedStr custom types with pyyaml representers - Add GitLabDumper with proper sequence indentation - Extract coverage job into static .gitlab/coverage.yml - Remove SYSTEM_TESTS_COMMENT, _insert_comment, and dead helpers
- Rename build-and-test-all-static.yml → build-and-test-common.yml - Rename build-and-test-fast-static.yml → checks.yml - Move .build-and-test-fast template to build-and-test-common.yml - Merge coverage job into checks.yml, delete coverage.yml - Sort includes in .gitlab-ci.yml
Replace hardcoded JOBS_PER_NGINX_VERSION/JOBS_PER_EXTRA_IMAGE constants with row_job_count() that computes jobs from FlowList lengths. Replace split_nginx_test_versions with generic split_rows that greedy-packs rows into arbitrarily many groups under the GitLab matrix limit.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
supported_versions.ymlas a single source of truth for all nginx, ingress-nginx, and openresty version listsbin/generate_gitlab_ci.pythat reads the YAML and produces.gitlab/build-and-test-{all,fast}.ymlwith stable semver-sorted outputcheck-generated-ciCI job that validates committed CI files match the YAML sourceAdding a new version is now a one-line change in
supported_versions.ymlfollowed byuv run bin/generate_gitlab_ci.py.Test plan
uv run bin/generate_gitlab_ci.py --checkpasses--checkexits 1 with diff