Skip to content

Add [tool.maturin.generate-ci.github] config support#3066

Merged
messense merged 3 commits intoPyO3:mainfrom
messense:generate-ci-config
Mar 7, 2026
Merged

Add [tool.maturin.generate-ci.github] config support#3066
messense merged 3 commits intoPyO3:mainfrom
messense:generate-ci-config

Conversation

@messense
Copy link
Member

@messense messense commented Mar 4, 2026

Split src/ci.rs into src/ci/mod.rs (shared types, resolution logic, CLI struct, tests) and src/ci/github.rs (GitHub Actions YAML generation).

Add declarative CI configuration via pyproject.toml:

  • [tool.maturin.generate-ci.github] with pytest, zig, skip-attestation
  • Per-platform config (linux, musllinux, windows, macos, emscripten, android)
  • Simple targets list or detailed [[target]] array with per-target overrides
  • Resolution chain: per-target -> platform-level -> hardcoded defaults
  • Smart YAML emission: uniform values at step level, varying via matrix
  • Platform presence in pyproject determines which platforms are built
  • CLI flags override pyproject values; --platform overrides platform selection
  • Validation: targets and [[target]] are mutually exclusive

Deprecate CLI options (--platform, --pytest, --zig, --skip-attestation) with warnings directing users to [tool.maturin.generate-ci.github].

@messense messense changed the title Add [tool.maturin.generate-ci.github] config support Add [tool.maturin.generate-ci.github] config support Mar 4, 2026
@messense messense force-pushed the generate-ci-config branch 2 times, most recently from d23a35c to 47beec1 Compare March 4, 2026 13:37
@messense messense requested a review from Copilot March 4, 2026 14:40
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR refactors src/ci.rs into a module (src/ci/mod.rs + src/ci/github.rs) and adds declarative CI configuration support via [tool.maturin.generate-ci.github] in pyproject.toml. It introduces a resolution chain (per-target → platform-level → hardcoded defaults) for CI options and smart YAML emission (step-level for uniform values, matrix references for varying values). CLI options (--platform, --pytest, --zig, --skip-attestation) are deprecated with warnings directing users to the new pyproject config.

Changes:

  • Splits src/ci.rs into src/ci/mod.rs (shared types, CLI, deprecation warnings) and src/ci/github.rs (GitHub Actions generation, resolution logic)
  • Adds GenerateCIConfig, GitHubCIConfig, PlatformCIConfig, and TargetCIConfig to src/pyproject_toml.rs for declarative configuration
  • Updates src/new_project.rs call site and tests/cmd/generate-ci.stdout for the new API

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
src/ci/mod.rs New module with shared types (Platform, ResolvedTarget, ResolvedCIConfig), GenerateCI CLI struct, generate() method, and deprecation warnings
src/ci/github.rs GitHub Actions YAML generation refactored to use resolve_config/generate_github; adds smart uniform-vs-matrix field emission; new tests for pyproject config integration
src/pyproject_toml.rs New config types (GenerateCIConfig, GitHubCIConfig, PlatformCIConfig, TargetCIConfig) with deserialization and tests
src/new_project.rs Updated from generate_github method to generate_github_from_cli function call
tests/cmd/generate-ci.stdout Updated help text to reflect deprecation messages and removal of [default: ...] for --platform
Comments suppressed due to low confidence (4)

src/ci/github.rs:854

  • Same YAML quoting issue: the extra_args value emitted at this line is user-provided and not quoted. If the args value contains YAML special characters (such as : or #), the generated YAML will be malformed. For example, an args value like --cfg 'feature="serialize"' (with single quotes) would need quoting to be valid YAML.
    src/ci/github.rs:432
  • When targets is empty (e.g., a user explicitly sets targets: [] in their pyproject.toml config), the generated YAML will include a job with runs-on: ${{ matrix.platform.runner }} but without a strategy.matrix block. This is invalid GitHub Actions YAML, as matrix.platform.runner would be undefined without a matrix definition.

The code should either skip the platform entirely when its target list is empty (by using continue in the loop), or validate and error out when an empty targets list is detected. Since platform_targets already contains an empty Vec for such a platform, the fix would be to add a check like if targets.is_empty() { continue; } at the start of the loop iteration.
src/ci/github.rs:62

  • User-provided values from PlatformCIConfig and TargetCIConfig fields (before_script_linux, docker_options, args, container, rustup_components, rust_toolchain) are emitted into the generated YAML without any quoting or escaping. This means that if a value contains a YAML special sequence like : (colon followed by space), # (comment start), or starts with {, [, >, |, etc., the generated YAML will be invalid.

For example, a docker_options value like --label "key: value" (which is a realistic Docker flag) or a before_script_linux value containing a comment like apt-get install libssl-dev # required for TLS would produce invalid YAML output. The emitted value should be quoted using YAML double-quote or single-quote syntax when it contains special characters.
src/ci/github.rs:847

  • Same YAML quoting issue as in emit_maturin_action_field: user-provided values emitted in matrix entries at line 846 are not quoted. Values for fields like before_script_linux or docker_options containing : (colon+space) or # (comment marker) would produce invalid YAML. For example, before_script_linux: apt-get install -y # setup would be truncated by YAML's comment parsing. The values should be YAML-quoted when they contain special characters.

@messense messense force-pushed the generate-ci-config branch 2 times, most recently from 363d685 to e83c57c Compare March 5, 2026 00:07
Split src/ci.rs into src/ci/mod.rs (shared types, resolution logic,
CLI struct, tests) and src/ci/github.rs (GitHub Actions YAML generation).

Add declarative CI configuration via pyproject.toml:
- [tool.maturin.generate-ci.github] with pytest, zig, skip-attestation
- Per-platform config (linux, musllinux, windows, macos, emscripten, android)
- Simple targets list or detailed [[target]] array with per-target overrides
- Resolution chain: per-target -> platform-level -> hardcoded defaults
- Smart YAML emission: uniform values at step level, varying via matrix
- Platform presence in pyproject determines which platforms are built
- CLI flags override pyproject values; --platform overrides platform selection
- Validation: targets and [[target]] are mutually exclusive

Deprecate CLI options (--platform, --pytest, --zig, --skip-attestation)
with warnings directing users to [tool.maturin.generate-ci.github].
@messense messense force-pushed the generate-ci-config branch from e83c57c to e468aa6 Compare March 5, 2026 11:26
@messense messense marked this pull request as ready for review March 6, 2026 12:08
- Use || instead of if/else for boolean resolution
- Deduplicate default_targets fallback with match
- Add resolve_optional() helper to DRY up resolution chain
- Remove redundant emscripten guard in generate_github
- Return &'static [Self] from Platform::defaults()/all()
- Remove redundant platforms field from ResolvedCIConfig
@messense messense force-pushed the generate-ci-config branch 5 times, most recently from 043d2eb to b66ac0f Compare March 7, 2026 02:48
@messense messense requested a review from Copilot March 7, 2026 02:49
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 18 out of 18 changed files in this pull request and generated 2 comments.

@messense messense force-pushed the generate-ci-config branch from b66ac0f to f82564c Compare March 7, 2026 03:01
@messense messense linked an issue Mar 7, 2026 that may be closed by this pull request
…to files

- Add `Yaml` struct for indentation-tracked YAML writing
- Add `gha_expr()` helper and constants to avoid `${{{{ }}}}` escaping noise
- Extract pytest generation into `emit_pytest_steps()` with per-platform helpers
- Deduplicate uv-based pytest (manylinux host/windows/macos) via `emit_uv_pytest()`
- Move 7 inline `expect!` snapshots to `src/ci/__snapshots__/` using `expect_file!`
- Reduces `github.rs` from ~2600 to ~1420 lines
@messense messense force-pushed the generate-ci-config branch from f82564c to ff58c30 Compare March 7, 2026 03:45
@messense messense merged commit 8bf07e9 into PyO3:main Mar 7, 2026
44 of 45 checks passed
@messense messense deleted the generate-ci-config branch March 7, 2026 04:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Maturin generate-ci should include android output?

2 participants