Skip to content

Add an integration test and debug issues with the meson backend#2718

Merged
joerick merged 10 commits intomainfrom
meson-test
Feb 8, 2026
Merged

Add an integration test and debug issues with the meson backend#2718
joerick merged 10 commits intomainfrom
meson-test

Conversation

@joerick
Copy link
Contributor

@joerick joerick commented Jan 25, 2026

Xref #2663.

Trying to make a test that recreates the issue in #2663.

The integration test for meson probably won't recreate it, but the bin/run_example_ci_configs.py should be able to.

@joerick joerick marked this pull request as draft January 25, 2026 22:56
@joerick
Copy link
Contributor Author

joerick commented Jan 25, 2026


Examples test run

Branch: example-config-test---meson-test-2026-01-25T23-02-46

Service Config Status
github examples/github-minimal.yml Build

Generated by bin/run_example_ci_configs.py --build-backend meson examples/github-minimal.yml


@joerick
Copy link
Contributor Author

joerick commented Jan 25, 2026

Well, the example config test did fail on windows. I'm a little disheartened that the error message doesn't match the one we saw in #2663.

  The Meson build system
  Version: 1.10.1
  Source dir: D:\a\cibuildwheel\cibuildwheel
  Build dir: D:\a\cibuildwheel\cibuildwheel\.mesonpy-veirak6b
  Build type: native build
  Project name: spam
  Project version: 0.1.0
  C compiler for the host machine: C:\Strawberry\c\bin\ccache.EXE cc (gcc 15.2.0 "cc (x86_64-posix-seh-rev0, Built by MinGW-Builds project) 15.2.0")
  C linker for the host machine: cc ld.bfd 2.45
  Host machine cpu family: x86_64
  Host machine cpu: x86_64
  Program python found: YES (C:\Users\runneradmin\AppData\Local\Temp\build-env-88vgdqqj\Scripts\python.exe)
  Need python for x86_64, but found x86
  Run-time dependency python found: NO (tried pkgconfig and sysconfig)
  
  ..\meson.build:9:3: ERROR: Python dependency not found

@joerick
Copy link
Contributor Author

joerick commented Jan 25, 2026

Aha! But it is the same error as we see in #2701 ! So I think perhaps this is the same issue appearing in different ways.

@joerick
Copy link
Contributor Author

joerick commented Jan 26, 2026

Hm... well the integration test (which is unaffected by the action.yml changes) is failing with the same error-

Command line: `cc -IC:\\Users\\runneradmin\\AppData\\Local\\pypa\\cibuildwheel\\Cache\\nuget-cpython\\pythonx86.3.12.10\\tools\\Include C:\\Users\\runneradmin\\AppData\\Local\\Temp\\pytest-of-runneradmin\\pytest-1\\popen-gw1\\test_meson_python_basic_build_0\\project\\.mesonpy-4w1qbvcz\\meson-private\\tmpstr2bsu7\\testfile.c -E -P -D_FILE_OFFSET_BITS=64 -P -O0` -> 0
Run-time dependency python found: NO (tried pkgconfig and sysconfig)

..\\meson.build:9:3: ERROR: Python dependency not found

Perhaps I'm missing something in my meson setup?

Test Meson sources here.

@joerick
Copy link
Contributor Author

joerick commented Jan 26, 2026

Ah, this error is in the win32 build. We have this warning earlier in the file-

Need python for x86_64, but found x86

So we've given meson a Python.exe that's x86, but it thinks it wants the x86_64 for some reason. I'd like to know how meson-python has decided it's targeting x86_64.

Edit: It's chosen x86_64 because the compiler it found on PATH is defaulting to that.

@joerick joerick changed the title Add an integration test with the meson backend Add an integration test and debug issues with the meson backend Jan 27, 2026
@joerick
Copy link
Contributor Author

joerick commented Jan 28, 2026

Here's what I've discovered.

By default, meson searches PATH for the first available cc compiler. On Github, even when running in pwsh, that compiler is MinGW.

Most projects out there have found ways to work around this. I've seen two ways-

  • ilammy/msvc-dev-cmd, example which pushes the VS compilers to the top of the PATH.
  • another approach, used by Matplotlib, is to pass --vsenv to setup-args using the config-settings option.

The other wrinkle is win32 builds. As far as I can tell, this isn't too easy with meson-python. Many of the scientific stack projects have already dropped win32. Most projects that support it seem to use separate builds and use ilammy/msvc-dev-cmd to choose a 32-bit toolchain before starting cibuildwheel. Curious if @rgommers has a take here?

Also I wonder if build-details.json PEP 739 could help here.

Anyway, for now, I've disabled win32 builds in this test.

I'd love to improve this interplay between cibuildwheel and meson-python. One idea I've had would be for cibuildwheel to call 'vsdevcmd.bat', like how we already do it for GraalPy. I suspect we might have to make that configurable, though. Some users might already have explicitly setup compilers on PATH.

This is so it can be tested with bin/run_example_ci_configs.py
@joerick
Copy link
Contributor Author

joerick commented Jan 28, 2026


Examples test run

Branch: example-config-test---meson-test-2026-01-28T09-28-54

Service Config Status
github examples/github-minimal.yml Build

Generated by bin/run_example_ci_configs.py --build-backend meson examples/github-minimal.yml


@rgommers
Copy link

Thanks for the ping @joerick. Yes I can see a few possible solutions. I don't think build-details.json is relevant here (meson-python can easily know the bitness of the interpreter either way). And gh-2673 should probably be merged independent of whatever solution is chosen, since it fixes regressions and is a much cleaner fix?

By default, meson searches PATH for the first available cc compiler.

That's one of the way. The others ways are:

  1. Native and cross files containing c = /path/to/c/compiler
  2. Checking for the relevant environment variable(s): CC, CXX
  3. Known compiler names on PATH
  4. If nothing was found for (3), try auto-activating MSVC on Windows (x64 on 64-bit Windows)

Note that if --vsenv is passed, that selects MSVC before checking on PATH - but always 64-bit MSVC on 64-bit Windows.

I'll also note that CMake is quite similar to Meson with these options, and this scikit-build-core issue described roughly the same problem: scikit-build/scikit-build-core#668.

I suspect we might have to make that configurable, though. Some users might already have explicitly setup compilers on PATH.

Agreed, it's quite hard to change defaults without breaking some build setup or use case for other users. You can't really go check all the ways existing compilers may already be configured. So opt-in is a safer. Or, if the backwards compat impact seems acceptable, then there should be an easy opt-out at the very least.

Meson is a generic build system, it's not going to change its compiler selection logic here (it's been considered and rejected, for good reasons). That means that if we want a more polished user experience by special-casing 32-bit Python on 64-bit Windows, we can either do it in cibuildwheel, or in meson-python. And as opt-out or opt-in.

There is one relevant Meson issue which would make it easier for the user to opt in to using 32-bit MSVC: mesonbuild/meson#11435 (this comment in particular).

(Second post with solution options coming)

@rgommers
Copy link

rgommers commented Jan 28, 2026

Actually, on second thought, the list of solution options is shorter than I thought.

I think the main one is to do both of these:

  1. Extending Meson's built-in functionality so one can easily do --vsenv=x86 or --vsenv=arm64, as discussed in meson#11435
  2. meson-python to check if the user added --vsenv, and if so then checking the target interpreter - and if it's 32-bit Python on Windows, then use --vsenv=x86. The same might apply to compiling for arm64 on an x64 machine with emulation (not 100% sure).

That makes it very easy for package authors to opt in to the automatic selection by using --vsenv in their pyproject.toml.

Cc @eli-schwartz and @dnicolodi for thoughts on this.

I'd love to improve this interplay between cibuildwheel and meson-python. One idea I've had would be for cibuildwheel to call 'vsdevcmd.bat',

I think that this is also reasonable to do, especially if it's easy and it's already being done for GraalPy. I'm not 100% sure that it won't be overridden anyway by --vsenv though.

@eli-schwartz
Copy link

Please note that the --vsenv command line option to meson instructs meson to pass force=True to the MSVC activation logic.

The MSVC activation logic is unconditionally run for all builds on all platforms. It:

  • returns False immediately if one is not on a Windows operating system
  • returns False immediately if cl.exe is on PATH and MSVC is already set up
  • checks if force=True and if so, skips checking on PATH for a GCC or clang or clang-cl compiler, otherwise it will early return if it finds one
  • tries to activate MSVC

Thus, passing --vsenv as a command line parameter to meson does not cause meson to detect MSVC, it just causes meson to prefer to detect MSVC even if it is "not on PATH && another compiler is on PATH".

@eli-schwartz
Copy link

Meson is a generic build system, it's not going to change its compiler selection logic here (it's been considered and rejected, for good reasons). That means that if we want a more polished user experience by special-casing 32-bit Python on 64-bit Windows, we can either do it in cibuildwheel, or in meson-python. And as opt-out or opt-in.

Yup -- note probably the biggest practical (rather than philosophical) issue here is that compiler detection has to happen during project() which occurs before import('python').find_installation(), after which it's too late.

I think the main one is to do both of these:

  1. Extending Meson's built-in functionality so one can easily do --vsenv=x86 or --vsenv=arm64, as discussed in meson#11435

This seems reasonable to me.

  1. meson-python to check if the user added --vsenv, and if so then checking the target interpreter - and if it's 32-bit Python on Windows, then use --vsenv=x86. The same might apply to compiling for arm64 on an x64 machine with emulation (not 100% sure).

This sounds like it might require parsing the meson command line and also project(default_options: [...]) and I'm not sure meson-python wants to go there. Although if it's just parsing the default setup arguments from pyproject.toml's tool.meson-python table then sure, that should be easy to do.

I think that this is also reasonable to do, especially if it's easy and it's already being done for GraalPy. I'm not 100% sure that it won't be overridden anyway by --vsenv though.

As per my previous comment, this will be fine.

@rgommers
Copy link

Although if it's just parsing the default setup arguments from pyproject.toml's tool.meson-python table then sure, that should be easy to do.

Yeah, that's what I was thinking, and is as far as I would go.

@joerick
Copy link
Contributor Author

joerick commented Jan 30, 2026

  • Extending Meson's built-in functionality so one can easily do --vsenv=x86 or --vsenv=arm64, as discussed in meson#11435

This would help, for sure! Users could set up a config override in cibuildwheel's config to set this depending on architecture, too.

Regarding the second solution,

Although if it's just parsing the default setup arguments from pyproject.toml's tool.meson-python table then sure, that should be easy to do.

Yeah, that's what I was thinking, and is as far as I would go.

How does a static config file specify both amd64 and x86? These are different per wheel build, not to be statically defined in a config. When cibuildwheel is using these options, it's passing them as config_settings.

On a separate note, it would be really cool if we could find some way, similar to ARCHFLAGS on macOS, for cibuildwheel to just do the right thing. Could we invent a MESON_PYTHON_VSENV env var that would have the effect of the config-settings = { "setup-args" = "--vsenv=<value>" }? That way cibuildwheel users wouldn't require any special config. (an environment variable would be better as we can namespace it to just MESON_PYTHON_ - the config_settings param might have undesirable effects on other backends).

@rgommers
Copy link

rgommers commented Jan 30, 2026

How does a static config file specify both amd64 and x86? These are different per wheel build, not to be statically defined in a config

I was thinking that if tool.meson-python contains setup-args = '--vsenv' (or --vsenv=auto), then meson-python will interpret that and translate to one of --vsenv=x86, --vsenv=x64, --vsenv=arm64 before passing it on to meson.

@joerick
Copy link
Contributor Author

joerick commented Jan 30, 2026

Ah, I see, yeah that could work. I was just confused because I thought that meson only knows the Python interpreter it's building for later in the meson.build file.

Having said that, for cibuildwheel usage, those are the same interpreter.

joerick added a commit that referenced this pull request Jan 30, 2026
Cherry-picked from #2718 so we can test the action.yml file
joerick added a commit that referenced this pull request Jan 30, 2026
Cherry-picked from #2718 and modified so we can test the action.yml file
@joerick joerick marked this pull request as ready for review January 30, 2026 17:17
@eli-schwartz
Copy link

eli-schwartz commented Jan 30, 2026

meson-python ordinarily instructs Meson to use the python interpreter that meson-python was run with. So it's still the same thing in that case, even outside of cibuildwheel. :)

joerick added a commit that referenced this pull request Feb 4, 2026
* Fix the PATH mangling in action.yml

Let's put the `uv` on PATH question aside, and fix #2663.

This change ensures that the PATH entries added by Github when you request `shell: bash` don't make it into the pwsh environment. It also keeps the uv binary on PATH if installed via `extras:`.

Would love to get this fixed soon, then we can figure out the right approach regarding `uv` discovery.

* Modify script to allow testing of the GHA action on a PR

Cherry-picked from #2718 and modified so we can test the action.yml file
@henryiii
Copy link
Contributor

henryiii commented Feb 6, 2026

Ready to go in? I want to add a scikit-build-core + uv workspaces test, and I'd like to build on this.

Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>
@joerick
Copy link
Contributor Author

joerick commented Feb 7, 2026

Yes, this is ready to go!

@joerick joerick merged commit a2f965f into main Feb 8, 2026
43 checks passed
@joerick joerick deleted the meson-test branch February 8, 2026 10:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants