Issue retrieving package metadata with 4.11.0

[avylove]

Issue

In GitHub Actions with 4.11.0
Example repo: https://github.com/Rockhopper-Technologies/prefixed

No issues with <= 4.10.0

Run tox -- --verbose
  tox -- --verbose
  shell: /usr/bin/bash -e {0}
  env:
    TOXENV: py3.8
    TOXPYTHON: python3.8
    pythonLocation: /opt/hostedtoolcache/Python/3.8.17/x64
    PKG_CONFIG_PATH: /opt/hostedtoolcache/Python/3.8.17/x64/lib/pkgconfig
    Python_ROOT_DIR: /opt/hostedtoolcache/Python/3.8.17/x64
    Python[2](https://github.com/Rockhopper-Technologies/prefixed/actions/runs/6032083693/job/16387986051#step:5:2)_ROOT_DIR: /opt/hostedtoolcache/Python/3.8.17/x64
    Python3_ROOT_DIR: /opt/hostedtoolcache/Python/3.8.17/x64
    LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.8.17/x64/lib
ROOT: will run in automatically provisioned tox, host /opt/hostedtoolcache/Python/3.8.17/x64/bin/python is missing [requires (has)]: virtualenv<=20.21.1 (20.24.4)
ROOT: install_deps> python -I -m pip install tox 'virtualenv<=20.21.1'
ROOT: freeze> python -m pip freeze --all
ROOT: cachetools==5.3.1,chardet==5.2.0,colorama==0.4.6,distlib==0.3.7,filelock==3.12.3,packaging==23.1,pip==23.2.1,platformdirs==3.10.0,pluggy==1.3.0,pyproject-api==1.6.1,setuptools==68.1.2,tomli==2.0.1,tox==4.5.1,typing_extensions==4.7.1,virtualenv==20.21.1,wheel==0.41.2
ROOT: provision> .tox/.tox/bin/python -m tox -- --verbose
.pkg: install_requires> python -I -m pip install 'setuptools>=40.8.0' wheel
.pkg: _optional_hooks> python /home/runner/work/prefixed/prefixed/.tox/.tox/lib/python[3](https://github.com/Rockhopper-Technologies/prefixed/actions/runs/6032083693/job/16387986051#step:5:3).8/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
.pkg: get_requires_for_build_editable> python /home/runner/work/prefixed/prefixed/.tox/.tox/lib/python3.8/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
.pkg: install_requires_for_build_editable> python -I -m pip install wheel
.pkg: freeze> python -m pip freeze --all
.pkg: pip==23.1,setuptools==67.6.1,wheel==0.[4](https://github.com/Rockhopper-Technologies/prefixed/actions/runs/6032083693/job/16387986051#step:5:4)0.0
py3.8: internal error
Traceback (most recent call last):
  File "/home/runner/work/prefixed/prefixed/.tox/.tox/lib/python3.8/site-packages/tox/session/cmd/run/single.py", line 4[5](https://github.com/Rockhopper-Technologies/prefixed/actions/runs/6032083693/job/16387986051#step:5:5), in _evaluate
    tox_env.setup()
  File "/home/runner/work/prefixed/prefixed/.tox/.tox/lib/python3.8/site-packages/tox/tox_env/api.py", line 250, in setup
    self._setup_with_env()
  File "/home/runner/work/prefixed/prefixed/.tox/.tox/lib/python3.8/site-packages/tox/tox_env/runner.py", line 143, in _setup_with_env
    self._setup_pkg()
  File "/home/runner/work/prefixed/prefixed/.tox/.tox/lib/python3.8/site-packages/tox/tox_env/runner.py", line 1[6](https://github.com/Rockhopper-Technologies/prefixed/actions/runs/6032083693/job/16387986051#step:5:6)6, in _setup_pkg
    self._packages = self._build_packages()
  File "/home/runner/work/prefixed/prefixed/.tox/.tox/lib/python3.8/site-packages/tox/tox_env/python/runner.py", line 118, in _build_packages
    packages = package_env.perform_packaging(self.conf)
  File "/home/runner/work/prefixed/prefixed/.tox/.tox/lib/python3.8/site-packages/tox/tox_env/package.py", line 4[7](https://github.com/Rockhopper-Technologies/prefixed/actions/runs/6032083693/job/16387986051#step:5:7), in _func
    return meth(*args, **kwargs)
  File "/home/runner/work/prefixed/prefixed/.tox/.tox/lib/python3.[8](https://github.com/Rockhopper-Technologies/prefixed/actions/runs/6032083693/job/16387986051#step:5:8)/site-packages/tox/tox_env/python/virtual_env/package/pyproject.py", line 186, in perform_packaging
    deps = self._load_deps(for_env)
  File "/home/runner/work/prefixed/prefixed/.tox/.tox/lib/python3.8/site-packages/tox/tox_env/package.py", line 47, in _func
    return meth(*args, **kwargs)
  File "/home/runner/work/prefixed/prefixed/.tox/.tox/lib/python3.8/site-packages/tox/tox_env/python/virtual_env/package/pyproject.py", line 23[9](https://github.com/Rockhopper-Technologies/prefixed/actions/runs/6032083693/job/16387986051#step:5:9), in _load_deps
    deps = self._load_deps_from_built_metadata(for_env)
  File "/home/runner/work/prefixed/prefixed/.tox/.tox/lib/python3.8/site-packages/tox/tox_env/package.py", line 47, in _func
    return meth(*args, **kwargs)
  File "/home/runner/work/prefixed/prefixed/.tox/.tox/lib/python3.8/site-packages/tox/tox_env/python/virtual_env/package/pyproject.py", line 283, in _load_deps_from_built_metadata
    reqs = self.get_package_dependencies(for_env)
  File "/home/runner/work/prefixed/prefixed/.tox/.tox/lib/python3.8/site-packages/tox/tox_env/package.py", line 47, in _func
    return meth(*args, **kwargs)
  File "/home/runner/work/prefixed/prefixed/.tox/.tox/lib/python3.8/site-packages/tox/tox_env/python/virtual_env/package/pyproject.py", line 292, in get_package_dependencies
    self._ensure_meta_present(for_env)
  File "/home/runner/work/prefixed/prefixed/.tox/.tox/lib/python3.8/site-packages/tox/tox_env/package.py", line 47, in _func
    return meth(*args, **kwargs)
  File "/home/runner/work/prefixed/prefixed/.tox/.tox/lib/python3.8/site-packages/tox/tox_env/python/virtual_env/package/pyproject.py", line 3[10](https://github.com/Rockhopper-Technologies/prefixed/actions/runs/6032083693/job/16387986051#step:5:10), in _ensure_meta_present
    dist_info = end.prepare_metadata_for_build_editable(self.meta_folder, self._wheel_config_settings).metadata
AttributeError: 'NoneType' object has no attribute 'metadata'
.pkg: _exit> python /home/runner/work/prefixed/prefixed/.tox/.tox/lib/python3.8/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
  py3.8: FAIL code 2 (5.96 seconds)
  evaluation failed :( (7.09 seconds)
Error: Process completed with exit code 2.

[gaborbernat]

ROOT: freeze> python -m pip freeze --all
ROOT: cachetools==5.3.1,chardet==5.2.0,colorama==0.4.6,distlib==0.3.7,filelock==3.12.3,packaging==23.1,pip==23.2.1,platformdirs==3.10.0,pluggy==1.3.0,pyproject-api==1.6.1,setuptools==68.1.2,tomli==2.0.1,tox==4.5.1,typing_extensions==4.7.1,virtualenv==20.21.1,wheel==0.41.2

The problem here is that virtualenv<=20.21.1 does not support latest tox. And latest pyproject-api==1.6.1 does not support old tox==4.5.1.

[avylove]

The virtualenv version hasn't been an issue, but yeah, if I pin pyproject_api tp 1.5.4 it's not a problem, so some issue introduced in 1.6.0.

[gaborbernat]

https://github.com/tox-dev/tox/releases/tag/4.5.1.1
https://github.com/tox-dev/tox/actions/runs/6039733358

should fix it.

[gaborbernat]

The virtualenv version hasn't been an issue,

It is because the reason you use an old tox release (4.5.1) is because you're requesting and old tox... but old tox doesn't work with new pyproject-api.

[avylove]

Does tox install an older version of itself if you set requires?

[gaborbernat]

Does tox install an older version of itself if you set requires?

tox does not install anything, it delegates it to pip. pip is one resolving dependencies. It does essentially pip install <other requirements> tox>=<minversion>. So if you set a requires that is not compatible with latest tox, yes you will get an older tox.

[avylove]

Ah, ok! Thanks for the quick fix!

[The-Compiler]

I still seem to be seeing this (though with end.prepare_metadata_for_build_wheel):

$ tox -e py311  --verbose -r
.pkg: find interpreter for spec PythonSpec(major=3)
.pkg: proposed PythonInfo(spec=CPython3.11.5.final.0-64, exe=/usr/bin/python, platform=linux, version='3.11.5 (main, Aug 28 2023, 20:02:58) [GCC 13.2.1 20230801]', encoding_fs_io=utf-8-utf-8)
.pkg: find interpreter for spec PythonSpec(path=/usr/bin/python)
.pkg: proposed PythonInfo(spec=CPython3.11.5.final.0-64, exe=/usr/bin/python, platform=linux, version='3.11.5 (main, Aug 28 2023, 20:02:58) [GCC 13.2.1 20230801]', encoding_fs_io=utf-8-utf-8)
.pkg: find interpreter for spec PythonSpec(major=3)
.pkg: proposed PythonInfo(spec=CPython3.11.5.final.0-64, exe=/usr/bin/python, platform=linux, version='3.11.5 (main, Aug 28 2023, 20:02:58) [GCC 13.2.1 20230801]', encoding_fs_io=utf-8-utf-8)
.pkg: find interpreter for spec PythonSpec(major=3)
.pkg: proposed PythonInfo(spec=CPython3.11.5.final.0-64, exe=/usr/bin/python, platform=linux, version='3.11.5 (main, Aug 28 2023, 20:02:58) [GCC 13.2.1 20230801]', encoding_fs_io=utf-8-utf-8)
.pkg: find interpreter for spec PythonSpec(path=/usr/bin/python)
.pkg: proposed PythonInfo(spec=CPython3.11.5.final.0-64, exe=/usr/bin/python, platform=linux, version='3.11.5 (main, Aug 28 2023, 20:02:58) [GCC 13.2.1 20230801]', encoding_fs_io=utf-8-utf-8)
.pkg: find interpreter for spec PythonSpec(major=3)
.pkg: proposed PythonInfo(spec=CPython3.11.5.final.0-64, exe=/usr/bin/python, platform=linux, version='3.11.5 (main, Aug 28 2023, 20:02:58) [GCC 13.2.1 20230801]', encoding_fs_io=utf-8-utf-8)
py311: remove tox env folder /home/florian/proj/pytest/.tox/py311
.pkg: remove tox env folder /home/florian/proj/pytest/.tox/.pkg
py311: find interpreter for spec PythonSpec(major=3, minor=11)
py311: proposed PythonInfo(spec=CPython3.11.5.final.0-64, exe=/usr/bin/python, platform=linux, version='3.11.5 (main, Aug 28 2023, 20:02:58) [GCC 13.2.1 20230801]', encoding_fs_io=utf-8-utf-8)
py311: create virtual environment via CPython3Posix(dest=/home/florian/proj/pytest/.tox/py311, clear=False, no_vcs_ignore=False, global=False)
py311: add seed packages via FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/florian/.local/share/virtualenv)
py311: add activators for Bash, CShell, Fish, Nushell, PowerShell, Python
.pkg: create virtual environment via CPython3Posix(dest=/home/florian/proj/pytest/.tox/.pkg, clear=False, no_vcs_ignore=False, global=False)
.pkg: add seed packages via FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/florian/.local/share/virtualenv)
.pkg: add activators for Bash, CShell, Fish, Nushell, PowerShell, Python
.pkg: install_requires> python -I -m pip install 'setuptools-scm[toml]>=6.2.3' 'setuptools>=45.0'
.pkg: exit 0 (5.64 seconds) /home/florian/proj/pytest> python -I -m pip install 'setuptools-scm[toml]>=6.2.3' 'setuptools>=45.0' pid=520465
.pkg: _optional_hooks> python /usr/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta
.pkg: exit None (0.73 seconds) /home/florian/proj/pytest> python /usr/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta pid=520545
.pkg: get_requires_for_build_editable> python /usr/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta
.pkg: exit None (0.07 seconds) /home/florian/proj/pytest> python /usr/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta pid=520545
.pkg: install_requires_for_build_editable> python -I -m pip install setuptools 'setuptools-scm>=6.0' wheel
.pkg: exit 0 (2.08 seconds) /home/florian/proj/pytest> python -I -m pip install setuptools 'setuptools-scm>=6.0' wheel pid=520563
.pkg: get_requires_for_build_sdist> python /usr/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta
.pkg: exit None (0.09 seconds) /home/florian/proj/pytest> python /usr/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta pid=520545
.pkg: install_requires_for_build_sdist> python -I -m pip install setuptools 'setuptools-scm>=6.0'
.pkg: exit 0 (2.05 seconds) /home/florian/proj/pytest> python -I -m pip install setuptools 'setuptools-scm>=6.0' pid=520603
py311: internal error
Traceback (most recent call last):
  File "/usr/lib/python3.11/site-packages/tox/session/cmd/run/single.py", line 45, in _evaluate
    tox_env.setup()
  File "/usr/lib/python3.11/site-packages/tox/tox_env/api.py", line 250, in setup
    self._setup_with_env()
  File "/usr/lib/python3.11/site-packages/tox/tox_env/runner.py", line 143, in _setup_with_env
    self._setup_pkg()
  File "/usr/lib/python3.11/site-packages/tox/tox_env/runner.py", line 166, in _setup_pkg
    self._packages = self._build_packages()
                     ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/tox/tox_env/python/runner.py", line 118, in _build_packages
    packages = package_env.perform_packaging(self.conf)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/tox/tox_env/package.py", line 47, in _func
    return meth(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/tox/tox_env/python/virtual_env/package/pyproject.py", line 186, in perform_packaging
    deps = self._load_deps(for_env)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/tox/tox_env/package.py", line 47, in _func
    return meth(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/tox/tox_env/python/virtual_env/package/pyproject.py", line 239, in _load_deps
    deps = self._load_deps_from_built_metadata(for_env)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/tox/tox_env/package.py", line 47, in _func
    return meth(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/tox/tox_env/python/virtual_env/package/pyproject.py", line 283, in _load_deps_from_built_metadata
    reqs = self.get_package_dependencies(for_env)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/tox/tox_env/package.py", line 47, in _func
    return meth(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/tox/tox_env/python/virtual_env/package/pyproject.py", line 292, in get_package_dependencies
    self._ensure_meta_present(for_env)
  File "/usr/lib/python3.11/site-packages/tox/tox_env/package.py", line 47, in _func
    return meth(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/tox/tox_env/python/virtual_env/package/pyproject.py", line 312, in _ensure_meta_present
    dist_info = end.prepare_metadata_for_build_wheel(self.meta_folder, self._wheel_config_settings).metadata
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'metadata'
.pkg: _exit> python /usr/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta
.pkg: exit None (0.00 seconds) /home/florian/proj/pytest> python /usr/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta pid=520545
  py311: FAIL code 2 (12.14 seconds)
  evaluation failed :( (12.67 seconds)

yet:

$ tox --version
4.5.1.1 from /usr/lib/python3.11/site-packages/tox/__init__.py
$ virtualenv --version
virtualenv 20.24.3 from /usr/lib/python3.11/site-packages/virtualenv/__init__.py

[gaborbernat]

You would need pip list to find out what's there 😊 https://github.com/tox-dev/pyproject-api version is that also matters.