Skip to content

Allow pytest discovery to find and run Python tests#30361

Open
cclauss wants to merge 1 commit intoArduPilot:masterfrom
cclauss:pytest-discovery
Open

Allow pytest discovery to find and run Python tests#30361
cclauss wants to merge 1 commit intoArduPilot:masterfrom
cclauss:pytest-discovery

Conversation

@cclauss
Copy link
Contributor

@cclauss cclauss commented Jun 15, 2025

  • Upgrade NumPy and other Python dependencies #30292 says that dependencies are being pinned to out-of-date versions. Let's use pytest discover to see which Python tests will be found and see what dependencies they require, and if they will pass with the current versions of those dependencies.

https://docs.pytest.org/en/stable/explanation/goodpractices.html#conventions-for-python-test-discovery

Test results: https://github.com/ArduPilot/ardupilot/actions/runs/16271173917/job/45938928611?pr=30361

% pytest --config-file=tests/pytest-whole-repo.toml

============================= test session starts ==============================
platform linux -- Python 3.14.0, pytest-8.4.2, pluggy-1.6.0
rootdir: /home/runner/work/ardupilot/ardupilot/tests
configfile: pytest-whole-repo.toml
collected 59 items

tests/Tools/autotest/unittest/annotate_params_unittest.py .............. [ 23%]
..........                                                               [ 40%]
tests/Tools/autotest/unittest/extract_param_defaults_unittest.py ....... [ 52%]
...........                                                              [ 71%]
tests/Tools/scripts/param_check_unittests.py ...........                 [ 89%]
tests/test_pre_commit_copyright.py ......                                [100%]

============================== 59 passed in 2.83s ==============================

@cclauss cclauss marked this pull request as draft June 15, 2025 09:45
@peterbarker
Copy link
Contributor

Just a note that upgrading numpy won't necessarily kill in-tree tools, but most definitely kills MAVProxy which is usually run in the same venv. This does suck.

@cclauss cclauss closed this Jun 16, 2025
@cclauss cclauss deleted the pytest-discovery branch June 16, 2025 21:11
@cclauss cclauss restored the pytest-discovery branch June 23, 2025 02:16
@cclauss cclauss reopened this Jun 23, 2025
@cclauss cclauss force-pushed the pytest-discovery branch 3 times, most recently from 12dc8e9 to 73cbc49 Compare June 23, 2025 03:10
@cclauss cclauss changed the title DRAFT: Check pytest discovery's ability to find Python tests Allow pytest discovery to find and run Python tests Jun 23, 2025
@cclauss cclauss marked this pull request as ready for review June 23, 2025 03:19
@cclauss cclauss force-pushed the pytest-discovery branch 2 times, most recently from 2fe57b3 to f2ff909 Compare June 24, 2025 11:06
@cclauss
Copy link
Contributor Author

cclauss commented Jun 24, 2025

Status:

  1. --ignore=modules/gtest DRAFT: gtest needs an upgrade for Python 3 #30426
    • Currently contains Python 2 code that would crash Py3. Probably always --ignore.
  2. --ignore=modules/mavlink/pymavlink/generator/C/test/posix/sha256_test.py
  3. --ignore=modules/mavlink/pymavlink/tests/test_wp.py
    • Mismatch in number of arguments! -- Help needed.
  4. --ignore=modules/waf/playground
  5. --ignore=Tools/autotest/test_build_options.py
  6. --ignore=Tools/autotest/test_param_upgrade.py
  7. --ignore=Tools/FilterTestTool/run_filter_test.py
  8. --ignore=Tools/ros2/ardupilot_dds_tests/test

@cclauss cclauss marked this pull request as draft June 25, 2025 08:11
@cclauss cclauss force-pushed the pytest-discovery branch from f2ff909 to 468860a Compare July 4, 2025 00:09
@cclauss cclauss marked this pull request as ready for review July 4, 2025 11:46
@cclauss cclauss force-pushed the pytest-discovery branch from 468860a to fe09429 Compare July 5, 2025 19:01
@cclauss cclauss closed this Jul 6, 2025
@cclauss cclauss deleted the pytest-discovery branch July 6, 2025 11:39
@cclauss cclauss restored the pytest-discovery branch July 14, 2025 14:17
@cclauss cclauss reopened this Jul 14, 2025
@cclauss cclauss force-pushed the pytest-discovery branch from fe09429 to 2f29dd3 Compare July 14, 2025 14:33
@cclauss cclauss requested a review from Ryanf55 July 14, 2025 14:49
@cclauss cclauss force-pushed the pytest-discovery branch from 2f29dd3 to 8117a1d Compare July 14, 2025 15:33
@cclauss cclauss force-pushed the pytest-discovery branch 2 times, most recently from 65fb090 to 3be54b8 Compare July 15, 2025 13:35
@cclauss
Copy link
Contributor Author

cclauss commented Jul 16, 2025

Based on the "too wildcardy" discussion in today's EU dev call, we should stop using --ignore to constrain the pytest discovery rules and instead use testpaths to focus pytest only on paths that are known to contain runnable tests.

testpaths = [
    "tests",
    "Tools/autotest/unittest",
    "Tools/scripts",
]

Also,

@cclauss cclauss marked this pull request as draft July 16, 2025 10:03
@peterbarker
Copy link
Contributor

Based on the "too wildcardy" discussion in today's EU dev call, we should stop using --ignore to constrain the pytest discovery rules and instead use testpaths to focus pytest only on paths that are known to contain runnable tests.

testpaths = [
    "tests",
    "Tools/autotest/unittest",
    "Tools/scripts",
]

Also,

* [tests: README.md caution about pytest run in GitHub Actions #30666](https://github.com/ArduPilot/ardupilot/pull/30666)

testpaths should allow us to get something in quite quickly. There are many PRs in your checklist in here which are stalled/stale, so perhaps pushing ahead on testpaths here would be a good idea?

@cclauss cclauss force-pushed the pytest-discovery branch 2 times, most recently from df8ef02 to 7e9e98b Compare November 3, 2025 13:21
@cclauss cclauss marked this pull request as ready for review November 3, 2025 13:21
@cclauss
Copy link
Contributor Author

cclauss commented Nov 3, 2025

59 passed in 2.83s -- Ready for review.

Can a repo maintainer please rerun the failing test?

* ArduPilot#30292 says that dependencies are being pinned to out-of-date versions.  Let's use
pytest discover to see which Python tests will be found and see what dependencies
they require, and if they will pass with the current versions of those dependencies.

https://docs.pytest.org/en/stable/explanation/goodpractices.html#conventions-for-python-test-discovery
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants