autotest: Fall back to relative imports in pytest discovery

[cclauss]

Tools/autotest/test_build_options.py and Tools/autotest/test_param_upgrade.py have filenames that will cause Pytest Discovery to think they should be tested, but these files do not contain Python unittests or pytest tests.

• https://docs.pytest.org/en/stable/explanation/goodpractices.html#conventions-for-python-test-discovery

These files also import pysim and vehicle_test_suite from the same directory, but do not use relative imports, so pytest raises errors.

The proposed change is to use relative imports:

- from pysim import util
+ try:
+     from pysim import util
+ except ImportError: # Fall back to a relative import in pytest
+     from .pysim import util

Searches for these files:

• https://github.com/search?q=repo%3AArduPilot%2Fardupilot%20test_build_options.py&type=code
• https://github.com/search?q=repo%3AArduPilot%2Fardupilot%20test_param_upgrade.py&type=code

How was this tested?
% uv tool run --python=3.12 --with=MAVProxy --with=numpy --with=pexpect pytest Tools/autotest

Warnings were raised instead of crashing with a confusing error.

[tpwrules]

These are not designed for relative imports as you can see from the CI errors.

Is there a way to warn users who try to do a global discovery that the project is not designed for it? And point them to our test scripts instead? There's this implicit assumption you are making that pytest is a thing to do but I personally am not sure it's valid. ArduPilot is not a Python project. It would be nice to make the problem clearer for those who try but reworking the whole test system to be compatible might not be feasible or desirable.

[cclauss]

When a change is proposed, we want to add tests that can be found and run automatically. What I was looking for was a GitHub Action that detects new tests based on naming conventions. If we need to pytest --ignore a lot of the codebase, then contributors will have to do a fair amount of verification to understand if their tests will be run.

[tpwrules]

That is what the autotest framework is for: https://ardupilot.org/dev/docs/the-ardupilot-autotest-framework.html

If you are proposing e.g. testing that framework itself, which is not unreasonable, we probably need a dedicated directory for that and to always point pytest there. But currently pytest is not a thing that ArduPilot itself uses. Many sub-projects and some areas within ArduPilot (like DDS) do use it though.

For better or worse, the current tests for the build system and the autotest framework are generally just CI, i.e. the things that use them. Plus a culture to think through and manually exercise changes (e.g. test_param_upgrade.py is a script designed to help with that). There are many problems that turn up only on physical vehicles as well.

We could always use more testing but I think it needs a more principled and tailored approach than making the whole project pytestable. Perhaps, like in my previous suggestion, just making it clear to users that pytest is not the right way to interact with it to avoid creating more confusion.

[cclauss]

% pytest --ignore=modules/gtest \
         --ignore=modules/mavlink/pymavlink/tests/test_wp.py \
         --ignore=modules/waf \
         --ignore=Tools/autotest/test_build_options.py
         --ignore=Tools/autotest/test_param_upgrade.py
         --ignore=Tools/FilterTestTool \
         --ignore=Tools/ros2/ardupilot_dds_tests/test

============================= test session starts ==============================
platform linux -- Python 3.13.5, pytest-8.4.1, pluggy-1.6.0
rootdir: /home/runner/work/ardupilot/ardupilot
configfile: pyproject.toml
collected 160 items

Tools/autotest/unittest/annotate_params_test.py ........................ [ 15%]
                                                                         [ 15%]
Tools/autotest/unittest/extract_param_defaults_test.py ................. [ 25%]
.                                                                        [ 26%]
Tools/scripts/param_check_test.py ...........                            [ 33%]
[ ... ]

================ 153 passed, 7 skipped, 127 warnings in 30.91s =================

[peterbarker]

Needs to pass CI.

@robertlong13 will need to be happy with the changes to the parameter extraction test stuff.

[peterbarker]

I'm really concerned about bending our code to suit a new tool like this.

ArduPilot is not a Python project. I'm not sure we should be conforming to the conventions of one.

Can pytest be configured via a file in the root directory to ignore these files instead of modifying them here?