Upgrade NumPy and other Python dependencies

[cclauss]

Blocked by pysim #45 and #46:

• setup.py is using distutils which was removed in Python >= 3.12 aldebjer/pysim#45
• setup.py xxx is deprecated and will be disabled in October 2025 aldebjer/pysim#46

---

@peterbarker wrote:

We have many warnings coming out of our tools saying we're using deprecated libraries and APIs; eg. we're restricted to numpy < 2 in some of our tooling, and that's not playing nice with tools created this century. If I got to point people at fixing problems with our Python code, that is definitely what I would suggest!

• waf: Fix flake8 whitespace issues in Python files #30278 (review)

I could not find any open issues about NumPy. I could not find where Python dependency versions were being pinned. For example, searching for numpy in this repo only libraries/AP_NavEKF3/derivation/generate_nix and Tools/environment_install/install-prereqs-arch.sh seem to be selecting NumPy versions. The former via a Nix package hash and the latter via apt package. Both must be out of date if they are still installing NumPy < v2 a year after it was released (NumPy v2.3 is current). The usual pyproject.toml, requirements.txt, setup.cfg, setup.py, or PEP723 headers methods of specifying Python dependencies are not used in this repo. Is there some other Python dependency that is preventing us from upgrading to NumPy v2?

% ruff check --select=NPY # Raises no issue of deprecated NumPy usage.

• https://docs.astral.sh/ruff/rules/#numpy-specific-rules-npy

Feature request

Is your feature request related to a problem? Please describe.
Understand how and where Python dependencies are being pinned in this repo on various operating systems.
Understand why Python dependencies are being pinned to out-of-date versions.
Enable them to be upgraded regardless of operating system.

Describe the solution you'd like
A clearly defined way to pin and upgrade Python dependencies such as pyproject.toml or PEP723 headers.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Platform
[x] All

Additional context
Add any other context or screenshots about the feature request here.

[tpwrules]

The AP_NavEKF3 mustn't be changed, it's pinned as that's the version that generates the EKF code in tree. We need to do a bunch of EKF testing if we want to change it.

[cclauss]

@peterbarker wrote #30361 (comment)

Just a note that upgrading numpy won't necessarily kill in-tree tools, but most definitely kills MAVProxy, which is usually run in the same venv. This does suck.

---

I do not see anything that restricts the version of NumPy used in MAVProxy.

• https://github.com/search?q=repo%3AArduPilot%2FMAVProxy%20numpy&type=code

[cclauss]

Fudge! pysim != pysim

https://github.com/ArduPilot/ardupilot/tree/master/Tools/autotest/pysim is not https://pysim.org