Adopt pytest-xdist

[davidorme]

Description

This PR follows @alexdewar's suggestion to use pytest-xdist to speed up testing. We're still going to run into test_ve_run being slow, but it should speed up testing generally. It also adds --jobs auto to the sphinx building options to distribute the running of MyST notebooks.

Fixes #941

Type of change

• New feature (non-breaking change which adds functionality)
• Optimization (back-end change that speeds up the code)
• Bug fix (non-breaking change which fixes an issue)

Key checklist

• Make sure you've run the pre-commit checks: $ pre-commit run -a
• All tests pass: $ poetry run pytest

Further checks

• Code is commented, particularly in hard-to-understand areas
• Tests added that prove fix is effective or that feature works
• Relevant documentation reviewed and updated

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.13%. Comparing base (0c3aece) to head (6c1fef8).

Additional details and impacted files

@@           Coverage Diff            @@
##           develop    #1023   +/-   ##
========================================
  Coverage    95.13%   95.13%           
========================================
  Files           80       80           
  Lines         6722     6722           
========================================
  Hits          6395     6395           
  Misses         327      327           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[alexdewar]

LGTM! I think this makes sense.

[alexdewar]

Would it also work if you just ran make here? Or will it not run without the poetry run prefix?

[davidorme]

I think poetry run make would work - but I think we probably do want to run the local sphinx-build in a slightly different way to the CI? So fail fast and not keep going locally but find everything wrong on CI? Doesn't make a huge difference but I'll leave as is for now.