Switch cell_xy checks over to using a STRtree query

[davidorme]

Description

This PR replaces the incredibly slow check that incoming cell coordinate data maps 1-to-1 onto the grid with a SRTtree spatial search structure (https://shapely.readthedocs.io/en/latest/strtree.html#).

I've added a test that is purely there to run speed checks for three pairs of (nx, ny) grid sizes. There might be a better place for it, but maybe that will emerge from the upcoming performance review.

Using pytest tests/core/test_grid.py::test_map_xy_to_cell_ids_performance --durations=0 -n 0 to run this test gives the following runtimes for the old code vs this PR.

Before:

1953.27s call test_grid.py::test_map_xy_to_cell_ids_performance[150-150]
371.25s call  test_grid.py::test_map_xy_to_cell_ids_performance[100-100]
23.24s call   test_grid.py::test_map_xy_to_cell_ids_performance[50-50]

After:

1.19s call    test_grid.py::test_map_xy_to_cell_ids_performance[150-150]
0.56s call    test_grid.py::test_map_xy_to_cell_ids_performance[100-100]
0.13s call    test_grid.py::test_map_xy_to_cell_ids_performance[50-50]

Fixes #1347

Type of change

• New feature (non-breaking change which adds functionality)
• Optimization (back-end change that speeds up the code)
• Bug fix (non-breaking change which fixes an issue)

Key checklist

• Make sure you've run the pre-commit checks: $ pre-commit run -a
• All tests pass: $ poetry run pytest

Further checks

• Code is commented, particularly in hard-to-understand areas
• Tests added that prove fix is effective or that feature works
• Relevant documentation reviewed and updated

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.96%. Comparing base (a9c0d7e) to head (01128a3).

Additional details and impacted files

@@           Coverage Diff            @@
##           develop    #1348   +/-   ##
========================================
  Coverage    94.95%   94.96%           
========================================
  Files           71       71           
  Lines         7379     7382    +3     
========================================
+ Hits          7007     7010    +3     
  Misses         372      372           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[dalonsoa]

Looks good to me. When the time comes (hopefully from next week, or maybe March), we can use pytest-benchmark to get some performance information out of the existing testsuite with minimum changes. We will look into it when the time comes.

[dalonsoa]

Actually, if you want to give it a try, just update the existing test_map_xy_to_cell_ids test and use benchmark. It should be as easy as replacing these:

virtual_ecosystem/tests/core/test_grid.py

Lines 436 to 437 in 01128a3

	cell_map = fixture_square_grid.map_xy_to_cell_id(
	np.array(x_coord), np.array(y_coord)

by these:

        cell_map = benchmark(
            fixture_square_grid.map_xy_to_cell_id
            np.array(x_coord)
            np.array(y_coord)

And you will have the performance information for free.

Then, the new custom test won't be needed, as the performance data will be readily available.

[davidorme]

Interesting - hadn't seen pytest-benchmark, which is annoying because we've been doing something more ersatz on pyrealm 😄

It does work - but I won't put it in this PR because it's only really useful once you have a database of previous profiling that you can use to compare runs to. I agree the current test is a hack, but I'd rather leave as is until we have a profiling plan and an initial DB setup.