Swap animal functional group configuration to CSV

[davidorme]

Description

This PR aligns the configuration of animal and plant functional groups. Both now load functional group definitions from a more user friendly and readable CSV file, which is configured from a single path setting in their config schema. The option to load them from an array of tables in TOML has been removed.

• The import_functional_groups function has been updated to do wider error checking.
• The schema has been updated to remove the [[animal.functional_groups]] syntax and replace it with:

[animals]
functional_group_definitions_path = "/path/to/data.csv"

• The AnimalModel.from_config method now calls import_functional_groups.
• Tests and test configurations updated.
• Example data and example data docs updated.

@TaranRallings - there are now two very similar definition files kicking around:

• One in tests/models/animals/data - used in animals testing
• One in tests/data - fed into the main config fixtures.

I suspect we could probably remove one, but wasn't about to try and guess which 😄

Fixes #1067

Type of change

• New feature (non-breaking change which adds functionality)
• Optimization (back-end change that speeds up the code)
• Bug fix (non-breaking change which fixes an issue)

Key checklist

• Make sure you've run the pre-commit checks: $ pre-commit run -a
• All tests pass: $ poetry run pytest

Further checks

• Code is commented, particularly in hard-to-understand areas
• Tests added that prove fix is effective or that feature works
• Relevant documentation reviewed and updated

[codecov-commenter]

Codecov Report

❌ Patch coverage is 54.16667% with 11 lines in your changes missing coverage. Please review.
✅ Project coverage is 95.24%. Comparing base (6749528) to head (e7d8777).
⚠️ Report is 1137 commits behind head on develop.

Files with missing lines	Patch %	Lines
...irtual_ecosystem/models/animal/functional_group.py	52.94%	8 Missing ⚠️
virtual_ecosystem/models/animal/animal_model.py	57.14%	3 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #1068      +/-   ##
===========================================
- Coverage    95.38%   95.24%   -0.15%     
===========================================
  Files           79       79              
  Lines         6876     6891      +15     
===========================================
+ Hits          6559     6563       +4     
- Misses         317      328      +11     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[TaranRallings]

This is a graceful solution! I expected it to be far more finicky but this is way better and now I don't have to update 45 different files every time I change the functional group definition. LGTM!

[davidorme]

Particularly now that the animal_fixture_config is synced (apart from grid size) with the fixture_config 😄

[nickwctan]

Is this part already updated on the readthedocs.io? I could not see the addition of "Animal functional group definitions" as of now.

[davidorme]

It is there but it is pretty brief: https://virtual-ecosystem.readthedocs.io/en/latest/using_the_ve/example_data.html#animal-functional-group-definitions

We need to link that to a more detailed explanation in the animal model implementation/configuration docs.

[nickwctan]

I could've sworn it was not there when I looked at it yesterday 😆. Thanks!

Thanks for streamlining configured paths. I might have more comments later on.

We need to link that to a more detailed explanation in the animal model implementation/configuration docs.

You can find more detailed explanation of functional group attributes in docs/source/virtual_ecosystem/theory/animals/animal_theory.md which we can link to. However we both felt like it fits much better in docs/source/virtual_ecosystem/implementation/animal_implementation.md, so this is in the list of things I need to do.

[nickwctan]

I could've sworn it was not there when I looked at it yesterday😆. Thanks!

Ok I realised I probably was not on the latest version, but instead on the 1050-revamp-the-existing-using-the-ve-documentation version. My bad!

[nickwctan]

Just got to go through this PR! This is great progress, thanks @davidorme! A much user friendly approach. So now only a CSV defining AFG is necessary to add as one of the data file while the TOML config file only has a relative path file that imports the CSV.

I have a few questions on these for clarification:

1. Would the TOML now be comma-delimited instead of array of tables? If the former is true, can the model take in information in a comma-delimited format? and if yes, how?

2. What does the config fixture do?

Thanks!

[davidorme]

Just got to go through this PR! This is great progress, thanks @davidorme! A much user friendly approach. So now only a CSV defining AFG is necessary to add as one of the data file while the TOML config file only has a relative path file that imports the CSV.

Just to be really pick about data files (and we need to make this distinction clearer in the docs). Basically:

• Array data sets (like elevation, soil PH, temperature per grid cell through time etc) are loaded by providing a configuration setting like this:

toml [[core.data.variable]] file_path = "/path/to/netcdf_data.nc" var_name = "elevation"

• Animal functional groups (and a couple of similar plant files) now side step this system and are loaded through model specific file paths:

[animals]
functional_group_definitions_path = "/path/to/data.csv"

I have a few questions on these for clarification:

1. Would the TOML now be comma-delimited instead of array of tables? If the former is true, can the model take in information in a comma-delimited format? and if yes, how?

Nope - the TOML (as above) provides the path of a CSV file to load from. This PR basically moves all of the data that was crammed into TOML format to an entirely separate file in an easier format, and the animal config then just points to that file.

2. What does the config fixture do?

A fixture is a feature of the pytest model testing framework. Essentially it's a function that returns something that can be re-used by multiple tests. It's not something that users interact with.

Thanks!