Fixing config parsing

[davidorme]

Description

The initial issue in #884 is that _resolve_config_paths iterates over the whole config dictionary looking for *_path entries. If it find a key, item pair where item is a list, it assumes it is a list of dictionaries. Sometimes it is (like with the variables: [{var...}, {var...}] structure) but sometimes it isn't (like with core.soil_layers = [-0.25, -1.0]).

So we can fix this by testing if item is a dictionary and only attempting to recursively resolve actual dictionaries and not random floats.

However that reveals another problem with the failing code in #884: _resolve_config_paths resolves paths relative to the ve_run execution directory to make sure that paths referred to in config files with different locations are conserved when those configs are merged. That is fine for making sure all paths are congruent during the actual run, but if the run outputs the compiled configuration, that file may very well not be in the execution directory. In fact, the example run doesn't put it there and generally you don't want it to be - it should go in some nice directory of curated configurations. So the file then contains relative links that are relative to an unknown previous execution directory.

There are potentially ways to update the paths to give relative paths from the output directory but that doesn't look well supported until Path(..., walk_up=True) in Python 3.12 and even then I'm not sure how well it works.

So, this PR changes _resolve_config_paths to convert everything to absolute paths. This is actually more robust (@dalonsoa noted issues with relative paths and drives on Windows) and works just fine - the main issue is that the resulting merged configuration output is tied to the file system on the execution machine.

In this PR I've also:

• Edited virtual_ecosystem.__init__.py to mute all of the pyrealm experimental warnings.
• Updated some legacy variable and file names that used vr_* to ve_*

Fixes #844
Fixes #808

Type of change

• New feature (non-breaking change which adds functionality)
• Optimization (back-end change that speeds up the code)
• Bug fix (non-breaking change which fixes an issue)

Key checklist

• Make sure you've run the pre-commit checks: $ pre-commit run -a
• All tests pass: $ poetry run pytest

Further checks

• Code is commented, particularly in hard-to-understand areas
• Tests added that prove fix is effective or that feature works
• Relevant documentation reviewed and updated

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.76%. Comparing base (fc8c6dc) to head (21a87f5).
⚠️ Report is 2125 commits behind head on develop.

Additional details and impacted files

@@           Coverage Diff            @@
##           develop     #845   +/-   ##
========================================
  Coverage    94.76%   94.76%           
========================================
  Files           75       75           
  Lines         5559     5563    +4     
========================================
+ Hits          5268     5272    +4     
  Misses         291      291           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[vgro]

LGTM, have you tested running ve_run from a config file?

[davidorme]

LGTM, have you tested running ve_run from a config file?

Very good point:

➜  virtual_ecosystem  ve_run --install-example /tmp
Example directory created at:
/tmp/ve_example
➜  virtual_ecosystem  cd /tmp/ve_example                             
➜  ve_example ve_run config -o out --logfile out/out.log
100%|████████████████████| 24/24 [00:17<00:00,  1.37it/s]
➜  ve_example mkdir out2
➜  ve_example ve_run out/ve_full_model_configuration.toml -o out2 --logfile out2/out.log
100%|████████████████████| 24/24 [00:17<00:00,  1.35it/s]
➜  ve_example 

[alexdewar]

LGTM!