Skip to content

feat(utils)!: store vcs info in metadata#268

Merged
anaprietonem merged 8 commits intoecmwf:mainfrom
ways:feat(utils)!-store-vcs-info-in-metadata
Feb 9, 2026
Merged

feat(utils)!: store vcs info in metadata#268
anaprietonem merged 8 commits intoecmwf:mainfrom
ways:feat(utils)!-store-vcs-info-in-metadata

Conversation

@ways
Copy link
Contributor

@ways ways commented Feb 2, 2026

Description

This closes #130 by adding more metadata to checkpoints.

What problem does this change solve?

Add more info to checkpoint metadata about modules used in training. The URL, subdirectory and more.

What issue or task does this change relate to?

Closes #130

Additional notes

If someone is parsing the metadata already, this could be a breaking change, but I don't think it is.

Old metadata:

"anemoi.models": "0.12.0",
New metadata:

            "anemoi.models": {
                "source": {
                    "subdirectory": "models",
                    "url": "https://github.com/ways/anemoi-core.git",
                    "vcs_info": {
                        "commit_id": "5abd0c8f3f05cb949800c8bb4878a29312c0e0aa",
                        "requested_revision": "models-0.12.0",
                        "vcs": "git"
                    }
                },
                "version": "0.12.0"
            },

By opening this pull request, I affirm that all authors agree to the Contributor License Agreement.

@ways
Copy link
Contributor Author

ways commented Feb 2, 2026

Tested like this:

deps =
    anemoi-training@git+https://github.com/ecmwf/anemoi-core.git@training-0.9.0\#egg=pkg&subdirectory=training
    anemoi-graphs@git+https://github.com/ecmwf/anemoi-core.git@graphs-0.8.4\#egg=pkg&subdirectory=graphs
    anemoi-models@git+https://github.com/ways/anemoi-core.git@models-0.12.0\#egg=pkg&subdirectory=models
    trimesh
  • change commands_pre = to:
commands_pre =
    # Copy config file to the temp directory.
    cp -vf {toxinidir}/config/tox_test_train.yaml {envtmpdir}/
    # Link training data
    ln -sf {toxworkdir}/bris_random_data.zarr {envtmpdir}/
    pip install 'anemoi-utils@git+https://github.com/ways/anemoi-utils.git@larstest'
  • run test uv tool run tox -e training
  • Look at checkpoint produced: uv tool run anemoi-utils metadata --dump .tox/training/tmp/training-output/checkpoint/*/inference-last.ckpt
  • Optional: compare with checkpoint produced with existing anemoi-utils, anemoi-utils == v0.4.43

@HCookie HCookie changed the title Feat(utils)! store vcs info in metadata feat(utils)! store vcs info in metadata Feb 2, 2026
@HCookie HCookie self-requested a review February 2, 2026 13:59
@HCookie HCookie changed the title feat(utils)! store vcs info in metadata feat(utils)!: store vcs info in metadata Feb 2, 2026
Co-authored-by: Harrison Cook <Harrison.cook@ecmwf.int>
@HCookie HCookie moved this from To be triaged to Under Review in Anemoi-dev Feb 3, 2026
@HCookie HCookie requested a review from anaprietonem February 3, 2026 09:37
@anaprietonem
Copy link
Collaborator

Thanks @ways, I left some comments (mostly to check my understanding). Regarding this being a breaking change - @HCookie could you check if this could lead to any issues in the catalogue? (just thinking that might be a place where we read/use the provenance info)

@anaprietonem anaprietonem self-requested a review February 9, 2026 17:48
@github-project-automation github-project-automation bot moved this from Under Review to For merging in Anemoi-dev Feb 9, 2026
@anaprietonem anaprietonem merged commit b28a62b into ecmwf:main Feb 9, 2026
12 checks passed
@github-project-automation github-project-automation bot moved this from For merging to Done in Anemoi-dev Feb 9, 2026
anaprietonem pushed a commit that referenced this pull request Feb 10, 2026
🤖 Automated Release PR

This PR was created by `release-please` to prepare the next release.
Once merged:

1. A new version tag will be created
2. A GitHub release will be published
3. The changelog will be updated

Changes to be included in the next release:
---


## [0.5.0](0.4.43...0.5.0)
(2026-02-09)


### ⚠ BREAKING CHANGES

* **utils:** store vcs info in metadata
([#268](#268))
* Update pyproject remove python 3.10
([#259](#259))

### Features

* Update pyproject remove python 3.10
([#259](#259))
([93a1c61](93a1c61))
* **utils:** Store vcs info in metadata
([#268](#268))
([b28a62b](b28a62b))


### Bug Fixes

* Add Filelock to cache
([#262](#262))
([20664f0](20664f0))
* Add provenance to tests deps
([#265](#265))
([942936b](942936b))

---
> [!IMPORTANT]
> Please do not change the PR title, manifest file, or any other
automatically generated content in this PR unless you understand the
implications. Changes here can break the release process.
> ⚠️ Merging this PR will:
> - Create a new release
> - Trigger deployment pipelines
> - Update package versions

 **Before merging:**
 - Ensure all tests pass
 - Review the changelog carefully
 - Get required approvals

[Release-please
documentation](https://github.com/googleapis/release-please)
HCookie added a commit to ecmwf/anemoi-inference that referenced this pull request Feb 18, 2026
## Description
ecmwf/anemoi-utils#268 broke provenance parsing
in inference comparing versions
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

Store more information regarding modules in checkpoint

3 participants