feat: migrate all scripts to plugins and tomte CLI by DhairyaPatel7 · Pull Request #2470 · valory-xyz/open-autonomy

DhairyaPatel7 · 2026-04-14T11:18:31Z

Summary

Replaces all 8 Python scripts from scripts/ with tomte CLI commands and plugin commands. After this PR, scripts/ contains only data files (whitelist.py), git hooks, and docs.

Tomte (0.6.5)

check_copyright.py → tomte check-copyright --author "Valory AG"
check_doc_links.py → tomte check-doc-links (with URL skip flags)
freeze_dependencies.py → tomte freeze-dependencies --exclude-package open-autonomy

aea-ci-helpers (open-aea >= 2.2.0)

generate_package_list.py → aea-ci generate-pkg-list
check_ipfs_hashes_pushed.py → aea-ci check-ipfs-pushed

aea-helpers (OA plugin)

check_third_party_hashes.py → aea-helpers check-third-party-hashes
generate_contract_list.py → aea-helpers generate-contract-list
generate_api_documentation.py → aea-helpers generate-api-docs (OA-specific, scans autonomy/ not aea/)

Other changes

Bump all tomte pins from 0.6.1 to 0.6.5
Fix pre-existing copyright end years
Regenerate API docs for new aea-helpers modules
Add pip to tox deps where tomte freeze-dependencies needs it

Scripts remaining

whitelist.py, pre-add, pre-push, RELEASE_PROCESS.md, __init__.py

Test plan

🤖 Generated with Claude Code

Replace all 8 Python scripts from scripts/ with tomte CLI commands and plugin commands: Tomte (0.6.5): - check_copyright.py → tomte check-copyright --author "Valory AG" - check_doc_links.py → tomte check-doc-links (with URL skip flags) - freeze_dependencies.py → tomte freeze-dependencies --exclude-package open-autonomy aea-ci-helpers (requires open-aea >= 2.2.0): - generate_api_documentation.py → aea-ci generate-api-docs - generate_package_list.py → aea-ci generate-pkg-list - check_ipfs_hashes_pushed.py → aea-ci check-ipfs-pushed aea-helpers (OA-specific): - check_third_party_hashes.py → aea-helpers check-third-party-hashes - generate_contract_list.py → aea-helpers generate-contract-list Also: - Bump all tomte pins from 0.6.1 to 0.6.5 (tox.ini, pyproject.toml, workflow) - Fix pre-existing copyright end years - Remaining in scripts/: whitelist.py, git hooks, RELEASE_PROCESS.md Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

The aea-ci generate-api-docs command is designed for open-aea's repo structure (scans aea/), not OA's (scans autonomy/). Moved OA's generate_api_documentation.py to aea-helpers plugin instead. Also regenerated API docs for new aea-helpers modules. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Change PyPI package name from "aea-helpers" to "open-aea-helpers" for naming consistency with open-aea-ci-helpers and open-aea-dev-helpers. The CLI entry point stays "aea-helpers" (console_scripts unchanged). The Python import name stays "aea_helpers" (no code changes needed). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

codecov · 2026-04-14T12:17:27Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 92.56%. Comparing base (6d66479) to head (95d7549).
⚠️ Report is 3 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #2470   +/-   ##
=======================================
  Coverage   92.55%   92.56%           
=======================================
  Files         243      243           
  Lines       17259    17261    +2     
=======================================
+ Hits        15974    15977    +3     
+ Misses       1285     1284    -1

Flag	Coverage Δ
unittests	`92.56% <ø> (+<0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

OjusWiZard · 2026-04-14T12:46:32Z

tox.ini

 deps =
    {[deps-packages]deps}
+    ; requires open-aea-ci-helpers >= 2.2.0
+    open-aea-ci-helpers


would be good to pin them, so that we don't surprise ourselves in the future.

LOCKhart07 · 2026-04-14T12:56:47Z

tox.ini

    aea init --reset --author ci --remote --ipfs --ipfs-node "/dns/registry.autonolas.tech/tcp/443/https"
    aea packages sync
-    {toxinidir}/scripts/check_doc_links.py
+    tomte check-doc-links -n "http://pmg.csail.mit.edu/papers/osdi99.pdf" -u "https://github.com/valory-xyz/open-operator" -u "https://gateway.autonolas.tech/ipfs/<hash>"


Blocking. This drops 3 URL skips that the deleted scripts/check_doc_links.py had, and they all still appear in current docs:

http://host.docker.internal:8545 — docs/guides/define_service.md, docs/configure_service/configure_access_external_chains.md, docs/configure_service/service_configuration_file.md, docs/advanced_reference/developer_tooling/debugging_in_the_cluster.md

http://www.fipa.org/repository/ips.php3 — docs/key_concepts/abci_app_async_behaviour.md

https://github.com/valory-xyz/open-autonomy/trunk/infrastructure — docs/using_stack_deployment.md, docs/application_deployment.md

tox -e check-doc-links-hashes is also missing from the PR test plan, so this likely wasn't exercised. Add the missing entries (-n for HTTP-allowlist, -u for status-code skips) before merge.

LOCKhart07 · 2026-04-14T12:56:47Z

tox.ini

-commands = {toxinidir}/scripts/check_copyright.py --check
-allowlist_externals = {toxinidir}/scripts/check_copyright.py
+    tomte[cli]==0.6.5
+commands = tomte check-copyright --author "Valory AG" --scan-path autonomy --scan-path tests --scan-path packages/valory/agents --scan-path packages/valory/connections/abci --scan-path packages/valory/contracts --scan-path packages/valory/skills --scan-path scripts --scan-path plugins -e protocols -e dummy_packages


Non-blocking. The deleted scripts/check_copyright.py excluded _pb2.py, _pb2_grpc.py, build/, t_protocol/, t_protocol_no_ct/. The new invocation only excludes protocols and dummy_packages, but generated _pb2.py files exist outside packages/valory/protocols/ (e.g. packages/valory/connections/abci/gogoproto/gogo_pb2.py, which has no Valory header). This only passes if tomte's check-copyright auto-skips generated files — worth confirming explicitly, otherwise add the exclusions.

LOCKhart07 · 2026-04-14T12:56:48Z

plugins/aea-helpers/aea_helpers/check_third_party_hashes.py

+@click.command(name="check-third-party-hashes")
+def check_third_party_hashes() -> None:
+    """Check that third-party package hashes match the open-aea repository."""
+    root_dir = Path.cwd()


Non-blocking. Hardcoding Path.cwd() as root_dir makes the contract implicit — the command silently does the wrong thing if invoked from outside the OA repo root. Expose it as a click option (--root-dir, default .) so the dependency on cwd is documented.

LOCKhart07 · 2026-04-14T12:56:48Z

plugins/aea-helpers/setup.py


 setup(
-    name="aea-helpers",
+    name="open-aea-helpers",


Non-blocking but worth calling out in the PR body / release notes: open-aea-helpers is not yet on PyPI (verified — pip index versions open-aea-helpers returns no matches), so the next release will be the first publish under the new name. Anyone with aea-helpers pinned externally will break, and the old PyPI name becomes orphaned (typosquat surface). Confirm someone will claim/reserve it.

Version bumps: - open-aea[all], open-aea-cli-ipfs, open-aea-ledger-ethereum, open-aea-ledger-ethereum-hwi, open-aea-ledger-cosmos, open-aea-ledger-solana: 2.1.0 → 2.2.0 (tox.ini, pyproject.toml, setup.py, aea-test-autonomy/setup.py) - open-aea-ci-helpers: pinned to ==2.2.0 - Python range: ^3.10 → >=3.10,<3.15 (matches open-aea plugins) - Regenerated poetry.lock Review fixes: - Add missing URL skips for check-doc-links: host.docker.internal, fipa.org, open-autonomy/trunk/infrastructure (LOCKhart07 blocking) - Add --root-dir option to check-third-party-hashes (LOCKhart07) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

- check-api-docs/generate-api-documentation: install aea-helpers via --no-deps in commands to avoid circular dep resolution (aea-helpers depends on open-autonomy from PyPI which pins open-aea==2.1.0) - Regenerate API docs for check_third_party_hashes (--root-dir added) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

DavidMinarsch · 2026-04-14T13:47:01Z

plugins/aea-helpers/aea_helpers/check_third_party_hashes.py

 import sys
 from pathlib import Path

+import click


why isn't this in open-aea? You can imagine agents building purely on open-aea wanting this

This script checks third-party packages against the open-aea repo's hashes. We put it in OA's aea-helpers initially, but any downstream repo building on open-aea would want the same check. Should I migrate it to open-aea-ci-helpers?

yes please - migrate there and re-release for use here

DavidMinarsch · 2026-04-14T13:48:00Z

plugins/aea-helpers/aea_helpers/cli.py

@@ -24,7 +24,10 @@
 from aea_helpers.check_binary import check_binary


So we're extending aea-helpers?

Yes — aea-helpers already existed in OA with commands like run-agent, run-service, make-release, config-replace, check-binary, etc. We added 3 commands that were previously in scripts/: check-third-party-hashes, generate-contract-list, and generate-api-docs. The first two are arguably OA-specific (contract list generation, third-party hash checking against open-aea). The API doc generator is OA-specific because it scans autonomy/ rather than aea/ — but per your other comment, that could be made configurable upstream instead.

third-party hash checking should be generic against any upstream repo (or list of repos)

let's make api thing configurable upstream so we don't need to maintain two versions

generate contract list can stay here. but what is it's purpose?

DavidMinarsch · 2026-04-14T13:48:19Z

plugins/aea-helpers/aea_helpers/generate_api_docs.py

-    )
-    arguments = parser.parse_args()
-
+def main(check_clean: bool = False) -> None:


Why wasn't this backported to open-aea?

The open-aea aea-ci generate-api-docs command is hardcoded to scan aea/ and plugins/ with open-aea's directory structure. OA's version scans autonomy/ and packages/valory/. We couldn't reuse the open-aea version as-is, so we kept OA's version in aea-helpers.

The right fix would be to make aea-ci generate-api-docs configurable (e.g. --source-dir, --packages-dir flags) so both repos can use the same command. Should I create a follow-up PR on open-aea for this?

DavidMinarsch · 2026-04-14T13:49:45Z

plugins/aea-helpers/setup.py


 setup(
-    name="aea-helpers",
+    name="open-aea-helpers",


why aea-helpers in autonomy?

The other OA plugin follows the same pattern: open-aea-test-autonomy — also an open-aea-* name living in the OA repo. We're renaming this one from aea-helpers to open-aea-helpers on PyPI in this PR to match that convention. The commands it provides (run-agent, run-service, make-release, config-replace, check-binary) are OA-specific and depend on autonomy.

DavidMinarsch · 2026-04-14T13:51:17Z

pyproject.toml

@@ -7,15 +7,15 @@ license = "Apache-2.0 license"
 packages = [{include = "autonomy"}, {include = "packages"}]



why are open-aea-ci-helpers and open-aea-dev-helpers not used here?

They're not runtime dependencies of open-autonomy — they're CI/dev tools. They're declared in tox.ini as deps for the specific tox envs that need them (e.g. open-aea-ci-helpers==2.2.0 in check-doc-links-hashes). Adding them to pyproject.toml would make them install-time dependencies for every user of open-autonomy, which isn't right since most users don't need CI tooling.

Should we add them under [tool.poetry.group.dev.dependencies] instead so they're available during development but not for end users?

yes dev dependencies

LOCKhart07

Re-reviewed commits aebd768 and eff13d0b since my previous review.

Prior items — resolved:

Missing URL skips (host.docker.internal, fipa.org, open-autonomy/trunk/infrastructure) added to check-doc-links-hashes (blocker).
--root-dir click option added to check-third-party-hashes.
open-aea-ci-helpers now pinned to ==2.2.0.

Minor, non-blocking polish (optional):

tox.ini check-third-party-hashes still installs aea-helpers via deps = {toxinidir}/plugins/aea-helpers, while the other new envs (check-api-docs, generate-api-documentation, check-doc-links-hashes, lock-packages) switched to python -m pip install --no-deps ... in commands to avoid the circular open-autonomy resolution. Lower risk here because this env omits {[deps-packages]deps}, but aligning for consistency would be nice.
http://host.docker.internal:8545 appears in both -n and -u for tomte check-doc-links — intentional (non-HTTPS skip + unreachable status skip), just worth noting so it isn't "simplified" later.
Still outstanding from prior pass (non-blocking): generated _pb2.py files outside packages/valory/protocols/ and the open-aea-helpers PyPI name reservation.

LGTM.

open-aea-cli-ipfs 2.2.0 replaced the underlying ipfshttpclient.Client with an in-tree IPFSHTTPClient that does not expose add_str. Switch to add_bytes which provides equivalent behavior. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Matches the other new tox envs (check-api-docs, generate-api-documentation, check-doc-links-hashes, lock-packages) which install aea-helpers via `python -m pip install --no-deps` in commands to avoid pulling the PyPI open-autonomy via aea-helpers' install_requires. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

DhairyaPatel7 · 2026-04-14T14:51:05Z

Thanks @LOCKhart07. Aligned check-third-party-hashes with the other new envs in 26d57a6 — now uses pip install --no-deps in commands for consistency.

Leaving the host.docker.internal:8545 entries in both -n and -u as-is per your note (intentional: non-HTTPS skip + unreachable status skip).

DhairyaPatel7 and others added 4 commits April 13, 2026 17:07

chore: regenerate poetry.lock for tomte 0.6.5 bump

95d7549

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

DhairyaPatel7 marked this pull request as ready for review April 14, 2026 12:30

LOCKhart07 self-requested a review April 14, 2026 12:32

OjusWiZard previously approved these changes Apr 14, 2026

View reviewed changes

LOCKhart07 reviewed Apr 14, 2026

View reviewed changes

DhairyaPatel7 dismissed OjusWiZard’s stale review via aebd768 April 14, 2026 13:25

DhairyaPatel7 requested review from LOCKhart07 and OjusWiZard April 14, 2026 13:48

DavidMinarsch reviewed Apr 14, 2026

View reviewed changes

LOCKhart07 previously approved these changes Apr 14, 2026

View reviewed changes

DhairyaPatel7 and others added 2 commits April 14, 2026 20:18

DhairyaPatel7 dismissed LOCKhart07’s stale review via 26d57a6 April 14, 2026 14:50

DhairyaPatel7 mentioned this pull request Apr 14, 2026

feat(aea-ci): generic check-third-party-hashes + configurable generate-api-docs valory-xyz/open-aea#876

Open

7 tasks

		@@ -24,7 +24,10 @@
		from aea_helpers.check_binary import check_binary

		@@ -7,15 +7,15 @@ license = "Apache-2.0 license"
		packages = [{include = "autonomy"}, {include = "packages"}]

Conversation

DhairyaPatel7 commented Apr 14, 2026

Summary

Tomte (0.6.5)

aea-ci-helpers (open-aea >= 2.2.0)

aea-helpers (OA plugin)

Other changes

Scripts remaining

Test plan

Uh oh!

codecov bot commented Apr 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

LOCKhart07 left a comment

Choose a reason for hiding this comment

Uh oh!

DhairyaPatel7 commented Apr 14, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

codecov bot commented Apr 14, 2026 •

edited

Loading