ci(l1): pin assertoor playbook URLs to pre-refactoring commit

[azteca1998]

Summary

Pin all assertoor playbook URL references from refs/heads/master to a specific commit (c35b5c3a5f) — the last master commit before ethpandaops/assertoor PR #137 (major refactoring) was merged.

Motivation

Since Feb 19 ~17:26 UTC, the assertoor upstream merged a major refactoring that introduced breaking changes:

• stopChildOnResult removed (tasks now self-complete)
• failTaskCount / succeedTaskCount replaced with failureThreshold / successThreshold
• Transaction submission engine replaced with spamoor

Since our configs fetched playbooks from refs/heads/master, every CI run after the merge fails the Blob & Stability Check — including runs with zero code changes.

Changes

Pinned 4 playbook URLs across 3 files:

• network_params_blob.yaml — blob-transactions-test.yaml
• network_params_tx.yaml — eoa-transactions-test.yaml
• network_params_ethrex_multiple_cl.yaml — blob-transactions-test.yaml + eoa-transactions-test.yaml

Note

Pinning the playbook URLs may not be sufficient on its own — if Kurtosis is also pulling the latest assertoor Docker image (with the new runtime), the old playbooks may still be interpreted differently. We may also need to pin the assertoor image version in the network params configs.

[github-actions]

🤖 Kimi Code Review

Review of PR #6238

Summary: The PR pins Assertoor test URLs to a specific commit hash instead of using the mutable master branch. This is a good practice for CI reproducibility.

Issues found:

1. Consistency concern: The PR only pins the Assertoor test URLs but leaves the ethrex-specific test URL unpinned:

   • .github/config/assertoor/network_params_blob.yaml line 31
   • .github/config/assertoor/network_params_ethrex_multiple_cl.yaml line 32

   Consider pinning the ethrex test URL as well for consistency:

   - https://raw.githubusercontent.com/lambdaclass/ethrex/<commit-hash>/.github/config/assertoor/el-stability-check.yaml

2. Future maintenance: While pinning improves reproducibility, it may require manual updates when Assertoor tests are updated. Consider adding a comment in the YAML files indicating when these URLs were last updated and how to check for newer versions.

Security/Reliability: The change improves CI reliability by preventing unexpected test failures from upstream changes. The commit hash c35b5c3a5fd543bcc843ee0e0af1e6255ca48dbd appears to be a valid Assertoor commit from the ethpandaops repository.

Overall: The change is appropriate and improves CI stability. Just address the consistency issue mentioned in point 1.

---

Automated review by Kimi (Moonshot AI) · custom prompt

[github-actions]

🤖 Claude Code Review

Review: ci(l1): pin assertoor playbook URLs to pre-refactoring commit

Type: CI configuration / dependency pinning
Scope: 3 YAML files, 4 URL changes — no production code affected

---

Summary

The change is correct and necessary. Replacing floating refs/heads/master references with the immutable commit SHA c35b5c3a5fd543bcc843ee0e0af1e6255ca48dbd restores CI stability after the breaking upstream refactor.

---

Observations

Correctness

All 4 URLs are consistently updated to the same commit SHA across all three files. No references were missed. The SHA points to the last pre-refactoring master commit, which is the right anchor point.

The el-stability-check.yaml reference is still floating

In all three files, the ethrex-owned playbook continues to use refs/heads/main:

- https://raw.githubusercontent.com/lambdaclass/ethrex/refs/heads/main/.github/config/assertoor/el-stability-check.yaml

This is intentional (it's under this repo's control), but worth noting explicitly: any breaking change to that file on main will immediately affect all CI runs referencing it. This is a pre-existing concern, not introduced by this PR.

Risk noted in PR description

The author correctly identifies that pinning playbook URLs may be insufficient if Kurtosis pulls the latest assertoor Docker image (which would use the new runtime). This is a real risk — if the assertoor runtime version is not pinned, the old playbooks may still fail or behave unexpectedly with the new engine. A follow-up task to pin the assertoor image version in the network params configs is advisable.

---

Verdict

The change is correct, minimal, and well-motivated. The commit SHA is immutable, making this a proper fix rather than a workaround. Approve with the suggestion to follow up on pinning the assertoor Docker image version.

---

Automated review by Claude (Anthropic) · custom prompt

[github-actions]

🤖 Codex Code Review

Changes are config-only: three YAML files now pin Assertoor playbooks to a specific commit hash. No Rust/EVM logic touched.

Review

• Good: pinning to a commit improves reproducibility and reduces supply‑chain drift. This is a net positive for stability and security.

Suggestions

1. Consider adding a brief comment noting why the commit is pinned and how/when it should be updated, to avoid it becoming stale. Example in .github/config/assertoor/network_params_blob.yaml near the URL line.
2. If these playbooks are expected to change with upstream fixes, consider using a tagged release instead of a raw commit hash, or add a periodic update task (to avoid silently missing important test changes). This applies to:
   • .github/config/assertoor/network_params_blob.yaml
   • .github/config/assertoor/network_params_ethrex_multiple_cl.yaml
   • .github/config/assertoor/network_params_tx.yaml

No correctness, security, or performance issues in the codebase itself since this PR only alters test config URLs.

---

Automated review by OpenAI Codex · custom prompt

[github-actions]

🤖 Kimi Code Review

Review Summary

This PR pins the assertoor image and test URLs to specific versions rather than using the master branch. This is a good practice for CI stability and reproducibility.

Issues Found

1. Pinning to a specific commit may become stale (lines 27-30 in network_params_blob.yaml, lines 31-34 in network_params_ethrex_multiple_cl.yaml, and line 37 in network_params_tx.yaml)

   • The commit hash c35b5c3a5fd543bcc843ee0e0af1e6255ca48dbd is hardcoded. While this provides stability, it may become outdated as assertoor evolves.
   • Consider adding a comment near these URLs indicating when this commit was current, or consider using a tagged release if available.

2. Inconsistent assertoor image version (lines 27, 29, 31)

   • The image is pinned to v0.0.17 but the test URLs point to a specific commit. Verify that this commit corresponds to the v0.0.17 release to ensure compatibility.

3. Missing validation of external dependencies

   • These URLs are external dependencies that could change or become unavailable. Consider:
     • Adding a CI check to validate these URLs are accessible
     • Mirroring critical test files in the ethrex repository for better control

Recommendations

1. Add a comment indicating the date when these pinned versions were chosen
2. Consider using a tagged release URL instead of a commit hash if assertoor provides stable releases
3. Document the process for updating these pinned versions in the future

The changes are minimal and focused, following good practices for CI configuration stability.

---

Automated review by Kimi (Moonshot AI) · custom prompt

[github-actions]

🤖 Codex Code Review

Review Summary

Looks good overall. This PR pins Assertoor playbooks to a specific commit and explicitly sets the Assertoor image tag, which improves reproducibility and reduces supply-chain drift risk.

Findings

1. Supply-chain hardening: You pinned the playbooks to a commit hash, which is great. However, the Docker image is still referenced by mutable tag v0.0.17. Consider pinning to a digest to fully lock the image and prevent tag retargeting.
   • /.github/config/assertoor/network_params_blob.yaml:27
   • /.github/config/assertoor/network_params_ethrex_multiple_cl.yaml:32
   • /.github/config/assertoor/network_params_tx.yaml:33

Notes

• No correctness, EVM, consensus, or performance concerns found; only CI config updates.

If you want, I can propose the exact digest pin change.

---

Automated review by OpenAI Codex · custom prompt

[github-actions]

🤖 Claude Code Review

Review: ci(l1): pin assertoor playbook URLs to pre-refactoring commit

Type: CI configuration / dependency pinning
Scope: 3 YAML files, 7 line changes — no production code affected

---

Summary

The change is correct and necessary. All 4 upstream playbook URLs are consistently updated to the immutable commit SHA c35b5c3a5fd543bcc843ee0e0af1e6255ca48dbd, and the assertoor Docker image is pinned to v0.0.17 across all three configs. This directly addresses the CI breakage caused by the upstream refactor.

---

Observations

Correctness

• All 4 external assertoor playbook URLs are consistently changed to the same SHA. No references were missed.
• The image ethpandaops/assertoor:v0.0.17 is added to all three files. The PR description's "Note" identified this as a potential gap, and the author correctly included it — good self-review.
• The ethrex-internal el-stability-check.yaml reference remains on refs/heads/main. This is intentional and correct since the project controls that ref.
• ethereum_package_branch in the workflow is already pinned to a commit SHA (82e5a7178138d892c0c31c3839c89d53ffd42d9a), so that dependency is already stable.

Scope of impact

network_params_ethrex_multiple_cl.yaml is currently commented out in the workflow matrix (line 172–174 of pr-main_l1.yaml, marked "Flaky, reenable when fixed"), so those two URL changes are proactive rather than immediately load-bearing. That is fine — better to have it correct before re-enabling.

One item worth confirming

The image tag v0.0.17 should be verified as a pre-refactoring release. The PR claims assertoor PR #137 merged on Feb 19 — if v0.0.17 was tagged after that merge, tests could still run against the incompatible runtime. Checking the assertoor releases page to confirm v0.0.17 predates the refactor would close this loop.

kurtosis-assertoor-github-action@v1 (out of scope)

The workflow uses a floating major-version tag for this action, which could theoretically introduce future breakage from the same category of problem. This is outside the scope of the current fix but could be a follow-up hardening item.

---

Verdict

Approved as-is if v0.0.17 is confirmed to predate the upstream refactor. The change is minimal, correct, and consistently applied.

---

Automated review by Claude (Anthropic) · custom prompt

[greptile-apps]

Greptile Summary

This PR addresses broken CI builds by pinning both the assertoor Docker image to v0.0.17 and all playbook URLs to commit c35b5c3a5f, preventing failures caused by assertoor's major refactoring merged on Feb 19.

Changes:

• Pinned assertoor_params.image to ethpandaops/assertoor:v0.0.17 (previously used implicit latest)
• Updated 4 playbook URLs from refs/heads/master to specific commit c35b5c3a5fd543bcc843ee0e0af1e6255ca48dbd
• Affects blob transaction tests, EOA transaction tests across all network configurations

Impact:

• Ensures CI stability while upstream assertoor settles post-refactoring
• Prevents breaking changes from propagating (removed stopChildOnResult, replaced counters, new spamoor engine)
• All 3 configuration files now use consistent versioning strategy

Confidence Score: 5/5

• This PR is safe to merge with minimal risk - it's a defensive pinning strategy to stabilize CI
• The changes are purely configuration updates that pin dependencies to known-working versions. The PR correctly addresses both the Docker image (v0.0.17) and playbook URLs (commit c35b5c3a5f), which resolves the issue mentioned in the PR description. The author also properly updated the commit message to reflect that the image pinning was included (not just playbooks). All changes are consistent across the 3 config files.
• No files require special attention

Important Files Changed

Filename	Overview
.github/config/assertoor/network_params_blob.yaml	Pinned assertoor image to v0.0.17 and blob-transactions-test playbook URL to pre-refactoring commit c35b5c3a5f
.github/config/assertoor/network_params_tx.yaml	Pinned assertoor image to v0.0.17 and eoa-transactions-test playbook URL to pre-refactoring commit c35b5c3a5f
.github/config/assertoor/network_params_ethrex_multiple_cl.yaml	Pinned assertoor image to v0.0.17 and both blob-transactions-test and eoa-transactions-test playbook URLs to pre-refactoring commit c35b5c3a5f

_{Last reviewed commit: d7b408a}