Update release workflow

[wileyj]

Note: relies on composite PR stacks-network/actions#91 due to the changes in registry.

This change refactors the release workflow as follows:

• changes the binary build process from using docker to building on the VM directly
• docker images use the binary artifact archive, rather than assets from a published github release
• multiple approval gates are removed in favor of a single one to start the build
• docker images publish to ghcr (this will require some additional setup after merge)
• workflows and inputs have been simplified to reduce duplication and remove variables that aren't needed
• docker tags are simplified: stacks-blockchain is no longer tagged by default for ghcr
• workflow filenames have been renamed slightly to better reflect what they do. release-*.yml etc.
• building the ad-hoc docker image using the same artifact method, but is coped only for glibc x64/arm64
• all archives and images are attested, with the docker images also being signed with cosign (https://github.com/sigstore/cosign)
• dockerfiles used to create the images are re-organized based on target and purpose
• macos builds are building on macos runners, with the only cross-compiliation being for cpu/c stdlib, which only involves updating linkers and other build-time flags.
• removed the cleanup step when setting up docker. via composite recent runners don't benefit from this, and the storage is more than adequate now to build stacks-core.

Overall, the performance is much better than using docker - what was taking ~30 minutes (discounting timeouts and failures) is now reliably down to ~12 minutes per build (macos being the outlier, since it was tested on a 7GB machine with the release-lite profile).

a full test release workflow: https://github.com/wileyj/stacks-core/actions/runs/19481419227

and a specific run of just the release workflow to show the expected timing of the changes proposed here: https://github.com/wileyj/stacks-core/actions/runs/19475022815

and the ghcr images produced-
signer: https://github.com/wileyj/stacks-core/pkgs/container/stacks-signer/versions?filters%5Bversion_type%5D=tagged
stacks-core: https://github.com/wileyj/stacks-core/pkgs/container/stacks-core/versions?filters%5Bversion_type%5D=tagged

published test release: https://github.com/wileyj/stacks-core/releases/tag/4.0.0.0.6
published singer test release: https://github.com/wileyj/stacks-core/releases/tag/signer-4.0.0.0.6.0

finally - the builds are set to use the release profile, which willl require at least the macos-latest-large runner, with the linux runners currently set to use the default ubuntu-latest (this may be changed, but the timing on default runners was acceptable).

the binaries/images need more verification, but initial testing of produced artifacts looks good.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 76.86%. Comparing base (39a0bbb) to head (573990b).
⚠️ Report is 50 commits behind head on develop.

Additional details and impacted files

@@             Coverage Diff              @@
##           develop    #6697       +/-   ##
============================================
+ Coverage    52.86%   76.86%   +24.00%     
============================================
  Files          578      579        +1     
  Lines       358379   360068     +1689     
============================================
+ Hits        189458   276772    +87314     
+ Misses      168921    83296    -85625     

see 445 files with indirect coverage changes

---

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 39a0bbb...573990b. Read the comment docs.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[wileyj]

moving to ghcr - one thing i don't like is how all the layers (tagged or not) are visible on the packages page (see link above).
i have a feeling it's a UI diff compared to something like dockerhub, but it may be confusing for users looking for what is the latest tagged release.

a few things to follow up on:

• schedule a nightly build from develop. i think this should be easier than expected with the move to using artifacts built natively. a simple conditional tag if it's run via cron should be all that's needed (along with hardcoding to build nightly from develop, compared to any desired branch when run manually)
• github release is created in draft mode, but i think we could also pre-populate most of the release message from the changelog(s) using a tool like https://github.com/sean0x42/markdown-extract

[wileyj]

there is also a question of if we should be re-running all the same tests (which should have been passing prior to release in order for the changes to merge), i.e. bitcoin tests.
on the other hand, there could be a case where a hotfix release breaks an existing test, and it woudn't be caught until the release ci.

[wileyj]

update - one thing i notice that i'm not sure about (but shoudln't be a blocker to merge) is that the artifacts are being attested 2x for the same archive. this may be a result of the the archive name being identical for stacks-core and the stacks-signer release.

Edit: it is indeed because of the archive file naming being a duplicate for the release. so stacks-core & stacks-signer both having the same archive name like linux-glibc-x64.zip will produce 2 quite similar attestations, even though the context of that archive is different.

[fabrizio-stacks]

update - one thing i notice that i'm not sure about (but shoudln't be a blocker to merge) is that the artifacts are being attested 2x for the same archive. this may be a result of the the archive name being identical for stacks-core and the stacks-signer release.

Edit: it is indeed because of the archive file naming being a duplicate for the release. so stacks-core & stacks-signer both having the same archive name like linux-glibc-x64.zip will produce 2 quite similar attestations, even though the context of that archive is different.

if you have a look to the sbtc repo I solved this problem!

[federico-stacks]

there is also a question of if we should be re-running all the same tests (which should have been passing prior to release in order for the changes to merge), i.e. bitcoin tests. on the other hand, there could be a case where a hotfix release breaks an existing test, and it woudn't be caught until the release ci.

just to clear my understanding: does it means we don't run the "ci" tests for the hotfix release?

[wileyj]

there is also a question of if we should be re-running all the same tests (which should have been passing prior to release in order for the changes to merge), i.e. bitcoin tests. on the other hand, there could be a case where a hotfix release breaks an existing test, and it woudn't be caught until the release ci.

just to clear my understanding: does it means we don't run the "ci" tests for the hotfix release?

no - just a note that we currently re-run all tests that are run as part of the PR process when doing a release, and perhaps there's an option to not do that in the future (with the expectation that tests would have already run. there are some caveats today, but i think it's possible).

for a hotfix release, we'd still need to run all the normal required steps. but, i think there may be a case to be made where we can ensure all tests are passing before a release workflow is triggered - not addressed in this PR, but something to think about for a future change.
for hotfixes, the tests should likewise already be run by the time the workflow is triggered. it's a good question though, i think it's worth the effort to determine when and how we run tests.

[aaronb-stacks]

This LGTM, just a comment on some of the dockerfiles' CMDs

[jacinta-stacks]

LGTM.

[francesco-stacks]

should the variable docker_tag be uppercase? if [[ "${DOCKER_TAG}" =~ -rc[0-9]*$ ]]; then

[wileyj]

i treated this as a global/local syntax, where var_docker_tag_rc (and others only used in the step) are lowercase, and the global vars are set to uppercase. this may not be the same in older workflows though, but in this PR i tried to treat them this way consistently.

there's no clear guideline on what's acceptable in workflows that i could find, but if you think uppercase makes more sense i can adjust and push.

[francesco-stacks]

Ah, I think we're talking about different things! I wasn't referring to the naming convention. The issue is that in that step the script is reading docker_tag, but the value exported via GITHUB_ENV is uppercase DOCKER_TAG. So i was wondering if ${docker_tag} is empty and the RC check always evaluates to false?