Use launcher and cert-tools in Testflinger job for testing DSS (New)#1947
Merged
Use launcher and cert-tools in Testflinger job for testing DSS (New)#1947
Conversation
…KBOX-1905-use-cert-tools-in-dss-tf-job
…KBOX-1905-use-cert-tools-in-dss-tf-job
I had seen this in the generic_source.yaml, but wanted to see if I could get away without it. Unfortunately, some devices do seem to need this, and we need to try if allowing degraded SSH to let the tests proceed actually have a bad impact or not.
…KBOX-1905-use-cert-tools-in-dss-tf-job
Base automatically changed from
CHECKBOX-1905-use-built-snap-in-dss-workflow
to
main
June 6, 2025 15:35
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1947 +/- ##
=======================================
Coverage 50.46% 50.46%
=======================================
Files 384 384
Lines 41111 41111
Branches 7531 7531
=======================================
Hits 20745 20745
Misses 19620 19620
Partials 746 746
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Contributor
Author
|
Had a full successful run since merging from |
Collaborator
fernando79513
left a comment
fernando79513
left a comment
There was a problem hiding this comment.
Good job here!
I've just left one small comment, but the rests looks good.
mreed8855
pushed a commit
that referenced
this pull request
Jul 30, 2025
…1947) * Temporarily reduce test matrix while developing * Trigger snap build and download atrifact * Add step to find the downloaded snap * Refactor step to build job.yaml from template * Accept built snap in testflinger job-def.yaml * Use attached pre-built snap in testflinger job * Temp disable submitting job to testflinger * Remove unused BRANCH env variable * Set path to download artifact in working directory * Make DSS and microk8s channels inputs * Add summary to each queue in matrix * Update image url for dell-precision-5680 queue * Re-enable all queues in the matrix * Add the default launcher for checkbox-dss * Use launcher in testflinger job * Remove validate-with-gpu from checkbox-dss snap * Fix some misspellings in snapcraft.yaml * Migrate testflinger job to use cert tools * Re-enable submitting job to testflinger * Fix path to built job.yaml * Remove awaiting rollout of outdated daemonset * Allow SSH to be degraded I had seen this in the generic_source.yaml, but wanted to see if I could get away without it. Unfortunately, some devices do seem to need this, and we need to try if allowing degraded SSH to let the tests proceed actually have a bad impact or not. * Wait for snap changes after each snap install * Add running snap refresh * Increase sleep time before checking rollout * Switch from using sed to envsubst to fill template * Fix typo in calling envsubst * Switch to ENV_ from REPLACE_ for template params
mreed8855
pushed a commit
that referenced
this pull request
Jul 31, 2025
…1947) * Temporarily reduce test matrix while developing * Trigger snap build and download atrifact * Add step to find the downloaded snap * Refactor step to build job.yaml from template * Accept built snap in testflinger job-def.yaml * Use attached pre-built snap in testflinger job * Temp disable submitting job to testflinger * Remove unused BRANCH env variable * Set path to download artifact in working directory * Make DSS and microk8s channels inputs * Add summary to each queue in matrix * Update image url for dell-precision-5680 queue * Re-enable all queues in the matrix * Add the default launcher for checkbox-dss * Use launcher in testflinger job * Remove validate-with-gpu from checkbox-dss snap * Fix some misspellings in snapcraft.yaml * Migrate testflinger job to use cert tools * Re-enable submitting job to testflinger * Fix path to built job.yaml * Remove awaiting rollout of outdated daemonset * Allow SSH to be degraded I had seen this in the generic_source.yaml, but wanted to see if I could get away without it. Unfortunately, some devices do seem to need this, and we need to try if allowing degraded SSH to let the tests proceed actually have a bad impact or not. * Wait for snap changes after each snap install * Add running snap refresh * Increase sleep time before checking rollout * Switch from using sed to envsubst to fill template * Fix typo in calling envsubst * Switch to ENV_ from REPLACE_ for template params
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR will be best merged after #1946.
The main goal is to migrate away from a custom implementation of the Testflinger job for testing DSS, and instead to use cert-tools in a (roughly) standard looking job (
job-def.yaml).The job spec in
tools/lab_dispatchwas the inspiration for the new implementation.The most important benefit of the new implementation is that the Testflinger agent will be the one controlling test execution, not the device itself, enabling us to do more advanced testing in the future. We also benefit from more stability using the scriptlets from cert-tools.
A default launcher for the provider has been added (
checkbox-dss.conf). It is the same one that was used in thevalidate-with-gpuscript in the Snap.validate-with-gpuscript has been removed from the Snap and the documentation now recommends using the launcher directly. This script would not have been useful in the advanced testing scenarios in the future anyway (e.g. for reboot testing).Resolved issues
Final part of CHECKBOX-1905.
Documentation
Only updates to the README of the provider.
Tests
See this run.