Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 44 additions & 0 deletions docs/.sphinx/_static/checkbox-doc.css
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
.mermaid {
display: flex;
justify-content: center;
padding-bottom: 0.5rem;
font-family: Ubuntu, sans-serif;
}

.mermaid .node rect,
.mermaid .node circle,
.mermaid .node ellipse,
.mermaid .node polygon,
.mermaid .node path,
.mermaid rect.actor,
.mermaid .labelBox polygon
{
fill: var(--color-background-item) !important;
stroke: var(--color-background-border) !important;
}


.mermaid .label text,
.mermaid span,
.mermaid p
{
color: var(--color-content-foreground) !important;
}

.mermaid .actor tspan,
.mermaid text.messageText
{
fill: var(--color-content-foreground) !important;
}

.mermaid .flowchart-link,
.mermaid line
{
stroke: var(--color-content-foreground) !important;
}

.mermaid .marker
{
fill: var(--color-content-foreground) !important;
stroke: var(--color-content-foreground) !important;
}
1 change: 1 addition & 0 deletions docs/.sphinx/spellingcheck.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -24,3 +24,4 @@ matrix:
- div.relatedlinks
- div.visually-hidden
- img
- div.mermaid
3 changes: 3 additions & 0 deletions docs/.sphinx/wordslist.txt
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
AMD
amd
APIs
autoconf
backend
Expand Down Expand Up @@ -94,6 +96,7 @@ sys
systemd
Systemd
templating
Testflinger
testplan
tmp
TODO
Expand Down
1 change: 1 addition & 0 deletions docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -153,6 +153,7 @@
html_css_files = [
'custom.css',
'github_issue_links.css',
'checkbox-doc.css'
]
html_js_files = [
'github_issue_links.js',
Expand Down
1 change: 1 addition & 0 deletions docs/explanation/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,4 @@ Explanation
understanding
remote
configs
release_process/canary
Original file line number Diff line number Diff line change
Expand Up @@ -4,55 +4,52 @@ Canary Testing for Checkbox Edge Version: In-depth Process
Introduction
------------

The following sections provide a detailed walkthrough of the Canary Testing process for the Checkbox Edge version, starting from snap building to the outcomes of the testing.
Canary testing applies on Checkbox snaps that are released through the `edge channel <https://snapcraft.io/docs/channels>`_ in the Snap Store. New versions are built daily if changes are made in the code repository.

The following sections provide a detailed walk-through of the Canary Testing process for the Checkbox Edge version, starting from snap building to the outcomes of the testing.

.. toctree::
:maxdepth: 1
:hidden:

canary_pipeline
validation_job_example
validation_pipeline_execution
validation_job_example


Building and Publishing the Checkbox Snap
-----------------------------------------
Snap Build and Release
-----------------------

Trigger Conditions
^^^^^^^^^^^^^^^^^^
The GitHub action responsible for building the snap runs every day. However, it only triggers when the following condition is met:

At least one (PR) has been merged since the last edge build.

Build URL
^^^^^^^^^
At least one pull request (PR) has been merged since the last edge build.

To monitor the build process, or to review the configuration and logs, visit the following URL:
Build Workflow
^^^^^^^^^^^^^^^

https://github.com/canonical/checkbox/actions/workflows/checkbox-snap-daily-builds.yml
To monitor the build process, or to review the configuration and logs, see `the GitHub workflow <https://github.com/canonical/checkbox/actions/workflows/checkbox-snap-daily-builds.yml>`_.

Post-Build Actions
^^^^^^^^^^^^^^^^^^

Once the build is successful:

The snap packages are automatically pushed to the Snap Store.
These snaps are released to the edge channel, allowing testers and early adopters to access the latest version.

Once the build is successful, the snap packages are automatically pushed to the Snap Store in the edge channel. Testers and early adopters can access the latest version through edge releases.

Jenkins Monitoring and Validation
---------------------------------

Snap Monitoring
^^^^^^^^^^^^^^^

Once the snap is published to the edge channel in the Snap Store, our Jenkins job titled `checkbox-edge-validation-detect-new-build` gets into action.
Once the snap is published to the edge channel in the Snap Store, our Jenkins job titled ``checkbox-edge-validation-detect-new-build`` gets into action.

Monitoring job URL
^^^^^^^^^^^^^^^^^^

This Jenkins job monitors the Snap Store for the presence of the new snap using the following URL:

[https://api.snapcraft.io/v2/snaps/find?q=checkbox22&channel=edge&fields=revision&architecture=amd64](https://api.snapcraft.io/v2/snaps/find?q=checkbox22&channel=edge&fields=revision&architecture=amd64)
``https://api.snapcraft.io/v2/snaps/find?q=checkbox22&channel=edge&fields=revision&architecture=amd64``

The specific JSON path that's being monitored for changes is:

Expand Down Expand Up @@ -86,19 +83,21 @@ The pipeline concurrently runs 3 distinct jobs on different platforms:
2. **amd64 ubuntu core 16**: Also using machines from the ``dearest team`` queue.
3. **arm64 ubuntu core 22**: Targeting machines in the ``cert-rpi4b4g`` queue, which corresponds to the Raspberry Pi4 4GB model.

The :doc:`validation_job_example` contains the jenkins job definition for the amd64 ubuntu core 22 validation.
The :doc:`validation_job_example` contains the Jenkins job definition for the amd64 ubuntu core 22 validation.

For a detailed look on how the job execution is carried out by all the entities in the chain,
refer to the :doc:`validation_pipeline_execution`
refer to the :doc:`validation_pipeline_execution`.

Canary Test Plan Criteria
^^^^^^^^^^^^^^^^^^^^^^^^^

The canary test plan outlines specific tests that are imperative for the new snap's validation. The pipeline's successful conclusion is contingent upon all these tests passing on each of the mentioned platforms.

## Outcome and Commit Tagging
Outcome and Commit Tagging
----------------------------

The outcome of the process is a tag that's being assigned to the commit from which the snaps were built.
The tag is either `edge-validation-succeeded` on success or `edge-validation-failed` otherwise.
The tag is either ``edge-validation-succeeded`` on success or ``edge-validation-failed`` otherwise.
The tag is pushed to the Checkbox repository on GitHub.

Conclusion
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Canary Pipeline
^^^^^^^^^^^^^^^
This is an example on how the whole pipeline can be implemented in Jenkins.

::
.. code-block:: groovy

pipeline {
agent any
Expand All @@ -24,7 +24,7 @@ This is an example on how the whole pipeline can be implemented in Jenkins.
},
"Checkbox series-16 for amd64": {
echo 'Running Canary on core16 amd64'
build job: 'ccheckbox-edge-validation-core16-amd64', wait: true, propagate: true
build job: 'checkbox-edge-validation-core16-amd64', wait: true, propagate: true
}

)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ Example of a validation job
Below is the shell code that can be used as a build step in Jenkins in a job
responsible for validating edge snap of Checkbox.

::
.. code-block:: shell

#!/bin/bash
set -e
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,12 +11,12 @@ and if anything new landed a new version is created.


.. mermaid::

graph TB

A[GitHub Workflow: Detect New Commits]

B{Run Metabox Tests on New Version}
B[Run Metabox Tests on New Version]
C[Mark Version as Edge and Build Snaps]
D{Did Metabox Tests Pass?}
E{Did Snap Builds Succeed?}
Expand All @@ -33,15 +33,17 @@ and if anything new landed a new version is created.
D -->|No| G
E -->|No| G


With new Edge version of Checkbox in the store we can start validating it.

Validating the Edge version
---------------------------

On the Cert Jenkins instance, the `{insert_final_job_name_here}` checks the store API for new Edge versions of Checkbox.
The job is defined in the hwcert-jenkins-jobs repository here: `{insert url to the trigger job}`.
On the Certification Jenkins instance, the ``{insert_final_job_name_here}`` job checks the store API for new Edge versions of Checkbox.
The job is defined in the |hwcert-jenkins-jobs|_ repository.

This job is also responsible for checking if all of the necessary snaps were published (for other series and architectures).
Once confirmed the "Canary Test Plan" defined here: `Canry test plan <https://github.com/canonical/checkbox/blob/main/providers/base/units/canary/test-plan.pxu>`_
Once confirmed, the "Canary Test Plan" is defined in `Canary test plan <https://github.com/canonical/checkbox/blob/main/providers/base/units/canary/test-plan.pxu>`_.

.. mermaid::

Expand All @@ -62,7 +64,7 @@ Once confirmed the "Canary Test Plan" defined here: `Canry test plan <https://gi



There's multiple entities participating in the chain of validating a Checkbox snap.
There are multiple entities participating in the chain of validating a Checkbox snap.

.. mermaid::

Expand Down Expand Up @@ -101,3 +103,8 @@ There's multiple entities participating in the chain of validating a Checkbox sn
deactivate TServer
Jenkins-->>Pipeline: Update Pipeline with Job Outcome
deactivate Pipeline

.. add code format to link text
.. |hwcert-jenkins-jobs| replace:: ``hwcert-jenkins-jobs``
.. _hwcert-jenkins-jobs: https://github.com

1 change: 0 additions & 1 deletion docs/how-to/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -11,4 +11,3 @@ These how-to guides cover key operations and processes in Checkbox.
nested-test-plan
side-loading
agent-service
release_process/canary