Skip to content

Add wait buffer after reboot/suspend stress tests (bugfix)#1981

Merged
Hook25 merged 9 commits intomainfrom
1918-add-buffer-after-reboot
Jul 16, 2025
Merged

Add wait buffer after reboot/suspend stress tests (bugfix)#1981
Hook25 merged 9 commits intomainfrom
1918-add-buffer-after-reboot

Conversation

@pieqq
Copy link
Copy Markdown
Collaborator

@pieqq pieqq commented Jun 30, 2025

Description

Translate the suspend triggering Shell script into Python, and add a --wait option to add some waiting time before actually triggering the suspend.

Some components (WWAN modules for instance) may take up to 2 minutes to
fully initialize after a reboot, so the script waits for this. (It also didn't make sense for a device to be suspended right after being rebooted)

Resolved issues

Fix CHECKBOX-1918

Documentation

Tests

https://github.com/canonical/checkbox/actions/runs/16020007949/job/45194470346

Each job run after a reboot now waits 120 seconds (or as defined by the STRESS_S3_INIT_DELAY env var), but not the subsequent jobs. For instance:

2025-07-02T09:29:23.4135609Z ----------------------------[ Running job 69 / 134 ]----------------------------
2025-07-02T09:29:23.4136290Z -------------[ Reboot device during stress test (reboot cycle 1). ]-------------
2025-07-02T09:29:23.4138340Z ID: com.canonical.certification::stress-tests/suspend_cycles_reboot1
2025-07-02T09:29:23.4139037Z Category: Suspend (S3) Stress Test
2025-07-02T09:29:23.4139537Z --------------------------------------------------------------------------------
2025-07-02T09:29:23.4140055Z Connection lost!
2025-07-02T09:29:23.4140377Z connection closed by peer
2025-07-02T09:29:33.9060585Z Reconnecting -�\�|�/�-�\�|�/�-�\�|�/�...
2025-07-02T09:29:33.9061617Z Reconnected (took: 25s)
2025-07-02T09:29:33.9064621Z -------------[ Reboot device during stress test (reboot cycle 1). ]-------------
2025-07-02T09:29:33.9065907Z ID: com.canonical.certification::stress-tests/suspend_cycles_reboot1
2025-07-02T09:29:33.9066692Z Category: Suspend (S3) Stress Test
2025-07-02T09:29:33.9082143Z --------------------------------------------------------------------------------
2025-07-02T09:29:33.9091126Z Outcome: job passed
2025-07-02T09:29:44.4299634Z ----------------------------[ Running job 76 / 140 ]----------------------------
2025-07-02T09:29:44.4300817Z --------[ Suspend and resume device (suspend cycle 1, reboot cycle 2) ]---------
2025-07-02T09:29:44.4302263Z ID: com.canonical.certification::stress-tests/suspend_cycles_1_reboot2
2025-07-02T09:29:44.4303137Z Category: Suspend (S3) Stress Test
2025-07-02T09:29:44.4303712Z --------------------------------------------------------------------------------
2025-07-02T09:29:44.4304639Z Current boot ID is: 3c47e7d6a34e469b9011c5a47eea4a8c
2025-07-02T09:33:03.4626849Z Waiting for 120 seconds...
2025-07-02T09:33:03.4627586Z Running FWTS to trigger suspend...

@pieqq pieqq marked this pull request as draft June 30, 2025 08:07
@codecov
Copy link
Copy Markdown

codecov bot commented Jun 30, 2025

Codecov Report

❌ Patch coverage is 90.32258% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 50.85%. Comparing base (2dab338) to head (4369476).
⚠️ Report is 123 commits behind head on main.

Files with missing lines Patch % Lines
...kbox-support/checkbox_support/scripts/fwts_test.py 40.00% 3 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1981      +/-   ##
==========================================
+ Coverage   50.60%   50.85%   +0.24%     
==========================================
  Files         384      386       +2     
  Lines       41180    41418     +238     
  Branches     7636     7689      +53     
==========================================
+ Hits        20841    21062     +221     
- Misses      19594    19602       +8     
- Partials      745      754       +9     
Flag Coverage Δ
checkbox-support 64.66% <40.00%> (+0.72%) ⬆️
provider-base 26.03% <100.00%> (+0.33%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@pieqq pieqq force-pushed the 1918-add-buffer-after-reboot branch 2 times, most recently from 02b243f to 9e4dd70 Compare June 30, 2025 10:24
pieqq added 3 commits July 2, 2025 09:20
The script mimics bin/suspend.sh, but adds a `--wait` optional argument
that can be used to wait for the device to be fully initialized.

Since it's a Python script, unit tests are added.

Also modify bin/fwts_test.py so that the main function can be called
with arguments, since this is what is done from the new
bin/suspend_trigger.py script.
Use the new Python script, with `--wait` option when it's the first
suspend cycle after a reboot, since this is when the device is
initializing.

Some components (WWAN modules for instance) may take up to 2 minutes to
fully initialize after a reboot, so the script waits for this.

(It also didn't make sense for a device to be suspended *riught* after
being rebooted)

Remove suspend.sh script as it's not used anymore.

Fix CHECKBOX-1918
@pieqq pieqq force-pushed the 1918-add-buffer-after-reboot branch 2 times, most recently from 4c2e84f to c6120f5 Compare July 2, 2025 09:47
@pieqq pieqq marked this pull request as ready for review July 2, 2025 09:48
Copy link
Copy Markdown
Collaborator

@Hook25 Hook25 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fantastic, this is way better than the bash version!

Please use check_call instead of check_output, saves you a bit of troubles, also consider the stuff about envvars. I know it was already that way, but still

@pieqq pieqq requested a review from Hook25 July 15, 2025 11:54
@Hook25 Hook25 merged commit 7b6dd5b into main Jul 16, 2025
32 checks passed
@Hook25 Hook25 deleted the 1918-add-buffer-after-reboot branch July 16, 2025 08:42
mreed8855 pushed a commit that referenced this pull request Jul 30, 2025
* Add suspend trigger Python script

The script mimics bin/suspend.sh, but adds a `--wait` optional argument
that can be used to wait for the device to be fully initialized.

Since it's a Python script, unit tests are added.

Also modify bin/fwts_test.py so that the main function can be called
with arguments, since this is what is done from the new
bin/suspend_trigger.py script.

* Update jobs

Use the new Python script, with `--wait` option when it's the first
suspend cycle after a reboot, since this is when the device is
initializing.

Some components (WWAN modules for instance) may take up to 2 minutes to
fully initialize after a reboot, so the script waits for this.

(It also didn't make sense for a device to be suspended *riught* after
being rebooted)

Remove suspend.sh script as it's not used anymore.

Fix CHECKBOX-1918

* Fix unit tests for Python 3.5

* Use check_call instead of check_output

* Use script arguments instead of envvars

* Adjust unit tests accordingly

* Call Checkbox jobs with arguments

* black <3 flake8

* Fix unit tests to work with py3.5
mreed8855 pushed a commit that referenced this pull request Jul 31, 2025
* Add suspend trigger Python script

The script mimics bin/suspend.sh, but adds a `--wait` optional argument
that can be used to wait for the device to be fully initialized.

Since it's a Python script, unit tests are added.

Also modify bin/fwts_test.py so that the main function can be called
with arguments, since this is what is done from the new
bin/suspend_trigger.py script.

* Update jobs

Use the new Python script, with `--wait` option when it's the first
suspend cycle after a reboot, since this is when the device is
initializing.

Some components (WWAN modules for instance) may take up to 2 minutes to
fully initialize after a reboot, so the script waits for this.

(It also didn't make sense for a device to be suspended *riught* after
being rebooted)

Remove suspend.sh script as it's not used anymore.

Fix CHECKBOX-1918

* Fix unit tests for Python 3.5

* Use check_call instead of check_output

* Use script arguments instead of envvars

* Adjust unit tests accordingly

* Call Checkbox jobs with arguments

* black <3 flake8

* Fix unit tests to work with py3.5
bladernr pushed a commit that referenced this pull request Aug 28, 2025
* Add suspend trigger Python script

The script mimics bin/suspend.sh, but adds a `--wait` optional argument
that can be used to wait for the device to be fully initialized.

Since it's a Python script, unit tests are added.

Also modify bin/fwts_test.py so that the main function can be called
with arguments, since this is what is done from the new
bin/suspend_trigger.py script.

* Update jobs

Use the new Python script, with `--wait` option when it's the first
suspend cycle after a reboot, since this is when the device is
initializing.

Some components (WWAN modules for instance) may take up to 2 minutes to
fully initialize after a reboot, so the script waits for this.

(It also didn't make sense for a device to be suspended *riught* after
being rebooted)

Remove suspend.sh script as it's not used anymore.

Fix CHECKBOX-1918

* Fix unit tests for Python 3.5

* Use check_call instead of check_output

* Use script arguments instead of envvars

* Adjust unit tests accordingly

* Call Checkbox jobs with arguments

* black <3 flake8

* Fix unit tests to work with py3.5
stanley31huang pushed a commit that referenced this pull request Oct 3, 2025
* Add suspend trigger Python script

The script mimics bin/suspend.sh, but adds a `--wait` optional argument
that can be used to wait for the device to be fully initialized.

Since it's a Python script, unit tests are added.

Also modify bin/fwts_test.py so that the main function can be called
with arguments, since this is what is done from the new
bin/suspend_trigger.py script.

* Update jobs

Use the new Python script, with `--wait` option when it's the first
suspend cycle after a reboot, since this is when the device is
initializing.

Some components (WWAN modules for instance) may take up to 2 minutes to
fully initialize after a reboot, so the script waits for this.

(It also didn't make sense for a device to be suspended *riught* after
being rebooted)

Remove suspend.sh script as it's not used anymore.

Fix CHECKBOX-1918

* Fix unit tests for Python 3.5

* Use check_call instead of check_output

* Use script arguments instead of envvars

* Adjust unit tests accordingly

* Call Checkbox jobs with arguments

* black <3 flake8

* Fix unit tests to work with py3.5
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants