Skip to content

Add: include certification-status fields (New)#882

Merged
pieqq merged 2 commits intocanonical:mainfrom
stanley31huang:231208_add-cert-blocker-data
Jan 10, 2024
Merged

Add: include certification-status fields (New)#882
pieqq merged 2 commits intocanonical:mainfrom
stanley31huang:231208_add-cert-blocker-data

Conversation

@stanley31huang
Copy link
Copy Markdown
Collaborator

@stanley31huang stanley31huang commented Dec 8, 2023

include the certification-status fields for list-bootstrapped command (New)

Description

Adding the certification-status attribute for list-bootstrapped command

Resolved issues

Documentation

N/A

Tests

stanley@stanley-ThinkPad-T490:~$ PYTHONPATH=/home/stanley/Desktop/Git_Repos/checkbox-2023/checkbox-ng:$PYTHONPATH checkbox-cli list-bootstrapped -f ? com.canonical.certification::client-cert-odm-ubuntucore-22-manual
Available fields are:
_description, _purpose, _siblings, _steps, _summary, _verification, after, category_id, certification_status, command, depends, environ, esimated_duration, estimate_duration, estimated_duration, flags, full_id, id, imports, plugin, requires, template-engine, unit, user
stanley@stanley-ThinkPad-T490:~/Desktop/Git_Repos/checkbox-2023$ PYTHONPATH=/home/stanley/Desktop/Git_Repos/checkbox-2023/checkbox-ng:$PYTHONPATH checkbox-cli list-bootstrapped -f "ID: {id}\nPlugin: {plugin}\nDescr: {_summary}\nCert-blocker: {certification_status}\n\n" com.canonical.certification::client-cert-odm-ubuntucore-22-manual
unable to find nested part: com.canonical.certification::edac-manual
unable to find nested part: com.canonical.certification::usb-dwc3-manual
ID: manifest
Plugin: resource
Descr: Hardware Manifest
Cert-blocker: unspecified

ID: executable
Plugin: resource
Descr: Enumerate available system executables
Cert-blocker: unspecified

ID: interface
Plugin: resource
Descr: Collect information about interfaces
Cert-blocker: unspecified

ID: connections
Plugin: resource
Descr: Collect information about connections
Cert-blocker: unspecified

ID: model_assertion
Plugin: resource
Descr: Collect model assertions on the device
Cert-blocker: unspecified

ID: serial_assertion
Plugin: resource
Descr: Collect serial assertions on the device
Cert-blocker: unspecified

ID: net_if_management
Plugin: resource
Descr: Identify what service is managing each physical network interface
Cert-blocker: unspecified

ID: net_if_management_attachment
Plugin: attachment
Descr: Collect logging from the net_if_management job
Cert-blocker: unspecified

ID: lspci_attachment
Plugin: attachment
Descr: Attach a list of PCI devices
Cert-blocker: unspecified

ID: lsusb_attachment
Plugin: attachment
Descr: Attach output of lsusb
Cert-blocker: unspecified

ID: rtc
Plugin: resource
Descr: Creates resource info for RTC
Cert-blocker: unspecified

ID: sleep
Plugin: resource
Descr: Create resource info for supported sleep states
Cert-blocker: unspecified

ID: parts_meta_info_attachment
Plugin: attachment
Descr: Attaches an information about all parts that constituted this snap
Cert-blocker: unspecified

ID: miscellanea/device_check
Plugin: user-interact-verify
Descr: Device Check
Cert-blocker: unspecified

ID: meminfo
Plugin: resource
Descr: Collect information about system memory (/proc/meminfo)
Cert-blocker: unspecified

ID: lspci_standard_config_json
Plugin: attachment
Descr: Attach PCI configuration space hex dump
Cert-blocker: unspecified

ID: dmi_present
Plugin: resource
Descr: Resource to detect if dmi data is present
Cert-blocker: unspecified

ID: raw_devices_dmi_json
Plugin: attachment
Descr: Attaches json dumps of raw dmi devices
Cert-blocker: unspecified

ID: efi
Plugin: resource
Descr: Collect information about the EFI configuration
Cert-blocker: unspecified

ID: dkms_info_json
Plugin: attachment
Descr: Attaches json dumps of installed dkms package information.
Cert-blocker: unspecified

ID: kernel_cmdline_attachment
Plugin: attachment
Descr: Attach a copy of /proc/cmdline
Cert-blocker: unspecified

ID: environment
Plugin: resource
Descr: Create resource info for environment variables
Cert-blocker: unspecified

ID: uname
Plugin: resource
Descr: Collect information about the running kernel
Cert-blocker: unspecified

ID: dpkg
Plugin: resource
Descr: Collect information about dpkg version
Cert-blocker: unspecified

ID: package
Plugin: resource
Descr: Collect information about installed software packages
Cert-blocker: unspecified

ID: cpuinfo
Plugin: resource
Descr: Collect information about the CPU
Cert-blocker: unspecified

ID: cdimage
Plugin: resource
Descr: Collect information about installation media (casper)
Cert-blocker: unspecified

ID: modprobe_json
Plugin: attachment
Descr: Attach the contents of /etc/modprobe.*
Cert-blocker: unspecified

ID: udev_json
Plugin: attachment
Descr: Attaches json dumps of udev_resource.py
Cert-blocker: unspecified

ID: udev_attachment
Plugin: attachment
Descr: Attach dump of udev database
Cert-blocker: unspecified

ID: system_info_json
Plugin: attachment
Descr: Attaches json dumps of system info tools
Cert-blocker: unspecified

ID: sysfs_attachment
Plugin: attachment
Descr: Attach detailed sysfs property output from udev
Cert-blocker: unspecified

ID: device
Plugin: resource
Descr: Collect information about hardware devices (udev)
Cert-blocker: unspecified

ID: requirements
Plugin: resource
Descr: Provide links to requirements documents
Cert-blocker: unspecified

ID: dmi_attachment
Plugin: attachment
Descr: Attach a copy of /sys/class/dmi/id/*
Cert-blocker: unspecified

ID: module
Plugin: resource
Descr: Collect information about kernel modules
Cert-blocker: unspecified

ID: lsblk_attachment
Plugin: attachment
Descr: Attach info block devices and their mount points
Cert-blocker: unspecified

ID: dmi
Plugin: resource
Descr: Collect information about hardware devices (DMI)
Cert-blocker: unspecified

ID: snap
Plugin: resource
Descr: Collect information about installed snap packages
Cert-blocker: unspecified

ID: lsb
Plugin: resource
Descr: Collect information about installed system (lsb-release)
Cert-blocker: unspecified

ID: miscellanea/submission-resources
Plugin: shell
Descr: Check that data for a complete result are present
Cert-blocker: unspecified

ID: info/systemd-analyze
Plugin: shell
Descr: System boot-up performance statistics
Cert-blocker: unspecified

ID: audio/detect-playback-devices
Plugin: shell
Descr: Check that at least one audio playback device exits
Cert-blocker: unspecified

ID: audio/alsa-playback
Plugin: user-interact-verify
Descr: Playback works
Cert-blocker: unspecified

ID: audio/detect-capture-devices
Plugin: shell
Descr: Check that at least one audio capture device exists
Cert-blocker: unspecified

ID: audio/alsa-loopback
Plugin: user-interact-verify
Descr: Captured sound matches played one
Cert-blocker: unspecified

ID: bluetooth/keyboard-manual
Plugin: manual
Descr: Bluetooth keyboard manual test
Cert-blocker: unspecified

ID: led/power
Plugin: manual
Descr: Power LED behavior when powered
Cert-blocker: unspecified

ID: led/power-blink-suspend
Plugin: manual
Descr: Power LED behavior when suspended
Cert-blocker: unspecified

ID: led/bluetooth
Plugin: manual
Descr: Bluetooth LED behavior
Cert-blocker: unspecified

ID: led/serial
Plugin: user-interact-verify
Descr: Serial ports LED behavior
Cert-blocker: unspecified

ID: led/fn
Plugin: manual
Descr: <missing _summary>
Cert-blocker: unspecified

ID: mediacard/sd-insert
Plugin: user-interact
Descr: Test that insertion of an SD card is detected
Cert-blocker: unspecified

ID: mediacard/sd-storage
Plugin: shell
Descr: Test reading & writing to a SD Card
Cert-blocker: unspecified

ID: mediacard/sd-remove
Plugin: user-interact
Descr: Test that removal of an SD card is detected
Cert-blocker: unspecified

ID: mediacard/sdhc-insert
Plugin: user-interact
Descr: Test that insertion of an SDHC card is detected
Cert-blocker: unspecified

ID: mediacard/sdhc-storage
Plugin: shell
Descr: Test reading & writing to a SDHC Card
Cert-blocker: unspecified

ID: mediacard/sdhc-remove
Plugin: user-interact
Descr: Test that removal of an SDHC card is detected
Cert-blocker: unspecified

ID: serial/rs232-console
Plugin: manual
Descr: Serial debugging console is enabled and operational
Cert-blocker: unspecified

ID: usb-c/hid
Plugin: manual
Descr: USB HID work on USB Type-C port
Cert-blocker: unspecified

ID: usb
Plugin: resource
Descr: Collect information about supported types of USB
Cert-blocker: unspecified

ID: usb-c/insert
Plugin: user-interact
Descr: USB 3.0 storage device insertion detected on USB Type-C port
Cert-blocker: unspecified

ID: usb-c/storage-automated
Plugin: shell
Descr: USB 3.0 storage device read & write works on USB Type-C port
Cert-blocker: unspecified

ID: usb-c/remove
Plugin: user-interact
Descr: USB 3.0 storage device removal detected on USB Type-C port
Cert-blocker: unspecified

ID: usb-c-otg/g_serial
Plugin: user-interact-verify
Descr: Check if USB OTG can work as a serial port.
Cert-blocker: unspecified

ID: usb-c-otg/g_serial-cleanup
Plugin: shell
Descr: Cleanup USB OTG serial interface setup after serial device test
Cert-blocker: unspecified

ID: usb-c-otg/g_mass_storage
Plugin: user-interact-verify
Descr: Check DUT can be detected as a mass storage device
Cert-blocker: unspecified

ID: usb-c-otg/g_mass_storage-cleanup
Plugin: shell
Descr: Cleanup mass storage setup after mass storage device test
Cert-blocker: unspecified

ID: usb-c-otg/g_ether
Plugin: user-interact-verify
Descr: Check DUT can be detected as USB ethernet device
Cert-blocker: unspecified

ID: usb-c-otg/g_ether-cleanup
Plugin: shell
Descr: Cleanup USB OTG ethernet interface setup after ethernet device test
Cert-blocker: unspecified

ID: usb/hid
Plugin: manual
Descr: USB keyboard works
Cert-blocker: unspecified

ID: usb/insert
Plugin: user-interact
Descr: USB 2.0 storage device insertion detected
Cert-blocker: unspecified

ID: usb/storage-automated
Plugin: shell
Descr: USB 2.0 storage device read & write works
Cert-blocker: unspecified

ID: usb/remove
Plugin: user-interact
Descr: USB 2.0 storage device removal detected
Cert-blocker: unspecified

ID: usb3/insert
Plugin: user-interact
Descr: USB 3.0 storage device insertion detected
Cert-blocker: unspecified

ID: usb3/storage-automated
Plugin: shell
Descr: USB 3.0 storage device read & write works
Cert-blocker: unspecified

ID: usb3/remove
Plugin: user-interact
Descr: USB 3.0 storage device removal detected
Cert-blocker: unspecified

ID: thunderbolt3/insert
Plugin: user-interact
Descr: Storage insert detection on Thunderbolt 3 port
Cert-blocker: blocker

ID: thunderbolt3/storage-test
Plugin: shell
Descr: Storage test on Thunderbolt 3
Cert-blocker: blocker

ID: thunderbolt3/remove
Plugin: user-interact
Descr: Storage removal detection on Thunderbolt 3 port
Cert-blocker: blocker

ID: wwan/detect-manual
Plugin: manual
Descr: Check if WWAN module is available
Cert-blocker: unspecified

ID: wwan/check-sim-present-manual
Plugin: manual
Descr: Check if a SIM card is present in a slot connected to the modem
Cert-blocker: unspecified

ID: wwan/gsm-connection-manual
Plugin: manual
Descr: Verify a GSM broadband modem can create a data connection
Cert-blocker: unspecified

ID: wwan/scan-networks-manual
Plugin: manual
Descr: Verify that modem can scan for available networks
Cert-blocker: unspecified

ID: wwan/gsm-connection-interrupted-manual
Plugin: manual
Descr: Verify a GSM broadband connection can be reconnected after the signal is lost
Cert-blocker: unspecified

ID: suspend/suspend_advanced_auto
Plugin: shell
Descr: Automated test of suspend function
Cert-blocker: unspecified

ID: after-suspend-audio/alsa-playback
Plugin: user-interact-verify
Descr: Playback works after suspend (S3)
Cert-blocker: unspecified

ID: after-suspend-audio/alsa-loopback
Plugin: user-interact-verify
Descr: Captured sound matches played one after suspend (S3)
Cert-blocker: unspecified

ID: after-suspend-bluetooth/keyboard-manual
Plugin: manual
Descr: Bluetooth keyboard manual test after suspend (S3)
Cert-blocker: unspecified

ID: ethernet/detect
Plugin: shell
Descr: Detect if at least one ethernet device is detected
Cert-blocker: unspecified

ID: after-suspend-ethernet/detect
Plugin: shell
Descr: Detect if at least one ethernet device is detected after suspend (S3)
Cert-blocker: blocker

ID: ethernet/hotplug-enx606d3c4b91af
Plugin: user-interact
Descr: Ensure hotplugging works on port {{ interface }}
Cert-blocker: unspecified

ID: after-suspend-ethernet/hotplug-enx606d3c4b91af
Plugin: user-interact
Descr: Ensure hotplugging works on port enx606d3c4b91af after suspend (S3)
Cert-blocker: blocker

ID: ethernet/hotplug-enp0s31f6
Plugin: user-interact
Descr: Ensure hotplugging works on port {{ interface }}
Cert-blocker: unspecified

ID: after-suspend-ethernet/hotplug-enp0s31f6
Plugin: user-interact
Descr: Ensure hotplugging works on port enp0s31f6 after suspend (S3)
Cert-blocker: blocker

ID: after-suspend-wwan/detect-manual
Plugin: manual
Descr: Check if WWAN module is available after suspend (S3)
Cert-blocker: unspecified

ID: after-suspend-wwan/gsm-connection-manual
Plugin: manual
Descr: Verify a GSM broadband modem can create a data connection after suspend (S3)
Cert-blocker: unspecified

ID: after-suspend-wwan/check-sim-present-manual
Plugin: manual
Descr: Check if a SIM card is present in a slot connected to the modem after suspend (S3)
Cert-blocker: unspecified

ID: after-suspend-wwan/scan-networks-manual
Plugin: manual
Descr: Verify that modem can scan for available networks after suspend (S3)
Cert-blocker: unspecified

ID: after-suspend-wwan/gsm-connection-interrupted-manual
Plugin: manual
Descr: Verify a GSM broadband connection can be reconnected after the signal is lost after suspend (S3)
Cert-blocker: unspecified

ID: after-suspend-thunderbolt3/insert
Plugin: user-interact
Descr: Storage insert detection on Thunderbolt 3 port after suspend (S3)
Cert-blocker: blocker

ID: after-suspend-thunderbolt3/storage-test
Plugin: shell
Descr: Storage test on Thunderbolt 3 after suspend (S3)
Cert-blocker: blocker

ID: after-suspend-thunderbolt3/remove
Plugin: user-interact
Descr: Storage removal detection on Thunderbolt 3 port after suspend (S3)
Cert-blocker: blocker

include the certification-status fields for list-bootstrapped command
@codecov
Copy link
Copy Markdown

codecov bot commented Dec 8, 2023

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (601f7ce) 35.70% compared to head (8035c1b) 35.78%.
Report is 3 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #882      +/-   ##
==========================================
+ Coverage   35.70%   35.78%   +0.08%     
==========================================
  Files         303      303              
  Lines       34250    34251       +1     
  Branches     5917     5917              
==========================================
+ Hits        12230    12258      +28     
+ Misses      21458    21430      -28     
- Partials      562      563       +1     
Flag Coverage Δ
checkbox-ng 61.90% <100.00%> (+0.18%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

adding unittest for the invoke function of ListBootstrapeed class
@pieqq
Copy link
Copy Markdown
Collaborator

pieqq commented Dec 11, 2023

/canonical/self-hosted-runners/run-workflows 8035c1b

Copy link
Copy Markdown
Collaborator

@pieqq pieqq left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for this QoL improvement! I don't know why I didn't approve this sooner...

@pieqq pieqq merged commit 941f021 into canonical:main Jan 10, 2024
LiaoU3 pushed a commit to LiaoU3/checkbox that referenced this pull request Mar 20, 2024
…nonical#882)

* Add: include certification-status fields

include the certification-status fields for list-bootstrapped command

* add unittest for ListBootstrapped.invoke function

adding unittest for the invoke function of ListBootstrapeed class
binli pushed a commit to binli/checkbox that referenced this pull request Mar 22, 2024
…nonical#882)

* Add: include certification-status fields

include the certification-status fields for list-bootstrapped command

* add unittest for ListBootstrapped.invoke function

adding unittest for the invoke function of ListBootstrapeed class
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants