Some laptops with MIPI camera fail to run multiple-resolution-images_video*

[kevinyehk]

Bug Description

During the racking, I found some laptops fail to run the multiple-resolution-images_video test.
The test passed at 4.3.0.dev71, but after a while, it failed at 4.4.0.dev54.
The camera-quality_video0 test is pass, so it should not be the hardware issue.

Cert-blocker Test Case

• cert-blocker

To Reproduce

1. run checkbox-cli
2. select camera test plan
3. run multiple-resolution-images_video0 test.

Expected Result

The test should pass.

Actual Result

The test fails.

Environment

• OS ubuntu 24.04
• Checkbox type debian
• Checkbox version 4.4.0.dev54.
• Hardware https://certification.canonical.com/hardware/202410-35906/, https://certification.canonical.com/hardware/202410-35907

Relevant log output

Failed to validate image /tmp/camera_test_NV121280x720wlny9pg_.jpg
Format: NV12 (Y/UV 4:2:0)
Resolutions: 1280x720

Taking multiple images using the NV12 pixelformat
Saving debug image to /var/tmp/checkbox-ng/sessions/session_title-2025-03-17T02.29.10.session/session-share/resolution_test_image_video0.jpg
Failed to capture image with fswebcam, using gstreamer
Starting GStreamer image capture pipeline with 3 second delay...
Opening valve!
Image saved to /var/tmp/checkbox-ng/sessions/session_title-2025-03-17T02.29.10.session/session-share/resolution_test_image_video0.jpg
Taking a picture at 1280x720
Failed to capture image with fswebcam, using gstreamer
Starting GStreamer image capture pipeline with 3 second delay...
Opening valve!
Image saved to /tmp/camera_test_NV121280x720wlny9pg_.jpg
Image is not a JPEG file

Additional context

No response

[syncronize-issues-to-jira]

Thank you for reporting your feedback to us!

The internal ticket has been created: https://warthogs.atlassian.net/browse/CHECKBOX-1795.

This message was autogenerated

[tomli380576]

I was able to reproduce this error message on usb cameras that directly outputs image/jpeg. I think the validator was looking for specific jpeg standards (JFIF, Exif, Adobe jpegs) instead of just the generic start and end bits of a jpeg file. Note that jpegenc produces JFIF if it successfully finishes.

# CameraTest._validate_image current impl
if (
    header[6:10] != b"JFIF"
    and header[6:10] != b"Exif"
    and header[6:11] != b"Adobe"
):
    print("Image is not a JPEG file")
    return False

We can fix this by either passing everything through jpegenc (decode it if it was already jpeg)

# if caps is jpeg
source ! image/jpeg ! jpegdec ! videoconvert ! jpegenc ! valve ! multifilesink
# else
source ! video/x-raw ! videoconvert ! jpegenc ! valve ! multifilesink
source ! video/x-bayer ! bayer2rgb ! videoconvert ! jpegenc ! valve ! multifilesink

or simplifying the validator:

# ...test file size first 
f.seek(0)
head = f.read(2)
f.seek(-2, 2)
tail = f.read(2)
if (head, tail) != (b'\xff\xd8', b'\xff\xd9'):
    print("Image is not a JPEG file")
    return False

maybe we can use the file command too:

$ file --mime-type --brief output.jpeg 
image/jpeg

See: https://github.com/corkami/formats/blob/master/image/jpeg.md#parameter-less-markers

This is a totally different problem and shouldn't fail on cameras that only produce video/x-raw

[tomli380576]

The pipeline itself was fine and wrote the image correctly

[tomli380576]

The camera test before the MIPI fix PR kinda got away with this because fswebcam was available most of the time. The gstreamer path was never visited unless fswebcam fails, which only happens when it encounters NV12, the format produced by intel MIPI cameras.

Another issue also helped mask this. When setting the MIME type in the caps filter, it was always choosing video/x-raw unless it sees RG10 as the pixel format (which doesn't happen 99% of the time on PCs). If the camera can produce all of its supported dimensions in video/x-raw, nothing goes wrong. But if the camera has 1 dimension that is only available in image/jpeg, then it would crash with Error: Not negotiated and won't even reach the validator.

In summary we have the following:

• USB cameras always used fswebcam to take photos
• Using fswebcam on Intel MIPI always failed due to NV12
• Intel MIPI always produced video/x-raw
• self._width and self._height was never updated from 640x480 (this was fixed)
• image/jpeg only dimensions were never tested using the gstreamer path
• Video previews was using camerabin which did some typefind magic for us

All these combined together made it look like only Intel MIPI cameras were failing, when it was actually failing for all cameras that would produce formats that are not video/x-raw. The PR made it fail in a different way; before it would fail due to forcing video/x-raw on image/jpeg sources, now it fails because it stored the raw jpegs from image/jpeg sources and the validator doesn't like it.

So yeah it's a sneaky multi-layer bug 🙀 But I think the 2 DUTs in the issue description actually have broken cameras and are expected to fail the test case. (EDIT: yes their cameras are actually broken at the moment)

[tomli380576]

TLDR: there is a bug with the test, but the test results are still accurate

[pieqq]

Thanks a lot for the deep-dive investigation @tomli380576 ! Could you propose a PR for this? It would help since you have easier direct access to different devices in the lab for testing.

[fernando79513]

I think going for the simpler jpeg check, just checking the start of image and end of image should be enough.
The header check was added here because we were using imghdr.what() before, and this was the logic that imghdr was using.

[tomli380576]

Seems like just checking the start and end bytes might not be enough :( some cameras like to produce trailing 0s after the FFD9 byte like this:

ubuntu@ubuntu:~$ hexdump -C out.jpeg | tail -3
0003f880  b2 a5 dd b3 3b 24 60 37  24 b6 07 e3 e9 4f 98 77  |....;$`7$....O.w|
0003f890  00 ff d9 00 00 00 00 00                           |........|
# these trailing zeroes ^^^^^^^^^

• https://certification.canonical.com/hardware/202503-36385/

ubuntu@ubuntu-Vostro-3420:~$ hexdump -C out.jpeg | tail -3
00009a40  6f a0 de 94 e0 32 28 12  76 0e 29 a4 9a a6 89 e6  |o....2(.v.).....|
00009a50  d4 ff d9 00 00 00 00 00                           |........|

• https://certification.canonical.com/hardware/202112-29820/

The amount of trailing zeroes is also completely random.

I think we might want to go with using the file command for this check since it probably does something more sophisticated to determine the file type.

[fernando79513]

Okey, then I think it should be fine just using the file command