Fix crash when using formats with bits per pixel size not divisible by 8#61
Conversation
nathanbabcock
left a comment
There was a problem hiding this comment.
Nice improvement to support YUV420P and potentially many other pixel formats as well. Also the better error handling is a welcome change.
There's an edge case here that needs to be handled: if the bytes per frame can't be calculated, it allocates a zero-size buffer and then the iterator constantly produces OutputFrame events with zero size and never finishes. That could happen, for example, with an irregularly-sized 321x421 YUV420P output.
I'll merge this as a starting point and follow up with a fallback to "chunked" mode — the same way it would handle encoded video data like H264. Depending on your use case that might be perfectly fine, or you may need to check for either OutputFrame or OutputChunk depending on the situation. We can iterate on this solution if needed.
oops. I didn't notice that I only exit from the lambda that's passed to |
### What * part of #7607 Recognizing ahead of time what format a piece of encoded data is unfortunately not that easy. To accomplish that I fell into the rabbit hole of parsing the Sequence Parameter Set (SPS). Which will hopefully be useful for other things in the future! Other than that, it turned out that determining the color space & color range are again different cans of worms, so I'm being now a lot more explicit with ffmpeg on what we expect. Which.. is yet another rabbit hole! Pretty much no matter what I tried I couldn't replicate exact colors some sample videos compared to VLC (unlike we did with AV1 where we have a lot more solid grasp on all these things and get satisfying parity!). But also here different video players seem to do slightly different things, so I decided to not follow this thread further for the moment as this whole affair already cost me a lot more time that I wanted to invest originally. ~Partially disabled unfortunately since we need this fix:~ * nathanbabcock/ffmpeg-sidecar#61 <img width="390" alt="image" src="https://github.com/user-attachments/assets/c340544a-0c32-406e-ba20-2acc630c0238"> (note that this uses 422, the video source material is 420 though) Fixes slow 4k video handling: Before: <img width="512" alt="image" src="https://github.com/user-attachments/assets/82a0f0a1-03e7-4eac-8a24-a17ba185a8b5"> After: <img width="548" alt="image" src="https://github.com/user-attachments/assets/fb8315f4-5a85-4c2e-83bb-7e7e0c8ded33"> ### Checklist * [x] I have read and agree to [Contributor Guide](https://github.com/rerun-io/rerun/blob/main/CONTRIBUTING.md) and the [Code of Conduct](https://github.com/rerun-io/rerun/blob/main/CODE_OF_CONDUCT.md) * [x] I've included a screenshot or gif (if applicable) * [x] I have tested the web demo (if applicable): * Using examples from latest `main` build: [rerun.io/viewer](https://rerun.io/viewer/pr/8002?manifest_url=https://app.rerun.io/version/main/examples_manifest.json) * Using full set of examples from `nightly` build: [rerun.io/viewer](https://rerun.io/viewer/pr/8002?manifest_url=https://app.rerun.io/version/nightly/examples_manifest.json) * [x] The PR title and labels are set such as to maximize their usefulness for the next release's CHANGELOG * [x] If applicable, add a new check to the [release checklist](https://github.com/rerun-io/rerun/blob/main/tests/python/release_checklist)! * [x] If have noted any breaking changes to the log API in `CHANGELOG.md` and the migration guide - [PR Build Summary](https://build.rerun.io/pr/8002) - [Recent benchmark results](https://build.rerun.io/graphs/crates.html) - [Wasm size tracking](https://build.rerun.io/graphs/sizes.html) To run all checks from `main`, comment on the PR with `@rerun-bot full-check`.
Two separate fixes actually:
get_bytes_per_frameassumed that a bits per pixel size that is not divisible by 8 means that it can't retrieve the frame size because that will yield a frame buffer with a non full-byte size. This is not entirely true since for instance YUV formats typically have size restrictions such that their raw frame buffers don't run into this problem. ➡️ The check got changed to the frame buffer's sizeget_bytes_per_framecan't return a size and instead report the error and exitThe former fixes using yuv formats for me, the later is a bit of a general requirement for me since I deal with arbitrary user data: not being able to handle data is sad, but crashing over it goes too far :)