Feat: Offscreen Rendering & Screenshots

[bungoboingo]

This PR adds support for offscreen rendering in Iced! 🎉

screenshot_demo.mov

📸 Users can now capture screenshots 🖼️ using the window::screenshot() command.

fn update(&mut self, message: Self::Message) -> Command<Self::Message> {
    match message {
        Message::Screenshot => {
            return iced::window::screenshot(Message::ScreenshotData); //new!
        }
        //...

The window::screenshot command requires a Message which takes a Screenshot. This is a new struct exposed in the iced_runtime crate which contains the RGBA bytes of the screenshot, in addition to its size.

For now this command is limited to the entire viewport, with an optional crop() method available which will crop the bytes to the specified Rectangle.

cropping.mov

let cropped = screenshot.crop(Rectangle::<u32> {
    x: 0,
    y: 0,
    width: 500,
    height: 500,
});

This method returns a Result<Screenshot, CropError> for situations where the cropped region is out of bounds, or is not visible.

🎉 Offscreen rendering capabilities are now added to both wgpu and tiny_skia!

Rendering offscreen is now exposed in the Compositor interface with a render_offscreen function. The outputted texture data is always guaranteed to be in RGBA format in the SRGB color space for compatibility with image libraries & iced images, and to simplify the Screenshot struct with a predetermined byte order & pixel size.

🔢 Implementation details

For wgpu, performing offscreen rendering was fairly straightforward. I create a texture based on the viewport size & the chosen texture format, and then pass that into the backend with a new backend.offscreen method. This method performs a normal render pass on the new texture, and optionally runs a tiny compute shader after drawing all primitives that converts it to wgpu::TextureFormat::Rgba8Unorm with a manual srgb conversion due to wgpu::TextureFormat::Rgba8UnormSrgb not being a supported format for a storage texture. The whole process on my (admittedly very powerful!) test suite of GPUs (AMD, NVIDIA, M1) took about ~2µs in the test example. The only added overhead over a normal render pass is the compute shader to convert to rgba.

For tiny-skia this was very straightforward, the same as a normal present call but rendering to a new buffer vs the surface buffer. Our backend returns ARGB format, so just had to do some bit shiftin' and we're all good.

🤔 Outstanding API questions

1. It was discussed whether it's a cleaner API to do a window::screenshot command which optionally can be cropped, or a window::screenshot(area) command which can either be Fullscreen or Region, which would be a rectangle. Obviously there is a performance hit when rendering the whole window to an offscreen buffer vs a potentially very small region if only a small portion is desired, but the execution is so fast it's almost negligible. I'm curious as to people's thoughts on this!

2. The offscreen render capabilities are only exposed at the moment through this window::screenshot command. The intent was to implement the capabilities for offscreen rendering in this PR and expand upon it further in later iterations if new applications of offscreen rendering are needed. Curious to see if anyone has any thoughts on how else this might be exposed!

🧪 Testing

Tested wgpu & tiny-skia implementations on MacOS + M1, Linux + PopOS + AMD, Windows 10 + Nvidia combos. Further testing would be greatly appreciated!

Feedback very much welcome! This was my first time working this deeply with textures on the GPU so might have made some rookie mistakes 😉 I also know that there is an open draft PR for offscreen rendering but it hasn't been updated in a few years so thought I'd take a crack at it.

[bungoboingo]

Lint wants me to change 0xFF_000000 to 0xFF00_0000 but imo it reads less clearly and is a silly lint. Propose to ignore unusual-byte-ordering?

[tarkah]

Lint wants me to change 0xFF_000000 to 0xFF00_0000 but imo it reads less clearly and is a silly lint. Propose to ignore unusual-byte-ordering?

How about 0b11111111_00000000_00000000_00000000 :P

[hecrj]

Looking good! Some suggestions here and there 🧐

Love your detailed PRs! 💯

[hecrj]

Lint wants me to change 0xFF_000000 to 0xFF00_0000 but imo it reads less clearly and is a silly lint. Propose to ignore unusual-byte-ordering?

Quite subjective, in my opinion. I think it's fairly standard to group evenly. Does 0xFF_00_00_00 work?

[bungoboingo]

Lint wants me to change 0xFF_000000 to 0xFF00_0000 but imo it reads less clearly and is a silly lint. Propose to ignore unusual-byte-ordering?

Quite subjective, in my opinion. I think it's fairly standard to group evenly. Does 0xFF_00_00_00 work?

Lint is happy with 0xFF_00_00_00; I adjusted it to that format, I'm fine with that 👍