[ICDPSession] Page.startScreencast causes subsequent operations to fail

[ben-biddington]

Hey I have been looking at getting Page.screencast to work and no matter what I try I find that sending Page.screencast succeeds, but any other operation after that point fails.

These two example differ only by this line:

await session.SendAsync("Page.startScreencast", new { format = "png", everyNthFrame = 1 });

The first one passes, the second one throws error.

using System;
using System.Threading.Tasks;
using NUnit.Framework;
using PuppeteerSharp;

[TestFixture]
public class AboutChromeDevToolsAndScreencasting
{
    [Test]
    public async Task Can_use_cdp_session_without_screencast()
    {
        await using var browser = await Puppeteer.LaunchAsync(new LaunchOptions { Headless = false });
        await using var page = await browser.NewPageAsync();

        var session = await page.CreateCDPSessionAsync();

        session.MessageReceived += (_, args) => { Console.WriteLine($"[EVENT] {args.MessageID} {args.MessageData}"); };

        await page.GoToAsync("https://www.rnz.co.nz");

        Assert.That(await page.GetTitleAsync(), Is.EqualTo("RNZ - NZ News, Current Affairs, Audio On Demand"));
    }

    [Test]
    public async Task Page_enable_causes_subsequent_operations_to_fail()
    {
        await using var browser = await Puppeteer.LaunchAsync(new LaunchOptions { Headless = false });
        await using var page = await browser.NewPageAsync();

        var session = await page.CreateCDPSessionAsync();

        /*

            Prints:

                [EVENT] Page.screencastVisibilityChanged {"visible":true}

        */
        session.MessageReceived += (_, args) => { Console.WriteLine($"[EVENT] {args.MessageID} {args.MessageData}"); };

        /*
            [!] This line causes all subsequent operations to fail

            PuppeteerSharp.NavigationException : Protocol error(Page.navigate): Target closed. (Connection failed to process System.Byte[].
                The JSON value could not be converted to System.String. Path: $.sessionId | LineNumber: 0 | BytePositionInLine: 10386..

        */
        await session.SendAsync("Page.startScreencast", new { format = "png", everyNthFrame = 1 });

        await page.GoToAsync("https://www.rnz.co.nz");

        Assert.That(await page.GetTitleAsync(), Is.EqualTo("RNZ - NZ News, Current Affairs, Audio On Demand"));
    }
}

What am I missing?

PuppeteerSharp 20.0.5

[ben-biddington]

Looks to be because when the obj.Method is Page.screencastFrame, obj.Params is not castable to ConnectionResponseParams because it looks more like:

internal class ScreencastFrame
{
    public string Data { get; set; }
}

If I do this crude thing (add if (method != "Page.screencastFrame") guard):

// lib/PuppeteerSharp/Cdp/Connection.cs
private void ProcessIncomingMessage(ConnectionResponse obj)
        {
            var method = obj.Method;

            if (method != "Page.screencastFrame")
            {
                var param = obj.Params?.ToObject<ConnectionResponseParams>();

                if (method == "Target.attachedToTarget")
                {
                    var sessionId = param.SessionId;
                    var session = new CdpCDPSession(this, param.TargetInfo.Type, sessionId, obj.SessionId);
                    _sessions.AddItem(sessionId, session);

                    SessionAttached?.Invoke(this, new SessionEventArgs(session));

                    if (obj.SessionId != null && _sessions.TryGetValue(obj.SessionId, out var parentSession))
                    {
                        parentSession.OnSessionAttached(session);
                    }
                }
                else if (method == "Target.detachedFromTarget")
                {
                    var sessionId = param.SessionId;
                    if (_sessions.TryRemove(sessionId, out var session) && !session.IsClosed)
                    {
                        session.Close("Target.detachedFromTarget");
                        SessionDetached?.Invoke(this, new SessionEventArgs(session));

                        if (_sessions.TryGetValue(sessionId, out var parentSession))
                        {
                            parentSession.OnSessionDetached(session);
                        }
                    }
                }
            }

            if (!string.IsNullOrEmpty(obj.SessionId))
            {
                var session = GetSession(obj.SessionId);
                session?.OnMessage(obj);
            }
            else if (obj.Id.HasValue)
            {
                // If we get the object we are waiting for we return if
                // if not we add this to the list, sooner or later some one will come for it
                if (_callbacks.TryRemove(obj.Id.Value, out var callback))
                {
                    MessageQueue.Enqueue(callback, obj);
                }
            }
            else
            {
                MessageReceived?.Invoke(this, new MessageEventArgs
                {
                    MessageID = method,
                    MessageData = (JsonElement)obj.Params,
                });
            }
        }

Then I can get MessageEventArgs with the base64 encoded data.

{
  "data": "iVBORw0KGgoAAAA...=",
  "metadata": {
    "offsetTop": 0,
    "pageScaleFactor": 1,
    "deviceWidth": 800,
    "deviceHeight": 600,
    "scrollOffsetX": 0,
    "scrollOffsetY": 0,
    "timestamp": 1754453959.046579
  },
  "sessionId": 1
}

[kblok]

That's interesting!

[kblok]

It looks like Puppeteer has a screencast feature that we don't support yet. We could do that.

[ben-biddington]

Happy to help, let me know how to proceed 👍

[kblok]

@ben-biddington we could start implementing this PR. There are a few challenges there. Like using ffmpeg.

[ben-biddington]

I have made progress on this, just having ffmpeg trouble

[kblok]

I have made progress on this, just having ffmpeg trouble

@ben-biddington yeah that can be challenging. If you need an early review, just create a PR