Collecting Playwright Coverage — Accessing the Playwright Page in a Custom StageCrewMember

[Gontrum]

Hi everyone,

I’m running UI tests with Cucumber-JS + Serenity/JS + Playwright (Chromium) and want to gather JS coverage (LCOV) across my entire suite. I’d like to write a small StageCrewMember that:

1. Before any tests run, calls

   await page.coverage.startJSCoverage();

2. After all tests finish, calls

   const coverage = await page.coverage.stopJSCoverage();
   // merge and write a single coverage.lcov

The blocker: inside my crew member, how do I actually get hold of the Playwright Page object that Serenity/JS is using? Everything I’ve tried fails to retrieve the page, and I’d rather not spin up a separate browser instance.

---

serenity.config.ts

import { configure, Cast }            from '@serenity-js/core';
import { BrowseTheWebWithPlaywright } from '@serenity-js/playwright';
import { PlaywrightAdapter }          from '@serenity-js/playwright';
import { ConsoleReporter }            from '@serenity-js/console-reporter';
import { ArtifactArchiver }           from '@serenity-js/serenity-bdd';
import { chromium }                   from 'playwright';

BeforeAll(async () => {
  const browser = await chromium.launch({ headless: true });

  configure({
    actors: Cast.where(actor =>
      actor.whoCan(BrowseTheWebWithPlaywright.using(browser))
    ),
    crew: [
      ConsoleReporter.forDarkTerminals(),
      PlaywrightAdapter.forCucumber(),
      ArtifactArchiver.storingArtifactsAt('./target/site/serenity'),
      CoverageCrew,    // ← plug in our custom crew here
    ],
  });
});

---

Pseudocode for CoverageCrew

import { StageCrewMember, Stage }     from '@serenity-js/core';
import { BrowseTheWebWithPlaywright }  from '@serenity-js/playwright';

export class CoverageCrew implements StageCrewMember {
  assignedTo(stage: Stage): StageCrewMember {
    return this;
  }

  async notifyOf(event: any, stage: Stage): Promise<void> {
    const actor = stage.theActorInTheSpotlight();

    if (/* suite is about to start */) {
      const page = BrowseTheWebWithPlaywright.as(actor).page;
      await page.coverage.startJSCoverage();
    }

    if (/* suite has finished */) {
      const page     = BrowseTheWebWithPlaywright.as(actor).page;
      const coverage = await page.coverage.stopJSCoverage();

      // ▶ merge all coverage fragments
      // ▶ generate LCOV
      // ▶ write out ./target/site/coverage/coverage.lcov
    }
  }
}

---

Questions

1. Suite-level hooks
   How can I run code exactly once before all tests and once after all tests in the latest Serenity/JS + Cucumber?

2. Accessing the Playwright Page
   What’s the correct API to grab the raw Playwright Page inside a StageCrewMember or interaction?

3. Alternative patterns
   Is there a more idiomatic way in current Serenity/JS to collect Playwright coverage across the suite (e.g. a built-in hook or extension)?

Any pointers, pseudo-snippets or links to examples would be hugely appreciated. Thanks! 🙏

[jan-molak]

Hi @Gontrum!

Have a look at the Photographer who does something similar, i.e. it retrieves the browser associated with the actor to take a screenshot.

The bit you're interested is here:

serenity-js/packages/web/src/stage/crew/photographer/strategies/PhotoTakingStrategy.ts

Lines 35 to 42 in 6df0ca6

	let browseTheWeb: BrowseTheWeb;
	try {
	browseTheWeb = BrowseTheWeb.as(stage.theActorInTheSpotlight());
	} catch {
	// actor doesn't have a browser, abort
	return void 0;
	}

Here, Stage.theActorInTheSpotlight() returns the most recently used actor.

Accessing the page

The difference between what you're trying to do and what the Photographer does is that you want to access a Playwright-specific API - page.

To do that, you could use PlaywrightPage to extract the page:

import { PlaywrightPage } from '@serenity-js/playwright'

async startCoverage(): Promise<void> {

  // ...

  try {
    const actor = stage.actorInTheSpotlight();
    const page = await actor.answer(PlaywrightPage.current().nativePage())
  
    await page.coverage.startJSCoverage();
    
    // ...
  } 
  catch (error) {
    // something's wrong, for example:
    // - there's no active actor
    // - there's an active actor, but they don't have a browser
    // - the browser the actor has is not a Playwright browser
    
    // ...
  }
}

Reacting to events

Serenity/JS stage crew members react to events emitted by the actors and the test runner adapter, and so your implementation will need to react to:

• SceneStarted - to enable the coverage right after the test scenario has started
• SceneFinishes - to process the coverage report right before the test scenario is about to finish

class CoverageReporter implements StageCrewMember {

    constructor(private stage?: Stage) {
    }

    assignedTo(stage: Stage): StageCrewMember {
        this.stage = stage;
        return this;
    }

    notifyOf(event: DomainEvent): void {
        if (! this.stage) {
            throw new LogicError(`CoverageReporter needs to be assigned to the Stage before it can be notified of any DomainEvents`);
        }

        if (! this.stage.theShowHasStarted()) {
            return void 0;
        }

        if (event instanceof SceneStarts) {
            this.startCoverage();
        }
        
        if (event instanceof SceneFinishes) {
            this.stopCoverage();
        }
    }
}

Performing async operations in a stage crew member

Note that both startCoverage and stopCoverage methods need to perform asynchronous operations, but the notifyOf(event: DomainEvent) is not an asynchronous method as it does not return a Promise.

To perform async operations in response to an event, your stage crew member must emit:

• AsyncOperationAttempted before you start the first async operation
• AsyncOperationCompleted, after your last async operation succeeded
• OR AsyncOperationFailed, if an async operation failed and you want it to fail the test scenario
• OR AsyncOperationAborted, when the async operation has not completed successfully for whatever reason, but you don't want it to fail the scenario

Again, have a look at the Photographer or SerenityBDDReporter for inspiration.

Emitting AsyncOperation* events makes Serenity/JS that it needs to wait for your operations to complete before marking the test scenario as completed.
If you're curious how this works under the hood, have a look at the Serenity/JS Cucumber adapter

Questions

Suite-level hooks

How can I run code exactly once before all tests and once after all tests in the latest Serenity/JS + Cucumber?

Instead of SceneStarted and SceneFinishes, you could make your stage crew member react to:

• TestRunStarts
• TestRunFinishes

Accessing the Playwright Page

What’s the correct API to grab the raw Playwright Page inside a StageCrewMember or interaction?

const page = await actor.answer(PlaywrightPage.current().nativePage())

Alternative patterns

Is there a more idiomatic way in current Serenity/JS to collect Playwright coverage across the suite (e.g. a built-in hook or extension)?

Not yet, please let me know if you'd like to contribute your coverage reporter to Serenity/JS when you're finished!

[Gontrum]

Hi @jan-molak, and thanks for the great pointers!

I’ve implemented my crew member to react to events and added a simple

console.log({ event });

inside notifyOf(event: DomainEvent) to inspect what I’m actually receiving. Here’s what I’ve observed:

• Running via CLI (npx cucumber-js features/my.feature):

  • No SceneStarts events are emitted.
  • I do see SceneFinishes events.

• Running via IntelliJ Cucumber plugin:

  • Still no SceneStarts.
  • I do see ActorEntersStage events (which don’t appear when using the CLI).
  • I do see SceneFinishes events.

This makes it hard to start coverage at the very beginning of each scenario, since my crew never sees SceneStarts.

---

Do you have any idea, why there are no SceneStarts emitted when running with cucumber-js from the CLI?
Is there a difference in event wiring between the CLI adapter and the IntelliJ plugin I should be aware of?
I know this is probably more related to Cucumber itself, but maybe you still have an idea.

I’d love to contribute the finished StageCrewMember back to the project once I’ve got everything up and running.

[Gontrum]

I realized I was missing this in my cucumber.yaml config:

  requireModule:
    - 'ts-node/register'

That fixed the problem with not emitting the SceneStarts (at least via cli, the plugin is still not producing SceneStarts, but that's for later...).

But now I’m running into another issue: there’s no Actor attached until the very first InteractionStarts event.
In my notifyOf I have:

if (!this.stage.theShowHasStarted()) {
  return void 0;
}

if (event instanceof SceneStarts) {
  this.startCoverage(this.stage);
}

But startCoverage is never called, because theShowHasStarted() is false. Is that expected behavior?

I could defer until the first InteractionStarts, but then I’d need to track state inside the crew member (since there are multiple interactions per scene). Do you know a cleaner way to kick off coverage exactly once at the start of each scenario, or whether I’m missing another event I should listen for?

[jan-molak]

OK, to delay the start until we have an actor with a browser, you could make your CoverageReporter react to ActorEntersStage. It would be a cleaner approach since not all actors have a browser, and when they do, they could be using different browsers...

ActorEntersStage event looks something like this when reported by the StreamReporter:

{
  "actor": {
    "name": "Phoebe",
    "abilities": [
      {
        "type": "PerformActivities"
      },
      {
        "type": "AnswerQuestions"
      },
      {
        "type": "RaiseErrors"
      },
      {
        "type": "ScheduleWork",
        "options": {
          "scheduler": {
            "clock": {
              "timeAdjustment": {
                "milliseconds": 0
              }
            },
            "interactionTimeout": {
              "milliseconds": 5000
            }
          }
        }
      },
      {
        "type": "ManageALocalServer"
      },
      {
        "class": "BrowseTheWebWithPlaywright",
        "type": "BrowseTheWeb"
      }
    ]
  },
  "sceneId": "ab3c3db8da84c61960b4-6c8831dcacf86d5df8ad",
  "timestamp": "2025-05-19T10:41:16.937Z"
}

One thing to note here is that the ActorEntersStage is currently emitted whenever an actor is activated. I.e. if your test calls actor('Alice'), then actor('Bob'), and then actor('Alice') again, you get three ActorEntersStage events as Alice is activated twice; we might want to revise that model to better support your usecase. I feel we will need to differentiate between ActorEntersStage and ActorActivated - #2855

With this approach, the CoverageReporter would need to hold state, associating the coverage reports with the actors the report is for, and then merge the reports when SceneFinishes.

type ActorName = string;
interface CoverageReport {
  // ...
}

class CoverageReporter implements StageCrewMember {

    private readonly coverageReports = new Map<ActorName, CoverageReport>();

    constructor(private stage?: Stage) {
    }

    assignedTo(stage: Stage): StageCrewMember {
        this.stage = stage;
        return this;
    }

    notifyOf(event: DomainEvent): void {
        if (! this.stage) {
            throw new LogicError(`CoverageReporter needs to be assigned to the Stage before it can be notified of any DomainEvents`);
        }

        if (
          event instanceof ActorEntersStage && 
          event.actor.abilities.some(ability => ability.type === 'BrowseTheWeb') && 
          ! this.coverageReports.has(event.actor.name)
        ) {
            this.startCoverageFor(stage.actor(event.actor.name));
            //This method would add an entry to this.coverageReports,
            // which would then be populated with the report
        }
        
        if (event instanceof SceneFinishes) {
            for (const [ actor, coverage ] of this.coverageReports) {
              this.stopCoverageFor(stage.actor(event.actor.name));
            }
        }
    }
}

[Gontrum]

Thank you for your ongoing support. I tried something like that in the meantime, but had no luck.
Now I prepared this minimal example:

export class AnotherCoverageReporter implements StageCrewMember {
  private readonly coverageReports = new Map<ActorName, CoverageReport>();

  constructor(private stage?: Stage) {}

  assignedTo(stage: Stage): StageCrewMember {
    this.stage = stage;
    return this;
  }

  notifyOf(event: DomainEvent): void {
    if (!this.stage) {
      throw new LogicError(
        `CoverageReporter needs to be assigned to the Stage before it can be notified of any DomainEvents`
      );
    }

    if (
      event instanceof ActorEntersStage &&
      event.actor.abilities.some((ability) => ability.type === 'BrowseTheWeb') &&
      !this.coverageReports.has(event.actor.name)
    ) {
      this.startCoverageFor(this.stage.actor(event.actor.name));
    }

    if (event instanceof SceneFinishes) {
      for (const [actor, coverage] of this.coverageReports) {
        this.stopCoverageFor(actorInTheSpotlight());
      }
    }
  }

  private async startCoverageFor(actor: Actor) {
    const id = CorrelationId.create();
    this.stage?.announce(
      new AsyncOperationAttempted(
        new Name(`CoverageCollector:${this.constructor.name}`),
        new Description('Starting collecting coverage...'),
        id,
        this.stage.currentTime()
      )
    );
    try {
      const page = await actor.answer(PlaywrightPage.current().nativePage());
      page.coverage.startJSCoverage({ resetOnNavigation: false });
      return this.stage?.announce(new AsyncOperationCompleted(id, this.stage.currentTime()));
    } catch (error) {
      if (error instanceof Error) {
        this.stage?.announce(new AsyncOperationFailed(error, id, this.stage.currentTime()));
      }
    }
  }

  private stopCoverageFor(actor: Actor) {
    // TODO needs to stop here
  }
}

I don't know why, but I always get the following error, when the CrewMember is registered and I run a test:

Error: selectors.register: "_sjs_closest" selector engine has been already registered
      at async SerenitySelectorEngines.ensureRegisteredWith (/Users/dgontrum/dev/experiments/collect-coverage/node_modules/@serenity-js/playwright/src/selector-engines/SerenitySelectorEngines.ts:32:17)
      at async PlaywrightBrowsingSessionWithBrowser.browserContext (/Users/dgontrum/dev/experiments/collect-coverage/node_modules/@serenity-js/playwright/src/screenplay/models/PlaywrightBrowsingSession.ts:54:13)
      at async PlaywrightBrowsingSessionWithBrowser.registerCurrentPage (/Users/dgontrum/dev/experiments/collect-coverage/node_modules/@serenity-js/playwright/src/screenplay/models/PlaywrightBrowsingSessionWithBrowser.ts:26:25)
      at async PlaywrightBrowsingSessionWithBrowser.currentPage (/Users/dgontrum/dev/experiments/collect-coverage/node_modules/@serenity-js/web/src/screenplay/models/BrowsingSession.ts:44:39)
      at Function.deserialiseFromStackTrace (/Users/dgontrum/dev/experiments/collect-coverage/node_modules/@serenity-js/core/src/errors/ErrorSerialiser.ts:82:20)
      at CucumberMessagesParser.outcomeFrom (/Users/dgontrum/dev/experiments/collect-coverage/node_modules/@serenity-js/cucumber/src/listeners/messages/parser/CucumberMessagesParser.ts:330:47)
      at CucumberMessagesParser.scenarioOutcomeFrom (/Users/dgontrum/dev/experiments/collect-coverage/node_modules/@serenity-js/cucumber/src/listeners/messages/parser/CucumberMessagesParser.ts:358:40)
      at CucumberMessagesParser.parseTestCaseFinished (/Users/dgontrum/dev/experiments/collect-coverage/node_modules/@serenity-js/cucumber/src/listeners/messages/parser/CucumberMessagesParser.ts:174:34)
      at EventEmitter.<anonymous> (/Users/dgontrum/dev/experiments/collect-coverage/node_modules/@serenity-js/cucumber/src/listeners/messages/CucumberMessagesListener.ts:72:41)
      at EventEmitter.emit (node:events:531:35)
      at EventEmitter.emit (node:domain:488:12)
      at TestCaseRunner.runAttempt (/Users/dgontrum/dev/experiments/collect-coverage/node_modules/@cucumber/cucumber/src/runtime/test_case_runner.ts:267:27)
      at async TestCaseRunner.run (/Users/dgontrum/dev/experiments/collect-coverage/node_modules/@cucumber/cucumber/src/runtime/test_case_runner.ts:195:29)
      at async Runtime.runTestCase (/Users/dgontrum/dev/experiments/collect-coverage/node_modules/@cucumber/cucumber/src/runtime/index.ts:91:20)

Also, when the test starts, a lot of browser windows open at once, but they all close again within a few milliseconds.
I could swear that a startCoverageJs() call in my CrewMember has worked before, but I can’t reproduce any setup right now where I don’t get the error.
The test I’m running is very minimalistic. The only step just contains await actor.attemptsTo(Navigate.to('/'));
Could you possibly give me a hand with this again?

// edit:
It seems like there is no actor when the ActorEntersStage Event is fired, only the actor name in the event.
When we call stage.actor(event.actor.name) a new context is created, the ActorEntersStage Event is fired again and the Call Stack exceeds.

[jan-molak]

@Gontrum - do you think you could put together an example based on one of the official Serenity/JS Project Templates and publish it on GitHub? I think it would make it easier for us to discuss the issue you're seeing.

[Gontrum]

@jan-molak Sure! I used a fresh Vue project as a starting point and added the Cucumber setup afterwards. Let me know if there’s anything I missed: https://github.com/Gontrum/cucumber-minimal-setup

// edit: sorry, didn't see your link and started a fresh project. Will have a look at the starters and make a fresh repo!
// edit2: here's one I created from a ProjectTemplate: https://github.com/Gontrum/coverage-crew-member