Adds status command; extend crypto with reset option by blackwood · Pull Request #466 · bitwarden/browser-interactions-testing

blackwood · 2026-03-11T15:30:13Z

Add dev tooling: status check, crypto reset, and improved error visibility

scripts/status.sh — prerequisite checker that inspects .env, SSL certs, node_modules, extension build, Docker/vault health, and test site reachability. Prints actionable hints on failure. Detects wrong-version stacks, port conflicts, orphaned BIT volumes from old compose projects, and missing vault accounts after a version switch.
scripts/reset-crypto.sh — strips generated crypto vars from .env and reruns setup:crypto, for when VAULT_EMAIL or VAULT_PASSWORD must change.
package.json — adds npm run status and npm run setup:crypto:reset.
scripts/seeder.sh — removes --quiet from bw login so errors surface; adds a process liveness check in the bw serve wait loop for faster failure detection.
scripts/create-account.ts — logs message and validationErrors on retry so server-side rejections (e.g. invalid email format) are visible.
README / CLAUDE.md — adds returning-workflow entry point, VAULT_EMAIL format constraint, setup:crypto:reset usage, Docker troubleshooting notes, and orphaned volume cleanup guidance.

github-actions · 2026-03-11T15:33:02Z

Checkmarx One – Scan Summary & Details – 3cfb19ab-92b8-4e7b-bd0d-164cccaa8bfd

Great job! No new security vulnerabilities introduced in this pull request

jprusik · 2026-03-11T17:55:30Z

package.json

    "seed:vault:ciphers": "./scripts/seeder.sh",
    "seed:vault:ciphers:refresh": "REFRESH=true ./scripts/seeder.sh",
    "seed:vault:import": "./scripts/vault-import.sh",
+    "status": "./scripts/status.sh",


How would you feel about calling this something like flightcheck? To me, status reads more as something that is constantly polling/monitoring/updating 🤔

jprusik · 2026-03-11T18:01:32Z

scripts/flightcheck.sh

+# ── Test site ─────────────────────────────────────────────────────────────────
+PAGES_URL="${PAGES_HOST:-https://127.0.0.1}${PAGES_HOST_PORT:+:${PAGES_HOST_PORT}}"
+PID_FILE="$ROOT_DIR/.test-site.pid"
+
+if curl -sf $CACERT_OPT --max-time 3 "${PAGES_URL}" > /dev/null 2>&1; then
+  row "$OK" "Test site" "running  (${PAGES_URL})"
+elif [ -f "$PID_FILE" ] && kill -0 "$(cat "$PID_FILE")" 2>/dev/null; then
+  row "$WARN" "Test site" "process alive but not yet responding"
+  warnings=$((warnings + 1))
+else
+  row "$WARN" "Test site" "not running  (${PAGES_URL})"
+  hint "npm run start:test-site"
+  warnings=$((warnings + 1))
+fi


This one probably shouldn't be checked for a running state; Playwright handles serving the test site for testing and tearing down. The only reason (I can think of?) to run the test site manually is to debug/develop the test site, and for that you're better off using the project's internal scripts.

What might be a more useful check is if it's runnable; maybe the script can try serving, hit the endpoint, teardown, and report on if it was successful?

If the test site is already running, does bit favor that? I only ask because I ran it manually without problems (and saw playwright's script hitting the endpoints).

Yes; the magic happens here:
https://github.com/bitwarden/browser-interactions-testing/blob/main/playwright.config.ts#L70-L75

And reuseExistingServer handles doing that check:

I still don't think the test-site's present running state should be in the checklist; it implies it's a requirement when it's not.

jprusik · 2026-03-11T18:25:21Z

scripts/flightcheck.sh

This is a great idea- could really help get people set up quickly and diagnose problems encountered. Nice work!

jprusik · 2026-03-11T18:28:11Z

scripts/flightcheck.sh

+  hint "npm run start:test-site"
+  warnings=$((warnings + 1))
+fi
+


Another status check I'd personally like to see is a summary of the feature flags; something that happens to me frequently is forgetting to sync the flags with my target configuration.

For example, we could hit the vault's /api/config and count the number of entries in featureStates, which would be a quick signal as to whether flags were loaded or not.

(We could also check for the presence of REMOTE_VAULT_CONFIG_MATCH to shortcut and exit early)

Similarly, we'll probably also want to eventually grab environment?.fillAssistRules (see bitwarden/server#7137) since that will directly impact BIT tests, but we're quite away from implementing those tests, so nbd here.

jprusik · 2026-03-11T18:35:40Z

scripts/flightcheck.sh

+
+# ── node_modules ──────────────────────────────────────────────────────────────
+if [ -d "$ROOT_DIR/node_modules/.bin" ]; then
+  row "$OK" "node_modules" "installed"


What is this telling us?

Might be overkill--checks whether npm install has been run or not.

whether it has ever been run (and not subsequently deleted); it could be quite stale. It does feel a bit overkill, yeah. The test-site check would fail in this case anyway.

jprusik · 2026-03-11T18:40:36Z

scripts/status.sh

+  set -o allexport; source "$ROOT_DIR/.env"; set +o allexport
+
+  missing=()
+  for var in VAULT_EMAIL VAULT_PASSWORD PAGES_HOST VAULT_HOST_URL VAULT_HOST_PORT; do


Also?

CLI_SERVE_HOST CLI_SERVE_PORT EXTENSION_BUILD_PATH BW_DOMAIN BW_DB_PROVIDER BW_DB_SERVER BW_DB_DATABASE BW_DB_USERNAME BW_DB_PASSWORD BW_DB_PORT BW_ENABLE_SSL

For most testing configurations, those are going to be needed, since they tell BIT not just where to set things up, but where to find them later. Granted some things are implicitly tested later by virtue of being downstream of this setup, but it's not a guarantee.

For example, the presence of SSL cert files doesn't mean those values are configured in the .env - so, I suppose it depends on what we're looking to communicate with ".env is configured"

(the truly optional values are marked as such in the .env.example; otherwise, we assume an empty environment where all the files/settings are coming from the generation scripts which use the .env values)

[ -z "$val" ] will be false if the value is an empty string or unset, so they'll be marked missing in that case.

jprusik · 2026-03-11T18:46:48Z

scripts/flightcheck.sh

+  hint "npm run setup:ssl"
+  errors=$((errors + 1))
+fi
+


It might be nice to check (as an independent concern, if VAULT_IMPORT_FILE is configured and if the file was found)

jprusik · 2026-03-11T18:48:38Z

scripts/flightcheck.sh

+MANIFEST="$ROOT_DIR/$BUILD_PATH/manifest.json"
+
+if [ -f "$MANIFEST" ]; then
+  MV=$(grep -o '"manifest_version":[[:space:]]*[0-9]*' "$MANIFEST" | grep -o '[0-9]*$')


This is a really nice/useful touch

jprusik · 2026-03-11T18:53:41Z

scripts/reset-crypto.sh

+  exit 1
+fi
+
+echo "Removing existing crypto values from .env..."


I'm really wary of doing this programmatically. Rotating the crypto values requires creating a new account, which requires a new email, which also seeds install values, which will require a server rebuild. You have to manually update the email in the .env anyway, so maybe the expectation should be that you manually remove the generated crypto values?

I suspect this might lead people to "break" their test vaults without understanding why.

Another check that might be worthwhile if we want to do this kind of regular rotation, is a validation script of the vault email in the .env with the generated crypto values; that way if they get out of sync (the user updates email but forgets to clear the old cypto), we can tell them exactly why and how to fix it.

Another thought - it feels like the case for this is really "create additional test accounts on the same vault server", which we probably want to do, but possibly as a separate effort.

It's worth noting that this calls generate-crypto.ts directly after cleaning up, which would also warn the user about existing values. Maybe the right call is to prompt the user then and there in that script to delete/override the values? (and nuke this script).

which would also warn the user about existing values

It won't though, because they were just removed, no?

scripts/seeder.sh

jprusik · 2026-03-11T19:40:58Z

.claude/CLAUDE.md


 1. **No real credentials in test data**: All vault passwords are fake (e.g., `"fakeBasicFormPassword"`). Prefix test credential values with `fake`.
 2. **No secrets in source**: Crypto material, master password hashes, and API keys live only in `.env` (gitignored). Generated by `npm run setup:crypto` — never set manually.
+    - `VAULT_EMAIL` is used as a PBKDF2 salt — changing it invalidates `MASTER_PASSWORD_HASH` and requires `npm run setup:crypto:reset` + a fresh DB.


also worth mentioning; it's a seeding value for generating the install keys

.claude/CLAUDE.md

jprusik · 2026-03-11T19:44:23Z

.claude/CLAUDE.md

A note here that Claude should never run the public tests is probably warranted.

jprusik · 2026-03-11T19:46:08Z

README.md

+
+```bash
+docker compose up -d --wait   # start the vault server
+npm run start:test-site


You shouldn't need this line - Playwright handles this concern

jprusik · 2026-03-11T19:49:27Z

README.md


+## Returning Workflow
+
+Run `npm run status` to check which prerequisites are satisfied and get specific commands for anything that needs attention. Typically all that's needed is:


Suggested change

Run `npm run status` to check which prerequisites are satisfied and get specific commands for anything that needs attention. Typically all that's needed is:

Run `npm run status` to check which prerequisites are satisfied and get specific commands for anything that needs attention. Typically you only need to do the initial set up once; after that all that's needed is:

jprusik · 2026-03-11T19:50:44Z

README.md


 > Important! Once you've generated installation and crypto values for your `.env` file, DO NOT CHANGE the seeding values (`VAULT_EMAIL`, `VAULT_PASSWORD`, `KDF_ITERATIONS`). Doing so requires regenerating your installation and crypto secret values and rebuilding/updating server.

+> If you do need to change `VAULT_EMAIL` or `VAULT_PASSWORD`, run `npm run setup:crypto:reset` to clear and regenerate the derived crypto values, then `docker compose down -v && docker compose up -d --wait` for a fresh database, then `npm run setup:vault`.


Similarly, if you change VAULT_EMAIL you may need to regenerate the install keys/rebuild the vault, depending on what you're trying to do.

jprusik · 2026-03-11T19:52:54Z

README.md


 Create and start the containers and volumes with `docker compose up -d --build --remove-orphans`, and teardown with `docker compose down -v`

+> If the image pull fails with a network error (e.g. `unexpected EOF`), re-run `docker compose pull` and then retry. If it continues to fail, try increasing the timeout: `COMPOSE_HTTP_TIMEOUT=120 docker compose pull`.


One reason this commonly fails is endpoint protection / firewall rules. In the case of the former, the request is significantly delayed. An allowlist rule is probably preferred (this guidance works, but we should also call out the common underlying causes).

jprusik · 2026-03-11T19:57:02Z

README.md

+
+> If startup fails with `port is already allocated`, a previous container session is holding the port — often a prior BIT stack (e.g. after updating the image version). Run `npm run status` to identify which project owns the port and get the exact teardown command.
+
+> Each compose project uses its own database volume. Switching to a different vault image version means a fresh database — re-run `npm run setup:vault` after bringing the new stack up. Old project volumes are not removed automatically; `npm run status` will list any orphaned BIT volumes and show the `docker volume rm` commands to clean them up.


You might also add guidance on:

You don't need/want to run setup on a previously seeded vault. You only need to start it. This is relevant when switching between several built environments (for example, with different server images, or feature flag configurations)

jprusik · 2026-03-11T19:59:10Z

README.md

+
+> If startup fails with `port is already allocated`, a previous container session is holding the port — often a prior BIT stack (e.g. after updating the image version). Run `npm run status` to identify which project owns the port and get the exact teardown command.
+
+> Each compose project uses its own database volume. Switching to a different vault image version means a fresh database — re-run `npm run setup:vault` after bringing the new stack up. Old project volumes are not removed automatically; `npm run status` will list any orphaned BIT volumes and show the `docker volume rm` commands to clean them up.


npm run status will list any orphaned BIT volumes

This presumes cleanup of old versions; it will not show BIT volumes that coexist but are simply not running (which is probably fine)

jprusik · 2026-03-11T20:01:39Z

scripts/flightcheck.sh

+        local proj
+        proj=$(docker inspect --format \
+          "{{index .Config.Labels \"com.docker.compose.project\"}}" "$cid" 2>/dev/null)
+        [ -n "$proj" ] && echo "docker project: ${proj}" || echo "docker container ${cid:0:12}"


Other useful information about the running container: name, db image, open ports

(name is particularly relevant when disambiguating multiple containers)

see also: how Docker Desktop displays this information

…utput.

jprusik · 2026-03-17T21:09:12Z

scripts/flightcheck.sh

✨ ✨ ✨ ✨

The one nit - up in that screenshot it says my flags are synced; they were not - my flags.json was {"flagValues": {}}. You could try parsing the json as well, but I think you should adjust the signal to what it's trying to communicate; flag syncing is pointed at x, flag state is y (loaded/empty)

jprusik · 2026-03-17T21:12:38Z

README.md


 > Important! Once you've generated installation and crypto values for your `.env` file, DO NOT CHANGE the seeding values (`VAULT_EMAIL`, `VAULT_PASSWORD`, `KDF_ITERATIONS`). Doing so requires regenerating your installation and crypto secret values and rebuilding/updating server.

+> If you do need to change `VAULT_EMAIL` or `VAULT_PASSWORD`, manually remove the generated crypto vars (`KDF_ITERATIONS`, `MASTER_PASSWORD_HASH`, `PROTECTED_SYMMETRIC_KEY`, `GENERATED_RSA_KEY_PAIR_PUBLIC_KEY`, `GENERATED_RSA_KEY_PAIR_PROTECTED_PRIVATE_KEY`) from your `.env`, run `npm run setup:crypto`, then `docker compose down -v && docker compose up -d --wait` for a fresh database, then `npm run setup:vault`. Note that changing `VAULT_EMAIL` may also require regenerating install keys (`npm run setup:install`) and rebuilding the vault, depending on what you're trying to do.


Removed unneeded step; only need to rebuild the server/db if you want the new account to be associated with the install (owner)

Suggested change

> If you do need to change `VAULT_EMAIL` or `VAULT_PASSWORD`, manually remove the generated crypto vars (`KDF_ITERATIONS`, `MASTER_PASSWORD_HASH`, `PROTECTED_SYMMETRIC_KEY`, `GENERATED_RSA_KEY_PAIR_PUBLIC_KEY`, `GENERATED_RSA_KEY_PAIR_PROTECTED_PRIVATE_KEY`) from your `.env`, run `npm run setup:crypto`, then `docker compose down -v && docker compose up -d --wait` for a fresh database, then `npm run setup:vault`. Note that changing `VAULT_EMAIL` may also require regenerating install keys (`npm run setup:install`) and rebuilding the vault, depending on what you're trying to do.

> If you do need to change `VAULT_EMAIL` or `VAULT_PASSWORD`, manually remove the generated crypto vars (`KDF_ITERATIONS`, `MASTER_PASSWORD_HASH`, `PROTECTED_SYMMETRIC_KEY`, `GENERATED_RSA_KEY_PAIR_PUBLIC_KEY`, `GENERATED_RSA_KEY_PAIR_PROTECTED_PRIVATE_KEY`) from your `.env`, run `npm run setup:crypto`, then `npm run setup:vault`. Note that changing `VAULT_EMAIL` may also require regenerating install keys (`npm run setup:install`) and rebuilding the vault, depending on what you're trying to do.

jprusik · 2026-03-17T21:22:53Z

scripts/create-account.ts

+      if (preCreationResponseData.message) {
+        console.log(`\x1b[2m  ${preCreationResponseData.message}\x1b[0m`);
+      }
+      if (preCreationResponseData.validationErrors) {
+        Object.entries(preCreationResponseData.validationErrors).forEach(
+          ([field, msgs]) =>
+            console.log(`\x1b[2m    ${field}: ${msgs.join(", ")}\x1b[0m`),
+        );
+      }


I haven't surfaced these previously because this is totally normal and expected:

I worry someone seeing "server error" for the first time here is going to think they did something wrong and reconfigure, when in reality, they just need to wait for everything to init.

On the flip side, misconfigurations can end up in the same state except without the eventual resolution. Right now the account creation script stops after 60 attempts. we could probably safely bump that down to 20-30

jprusik

I've left a few more nits and notes, but I don't think they're significant enough to hold things up; I'll leave it to you.

blackwood requested a review from a team as a code owner March 11, 2026 15:30

blackwood requested a review from jprusik March 11, 2026 15:30

jprusik reviewed Mar 11, 2026

View reviewed changes

scripts/seeder.sh Show resolved Hide resolved

jprusik reviewed Mar 11, 2026

View reviewed changes

.claude/CLAUDE.md Outdated Show resolved Hide resolved

jprusik reviewed Mar 11, 2026

View reviewed changes

blackwood force-pushed the feature/status branch from 276c2bc to 6950230 Compare March 12, 2026 14:33

blackwood added 5 commits March 17, 2026 11:28

Adds status command; extend crypto with reset option; improve error o…

9075f9d

…utput.

Removes reference to potential fluke issue.

fef17f1

flightcheck script, lifecycle docs, README/CLAUDE updates

546c523

Removes node_modules check.

d9e4378

Removes reset-crypto.sh; separate flightcheck err and warn cases.

8e09dee

blackwood force-pushed the feature/status branch from 6aeb189 to 8e09dee Compare March 17, 2026 15:28

jprusik reviewed Mar 17, 2026

View reviewed changes

jprusik approved these changes Mar 17, 2026

View reviewed changes


		## Returning Workflow

		Run `npm run status` to check which prerequisites are satisfied and get specific commands for anything that needs attention. Typically all that's needed is:


		> Important! Once you've generated installation and crypto values for your `.env` file, DO NOT CHANGE the seeding values (`VAULT_EMAIL`, `VAULT_PASSWORD`, `KDF_ITERATIONS`). Doing so requires regenerating your installation and crypto secret values and rebuilding/updating server.

		> If you do need to change `VAULT_EMAIL` or `VAULT_PASSWORD`, run `npm run setup:crypto:reset` to clear and regenerate the derived crypto values, then `docker compose down -v && docker compose up -d --wait` for a fresh database, then `npm run setup:vault`.


		Create and start the containers and volumes with `docker compose up -d --build --remove-orphans`, and teardown with `docker compose down -v`

		> If the image pull fails with a network error (e.g. `unexpected EOF`), re-run `docker compose pull` and then retry. If it continues to fail, try increasing the timeout: `COMPOSE_HTTP_TIMEOUT=120 docker compose pull`.


		> If startup fails with `port is already allocated`, a previous container session is holding the port — often a prior BIT stack (e.g. after updating the image version). Run `npm run status` to identify which project owns the port and get the exact teardown command.

		> Each compose project uses its own database volume. Switching to a different vault image version means a fresh database — re-run `npm run setup:vault` after bringing the new stack up. Old project volumes are not removed automatically; `npm run status` will list any orphaned BIT volumes and show the `docker volume rm` commands to clean them up.

Conversation

blackwood commented Mar 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Mar 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Great job! No new security vulnerabilities introduced in this pull request

Uh oh!

jprusik Mar 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jprusik Mar 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jprusik Mar 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jprusik Mar 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jprusik Mar 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jprusik Mar 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jprusik Mar 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jprusik Mar 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

blackwood Mar 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jprusik Mar 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

blackwood commented Mar 11, 2026 •

edited

Loading

github-actions bot commented Mar 11, 2026 •

edited

Loading

jprusik Mar 11, 2026 •

edited

Loading

jprusik Mar 11, 2026 •

edited

Loading

jprusik Mar 11, 2026 •

edited

Loading

jprusik Mar 11, 2026 •

edited

Loading

jprusik Mar 12, 2026 •

edited

Loading

jprusik Mar 11, 2026 •

edited

Loading

jprusik Mar 11, 2026 •

edited

Loading

jprusik Mar 11, 2026 •

edited

Loading

blackwood Mar 12, 2026 •

edited

Loading

jprusik Mar 11, 2026 •

edited

Loading

jprusik Mar 11, 2026 •

edited

Loading

jprusik Mar 17, 2026 •

edited

Loading