Switchboard

Profile manager and benchmarking tool for AI toolchains. Atomically swap config files, validate environments, and compare profiles with golden tasks.

Also available as sb.

Quickstart

pnpm install
pnpm run build
pnpm link --global

switchboard init          # scaffold config + dirs
switchboard use default   # apply the default profile
switchboard doctor        # validate files, env, MCP

Commands

switchboard / switchboard status

Shows current config path, active profile, profile count, and last applied time. Falls back gracefully if no switchboard.yaml exists.

switchboard init

Scaffolds a new project: creates switchboard.yaml, profiles/default/, and goldens/ with an example task. Guards against overwriting an existing config.

switchboard use <profile> [--dry-run]

Atomically applies a profile's config files with backups. Use --dry-run to preview what would happen without making changes.

switchboard use stable --dry-run   # preview
switchboard use stable             # apply

switchboard doctor [profile] [--no-mcp]

Validates the active (or specified) profile: checks required env vars, file existence, JSON validity, and pings MCP servers (SSE + stdio).

switchboard capture <profile>

Captures current live config files into a profile folder. Useful for onboarding — snapshot your working setup into profiles/<name>/.

switchboard profile <subcommand>

Subcommand	Alias	Description
profile list	ls	List profiles with active marker, file/env/mcp counts
profile show [name]		Show file mappings, env vars, MCP servers (defaults to active)
profile create <name> [--from <existing>]		Create empty or clone an existing profile
profile delete <name>	rm	Remove from config (won't delete files on disk)

switchboard golden <subcommand>

Run reproducible tasks across profiles and compare results.

Subcommand	Alias	Description
golden list	ls	List golden tasks from goldensDir
golden run [-p profile] [-t task]		Execute tasks, save results, print pass/fail
golden compare <profileA> <profileB>		Side-by-side comparison with winner

Golden tasks are YAML files in the goldensDir (default: ./goldens/):

name: hello
description: Verify basic script execution
type: script        # or "prompt"
path: hello.sh
expect:
  exit_code: 0
  output_contains:
    - "hello"

Config

See switchboard.example.yaml for a full example. Key fields:

version: 1
activeProfile: stable
runtimeCommand: claude    # for golden prompt tasks

paths:
  profilesDir: "./profiles"
  stateDir: "~/.switchboard"
  goldensDir: "./goldens"

profiles:
  stable:
    files:
      - from: ".claude/settings.json"
        to: "~/.claude/settings.json"
    env:
      required: [ANTHROPIC_API_KEY]
    mcp:
      servers:
        - name: "local-tools"
          type: "sse"
          url: "http://127.0.0.1:9000/sse"