feat(cli): add roll-call command for batch-testing model connectivity by cbrunnkvist · Pull Request #7201 · Kilo-Org/kilocode

cbrunnkvist · 2026-03-18T02:04:07Z

Summary

This PR adds a new roll-call subcommand that allows users to batch-test multiple models for connectivity and latency. This helps users discover which provider/model pairs are available and working.

Features

Filter models by regex pattern (required for safety - prevents accidental testing of all 300+ models)
Configurable prompt, timeout, and parallelization
JSON or table output formats
Proper provider options handling for reasoning models

Technical Details

The implementation:

Applies ProviderTransform options (maxOutputTokens, temperature, topP, topK, providerOptions)
Enables reasoning for dedicated thinking models via Kilo Gateway (models with "thinking" in their ID)
Uses proper session-based provider option generation

Testing

Tested with various model filters:

% bun run dev -- roll-call "free"                                                                                                                                                   [1]
$ bun run --cwd packages/opencode --conditions=browser src/index.ts roll-call free
Starting roll call for models with prompt: "Hello"
Timeout per model: 25000ms, Parallel calls: 5
Prompting 6 models...
Model                                    | Access | Snippet                                                                                      | Latency
----------------------------------------------------------------------------------------------------------------------------------------------------------
kilo/arcee-ai/trinity-large-preview:free | YES    | Hey there! I'm AFM, your multipurpose autonomous software agent. How can I assist you today? | 1215ms
kilo/openrouter/free                     | YES    | Hello! How can I assist you today?                                                           | 1681ms
kilo/minimax/minimax-m2.5:free           | YES    | Hello! How can I help you today?                                                             | 3171ms
kilo/stepfun/step-3.5-flash:free         | YES    | Hello! 👋 How can I assist you today?                                                        | 6340ms
kilo/z-ai/glm-5:free                     | YES    | Hello! How can I help you today?                                                             | 17428ms
kilo/corethink:free                      | NO     | (Invalid JSON response)                                                                      | 19473ms

5 accessible, 1 failed

Demonstrating truncation, error handling, color coding:

Related Issue

Closes #6304

packages/opencode/src/cli/cmd/roll-call.ts

kilo-code-bot · 2026-03-18T02:19:41Z

Code Review Summary

Status: 1 Issue Found | Recommendation: Address before merge

Overview

Severity	Count
CRITICAL	0
WARNING	1
SUGGESTION	0

Issue Details (click to expand)

WARNING

File	Line	Issue
`packages/opencode/src/cli/cmd/roll-call.ts`	312	Full roll-call failures still exit successfully

Fix these issues in Kilo Cloud

Other Observations (not in diff)

None.

Files Reviewed (3 files)

packages/opencode/src/cli/cmd/roll-call.ts - 1 unresolved warning
packages/opencode/src/index.ts
packages/opencode/test/cli/roll-call.test.ts

_{Reviewed by gpt-5.4-20260305 · 1,736,380 tokens}

packages/opencode/src/cli/cmd/roll-call.ts

Adds a new CLI command that allows users to test multiple models for connectivity and latency in parallel. This helps users discover which provider/model pairs are available and working. Features: - Filter models by regex pattern (e.g., 'kilo roll-call "free"') - Configurable prompt, timeout, and parallelization - JSON or table output formats - Verbose mode for real-time progress Closes Kilo-Org#6304

Reject non-positive --parallel values that would cause the scheduler to spin forever, and reject empty filter strings that would bypass the safety guard and fan out to every configured model.

Filter out models without text output capability (e.g. embeddings, image-only) since generateText is the wrong probe for them and would produce false negatives.

Extract isTextModel predicate and add tests covering multimodal, audio-only, image-only, and embedding models.

JSON and md modes emit clean data to stdout with no decoration. --quiet now only suppresses progress, never the result output. Add formatMarkdown and tests.

Escape | in markdown table cells to prevent broken rows. Add process.exitCode = 1 on the invalid regex path.

cbrunnkvist force-pushed the feat/roll-call-command branch from c204aaa to fe98bc1 Compare March 18, 2026 02:10

cbrunnkvist marked this pull request as ready for review March 18, 2026 02:14