Skip to content

test: add Gemini thoughtSignature metadata preservation tests#6368

Open
peterkc wants to merge 1 commit intoKilo-Org:mainfrom
peterkc:fix/gemini-thought-signature
Open

test: add Gemini thoughtSignature metadata preservation tests#6368
peterkc wants to merge 1 commit intoKilo-Org:mainfrom
peterkc:fix/gemini-thought-signature

Conversation

@peterkc
Copy link

@peterkc peterkc commented Feb 25, 2026

Context

While investigating #6018 (Gemini 3.1 Pro thought_signature errors on v5.8.0), I traced the full message pipeline to understand where thoughtSignature metadata could be lost. The pipeline works correctly on current main — the v5.9.0 fix ("preserve extra_content for Gemini 3 thought_signature support") resolves the issue.

However, the differentModel guard in message-v2.ts (line 600) — which strips provider-specific metadata when models differ — had zero test coverage for the positive case (same-model metadata preservation). These tests fill that gap to prevent regression.

Refs: #6018, diagnostic analysis

Implementation

No production code changes. Test-only PR adding regression coverage to message-v2.test.ts:

  • Gemini model fixture (geminiModel): A Provider.Model matching the existing test mock pattern, with providerID: "google", id: "gemini-3.1-pro-preview", npm: "@ai-sdk/google"
  • Metadata fixture (geminiThoughtSignature): { google: { thoughtSignature: "opaque-test-sig==" } } — the shape @ai-sdk/google emits via providerMetadata
  • Test 1 — same-model tool call: Verifies callProviderMetadata with thoughtSignature is preserved on tool-call parts when differentModel evaluates false
  • Test 2 — same-model reasoning: Verifies providerMetadata with thoughtSignature is preserved on reasoning parts

Both tests use the existing assistantInfo() helper with its meta parameter to set matching providerID/modelID, ensuring the differentModel guard evaluates correctly.

Data Flow Under Test

The differentModel guard at message-v2.ts:600 controls metadata flow:

Same model (differentModel = false) — Tests 1 & 2 verify this path:

  processor.ts        DB (ToolPart)       message-v2.ts        AI SDK            Gemini API
  providerMetadata -> part.metadata  ->  callProviderMetadata -> providerOptions -> thoughtSignature
  { google:           stored as-is       preserved by guard      maps to            on functionCall
    thoughtSignature                                              google.            part
    : "opaque..." }                                               thoughtSignature

Model switch (differentModel = true) — Test 3 verifies this path:

  processor.ts        DB (ToolPart)       message-v2.ts        AI SDK
  providerMetadata -> part.metadata  ->  (stripped)            (nothing sent)
                                         guard correctly        prevents cross-
                                         omits metadata         provider leakage

Test Matrix

# Scenario Model differentModel Expected Status
1 Same-model tool call google/gemini-3.1-pro-preview false callProviderMetadata preserved New
2 Same-model reasoning google/gemini-3.1-pro-preview false providerMetadata preserved New
3 Cross-model switch test/test-model vs other/other true metadata stripped Existing (line 358)
4 Same-model text google/gemini-3.1-pro-preview false providerMetadata preserved Future
5 Sequential tool calls google/gemini-3.1-pro-preview false each signature preserved Future
6 Error/pending parts google/gemini-3.1-pro-preview false callProviderMetadata preserved Future

Rows 1-3 are covered by this PR. Rows 4-6 are natural follow-ups if the team wants broader coverage.

How to Test

cd packages/opencode && bun test test/session/message-v2.test.ts
  • 21/21 pass (19 existing + 2 new)
  • Existing differentModel regression test (line 358) continues to pass
  • Typecheck passes (10/10 packages)

Additionally verified via E2E test against live gemini-3.1-pro-preview API — thoughtSignature flows through all streaming events (tool-input-start, tool-call, tool-result) and appears in response message providerOptions.

Get in Touch

Happy to iterate on these tests or expand coverage (rows 4-6 in the matrix). Available here on GitHub.

Add regression tests verifying that providerMetadata containing
thoughtSignature is preserved through toModelMessages() for
same-model Gemini sessions. Covers tool-call and reasoning parts.

Includes Gemini 3.1 Pro model fixture and thoughtSignature
metadata constant for future test use.

Refs: Kilo-Org#6018

AI-assisted: intent and review human-driven
Co-Authored-By: Claude <noreply@anthropic.com>
@peterkc peterkc force-pushed the fix/gemini-thought-signature branch from d1c7bfe to c72a8ac Compare February 25, 2026 21:11
@peterkc peterkc marked this pull request as ready for review February 25, 2026 21:12
@kilo-code-bot
Copy link
Contributor

kilo-code-bot bot commented Feb 25, 2026

Code Review Summary

Status: No Issues Found | Recommendation: Merge

Files Reviewed (1 files)
  • packages/opencode/test/session/message-v2.test.ts

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant