litellm oss staging 04/15/2026

[krrish-berri-2]

…uests (#25800)

The Vertex AI count-tokens endpoint rejects model names that include version suffixes (@default, @20251001, etc.) with: "claude-sonnet-4-6@default is not supported for token counting"

The same model without the suffix ("claude-sonnet-4-6") works correctly.

Strip @suffix from both the model parameter and request_data["model"] in handle_count_tokens_request before sending to the API.

Relevant issues

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have Added testing in the tests/test_litellm/ directory, Adding at least 1 test is a hard requirement - see details
• My PR passes all unit tests on make test-unit
• My PR's scope is as isolated as possible, it only solves 1 specific problem
• I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

CI (LiteLLM team)

CI status guideline:

• 50-55 passing tests: main is stable with minor issues.
• 45-49 passing tests: acceptable but needs attention
• <= 40 passing tests: unstable; be careful with your merges and assess the risk.

• Branch creation CI run
  Link:

• CI run for the last commit
  Link:

• Merge / cherry-pick CI run
  Links:

Screenshots / Proof of Fix

Type

🆕 New Feature
🐛 Bug Fix
🧹 Refactoring
📖 Documentation
🚄 Infrastructure
✅ Test

Changes

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
litellm	Ready	Preview, Comment	Apr 16, 2026 2:01pm

[greptile-apps]

Greptile Summary

This is a staging PR bundling five targeted bug fixes: (1) stripping @version suffixes from Vertex AI partner-model count-tokens requests, (2) propagating Ollama done_reason='length' as finish_reason, (3) fixing allowed_openai_params forwarding None for unset params, (4) adding a user_api_key_alias fallback for Noma v2 application_id, and (5) syncing the in-memory credential list after a PATCH update. Every fix is accompanied by targeted unit tests.

Confidence Score: 5/5

Safe to merge — all five fixes are well-scoped, isolated, and covered by mock-only unit tests.

All findings are P2 or below. The primary fix (Vertex AI suffix stripping) is simple, correct, and thoroughly tested. Remaining changes are equally targeted with no regression risk identified.

No files require special attention.

Important Files Changed

Filename	Overview
litellm/llms/vertex_ai/vertex_ai_partner_models/count_tokens/handler.py	Adds _strip_version_suffix static method and applies it to both model and request_data["model"] before the count-tokens API call; uses dict-spread immutable copy per style guide.
tests/test_litellm/llms/vertex_ai/vertex_ai_partner_models/count_tokens/test_count_tokens_location.py	Adds TestCountTokensVersionSuffixStripping class with four mock-only tests covering @default, @date, no-suffix, and end-to-end stripping of request_data["model"].
litellm/llms/ollama/chat/transformation.py	Propagates Ollama done_reason='length' as OpenAI finish_reason='length' in both streaming and non-streaming paths via map_finish_reason.
litellm/utils.py	Fixes _apply_openai_param_overrides to only forward params that the caller actually sent, preventing None values from reaching provider SDKs as unknown kwargs.
litellm/proxy/guardrails/guardrail_hooks/noma/noma_v2.py	Adds user_api_key_alias as final fallback for application_id when neither dynamic params nor the configured value is available.
litellm/proxy/credential_endpoints/endpoints.py	PATCH handler now syncs in-memory credential_list after DB update, handling rename by removing the old entry before upserting the new one.
litellm/llms/bedrock/chat/converse_transformation.py	No logic changes; new unit tests added for the existing _transform_usage method covering both base and reasoning-content token accounting.

Sequence Diagram

sequenceDiagram
    participant C as Caller
    participant H as handle_count_tokens_request
    participant S as _strip_version_suffix
    participant API as Vertex AI count-tokens API

    C->>H: model="claude-sonnet-4-6@default"<br/>request_data={"model":"claude-sonnet-4-6@default", ...}
    H->>S: _strip_version_suffix("claude-sonnet-4-6@default")
    S-->>H: "claude-sonnet-4-6"
    H->>H: model = "claude-sonnet-4-6"
    H->>S: _strip_version_suffix(request_data["model"])
    S-->>H: "claude-sonnet-4-6"
    H->>H: request_data = {**request_data, "model": "claude-sonnet-4-6"}
    H->>API: POST /count-tokens {"model": "claude-sonnet-4-6", "messages": [...]}
    API-->>H: {"input_tokens": 42}
    H-->>C: {"input_tokens": 42, "tokenizer_used": "vertex_ai_partner_models"}

Loading

_{Reviews (6): Last reviewed commit: "Fix code qa" | Re-trigger Greptile}

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
5 out of 6 committers have signed the CLA.

✅ Sameerlite
✅ xr843
✅ dkindlund
✅ jarsever
✅ bse-ai
❌ kuun993
_{You have signed the CLA already but the status is still pending? Let us recheck it.}