Follow ADDINGMODEL.MD and add DeepSeek-V4-Pro

[juanmichelini]

See https://api-docs.deepseek.com/news/news260424 for details

make sure to run the integration tests through gh

[juanmichelini]

@OpenHands please fix this

[openhands-ai]

I'm on it! juanmichelini can track my progress at all-hands.dev

[all-hands-bot]

Issue #2976 requests adding DeepSeek-V4-Pro, which is fully covered by the already-open #2947 that explicitly lists deepseek-v4-pro in its checklist and references the same DeepSeek API announcement. #2947 is more comprehensive (covers both V4-Pro and V4-Flash, eval config updates, workflow defaults). #2976 adds no new scope beyond mentioning ADDINGMODEL.MD as the process guide.

Found 1 possible duplicate issue:

1. #2947 — Add explicit DeepSeek V4 model support to eval configs

This issue will be automatically closed as a duplicate in 3 days.

• If your issue is a duplicate, please close it and 👍 the existing issue instead
• To prevent auto-closure, add a comment or 👎 this comment

This comment was created by an AI assistant (OpenHands) on behalf of the repository maintainer.

[openhands-ai]

Summary of Work Completed

I have successfully implemented the fix for issue #2976 by adding the DeepSeek-V4-Pro model configuration following the ADDINGMODEL.MD guidelines.

Changes Made (3 files modified):

1. .github/run-eval/resolve_model_config.py - Added model configuration entry:

   • Model ID: deepseek-v4-pro
   • Display name: "DeepSeek V4 Pro"
   • Model path: litellm_proxy/deepseek/deepseek-v4-pro
   • Reference URL comment to the DeepSeek API documentation

2. tests/cross/test_resolve_model_config.py - Added test function test_deepseek_v4_pro_config() to verify the model configuration

3. openhands-sdk/openhands/sdk/llm/utils/verified_models.py - Added deepseek-v4-pro to both VERIFIED_DEEPSEEK_MODELS and VERIFIED_OPENHANDS_MODELS lists

Verification:

• ✅ All 35 unit tests pass
• ✅ Pre-commit checks pass (ruff, pyright, etc.)
• ✅ Changes follow the ADDINGMODEL.MD guidelines exactly

Git Status:

• Branch pushed: openhands/add-deepseek-v4-pro
• PR created: Add DeepSeek-V4-Pro model configuration #2978
• Integration tests triggered via integration-test label and workflow_dispatch

[github-actions]

🧪 Integration Tests Results

Overall Success Rate: 0.0%
Total Cost: $0.00
Models Tested: 1
Timestamp: 2026-04-27 17:26:11 UTC

📊 Summary

Model	Overall	Tests Passed	Skipped	Total	Cost	Tokens
litellm_proxy_deepseek_deepseek_v4_pro	0.0%	0/17	2	19	$0.00	124,412

📋 Detailed Results

litellm_proxy_deepseek_deepseek_v4_pro

• Success Rate: 0.0% (0/17)
• Total Cost: $0.00
• Token Usage: prompt: 121,324, completion: 3,088, cache_read: 51,456, reasoning: 1,167
• Run Suffix: litellm_proxy_deepseek_deepseek_v4_pro_0f11afe_deepseek_v4_pro_run_N19_20260427_172449
• Skipped Tests: 2

Skipped Tests:

• t08_image_file_viewing: This test requires a vision-capable LLM model. Please use a model that supports image input.
• c01_thinking_block_condenser: Model litellm_proxy/deepseek/deepseek-v4-pro does not support extended thinking (produces reasoning items instead of thinking blocks)

Failed Tests:

• c02_hard_context_reset: Test execution failed: Conversation run failed for id=7a9826d6-88e5-48e2-9f36-d50945182728: litellm.BadRequestError: Error code: 400 - {'error': {'message': 'litellm.BadRequestError: DeepseekException - {"error":{"message":"The reasoning_content in the thinking mode must be passed back to the API.","type":"invalid_request_error","param":null,"code":"invalid_request_error"}}No fallback model group found for original model_group=deepseek/deepseek-v4-pro. Fallbacks=[{'minimax-m2.5': ['minimax-m2.5-api']}]. Received Model Group=deepseek/deepseek-v4-pro\nAvailable Model Group Fallbacks=None\nError doing the fallback: litellm.BadRequestError: DeepseekException - {"error":{"message":"The reasoning_content in the thinking mode must be passed back to the API.","type":"invalid_request_error","param":null,"code":"invalid_request_error"}}No fallback model group found for original model_group=deepseek/deepseek-v4-pro. Fallbacks=[{'minimax-m2.5': ['minimax-m2.5-api']}]', 'type': None, 'param': None, 'code': '400'}} (Cost: $0.00)
• t04_git_staging: Test execution failed: Conversation run failed for id=77ace009-9b93-42ca-827c-7fa5baef92ce: litellm.BadRequestError: Error code: 400 - {'error': {'message': 'litellm.BadRequestError: DeepseekException - {"error":{"message":"The reasoning_content in the thinking mode must be passed back to the API.","type":"invalid_request_error","param":null,"code":"invalid_request_error"}}No fallback model group found for original model_group=deepseek/deepseek-v4-pro. Fallbacks=[{'minimax-m2.5': ['minimax-m2.5-api']}]. Received Model Group=deepseek/deepseek-v4-pro\nAvailable Model Group Fallbacks=None\nError doing the fallback: litellm.BadRequestError: DeepseekException - {"error":{"message":"The reasoning_content in the thinking mode must be passed back to the API.","type":"invalid_request_error","param":null,"code":"invalid_request_error"}}No fallback model group found for original model_group=deepseek/deepseek-v4-pro. Fallbacks=[{'minimax-m2.5': ['minimax-m2.5-api']}]', 'type': None, 'param': None, 'code': '400'}} (Cost: $0.00)
• t06_github_pr_browsing: Test execution failed: Conversation run failed for id=87cbc23b-1406-46e7-905e-e992744a4174: litellm.BadRequestError: Error code: 400 - {'error': {'message': 'litellm.BadRequestError: DeepseekException - {"error":{"message":"The reasoning_content in the thinking mode must be passed back to the API.","type":"invalid_request_error","param":null,"code":"invalid_request_error"}}No fallback model group found for original model_group=deepseek/deepseek-v4-pro. Fallbacks=[{'minimax-m2.5': ['minimax-m2.5-api']}]. Received Model Group=deepseek/deepseek-v4-pro\nAvailable Model Group Fallbacks=None\nError doing the fallback: litellm.BadRequestError: DeepseekException - {"error":{"message":"The reasoning_content in the thinking mode must be passed back to the API.","type":"invalid_request_error","param":null,"code":"invalid_request_error"}}No fallback model group found for original model_group=deepseek/deepseek-v4-pro. Fallbacks=[{'minimax-m2.5': ['minimax-m2.5-api']}]', 'type': None, 'param': None, 'code': '400'}} (Cost: $0.00)
• b04_each_tool_call_has_a_concise_explanation: Test execution failed: Conversation run failed for id=c5dc9fbe-fe82-45f8-ae5f-8ff893308250: litellm.BadRequestError: Error code: 400 - {'error': {'message': 'litellm.BadRequestError: DeepseekException - {"error":{"message":"The reasoning_content in the thinking mode must be passed back to the API.","type":"invalid_request_error","param":null,"code":"invalid_request_error"}}No fallback model group found for original model_group=deepseek/deepseek-v4-pro. Fallbacks=[{'minimax-m2.5': ['minimax-m2.5-api']}]. Received Model Group=deepseek/deepseek-v4-pro\nAvailable Model Group Fallbacks=None\nError doing the fallback: litellm.BadRequestError: DeepseekException - {"error":{"message":"The reasoning_content in the thinking mode must be passed back to the API.","type":"invalid_request_error","param":null,"code":"invalid_request_error"}}No fallback model group found for original model_group=deepseek/deepseek-v4-pro. Fallbacks=[{'minimax-m2.5': ['minimax-m2.5-api']}]', 'type': None, 'param': None, 'code': '400'}} (Cost: $0.00)
• c03_delayed_condensation: Test execution failed: Conversation run failed for id=228e32f3-5a1e-4d13-98ec-5eaa587b8749: litellm.BadRequestError: Error code: 400 - {'error': {'message': 'litellm.BadRequestError: DeepseekException - {"error":{"message":"The reasoning_content in the thinking mode must be passed back to the API.","type":"invalid_request_error","param":null,"code":"invalid_request_error"}}No fallback model group found for original model_group=deepseek/deepseek-v4-pro. Fallbacks=[{'minimax-m2.5': ['minimax-m2.5-api']}]. Received Model Group=deepseek/deepseek-v4-pro\nAvailable Model Group Fallbacks=None\nError doing the fallback: litellm.BadRequestError: DeepseekException - {"error":{"message":"The reasoning_content in the thinking mode must be passed back to the API.","type":"invalid_request_error","param":null,"code":"invalid_request_error"}}No fallback model group found for original model_group=deepseek/deepseek-v4-pro. Fallbacks=[{'minimax-m2.5': ['minimax-m2.5-api']}]', 'type': None, 'param': None, 'code': '400'}} (Cost: $0.00)
• t01_fix_simple_typo: Test execution failed: Conversation run failed for id=6326528b-2b12-4298-9cb9-41d12f30b1c5: litellm.BadRequestError: Error code: 400 - {'error': {'message': 'litellm.BadRequestError: DeepseekException - {"error":{"message":"The reasoning_content in the thinking mode must be passed back to the API.","type":"invalid_request_error","param":null,"code":"invalid_request_error"}}No fallback model group found for original model_group=deepseek/deepseek-v4-pro. Fallbacks=[{'minimax-m2.5': ['minimax-m2.5-api']}]. Received Model Group=deepseek/deepseek-v4-pro\nAvailable Model Group Fallbacks=None\nError doing the fallback: litellm.BadRequestError: DeepseekException - {"error":{"message":"The reasoning_content in the thinking mode must be passed back to the API.","type":"invalid_request_error","param":null,"code":"invalid_request_error"}}No fallback model group found for original model_group=deepseek/deepseek-v4-pro. Fallbacks=[{'minimax-m2.5': ['minimax-m2.5-api']}]', 'type': None, 'param': None, 'code': '400'}} (Cost: $0.00)
• b03_no_useless_backward_compatibility: Test execution failed: Conversation run failed for id=1b1a41e2-81a9-4382-a19e-768b7ffbea08: litellm.BadRequestError: Error code: 400 - {'error': {'message': 'litellm.BadRequestError: DeepseekException - {"error":{"message":"The reasoning_content in the thinking mode must be passed back to the API.","type":"invalid_request_error","param":null,"code":"invalid_request_error"}}No fallback model group found for original model_group=deepseek/deepseek-v4-pro. Fallbacks=[{'minimax-m2.5': ['minimax-m2.5-api']}]. Received Model Group=deepseek/deepseek-v4-pro\nAvailable Model Group Fallbacks=None\nError doing the fallback: litellm.BadRequestError: DeepseekException - {"error":{"message":"The reasoning_content in the thinking mode must be passed back to the API.","type":"invalid_request_error","param":null,"code":"invalid_request_error"}}No fallback model group found for original model_group=deepseek/deepseek-v4-pro. Fallbacks=[{'minimax-m2.5': ['minimax-m2.5-api']}]', 'type': None, 'param': None, 'code': '400'}} (Cost: $0.00)
• c05_size_condenser: Test execution failed: Conversation run failed for id=110fd167-4eef-4788-be49-ed43c86dad24: litellm.BadRequestError: Error code: 400 - {'error': {'message': 'litellm.BadRequestError: DeepseekException - {"error":{"message":"The reasoning_content in the thinking mode must be passed back to the API.","type":"invalid_request_error","param":null,"code":"invalid_request_error"}}No fallback model group found for original model_group=deepseek/deepseek-v4-pro. Fallbacks=[{'minimax-m2.5': ['minimax-m2.5-api']}]. Received Model Group=deepseek/deepseek-v4-pro\nAvailable Model Group Fallbacks=None\nError doing the fallback: litellm.BadRequestError: DeepseekException - {"error":{"message":"The reasoning_content in the thinking mode must be passed back to the API.","type":"invalid_request_error","param":null,"code":"invalid_request_error"}}No fallback model group found for original model_group=deepseek/deepseek-v4-pro. Fallbacks=[{'minimax-m2.5': ['minimax-m2.5-api']}]', 'type': None, 'param': None, 'code': '400'}} (Cost: $0.00)
• t02_add_bash_hello: Test execution failed: Conversation run failed for id=15096196-2728-4737-9f90-915b0bc533a6: litellm.BadRequestError: Error code: 400 - {'error': {'message': 'litellm.BadRequestError: DeepseekException - {"error":{"message":"The reasoning_content in the thinking mode must be passed back to the API.","type":"invalid_request_error","param":null,"code":"invalid_request_error"}}No fallback model group found for original model_group=deepseek/deepseek-v4-pro. Fallbacks=[{'minimax-m2.5': ['minimax-m2.5-api']}]. Received Model Group=deepseek/deepseek-v4-pro\nAvailable Model Group Fallbacks=None\nError doing the fallback: litellm.BadRequestError: DeepseekException - {"error":{"message":"The reasoning_content in the thinking mode must be passed back to the API.","type":"invalid_request_error","param":null,"code":"invalid_request_error"}}No fallback model group found for original model_group=deepseek/deepseek-v4-pro. Fallbacks=[{'minimax-m2.5': ['minimax-m2.5-api']}]', 'type': None, 'param': None, 'code': '400'}} (Cost: $0.00)
• b02_no_oververification: Test execution failed: Conversation run failed for id=f3021a19-2e98-4611-bb1d-606603cf71ae: litellm.BadRequestError: Error code: 400 - {'error': {'message': 'litellm.BadRequestError: DeepseekException - {"error":{"message":"The reasoning_content in the thinking mode must be passed back to the API.","type":"invalid_request_error","param":null,"code":"invalid_request_error"}}No fallback model group found for original model_group=deepseek/deepseek-v4-pro. Fallbacks=[{'minimax-m2.5': ['minimax-m2.5-api']}]. Received Model Group=deepseek/deepseek-v4-pro\nAvailable Model Group Fallbacks=None\nError doing the fallback: litellm.BadRequestError: DeepseekException - {"error":{"message":"The reasoning_content in the thinking mode must be passed back to the API.","type":"invalid_request_error","param":null,"code":"invalid_request_error"}}No fallback model group found for original model_group=deepseek/deepseek-v4-pro. Fallbacks=[{'minimax-m2.5': ['minimax-m2.5-api']}]', 'type': None, 'param': None, 'code': '400'}} (Cost: $0.00)
• b05_do_not_create_redundant_files: Test execution failed: Conversation run failed for id=769317f5-face-4786-8904-0b6ee887a6c1: litellm.BadRequestError: Error code: 400 - {'error': {'message': 'litellm.BadRequestError: DeepseekException - {"error":{"message":"The reasoning_content in the thinking mode must be passed back to the API.","type":"invalid_request_error","param":null,"code":"invalid_request_error"}}No fallback model group found for original model_group=deepseek/deepseek-v4-pro. Fallbacks=[{'minimax-m2.5': ['minimax-m2.5-api']}]. Received Model Group=deepseek/deepseek-v4-pro\nAvailable Model Group Fallbacks=None\nError doing the fallback: litellm.BadRequestError: DeepseekException - {"error":{"message":"The reasoning_content in the thinking mode must be passed back to the API.","type":"invalid_request_error","param":null,"code":"invalid_request_error"}}No fallback model group found for original model_group=deepseek/deepseek-v4-pro. Fallbacks=[{'minimax-m2.5': ['minimax-m2.5-api']}]', 'type': None, 'param': None, 'code': '400'}} (Cost: $0.00)
• t09_invoke_skill: Test execution failed: Conversation run failed for id=c3906423-df8e-4334-a97c-b7ca6dff1087: litellm.BadRequestError: Error code: 400 - {'error': {'message': 'litellm.BadRequestError: DeepseekException - {"error":{"message":"The reasoning_content in the thinking mode must be passed back to the API.","type":"invalid_request_error","param":null,"code":"invalid_request_error"}}No fallback model group found for original model_group=deepseek/deepseek-v4-pro. Fallbacks=[{'minimax-m2.5': ['minimax-m2.5-api']}]. Received Model Group=deepseek/deepseek-v4-pro\nAvailable Model Group Fallbacks=None\nError doing the fallback: litellm.BadRequestError: DeepseekException - {"error":{"message":"The reasoning_content in the thinking mode must be passed back to the API.","type":"invalid_request_error","param":null,"code":"invalid_request_error"}}No fallback model group found for original model_group=deepseek/deepseek-v4-pro. Fallbacks=[{'minimax-m2.5': ['minimax-m2.5-api']}]', 'type': None, 'param': None, 'code': '400'}} (Cost: $0.00)
• t03_jupyter_write_file: Test execution failed: Conversation run failed for id=1dae2f27-f705-4367-924b-f57b8683e649: litellm.BadRequestError: Error code: 400 - {'error': {'message': 'litellm.BadRequestError: DeepseekException - {"error":{"message":"The reasoning_content in the thinking mode must be passed back to the API.","type":"invalid_request_error","param":null,"code":"invalid_request_error"}}No fallback model group found for original model_group=deepseek/deepseek-v4-pro. Fallbacks=[{'minimax-m2.5': ['minimax-m2.5-api']}]. Received Model Group=deepseek/deepseek-v4-pro\nAvailable Model Group Fallbacks=None\nError doing the fallback: litellm.BadRequestError: DeepseekException - {"error":{"message":"The reasoning_content in the thinking mode must be passed back to the API.","type":"invalid_request_error","param":null,"code":"invalid_request_error"}}No fallback model group found for original model_group=deepseek/deepseek-v4-pro. Fallbacks=[{'minimax-m2.5': ['minimax-m2.5-api']}]', 'type': None, 'param': None, 'code': '400'}} (Cost: $0.00)
• t07_interactive_commands: Test execution failed: Conversation run failed for id=e30fe187-1f54-4921-bdbd-e1055a7b7085: litellm.BadRequestError: Error code: 400 - {'error': {'message': 'litellm.BadRequestError: DeepseekException - {"error":{"message":"The reasoning_content in the thinking mode must be passed back to the API.","type":"invalid_request_error","param":null,"code":"invalid_request_error"}}No fallback model group found for original model_group=deepseek/deepseek-v4-pro. Fallbacks=[{'minimax-m2.5': ['minimax-m2.5-api']}]. Received Model Group=deepseek/deepseek-v4-pro\nAvailable Model Group Fallbacks=None\nError doing the fallback: litellm.BadRequestError: DeepseekException - {"error":{"message":"The reasoning_content in the thinking mode must be passed back to the API.","type":"invalid_request_error","param":null,"code":"invalid_request_error"}}No fallback model group found for original model_group=deepseek/deepseek-v4-pro. Fallbacks=[{'minimax-m2.5': ['minimax-m2.5-api']}]', 'type': None, 'param': None, 'code': '400'}} (Cost: $0.00)
• c04_token_condenser: Test execution failed: Conversation run failed for id=d298fae9-36de-4982-8a48-573b111ce06a: litellm.BadRequestError: Error code: 400 - {'error': {'message': 'litellm.BadRequestError: DeepseekException - {"error":{"message":"The reasoning_content in the thinking mode must be passed back to the API.","type":"invalid_request_error","param":null,"code":"invalid_request_error"}}No fallback model group found for original model_group=deepseek/deepseek-v4-pro. Fallbacks=[{'minimax-m2.5': ['minimax-m2.5-api']}]. Received Model Group=deepseek/deepseek-v4-pro\nAvailable Model Group Fallbacks=None\nError doing the fallback: litellm.BadRequestError: DeepseekException - {"error":{"message":"The reasoning_content in the thinking mode must be passed back to the API.","type":"invalid_request_error","param":null,"code":"invalid_request_error"}}No fallback model group found for original model_group=deepseek/deepseek-v4-pro. Fallbacks=[{'minimax-m2.5': ['minimax-m2.5-api']}]', 'type': None, 'param': None, 'code': '400'}} (Cost: $0.00)
• b01_no_premature_implementation: Test execution failed: Conversation run failed for id=2dccbea5-04d3-4efa-ae21-f406e544ab9d: litellm.BadRequestError: Error code: 400 - {'error': {'message': 'litellm.BadRequestError: DeepseekException - {"error":{"message":"The reasoning_content in the thinking mode must be passed back to the API.","type":"invalid_request_error","param":null,"code":"invalid_request_error"}}No fallback model group found for original model_group=deepseek/deepseek-v4-pro. Fallbacks=[{'minimax-m2.5': ['minimax-m2.5-api']}]. Received Model Group=deepseek/deepseek-v4-pro\nAvailable Model Group Fallbacks=None\nError doing the fallback: litellm.BadRequestError: DeepseekException - {"error":{"message":"The reasoning_content in the thinking mode must be passed back to the API.","type":"invalid_request_error","param":null,"code":"invalid_request_error"}}No fallback model group found for original model_group=deepseek/deepseek-v4-pro. Fallbacks=[{'minimax-m2.5': ['minimax-m2.5-api']}]', 'type': None, 'param': None, 'code': '400'}} (Cost: $0.00)
• t05_simple_browsing: Test execution failed: Conversation run failed for id=e0a2cca9-5b73-4aaa-ae4d-fe5c94420971: litellm.BadRequestError: Error code: 400 - {'error': {'message': 'litellm.BadRequestError: DeepseekException - {"error":{"message":"The reasoning_content in the thinking mode must be passed back to the API.","type":"invalid_request_error","param":null,"code":"invalid_request_error"}}No fallback model group found for original model_group=deepseek/deepseek-v4-pro. Fallbacks=[{'minimax-m2.5': ['minimax-m2.5-api']}]. Received Model Group=deepseek/deepseek-v4-pro\nAvailable Model Group Fallbacks=None\nError doing the fallback: litellm.BadRequestError: DeepseekException - {"error":{"message":"The reasoning_content in the thinking mode must be passed back to the API.","type":"invalid_request_error","param":null,"code":"invalid_request_error"}}No fallback model group found for original model_group=deepseek/deepseek-v4-pro. Fallbacks=[{'minimax-m2.5': ['minimax-m2.5-api']}]', 'type': None, 'param': None, 'code': '400'}} (Cost: $0.00)

[all-hands-bot]

[Automatic Post]: This issue has been waiting for triage. @xingyaoww @neubig @enyst, could you please take a look and add the appropriate priority label when you have a chance?