Add response_format parameter support for structured output

[fede-kamel]

Summary

Implements response_format parameter to enable JSON mode and JSON schema output formatting for both Meta Llama and Cohere models. This allows users to leverage structured output capabilities via with_structured_output() and direct response_format configuration.

Changes

• Add response_format field to OCIGenAIBase with comprehensive documentation
• Implement response_format handling in ChatOCIGenAI._prepare_request method
• Support response_format via class initialization, bind(), and model_kwargs
• Add comprehensive unit tests covering all configuration methods
• Add integration tests validating end-to-end functionality with real API calls

Implementation Details

The implementation ensures response_format is properly passed to both GenericChatRequest (Meta Llama) and CohereChatRequest (Cohere) models by adding the parameter to the model_kwargs in the _prepare_request method.

Testing

• Unit Tests: 9/9 tests passing

  • Class-level response_format
  • Response_format via bind()
  • Response_format via model_kwargs
  • Pass-through to API for Generic and Cohere models
  • with_structured_output integration (json_mode and json_schema)

• Integration Tests: 10/10 tests passing with real OCI API calls

  • JSON mode with Meta Llama and Cohere
  • Structured output with Pydantic models
  • JSON schema mode with Meta Llama
  • Complex nested structures

Fixes

Fixes #33

Breaking Changes

None - this is a new feature that is fully backward compatible.

[fede-kamel]

Hi @YouNeedCryDear! 👋

I hope you're doing well! I wanted to gently follow up on this PR that adds response_format parameter support for structured output.

Why this matters:
This implements a feature requested in issue #33 and enables users to leverage JSON mode and JSON schema output formatting for both Meta Llama and Cohere models through LangChain's standard with_structured_output() method.

What's been validated:

• ✅ All 9 unit tests passing
• ✅ All 10 integration tests passing with live OCI API calls
• ✅ Fully backward compatible (no breaking changes)
• ✅ Comprehensive documentation and examples included

This follows the same parameter pattern as other recent enhancements and should be a straightforward addition to the SDK's capabilities.

Would you have a chance to take a look when you get a moment? I'm happy to address any feedback or questions you might have!

Thanks so much for maintaining this project! 🙏

[YouNeedCryDear]

@fede-kamel Why do we need response_format in the class? Can it be inside the model_kwargs? I don't see other vendors doing this.

[fede-kamel]

Response to Review Feedback

@YouNeedCryDear
Thank you for the feedback! You're absolutely right that having response_format as a class field is inconsistent with other LangChain providers. I've addressed this concern by removing the class field and ensuring the functionality works through model_kwargs like other providers.

Changes Made

1. Removed response_format class field from OCIGenAIBase - no longer needed as a class parameter
2. Removed special handling in ChatOCIGenAI._prepare_request() that checked for the class field
3. Updated all tests to use model_kwargs={"response_format": {...}} instead of class initialization
4. Maintained full functionality - users can still set response format, just via model_kwargs

Why This Approach

This makes the OCI implementation consistent with other LangChain providers (OpenAI, Anthropic, etc.) where response format parameters are handled through model_kwargs rather than dedicated class fields. Users now have the same interface pattern across all providers.

Real-World Testing

To verify the functionality still works correctly after these changes, I tested with real OCI Generative AI API calls:

With Meta Llama 4 model:

llm = ChatOCIGenAI(
    model_id="meta.llama-4-maverick-17b-128e-instruct-fp8",
    model_kwargs={"response_format": {"type": "JSON_OBJECT"}},
    # ... other config
)

response = llm.invoke("Create JSON with name and age for Alice who is 30")
# Returns: {"name": "Alice", "age": 30}

Tested successfully with:

• ✅ xAI Grok models
• ✅ Meta Llama 3.3 models
• ✅ Meta Llama 4 models

The response format functionality works perfectly and enforces proper JSON output from the models.

This change maintains backward compatibility while providing a cleaner, more consistent API design. Users can still access all the structured output features through with_structured_output() and direct model_kwargs configuration.

[fede-kamel]

Hi @YouNeedCryDear, just following up—this PR is ready for another look whenever you have a moment. Appreciate your previous feedback!

[YouNeedCryDear]

@fede-kamel Sorry I don't see any new changes after our discussion.

[fede-kamel]

Hi @YouNeedCryDear! 👋

The changes have now been pushed to the PR. I apologize for the confusion - the commit addressing your feedback was created locally but wasn't pushed to GitHub until now.

Changes Applied:

• ✅ Removed response_format class field from OCIGenAIBase
• ✅ Removed special handling in ChatOCIGenAI._prepare_request()
• ✅ Updated all tests to use model_kwargs={"response_format": ...} instead
• ✅ Rebased on latest main branch

The functionality remains the same, but now follows the same pattern as other LangChain providers by handling response_format through model_kwargs rather than as a dedicated class field.

You should now see two commits in the PR:

1. The original feature implementation
2. The changes addressing your feedback

Ready for another review when you have a chance. Thanks for your patience!

[fede-kamel]

The CI failures in libs/oracledb are due to GitHub runner disk space issues ("No space left on device"), not code issues.

All libs/oci checks pass successfully:

• ✅ make lint Create langchain-oracledb Package for Oracle AI Vector Search #3.12 - pass
• ✅ make lint Create langchain-oracledb Package for Oracle AI Vector Search #3.9 - pass
• ✅ make test Create langchain-oracledb Package for Oracle AI Vector Search #3.12 - pass
• ✅ make test Create langchain-oracledb Package for Oracle AI Vector Search #3.9 - pass

The oracledb failures are unrelated to this PR (no oracledb files were modified). Could you re-run the failed checks when you get a chance? Thanks!

@YouNeedCryDear