Universal Local Model Support via Ollama/OpenAI-Compatible Providers

[magudeshhmw]

Problem Statement

Gemini CLI currently only supports Google Gemini models and limited local model options (LiteRT-LM).
Many users want to run AI workflows entirely offline using local LLM providers like Ollama, LM
Studio, or LocalAI.

Proposed Solution

Extend the ContentGenerator interface to support multiple local model providers:

Implementation Plan

1. New AuthTypes: Add AuthType.OLLAMA and AuthType.OPENAI_COMPATIBLE to the enum
2. New ContentGenerators: Create OllamaContentGenerator and OpenAiCompatibleContentGenerator
   implementing the ContentGenerator interface
3. Factory Update: Modify createContentGenerator() to route to the appropriate provider based
   on AuthType
4. Configuration: Add config options for local endpoint URLs (e.g., OLLAMA_HOST,
   OPENAI_BASE_URL)

Timeline (350 hours)

• Phase 1 (Weeks 1-3): Ollama API client implementation
• Phase 2 (Weeks 4-6): OpenAI-compatible API client
• Phase 3 (Weeks 7-9): Configuration system and CLI integration
• Phase 4 (Weeks 10-12): Testing, streaming support, documentation

Relevant Experience

I've built offline AI applications using Ollama + FastAPI, and I'm familiar with the OpenAI API
specification.

Questions for Maintainers

1. Is this aligned with the project's roadmap?
2. Are there any existing efforts in this direction we should build upon?
3. What's the preferred approach for configuration (env vars vs config file)?

Looking forward to feedback!