Support most AI providers

[gvanrossum-ms]

We should completely redesign the way models (both embedding and LLM) are accessed. Most likely we can use Pydantic.AI for this. (We could use it instead of TypeChat too, but when I tried that, results were worse than TypeChat, and I didn't have the time to delve deeper into the reasons.)

• The AsyncEmbeddingModel class should wrap a Pydantic.AI embedding API (assuming one exists) instead of going straight to the openai package.
• The TypeChatModel (sp?) class should wrap a Pydantic.AI model class.
• Environment variables should be redesigned to support setting the provider and model name; but you should also be allowed to create your own Pydantic.AI-based implementations of those classes.
• The current set of environment variables (esp. for AZURE_OPENAI) should still be supported.

[Abdur-rahmaanJ]

@gvanrossum-ms PyDantic Ai does not seem to have an embedding class

[wirthual]

LiteLLM might also be an option for this. It also supports embeddings and Azure Openai

[gvanrossum]

@gvanrossum-ms PyDantic Ai does not seem to have an embedding class

Ah, you're right: pydantic/pydantic-ai#58

[gvanrossum]

Oh, looks impressive.

…

--Guido (mobile)

On Thu, Nov 13, 2025 at 08:33 wirthual ***@***.***> wrote: *wirthual* left a comment (microsoft/typeagent-py#84) <#84 (comment)> LiteLLM <https://github.com/BerriAI/litellm> might also be an option for this. It also supports embeddings and Azure Openai <https://docs.litellm.ai/docs/providers/azure/> — Reply to this email directly, view it on GitHub <#84 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAWCWMW7RNBRPLJTDCQLBXD34SXF5AVCNFSM6AAAAACL5JCXBOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTKMRYGY2TONRQGI> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[Abdur-rahmaanJ]

@gvanrossum Some returns on experiment: Asked Google-cli to integrate litellm in such a way that it passes all tests. It did and added a my usecase, Gemini example to the repo. But it was unusable.

So, i think it's worth it understanding how the code works before jumping in.