Issues with Gemini quickstart flow (retry behavior, model errors, and tool usage

[MohidNaghman1]

Hi, I found a few issues while testing the quickstart flow with Gemini that seem worth addressing for first-run usability.

Tagging @jackiekazil and @colinfrisch in case this is helpful context.

---

Observed issues

Unbounded retry behavior in ModuleLLM

ModuleLLM.generate() / agenerate() retry transient LiteLLM errors such as APIConnectionError, Timeout, and RateLimitError using exponential backoff, but without a clear stop condition.

In practice, this can make the script appear stuck while LiteLLM repeatedly prints its help/debug banner.

---

Outdated Gemini model names causing 404 errors

Older Gemini model names such as:

• gemini-pro
• gemini-1.5-pro

now return 404 errors from the Google API.

This currently surfaces as a raw LiteLLM traceback, which makes it harder for users to understand that the issue is an outdated model name rather than a runtime failure.

---

Missing actionable error message for NotFoundError

It may be helpful to rewrite Gemini NotFoundError into a more actionable message, for example:

Model 'gemini/gemini-1.5-pro' was not found. This Gemini model may no longer be available via this API. Try 'gemini/gemini-2.0-flash' instead.

---

Tool usage in quickstart (tool_choice="required")

It may also be worth reviewing whether tool_choice="required" in the CoT executor step is ideal for minimal examples.

In practice, it can make simple quickstart usage more dependent on tool-calling behavior than users might expect, and may introduce unnecessary failures for providers that do not fully support tool calling.

---

Possible improvements

• Add a bounded retry policy in ModuleLLM so transient failures eventually surface clearly (instead of potentially looping indefinitely).
• Rewrite Gemini NotFoundError responses into a more actionable error message with a suggested replacement model.
• Update quickstart/default examples to use a currently available Gemini model (e.g. gemini-2.0-flash, depending on API availability).
• Optionally revisit whether tool_choice="required" is appropriate for minimal quickstart examples.

---

Example of clearer error messaging

Something like:

Model 'gemini/gemini-1.5-pro' was not found. This Gemini model may no longer be available via this API. Try 'gemini/gemini-2.0-flash' instead.

---

If this direction sounds reasonable, I’d be happy to work on a fix and open a PR.