Add Pydantic Field descriptions to improve LLM JSON schema parsing#296
Add Pydantic Field descriptions to improve LLM JSON schema parsing#296
Conversation
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #296 +/- ##
==========================================
+ Coverage 91.73% 92.09% +0.35%
==========================================
Files 19 19
Lines 1658 1695 +37
==========================================
+ Hits 1521 1561 +40
+ Misses 137 134 -3 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
@wang-boyu can i get review when u have a time |
|
Thanks for the PR. I'm still working through and cleaning up a fairly large backlog of open PRs, so reviews have been slower than I wanted. I’ve added a few tests for the changes in this PR. Merging now. |
Thanks for merging, and no worries about the timeline I really appreciate you taking the time to review. |
Summary
This PR adds explicit Pydantic
Field(description="...")definitions to theReActOutputandEventGrademodels.Why this is important
When Pydantic models are converted into JSON Schemas (for LiteLLM tool-calling or structured outputs), these field descriptions are injected directly into the prompt's schema framework.
While large models (like GPT-4) can often infer the required output just from variable names, smaller local models (like Llama-3 or Mistral via Ollama) often struggle. By supplying explicit field instructions, we significantly reduce misformatted JSON outputs, minimize hallucinations, and make
mesa-llmfar more reliable when users opt for local model providers.