Skip to content

Add Pydantic Field descriptions to improve LLM JSON schema parsing#296

Merged
wang-boyu merged 3 commits intomesa:mainfrom
gamal1osama:enhance-pydantic-schemas
Apr 15, 2026
Merged

Add Pydantic Field descriptions to improve LLM JSON schema parsing#296
wang-boyu merged 3 commits intomesa:mainfrom
gamal1osama:enhance-pydantic-schemas

Conversation

@gamal1osama
Copy link
Copy Markdown
Contributor

Summary
This PR adds explicit Pydantic Field(description="...") definitions to the ReActOutput and EventGrade models.

Why this is important
When Pydantic models are converted into JSON Schemas (for LiteLLM tool-calling or structured outputs), these field descriptions are injected directly into the prompt's schema framework.

While large models (like GPT-4) can often infer the required output just from variable names, smaller local models (like Llama-3 or Mistral via Ollama) often struggle. By supplying explicit field instructions, we significantly reduce misformatted JSON outputs, minimize hallucinations, and make mesa-llm far more reliable when users opt for local model providers.

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai bot commented Apr 13, 2026

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 38cc16e3-f4c4-4d85-a0af-f1c3536696e1

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@codecov
Copy link
Copy Markdown

codecov bot commented Apr 13, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 92.09%. Comparing base (c014479) to head (c880920).
⚠️ Report is 22 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #296      +/-   ##
==========================================
+ Coverage   91.73%   92.09%   +0.35%     
==========================================
  Files          19       19              
  Lines        1658     1695      +37     
==========================================
+ Hits         1521     1561      +40     
+ Misses        137      134       -3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@gamal1osama
Copy link
Copy Markdown
Contributor Author

@wang-boyu can i get review when u have a time

@wang-boyu wang-boyu added the maintenance Release notes label label Apr 15, 2026
@wang-boyu
Copy link
Copy Markdown
Member

Thanks for the PR. I'm still working through and cleaning up a fairly large backlog of open PRs, so reviews have been slower than I wanted.

I’ve added a few tests for the changes in this PR. Merging now.

@wang-boyu wang-boyu merged commit 04b1068 into mesa:main Apr 15, 2026
14 checks passed
@gamal1osama
Copy link
Copy Markdown
Contributor Author

Thanks for the PR. I'm still working through and cleaning up a fairly large backlog of open PRs, so reviews have been slower than I wanted.

I’ve added a few tests for the changes in this PR. Merging now.

Thanks for merging, and no worries about the timeline I really appreciate you taking the time to review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

maintenance Release notes label

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants