Skip to content

add try-except around json loaders#177

Open
IlamaranMagesh wants to merge 3 commits intomesa:mainfrom
IlamaranMagesh:fix-json-load
Open

add try-except around json loaders#177
IlamaranMagesh wants to merge 3 commits intomesa:mainfrom
IlamaranMagesh:fix-json-load

Conversation

@IlamaranMagesh
Copy link
Copy Markdown

Pre-PR Checklist

  • This PR is a bug fix, not a new feature or enhancement.

Summary

Closes #173

This PR has added try-except block for the JSON loaders in all the files except for the files in tests folder.

Bug / Issue

Issue - #173

There were couple of JSON objects parsed without any validation. This can crash the system when the response received from the APIs or the file that has been loaded is either not a JSON or corrupted.

Implementation

Added try-except blocks across files, except in the tests folder. It can be updated, if it's required by the maintainers

Testing

  • Run pre-commit
  • Run Pytest
Pytest Coverage Report
================================ tests coverage ================================
______________ coverage: platform darwin, python 3.12.12-final-0 _______________

Name                                        Stmts   Miss  Cover
---------------------------------------------------------------
mesa_llm/__init__.py                           13      0   100%
mesa_llm/llm_agent.py                         114      9    92%
mesa_llm/memory/episodic_memory.py            103      6    94%
mesa_llm/memory/lt_memory.py                   63      4    94%
mesa_llm/memory/memory.py                      66      5    92%
mesa_llm/memory/st_lt_memory.py                73      8    89%
mesa_llm/memory/st_memory.py                   41      2    95%
mesa_llm/module_llm.py                         54      1    98%
mesa_llm/parallel_stepping.py                  69     16    77%
mesa_llm/reasoning/cot.py                      68      4    94%
mesa_llm/reasoning/react.py                    62      8    87%
mesa_llm/reasoning/reasoning.py                41      9    78%
mesa_llm/reasoning/rewoo.py                    72      2    97%
mesa_llm/recording/agent_analysis.py          281     30    89%
mesa_llm/recording/record_model.py             45      4    91%
mesa_llm/recording/simulation_recorder.py      79      1    99%
mesa_llm/tools/inbuilt_tools.py                78      5    94%
mesa_llm/tools/tool_decorator.py              151     34    77%
mesa_llm/tools/tool_manager.py                 85      6    93%
---------------------------------------------------------------
TOTAL                                        1558    154    90%
273 passed, 22 warnings in 1.55s

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai bot commented Mar 8, 2026

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: b9134dde-ac0a-4cbf-afeb-4ee23b5833ed

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@Komatlakarthik
Copy link
Copy Markdown

Hi @IlamaranMagesh, great work on adding these validations. After a senior-level review of the changes in this PR, I've identified a few critical edge cases that should be addressed to ensure reliability:

  1. TypeError in JSON Parsing: In both episodic_memory.py and react.py, json.loads() is called on rsp.choices[0].message.content. If the model returns None (common in cases of filtered content or API errors), this will raise a TypeError. We should verify that the content is a string before parsing.
  2. KeyError on Response Access: The code currently assumes the "grade" key will always be present in the parsed JSON. If the LLM output is malformed or missing this key, the agent will crash with a KeyError. A safer approach would be using .get("grade") or adding a schema validation step.
  3. Import Consistency: Double-check that the style function is imported correctly in all files where it was added.

Addressing these would significantly harden the error handling for LLM responses!

@IlamaranMagesh
Copy link
Copy Markdown
Author

Thanks for the review.

Updates made:

  1. Caught the TypeError and handled it gracefully, but this brings the question. Should the JSON parse error also be handled gracefully? Need some clarity here.

  2. Updated .get

  3. All files already import terminal-style. The agent-analysis.py aliases it since there is an another usage of variable name style

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: no validation of JSON object returned by LLM models

2 participants