[BUG] Pydantic Validation error while saving in Memory

[sayoojbkumar]

Description

Getting the following Pydantic error while saving to memory.

ERROR:root:OpenAI API call failed: 1 validation error for TaskEvaluation
  Invalid JSON: expected value at line 1 column 1 [type=json_invalid, input_value='```json\n{\n  "assessmen...  ]\n    }\n  ]\n}\n```', input_type=str]
    For further information visit https://errors.pydantic.dev/2.11/v/json_invalid

[2026-02-18 18:17:49][ERROR]: Failed to add to long term memory: Failed to convert text into a Pydantic model due to validation error: 1 validation error for TaskEvaluation
  Invalid JSON: expected value at line 1 column 1 [type=json_invalid, input_value='```json\n{\n  "assessmen...  ]\n    }\n  ]\n}\n```', input_type=str]
    For further information visit https://errors.pydantic.dev/2.11/v/json_invalid

Steps to Reproduce

1. Run the POC code below.
2. Go through the console logs.

from crewai import Agent, Task, Crew, Process
import os
from dotenv import load_dotenv

# Define agents
load_dotenv()
researcher = Agent(
    role="Senior Researcher",
    goal="Find comprehensive information on the given topic",
    backstory="You are an experienced researcher skilled at finding relevant information.",
    verbose=True,
)

writer = Agent(
    role="Report Writer",
    goal="Create a well-structured report based on research findings",
    backstory="You are a skilled writer who turns research into clear reports.",
    verbose=True,
)

# Define tasks
research_task = Task(
    description="Research the topic: {topic}",
    expected_output="A list of key findings about the topic.",
    agent=researcher,
)

writing_task = Task(
    description="Write a report based on the research findings.",
    expected_output="A comprehensive report in markdown format.",
    agent=writer,
    context=[research_task],
)

# Create crew with memory enabled and a custom embedding model
crew = Crew(
    agents=[researcher, writer],
    tasks=[research_task, writing_task],
    process=Process.sequential,
    memory=True,
    embedder={
        "provider": "openai",
        "config": {
            "model_name": "text-embedding-3-small",
            "api_key":os.getenv("API_KEY"),
            "api_base": os.getenv("BASE_URL")
        },
    },
    verbose=True,
)

# Kick off the crew
result = crew.kickoff(inputs={"topic": "AI in Healthcare"})
print(result.raw)

Expected behavior

CrewAI should successfully parse LLM output and save task evaluations to long-term memory without validation errors.

Screenshots/Code snippets

Operating System

macOS Sonoma

Python Version

3.10

crewAI Version

1.9.3

crewAI Tools Version

1.9.3

Virtual Environment

Venv

Evidence

.

Possible Solution

.

Additional context

I am already aware of the following issues, and I am on the latest version of CrewAI. It looks like this hasn’t been resolved yet:

#3915

[Vidit-Ostwal]

Which LLM are you using?

[sayoojbkumar]

Which LLM are you using?

Hey @Vidit-Ostwal , iam using gemini 3 pro

[Vidit-Ostwal]

Mind bumping it up, to the latest version, they have unified all the memory.
Can debug if required post that #4484

[sayoojbkumar]

A ValidationError occurs during the task evaluation process because the LLM returns a JSON response wrapped in Markdown code blocks. The pydantic validator expects a raw JSON string and fails when it encounters the leading characters: ```json.

Root Cause Analysis:
The error propagates through the following execution flow:

1. TaskEvaluator.evaluate(): Initiates the evaluation by calling the Converter class with specific queries and instructions.
2. Converter.to_pydantic(): Triggers an LLM call to retrieve structured data.
3. LLM Response: The model returns the result formatted as:

```JSON \n \n {"assessment": "..."}```

4. model_validate_json(response): This method is called within Converter, but it fails because the response variable contains the Markdown backticks and language identifier rather than just the object.

Here is a simplified POC for the same:

from pydantic import BaseModel

class User(BaseModel):
    id: int
    name: str

# The "One-Liner" in action
user = User.model_validate_json('{"id": 42, "name": "Jerry"}')
user = User.model_validate_json('```json{"id": 42, "name": "Jerry"}```')

[sayoojbkumar]

Just saw the devin-ai-integration PR:
#4510

It looks like Devin AI has correctly identified the issue, and the PR looks good.
Could someone from the maintainers please take a look at this?
@greysonlalonde @joaomdmoura

[Vidit-Ostwal]

Just saw the devin-ai-integration PR: #4510

It looks like Devin AI has correctly identified the issue, and the PR looks good. Could someone from the maintainers please take a look at this? @greysonlalonde @joaomdmoura

That's great

Mind checking this output_json or output_pydantic attribute here
https://docs.crewai.com/en/concepts/tasks

I think that re-inforces the response into pydantic or json format.

[sayoojbkumar]

Mind checking this output_json or output_pydantic attribute here https://docs.crewai.com/en/concepts/tasks

I think that re-inforces the response into pydantic or json format.

These concepts are related to tasks where the output needs to be in a specific JSON or Pydantic format. Those are optional attributes and have nothing to do with the error I’m facing.

[Vidit-Ostwal]

I can see TaskEvaluation is being used, are you by any chance training the cre.
https://docs.crewai.com/en/concepts/training?