[BUG] Agent Framework: Handle model response when toolUse is not accompanied by text

[pyek-bot]

Description

Fix for when model doesn't return a text along with a toolUse request.

Example failure payload:

{"metrics":{"latencyMs":2875},"output":{"message":{"content":[{"toolUse":{"input":{"index":"ss4o_logs-2025.04.18","query":{"size":10,"_source":["body","time","severityText","log.attributes","resource.attributes.service.name"],"query":{"bool":{"must":[{"wildcard":{"body":"*cart*"}},{"term":{"severityText":"ERROR"}}]}}}},"name":"SearchIndexTool","toolUseId":"tooluse_c33B609fSbCsgDeTTj8VmA"}}],"role":"assistant"}},"stopReason":"tool_use","usage":{"cacheReadInputTokenCount":0,"cacheReadInputTokens":0,"cacheWriteInputTokenCount":0,"cacheWriteInputTokens":0,"inputTokens":9000,"outputTokens":152,"totalTokens":9152}}

Related Issues

Resolves #3751

Check List

• New functionality includes testing.
• New functionality has been documented.
• API changes companion pull request created.
• Commits are signed per the DCO using --signoff.
• Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[mingshl]

is it easier just to check if LLM_RESPONSE_FILTER in parameters.keys then get the response, else you read from the TOOL_CALLS_PATH? this exception is not necessary

[mingshl]

boolean hasResponseFilter = parameters.containsKey(LLM_RESPONSE_FILTER);

if (hasResponseFilter) {
    response = JsonPath.read(dataAsMap, parameters.get(LLM_RESPONSE_FILTER));
} else {
    response = JsonPath.read(dataAsMap, parameters.get(TOOL_CALLS_PATH));
}

[pyek-bot]

both of the keys will be present in the map for all models supporting function calling:

LLM_RESPONSE_FILTER points to where we can fetch the text returned by the model. Once we fetch the text, the logic checks for whether there is a toolUse block after or not.

TOOL_CALLS_PATH points to where we can fetch a toolUse block returned by the model.

For example, these are values for claude
LLM_RESPONSE_FILTER: $.choices[0].message.content
TOOL_CALLS_PATH: $.choices[0].message.tool_calls

Before this fix, we would always look for LLM_RESPONSE_FILTER and in the case of this payload:

{"metrics":{"latencyMs":2875},"output":{"message":{"content":[{"toolUse":{"input":{"index":"ss4o_logs-2025.04.18","query":{"size":10,"_source":["body","time","severityText","log.attributes","resource.attributes.service.name"],"query":{"bool":{"must":[{"wildcard":{"body":"*cart*"}},{"term":{"severityText":"ERROR"}}]}}}},"name":"SearchIndexTool","toolUseId":"tooluse_c33B609fSbCsgDeTTj8VmA"}}],"role":"assistant"}},"stopReason":"tool_use","usage":{"cacheReadInputTokenCount":0,"cacheReadInputTokens":0,"cacheWriteInputTokenCount":0,"cacheWriteInputTokens":0,"inputTokens":9000,"outputTokens":152,"totalTokens":9152}}

It would lead to an error since there is no text field. Therefore, we then check for toolUse path. This is required as different models have a different response format. A better way to handle this would be to first check for a parent path like $.choices[0].message which might contain both the text and toolUse as children. However, this is not guaranteed as the response heavily depends on the model. Hence, the try-catch.

[codecov]

Codecov Report

Attention: Patch coverage is 85.71429% with 1 line in your changes missing coverage. Please review.

Project coverage is 78.24%. Comparing base (9e417e6) to head (0b565a3).
Report is 5 commits behind head on main.

Files with missing lines	Patch %	Lines
...nsearch/ml/engine/algorithms/agent/AgentUtils.java	85.71%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #3755      +/-   ##
============================================
+ Coverage     78.12%   78.24%   +0.12%     
- Complexity     7156     7161       +5     
============================================
  Files           631      631              
  Lines         32211    32216       +5     
  Branches       3660     3660              
============================================
+ Hits          25164    25207      +43     
+ Misses         5492     5452      -40     
- Partials       1555     1557       +2     

Flag	Coverage Δ
ml-commons	78.24% <85.71%> (+0.12%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[pyek-bot]

Added a testcase to validate 3 cases for the 3 models:

1. Only tooluse
2. Only text
3. Both tooluse and text

@ylwu-amzn can you review again please? thanks!

[pyek-bot]

Please add 3.0 backport label!