fix: handle missing </tool_call> closing tag in parse_tool_call

[dzorlu]

Summary

• Fix two critical bugs in parse_tool_call() causing training crashes and 0% reward

Bug 1: Missing closing tag (stop string issue)

When using </tool_call> as the stop string:

generator.sampling_params.stop='["</tool_call>"]'

vLLM stops generation before outputting the closing tag. Model generates:

<tool_call>{"name": "func", "arguments": {...}}<|im_end|>

But regex required both tags → every tool call rejected as "No tool call found" → 0% reward.

Evidence: Run fleet_tool_use_024ac38b - all trajectories show model generating valid tool calls but environment rejecting them.

Bug 2: Non-dict JSON type crash

json.loads() can return any valid JSON type (string, list, number, etc). If model generates:

<tool_call>"some_string"</tool_call>

Then json.loads() returns a Python string, and parsed.get("name") crashes:

AttributeError: 'str' object has no attribute 'get'

Evidence: Run fleet_tool_use_287cf214 crashed at step 10 with this exact error during training rollout.

Fix

# Handle missing closing tag
match = re.search(rf"<{tag}>(.*?)</{tag}>", action, re.DOTALL)
if not match:
    match = re.search(rf"<{tag}>(.*?)(?:<\||\Z)", action, re.DOTALL)

# Check parsed is dict before calling .get()
parsed = json.loads(match.group(1).strip())
if not isinstance(parsed, dict):
    continue

Test plan

• Unit test showing both fixes work correctly
• Run training to verify no crashes and non-zero rewards

🤖 Generated with Claude Code