[Router] use call_id instead of id for matching function calls in Responses API for Harmony

[zhaowenzi]

Motivation

Fix function call matching logic in sgl-router's Responses API implementation to comply with OpenAI specification.

Currently, the router incorrectly uses the id field to match function_call and function_call_output items. According to OpenAI's Responses API spec, function_call_output only requires call_id field (where id is optional), and matching should be done via call_id.

Refer: https://platform.openai.com/docs/guides/function-calling

for item in response.output:
    if item.type == "function_call":
        if item.name == "get_horoscope":
            # 3. Execute the function logic for get_horoscope
            horoscope = get_horoscope(json.loads(item.arguments))
            
            # 4. Provide function call results to the model
            input_list.append({
                "type": "function_call_output",
                "call_id": item.call_id,
                "output": json.dumps({
                  "horoscope": horoscope
                })
            })

Modifications

• Updated matching logic in src/routers/grpc/harmony/responses.rs to use call_id instead of id
• Updated matching logic in src/routers/grpc/harmony/builder.rs to use call_id instead of id

Accuracy Tests

1. Before this fix

2. After this fix

3. Test with OpenAI

Benchmarking and Profiling

N/A

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.

[gemini-code-assist]

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

[CatherineSue]

Since there is no ut added for this, please use Accuracy Tests section to upload the before vs after fix for this.

[CatherineSue]

Can you fix regular/responses as well? Otherwise, can we update PR title to indicate this is for Harmony only?

[zhaowenzi]

Hi @CatherineSue, I've added three screenshots to the PR description.

I think this could also fix regular/responses I am not very sure about the regular/responses. However, Harmony has another issue where 'type': 'function_call' always has the same call_id and id, though it's not urgent to fix.

Edit: updated the title

[key4ng]

pls run cargo +nightly fmt -- --check to fix the format otherwise lgtm

[zhaowenzi]

Hi @key4ng , done

[CatherineSue]

It seems the fmt check still failed

[zhaowenzi]

Hi @CatherineSue The fmt error came from the sgl-router/src/core/workflow/steps/worker_registration.rs file. It wasn’t caused by my changes, but I’ve run the command to format it now.

[CatherineSue]

Hi @CatherineSue The fmt error came from the sgl-router/src/core/workflow/steps/worker_registration.rs file. It wasn’t caused by my changes, but I’ve run the command to format it now.

I see. I think that was added by someone else. No need to change it for now.