Return type mismatch in training script. train_fold.py

[rexxy-sasori]

Thank you for uploading your training script. Just one question. I am new to verl but I could be wrong. The AgentLoop structure returns AgentLoopOutput as far as I know. However, the process_item of fold agent returns DataProto. Could there be a mismatch in this case??

[sunnweiwei]

Hi, there may have been some stale code, but the current version returns AgentLoopOutput:
https://github.com/sunnweiwei/FoldAgent/blob/main/agents/fold_agent.py

[rexxy-sasori]

Thanks. Seems that I forked your code so early that I was referencing stale code.

Another question though. I try to set up my own ReAct agent with GRPO and the Seed-OSS model. I am wondering whether you are giving the trajectory any intermediate reward, like +0.5 for actually using a tool. The reason why I am asking is that I noticed during training the model is simulating the search results without actually running any searches, even though I used the same system prompt + tool description. Is this expected behavior during first few steps of training?