Skip to content

fix: ensure that we check for pad_token and not assume pad_token==eos…#120

Merged
parthchadha merged 2 commits intomainfrom
pchadha/fix-pad-token
Apr 2, 2025
Merged

fix: ensure that we check for pad_token and not assume pad_token==eos…#120
parthchadha merged 2 commits intomainfrom
pchadha/fix-pad-token

Conversation

@parthchadha
Copy link
Copy Markdown
Contributor

…_token

What does this PR do ?

There were wrong assumptions in grpo and sft that pad_token will be the same as eos_token.

Issues

Closes #119.

Usage

  • You can potentially add a usage example below
# Add a code snippet demonstrating how to use this 

Before your PR is "Ready for review"

Pre checks:

  • Make sure you read and followed Contributor guidelines
  • Did you write any new necessary tests?
  • Did you run the unit tests and functional tests locally? Visit our Testing Guide for how to run tests
  • Did you add or update any necessary documentation? Visit our Document Development Guide for how to write, build and test the docs.

Additional Information

  • ...

…_token

Signed-off-by: Parth Chadha <pchadha@nvidia.com>
@parthchadha
Copy link
Copy Markdown
Contributor Author

#121 will add functional tests. Unit tests won't catch this failure since we need to invoke the grpo or sft algorithm for an iteration to catch this issue.

@parthchadha parthchadha requested review from SahilJain314, ashors1 and terrykong and removed request for terrykong April 2, 2025 18:23
Comment thread nemo_reinforcer/algorithms/grpo.py
Comment thread nemo_reinforcer/algorithms/grpo.py Outdated
Signed-off-by: Parth Chadha <pchadha@nvidia.com>
@parthchadha parthchadha force-pushed the pchadha/fix-pad-token branch from 728e534 to 2983ca7 Compare April 2, 2025 18:44
@parthchadha parthchadha enabled auto-merge (squash) April 2, 2025 21:29
@parthchadha parthchadha merged commit afefe66 into main Apr 2, 2025
11 checks passed
@parthchadha parthchadha deleted the pchadha/fix-pad-token branch April 2, 2025 21:57
KiddoZhu pushed a commit that referenced this pull request May 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

qwen1.5b-instruct grpo padding failure

3 participants