Fix policy type check in eval_policy function to use unwrapped model

[andrearosasco]

Title

When using accelerate to run the training the eval function check the policy type which being wrapped raises and exception. This PR fixes it.

Type / Scope

• Type: Bug
• Scope: eval

How was this tested (or how to run locally)

• Running a training with periodic evaluations

Checklist (required before merge)

• Linting/formatting run (pre-commit run -a)
• All tests pass locally (pytest)
• Documentation updated
• CI is green

[Copilot]

Pull request overview

Fixes evaluation-time policy type checking when the policy model is wrapped by accelerate (e.g., DDP/FSDP wrappers), by unwrapping before performing isinstance checks.

Changes:

• Add extract_model_from_parallel and use it to obtain unwrapped_policy for the PreTrainedPolicy / PeftModel type checks in eval_policy.
• Update the raised ValueError message to report the unwrapped model type.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

from accelerate import accelerator appears to be an invalid/unused import (it’s not referenced anywhere in this file, and the Accelerate API typically exposes Accelerator, not accelerator). This will likely raise an ImportError at runtime. Remove this import, or replace it with the correct symbol if you intended to use it.

Suggested change

from accelerate import accelerator