Is your feature request related to a problem? Please describe.
This feature request issue is opened as a follow-up of #910
The goal is to support megatron backend for on-policy distillation.
Describe the solution you'd like
Add megatron backend support to on-policy distillation
Is your feature request related to a problem? Please describe.
This feature request issue is opened as a follow-up of #910
The goal is to support megatron backend for on-policy distillation.
Describe the solution you'd like
Add megatron backend support to on-policy distillation