Reorder Trainer methods

[SunMarc]

What does this PR do?

This PR reorder all methods from Trainer. No modifications was done to the methods itself. With this final PR, all the major changes should be done and we can focus on simplifying the core logic + testing.

Reordering makes it hard to go back to previous PRs with git blame so a follow-up would be to create a .git-blame-ignore-revs file as this enables github to automatically skip those commits when using blame.

Here's the structure for now:

# ---- Initialization & Validation ----
 - __init__
 - _validate_args
 - create_accelerator_and_postprocess

# ---- Data Loading ----
 - get_train_dataloader
 - get_eval_dataloader
 - get_test_dataloader
 - num_examples
 - _get_dataloader
 - _get_train_sampler
 - _get_eval_sampler
 - _set_signature_columns_if_needed
 - _remove_unused_columns
 - _get_collator_with_removed_columns

# ---- Optimizer & Scheduler & Learning rate ----
 - create_optimizer_and_scheduler
 - create_optimizer
 - create_scheduler
 - get_optimizer_cls_and_kwargs
 - get_decay_parameter_names
 - _get_learning_rate

# ---- Training ----
 - train
 - _inner_training_loop
 - training_step
 - compute_loss
 - compute_loss_context_manager
 - autocast_smart_context_manager
 - _maybe_log_save_evaluate

# ---- Training Utilites ----
 - get_batch_samples
 - _get_num_items_in_batch
 - _prepare_input
 - _prepare_inputs
 - _prepare_context_parallel_inputs
 - set_initial_training_values
 - get_total_train_batch_size
 - get_sp_size
 - get_cp_size
 - get_tp_size
 - _wrap_model

# ---- Evaluation & Prediction ----
 - evaluate
 - evaluation_loop
 - predict
 - prediction_step
 - _evaluate

# ---- Checkpoint Saving ----
 - _get_output_dir
 - _save_checkpoint
 - _determine_best_metric
 - _save_rng_state
 - _save_optimizer_and_scheduler
 - _save_scaler

# ---- Checkpoint Resuming ----
 - _load_from_checkpoint
 - _load_best_model
 - _load_rng_state
 - _load_optimizer_and_scheduler
 - _load_scaler
 - _load_callback_state
 - _issue_warnings_after_load

# ---- Saving & Serialization ----
 - save_model
 - _save

# ---- Logging & Metrics ----
 - log
 - store_flos
 - floating_point_ops

# ---- Hub Integration ----
 - init_hf_repo
 - create_model_card
 - push_to_hub
 - _push_from_checkpoint
 - _finish_current_push

# ---- Hyperparameter Search ----
 - hyperparameter_search
 - call_model_init
 - _hp_search_setup
 - _report_to_hp_search
 - _tune_save_checkpoint

# ---- Callbacks ----
 - add_callback
 - pop_callback
 - remove_callback

# ---- Utilities ----
 - is_local_process_zero
 - is_world_process_zero
 - _move_model_to_device

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[qgallouedec]

/trl-ci

[qgallouedec]

if it's just a reordering, why +2,390 −2,304?

[SunMarc]

I've updated the docstring + typing also ! Also with the creation of sections, there are some comments that were added.
The first commit where the reordering was done have +2,417 -2,383. So 30 lines seems reasonable as each section add ~2 lines (empty line + title)

[qgallouedec]

lgtm! I didn't check that the re-ordering is correct, but the new order from the PR description looks good! https://github.com/huggingface/trl/actions/runs/21912922535 is red but it's not related to this PR