[train] Enable v2 for ray/train/tests

[justinvyu]

---

Note

Enable Train v2 across CI with new CPU/GPU jobs, migrate tests to v2 with env flags and tag updates, remove legacy Tune examples/utilities, and fix docs/BUILD configurations.

• CI (Buildkite):
  • Split Train tests into v1 and v2, add dedicated GPU job for train_v2_gpu.
  • Standardize tags (drop gpu_only, use gpu), adjust include/except tag filters.
• Tests (BUILD and test code):
  • Gate tests with RAY_TRAIN_V2_ENABLED (on/off per test); retag v2 tests (train_v2, train_v2_gpu).
  • Replace deprecated imports/usages; simplify some assertions to just run fit().
  • Remove legacy tests/examples (e.g., Tune Torch/TensorFlow tuning examples, datasets_train, utils), and adjust v2 HF/XGBoost tests (steps/save intervals).
  • Data tests retagged from team:ml/ray_air to team:data.
• Code cleanup:
  • Remove construct_path from train/_internal/utils.py and its unit test; update pydoclint baseline accordingly.
• Docs:
  • Fix outdated example links and remove obsolete Tune example pages/links.
• Bazel BUILD:
  • Widespread tag/env updates; add/remove tests to reflect v2 enablement and GPU tagging changes.

^{Written by Cursor Bugbot for commit b88db74. This will update automatically on new commits. Configure here.}

Legacy V1 tests

This PR explicitly marked legacy v1 tests by running them with RAY_TRAIN_V2_ENABLED=0. See the BUILD files.

Followups

There are some tests where some tests should be migrated to V2 and some other tests in the file are legacy V1 tests. I've marked these as TODOs in the BUILD files to migrate in a follow-up.

[gemini-code-assist]

Code Review

This pull request introduces significant changes to enable Ray Train v2. The modifications include updating CI configurations, migrating numerous tests to use the v2 API by setting RAY_TRAIN_V2_ENABLED=1, and cleaning up obsolete v1 examples and tests. The changes are generally consistent and well-structured.

My main concern is the removal of tests in python/ray/train/tests/test_gpu_auto_transfer.py that verified critical GPU-related functionality. While the tests used v1 APIs, the underlying features are still important for v2. I've left a specific comment on this.

Otherwise, the migration strategy seems sound, with clear use of environment variables to toggle between API versions and TODOs marking areas for future work.