[Data] Deprecate V1 cluster autoscaler in favor of V2 by ryankert01 · Pull Request #60474 · ray-project/ray

ryankert01 · 2026-01-24T13:39:29Z

Description

This document describes how to verify the V1 cluster autoscaler deprecation.

Added @Deprecated annotation to DefaultClusterAutoscaler in default_cluster_autoscaler.py
Changed default from V1 to V2 in __init__.py

Related issues

Closes #60459

Additional information

Testing

Test V1 via env var emits deprecation warning:

RAY_DATA_CLUSTER_AUTOSCALER=V1 python -c "
import warnings
warnings.simplefilter('always')
from unittest.mock import MagicMock
from ray.data._internal.cluster_autoscaler import create_cluster_autoscaler

mock_data_context = MagicMock()
mock_data_context.execution_options.resource_limits = None
autoscaler = create_cluster_autoscaler(MagicMock(), MagicMock(), mock_data_context, execution_id='test')
print(f'Created: {type(autoscaler).__name__}')
"

...RayDeprecationWarning: This API is deprecated and may be removed in future Ray releases...
DefaultClusterAutoscaler (V1) is deprecated. Use DefaultClusterAutoscalerV2 instead by setting RAY_DATA_CLUSTER_AUTOSCALER=V2 or using the default.
Created: DefaultClusterAutoscaler

Test V2 (default) has no deprecation warning:

python -c "
import warnings
warnings.filterwarnings('always', category=DeprecationWarning)
from unittest.mock import MagicMock
from ray.data._internal.cluster_autoscaler import create_cluster_autoscaler

mock_data_context = MagicMock()
mock_data_context.execution_options.resource_limits = None
autoscaler = create_cluster_autoscaler(MagicMock(), MagicMock(), mock_data_context, execution_id='test')
print(f'Created: {type(autoscaler).__name__}')
"

Created: DefaultClusterAutoscalerV2

gemini-code-assist

Code Review

This pull request correctly deprecates the V1 cluster autoscaler in favor of V2. The default version is now V2, and using V1 will trigger a deprecation warning, as demonstrated by the tests in the description. The changes are straightforward and well-implemented.

As a suggestion for a follow-up, you might consider updating the existing unit tests for DefaultClusterAutoscaler to handle the new deprecation warnings (e.g., using pytest.warns) to keep the test logs clean. This is not a blocking issue for this PR.

Signed-off-by: Ryan Huang <ryankert01@gmail.com>

bveeramani

I think we still need to add a warnings.warn in the v1 constructor. The message for the deprecated decorator is for docs only, IiRC

ryankert01 · 2026-01-25T04:06:48Z

@bveeramani I initially think it will do it there. Is that okay?

ray/python/ray/util/annotations.py

Lines 164 to 169 in bcb27af

    
           def patched_init(*args, **kwargs): 
        
               warnings.warn(warning_message, RayDeprecationWarning, stacklevel=2) 
        
               return obj_init(*args, **kwargs) 
        
           obj.__init__ = patched_init 
        
           return obj

bveeramani · 2026-01-26T23:26:55Z

@ryankert01 you're right. Looks like we do emit a warning when we set warning=True in Deprecated.

I think the emitted warning is verbose and redundant with this approach.

Users/balaji/ray/python/ray/data/_internal/cluster_autoscaler/init.py:48: RayDeprecationWarning: This API is deprecated and may be removed in future Ray releases. You could suppress this warning by setting env variable PYTHONWARNINGS="ignore::DeprecationWarning"
DefaultClusterAutoscaler (V1) is deprecated. Use DefaultClusterAutoscalerV2 instead by setting RAY_DATA_CLUSTER_AUTOSCALER=V2 or using the default.

Rather than using the warning=True flag (which introduces the redundant deprecation message), could we not set warning=True and instead explicitly emit a warning in the autoscaler constructor, and also add "is deprecated and will be removed after June 2026."?

Signed-off-by: Ryan Huang <ryankert01@gmail.com>

python/ray/data/_internal/cluster_autoscaler/default_cluster_autoscaler.py

Signed-off-by: Ryan Huang <ryankert01@gmail.com>

ryankert01 · 2026-01-27T10:56:43Z

cc @bveeramani

bveeramani

nice ty

cursor

Cursor Bugbot has reviewed your changes and found 1 potential issue.

cursor · 2026-01-28T02:38:22Z

python/ray/data/_internal/cluster_autoscaler/default_cluster_autoscaler.py

+            "RAY_DATA_CLUSTER_AUTOSCALER=V2 or using the default.",
+            RayDeprecationWarning,
+            stacklevel=2,
+        )


Duplicated deprecation message string in two places

Low Severity

The deprecation message string "DefaultClusterAutoscaler (V1) is deprecated and will be removed after June 2026..." is duplicated verbatim in both the @Deprecated decorator and the warnings.warn() call. This creates a maintenance burden since updates to the deprecation message require changes in two places, risking inconsistency if one is modified without the other.

Additional Locations (1)

python/ray/data/_internal/cluster_autoscaler/default_cluster_autoscaler.py#L18-L23

@ryankert01 if you want, feel free to deduplicate this with a constant in a follow-up PR

Signed-off-by: Ryan Huang <ryankert01@gmail.com>

ryankert01 · 2026-01-28T04:54:31Z

The test failed because I changed default from v1 to v2. I change requestor to coordinator to make the test respect autoscaler v2.
cc @bveeramani

bveeramani

Data changes LGTM

bveeramani · 2026-01-28T22:17:08Z

python/ray/train/v2/tests/test_data_resource_cleanup.py

@@ -101,16 +101,15 @@ def test_after_worker_group_shutdown():
 def test_split_coordinator_shutdown_executor(ray_start_4_cpus):


Possibly out-of-scope for this PR since it's an existing issue, but this test breaks Ray Core and Ray Data abstraction barriers. I think we should refactor it to test against a higher level of abstraction (e.g., the ClusterAutoscaler interface).

TimothySeah · 2026-01-30T18:57:54Z

python/ray/data/_internal/cluster_autoscaler/default_cluster_autoscaler.py

+            "RAY_DATA_CLUSTER_AUTOSCALER=V2 or using the default.",
+            RayDeprecationWarning,
+            stacklevel=2,
+        )


) ## Description This document describes how to verify the V1 cluster autoscaler deprecation. 1. Added `@Deprecated` annotation to `DefaultClusterAutoscaler` in `default_cluster_autoscaler.py` 2. Changed default from `V1` to `V2` in `__init__.py` ## Related issues Closes ray-project#60459 ## Additional information ### Testing **Test V1 via env var emits deprecation warning:** ```bash RAY_DATA_CLUSTER_AUTOSCALER=V1 python -c " import warnings warnings.simplefilter('always') from unittest.mock import MagicMock from ray.data._internal.cluster_autoscaler import create_cluster_autoscaler mock_data_context = MagicMock() mock_data_context.execution_options.resource_limits = None autoscaler = create_cluster_autoscaler(MagicMock(), MagicMock(), mock_data_context, execution_id='test') print(f'Created: {type(autoscaler).__name__}') " ``` ``` ...RayDeprecationWarning: This API is deprecated and may be removed in future Ray releases... DefaultClusterAutoscaler (V1) is deprecated. Use DefaultClusterAutoscalerV2 instead by setting RAY_DATA_CLUSTER_AUTOSCALER=V2 or using the default. Created: DefaultClusterAutoscaler ``` **Test V2 (default) has no deprecation warning:** ```bash python -c " import warnings warnings.filterwarnings('always', category=DeprecationWarning) from unittest.mock import MagicMock from ray.data._internal.cluster_autoscaler import create_cluster_autoscaler mock_data_context = MagicMock() mock_data_context.execution_options.resource_limits = None autoscaler = create_cluster_autoscaler(MagicMock(), MagicMock(), mock_data_context, execution_id='test') print(f'Created: {type(autoscaler).__name__}') " ``` ``` Created: DefaultClusterAutoscalerV2 ``` --------- Signed-off-by: Ryan Huang <ryankert01@gmail.com>

) ## Description This document describes how to verify the V1 cluster autoscaler deprecation. 1. Added `@Deprecated` annotation to `DefaultClusterAutoscaler` in `default_cluster_autoscaler.py` 2. Changed default from `V1` to `V2` in `__init__.py` ## Related issues Closes ray-project#60459 ## Additional information ### Testing **Test V1 via env var emits deprecation warning:** ```bash RAY_DATA_CLUSTER_AUTOSCALER=V1 python -c " import warnings warnings.simplefilter('always') from unittest.mock import MagicMock from ray.data._internal.cluster_autoscaler import create_cluster_autoscaler mock_data_context = MagicMock() mock_data_context.execution_options.resource_limits = None autoscaler = create_cluster_autoscaler(MagicMock(), MagicMock(), mock_data_context, execution_id='test') print(f'Created: {type(autoscaler).__name__}') " ``` ``` ...RayDeprecationWarning: This API is deprecated and may be removed in future Ray releases... DefaultClusterAutoscaler (V1) is deprecated. Use DefaultClusterAutoscalerV2 instead by setting RAY_DATA_CLUSTER_AUTOSCALER=V2 or using the default. Created: DefaultClusterAutoscaler ``` **Test V2 (default) has no deprecation warning:** ```bash python -c " import warnings warnings.filterwarnings('always', category=DeprecationWarning) from unittest.mock import MagicMock from ray.data._internal.cluster_autoscaler import create_cluster_autoscaler mock_data_context = MagicMock() mock_data_context.execution_options.resource_limits = None autoscaler = create_cluster_autoscaler(MagicMock(), MagicMock(), mock_data_context, execution_id='test') print(f'Created: {type(autoscaler).__name__}') " ``` ``` Created: DefaultClusterAutoscalerV2 ``` --------- Signed-off-by: Ryan Huang <ryankert01@gmail.com> Signed-off-by: 400Ping <jiekaichang@apache.org>

) ## Description This document describes how to verify the V1 cluster autoscaler deprecation. 1. Added `@Deprecated` annotation to `DefaultClusterAutoscaler` in `default_cluster_autoscaler.py` 2. Changed default from `V1` to `V2` in `__init__.py` ## Related issues Closes ray-project#60459 ## Additional information ### Testing **Test V1 via env var emits deprecation warning:** ```bash RAY_DATA_CLUSTER_AUTOSCALER=V1 python -c " import warnings warnings.simplefilter('always') from unittest.mock import MagicMock from ray.data._internal.cluster_autoscaler import create_cluster_autoscaler mock_data_context = MagicMock() mock_data_context.execution_options.resource_limits = None autoscaler = create_cluster_autoscaler(MagicMock(), MagicMock(), mock_data_context, execution_id='test') print(f'Created: {type(autoscaler).__name__}') " ``` ``` ...RayDeprecationWarning: This API is deprecated and may be removed in future Ray releases... DefaultClusterAutoscaler (V1) is deprecated. Use DefaultClusterAutoscalerV2 instead by setting RAY_DATA_CLUSTER_AUTOSCALER=V2 or using the default. Created: DefaultClusterAutoscaler ``` **Test V2 (default) has no deprecation warning:** ```bash python -c " import warnings warnings.filterwarnings('always', category=DeprecationWarning) from unittest.mock import MagicMock from ray.data._internal.cluster_autoscaler import create_cluster_autoscaler mock_data_context = MagicMock() mock_data_context.execution_options.resource_limits = None autoscaler = create_cluster_autoscaler(MagicMock(), MagicMock(), mock_data_context, execution_id='test') print(f'Created: {type(autoscaler).__name__}') " ``` ``` Created: DefaultClusterAutoscalerV2 ``` --------- Signed-off-by: Ryan Huang <ryankert01@gmail.com> Signed-off-by: Sirui Huang <ray.huang@anyscale.com>

## Description This document describes how to verify the V1 cluster autoscaler deprecation. 1. Added `@Deprecated` annotation to `DefaultClusterAutoscaler` in `default_cluster_autoscaler.py` 2. Changed default from `V1` to `V2` in `__init__.py` ## Related issues Closes #60459 ## Additional information ### Testing **Test V1 via env var emits deprecation warning:** ```bash RAY_DATA_CLUSTER_AUTOSCALER=V1 python -c " import warnings warnings.simplefilter('always') from unittest.mock import MagicMock from ray.data._internal.cluster_autoscaler import create_cluster_autoscaler mock_data_context = MagicMock() mock_data_context.execution_options.resource_limits = None autoscaler = create_cluster_autoscaler(MagicMock(), MagicMock(), mock_data_context, execution_id='test') print(f'Created: {type(autoscaler).__name__}') " ``` ``` ...RayDeprecationWarning: This API is deprecated and may be removed in future Ray releases... DefaultClusterAutoscaler (V1) is deprecated. Use DefaultClusterAutoscalerV2 instead by setting RAY_DATA_CLUSTER_AUTOSCALER=V2 or using the default. Created: DefaultClusterAutoscaler ``` **Test V2 (default) has no deprecation warning:** ```bash python -c " import warnings warnings.filterwarnings('always', category=DeprecationWarning) from unittest.mock import MagicMock from ray.data._internal.cluster_autoscaler import create_cluster_autoscaler mock_data_context = MagicMock() mock_data_context.execution_options.resource_limits = None autoscaler = create_cluster_autoscaler(MagicMock(), MagicMock(), mock_data_context, execution_id='test') print(f'Created: {type(autoscaler).__name__}') " ``` ``` Created: DefaultClusterAutoscalerV2 ``` --------- Signed-off-by: Ryan Huang <ryankert01@gmail.com> Signed-off-by: elliot-barn <elliot.barnwell@anyscale.com>

## Description This document describes how to verify the V1 cluster autoscaler deprecation. 1. Added `@Deprecated` annotation to `DefaultClusterAutoscaler` in `default_cluster_autoscaler.py` 2. Changed default from `V1` to `V2` in `__init__.py` ## Related issues Closes #60459 ## Additional information ### Testing **Test V1 via env var emits deprecation warning:** ```bash RAY_DATA_CLUSTER_AUTOSCALER=V1 python -c " import warnings warnings.simplefilter('always') from unittest.mock import MagicMock from ray.data._internal.cluster_autoscaler import create_cluster_autoscaler mock_data_context = MagicMock() mock_data_context.execution_options.resource_limits = None autoscaler = create_cluster_autoscaler(MagicMock(), MagicMock(), mock_data_context, execution_id='test') print(f'Created: {type(autoscaler).__name__}') " ``` ``` ...RayDeprecationWarning: This API is deprecated and may be removed in future Ray releases... DefaultClusterAutoscaler (V1) is deprecated. Use DefaultClusterAutoscalerV2 instead by setting RAY_DATA_CLUSTER_AUTOSCALER=V2 or using the default. Created: DefaultClusterAutoscaler ``` **Test V2 (default) has no deprecation warning:** ```bash python -c " import warnings warnings.filterwarnings('always', category=DeprecationWarning) from unittest.mock import MagicMock from ray.data._internal.cluster_autoscaler import create_cluster_autoscaler mock_data_context = MagicMock() mock_data_context.execution_options.resource_limits = None autoscaler = create_cluster_autoscaler(MagicMock(), MagicMock(), mock_data_context, execution_id='test') print(f'Created: {type(autoscaler).__name__}') " ``` ``` Created: DefaultClusterAutoscalerV2 ``` --------- Signed-off-by: Ryan Huang <ryankert01@gmail.com>

) ## Description This document describes how to verify the V1 cluster autoscaler deprecation. 1. Added `@Deprecated` annotation to `DefaultClusterAutoscaler` in `default_cluster_autoscaler.py` 2. Changed default from `V1` to `V2` in `__init__.py` ## Related issues Closes ray-project#60459 ## Additional information ### Testing **Test V1 via env var emits deprecation warning:** ```bash RAY_DATA_CLUSTER_AUTOSCALER=V1 python -c " import warnings warnings.simplefilter('always') from unittest.mock import MagicMock from ray.data._internal.cluster_autoscaler import create_cluster_autoscaler mock_data_context = MagicMock() mock_data_context.execution_options.resource_limits = None autoscaler = create_cluster_autoscaler(MagicMock(), MagicMock(), mock_data_context, execution_id='test') print(f'Created: {type(autoscaler).__name__}') " ``` ``` ...RayDeprecationWarning: This API is deprecated and may be removed in future Ray releases... DefaultClusterAutoscaler (V1) is deprecated. Use DefaultClusterAutoscalerV2 instead by setting RAY_DATA_CLUSTER_AUTOSCALER=V2 or using the default. Created: DefaultClusterAutoscaler ``` **Test V2 (default) has no deprecation warning:** ```bash python -c " import warnings warnings.filterwarnings('always', category=DeprecationWarning) from unittest.mock import MagicMock from ray.data._internal.cluster_autoscaler import create_cluster_autoscaler mock_data_context = MagicMock() mock_data_context.execution_options.resource_limits = None autoscaler = create_cluster_autoscaler(MagicMock(), MagicMock(), mock_data_context, execution_id='test') print(f'Created: {type(autoscaler).__name__}') " ``` ``` Created: DefaultClusterAutoscalerV2 ``` --------- Signed-off-by: Ryan Huang <ryankert01@gmail.com> Signed-off-by: Adel Nour <ans9868@nyu.edu>

) ## Description This document describes how to verify the V1 cluster autoscaler deprecation. 1. Added `@Deprecated` annotation to `DefaultClusterAutoscaler` in `default_cluster_autoscaler.py` 2. Changed default from `V1` to `V2` in `__init__.py` ## Related issues Closes ray-project#60459 ## Additional information ### Testing **Test V1 via env var emits deprecation warning:** ```bash RAY_DATA_CLUSTER_AUTOSCALER=V1 python -c " import warnings warnings.simplefilter('always') from unittest.mock import MagicMock from ray.data._internal.cluster_autoscaler import create_cluster_autoscaler mock_data_context = MagicMock() mock_data_context.execution_options.resource_limits = None autoscaler = create_cluster_autoscaler(MagicMock(), MagicMock(), mock_data_context, execution_id='test') print(f'Created: {type(autoscaler).__name__}') " ``` ``` ...RayDeprecationWarning: This API is deprecated and may be removed in future Ray releases... DefaultClusterAutoscaler (V1) is deprecated. Use DefaultClusterAutoscalerV2 instead by setting RAY_DATA_CLUSTER_AUTOSCALER=V2 or using the default. Created: DefaultClusterAutoscaler ``` **Test V2 (default) has no deprecation warning:** ```bash python -c " import warnings warnings.filterwarnings('always', category=DeprecationWarning) from unittest.mock import MagicMock from ray.data._internal.cluster_autoscaler import create_cluster_autoscaler mock_data_context = MagicMock() mock_data_context.execution_options.resource_limits = None autoscaler = create_cluster_autoscaler(MagicMock(), MagicMock(), mock_data_context, execution_id='test') print(f'Created: {type(autoscaler).__name__}') " ``` ``` Created: DefaultClusterAutoscalerV2 ``` --------- Signed-off-by: Ryan Huang <ryankert01@gmail.com> Signed-off-by: peterxcli <peterxcli@gmail.com>

ryankert01 requested a review from a team as a code owner January 24, 2026 13:39

gemini-code-assist bot reviewed Jan 24, 2026

View reviewed changes

Deprecate V1 cluster autoscaler in favor of V2

4f728a3

Signed-off-by: Ryan Huang <ryankert01@gmail.com>

ryankert01 force-pushed the deprecate-autoscaler-v1 branch from ff2eff1 to 4f728a3 Compare January 24, 2026 13:47

ray-gardener bot added docs An issue or change related to documentation data Ray Data-related issues deprecation Scheduled for deprecation community-contribution Contributed by the community labels Jan 24, 2026

bveeramani reviewed Jan 24, 2026

View reviewed changes

Merge branch 'master' into deprecate-autoscaler-v1

a1ded7a

ryankert01 requested a review from bveeramani January 25, 2026 04:07

Merge branch 'master' into deprecate-autoscaler-v1

bb59acb

ryankert01 assigned bveeramani Jan 26, 2026

use warn.warning to reduce warning message redundancy

215cc2a

Signed-off-by: Ryan Huang <ryankert01@gmail.com>

cursor bot reviewed Jan 27, 2026

View reviewed changes

python/ray/data/_internal/cluster_autoscaler/default_cluster_autoscaler.py Show resolved Hide resolved

ryankert01 added 2 commits January 27, 2026 18:54

fix cursor comments

9c77075

Signed-off-by: Ryan Huang <ryankert01@gmail.com>

Merge branch 'master' into deprecate-autoscaler-v1

b927ef9

bveeramani approved these changes Jan 27, 2026

View reviewed changes

bveeramani enabled auto-merge (squash) January 27, 2026 17:34

github-actions bot added the go add ONLY when ready to merge, run all tests label Jan 27, 2026

Merge branch 'master' into deprecate-autoscaler-v1

6015036

github-actions bot disabled auto-merge January 28, 2026 02:32

cursor bot reviewed Jan 28, 2026

View reviewed changes

update test to respect autoscalar v2

55463d6

Signed-off-by: Ryan Huang <ryankert01@gmail.com>

ryankert01 requested a review from a team as a code owner January 28, 2026 04:45

Merge branch 'master' into deprecate-autoscaler-v1

0ee3c2a

bveeramani approved these changes Jan 28, 2026

View reviewed changes

ryankert01 requested a review from JasonLi1909 January 29, 2026 14:06

TimothySeah approved these changes Jan 30, 2026

View reviewed changes

bveeramani merged commit 76304d0 into ray-project:master Jan 30, 2026
6 checks passed

		@@ -101,16 +101,15 @@ def test_after_worker_group_shutdown():
		def test_split_coordinator_shutdown_executor(ray_start_4_cpus):

Conversation

ryankert01 commented Jan 24, 2026

Description

Related issues

Additional information

Testing

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

bveeramani left a comment

Choose a reason for hiding this comment

Uh oh!

ryankert01 commented Jan 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bveeramani commented Jan 26, 2026

Uh oh!

Uh oh!

ryankert01 commented Jan 27, 2026

Uh oh!

bveeramani left a comment

Choose a reason for hiding this comment

Uh oh!

cursor bot left a comment

Choose a reason for hiding this comment

Uh oh!

cursor bot Jan 28, 2026

Choose a reason for hiding this comment

Duplicated deprecation message string in two places

Uh oh!

TimothySeah Jan 30, 2026

Choose a reason for hiding this comment

Uh oh!

bveeramani Jan 30, 2026

Choose a reason for hiding this comment

Uh oh!

ryankert01 commented Jan 28, 2026

Uh oh!

bveeramani left a comment

Choose a reason for hiding this comment

Uh oh!

bveeramani Jan 28, 2026

Choose a reason for hiding this comment

Uh oh!

TimothySeah Jan 30, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ryankert01 commented Jan 25, 2026 •

edited

Loading