Add request parameter 'cluster_manager_timeout' and deprecate 'master_timeout' - in Snapshot APIs by tlfeng · Pull Request #2680 · opensearch-project/OpenSearch

tlfeng · 2022-03-31T04:36:31Z

Description

Deprecate the request parameter master_timeout that used in Snapshot APIs which have got the parameter.
Add alternative new request parameter cluster_manager_timeout.
Add unit tests.

List of the Snapshot APIs that have got request parameter master_timeout:
Put snapshot repository API PUT /_snapshot/<repository>, POST /_snapshot/<repository>
Get snapshot repository API
Delete snapshot repository API
Clean up snapshot repository API POST /_snapshot/<repository>/_cleanup
Verify snapshot repository API POST /_snapshot/<repository>/_verify
Create snapshot API
Get snapshot API
Delete snapshot API
Restore snapshot API POST /_snapshot/<repository>/<snapshot>/_restore
Clone snapshot API
Get snapshot status API GET _snapshot/<repository>/<snapshot>/_status

Issues Resolved

A part of issue #2511

Check List

New functionality includes testing.
- All tests pass
New functionality has been documented.
- New functionality has javadoc added
Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Tianli Feng <ftianli@amazon.com>

…pshot-api

Signed-off-by: Tianli Feng <ftianli@amazon.com>

opensearch-ci-bot · 2022-03-31T05:15:24Z

✅ Gradle Check success c998932
Log 3958

Reports 3958

opensearch-ci-bot · 2022-03-31T05:15:34Z

❌ Gradle Check failure 138eafddb434351306909d14b6539fe185ce8f1f
Log 3959

Reports 3959

opensearch-ci-bot · 2022-03-31T05:49:34Z

✅ Gradle Check success 6142b93
Log 3962

Reports 3962

Signed-off-by: Tianli Feng <ftianli@amazon.com> # Conflicts: # server/src/main/java/org/opensearch/rest/BaseRestHandler.java

…rameter Signed-off-by: Tianli Feng <ftianli@amazon.com>

opensearch-ci-bot · 2022-04-01T08:47:56Z

✅ Gradle Check success 2e293caaed8ed8c230fc1d4786b90a1f4456e85b
Log 4022

Reports 4022

opensearch-ci-bot · 2022-04-01T08:47:57Z

❌ Gradle Check failure 6142b93
Log 4023

Reports 4023

Signed-off-by: Tianli Feng <ftianli@amazon.com>

tlfeng · 2022-04-01T08:50:51Z

In log 4023:

REPRODUCE WITH: ./gradlew ':server:internalClusterTest' --tests "org.opensearch.search.SearchCancellationIT.testMSearchChildRequestCancellationWithClusterLevelTimeout" -Dtests.seed=60DB4EDF578765F3 -Dtests.security.manager=true -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=sq -Dtests.timezone=Asia/Beirut -Druntime.java=17

org.opensearch.search.SearchCancellationIT > testMSearchChildRequestCancellationWithClusterLevelTimeout FAILED
    java.lang.AssertionError: Actual child request with cancellation failure is different that expected expected:<[0, 1]> but was:<[]>
        at __randomizedtesting.SeedInfo.seed([60DB4EDF578765F3:DE66853AC6918584]:0)
        at org.junit.Assert.fail(Assert.java:89)
        at org.junit.Assert.failNotEquals(Assert.java:835)
        at org.junit.Assert.assertEquals(Assert.java:120)
        at org.opensearch.search.SearchCancellationIT.ensureMSearchWasCancelled(SearchCancellationIT.java:194)
        at org.opensearch.search.SearchCancellationIT.testMSearchChildRequestCancellationWithClusterLevelTimeout(SearchCancellationIT.java:537)

> Task :client:rest-high-level:asyncIntegTest

REPRODUCE WITH: ./gradlew ':client:rest-high-level:asyncIntegTest' --tests "org.opensearch.client.SnapshotIT.testSnapshotsStatus" -Dtests.seed=60DB4EDF578765F3 -Dtests.security.manager=true -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=en -Dtests.timezone=Mexico/BajaSur -Druntime.java=17

org.opensearch.client.SnapshotIT > testSnapshotsStatus FAILED
    org.opensearch.client.WarningFailureException: method [PUT], host [http://[::1]:39105], URI [/_snapshot/test?master_timeout=30s&timeout=30s], status line [HTTP/1.1 200 OK]
    Warnings: [Deprecated parameter [master_timeout] used. To promote inclusive language, please use [cluster_manager_timeout] instead. It will be unsupported in a future major version.]
    {"acknowledged":true}
        at __randomizedtesting.SeedInfo.seed([60DB4EDF578765F3:9713EF5C5009D61B]:0)
        at app//org.opensearch.client.RestClient.convertResponse(RestClient.java:346)
        at app//org.opensearch.client.RestClient$1.completed(RestClient.java:400)
        at app//org.opensearch.client.RestClient$1.completed(RestClient.java:396)
        at app//org.apache.http.concurrent.BasicFuture.completed(BasicFuture.java:122)
        at app//org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.responseCompleted(DefaultClientExchangeHandlerImpl.java:181)

opensearch-ci-bot · 2022-04-01T08:59:09Z

✅ Gradle Check success 2a289e4
Log 4024

Reports 4024

opensearch-ci-bot · 2022-04-01T08:59:09Z

❌ Gradle Check failure d04e4bd
Log 4025

Reports 4025

opensearch-ci-bot · 2022-04-01T09:29:43Z

✅ Gradle Check success 336e7c7
Log 4027

Reports 4027

Signed-off-by: Tianli Feng <ftianli@amazon.com> # Conflicts: # server/src/test/java/org/opensearch/action/RenamedTimeoutRequestParameterTests.java

opensearch-ci-bot · 2022-04-02T01:18:47Z

❌ Gradle Check failure 697712a
Log 4061

Reports 4061

tlfeng · 2022-04-03T22:59:04Z

In log 4061:


REPRODUCE WITH: ./gradlew ':qa:remote-clusters:integTest' --tests "org.opensearch.cluster.remote.test.RemoteClustersIT.testHAProxyModeConnectionWorks" -Dtests.seed=2743972FDE83A85C -Dtests.security.manager=true -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=ar-BH -Dtests.timezone=Asia/Irkutsk -Druntime.java=17

org.opensearch.cluster.remote.test.RemoteClustersIT > testHAProxyModeConnectionWorks FAILED
    java.lang.AssertionError
        at __randomizedtesting.SeedInfo.seed([2743972FDE83A85C:203542E4E505DE01]:0)
        at org.junit.Assert.fail(Assert.java:87)
        at org.junit.Assert.assertTrue(Assert.java:42)
        at org.junit.Assert.assertTrue(Assert.java:53)
        at org.opensearch.cluster.remote.test.RemoteClustersIT.testHAProxyModeConnectionWorks(RemoteClustersIT.java:125)

It's reported in issue #1703

Signed-off-by: Tianli Feng <ftianli@amazon.com>

opensearch-ci-bot · 2022-04-04T17:34:46Z

✅ Gradle Check success bd1f0a3
Log 4129

Reports 4129

Signed-off-by: Tianli Feng <ftianli@amazon.com> # Conflicts: # server/src/test/java/org/opensearch/action/RenamedTimeoutRequestParameterTests.java

opensearch-ci-bot · 2022-04-04T22:27:35Z

✅ Gradle Check success b4e9c8b
Log 4155

Reports 4155

dreamer-89

Nice! LGTM

Signed-off-by: Tianli Feng <ftianli@amazon.com> # Conflicts: # server/src/test/java/org/opensearch/action/RenamedTimeoutRequestParameterTests.java

opensearch-ci-bot · 2022-04-05T18:46:01Z

❌ Gradle Check failure 0d53a86
Log 4209

Reports 4209

tlfeng · 2022-04-05T18:47:47Z

In log 4209:

> Task :sandbox:plugins:concurrent-search:test

REPRODUCE WITH: ./gradlew ':sandbox:plugins:concurrent-search:test' --tests "org.opensearch.search.query.QueryProfilePhaseTests" -Dtests.method="testTerminateAfterEarlyTermination {p0=5 p1=org.opensearch.search.query.ConcurrentQueryPhaseSearcher@2c4eae94}" -Dtests.seed=CDA03188B38F0C0D -Dtests.security.manager=true -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=ar-TN -Dtests.timezone=America/Noronha -Druntime.java=17

org.opensearch.search.query.QueryProfilePhaseTests > testTerminateAfterEarlyTermination {p0=5 p1=org.opensearch.search.query.ConcurrentQueryPhaseSearcher@2c4eae94} FAILED
    java.lang.AssertionError: 
    Expected: a value greater than <0L>
         but: <0L> was equal to <0L>
        at __randomizedtesting.SeedInfo.seed([CDA03188B38F0C0D:5B04881FBABD7064]:0)
        at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:18)
        at org.junit.Assert.assertThat(Assert.java:964)
        at org.junit.Assert.assertThat(Assert.java:930)
        at org.opensearch.search.query.QueryProfilePhaseTests.lambda$testTerminateAfterEarlyTermination$26(QueryProfilePhaseTests.java:553)
        at org.opensearch.search.query.QueryProfilePhaseTests.lambda$assertProfileData$55(QueryProfilePhaseTests.java:1072)
        at org.opensearch.search.query.QueryProfilePhaseTests.assertProfileData(QueryProfilePhaseTests.java:1123)
        at org.opensearch.search.query.QueryProfilePhaseTests.assertProfileData(QueryProfilePhaseTests.java:1078)
        at org.opensearch.search.query.QueryProfilePhaseTests.assertProfileData(QueryProfilePhaseTests.java:1069)
        at org.opensearch.search.query.QueryProfilePhaseTests.testTerminateAfterEarlyTermination(QueryProfilePhaseTests.java:534)

REPRODUCE WITH: ./gradlew ':sandbox:plugins:concurrent-search:test' --tests "org.opensearch.search.query.QueryProfilePhaseTests" -Dtests.method="testTerminateAfterEarlyTermination {p0=0 p1=org.opensearch.search.query.QueryPhase$DefaultQueryPhaseSearcher@6ad198c9}" -Dtests.seed=CDA03188B38F0C0D -Dtests.security.manager=true -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=ar-TN -Dtests.timezone=America/Noronha -Druntime.java=17

org.opensearch.search.query.QueryProfilePhaseTests > testTerminateAfterEarlyTermination {p0=0 p1=org.opensearch.search.query.QueryPhase$DefaultQueryPhaseSearcher@6ad198c9} FAILED
    java.lang.AssertionError: 
    Expected: a value greater than <0L>
         but: <0L> was equal to <0L>
... (same as above)

* What went wrong:
Execution failed for task ':sandbox:plugins:concurrent-search:test'.
> org.gradle.test-retry was unable to retry the following test methods, which is unexpected. Please file a bug report at https://github.com/gradle/test-retry-gradle-plugin/issues
     org.opensearch.search.query.QueryProfilePhaseTests#testTerminateAfterEarlyTermination {p0=5 p1=org.opensearch.search.query.ConcurrentQueryPhaseSearcher@2c4eae94}
     org.opensearch.search.query.QueryProfilePhaseTests#testTerminateAfterEarlyTermination {p0=0 p1=org.opensearch.search.query.QueryPhase$DefaultQueryPhaseSearcher@6ad198c9}

It's reproducible in main branch: ./gradlew ':sandbox:plugins:concurrent-search:test' --tests "org.opensearch.search.query.QueryProfilePhaseTests" -Dtests.method="testTerminateAfterEarlyTermination" -Dtests.seed=CDA03188B38F0C0D. I created issue #2775

tlfeng · 2022-04-05T19:02:57Z

start gradle check

opensearch-ci-bot · 2022-04-05T19:32:23Z

✅ Gradle Check success 0d53a86
Log 4213

Reports 4213

# Conflicts: # server/src/test/java/org/opensearch/action/RenamedTimeoutRequestParameterTests.java

Signed-off-by: Tianli Feng <ftianli@amazon.com>

opensearch-ci-bot · 2022-04-12T01:27:03Z

✅ Gradle Check success 712952e
Log 4392

Reports 4392

…_timeout' - in Snapshot APIs (#2680) * Deprecate the request parameter `master_timeout` that used in Snapshot APIs which have got the parameter. * Add alternative new request parameter `cluster_manager_timeout`. * Add unit tests. Signed-off-by: Tianli Feng <ftianli@amazon.com> (cherry picked from commit a89b7e6)

…_timeout' - in Snapshot APIs (#2680) (#2870) * Deprecate the request parameter `master_timeout` that used in Snapshot APIs which have got the parameter. * Add alternative new request parameter `cluster_manager_timeout`. * Add unit tests. Signed-off-by: Tianli Feng <ftianli@amazon.com> (cherry picked from commit a89b7e6)

…_timeout' - in Snapshot APIs (#2680) (#2871) * Deprecate the request parameter `master_timeout` that used in Snapshot APIs which have got the parameter. * Add alternative new request parameter `cluster_manager_timeout`. * Add unit tests. Signed-off-by: Tianli Feng <ftianli@amazon.com> (cherry picked from commit a89b7e6)

Tianli Feng added 2 commits March 30, 2022 21:21

Deprecate master_timeout parameter in snapshot APIs

abfb410

Signed-off-by: Tianli Feng <ftianli@amazon.com>

Merge remote-tracking branch 'upstream/main' into manager-timeout-sna…

c998932

…pshot-api

tlfeng added enhancement Enhancement or improvement to existing feature or request v2.0.0 Version 2.0.0 backport 2.x Backport to 2.x branch backport 2.0 Backport to 2.0 branch labels Mar 31, 2022

tlfeng mentioned this pull request Mar 31, 2022

Add a new request parameter "cluster_manager_timeout" in the REST APIs that accept "master_timeout" parameter, and deprecate "master_timeout" #2511

Closed

11 tasks

Deprecate master_timeout parameter in snapshot APIs rest-api-spec

6142b93

Signed-off-by: Tianli Feng <ftianli@amazon.com>

tlfeng force-pushed the manager-timeout-snapshot-api branch from 138eafd to 6142b93 Compare March 31, 2022 05:05

tlfeng added the WIP Work in progress label Mar 31, 2022

tlfeng force-pushed the manager-timeout-snapshot-api branch from 2e293ca to 6142b93 Compare April 1, 2022 08:08

Merge branch 'main' into manager-timeout-snapshot-api

2a289e4

Signed-off-by: Tianli Feng <ftianli@amazon.com> # Conflicts: # server/src/main/java/org/opensearch/rest/BaseRestHandler.java

tlfeng force-pushed the manager-timeout-snapshot-api branch from 7e6e9dd to 2a289e4 Compare April 1, 2022 08:17

Add unit test for snapshot APIs of parsing cluster_manager_timeout pa…

d04e4bd

…rameter Signed-off-by: Tianli Feng <ftianli@amazon.com>

tlfeng marked this pull request as ready for review April 1, 2022 08:34

tlfeng requested a review from a team as a code owner April 1, 2022 08:34

tlfeng removed the WIP Work in progress label Apr 1, 2022

Adjust format by spotlessApply task

336e7c7

Signed-off-by: Tianli Feng <ftianli@amazon.com>

tlfeng mentioned this pull request Apr 1, 2022

Make High-Level-Rest-Client tests allow deprecation warning temporarily, during deprecation of request parameter 'master_timeout' #2702

Merged

5 tasks

Merge branch 'main' into manager-timeout-snapshot-api

697712a

Signed-off-by: Tianli Feng <ftianli@amazon.com> # Conflicts: # server/src/test/java/org/opensearch/action/RenamedTimeoutRequestParameterTests.java

Merge branch 'main' into manager-timeout-snapshot-api

bd1f0a3

Signed-off-by: Tianli Feng <ftianli@amazon.com>

Merge branch 'main' into manager-timeout-snapshot-api

b4e9c8b

Signed-off-by: Tianli Feng <ftianli@amazon.com> # Conflicts: # server/src/test/java/org/opensearch/action/RenamedTimeoutRequestParameterTests.java

dreamer-89 approved these changes Apr 5, 2022

View reviewed changes

Merge branch 'main' into manager-timeout-snapshot-api

0d53a86

Signed-off-by: Tianli Feng <ftianli@amazon.com> # Conflicts: # server/src/test/java/org/opensearch/action/RenamedTimeoutRequestParameterTests.java

tlfeng mentioned this pull request Apr 5, 2022

[BUG] test failure ':sandbox:plugins:concurrent-search:test' QueryProfilePhaseTests.testTerminateAfterEarlyTermination #2775

Closed

xuezhou25 approved these changes Apr 7, 2022

View reviewed changes

Tianli Feng added 2 commits April 11, 2022 17:35

Merge branch 'main' into manager-timeout-snapshot-api

dfa3ac5

# Conflicts: # server/src/test/java/org/opensearch/action/RenamedTimeoutRequestParameterTests.java

Use support instead of promote in the deprecation message

712952e

Signed-off-by: Tianli Feng <ftianli@amazon.com>

tlfeng merged commit a89b7e6 into opensearch-project:main Apr 12, 2022

tlfeng deleted the manager-timeout-snapshot-api branch April 12, 2022 03:43

opensearch-trigger-bot bot mentioned this pull request Apr 12, 2022

[Backport 2.x] Add request parameter 'cluster_manager_timeout' and deprecate 'master_timeout' - in Snapshot APIs #2870

Merged

opensearch-trigger-bot bot mentioned this pull request Apr 12, 2022

[Backport 2.0] Add request parameter 'cluster_manager_timeout' and deprecate 'master_timeout' - in Snapshot APIs #2871

Merged

VachaShah mentioned this pull request Apr 13, 2022

Jenkins Randomized Gradle Check Failed on 6142b9340cb4627fa72c047f7e51422bfac0b596 #2701

Closed

Conversation

tlfeng commented Mar 31, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Issues Resolved

Check List

Uh oh!

opensearch-ci-bot commented Mar 31, 2022

Uh oh!

opensearch-ci-bot commented Mar 31, 2022

Uh oh!

opensearch-ci-bot commented Mar 31, 2022

Uh oh!

opensearch-ci-bot commented Apr 1, 2022

Uh oh!

opensearch-ci-bot commented Apr 1, 2022

Uh oh!

tlfeng commented Apr 1, 2022

Uh oh!

opensearch-ci-bot commented Apr 1, 2022

Uh oh!

opensearch-ci-bot commented Apr 1, 2022

Uh oh!

opensearch-ci-bot commented Apr 1, 2022

Uh oh!

opensearch-ci-bot commented Apr 2, 2022

Uh oh!

tlfeng commented Apr 3, 2022

Uh oh!

opensearch-ci-bot commented Apr 4, 2022

Uh oh!

opensearch-ci-bot commented Apr 4, 2022

Uh oh!

dreamer-89 left a comment

Choose a reason for hiding this comment

Uh oh!

opensearch-ci-bot commented Apr 5, 2022

Uh oh!

tlfeng commented Apr 5, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tlfeng commented Apr 5, 2022

Uh oh!

opensearch-ci-bot commented Apr 5, 2022

Uh oh!

opensearch-ci-bot commented Apr 12, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

tlfeng commented Mar 31, 2022 •

edited

Loading

tlfeng commented Apr 5, 2022 •

edited

Loading