[SnapshotV2] Add timestamp of last successful fetch of pinned timestamps in node stats by ltaragi · Pull Request #15611 · opensearch-project/OpenSearch

ltaragi · 2024-09-03T12:15:14Z

Description

This change adds the timestamp for the last successful fetch of pinned timestamps to the node stats:

We make use of the static method getPinnedTimestamps() of RemoteStorePinnedTimestampService. The first value of the returned tuple is when the last fetch of pinned timestamps is triggered . This is the timestamp we need to add to node stats.
We have ensured backward compatibility with version checks
We have added a new remote store specific metric class RemoteStoreNodeStats in the node stats

Related Issues

Resolves #15896

Check List

Functionality includes testing.
~~API changes companion pull request created, if applicable.~~
~~Public documentation issue/PR created, if applicable.~~

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

github-actions · 2024-09-03T12:16:22Z

❌ Gradle check result for 7500c9d: null

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2024-09-03T16:03:08Z

❌ Gradle check result for 42a1030: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2024-09-03T17:32:43Z

❌ Gradle check result for a617c18: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2024-09-10T05:29:51Z

✅ Gradle check result for b238922: SUCCESS

codecov · 2024-09-10T05:30:33Z

Codecov Report

Attention: Patch coverage is 48.57143% with 18 lines in your changes missing coverage. Please review.

Project coverage is 71.98%. Comparing base (d131d58) to head (b0a71d2).
Report is 6 commits behind head on main.

Files with missing lines	Patch %	Lines
...nsearch/node/remotestore/RemoteStoreNodeStats.java	45.00%	8 Missing and 3 partials ⚠️
...rch/action/admin/cluster/node/stats/NodeStats.java	44.44%	3 Missing and 2 partials ⚠️
...src/main/java/org/opensearch/node/NodeService.java	0.00%	0 Missing and 2 partials ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #15611      +/-   ##
============================================
+ Coverage     71.89%   71.98%   +0.09%     
- Complexity    64466    64528      +62     
============================================
  Files          5288     5289       +1     
  Lines        301439   301471      +32     
  Branches      43552    43558       +6     
============================================
+ Hits         216706   217011     +305     
+ Misses        66931    66676     -255     
+ Partials      17802    17784      -18

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

github-actions · 2024-09-11T06:51:59Z

❌ Gradle check result for 7f81828: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2024-09-11T10:15:46Z

❌ Gradle check result for 7fae8db: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

ltaragi · 2024-09-11T10:32:02Z

❌ Gradle check result for 7fae8db: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Flaky test: #14316

…mps in node stats (opensearch-project#15611) --------- Signed-off-by: Lakshya Taragi <lakshya.taragi@gmail.com>

…mps in node stats (#15611) --------- Signed-off-by: Lakshya Taragi <lakshya.taragi@gmail.com> (cherry picked from commit be9f942) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

…mps in node stats (#15611) (#16155) --------- (cherry picked from commit be9f942) Signed-off-by: Lakshya Taragi <lakshya.taragi@gmail.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

…mps in node stats (opensearch-project#15611) --------- Signed-off-by: Lakshya Taragi <lakshya.taragi@gmail.com>

dblock · 2024-10-24T19:17:31Z

This needed an API spec change as remote_store is newly returned. Hitting it in opensearch-project/opensearch-api-specification#643.

This field is returned since OpenSearch 2.18. (see opensearch-project/OpenSearch#15611) Signed-off-by: Tim Holdaway <timothy.holdaway@clever.com>

* Add Highlight field to SearchHit struct Signed-off-by: Tim Holdaway <timothy.holdaway@clever.com> * Update changelog Signed-off-by: Tim Holdaway <timothy.holdaway@clever.com> * Update SearchTemplate struct with Status field This field is returned since OpenSearch 2.18 Signed-off-by: Tim Holdaway <timothy.holdaway@clever.com> * Update NodeStatsstruct with RemoteStore field This field is returned since OpenSearch 2.18. (see opensearch-project/OpenSearch#15611) Signed-off-by: Tim Holdaway <timothy.holdaway@clever.com> * Always attempt to pull the docker base image for integration tests. Otherwise we can end up with a cached version of opensearch:latest that is not the current latest, and would use that instead, resulting in tests running against a different version than in the CI build. Signed-off-by: Tim Holdaway <timothy.holdaway@clever.com> --------- Signed-off-by: Tim Holdaway <timothy.holdaway@clever.com>

…mps in node stats (opensearch-project#15611) --------- Signed-off-by: Lakshya Taragi <lakshya.taragi@gmail.com>

ltaragi self-assigned this Sep 3, 2024

ltaragi added the skip-changelog label Sep 3, 2024

ltaragi force-pushed the ptsts-in-nodestats branch from 7500c9d to 42a1030 Compare September 3, 2024 15:12

ltaragi force-pushed the ptsts-in-nodestats branch from 42a1030 to a617c18 Compare September 3, 2024 16:08

This was referenced Sep 6, 2024

[AUTOCUT] Gradle Check Flaky Test Report for RemoteStoreMultipartIT #15819

Open

[AUTOCUT] Gradle Check Flaky Test Report for ShardIndexingPressureIT #15830

Closed

ltaragi force-pushed the ptsts-in-nodestats branch from a617c18 to b238922 Compare September 10, 2024 04:25

ltaragi force-pushed the ptsts-in-nodestats branch from b238922 to 7f81828 Compare September 11, 2024 05:55

ltaragi force-pushed the ptsts-in-nodestats branch from 7f81828 to 7fae8db Compare September 11, 2024 09:40

github-actions bot added enhancement Enhancement or improvement to existing feature or request Storage:Snapshots v2.18.0 Issues and PRs related to version 2.18.0 labels Sep 11, 2024

ltaragi marked this pull request as ready for review September 11, 2024 10:32

ltaragi requested review from Bukhtawar, CEHENKLE, anasalkouz, andrross, ashking94, dblock, dbwiddis, gbbafna and jainankitk as code owners September 11, 2024 10:32

sachinpkale added the skip-changelog label Oct 1, 2024

sachinpkale merged commit be9f942 into opensearch-project:main Oct 1, 2024

hainenber pushed a commit to hainenber/OpenSearch that referenced this pull request Oct 1, 2024

[SnapshotV2] Add timestamp of last successful fetch of pinned timesta…

e8f6588

…mps in node stats (opensearch-project#15611) --------- Signed-off-by: Lakshya Taragi <lakshya.taragi@gmail.com>

sachinpkale added the backport 2.x Backport to 2.x branch label Oct 1, 2024

opensearch-trigger-bot bot mentioned this pull request Oct 1, 2024

[Backport 2.x] [SnapshotV2] Add timestamp of last successful fetch of pinned timestamps in node stats #16155

Merged

ltaragi mentioned this pull request Oct 3, 2024

Change version checks from CURRENT to 2.18 in #15611 #16174

Merged

3 tasks

prudhvigodithi mentioned this pull request Oct 16, 2024

[AUTOCUT] Gradle Check Flaky Test Report for MixedClusterClientYamlTestSuiteIT prudhvigodithi/opensearch-build#82

Open

dk2k pushed a commit to dk2k/OpenSearch that referenced this pull request Oct 16, 2024

[SnapshotV2] Add timestamp of last successful fetch of pinned timesta…

ee3631b

…mps in node stats (opensearch-project#15611) --------- Signed-off-by: Lakshya Taragi <lakshya.taragi@gmail.com>

dk2k pushed a commit to dk2k/OpenSearch that referenced this pull request Oct 17, 2024

[SnapshotV2] Add timestamp of last successful fetch of pinned timesta…

148928d

…mps in node stats (opensearch-project#15611) --------- Signed-off-by: Lakshya Taragi <lakshya.taragi@gmail.com>

dk2k pushed a commit to dk2k/OpenSearch that referenced this pull request Oct 21, 2024

[SnapshotV2] Add timestamp of last successful fetch of pinned timesta…

57b2783

…mps in node stats (opensearch-project#15611) --------- Signed-off-by: Lakshya Taragi <lakshya.taragi@gmail.com>

BrewTestBot mentioned this pull request Nov 6, 2024

opensearch 2.18.0 Homebrew/homebrew-core#196785

Merged

tim-holdaway-clever mentioned this pull request Jan 17, 2025

Add Highlight field to SearchHit struct opensearch-project/opensearch-go#654

Merged

opensearch-ci-bot mentioned this pull request Oct 22, 2025

[AUTOCUT] Gradle Check Flaky Test Report for AbstractAsyncTaskTests #14316

Open

This was referenced Jan 10, 2026

docs: add remote-store-metrics report for v2.18.0 tkykenmt/opensearch-feature-explorer#758

Merged

[feature] Remote Store Metrics tkykenmt/opensearch-feature-explorer#633

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[SnapshotV2] Add timestamp of last successful fetch of pinned timestamps in node stats#15611

[SnapshotV2] Add timestamp of last successful fetch of pinned timestamps in node stats#15611
sachinpkale merged 5 commits intoopensearch-project:mainfrom
ltaragi:ptsts-in-nodestats

ltaragi commented Sep 3, 2024 •

edited

Loading

Uh oh!

github-actions bot commented Sep 3, 2024

Uh oh!

github-actions bot commented Sep 3, 2024

Uh oh!

github-actions bot commented Sep 3, 2024

Uh oh!

github-actions bot commented Sep 10, 2024

Uh oh!

codecov bot commented Sep 10, 2024 •

edited

Loading

Uh oh!

github-actions bot commented Sep 11, 2024

Uh oh!

github-actions bot commented Sep 11, 2024

Uh oh!

ltaragi commented Sep 11, 2024

Uh oh!

dblock commented Oct 24, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

ltaragi commented Sep 3, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Related Issues

Check List

Uh oh!

github-actions bot commented Sep 3, 2024

Uh oh!

github-actions bot commented Sep 3, 2024

Uh oh!

github-actions bot commented Sep 3, 2024

Uh oh!

github-actions bot commented Sep 10, 2024

Uh oh!

codecov bot commented Sep 10, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

github-actions bot commented Sep 11, 2024

Uh oh!

github-actions bot commented Sep 11, 2024

Uh oh!

ltaragi commented Sep 11, 2024

Uh oh!

dblock commented Oct 24, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ltaragi commented Sep 3, 2024 •

edited

Loading

codecov bot commented Sep 10, 2024 •

edited

Loading