Skip to content

Crypto Metadata support in repo registration#9802

Merged
gbbafna merged 2 commits intoopensearch-project:mainfrom
gbbafna:crypto-metadata
Sep 6, 2023
Merged

Crypto Metadata support in repo registration#9802
gbbafna merged 2 commits intoopensearch-project:mainfrom
gbbafna:crypto-metadata

Conversation

@gbbafna
Copy link
Copy Markdown
Contributor

@gbbafna gbbafna commented Sep 6, 2023 •
edited
Loading

Description

Crypto Metadata support in repo registration

Sample repo details

"node.attr.remote_store.segment.repository" , "remote-store-A",
"node.attr.remote_store.repository.remote-store-A.type" , "s3",
"node.attr.remote_store.repository.remote-store-A.settings.bucket" , "abc",
"node.attr.remote_store.repository.remote-store-A.settings.base_path" , "xyz",
"node.attr.remote_store.repository.remote-store-A.crypto_metadata.key_provider_name" , "store-test",
"node.attr.remote_store.repository.remote-store-A.crypto_metadata.key_provider_type" , "aws-kms",
"node.attr.remote_store.repository.remote-store-A.crypto_metadata.settings.region" , "us-east-1",
"node.attr.remote_store.repository.remote-store-A.crypto_metadata.settings.key_arn" , "arn:aws:kms:us-east-1:12345:key/6e9aa906"

Related Issues

Partially Resolves #8623

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Commits are signed per the DCO using --signoff
  • Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@gbbafna
Crypto Metadata support in repo registration
975ede2 
Signed-off-by: Gaurav Bafna <gbbafna@amazon.com>
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Sep 6, 2023

Compatibility status:

Checks if related components are compatible with change 975ede2

Incompatible components

Incompatible components: [https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/neural-search.git]

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/reporting.git]

psychbot
psychbot reviewed Sep 6, 2023
View reviewed changes
Copy link
Copy Markdown
Member

@psychbot psychbot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added few comments.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Sep 6, 2023

Gradle Check (Jenkins) Run Completed with:

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Sep 6, 2023

Compatibility status:

Checks if related components are compatible with change 64bd991

Incompatible components

Incompatible components: [https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/cross-cluster-replication.git]

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/reporting.git]

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Sep 6, 2023

Gradle Check (Jenkins) Run Completed with:

@gbbafna gbbafna marked this pull request as ready for review September 6, 2023 12:31
@psychbot
Copy link
Copy Markdown
Member

psychbot commented Sep 6, 2023

LGTM!

@gbbafna
Copy link
Copy Markdown
Contributor Author

gbbafna commented Sep 6, 2023

Failing tests :

org.opensearch.search.profile.query.QueryProfilerIT.testProfileQuery {p0={"search.concurrent_segment_search.enabled":"true"} p1=true}
org.opensearch.search.profile.query.QueryProfilerIT.testProfileMatchesRegular {p0={"search.concurrent_segment_search.enabled":"true"} p1=true}
org.opensearch.snapshots.DedicatedClusterSnapshotRestoreIT.testIndexDeletionDuringSnapshotCreationInQueue
org.opensearch.remotestore.RemoteIndexPrimaryRelocationIT.testPrimaryRelocationWhileIndexing

testPrimaryRelocationWhileIndexing is getting fixed in a PR and the rest are flaky .

@psychbot
Copy link
Copy Markdown
Member

psychbot commented Sep 6, 2023

#9807 - Flaky test - testPrimaryRelocationWhileIndexing

@gbbafna gbbafna merged commit 742267d into opensearch-project:main Sep 6, 2023
@gbbafna gbbafna added the backport 2.x Backport to 2.x branch label Sep 6, 2023
@opensearch-trigger-bot opensearch-trigger-bot bot mentioned this pull request Sep 6, 2023
Merged
opensearch-trigger-bot bot pushed a commit that referenced this pull request Sep 6, 2023
@github-actions
Crypto Metadata support in repo registration (#9802)
e60750d 
Signed-off-by: Gaurav Bafna <gbbafna@amazon.com>
(cherry picked from commit 742267d)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
gbbafna pushed a commit that referenced this pull request Sep 6, 2023
@opensearch-trigger-bot @github-actions
Crypto Metadata support in repo registration (#9802) (#9827)
655d223 
(cherry picked from commit 742267d)

Signed-off-by: Gaurav Bafna <gbbafna@amazon.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
@gbbafna gbbafna added the backport 2.10 Backport to 2.10 branch label Sep 7, 2023
@gbbafna gbbafna deleted the crypto-metadata branch September 7, 2023 13:28
@opensearch-trigger-bot opensearch-trigger-bot bot mentioned this pull request Sep 7, 2023
Merged
opensearch-trigger-bot bot pushed a commit that referenced this pull request Sep 7, 2023
@github-actions
Crypto Metadata support in repo registration (#9802)
1e69c73 
Signed-off-by: Gaurav Bafna <gbbafna@amazon.com>
(cherry picked from commit 742267d)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
gbbafna pushed a commit that referenced this pull request Sep 7, 2023
@opensearch-trigger-bot @github-actions
Crypto Metadata support in repo registration (#9802) (#9888)
c0e9f4f 
(cherry picked from commit 742267d)

Signed-off-by: Gaurav Bafna <gbbafna@amazon.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
kaushalmahi12 pushed a commit to kaushalmahi12/OpenSearch that referenced this pull request Sep 12, 2023
@gbbafna @kaushalmahi12
Crypto Metadata support in repo registration (opensearch-project#9802)
0689d3b 
Signed-off-by: Gaurav Bafna <gbbafna@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>
brusic pushed a commit to brusic/OpenSearch that referenced this pull request Sep 25, 2023
@gbbafna @brusic
Crypto Metadata support in repo registration (opensearch-project#9802)
f4cea9e 
Signed-off-by: Gaurav Bafna <gbbafna@amazon.com>
Signed-off-by: Ivan Brusic <ivan.brusic@flocksafety.com>
shiv0408 pushed a commit to Gaurav614/OpenSearch that referenced this pull request Apr 25, 2024
@gbbafna @shiv0408
Crypto Metadata support in repo registration (opensearch-project#9802)
8540027 
Signed-off-by: Gaurav Bafna <gbbafna@amazon.com>
Signed-off-by: Shivansh Arora <hishiv@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@sachinpkale sachinpkale sachinpkale approved these changes

@reta reta Awaiting requested review from reta

@anasalkouz anasalkouz Awaiting requested review from anasalkouz

@andrross andrross Awaiting requested review from andrross

@Bukhtawar Bukhtawar Awaiting requested review from Bukhtawar

@CEHENKLE CEHENKLE Awaiting requested review from CEHENKLE

@dblock dblock Awaiting requested review from dblock

@setiah setiah Awaiting requested review from setiah

@kartg kartg Awaiting requested review from kartg

@kotwanikunal kotwanikunal Awaiting requested review from kotwanikunal

@mch2 mch2 Awaiting requested review from mch2

@nknize nknize Awaiting requested review from nknize

@owaiskazi19 owaiskazi19 Awaiting requested review from owaiskazi19

@peternied peternied Awaiting requested review from peternied

@Rishikesh1159 Rishikesh1159 Awaiting requested review from Rishikesh1159

@ryanbogan ryanbogan Awaiting requested review from ryanbogan

@saratvemulapalli saratvemulapalli Awaiting requested review from saratvemulapalli

@shwetathareja shwetathareja Awaiting requested review from shwetathareja

@dreamer-89 dreamer-89 Awaiting requested review from dreamer-89

@tlfeng tlfeng Awaiting requested review from tlfeng

@VachaShah VachaShah Awaiting requested review from VachaShah

@dbwiddis dbwiddis Awaiting requested review from dbwiddis

@sohami sohami Awaiting requested review from sohami

@msfroh msfroh Awaiting requested review from msfroh

+1 more reviewer

@psychbot psychbot psychbot approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

backport 2.x Backport to 2.x branch backport 2.10 Backport to 2.10 branch skip-changelog

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

RFC : Repository Registration for Remote Backed Storage

3 participants

@gbbafna @psychbot @sachinpkale