[Feature] Introduces Centralized Resource Access Control and Sharing

[DarshitChanpura]

Description

Introduces a new authorization mechanism to control access to resources defined by plugins.
There are 4 major components to this PR:

1. Introduces an SPI that defines the contracts for a resource sharing plugin as well as Resource Access Control plugin.
2. Introduces a client that can be used by plugins to implement resource-access-control.
3. Introduces 4 new APIs that expose resource-sharing and access verification capabilities.
4. Adds a sample resource plugin with tests to demonstrate the usage of this new feature.
5. Adds a feature flag to toggle the feature. Feature is enabled by default.

• Category : New feature
• Why these changes are required?
  • At present, plugins have implemented in-house authorization mechanisms for a lack of centralized framework. This PR introduces a centralized way to offload resource permissions check to security plugin.

Issues Resolved

• Related to Resource Permissions and Sharing #4500

Testing

• automated and manual testing

Check List

• New functionality includes testing
• New functionality has been documented
• New Roles/Permissions have a corresponding security dashboards plugin PR
• API changes companion pull request created
• Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[DarshitChanpura]

Sample-resource-plugin integrations tests pass locally:

➜  security git:(resource-sharing-spi) ./gradlew clean :opensearch-sample-resource-plugin:integrationTest
=======================================
OpenSearch Build Hamster says Hello!
  Gradle Version        : 8.13
  OS Info               : Mac OS X 15.3.1 (aarch64)
  JDK Version           : 21 (Eclipse Temurin JDK)
  JAVA_HOME             : /.sdkman/candidates/java/21.0.4-tem
  Random Testing Seed   : 20F70E3D35573259
  In FIPS 140 mode      : false
=======================================

> Task :opensearch-security-common:compileJava
Note: ~/security/common/src/main/java/org/opensearch/security/common/support/Utils.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.

> Task :compileJava
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.

> Task :compileIntegrationTestJava
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.

[Incubating] Problems report is available at: file:///~/security/build/reports/problems/problems-report.html

Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

For more on this, please refer to https://docs.gradle.org/8.13/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.

BUILD SUCCESSFUL in 42s
19 actionable tasks: 19 executed
➜  security git:(resource-sharing-spi) 

[nibix]

This is a huge change. Meanwhile, it's architectural decisions like the common module are not entirely clear to me, which makes reviewing quite difficult.

Thus, I could only skim the changes, and added a few comments.

For a thorough review, I do not have the time ATM, though :-(

[nibix]

How do plugins consume this? Is this published as a separate maven package?

This README should probably have instructions on how to consume it.

[nibix]

It is unclear to me why these classes need to be moved into an own module. I see that the client module depends on it via an implementation dependency, right? So, will each plugin then carry a JAR with these implementations?

As some of them feel rather security-core-ish, like AdminDNs, I am not sure if it is good to move these out of the security core.

[nibix]

A resource sharing plugin indirectly cosumes the code via the client dependency, right?

Will this be also the case when security is disabled?

If so, what's the strategy for the case that security is disabled? Here it looks like that resources are not available at all when security is disabled (i.e., the user object is null).

[DarshitChanpura]

When security is disabled, the call will not reach here. Instead it would fail [here in the client] (

security/client/src/main/java/org/opensearch/security/client/resources/ResourceSharingNodeClient.java

Line 68 in e6d43b6

client.execute(ResourceAccessAction.INSTANCE, request, verifyAccessResponseListener(listener));

) with action not found

[nibix]

Does that mean that resource plugins cannot be used when security is disabled?

[nibix]

Why is this change necessary?

[DarshitChanpura]

At present, the persistent header is only utilized in ResourceAccessHandler. And since we have an implementation of User in common package, we utilize that package's User here to avoid mismatch issues during runtime. This will be cleaned up once User is completely moved to common package.

[nibix]

Following up on this: https://github.com/opensearch-project/security/pull/5016/files#r1992120543

I think this indicates a deeper problem that needs to be investigated.

[DarshitChanpura]

I believe this might require updating the test framework. But I believe that could be done in a separate PR.

[DarshitChanpura]

@nibix I understand this is a huge change, but I've tried to split it into multiple components, and highlighted the same in PR description.

1. Client was added to allow plugins to call the resource sharing APIs. I've followed the framework implement by ml-commons
2. Common module was added to avoid circular dependency between SPI, client and security. This pattern is already present in ml-commons.
3. There are some redundant files that are needed in common, which will be removed in a fast follow PR. (adding those changes is out of scope for this PR)
4. I've added a comprehensive developer documentation here: https://github.com/opensearch-project/security/blob/d88ca438d69c17e0eb1b904178e10373a617e37b/RESOURCE_ACCESS_CONTROL_FOR_PLUGINS.md

I've documented the final design in detail here: #4500 (comment)

[DarshitChanpura]

As reviewers are leaning towards smaller PR, I'm closing this monolith, in lieu of individual PRs:

• [Resource Access Control] [Part1] Introduces SPI for resource access control #5185
• [Resource Access Control] [Part2] Introduces a client for Resource Access Control and adds concrete implementation via common package #5186
• [Resource Access Control] [Part3] Introduces a sample plugin to demonstrate usage of Resource Access Control feature #5187

[nibix]

@DarshitChanpura

I understand this is a huge change, but I've tried to split it into multiple components, and highlighted the same in PR description.

Apologies, I did not want to sound harsh. Just wanted to point out that I cannot give a thorough and complete review at the moment due to the size and time constraints.