[Meta][FEATURE] Release extensions as experimental feature

[saratvemulapalli]

Is your feature request related to a problem?

Release extensions support as an experimental release in OpenSearch 2.x line.

• Need to have this behind a feature flag.
• Merge all code changes from feature/extensions to main and 2.x
• [FEATURE] Publish artifacts to Maven #75
• Publish SDK snapshots to maven Adding maven Snapshots repo for SDK #311
• Add versioning support for SDK #337
• Add support for versioning
• Add test framework support to test extensions
• Release opensearch-java-sdk v0.1 [FEATURE] Experimental release of opensearch-sdk-java #836
• Documentation on how to write an extension https://github.com/opensearch-project/opensearch-sdk-java/blob/main/CREATE_YOUR_FIRST_EXTENSION.md

[peternied]

From the Identity roadmap [1] the minimum viable system includes

• internal users system
• authentication via http BASIC authentication [2]
• identity information is passed to extensions
• permissions checked via API

As roadmap items are completed more details will be added to the roadmap document [1].

Staging feature integration

It sounds like there are two options for accessing the features that extensions and the anomaly detection extension will rely on:

A) feature/identity merges into main -> latest minor before feature/extensions
B) feature/identity merges into feature/extensions, all changes move as one from main -> latest minor

I think I would prefer option A for identity to be handled on its own merge to main and then is backported into the next minor release, it will be a smaller change to get reviewed and approved. The disadvantages is that it needs to complete this process first, and the two features areas might arrive at different times impacting the amount of time to integrate the components.

Alternatively with option B moving into main together, integration between components will happen earlier providing more time to iron out missing functionality.

Detailed integration list

In order to prepare for the integration window between both features, I suggest we chart out the user scenarios that are required to support the extension release - @minalsha can you help us build up this list of scenarios?

[1] https://github.com/opensearch-project/OpenSearch/blob/feature/identity/IDENTITY_ROADMAP.md#minimal-viable-system
[2] https://developer.mozilla.org/en-US/docs/Web/HTTP/Authentication#basic_authentication_scheme

[dbwiddis]

I'd like a little bit more specificity between these two bullets:

• identity information is passed to extensions
• permissions checked via API

I'll repeat my "identity and access are different things" autoreply and add that we're considering two different types of access tied to identity:

• Short-duration "logged-in" access to enable real-time REST requests from users; access only needs to last until the request is responded to and then terminates, but this allows our current relay model (from the DESIGN document) to work securely
• Long-duration "execute on my behalf" access to enable unattended future actions (job scheduler + anomaly detector + alerting example)

[peternied]

identity information is passed to extensions

[Use Case 1] Anomaly Detection extension checks if the current user is the same as the user that created a detector object.

Extension will receive a PrincipalIdentifierToken [1] with every OS->Extension request that is an encrypted representation of the user. The naïve algorithm is already in the extension feature branch, and our feature branch has a cryptographically secure algorithm.

Are there other scenarios that need identity information such as a display name?

permissions checked via API

[Use Case 2] Anomaly Detection extension only allows users that have been granted permissions to retrieve, describe, modify, delete detectors. This should be set for all registered actions in Anomaly Detection.

Before actions are executed permissions are checked based on the action name. When classes extend ActionType the name passed to the constructor is the permission name, for GetAnomalyDetectorAction [2] the name is cluster:admin/opendistro/ad/detectors/get.

[1] https://github.com/opensearch-project/OpenSearch/blob/feature/extensions/server/src/main/java/org/opensearch/identity/PrincipalIdentifierToken.java

[2] https://github.com/opensearch-project/anomaly-detection/blob/4baf75ca46da02c4e70016d34982f2c5a34d0acd/src/main/java/org/opensearch/ad/transport/GetAnomalyDetectorAction.java#L19

Note: I'll codify these in our documents

[owaiskazi19]

SDK is released now: https://aws.oss.sonatype.org/content/repositories/releases/org/opensearch/sdk/opensearch-sdk-java/0.1.0/.