[Documentation] Breaking changes in 2.0

[saratvemulapalli]

This is a meta issue which lists down all breaking changes in OpenSearch 2.0

Inclusive Naming

• Add alternative usages for the REST APIs and configurations that contain "master" terminology #1549 - Breaking changes in master REST APIs
• Add alternative usage for the configuration that contains "blacklist/whitelist" terminology #1547

Concerning changes

• All REST APIs are backwards compatible but changes have to be made in the consumers (like plugins, clients) as the masterand blacklist/whitelist usages will deprecated in 2.0, and eventually be removed.
• master renamed to cluster_manager
• blacklist / whitelist renamed to denylist / allowlist
• Please check the issue [FEATURE] Change the "master" nomenclature documentation-website#450 for the changes, and issues created in each impacted plugin repository are tracked in issue [Meta] Deprecate non-inclusive naming  #2589

Type Changes:

• [Meta] Remove mapping types #1940

Concerning changes

• RestIndexAction Remove type mapping from document index API #2026
• RestUpdateAction [Remove] Type mapping parameter from document update API #2204
• RestDeleteAction [Remove] Types from DocWrite Request and Response #2239
• RestGetAction [Remove] Types from GET/MGET #2168
• RestMultiGetAction [Remove] Types from GET/MGET #2168
• RestExplainAction [Remove] types from SearchHit and Explain API #2205
• RestBulkAction [Remove] type support from Bulk API #2215
• RestNoopBulkAction Remove type end-points from NOOP bulk and search action #2261
• RestSearchAction Remove type end-points from search and related APIs #2263 and [Remove] Types from GET/MGET #2168
• RestMultiSearchAction Remove type end-points from search and related APIs #2263
• RestSearchTemplateAction Remove type end-points from search and related APIs #2263
• RestMultiSearchTemplateAction [Remove] Type mapping end-points from RestMultiSearchTemplateAction #2433 & Remove type end-points from search and related APIs #2263
• RestNoopSearchAction Remove type end-points from NOOP bulk and search action #2261
• RestCountAction Remove type end-points from count action #2379
• RestUpdateByQueryAction Remove type end-points from search and related APIs #2263
  RestDeleteByQueryAction Remove type end-points from search and related APIs #2263
• RestGetFieldMappingAction Removes types from mapping APIs #2238
• RestGetMappingAction Removes types from mapping APIs #2238
• RestPutMappingAction Removes types from mapping APIs #2238
• RestValidateQueryAction Remove type from validate query API #2255
• RestGetSourceAction [Remove] Types from GET/MGET #2168
• RestTermVectorsAction [Remove] Type parameter from TermVectors API #2104 and Remove type end-points from search and related APIs #2263
• RestMultiTermVectorsAction Remove type end-points from search and related APIs #2263
• SearchRequest, SearchResponse`
• ShardGetService
• DocWrite Request, DocWriteResponse
• Client.prepare
• AbstractClient, Client
• getMapping in IndicesClient
• PutMappingRequest

Example Changes in plugins

opensearch-project/asynchronous-search#110

Lucene 9 upgrade

• [Upgrade] OpenSearch 2.0 to Lucene 9.0.0 release #1067
• Set target and source compatibility to 11, required by Lucene 9. #2407

Gradle 7 upgrade

• Upgrade to Gradle 7 opensearch-plugins#107

Example Changes in plugins

opensearch-project/job-scheduler#133

Deprecated code in 2.0+

• Remove LegacyVersion.V_6.x Constants and Deprecated Code in OpenSearch 2.0 #1674
• Remove Deprecated TransportClient #1669

Concerning changes

• All usages of TransportClient are now deprecated for 2.0. They should start moving to High Level Rest Client.
• Removed all version constants V_6_x_x
• Removed Synced Flush API [Remove] Deprecated Synced Flush API #1761
• Removed search.remote settings Remove deprecated search.remote settings #1870
• Removed all default mappings [Remove] Default Mapping #2151
• Removed SimpleFS Remove Deprecated SimpleFS #1639
• Removed Zen1 Discovery [Remove] Deprecated Zen1 Discovery #1216

JDK Support

• Add support for JDK 17 opensearch-plugins#129
• Remove JavaVersion in favour of standard Runtime.Version #3003

Settings Changes

• override_main_response_version was removed, [BUG] Breaking changes with opensearch.http.override_main_response_version=true #3023, [Remove] MainResponse version override cluster setting #3031

[cliu123]

Security plugin breaking changes:

• Security plugin will drop the rolling upgrade support ODFE -> OpenSearch 1.2.0, but will still support the upgrade patch ODFE 1.13 -> OpenSearch 1.x -> OpenSearch 2.0.0
• Security plugin no longer supports configuration migration DynamicConfigModelV6 -> DynamicConfigModelV7.

Security Dashboards plugin breaking changes:

• Enforce authentication on api/status by default. Request to api/status API is required to provide valid credential unless it's explicitly configured to bypass authentication(opensearch_security.auth.unauthenticated_routes: ['/api/reporting/stats', '/api/status']).

[brijos]

Destination API will break in 2.0 - opensearch-project/alerting#105

[travisbenedict]

Are there any breaking changes related to the expression scripting language?

I ran tests for OpenSearch 2.0 using OpenSearch Benchmark and was receiving errors for this query which uses expression scripting. The error message was:

"error-description": "search_phase_execution_exception ({'error': {'root_cause': [{'type': 'illegal_argument_exception', 'reason': 'Unsupported api 589824'}], 'type': 'search_phase_execution_exception', 'reason': 'all shards failed', 'phase': 'query', 'grouped': True, 'failed_shards': [{'shard': 0, 'index': 'geonames', 'node': 'G2m1_1_mR0qqSCHaYUiwMA', 'reason': {'type': 'query_shard_exception', 'reason': 'script_score: the script could not be loaded', 'index': 'geonames', 'index_uuid': 'sXxM4QaDSlGryh1Slg5pbg', 'caused_by': {'type': 'illegal_argument_exception', 'reason': 'Unsupported api 589824'}}}], 'caused_by': {'type': 'illegal_argument_exception', 'reason': 'Unsupported api 589824'}}, 'status': 400})"

All other operations for all other OpenSearch Benchmark workloads were successful. I also tested against OpenSearch 1.3 and did not receive this error.

More information can be found in opensearch-project/opensearch-benchmark-workloads#41