Skip to content

Commit c4fca2c

Browse files
aparajita31pandeykaushalmahi12vigyasharmashatejas
authored andcommitted
Introduce gRPC Interceptor Chain for transport-grpc (opensearch-project#19005)
* Introduce Interceptor to Grpc Plugin - Signed-off-by: Aparajita Pandey <aparajita29pandey@gmail.com> Signed-off-by: Aparajita Pandey <aparajita31pandey@gmail.com> * This is a combination of 7 commits. Add testcases for gRPC Interceptor Signed-off-by: Aparajita Pandey <aparajita31pandey@gmail.com> Ran spotlessAppy Signed-off-by: Aparajita Pandey <aparajita31pandey@gmail.com> refactor Signed-off-by: Aparajita Pandey <aparajita31pandey@gmail.com> spotlesssCheck Signed-off-by: Aparajita Pandey <aparajita31pandey@gmail.com> check if it verified Signed-off-by: Aparajita Pandey <aparajita31pandey@gmail.com> Added GrpcInterceptorChain check if it verified Signed-off-by: Aparajita Pandey <aparajita31pandey@gmail.com> revert Signed-off-by: Aparajita Pandey <aparajita31pandey@gmail.com> empty commit Signed-off-by: Aparajita Pandey <aparajita31pandey@gmail.com> Added GrpcInterceptorChain Signed-off-by: Aparajita Pandey <aparajita31pandey@gmail.com> Added GrpcInterceptorChain Signed-off-by: Aparajita Pandey <aparajita31pandey@gmail.com> Added GrpcInterceptorChain Signed-off-by: Aparajita Pandey <aparajita31pandey@gmail.com> testCases Signed-off-by: Aparajita Pandey <aparajita31pandey@gmail.com> logger Signed-off-by: Aparajita Pandey <aparajita31pandey@gmail.com> Resolve Conflict Signed-off-by: Aparajita Pandey <aparajita31pandey@gmail.com> javadoc Signed-off-by: Aparajita Pandey <aparajita31pandey@gmail.com> * javadoc Signed-off-by: Aparajita Pandey <aparajita31pandey@gmail.com> * cleanuo Signed-off-by: Aparajita Pandey <aparajita31pandey@gmail.com> * move GrpcInterceptor Interface to spi Signed-off-by: Aparajita Pandey <aparajita31pandey@gmail.com> * move GrpcInterceptor Interface to spi Signed-off-by: Aparajita Pandey <aparajita31pandey@gmail.com> * build.gradle fixes Signed-off-by: Aparajita Pandey <aparajita31pandey@gmail.com> * GrpcInterceptorChain refactoring Signed-off-by: Aparajita Pandey <aparajita31pandey@gmail.com> * GrpcInterceptorChain refactoring Signed-off-by: Aparajita Pandey <aparajita31pandey@gmail.com> * rebase Signed-off-by: Aparajita Pandey <aparajita31pandey@gmail.com> * coverage Signed-off-by: Aparajita Pandey <aparajita31pandey@gmail.com> * spotless Signed-off-by: Aparajita Pandey <aparajita31pandey@gmail.com> * Add logic to throw exception on workload group deletion with associated rules (opensearch-project#19502) Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> Signed-off-by: Aparajita Pandey <aparajita31pandey@gmail.com> * add documentation about debug-server-jvm in testing.md (opensearch-project#19567) Signed-off-by: Vigya Sharma <vigya.work@gmail.com> Signed-off-by: Aparajita Pandey <aparajita31pandey@gmail.com> * Adds a mapper for context aware segments grouping criteria (opensearch-project#19233) * Adds Context aware grouping mapper The change also stores grouping criteria for each doc in parsed context Signed-off-by: Tejas Shah <shatejas@amazon.com> * Adds a test for ContextAwareGroupingScript Signed-off-by: Tejas Shah <shatejas@amazon.com> * Adds ContextAwareGroupingScript support in mockScriptEngine to fix integ tests Signed-off-by: Tejas Shah <shatejas@amazon.com> * Corrects CHANGELOG.md Signed-off-by: Tejas Shah <shatejas@amazon.com> * Fixes LangPainlessClientYamlTestSuiteIT Signed-off-by: Tejas Shah <shatejas@amazon.com> --------- Signed-off-by: Tejas Shah <shatejas@amazon.com> Signed-off-by: Aparajita Pandey <aparajita31pandey@gmail.com> * Refactor README.md and GrpcInterceptorChain Constructor Signed-off-by: Aparajita Pandey <aparajita31pandey@gmail.com> * UpdateREADME & switch to implmentation Signed-off-by: Aparajita Pandey <aparajita31pandey@gmail.com> * refactoring Signed-off-by: Aparajita Pandey <aparajita31pandey@gmail.com> * refactoring Signed-off-by: Aparajita Pandey <aparajita31pandey@gmail.com> * Address PR review comments - Changed SPI dependencies from api to implementation - Made OrderedGrpcInterceptor a nested interface inside GrpcInterceptorProvider - Updated README with build.gradle extension declaration examples - Simplified duplicate order handling documentation Signed-off-by: Aparajita Pandey <aparajita31pandey@gmail.com> * empty commit Signed-off-by: Aparajita Pandey <aparajita31pandey@gmail.com> * resolve conflict Signed-off-by: Aparajita Pandey <aparajita31pandey@gmail.com> --------- Signed-off-by: Aparajita Pandey <aparajita31pandey@gmail.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com> Signed-off-by: Vigya Sharma <vigya.work@gmail.com> Signed-off-by: Tejas Shah <shatejas@amazon.com> Signed-off-by: Aparajita Pandey <37636092+aparajita31pandey@users.noreply.github.com> Co-authored-by: Kaushal Kumar <ravi.kaushal97@gmail.com> Co-authored-by: Vigya Sharma <vigyas@amazon.com> Co-authored-by: Tejas Shah <shatejas@amazon.com>
1 parent 26ab35c commit c4fca2c

16 files changed

Lines changed: 1698 additions & 32 deletions

File tree

CHANGELOG.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -61,6 +61,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
6161
- Use Lucene `pack` method for `half_float` and `usigned_long` when using `ApproximatePointRangeQuery`.
6262
- Add a mapper for context aware segments grouping criteria ([#19233](https://github.com/opensearch-project/OpenSearch/pull/19233))
6363
- Return full error for GRPC error response ([#19568](https://github.com/opensearch-project/OpenSearch/pull/19568))
64+
- Add pluggable gRPC interceptors with explicit ordering([#19005](https://github.com/opensearch-project/OpenSearch/pull/19005))
6465

6566
### Changed
6667
- Faster `terms` query creation for `keyword` field with index and docValues enabled ([#19350](https://github.com/opensearch-project/OpenSearch/pull/19350))

modules/transport-grpc/spi/README.md

Lines changed: 128 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@ The `transport-grpc-spi` module enables plugin developers to:
88
- Implement custom query converters for gRPC transport
99
- Extend gRPC protocol buffer handling
1010
- Register custom query types that can be processed via gRPC
11+
- Register gRPC interceptors with explicit ordering
1112

1213
## Key Components
1314

@@ -26,6 +27,26 @@ public interface QueryBuilderProtoConverter {
2627

2728
Interface for accessing the query converter registry. This provides a clean abstraction for plugins that need to convert nested queries without exposing internal implementation details.
2829

30+
### GrpcInterceptorProvider
31+
32+
Interface for providing gRPC interceptors that can intercept and process all incoming gRPC requests. This enables plugins to implement cross-cutting concerns like authentication, authorization, logging, and metrics collection.
33+
34+
```java
35+
public interface GrpcInterceptorProvider {
36+
List<OrderedGrpcInterceptor> getOrderedGrpcInterceptors();
37+
38+
/**
39+
* Nested interface for ordered gRPC interceptors.
40+
* Interceptors are executed in order based on their order() value,
41+
* with lower values executing first (e.g., 10 before 20).
42+
*/
43+
interface OrderedGrpcInterceptor {
44+
int order(); // Lower values execute first
45+
ServerInterceptor getInterceptor();
46+
}
47+
}
48+
```
49+
2950
## Usage for Plugin Developers
3051

3152
### 1. Add Dependency
@@ -36,10 +57,47 @@ Add the SPI dependency to your plugin's `build.gradle`:
3657
dependencies {
3758
compileOnly 'org.opensearch.plugin:transport-grpc-spi:${opensearch.version}'
3859
compileOnly 'org.opensearch:protobufs:${protobufs.version}'
60+
compileOnly 'io.grpc:grpc-api:${versions.grpc}'
3961
}
4062
```
4163

42-
### 2. Implement Custom Query Converter
64+
### 2. Declare Extension in build.gradle
65+
66+
In your `build.gradle`, declare that your plugin extends `transport-grpc`. This automatically adds the `extended.plugins=transport-grpc` entry to the auto-generated `plugin-descriptor.properties` file: :
67+
68+
```groovy
69+
opensearchplugin {
70+
name 'your-plugin-name'
71+
description 'Your plugin description'
72+
classname 'org.opensearch.yourplugin.YourPlugin'
73+
extendedPlugins = ['transport-grpc'] // Declare extension here
74+
}
75+
```
76+
**Real-world examples:**
77+
- [OpenSearch Reporting Plugin](https://github.com/opensearch-project/reporting/blob/main/build.gradle#L92)
78+
- [OpenSearch k-NN Plugin](https://github.com/opensearch-project/k-NN/blob/main/build.gradle#L319)
79+
80+
### 3. Create SPI Registration File(s)
81+
82+
Create a service file denoting your plugin's implementation of a service interface.
83+
84+
For QueryBuilderProtoConverter implementations:
85+
`src/main/resources/META-INF/services/org.opensearch.transport.grpc.spi.QueryBuilderProtoConverter`:
86+
87+
```
88+
org.opensearch.mypackage.MyCustomQueryConverter
89+
```
90+
91+
For `GrpcInterceptorProvider` implementations: `src/main/resources/META-INF/services/org.opensearch.transport.grpc.spi.GrpcInterceptorProvider`:
92+
93+
```
94+
org.opensearch.mypackage.SampleInterceptorProvider
95+
```
96+
97+
98+
99+
## QueryBuilderProtoConverter
100+
### 1. Implement Custom Query Converter
43101

44102
```java
45103
public class MyCustomQueryConverter implements QueryBuilderProtoConverter {
@@ -58,7 +116,7 @@ public class MyCustomQueryConverter implements QueryBuilderProtoConverter {
58116
}
59117
```
60118

61-
### 3. Register Your Converter
119+
### 2. Register Your Converter
62120

63121
In your plugin's main class, return the converter from createComponents:
64122

@@ -80,23 +138,7 @@ public class MyPlugin extends Plugin {
80138
}
81139
```
82140

83-
**Step 3b: Create SPI Registration File**
84-
85-
Create a file at `src/main/resources/META-INF/services/org.opensearch.transport.grpc.spi.QueryBuilderProtoConverter`:
86-
87-
```
88-
org.opensearch.mypackage.MyCustomQueryConverter
89-
```
90-
91-
**Step 3c: Declare Extension in Plugin Descriptor**
92-
93-
In your `plugin-descriptor.properties`, declare that your plugin extends transport-grpc:
94-
95-
```properties
96-
extended.plugins=transport-grpc
97-
```
98-
99-
### 4. Accessing the Registry (For Complex Queries)
141+
### 3. Accessing the Registry (For Complex Queries)
100142

101143
If your converter needs to handle nested queries (like k-NN's filter clause), you'll need access to the registry to convert other query types. The transport-grpc plugin will inject the registry into your converter.
102144

@@ -188,7 +230,7 @@ public class KNNQueryBuilderProtoConverter implements QueryBuilderProtoConverter
188230
./gradlew :modules:transport-grpc:spi:test
189231
```
190232

191-
### Testing Your Custom Converter
233+
### 4. Testing Your Custom Converter
192234

193235
```java
194236
@Test
@@ -275,3 +317,69 @@ org.opensearch.knn.grpc.proto.request.search.query.KNNQueryBuilderProtoConverter
275317

276318
**Why k-NN needs the registry:**
277319
The k-NN query's `filter` field is a `QueryContainer` protobuf type that can contain any query type (MatchAll, Term, Terms, etc.). The k-NN converter needs access to the registry to convert these nested queries to their corresponding QueryBuilder objects.
320+
321+
322+
## gRPC Interceptors
323+
324+
### Overview
325+
326+
Intercept incoming gRPC requests for authentication, authorization, logging, metrics, rate limiting,etc
327+
328+
### Basic Usage
329+
330+
**1. Implement Provider:**
331+
```java
332+
public class SampleInterceptorProvider implements GrpcInterceptorProvider {
333+
@Override
334+
public List<GrpcInterceptorProvider.OrderedGrpcInterceptor> getOrderedGrpcInterceptors() {
335+
return Arrays.asList(
336+
// First interceptor (order = 5, runs first)
337+
new GrpcInterceptorProvider.OrderedGrpcInterceptor() {
338+
@Override
339+
public int order() { return 5; } // Lower = higher priority
340+
341+
@Override
342+
public ServerInterceptor getInterceptor() {
343+
return (call, headers, next) -> {
344+
String methodName = call.getMethodDescriptor().getFullMethodName();
345+
System.out.println("First interceptor - Method: " + methodName);
346+
return next.startCall(call, headers);
347+
};
348+
}
349+
},
350+
351+
// Second interceptor (order = 10, runs after first)
352+
new GrpcInterceptorProvider.OrderedGrpcInterceptor() {
353+
@Override
354+
public int order() { return 10; }
355+
356+
@Override
357+
public ServerInterceptor getInterceptor() {
358+
return (call, headers, next) -> {
359+
System.out.println("Second interceptor - Processing request");
360+
return next.startCall(call, headers);
361+
};
362+
}
363+
}
364+
);
365+
}
366+
}
367+
```
368+
369+
### Understanding Interceptor Ordering
370+
371+
#### How Order Values Work
372+
373+
Interceptors are executed based on their `order()` value. Lower order values execute first. The interceptor chain processes requests from lowest to highest order value.
374+
375+
```
376+
Request → [order=5] → [order=10] → [order=100] → Service Handler
377+
```
378+
379+
#### Duplicate Order Values
380+
381+
Each interceptor must have a unique order value. If duplicate order values are detected, OpenSearch will fail to start with an `IllegalArgumentException`.
382+
```
383+
IllegalArgumentException: Multiple gRPC interceptors have the same order value: 10.
384+
Each interceptor must have a unique order value.
385+
```

modules/transport-grpc/spi/build.gradle

Lines changed: 22 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -15,8 +15,28 @@ base {
1515
}
1616

1717
dependencies {
18-
api project(":server")
19-
api "org.opensearch:protobufs:${versions.opensearchprotobufs}"
18+
implementation project(":server")
19+
implementation "org.opensearch:protobufs:${versions.opensearchprotobufs}"
20+
implementation "io.grpc:grpc-api:${versions.grpc}"
2021

2122
testImplementation project(":test:framework")
2223
}
24+
25+
thirdPartyAudit {
26+
ignoreMissingClasses(
27+
'com.google.common.base.Joiner',
28+
'com.google.common.base.MoreObjects',
29+
'com.google.common.base.MoreObjects$ToStringHelper',
30+
'com.google.common.base.Objects',
31+
'com.google.common.base.Preconditions',
32+
'com.google.common.base.Strings',
33+
'com.google.common.base.Throwables',
34+
'com.google.common.collect.ImmutableList',
35+
'com.google.common.collect.ImmutableMap',
36+
'com.google.common.collect.Maps',
37+
'com.google.common.collect.Sets',
38+
'com.google.common.io.BaseEncoding',
39+
'com.google.common.io.ByteStreams',
40+
'com.google.common.util.concurrent.ListenableFuture'
41+
)
42+
}
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
18ddd409fb9bc0209d216854ca584d027e68210b

0 commit comments

Comments
 (0)