-
Notifications
You must be signed in to change notification settings - Fork 2.5k
Closed
Labels
SearchSearch query, autocomplete ...etcSearch query, autocomplete ...etcbugSomething isn't workingSomething isn't workinggood first issueGood for newcomersGood for newcomers
Description
Describe the bug
Currently, IndicesQueryCache$CachingWeightWrapper#count does not delegate to internal weight object and keeps returning -1 from the Weight#count implementation. Due to this leaf counting cannot be done even when it is supported by Weight implementations like PointRangeQuery#createWeight
Expected behavior
Instrumenting the code for FiltersAggregator and observing with and without the change:
Without this change:
Original Query : class org.apache.lucene.document.LongPoint$1
Rewritten Query : class org.apache.lucene.search.IndexOrDocValuesQuery
Original Query : class org.apache.lucene.document.LongPoint$1
Rewritten Query : class org.apache.lucene.search.IndexOrDocValuesQuery
Wrapper class type : class org.apache.lucene.search.LRUQueryCache$CachingWrapperWeight
Aggregation count from the filters : 0 value : -1 class type : class org.apache.lucene.document.LongPoint$1 : class org.opensearch.indices.IndicesQueryCache$CachingWeightWrapper
Wrapper class type : class org.apache.lucene.search.LRUQueryCache$CachingWrapperWeight
Aggregation count from the filters : 1 value : -1 class type : class org.apache.lucene.document.LongPoint$1 : class org.opensearch.indices.IndicesQueryCache$CachingWeightWrapper
With the change:
Original Query : class org.apache.lucene.document.LongPoint$1
Rewritten Query : class org.apache.lucene.search.IndexOrDocValuesQuery
Original Query : class org.apache.lucene.document.LongPoint$1
Rewritten Query : class org.apache.lucene.search.IndexOrDocValuesQuery
Wrapper class type : class org.apache.lucene.search.LRUQueryCache$CachingWrapperWeight
PointRangeQuery#count method was invoked
Aggregation count from the filters : 0 value : 4349068 class type : class org.apache.lucene.document.LongPoint$1 : class org.opensearch.indices.IndicesQueryCache$CachingWeightWrapper
Wrapper class type : class org.apache.lucene.search.LRUQueryCache$CachingWrapperWeight
PointRangeQuery#count method was invoked
Aggregation count from the filters : 1 value : 5981766 class type : class org.apache.lucene.document.LongPoint$1 : class org.opensearch.indices.IndicesQueryCache$CachingWeightWrapper
Additional context
Instrumentation Code
diff --git a/server/src/main/java/org/opensearch/search/aggregations/bucket/filter/FiltersAggregator.java b/server/src/main/java/org/opensearch/search/aggregations/bucket/filter/FiltersAggregator.java
index 84e0218d8ea..b54b46a8254 100644
--- a/server/src/main/java/org/opensearch/search/aggregations/bucket/filter/FiltersAggregator.java
+++ b/server/src/main/java/org/opensearch/search/aggregations/bucket/filter/FiltersAggregator.java
@@ -175,6 +175,7 @@ public class FiltersAggregator extends BucketsAggregator {
Weight[] filters = this.filters.get();
final Bits[] bits = new Bits[filters.length];
for (int i = 0; i < filters.length; ++i) {
+ System.out.println("Aggregation count from the filters : " + i + " value : " + filters[i].count(ctx) + " class type : " + filters[i].getQuery().getClass() + " : " + filters[i].getClass());
bits[i] = Lucene.asSequentialAccessBits(ctx.reader().maxDoc(), filters[i].scorerSupplier(ctx));
}
return new LeafBucketCollectorBase(sub, null) {
diff --git a/server/src/main/java/org/opensearch/search/aggregations/bucket/filter/FiltersAggregatorFactory.java b/server/src/main/java/org/opensearch/search/aggregations/bucket/filter/FiltersAggregatorFactory.java
index 8741213f988..be96c0505b2 100644
--- a/server/src/main/java/org/opensearch/search/aggregations/bucket/filter/FiltersAggregatorFactory.java
+++ b/server/src/main/java/org/opensearch/search/aggregations/bucket/filter/FiltersAggregatorFactory.java
@@ -32,10 +32,7 @@
package org.opensearch.search.aggregations.bucket.filter;
-import org.apache.lucene.search.IndexSearcher;
-import org.apache.lucene.search.Query;
-import org.apache.lucene.search.ScoreMode;
-import org.apache.lucene.search.Weight;
+import org.apache.lucene.search.*;
import org.opensearch.index.query.QueryShardContext;
import org.opensearch.search.aggregations.AggregationInitializationException;
import org.opensearch.search.aggregations.Aggregator;
@@ -102,7 +99,9 @@ public class FiltersAggregatorFactory extends AggregatorFactory {
IndexSearcher contextSearcher = searchContext.searcher();
weights = new Weight[filters.length];
for (int i = 0; i < filters.length; ++i) {
- this.weights[i] = contextSearcher.createWeight(contextSearcher.rewrite(filters[i]), ScoreMode.COMPLETE_NO_SCORES, 1);
+ System.out.println("Original Query : " + ((IndexOrDocValuesQuery)filters[i]).getIndexQuery().getClass());
+ System.out.println("Rewritten Query : " + contextSearcher.rewrite(filters[i]).getClass());
+ this.weights[i] = contextSearcher.createWeight(((IndexOrDocValuesQuery)filters[i]).getIndexQuery(), ScoreMode.COMPLETE_NO_SCORES, 1);
}
} catch (IOException e) {
throw new AggregationInitializationException("Failed to initialse filters for aggregation [" + name() + "]", e);
Code change:
diff --git a/server/src/main/java/org/opensearch/indices/IndicesQueryCache.java b/server/src/main/java/org/opensearch/indices/IndicesQueryCache.java
index 2669da3f417..97718aeca34 100644
--- a/server/src/main/java/org/opensearch/indices/IndicesQueryCache.java
+++ b/server/src/main/java/org/opensearch/indices/IndicesQueryCache.java
@@ -186,6 +186,12 @@ public class IndicesQueryCache implements QueryCache, Closeable {
return in.bulkScorer(context);
}
+ @Override
+ public int count(LeafReaderContext context) throws IOException {
+ System.out.println("Wrapper class type : " + in.getClass());
+ return in.count(context);
+ }
+
@Override
public boolean isCacheable(LeafReaderContext ctx) {
return in.isCacheable(ctx);
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
SearchSearch query, autocomplete ...etcSearch query, autocomplete ...etcbugSomething isn't workingSomething isn't workinggood first issueGood for newcomersGood for newcomers
Type
Projects
Status
✅ Done