EQL: Filter out null join keys in sequence queries#78195
EQL: Filter out null join keys in sequence queries#78195astefan wants to merge 3 commits intoelastic:masterfrom
Conversation
elasticmachine
added
the
Team:QL (Deprecated)
|
Pinging @elastic/es-ql (Team:QL) |
| // do not join on null values | ||
| if (keyNames.isEmpty() == false) { | ||
| BoolQueryBuilder nullValuesFilter = boolQuery(); | ||
| for (int keyIndex = 0; keyIndex < keyNames.size(); keyIndex++) { |
There was a problem hiding this comment.
Use foreach instead:
for (String keyName : keyNames) {
}
| BoolQueryBuilder nullValuesFilter = boolQuery(); | ||
| for (int keyIndex = 0; keyIndex < keyNames.size(); keyIndex++) { | ||
| // add an "exists" query for each join key to filter out any non-existent values | ||
| nullValuesFilter.must(existsQuery(keyNames.get(keyIndex))); |
There was a problem hiding this comment.
No need to score, just use filter. Which removes the need for wrapping it into a bool query and simply use RuntimeUtils.addFilter for every existsQuery.
is enabled in this cluster).
|
@elasticmachine run elasticsearch-ci/bwc elasticsearch-ci/part-2 |
|
@elasticmachine run elasticsearch-ci/part-2 |
| .setHttpClientConfigCallback(new HttpClientConfigCallback() { | ||
| @Override | ||
| public HttpAsyncClientBuilder customizeHttpClient(HttpAsyncClientBuilder httpClientBuilder) { | ||
| return httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider); | ||
| } | ||
| }) |
There was a problem hiding this comment.
I think it makes it generally easier with testing, so good to have it, but curious if this was added for a specific reason.
There was a problem hiding this comment.
Yes, there is a better reason: since #70114 security is enabled in this test cluster.
There was a problem hiding this comment.
Thank you.
(teaching me a lesson about reading PRs chronologically...)
|
Superseded by #79677. |
6 participants
Joining on
nullkeys in sequences can lead to a high amount of queries and matches in a cluster or CCS multi-cluster scenario where some indices have mappings with existent fields while others don't. This change removes support for joining onnullvalues by filtering them out pro-actively with anexistsfilter.