Enable skip_list for @timestamp field or index sort field by default by asimmahmood1 · Pull Request #19480 · opensearch-project/OpenSearch

asimmahmood1 · 2025-09-30T16:54:24Z

Description

this will help logging indexing that do not explictly enable sort, e.g. big5 benchmark
currently skip_list is set to false by default
this will help with date histogram aggregation queries like in Adding logic for histogram aggregation using skiplist #19130

Related Issues

Resolves #19123

Check List

Functionality includes testing.
[n/a] API changes companion pull request created, if applicable.
Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

github-actions · 2025-09-30T17:21:24Z

❌ Gradle check result for 89c3fba: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

codecov · 2025-09-30T23:26:57Z

Codecov Report

❌ Patch coverage is 87.50000% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 72.94%. Comparing base (83b2a6d) to head (13ec937).
⚠️ Report is 5 commits behind head on main.

Files with missing lines	Patch %	Lines
...a/org/opensearch/index/mapper/DateFieldMapper.java	87.50%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #19480      +/-   ##
============================================
+ Coverage     72.93%   72.94%   +0.01%     
  Complexity    70236    70236              
============================================
  Files          5700     5700              
  Lines        322150   322157       +7     
  Branches      46616    46618       +2     
============================================
+ Hits         234966   235007      +41     
+ Misses        68225    68163      -62     
- Partials      18959    18987      +28

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

jainankitk

LGTM!

@timestamp

* to test out skiplist based date histogram * skiplist is enabled by default http_logs has @timestamp field: opensearch-project/OpenSearch#19480 * TODO: add index.sort.field option on @timestamp Signed-off-by: Asim Mahmood <asim.seng@gmail.com>

@timestamp

* to test out skiplist based date histogram * skiplist is enabled by default http_logs has @timestamp field: opensearch-project/OpenSearch#19480 * TODO: add index.sort.field option on @timestamp Signed-off-by: Asim Mahmood <asim.seng@gmail.com>

@timestamp

* to test out skiplist based date histogram * skiplist is enabled by default http_logs has @timestamp field: opensearch-project/OpenSearch#19480 * TODO: add index.sort.field option on @timestamp Signed-off-by: Asim Mahmood <asim.seng@gmail.com> (cherry picked from commit bf2c36e) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

@timestamp

* to test out skiplist based date histogram * skiplist is enabled by default http_logs has @timestamp field: opensearch-project/OpenSearch#19480 * TODO: add index.sort.field option on @timestamp (cherry picked from commit bf2c36e) Signed-off-by: Asim Mahmood <asim.seng@gmail.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

…-project#19480) Signed-off-by: Asim Mahmood <asim.seng@gmail.com>

…ensearch-project#19480)" Fixes opensearch-project#19660 This reverts commit 08ed9ee.

…ensearch-project#19480)" Fixes opensearch-project#19660 This reverts commit 08ed9ee. Signed-off-by: Asim Mahmood <asim.seng@gmail.com>

…ensearch-project#19480)" This reverts commit 08ed9ee. Fixes opensearch-project#19660 Signed-off-by: Asim Mahmood <asim.seng@gmail.com>

@timestamp

* to test out skiplist based date histogram * skiplist is enabled by default http_logs has @timestamp field: opensearch-project/OpenSearch#19480 * TODO: add index.sort.field option on @timestamp Signed-off-by: Asim Mahmood <asim.seng@gmail.com>

@timestamp

* to test out skiplist based date histogram * skiplist is enabled by default http_logs has @timestamp field: opensearch-project/OpenSearch#19480 * TODO: add index.sort.field option on @timestamp Signed-off-by: Asim Mahmood <asim.seng@gmail.com>

@timestamp

* to test out skiplist based date histogram * skiplist is enabled by default http_logs has @timestamp field: opensearch-project/OpenSearch#19480 * TODO: add index.sort.field option on @timestamp Signed-off-by: Asim Mahmood <asim.seng@gmail.com> (cherry picked from commit 2559178) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

@timestamp

) * to test out skiplist based date histogram * skiplist is enabled by default http_logs has @timestamp field: opensearch-project/OpenSearch#19480 * TODO: add index.sort.field option on @timestamp (cherry picked from commit 2559178) Signed-off-by: Asim Mahmood <asim.seng@gmail.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

@timestamp

…ch-project#727) * to test out skiplist based date histogram * skiplist is enabled by default http_logs has @timestamp field: opensearch-project/OpenSearch#19480 * TODO: add index.sort.field option on @timestamp Signed-off-by: Asim Mahmood <asim.seng@gmail.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com>

asimmahmood1 requested review from a team, Bukhtawar, CEHENKLE, Rishikesh1159, anasalkouz, andrross, ashking94, cwperks, dbwiddis, gbbafna, jed326, kotwanikunal, mch2, msfroh, owaiskazi19, reta, sachinpkale, saratvemulapalli, shwetathareja and sohami as code owners September 30, 2025 16:54

github-actions bot added enhancement Enhancement or improvement to existing feature or request Indexing Indexing, Bulk Indexing and anything related to indexing Roadmap:Search Project-wide roadmap label Search:Performance v3.3.0 labels Sep 30, 2025

asimmahmood1 self-assigned this Sep 30, 2025

asimmahmood1 added this to Performance Roadmap Sep 30, 2025

github-project-automation bot moved this to Todo in Performance Roadmap Sep 30, 2025

asimmahmood1 added the Search:Aggregations label Sep 30, 2025

jainankitk approved these changes Sep 30, 2025

View reviewed changes

jainankitk merged commit 08ed9ee into opensearch-project:main Sep 30, 2025
61 of 63 checks passed

github-project-automation bot moved this from In Progress to Done in Performance Roadmap Sep 30, 2025

asimmahmood1 mentioned this pull request Oct 8, 2025

Add date histogram with filter, and filter and metrics opensearch-project/opensearch-benchmark-workloads#700

Merged

6 tasks

BrewTestBot mentioned this pull request Oct 15, 2025

opensearch 3.3.0 Homebrew/homebrew-core#249627

Merged

Luzifer mentioned this pull request Oct 15, 2025

[BUG] illegal_argument_exception error after upgrading to 3.3.0 #19646

Open

peteralfonsi pushed a commit to peteralfonsi/OpenSearch that referenced this pull request Oct 15, 2025

Enable skip_list for @timestamp field or index sort field (opensearch…

e15efd1

…-project#19480) Signed-off-by: Asim Mahmood <asim.seng@gmail.com>

harshavamsi mentioned this pull request Oct 16, 2025

[BUG] Upgrade from 2.19.2 to 3.3.0 failed #19660

Closed

asimmahmood1 added a commit to asimmahmood1/OpenSearch that referenced this pull request Oct 16, 2025

Revert "Enable skip_list for @timestamp field or index sort field (op…

35ef4da

…ensearch-project#19480)" Fixes opensearch-project#19660 This reverts commit 08ed9ee.

jainankitk mentioned this pull request Oct 17, 2025

Missing bwc test for indexing data for index created with older version #19682

Open

This was referenced Oct 22, 2025

[AUTOCUT] Gradle Check Flaky Test Report for FieldDataLoadingIT #19431

Closed

[AUTOCUT] Gradle Check Flaky Test Report for IngestFromKafkaIT #17215

Open

asimmahmood1 mentioned this pull request Oct 27, 2025

[META] Skiplist plan for 3.x #18882

Open

7 tasks

asimmahmood1 mentioned this pull request Dec 12, 2025

Add date_histogram_hourly_with_filter_agg operation to big5 opensearch-project/opensearch-benchmark-workloads#727

Merged

6 tasks

This was referenced Jan 10, 2026

docs: add Skip List report for v3.3.0 tkykenmt/opensearch-feature-explorer#1506

Merged

[feature] Skip List tkykenmt/opensearch-feature-explorer#1403

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enable skip_list for @timestamp field or index sort field by default#19480

Enable skip_list for @timestamp field or index sort field by default#19480
jainankitk merged 1 commit intoopensearch-project:mainfrom
asimmahmood1:skiplist_default

asimmahmood1 commented Sep 30, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Sep 30, 2025

Uh oh!

codecov bot commented Sep 30, 2025 •

edited

Loading

Uh oh!

jainankitk left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

asimmahmood1 commented Sep 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Related Issues

Check List

Uh oh!

github-actions bot commented Sep 30, 2025

Uh oh!

codecov bot commented Sep 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

jainankitk left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

asimmahmood1 commented Sep 30, 2025 •

edited

Loading

codecov bot commented Sep 30, 2025 •

edited

Loading