Skip to content

Refactor query shape field data maps#111

Merged
jainankitk merged 1 commit intoopensearch-project:mainfrom
dzane17:field-name-simplify
Sep 19, 2024
Merged

Refactor query shape field data maps#111
jainankitk merged 1 commit intoopensearch-project:mainfrom
dzane17:field-name-simplify

Conversation

@dzane17
Copy link
Copy Markdown
Member

@dzane17 dzane17 commented Sep 9, 2024

Description

Refactor query shape field data maps

With the introduction of WithFieldName interface in OpenSearch core (see opensearch-project/OpenSearch#15705), we can now simplify the field data logic in Query-Insights.

Issues Resolved

Resolves #109
Related RFC #69

Testing

Large Search

curl -XGET 'http://localhost:9200/_search?pretty' -H 'Content-Type: application/json' -d '{
  "size": 0,
  "query": {
    "bool": {
      "must": [
        {
          "range": {
            "order_date": {
              "gte": "2023-01-01",
              "lte": "2023-12-31"
            }
          }
        },
        {
          "term": {
            "status": "completed"
          }
        }
      ],
      "filter": [
        {
          "terms": {
            "category": ["electronics", "clothing"]
          }
        }
      ]
    }
  },
  "aggs": {
    "sales_per_month": {
      "date_histogram": {
        "field": "order_date",
        "calendar_interval": "month",
        "format": "yyyy-MM",
        "order": {
          "_key": "desc"
        }
      },
      "aggs": {
        "total_sales": {
          "sum": {
            "field": "total_amount"
          }
        },
        "avg_order_quantity": {
          "avg": {
            "field": "order_quantity"
          }
        },
        "max_order_amount": {
          "max": {
            "field": "total_amount"
          }
        }
      }
    },
    "avg_monthly_sales": {
      "avg_bucket": {
        "buckets_path": "sales_per_month>total_sales"
      }
    },
    "avg_max_order_amount": {
      "avg_bucket": {
        "buckets_path": "sales_per_month>max_order_amount"
      }
    }
  },
  "sort": [
    { "sales_per_month>key": { "order": "desc" } },
    { "total_sales": { "order": "desc" } },
    { "avg_order_quantity": { "order": "asc" } }
  ]
}'
[2024-09-09T13:15:16,352][INFO ][stdout                   ] [integTest-0] bool []
[2024-09-09T13:15:16,354][INFO ][stdout                   ] [integTest-0]   must:
[2024-09-09T13:15:16,354][INFO ][stdout                   ] [integTest-0]     range [order_date]
[2024-09-09T13:15:16,355][INFO ][stdout                   ] [integTest-0]     term [status]
[2024-09-09T13:15:16,355][INFO ][stdout                   ] [integTest-0]   filter:
[2024-09-09T13:15:16,355][INFO ][stdout                   ] [integTest-0]     terms [category]
[2024-09-09T13:15:16,355][INFO ][stdout                   ] [integTest-0] aggregation:
[2024-09-09T13:15:16,356][INFO ][stdout                   ] [integTest-0]   date_histogram [order_date]
[2024-09-09T13:15:16,356][INFO ][stdout                   ] [integTest-0]     aggregation:
[2024-09-09T13:15:16,356][INFO ][stdout                   ] [integTest-0]       avg [order_quantity]
[2024-09-09T13:15:16,357][INFO ][stdout                   ] [integTest-0]       max [total_amount]
[2024-09-09T13:15:16,357][INFO ][stdout                   ] [integTest-0]       sum [total_amount]
[2024-09-09T13:15:16,357][INFO ][stdout                   ] [integTest-0]   pipeline aggregation:
[2024-09-09T13:15:16,357][INFO ][stdout                   ] [integTest-0]     avg_bucket
[2024-09-09T13:15:16,358][INFO ][stdout                   ] [integTest-0]     avg_bucket
[2024-09-09T13:15:16,358][INFO ][stdout                   ] [integTest-0] sort:
[2024-09-09T13:15:16,358][INFO ][stdout                   ] [integTest-0]   desc [sales_per_month>key]
[2024-09-09T13:15:16,358][INFO ][stdout                   ] [integTest-0]   desc [total_sales]
[2024-09-09T13:15:16,358][INFO ][stdout                   ] [integTest-0]   asc [avg_order_quantity]

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@dzane17 dzane17 force-pushed the field-name-simplify branch 2 times, most recently from 715e373 to dc1e87a Compare September 9, 2024 21:12
@dzane17 dzane17 force-pushed the field-name-simplify branch 3 times, most recently from 77e08c9 to 5c55429 Compare September 17, 2024 00:33
@dzane17
Copy link
Copy Markdown
Member Author

dzane17 commented Sep 17, 2024

Unrelated build failure fix: #117

@dzane17 dzane17 force-pushed the field-name-simplify branch 2 times, most recently from 030b5ef to 00c8f18 Compare September 18, 2024 17:49
@dzane17
Copy link
Copy Markdown
Member Author

dzane17 commented Sep 18, 2024

@jainankitk @deshsidd @ansjcy Please take a look again. Since all relevant builders now implement the WithFieldName interface, I combined all three query, aggregation, sort maps into one map.

Signed-off-by: David Zane <davizane@amazon.com>
@dzane17 dzane17 force-pushed the field-name-simplify branch from 00c8f18 to 8539426 Compare September 19, 2024 23:44
@jainankitk jainankitk merged commit f1827d8 into opensearch-project:main Sep 19, 2024
opensearch-trigger-bot bot pushed a commit that referenced this pull request Sep 19, 2024
…Name interface (#111)

Signed-off-by: David Zane <davizane@amazon.com>
(cherry picked from commit f1827d8)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
@dzane17 dzane17 deleted the field-name-simplify branch September 20, 2024 00:12
dzane17 pushed a commit that referenced this pull request Sep 20, 2024
…Name interface (#111) (#125)

(cherry picked from commit f1827d8)

Signed-off-by: David Zane <davizane@amazon.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Remove unnecessary map of QueryBuilders to fieldName function

4 participants