Skip to content

TSDB: disable _tsid in composite aggregation#81646

Closed
weizijun wants to merge 1 commit intoelastic:masterfrom
weizijun:disable-_tsid-in-composite-aggregation
Closed

TSDB: disable _tsid in composite aggregation#81646
weizijun wants to merge 1 commit intoelastic:masterfrom
weizijun:disable-_tsid-in-composite-aggregation

Conversation

@weizijun
Copy link
Copy Markdown
Contributor

in composite aggregation, group by _tsid will return the error:

{
  "error" : {
    "root_cause" : [
      {
        "type" : "unsupported_operation_exception",
        "reason" : null
      }
    ],
    "type" : "unsupported_operation_exception",
    "reason" : null,
    "suppressed" : [
      {
        "type" : "illegal_state_exception",
        "reason" : "Failed to close the XContentBuilder",
        "caused_by" : {
          "type" : "i_o_exception",
          "reason" : "Unclosed object or array found"
        }
      }
    ]
  },
  "status" : 500
}

reproduce:

PUT test
{
  "settings": {
    "index": {
      "time_series.start_time": 1,
      "time_series.end_time": 99999999999,
      "mode": "time_series",
      "routing_path": [
        "test"
      ]
    }
  },
  "mappings": {
    "properties": {
      "test": {
        "type": "keyword",
        "time_series_dimension": true
      }
    }
  }
}

POST test/_doc?refresh
{
  "test" : "test",
  "@timestamp" : 2
}

GET test/_search
{
  "aggs": {
    "1": {
      "composite": {
        "sources": [
          {
            "2": {
              "terms": {
                "field": "_tsid"
              }
            }
          }
        ]
      }
    }
  }
}

The reason why failed is that in InternalComposite.getKeyAsString, it will return a string for after key, but _tsid is a Map.

I think _tsid is not suitable for composite aggregation, so I add a check to failed composite aggregation with _tsid.

@elasticsearchmachine elasticsearchmachine added v8.1.0 external-contributor Pull request authored by a developer outside the Elasticsearch team labels Dec 13, 2021
@jtibshirani jtibshirani added the :StorageEngine/TSDB You know, for Metrics label Dec 13, 2021
@elasticmachine elasticmachine added the Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) label Dec 13, 2021
@elasticmachine
Copy link
Copy Markdown
Collaborator

Pinging @elastic/es-analytics-geo (Team:Analytics)

@nik9000
Copy link
Copy Markdown
Member

nik9000 commented Dec 14, 2021

I think we want _tsid to work in composite aggregation. It's the sort key of the index so it's the most efficient way to run composite. @imotov, do you agree?

@weizijun
Copy link
Copy Markdown
Contributor Author

I think we want _tsid to work in composite aggregation. It's the sort key of the index so it's the most efficient way to run composite. @imotov, do you agree?

is hard to use _tsid in composite aggregation, because of the after key need a string value. if we can implement the _tsid string format, it is ok to use in composite aggregation.

now, _tsid in composite aggregation will throw an unexpected exception.

@nik9000
Copy link
Copy Markdown
Member

nik9000 commented Dec 14, 2021

Yeah. I think @csoulios will try and fix it soon.

@csoulios
Copy link
Copy Markdown
Contributor

@weizijun PR #81998 should fix issues with composite aggregation

@weizijun
Copy link
Copy Markdown
Contributor Author

@weizijun PR #81998 should fix issues with composite aggregation

yeah, I see, thank you!

@weizijun
Copy link
Copy Markdown
Contributor Author

the exception fixed in #81998

@weizijun weizijun closed this Jan 23, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

external-contributor Pull request authored by a developer outside the Elasticsearch team :StorageEngine/TSDB You know, for Metrics Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) v8.1.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants