Include metric types in tsdb index and template mappings#316
Conversation
This commit includes three changes: 1. extract the mappings which are common between the index and the data stream template 2. include the metric type for (almost) every field (skipped histogram fields like latencies and others) 3. fix some inconsistency around synthetic source settings
| } | ||
| }, | ||
| { | ||
| "operation": "date-histo-entire-range-1m", |
There was a problem hiding this comment.
I am wondering if it makes sense to have these three aggregations here or if we want to just have them under an "aggs" challenge for the tsdb track (not part of the downsampling challenge). Maybe we can keep them until we have an "aggs" challenge for the tsdb track?
nik9000
left a comment
nik9000
left a comment
There was a problem hiding this comment.
I think this looks good. We probably need the new rally operator first. But once we have it I'd be happy to get this run nightly as well.
|
It'll want a review from a perf person as well! |
tsdb/operations/default.json
Outdated
| { | ||
| "name": "date-histo-entire-range-1d", | ||
| "operation-type": "search", | ||
| "index": "tsdb-1h", |
There was a problem hiding this comment.
did you intend
| "index": "tsdb-1h", | |
| "index": "tsdb-1d", |
?
| "aggs": { | ||
| "date": { | ||
| "date_histogram": { | ||
| "field": "@timestamp", |
There was a problem hiding this comment.
Just a question, do we also want to benchmark the time_series agg here? (as separate operation)
There was a problem hiding this comment.
My plan is to have later a new "aggs" challenge in the tsdb track to benchmark aggregations. Those aggregations are there just to roughly evaluate the result of downsampling and I might remove them later once we have the new "aggs" challenge.
4 participants
This PR includes the following changes:
1. extract the mappings which are common between the index and the data stream template (see
index-mappings.json)2. include the metric type for (almost) every field (skipped histogram fields like latencies and others)
3. fix some inconsistency around synthetic source settings
4. include a new
downsamplingchallenge in thetsdbtrack (create an index, downsample it using 3 different intervals and run 3 date histogram aggregations on the downsampled indices).NOTE: the
downsamplechallenge uses a new operation (downsample) which is required to run the challenge (see Rally PR 1574).