Skip to content

Include a new downsample challenge to the tsdb track#319

Merged
salvatore-campagna merged 10 commits intoelastic:masterfrom
salvatore-campagna:feature/tsdb-downsample-challenge
Sep 29, 2022
Merged

Include a new downsample challenge to the tsdb track#319
salvatore-campagna merged 10 commits intoelastic:masterfrom
salvatore-campagna:feature/tsdb-downsample-challenge

Conversation

@salvatore-campagna
Copy link
Copy Markdown
Contributor

No description provided.

The downsample operation requires that the new indices it creates
by aggregating data in the source index not to exist. For this
reason we need to delete all of the indices, the source and the
target.
@salvatore-campagna
Copy link
Copy Markdown
Contributor Author

The CI failures here require changes in Rally that are part of PR 1574 in Rally repo.

Comment on lines +57 to +58
"operation-type": "delete-index",
"only-if-exists": false
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This shouldn't be needed anymore as of elastic/rally#1580

Suggested change
"operation-type": "delete-index",
"only-if-exists": false
"operation-type": "delete-index"

{
"operation": {
"operation-type": "delete-index",
"only-if-exists": false,
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"only-if-exists": false,

The default value for 'only-if-exists' is false. As a result,
we don't need to make the value explicit in the delete index
operation.
Include a new downsampling operation using a 'fixed_interval' of
1 second and and operation to perform a date histogram aggregation
on the resulting target index.
@salvatore-campagna
Copy link
Copy Markdown
Contributor Author

I will remove the source-index here so that we use the default index value as the source for all downsampling operations.

@salvatore-campagna
Copy link
Copy Markdown
Contributor Author

The CI fails with
[ERROR] Cannot race. Error in load generator [0] 14:19:34 No runner available for operation-type: [downsample]

As a result this will be fixed once the other PR is merged (the PR adding the downsample operation).

Copy link
Copy Markdown
Contributor

@DJRickyB DJRickyB left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just comments, since this PR is blocked on elastic/rally#1574 for now

@salvatore-campagna
Copy link
Copy Markdown
Contributor Author

I will re-run the failing CI test once the other PR (adding support for the downsample operation to Rally) is merged.

@salvatore-campagna
Copy link
Copy Markdown
Contributor Author

@elasticmachine run rally-tracks/it

@DJRickyB DJRickyB added the backport pending Awaiting backport to stable release branch label Sep 15, 2022
@salvatore-campagna
Copy link
Copy Markdown
Contributor Author

@elasticmachine run rally-tracks/it

@salvatore-campagna
Copy link
Copy Markdown
Contributor Author

@DJRickyB I executed the test after merging the other PR and everything looks ok.

Copy link
Copy Markdown
Contributor

@DJRickyB DJRickyB left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thank you

@salvatore-campagna salvatore-campagna merged commit 7a14c10 into elastic:master Sep 29, 2022
@salvatore-campagna salvatore-campagna self-assigned this Sep 29, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport pending Awaiting backport to stable release branch

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants