Skip to content

Rebuild step on PolicyStepsRegistry.getStep#33780

Merged
dakrone merged 3 commits intoelastic:index-lifecyclefrom
dakrone:ilm-get-step-rebuild
Sep 18, 2018
Merged

Rebuild step on PolicyStepsRegistry.getStep#33780
dakrone merged 3 commits intoelastic:index-lifecyclefrom
dakrone:ilm-get-step-rebuild

Conversation

@dakrone
Copy link
Copy Markdown
Member

@dakrone dakrone commented Sep 17, 2018

This moves away from caching a list of steps for a current phase, instead
rebuilding the necessary step from the phase JSON stored in the index's
metadata.

Relates to #29823

This moves away from caching a list of steps for a current phase, instead
rebuilding the necessary step from the phase JSON stored in the index's
metadata.

Relates to elastic#29823
@dakrone dakrone added the :Data Management/ILM+SLM DO NOT USE. Use ":StorageEngine/ILM" or ":Distributed Coordination/SLM" instead. label Sep 17, 2018
}

static Step getCurrentStep(PolicyStepsRegistry stepRegistry, String policy, Index index, Settings indexSettings) {
static Step getCurrentStep(PolicyStepsRegistry stepRegistry, String policy, IndexMetaData indexMetaData, Settings indexSettings) {
Copy link
Copy Markdown
Contributor

@AthenaEryma AthenaEryma Sep 17, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is going to cause a (not very big, I think?) merge conflict with #33783. Not sure which side that should be handled on, but fair warning.

indexPhaseSteps.get(index).stream().filter(step -> step.getKey().equals(stepKey)).findFirst().orElse(null));

// parse phase steps from the phase definition in the index settings
final String phaseJson = indexMetaData.getSettings().get(LifecycleSettings.LIFECYCLE_PHASE_DEFINITION,
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will also need to be updated for #33783, depending on which gets merged first.

Copy link
Copy Markdown
Contributor

@bleskes bleskes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, as a first start. I think we should follow up with removing PolicyStepsRegistry completely. There is no reason for it to exist IMO. Let me know if I miss something.

default:
throw new IllegalArgumentException("invalid action [" + action + "]");
}};
for (String phase : phaseNames) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe we're not ready for this yet but don't we want a random subset of the phases and actions?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For now this made it easier to pick a random step for the entire policy because it guaranteed that I could pick a random phase that would have a random step. For this particular test it also tests (through randomization) that all phases and steps can be serialized/deserialized to/from JSON in the metadata

static Step getCurrentStep(PolicyStepsRegistry stepRegistry, String policy, IndexMetaData indexMetaData, Settings indexSettings) {
StepKey currentStepKey = getCurrentStepKey(indexSettings);
if (currentStepKey == null) {
return stepRegistry.getFirstStep(policy);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As far as I can tell, this only needed to onboard an index. Can we move this (as a follow up) to a dedicated place? on boarding is an exceptional case - we should put it in it's own method.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah as a followup definitely, we'd need to remove it if we want to remove PolicyStepsRegistry too

@dakrone dakrone merged commit 27dd258 into elastic:index-lifecycle Sep 18, 2018
dakrone added a commit that referenced this pull request Sep 19, 2018
This moves away from caching a list of steps for a current phase, instead
rebuilding the necessary step from the phase JSON stored in the index's
metadata.

Relates to #29823
@dakrone dakrone deleted the ilm-get-step-rebuild branch February 4, 2019 14:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

:Data Management/ILM+SLM DO NOT USE. Use ":StorageEngine/ILM" or ":Distributed Coordination/SLM" instead.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants