Make index.lifecycle.name setting internal

[dakrone]

This commit makes the index.lifecycle.name setting internal an index, this
means that the policy can only be set on the index creation, or with the
specialized RestSetIndexLifecyclePolicy action.

Relates to #29823

@colings86 I'm not 100% sure we should do as-is. You currently can't remove a policy from the index, unlike a setting (which you could set to null to remove it) we don't currently have a way to do that, in order to delete a policy then you have to create a dummy policy and update the index to use it. Maybe someone can explain more of the background for why we originally wanted this to be an internal setting?

I think we should either add an API to remove the lifecycle policy from an index, or keep this as a dynamic setting, what do you think?

[elasticmachine]

Pinging @elastic/es-core-infra

[talevy]

Maybe someone can explain more of the background for why we originally wanted this to be an internal setting?

We did not preserve the current version of the policy that was mid-execution, so it was not always safe (see Shrink) to switch policies without extra validation and guards against potential issues.

That being said, latest meetings around this led to the new design goal laid out in the meta issue:

Store phase JSON in index metadata when an index moves to a new phase and use that version to drive execution until the end of the phase regardless of changes to the underlying policy (#29823)

The above state preservation would void any validation and guards since each index would remember what it is currently executing, so switching the policy underneath would not be a problem.

Let me know if that made sense @dakrone. We may not need this API at all anymore

[dakrone]

I think that does make sense, in that case I'm extra-+1 on the solution where we preserved the state in the metadata so that we can completely remove this API.

@colings86 does that sound good? If so, I will work on removing the SetIndexLifecyclePolicy* stuff

[colings86]

You currently can't remove a policy from the index

I'm not sure I follow here because we have RemovePolicyForIndexAction to remove a policy from an index?

does that sound good? If so, I will work on removing the SetIndexLifecyclePolicy* stuff

I think we should wait until we have done the work to store the current phase in the index metadata before we remove this API so we can make sure its definitely not needed anymore, from both a technical and non-technical standpoint

[dakrone]

I'm not sure I follow here because we have RemovePolicyForIndexAction to remove a policy from an index?

Gah, I (erroneously) assumed that because all the tests were using the settings version that we didn't have an API for it.

I think we should wait until we have done the work to store the current phase in the index metadata before we remove this API so we can make sure its definitely not needed anymore, from both a technical and non-technical standpoint

Okay, I've changed the test to use the remove policy action, do we want to merge this for now and discuss removing it at a later time?

[colings86]

LGTM