Webhooks should not be attached to the default admin provisioner

[tashian]

We should have an issue label called "annoying edge case foot gun" for this one.

In the default case where there's only one initial admin provisioner, If a webhook is added to that provisioner, and the webhook is misconfigured or becomes unavailable, the user will effectively become locked out of provisioner configuration. I wonder if the CLI should block new webhooks on the initial admin provisioner. My hunch is that people, like me, will be tempted to add a webhook to the default provisioner so they can test out the feature.

The question is whether we address this one in code, or in the docs?

[hslatman]

Didn't think of this during the triage meeting, but this sounds a lot like the way a user could lock themselves out when configuring a policy. In the policy management APIs the new policy is checked to not lock out the current admin, nor the other admins, before persisting and making it active. This is done by instantiating a temporary policy engine and doing a dry run, essentially. I think a similar mechanism could work for the webhook, although there's still the case that the behavior of the system handling the webhook changes after first configuration.

[tashian]

From Discord: