[Dataset Quality] Create rules for degraded documents

[gbamparop]

📓 Summary

Users should be able to configure alerts for one or all their datasets to be informed when their quality is degrading based on _ignored fields. In the future, a new rule type will be created for degradation based on the failure store.

🎨 Design

TBD

✔️ Acceptance Criteria

TBD

❓ Open questions

• Is it acceptable for UX to use the custom threshold rules with filters for ignored fields?
• Since the percentage of degraded documents is calculated on the fly, if old documents are updated the charts in the alert details page will be affected. Is it worth considering an approach of having degraded metrics instead of relying on individual docs?

[elasticmachine]

Pinging @elastic/obs-ux-logs-team (Team:obs-ux-logs)

[weltenwort]

💭 There are probably some learnings to be transferred from the managed stack monitoring rules. I wasn't very involved in that, but I vaguely remember the space-awareness of the alerting rules to be a problem.

I also wonder how we'd be able to recognize the same rule already being present. And how we'd update it in case we need to ship fixes/improvements to its parameters.

[ruflin]

Is it acceptable for UX to use the custom threshold rules with filters for ignored fields?

I would think that is an easy way to get us started. What would be the alternative? If we have a dedicated rule type, that could potentially updates / fixing rules easier as @weltenwort brings up. We at least need to have a way to identify that it is a rule that is created by dataset quality.

failure store

Eventually yes, but I think for now it is ok if these would be 2 different rules.

if old documents are updated

How are documents updated in a data stream? Not saying we should not have metrics in addition but I would hope old docs do not change?

[flash1293]

I would think that is an easy way to get us started

I agree it would probably be the easiest way to get a first version out, but we should consider the following points:

• If we want to add a dedicated rule type later on, migration will become difficult
• We can set up the custom threshold rule correctly, but users will have a hard time configuring these alerts when not going through our UI - this would apply to the central alerting UI and maybe even more importantly to the API

I'm not sure how much harder it is to go with a dedicated rule type compared to parameterizing the custom threshold rule correctly, but if it's a marginal difference it feels the short-term / long-term tradeoff is worth it here. For example, we could register it as a new rule type, but use the custom threshold executor under the hood.

I also wonder how we'd be able to recognize the same rule already being present. And how we'd update it in case we need to ship fixes/improvements to its parameters.

This also seems easier with a custom rule type, as we can structure the parameteres in a way that makes it easy to find them.

About the failure store - IMHO this could be a separate rule type or an option on a dedicated rule type, the dedicated rule type also makes this addition easier in the future.

[flash1293]

About UI - the easiest integration would be a simple deeplink to create an alert like it's integrated in the APM app:

Thinking further there could be an indication if there is already a configured rule for degraded documents in a certain dataset like Felix mentioned, but it doesn't seem like an absolute blocker.

[gbamparop]

Updated the description and removed the question about the failure store.

• We can set up the custom threshold rule correctly, but users will have a hard time configuring these alerts when not going through our UI - this would apply to the central alerting UI and maybe even more importantly to the API

I'm not sure how much harder it is to go with a dedicated rule type compared to parameterizing the custom threshold rule correctly, but if it's a marginal difference it feels the short-term / long-term tradeoff is worth it here. For example, we could register it as a new rule type, but use the custom threshold executor under the hood.

I don't think there is much overhead to create a new rule type but it would be great to get input from @elastic/obs-ux-management-team. These are good points, the rules should be straightforward to add outside of the app and through APIs.

[maryam-saeidi]

Hi team,

I've created an RFC to consider different implementations of the DQ rule to see if reusing the custom threshold rule is a good idea. Please let me know if you have any comments. Thanks!