Skip to content

ref(grouping): Improve message parameterization metrics#110612

Merged
lobsterkatie merged 5 commits intomasterfrom
kmclb-improve-parameterization-metrics
Mar 13, 2026
Merged

ref(grouping): Improve message parameterization metrics#110612
lobsterkatie merged 5 commits intomasterfrom
kmclb-improve-parameterization-metrics

Conversation

@lobsterkatie
Copy link
Member

@lobsterkatie lobsterkatie commented Mar 13, 2026

This makes a number of adjustments to our parameterization metrics, both to improve them and to reflect recent changes to the code.

  • Right now, we have a message_parameterized metric, which only fires when the parameterized message doesn't match the original message, not every time we run parameterization. We therefore can't answer the question of how often parameterization has any effect. To answer that question, this adds a new metric inside the parameterize function, parameterizer_called, which fires any time the function runs, and which includes a boolean changed tag.

  • Now that we cache parameterization results, firing the message_parameterized metric any time normalize_message_for_grouping is called isn't actually accurate anymore. This therefore replaces it with a new message_used metric, so that we can continue to track our various reasons for using messages.

  • Caching parameterization results doesn't actually make things any better if we only use the cached value once. To better track how effective caching is, this adds to the context usage flags for both the parameterization result and the parameterizer itself (which is also cached, to save us having to check which one to use over and over). These are then checked (and flipped to True if they weren't already) each time we use one, and new metrics (cached_param_result_used and cached_parameterizer_used, respectively) are fired any time the flag indicates multiple usages.

@github-actions github-actions bot added the Scope: Backend Automatically applied to PRs that change backend components label Mar 13, 2026
@github-actions

This comment was marked as outdated.

@lobsterkatie lobsterkatie force-pushed the kmclb-cache-parameterized-version-of-message-variable branch from 7a47d5b to bec3b75 Compare March 13, 2026 17:40
@lobsterkatie lobsterkatie force-pushed the kmclb-improve-parameterization-metrics branch from 8c97176 to e11bcd0 Compare March 13, 2026 17:40
@lobsterkatie lobsterkatie marked this pull request as ready for review March 13, 2026 18:20
@lobsterkatie lobsterkatie requested a review from a team as a code owner March 13, 2026 18:20
Copy link
Member

@shashjar shashjar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

left a couple of suggestions

Base automatically changed from kmclb-cache-parameterized-version-of-message-variable to master March 13, 2026 18:53
@lobsterkatie lobsterkatie force-pushed the kmclb-improve-parameterization-metrics branch from e11bcd0 to 00752a1 Compare March 13, 2026 19:43
@lobsterkatie lobsterkatie force-pushed the kmclb-improve-parameterization-metrics branch from a16ea80 to 4efa932 Compare March 13, 2026 20:06
@lobsterkatie lobsterkatie merged commit 56f9f83 into master Mar 13, 2026
77 checks passed
@lobsterkatie lobsterkatie deleted the kmclb-improve-parameterization-metrics branch March 13, 2026 20:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Scope: Backend Automatically applied to PRs that change backend components

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants