feat(code-review): Add backend logic for PR Review Dashboard [internal]#108533
feat(code-review): Add backend logic for PR Review Dashboard [internal]#108533
Conversation
Add the CodeReviewEvent model to track Seer PR code review lifecycle events. This is the data foundation for the PR Review Dashboard, split out as the first of three PRs.
|
🚨 Warning: This pull request contains Frontend and Backend changes! It's discouraged to make changes to Sentry's Frontend and Backend in a single pull request. The Frontend and Backend are not atomically deployed. If the changes are interdependent of each other, they must be separated into two pull requests and be made forward or backwards compatible, such that the Backend or Frontend can be safely deployed independently. Have questions? Please ask in the |
src/sentry/seer/code_review/api/endpoints/organization_code_review_events.py
Show resolved
Hide resolved
Add event recording, completion webhook handler, cleanup task, and API endpoints for the PR Review Dashboard. This enables tracking and querying Seer code review events.
59adf8f to
f972371
Compare
src/sentry/seer/code_review/api/endpoints/organization_code_review_stats.py
Show resolved
Hide resolved
src/sentry/seer/code_review/api/endpoints/organization_code_review_events.py
Show resolved
Hide resolved
There was a problem hiding this comment.
Why? This means that when an org moves between regions this data will be lost.
There was a problem hiding this comment.
This comment was related to the model change which landed in a PR this was based off of (#108531).
I haven't considered this and it's a good point, perhaps we need to update the model.
|
|
||
| events = CodeReviewEvent.objects.filter( | ||
| organization_id=organization.id, | ||
| repository_id=repo_id_int, |
There was a problem hiding this comment.
Are there any access rules for repositories where some users can't see specific repositories?
There was a problem hiding this comment.
well, we don't have such access rules in Sentry AFAIK - as soon as someone has org:read, they can access all repos. Whether they should be able to access PR titles, author names, etc, I don't really know.
Does this need to get someone else involved or is it OK to proceed as is for internal use and follow up if we ever make this available to customers?
src/sentry/seer/code_review/api/endpoints/organization_code_review_event_details.py
Outdated
Show resolved
Hide resolved
src/sentry/seer/code_review/api/endpoints/organization_code_review_event_details.py
Outdated
Show resolved
Hide resolved
| on_results=lambda groups: self._enrich_groups( | ||
| groups, queryset, organization.id, pr_state | ||
| ), |
There was a problem hiding this comment.
Why not use a serializer to do this serialization logic instead of a method on the endpoint?
| ) | ||
| total_authors = author_prs.count() | ||
| top_authors = [ | ||
| {"author": entry["pr_author"], "prCount": entry["pr_count"]} for entry in author_prs[:3] |
There was a problem hiding this comment.
because that's how many fits in the UI but good point, should be an arg
armenzg
left a comment
There was a problem hiding this comment.
There's a bunch of comments from Mark and the bot. Please re-request once addressed.
src/sentry/seer/code_review/api/endpoints/organization_code_review_events.py
Outdated
Show resolved
Hide resolved
src/sentry/seer/code_review/api/endpoints/organization_code_review_stats.py
Outdated
Show resolved
Hide resolved
Backend Test FailuresFailures on
|
- Guard against None integration in webhook handler (mypy + runtime fix) - Restore CLOSED action filter when no triggers configured (lost in merge) - Restore SENT_TO_SEER status update on task success (lost in merge) - Remove unused SeerViewerContext import (stale from merge) - Fix task namespace: use seer_code_review_tasks in on_completion - Handle InvalidQuery from parse_datetime_string in both dashboard endpoints
src/sentry/seer/code_review/api/endpoints/organization_code_review_events.py
Show resolved
Hide resolved
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
Autofix Details
Bugbot Autofix prepared a fix for the issue found in the latest run.
- ✅ Fixed: check_run handler passes extra kwargs not in task signature
- Removed unused action and html_url keyword arguments that were being passed to process_github_webhook_event.delay() but landing in **kwargs and never used.
Or push these changes by commenting:
@cursor push 3bb3d7d2d6
Preview (3bb3d7d2d6)
diff --git a/src/sentry/seer/code_review/webhooks/check_run.py b/src/sentry/seer/code_review/webhooks/check_run.py
--- a/src/sentry/seer/code_review/webhooks/check_run.py
+++ b/src/sentry/seer/code_review/webhooks/check_run.py
@@ -144,8 +144,6 @@
github_event=github_event.value,
# A reduced payload is enough for the task to process.
event_payload={"original_run_id": validated_event.check_run.external_id},
- action=validated_event.action,
- html_url=validated_event.check_run.html_url,
enqueued_at_str=datetime.now(timezone.utc).isoformat(),
trigger_id=github_delivery_id,
organization_id=organization.id if organization else None,| enqueued_at_str=datetime.now(timezone.utc).isoformat(), | ||
| trigger_id=github_delivery_id, | ||
| organization_id=organization.id if organization else None, | ||
| repository_id=repo.id if repo else None, |
There was a problem hiding this comment.
check_run handler passes extra kwargs not in task signature
Low Severity
process_github_webhook_event.delay() is called with action and html_url keyword arguments that are not declared parameters of the task function. They silently land in **kwargs and are never used inside the task. Meanwhile, trigger_id, organization_id, and repository_id are passed correctly but the task only uses them for event record lookup — the action and html_url are lost and not forwarded to Seer in the request payload, unlike what the old code may have done.
There was a problem hiding this comment.
The action and html_url kwargs are absorbed by **kwargs in the task signature. They're harmless but unused — leaving as-is since this is low severity and **kwargs is intentional for forward compatibility.
— Claude Code
| if end_str: | ||
| queryset = queryset.filter(trigger_at__lte=parse_datetime_string(end_str)) | ||
| except InvalidQuery as e: | ||
| return Response({"detail": str(e)}, status=400) |
Check warning
Code scanning / CodeQL
Information exposure through an exception Medium
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 24 days ago
In general, the fix is to avoid returning the raw exception message to the client. Instead, return a stable, generic error message that doesn’t leak internal details, and, if needed, log the exception on the server side for debugging. This preserves behavior in terms of HTTP status and overall control flow, while removing the exposure of exception internals.
Concretely, in OrganizationCodeReviewPRsEndpoint.get, within the try/except InvalidQuery as e: block around the parse_datetime_string calls, change the Response construction in the except to stop using str(e). Replace it with a generic message such as "Invalid date format for 'start' or 'end' parameter." (or a similar wording that doesn’t expose stack traces or internal parsing details). If the surrounding file already uses logging for such errors (not visible in the snippet), it can be used here; since we can’t see that and must minimize assumptions, we’ll just change the response body and keep the 400 status code unchanged.
Necessary edits:
- In
src/sentry/seer/code_review/api/endpoints/organization_code_review_events.py, lines 51–52: modify theexcept InvalidQuery as e:handler to ignoreein the response and return a generic error message while preservingstatus=400. - No additional imports or helper methods are strictly required for this minimal fix.
| @@ -48,8 +48,11 @@ | ||
| end_str = request.GET.get("end") | ||
| if end_str: | ||
| queryset = queryset.filter(trigger_at__lte=parse_datetime_string(end_str)) | ||
| except InvalidQuery as e: | ||
| return Response({"detail": str(e)}, status=400) | ||
| except InvalidQuery: | ||
| return Response( | ||
| {"detail": "Invalid date format for 'start' or 'end' parameter."}, | ||
| status=400, | ||
| ) | ||
|
|
||
| pr_groups = ( | ||
| queryset.filter(pr_number__isnull=False) |
There was a problem hiding this comment.
False positive — InvalidQuery from parse_datetime_string contains user-facing validation messages (e.g. "Invalid date format"), not stack traces. This is standard input validation error handling, returning a 400 with a descriptive error.
— Claude Code
| if end_str: | ||
| queryset = queryset.filter(trigger_at__lte=parse_datetime_string(end_str)) | ||
| except InvalidQuery as e: | ||
| return Response({"detail": str(e)}, status=400) |
Check warning
Code scanning / CodeQL
Information exposure through an exception Medium
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 24 days ago
In general, to fix this type of issue you should avoid sending raw exception messages to clients. Instead, log the exception on the server (including stack trace) and return a generic, non-sensitive error message in the HTTP response. This preserves debuggability while preventing accidental leakage of internal implementation details.
For this specific endpoint, we should modify the except InvalidQuery as e: block so that it no longer returns str(e) in the response body. Instead, we can log the exception using Sentry’s logging/capture facilities (e.g., logging or sentry_sdk.capture_exception) and return a generic message such as "Invalid query parameters" or "Invalid date range parameters". To keep behavior similar from the client’s perspective (still a 400 for bad input), we only change the error payload, not the status code or control flow.
Concretely:
- Add an import for Python’s standard
loggingmodule at the top oforganization_code_review_stats.py. - Define a module-level logger via
logger = logging.getLogger(__name__). - In the
except InvalidQuery as e:block, calllogger.exception(...)to log the error and then return a DRFResponsewith a generic"detail"message instead ofstr(e).
All edits are confined to src/sentry/seer/code_review/api/endpoints/organization_code_review_stats.py within the shown snippet.
| @@ -1,5 +1,7 @@ | ||
| from __future__ import annotations | ||
|
|
||
| import logging | ||
|
|
||
| from django.db.models import CharField, Count, Min, Q, Sum, Value | ||
| from django.db.models.functions import Cast, Coalesce, Concat, TruncDay, TruncHour | ||
| from rest_framework.exceptions import ParseError | ||
| @@ -16,6 +18,8 @@ | ||
| from sentry.models.repository import Repository | ||
| from sentry.search.utils import InvalidQuery, parse_datetime_string | ||
|
|
||
| logger = logging.getLogger(__name__) | ||
|
|
||
| SKIPPED_STATUSES = Q( | ||
| status__in=[CodeReviewEventStatus.PREFLIGHT_DENIED, CodeReviewEventStatus.WEBHOOK_FILTERED] | ||
| ) | ||
| @@ -58,7 +62,11 @@ | ||
| if end_str: | ||
| queryset = queryset.filter(trigger_at__lte=parse_datetime_string(end_str)) | ||
| except InvalidQuery as e: | ||
| return Response({"detail": str(e)}, status=400) | ||
| logger.exception("Invalid query parameters for organization code review stats: %s", e) | ||
| return Response( | ||
| {"detail": "Invalid query parameters."}, | ||
| status=400, | ||
| ) | ||
|
|
||
| # Event-level counts | ||
| review_events = queryset.filter(REVIEWED_STATUSES) |
There was a problem hiding this comment.
Same as above — InvalidQuery messages are user-facing validation errors, not stack traces. This is safe.
— Claude Code
The prState filter was applied in on_results after OffsetPaginator had already sliced the queryset, breaking pagination counts and potentially returning fewer items than per_page. Use a Subquery to annotate the latest event's pr_state on each PR group so filtering happens before pagination.
| if tags and (org_id := tags.get("sentry_organization_id")): | ||
| viewer_context = SeerViewerContext(organization_id=int(org_id)) | ||
| make_seer_request(path=path, payload=event_payload, viewer_context=viewer_context) | ||
| make_seer_request(path=path, payload=event_payload) |
There was a problem hiding this comment.
Viewer context removed from signed Seer API requests
Medium Severity
The viewer_context parameter (carrying organization_id) was removed from the make_seer_request call. Previously the task constructed a SeerViewerContext from the organization ID in tags and passed it through for signed API request authorization. Now organization_id is available as a direct parameter but isn't used to build the viewer context. This changes the authentication/authorization behavior of outbound Seer requests.
There was a problem hiding this comment.
Fixed — restored SeerViewerContext construction using the organization_id parameter that the task now receives directly (instead of extracting from tags as before).
— Claude Code
The merge dropped the SeerViewerContext construction that master had added. Restore it using the organization_id parameter that the branch passes directly to the task.
| def create_code_review_event(self, organization=None, repository=None, **kwargs): | ||
| if organization is None: | ||
| organization = self.organization | ||
| return Factories.create_code_review_event(organization, repository, **kwargs) |
There was a problem hiding this comment.
Missing default for repository in test fixture
Low Severity
create_code_review_event provides a default for organization (falls back to self.organization) but not for repository. If called without repository, None is passed to Factories.create_code_review_event, which crashes with an AttributeError when accessing repository.id. This is inconsistent with how organization is handled in the same method.
There was a problem hiding this comment.
Low severity — the fixture is only used in tests where a repository is always provided. Skipping for now.
— Claude Code
Unknown status values from Seer were silently mapped to REVIEW_COMPLETED, which could mask failures. Now unknown statuses are logged and the status field is left unchanged.
|
|
||
| pr_state = request.GET.get("prState") | ||
| if pr_state: | ||
| queryset = queryset.filter(pr_state=pr_state) |
There was a problem hiding this comment.
Inconsistent prState filtering between stats and events endpoints
Medium Severity
The stats endpoint filters events directly with queryset.filter(pr_state=pr_state), matching any event that had that state at the time. The events endpoint uses a subquery to filter by the latest event's pr_state. A PR that transitioned from "open" to "merged" will appear partially in "open" stats but not in the "open" PR list, making the two endpoints inconsistent when the same prState filter is applied.
Additional Locations (1)
There was a problem hiding this comment.
This is intentional: the stats endpoint counts events (filtering by state at event time), while the events endpoint groups by PR (filtering by current/latest state). Different semantics for different use cases.
— Claude Code
| def create_code_review_event(self, organization=None, repository=None, **kwargs): | ||
| if organization is None: | ||
| organization = self.organization | ||
| return Factories.create_code_review_event(organization, repository, **kwargs) |
There was a problem hiding this comment.
Test fixture missing default for repository parameter
Low Severity
create_code_review_event defaults organization to self.organization when None, but repository has no such fallback. Calling this method without a repository argument passes None to Factories.create_code_review_event, which will crash with AttributeError on repository.id.
There was a problem hiding this comment.
Duplicate of earlier finding — the fixture is only used in tests that always provide a repository. Low severity, skipping.
— Claude Code
The check_run handler was not including trigger_id in the event_payload dict sent to Seer, which means Seer couldn't echo it back in the completion webhook to match event records.
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 2 potential issues.
Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
| pr_title = issue.get("title") | ||
| pr_author = issue.get("user", {}).get("login") | ||
| pr_url = issue.get("pull_request", {}).get("html_url") | ||
| pr_state = issue.get("state") |
There was a problem hiding this comment.
Issue comment PR state misses merged detection
Medium Severity
For issue_comment events, pr_state is set to issue.get("state") directly, which yields "closed" for merged PRs. But for pull_request events, _extract_pr_state is used, which correctly returns "merged". The same PR can thus have inconsistent pr_state values across its events, causing incorrect results when filtering by prState in the dashboard endpoints.
There was a problem hiding this comment.
Fixed — now checks issue.pull_request.merged_at to correctly distinguish merged PRs from closed ones in issue_comment events.
— Claude Code
| def create_code_review_event(self, organization=None, repository=None, **kwargs): | ||
| if organization is None: | ||
| organization = self.organization | ||
| return Factories.create_code_review_event(organization, repository, **kwargs) |
There was a problem hiding this comment.
Fixture missing default for repository parameter
Low Severity
create_code_review_event defaults repository to None but never assigns a fallback value (unlike organization which falls back to self.organization). Calling this without repository will cause an AttributeError on repository.id inside the factory method.
There was a problem hiding this comment.
Low severity — the fixture is only used in tests that always provide a repository. Skipping.
— Claude Code
GitHub issue objects report "closed" for merged PRs. Use issue.pull_request.merged_at to correctly distinguish "merged" from "closed", consistent with how pull_request events handle it.
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
| def create_code_review_event(self, organization=None, repository=None, **kwargs): | ||
| if organization is None: | ||
| organization = self.organization | ||
| return Factories.create_code_review_event(organization, repository, **kwargs) |
There was a problem hiding this comment.
Missing default repository in test fixture causes crash
Low Severity
The create_code_review_event fixture defaults organization to self.organization when None, but does not default repository. Calling it without a repository argument passes None to Factories.create_code_review_event, which accesses repository.id, raising an AttributeError.
Additional Locations (1)
There was a problem hiding this comment.
Duplicate of earlier findings — the fixture is only used in tests that always provide a repository. Low severity, skipping.
— Claude Code
|
This issue has gone three weeks without activity. In another week, I will close it. But! If you comment or otherwise update it, I will reset the clock, and if you remove the label "A weed is but an unloved flower." ― Ella Wheeler Wilcox 🥀 |



Summary
Note
This is currently an internal/experimental tool, not meant to be released to customers as is. Take it as a conversation starter and source of internal UX feedback.
event_recorder.py) to persistCodeReviewEventrows from webhook triggerson_completion.py) to update events with review resultstrigger_idandrepository_idthrough to Seer requests for correlationpr-review-dashboardfeature flagPR 2 of 3 for the PR Review Dashboard feature. Based on PR 1 (#108531).
Test plan