Skip to content

ref(integrations): Refactor platform detection to composable framework definitions#109700

Merged
jaydgoss merged 15 commits intomasterfrom
jaygoss/vdy-15-platform-detection-composable
Mar 12, 2026
Merged

ref(integrations): Refactor platform detection to composable framework definitions#109700
jaydgoss merged 15 commits intomasterfrom
jaygoss/vdy-15-platform-detection-composable

Conversation

@jaydgoss
Copy link
Member

@jaydgoss jaydgoss commented Mar 2, 2026

Summary

  • Refactor flat framework detection into composable FrameworkDef / DetectorRule system
  • Add three signal types: path (config file existence), match_content (regex on file content), match_package (dependency lookup in parsed manifests)
  • Add every (AND) and some (OR) rule composition
  • Add priority ranking (sort field) and supersession (e.g. Next.js supersedes React)
  • Refactor file content fetching into a single batch pass to minimize API calls

No behavior change for existing detections, but the architecture now supports easy addition of new frameworks as data-only entries.

Stack:

  • PR 1: Core detection + API endpoint
  • PR 2 (this): Composable framework definitions refactor
  • PR 3: Expanded coverage (98% of picker platforms)

@jaydgoss jaydgoss requested review from a team as code owners March 2, 2026 19:40
@linear
Copy link

linear bot commented Mar 2, 2026

@github-actions github-actions bot added Scope: Frontend Automatically applied to PRs that change frontend components Scope: Backend Automatically applied to PRs that change backend components labels Mar 2, 2026
@github-actions

This comment was marked as outdated.

@jaydgoss jaydgoss force-pushed the jaygoss/vdy-15-platform-detection-composable branch from 61cacd1 to 0815067 Compare March 2, 2026 19:44
@jaydgoss jaydgoss marked this pull request as draft March 2, 2026 20:14
@jaydgoss jaydgoss force-pushed the jaygoss/vdy-15-platform-detection-composable branch from 0815067 to efa46fe Compare March 2, 2026 23:11
@jaydgoss jaydgoss force-pushed the jaygoss/vdy-15-platform-detection-composable branch from 2ae4a7f to b30fee7 Compare March 10, 2026 17:04
@jaydgoss jaydgoss force-pushed the jaygoss/vdy-15-platform-detection-composable branch from b30fee7 to 5d8136e Compare March 10, 2026 17:15
Copy link
Contributor

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

@github-actions
Copy link
Contributor

Backend Test Failures

Failures on 8730df5 in this run:

tests/sentry/integrations/api/endpoints/test_organization_repository_platforms.py::OrganizationRepositoryPlatformsGetTest::test_detects_platformslog
tests/sentry/integrations/api/endpoints/test_organization_repository_platforms.py:70: in test_detects_platforms
    assert response.data == {
E   AssertionError: assert {'platforms':...cript', ...}]} == {'platforms':...ython', ...}]}
E     
E     Differing items:
E     {'platforms': [{'bytes': 50000, 'confidence': 'medium', 'language': 'Python', 'platform': 'python', ...}, {'bytes': 30000, 'confidence': 'medium', 'language': 'JavaScript', 'platform': 'javascript', ...}]} != {'platforms': [{'bytes': 50000, 'confidence': 'medium', 'language': 'Python', 'platform': 'python', ...}]}
E     
E     Full diff:
E       {
E           'platforms': [
E               {
E                   'bytes': 50000,
E                   'confidence': 'medium',
E                   'language': 'Python',
E                   'platform': 'python',
E                   'priority': 1,
E               },
E     +         {
E     +             'bytes': 30000,
E     +             'confidence': 'medium',
E     +             'language': 'JavaScript',
E     +             'platform': 'javascript',
E     +             'priority': 1,
E     +         },
E           ],
E       }

jaydgoss added a commit that referenced this pull request Mar 12, 2026
## Summary

- Add `get_languages()` method to GitHub API client to fetch repository
language statistics
- Create `platform_detection` module with language-to-platform mapping,
framework detection from manifest files (package.json, requirements.txt,
pyproject.toml, Pipfile, Gemfile, composer.json, build.gradle, pom.xml,
go.mod), and confidence scoring
- Add REST endpoint `GET
/api/0/organizations/{org}/repos/{repo_id}/platforms/` to expose
detected platforms
- 18 base language mappings, 23 framework detection rules, 24 ignored
languages

This is the foundation for automatic platform detection from GitHub
repositories to streamline onboarding. Part 1 of 3.

**Stack:**
- **PR 1 (this):** Core detection + API endpoint
- [PR 2](#109700): Composable
framework definitions refactor
- [PR 3](#109701): Expanded
coverage (~80 platforms)

## Test plan
- 44 unit tests covering language mapping, manifest parsing, framework
detection, supersession, and edge cases
- 7 integration tests covering endpoint success/error cases, IDOR
prevention, and auth requirements

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: getsantry[bot] <66042841+getsantry[bot]@users.noreply.github.com>
Base automatically changed from jaygoss/vdy-15-platform-detection-core to master March 12, 2026 16:06
jaydgoss and others added 15 commits March 12, 2026 11:09
…k definitions

Replace flat framework detection with a composable FrameworkDef /
DetectorRule system inspired by Vercel's framework detection. Each
framework is a self-contained definition with three signal types:
path (config file existence), match_content (regex on file content),
and match_package (dependency lookup in parsed manifests).

Add every (AND) and some (OR) rule composition, priority ranking via
sort field, and supersession so meta-frameworks like Next.js remove
redundant base frameworks like React from results. Batch file content
fetching into a single pass to minimize GitHub API calls.

Co-Authored-By: Claude <noreply@anthropic.com>
…types

Use `or {}` instead of `.get(key, {})` when accessing dependency fields
in package.json and composer.json parsing. When a field is explicitly
null (e.g., `"dependencies": null`), `.get()` returns None instead of
the default, causing an AttributeError on `.keys()`.

Also change FrameworkDef from `total=False` to use NotRequired for
optional fields only, so the type checker enforces that platform, sort,
and base_platform are always present.

Co-Authored-By: Claude <noreply@anthropic.com>
Catch KeyError, TypeError, and AttributeError in addition to ApiError
when parsing the GitHub Contents API response. Handles cases where items
are missing the "name" field or the response is not a list.

Co-Authored-By: Claude <noreply@anthropic.com>
Annotate inline dict literals with DetectorRule and FrameworkDef types
so mypy recognizes them as compatible TypedDict instances rather than
plain dict[str, str] or dict[str, object].

Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
Skip items missing "name" key instead of discarding all results.
Previously a single malformed item would cause KeyError, caught by
the blanket except, returning an empty set for the entire listing.

Co-Authored-By: Claude <noreply@anthropic.com>
…n tests

Add TestFrameworksIntegrity to catch structural errors in framework
definitions: duplicate platform IDs, invalid base_platforms, dangling
supersedes targets, missing rules, and match_content without path.

Add TestDetectPlatformsMultiStack to exercise the full pipeline against
a realistic repo with Python (Django + Celery), JavaScript (Next.js),
Go (Gin), TypeScript, and ignored languages — validating framework
detection, supersession, priority ordering, and language filtering all
work together.

Co-Authored-By: Claude <noreply@anthropic.com>
The detection pattern is a common technique, not derived from
Vercel's code. Remove references to avoid implying a stronger
connection than exists.

Co-Authored-By: Claude <noreply@anthropic.com>
- Fold None-manifest check into _package_in_manifest
- Simplify _framework_matches since all([]) returns True
- Rename detected_ids to platform_ids

Co-Authored-By: Claude <noreply@anthropic.com>
…amework tier

Sort by (bytes, priority) instead of (priority, bytes) so that the
dominant language in a repo ranks first. Previously a JS meta-framework
like Next.js would always outrank Django even in a Python-majority repo.
Now sort values only control ordering within a language group.

Co-Authored-By: Claude <noreply@anthropic.com>
Restore full response assertions that were lost during rebase conflict
resolution. Tests now verify the complete response dict including
platform, language, bytes, confidence, and priority fields.

Co-Authored-By: Claude <noreply@anthropic.com>
…ting fails

_get_root_file_names now returns None on API failure instead of an empty
set, so detect_platforms can distinguish "API failed" from "empty repo
root". When the root listing is unavailable, content-based and
package-based framework detection still works by fetching files
individually rather than silently degrading to base platforms only.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Scope: Backend Automatically applied to PRs that change backend components Scope: Frontend Automatically applied to PRs that change frontend components

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants