Skip to content

fix(tests): Use override_settings instead of direct settings mutation#109414

Merged
joshuarli merged 3 commits intomasterfrom
fix/flaky-metrics-settings-leak
Feb 26, 2026
Merged

fix(tests): Use override_settings instead of direct settings mutation#109414
joshuarli merged 3 commits intomasterfrom
fix/flaky-metrics-settings-leak

Conversation

@joshuarli
Copy link
Member

@joshuarli joshuarli commented Feb 25, 2026

Summary

  • Replace direct settings.X = value mutations with @override_settings across the test suite
  • Prevents settings state from leaking to subsequent tests

Files fixed

  • tests/sentry/sentry_metrics/test_batch.py (8 occurrences) — SENTRY_METRICS_INDEXER_DEBUG_LOG_SAMPLE_RATE
  • tests/sentry/sentry_metrics/test_gen_metrics_multiprocess_steps.py (1) — same
  • tests/sentry/sentry_metrics/test_rh_metrics_multiprocess_steps.py (1) — same
  • tests/sentry/web/frontend/test_js_sdk_loader.py (~15 occurrences) — JS_SDK_LOADER_DEFAULT_SDK_URL, JS_SDK_LOADER_SDK_VERSION; also replaced broken autouse fixture with @override_settings on the class
  • tests/acceptance/test_create_organization.py (2) — PRIVACY_URL, TERMS_URL
  • tests/acceptance/test_quick_start.py (2) — PRIVACY_URL, TERMS_URL
  • tests/sentry/loader/test_browsersdkversion.py (1) — JS_SDK_LOADER_SDK_VERSION

Why

Per PYTEST-GUIDELINES.md §4a: "Mutates settings globally, leaks to subsequent tests." Direct settings.X = value assignment bypasses Django's settings restoration mechanism, causing test pollution when test execution order changes.

Direct `settings.SENTRY_METRICS_INDEXER_DEBUG_LOG_SAMPLE_RATE = 1.0`
mutations leak state to subsequent tests. Use `@override_settings` to
ensure automatic cleanup after each test.
…browser SDK tests [skip ci]

Expand scope to cover all direct `settings.X = value` mutations across the
test suite. The JS SDK loader tests had ~15 direct mutations in test methods
plus an autouse fixture that set settings without cleanup.

- test_js_sdk_loader.py: Replace autouse fixture with @override_settings on
  class; replace all per-method settings mutations with @override_settings
- test_create_organization.py: Wrap PRIVACY_URL/TERMS_URL in @override_settings
- test_quick_start.py: Same
- test_browsersdkversion.py: Wrap JS_SDK_LOADER_SDK_VERSION in @override_settings
@joshuarli joshuarli changed the title fix(tests): Use override_settings for metrics indexer settings fix(tests): Use override_settings instead of direct settings mutation Feb 26, 2026
@joshuarli joshuarli marked this pull request as ready for review February 26, 2026 00:36
@joshuarli joshuarli requested a review from a team as a code owner February 26, 2026 00:36
@joshuarli joshuarli merged commit 09b7121 into master Feb 26, 2026
49 checks passed
@joshuarli joshuarli deleted the fix/flaky-metrics-settings-leak branch February 26, 2026 01:26
@github-actions github-actions bot locked and limited conversation to collaborators Mar 13, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants