Skip to content

[Infra] Remove flaky proxy_e2e_azure_batches_tests CI workflow#25247

Merged
ishaan-berri merged 1 commit intomainfrom
litellm_remove_flaky_azure_batches_e2e
Apr 6, 2026
Merged

[Infra] Remove flaky proxy_e2e_azure_batches_tests CI workflow#25247
ishaan-berri merged 1 commit intomainfrom
litellm_remove_flaky_azure_batches_e2e

Conversation

@yuneng-berri
Copy link
Copy Markdown
Collaborator

Summary

Problem

The proxy_e2e_azure_batches_tests workflow is consistently flaky — it fails intermittently regardless of code changes, providing no reliable signal on whether a PR actually breaks anything. This leads to wasted CI minutes, unnecessary re-runs, and developer friction when legitimate PRs are blocked by unrelated test failures.

Fix

Remove the proxy_e2e_azure_batches_tests workflow entirely from both CircleCI and GitHub Actions, along with the test directory it exclusively used.

Changes

  • Removed the proxy_e2e_azure_batches_tests job definition from .circleci/config.yml
  • Removed its workflow trigger entry from the CircleCI workflows section
  • Removed .github/workflows/test-proxy-e2e-azure-batches.yml
  • Removed tests/proxy_e2e_azure_batches_tests/ directory (all test files, fixtures, mock servers)

No other tests or workflows referenced this directory, so no other CI jobs are affected.

Type

🚄 Infrastructure

The proxy_e2e_azure_batches_tests workflow is consistently flaky and
does not provide reliable signal on whether changes break anything.
Remove the workflow from both CircleCI and GitHub Actions, along with
the test directory it exclusively used.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@yuneng-berri yuneng-berri requested a review from a team April 6, 2026 22:04
@vercel
Copy link
Copy Markdown

vercel bot commented Apr 6, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
litellm Ready Ready Preview, Comment Apr 6, 2026 10:05pm

Request Review

@CLAassistant
Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps bot commented Apr 6, 2026

Greptile Summary

This PR removes the consistently flaky proxy_e2e_azure_batches_tests CI workflow and its associated test directory, which were causing wasted CI minutes and developer friction without providing a reliable signal.

  • Removes the proxy_e2e_azure_batches_tests job definition and its workflow trigger entry from .circleci/config.yml
  • Deletes .github/workflows/test-proxy-e2e-azure-batches.yml (the GitHub Actions counterpart)
  • Removes the entire tests/proxy_e2e_azure_batches_tests/ directory (~3,600 lines: base integration tests, conftest, mock Azure batch server, fixtures, E2E tests, and validation utilities)
  • A search across the codebase confirms zero remaining references to the deleted directory or workflow — no orphaned imports or CI references remain

Confidence Score: 5/5

Safe to merge — pure deletion of a flaky CI workflow and its test directory with no impact on production code.

This is a straightforward infrastructure cleanup consisting entirely of deletions. No production or library code is touched, no other CI jobs reference the removed workflow or test directory (verified by codebase-wide search), and the CircleCI config correctly retains all other workflow triggers. There are no P0 or P1 findings.

No files require special attention.

Important Files Changed

Filename Overview
.circleci/config.yml Removes the proxy_e2e_azure_batches_tests job definition (~114 lines) and its workflow trigger; all other CircleCI jobs are untouched
.github/workflows/test-proxy-e2e-azure-batches.yml Entire GitHub Actions workflow file deleted; was the GHA counterpart running the same flaky E2E suite
tests/proxy_e2e_azure_batches_tests/test_proxy_e2e_azure_batches.py Deleted primary E2E test file (~324 lines) exclusively used by the removed CI workflow
tests/proxy_e2e_azure_batches_tests/test_managed_files_base.py Largest deleted file (~1,085 lines) containing base-class tests for managed file operations in the Azure batch suite
tests/proxy_e2e_azure_batches_tests/fixtures/mock_azure_batch_server/mock_azure_batch.py Deleted mock server implementation (~517 lines) simulating Azure Batch API responses; no longer needed

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[PR merged to main] --> B{CI Triggers}
    B --> C[CircleCI Workflows]
    B --> D[GitHub Actions Workflows]
    C --> E[Remaining CircleCI jobs\ne.g. llm_translation_testing, upload-coverage...]
    C --> F[proxy_e2e_azure_batches_tests\n❌ REMOVED]
    D --> G[Remaining GHA workflows]
    D --> H[test-proxy-e2e-azure-batches.yml\n❌ REMOVED]
    style F fill:#ffcccc,stroke:#cc0000,color:#000000
    style H fill:#ffcccc,stroke:#cc0000,color:#000000
Loading

Reviews (1): Last reviewed commit: "Remove flaky proxy_e2e_azure_batches_tes..." | Re-trigger Greptile

@codspeed-hq
Copy link
Copy Markdown
Contributor

codspeed-hq bot commented Apr 6, 2026

Merging this PR will not alter performance

✅ 16 untouched benchmarks


Comparing litellm_remove_flaky_azure_batches_e2e (1bcf014) with main (39c1042)

Open in CodSpeed

Copy link
Copy Markdown
Collaborator

@ryan-crabbe-berri ryan-crabbe-berri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Copy Markdown
Contributor

@ishaan-berri ishaan-berri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@ishaan-berri ishaan-berri merged commit a60e19a into main Apr 6, 2026
108 of 113 checks passed
@ishaan-berri ishaan-berri deleted the litellm_remove_flaky_azure_batches_e2e branch April 6, 2026 23:49
harish876 pushed a commit to harish876/litellm that referenced this pull request Apr 8, 2026
The proxy_e2e_azure_batches_tests workflow is consistently flaky and
does not provide reliable signal on whether changes break anything.
Remove the workflow from both CircleCI and GitHub Actions, along with
the test directory it exclusively used.

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants