Skip to content

[Infra] Remove Redundant Matrix Unit Test Workflow#25251

Merged
ishaan-berri merged 3 commits intomainfrom
litellm_/elastic-goodall
Apr 6, 2026
Merged

[Infra] Remove Redundant Matrix Unit Test Workflow#25251
ishaan-berri merged 3 commits intomainfrom
litellm_/elastic-goodall

Conversation

@yuneng-berri
Copy link
Copy Markdown
Collaborator

Summary

Problem

The test-litellm-matrix.yml workflow duplicates all test coverage already provided by the newer semantic unit test workflows (test-unit-*.yml). This results in redundant CI spend on every PR.

Fix

Remove test-litellm-matrix.yml. Every matrix group maps 1:1 to an existing semantic workflow:

Matrix Group Replacement Workflow
llms-vertex, llms-other test-unit-llm-providers.yml
proxy-guardrails, proxy-misc (endpoints) test-unit-proxy-endpoints.yml
proxy-core (auth/hooks/policy) test-unit-proxy-auth.yml
proxy-core (db), proxy-misc (infra) test-unit-proxy-infra.yml
integrations test-unit-integrations.yml
core-utils test-unit-core-utils.yml
other-1 (responses/caching/types) test-unit-responses-caching-types.yml
other-2 (enterprise/genai/routing) test-unit-enterprise-routing.yml
other-3 + root test-unit-misc.yml
proxy-unit-a1 through b7 test-unit-proxy-legacy.yml

Testing

  • Verified every test path in the matrix is covered by at least one semantic workflow
  • No required status checks reference the matrix workflow

Type

🚄 Infrastructure

All test paths in test-litellm-matrix.yml are fully covered by the
newer semantic unit test workflows (test-unit-*.yml), making the
matrix workflow redundant CI spend.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@yuneng-berri yuneng-berri requested a review from a team April 6, 2026 22:27
@vercel
Copy link
Copy Markdown

vercel bot commented Apr 6, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
litellm Ready Ready Preview, Comment Apr 6, 2026 10:59pm

Request Review

@CLAassistant
Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@codspeed-hq
Copy link
Copy Markdown
Contributor

codspeed-hq bot commented Apr 6, 2026

Merging this PR will not alter performance

✅ 16 untouched benchmarks


Comparing litellm_/elastic-goodall (e205104) with main (39c1042)

Open in CodSpeed

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps bot commented Apr 6, 2026

Greptile Summary

This PR removes the redundant test-litellm-matrix.yml workflow and adds code coverage reporting (via pytest-cov + Codecov OIDC upload) to all semantic unit test workflows.

  • Deleted test-litellm-matrix.yml — every matrix group maps 1:1 to an existing semantic workflow, so this was pure duplicate CI spend on every PR
  • Added coverage reporting to both _test-unit-base.yml and _test-unit-services-base.yml via --cov=litellm --cov-report=xml:coverage.xml --cov-config=pyproject.toml pytest flags
  • Added upload-coverage job to both base templates — runs after run with if: always(), downloads the coverage artifact, and uploads to Codecov using OIDC (use_oidc: true, fail_ci_if_error: false prevents blocking CI)
  • Updated all 13 caller workflows to supply a unique artifact-name and to include id-token: write / pull-requests: write permissions required for the OIDC-based Codecov upload
  • Minor inconsistency: _test-unit-base.yml declares artifact-name as required: true, while _test-unit-services-base.yml defaults it to "run" — this could cause silent artifact name collisions if a future matrix caller omits the parameter

Confidence Score: 5/5

Safe to merge — all changes are confined to GitHub Actions workflow files with no production code impact

All 15 changed files are GitHub Actions YAML. The matrix workflow deletion is cleanly mapped to existing semantic workflows. Coverage instrumentation is additive and non-blocking (fail_ci_if_error: false). The only finding is a P2 inconsistency in the artifact-name default between the two base templates, which does not affect any current caller. P2-only findings warrant a 5/5 score.

_test-unit-services-base.yml has a minor required: false vs required: true inconsistency for artifact-name relative to _test-unit-base.yml

Important Files Changed

Filename Overview
.github/workflows/test-litellm-matrix.yml Deleted — redundant matrix test workflow replaced 1:1 by the existing semantic unit test workflows
.github/workflows/_test-unit-base.yml Added required artifact-name input, pytest-cov coverage flags, artifact upload, and a new upload-coverage job with OIDC-based Codecov integration
.github/workflows/_test-unit-services-base.yml Same coverage/Codecov additions as _test-unit-base.yml; artifact-name is optional with default 'run' (inconsistent with _test-unit-base.yml's required: true)
.github/workflows/test-unit-proxy-db.yml Added job-level id-token/pull-requests write permissions and unique per-matrix-group artifact names for coverage upload
.github/workflows/test-unit-llm-providers.yml Added job-level id-token/pull-requests write permissions and unique artifact names for vertex-ai and other-providers jobs

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    PR[Pull Request / Push] --> WF1[test-unit-llm-providers.yml]
    PR --> WF2[test-unit-proxy-auth.yml]
    PR --> WF3[test-unit-proxy-endpoints.yml]
    PR --> WF4[test-unit-proxy-infra.yml]
    PR --> WF5[test-unit-misc.yml]
    PR --> WF6[test-unit-enterprise-routing.yml]
    PR --> WF7[test-unit-integrations.yml]
    PR --> WF8[test-unit-responses-caching-types.yml]
    PUSH[Push to main] --> WF9[test-unit-proxy-db.yml]
    PUSH --> WF10[test-unit-caching-redis.yml]
    PUSH --> WF11[test-unit-security.yml]
    WF1 --> BASE[_test-unit-base.yml]
    WF2 --> BASE
    WF3 --> BASE
    WF4 --> BASE
    WF5 --> BASE
    WF6 --> BASE
    WF7 --> BASE
    WF8 --> BASE
    WF9 --> SBASE[_test-unit-services-base.yml]
    WF10 --> SBASE
    WF11 --> SBASE
    BASE --> RUN1[job: run\npytest + --cov]
    BASE --> UP1[job: upload-coverage\nOIDC to Codecov]
    SBASE --> RUN2[job: run\npytest + --cov]
    SBASE --> UP2[job: upload-coverage\nOIDC to Codecov]
    RUN1 -->|coverage artifact| UP1
    RUN2 -->|coverage artifact| UP2
    DELETED[~~test-litellm-matrix.yml~~ DELETED]
    style DELETED fill:#ffcccc,stroke:#cc0000
Loading

Reviews (3): Last reviewed commit: "Move id-token/pull-requests permissions ..." | Re-trigger Greptile

Add coverage collection (--cov) and Codecov OIDC upload to both
reusable base workflows and all 12 caller workflows, replacing the
coverage reporting that was previously only in the matrix workflow.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…rkflows

For workflows with multiple jobs (llm-providers, proxy-db), move
id-token: write and pull-requests: write from workflow level to job
level so permissions are scoped to only the jobs that need them.
Removes zizmor inline suppressions that were masking the issue.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Copy link
Copy Markdown
Contributor

@ishaan-berri ishaan-berri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@ishaan-berri ishaan-berri merged commit d132b1b into main Apr 6, 2026
96 of 99 checks passed
@ishaan-berri ishaan-berri deleted the litellm_/elastic-goodall branch April 6, 2026 23:52
harish876 pushed a commit to harish876/litellm that referenced this pull request Apr 8, 2026
* Remove redundant matrix unit test workflow

All test paths in test-litellm-matrix.yml are fully covered by the
newer semantic unit test workflows (test-unit-*.yml), making the
matrix workflow redundant CI spend.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Add Codecov coverage reporting to semantic unit test workflows

Add coverage collection (--cov) and Codecov OIDC upload to both
reusable base workflows and all 12 caller workflows, replacing the
coverage reporting that was previously only in the matrix workflow.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Move id-token/pull-requests permissions to job level for multi-job workflows

For workflows with multiple jobs (llm-providers, proxy-db), move
id-token: write and pull-requests: write from workflow level to job
level so permissions are scoped to only the jobs that need them.
Removes zizmor inline suppressions that were masking the issue.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants