-
Notifications
You must be signed in to change notification settings - Fork 2.3k
CI: Improve reliability of codecov workflow with larger runner #18992
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
By using a larger runner Signed-off-by: Matt Lord <[email protected]>
Review ChecklistHello reviewers! 👋 Please follow this checklist when reviewing this Pull Request. General
Tests
Documentation
New flags
If a workflow is added or modified:
Backward compatibility
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #18992 +/- ##
==========================================
+ Coverage 69.82% 69.85% +0.03%
==========================================
Files 1610 1610
Lines 215303 215303
==========================================
+ Hits 150328 150398 +70
+ Misses 64975 64905 -70 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
@mattlord nice catch! I've been looking into this as well and was a bit lost 🚀 |
Signed-off-by: Matt Lord <[email protected]>
Signed-off-by: Matt Lord <[email protected]>
…r runner (#18992) (#18994) Signed-off-by: Matt Lord <[email protected]> Co-authored-by: vitess-bot[bot] <108069721+vitess-bot[bot]@users.noreply.github.com>
…r runner (#18992) (#18995) Signed-off-by: Matt Lord <[email protected]> Co-authored-by: vitess-bot[bot] <108069721+vitess-bot[bot]@users.noreply.github.com>
Description
The
codecovworkflow has been VERY flaky lately. Typically taking 3+ attempts before it succeeds. This workflow runs every unit test with code coverage enabled — so it's heavier than our other unit test workflows which have split off the evalengine tests into a separate workflow — and the failures I kept seeing made no sense, unless we were resource starved. So to test that theory I used a larger runner (up until 9 months ago, we were using a larger one). You can see the results here from 10 runs: https://github.com/vitessio/vitess/actions/runs/19942952837?pr=18992We went from having to run it 2-5 times to get it to pass once, so let's say it was passing ~ 30% of the time. Whereas on this branch it passed 7 out of 10 times, so 70%. That is pretty good, considering that we DO have a number of flaky individual unit tests.
Note
Backporting to v22 so that we have an improved CI for all supported release branches.
Related Issue(s)
Checklist
AI Disclosure