-
Notifications
You must be signed in to change notification settings - Fork 2.3k
feat: Allow specifying the hex width to use when generating shard ranges. #18633
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: Allow specifying the hex width to use when generating shard ranges. #18633
Conversation
Review ChecklistHello reviewers! 👋 Please follow this checklist when reviewing this Pull Request. General
Tests
Documentation
New flags
If a workflow is added or modified:
Backward compatibility
|
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #18633 +/- ##
=======================================
Coverage 67.48% 67.48%
=======================================
Files 1613 1613
Lines 263962 263948 -14
=======================================
- Hits 178146 178137 -9
+ Misses 85816 85811 -5 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
…d ranges. Signed-off-by: Arthur Schreiber <[email protected]>
Signed-off-by: Arthur Schreiber <[email protected]>
0018ddf to
92afd27
Compare
This allows specifying how many characters should be used in the generated shard labels. Using a higher number if characters reduces unevenness of shard sizes when using shard counts that are not a power of two. Signed-off-by: Arthur Schreiber <[email protected]>
nickvanw
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks good to me, will wait for feedback from others on the command line flag.
It would be good to add some more exhaustive tests for common scenarios, I think - passing 0 in for various values, and overriding to 3 or 4 at smaller shard values.
| // equal distribution of N shards. | ||
| GenerateShardRanges = &cobra.Command{ | ||
| Use: "GenerateShardRanges <num_shards>", | ||
| Use: "GenerateShardRanges <num_shards> [--chars]", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think --chars makes sense. Other things I thought about:
--num-digits--digits--hex-char-count--char-count--range-digits
Let me know if you or anyone thinks any of those are better.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Claude likes --digits. It also suggests --width.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
what about --hex-format-length or --format-length
Signed-off-by: Arthur Schreiber <[email protected]>
I added another test case for 512 shards (to verify that using a shard number > 256 uses 4 characters). |
a6e7216 to
7874274
Compare
harshit-gangal
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Like @nickvanw said, need some test with different hex length passed to the function and finalizing on the flag name
A test like 8 shards with 4 length |
Isn't this covered by https://github.com/vitessio/vitess/pull/18633/files#diff-742b022d7e1f659dd60ba4e410fa90b9597aa4c941e0edc19b56e8ab068d6f6bR1596-R1637? |
Signed-off-by: Arthur Schreiber <[email protected]>
472910a to
b683f2f
Compare
…/generate-shard-ranges Signed-off-by: Arthur Schreiber <[email protected]>
Signed-off-by: Arthur Schreiber <[email protected]>
Signed-off-by: Arthur Schreiber <[email protected]>
|
I renamed the option to |
nickvanw
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
![]()
Description
This adds a new
--hex-widthcli flag tovtctldclient GenerateShardRanges. This flag allows specifying how many characters to use for start and end parts of the shard range labels. This can be used to reduce the shard size differences when using a shard count that is not a power of 2.I'm not super happy with the naming of the CLI flag, suggestions for alternative naming are welcome.
Related Issue(s)
Fixes: #15744
References: #17688
Checklist
Deployment Notes