Skip to content

Fix GetAddSubPrec calculation#406

Merged
tompng merged 1 commit intoruby:masterfrom
tompng:addsub_prec_wrong_error_fix
Aug 20, 2025
Merged

Fix GetAddSubPrec calculation#406
tompng merged 1 commit intoruby:masterfrom
tompng:addsub_prec_wrong_error_fix

Conversation

@tompng
Copy link
Member

@tompng tompng commented Aug 20, 2025

x = BigDecimal('1' * 100) + 12345
# => 0.1111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111123456e100
x.precs #=> [108, 234] → [108, 117]

Removes overflow/underflow checks. It won't happen.
a->Prec * SIZEOF_DECDIG is allocated memory size. fits in size_t. SIZEOF_DECDIG is 2 or 4.
a->exponent * BASE_FIG fits in ssize_t and BASE_FIG is 4 or 9.

@tompng tompng merged commit c2cb664 into ruby:master Aug 20, 2025
79 checks passed
@tompng tompng deleted the addsub_prec_wrong_error_fix branch August 20, 2025 16:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant