Skip to content

Refactor AddExponent overflow/underflow check#368

Merged
tompng merged 1 commit intoruby:masterfrom
tompng:addexponent_strict_minmax
Jul 11, 2025
Merged

Refactor AddExponent overflow/underflow check#368
tompng merged 1 commit intoruby:masterfrom
tompng:addexponent_strict_minmax

Conversation

@tompng
Copy link
Member

@tompng tompng commented Jul 11, 2025

Just a refactor for BASE_FIG=9 (DECDIG=int32_t) and SIGNED_VALUE is 64bit

Fix this bug

RUBY_PLATFORM
# => "arm-linux-eabihf" (arm32v7/ruby docker image)
x=BigDecimal("0.1e-214748364")
y = x*x*x*x*x*x*x*x*x*x*0.1
# => 0.1e2147483646 # should underflow

Fix this bug for BASE_FIG=4 (DECDIG=int16_t)

x=BigDecimal("0.1E-922337203685477579");
y = x*x*x*x*x*x*x*x*x*x
#=> 0.1e-9223372036854775799
y.precs
# => [4, 12] # by undefining HAVE_INT64_T
z = y*0.00000000001
# => 0.1e9223372036854775806 # exponent underflow. result should be 0.0

# Reason of the bug
# z is 0.0001_0000_0000_0000e-9223372036854775808
# real->exponent is 2305843009213693952 and real->exponent * BASE_FIG does not underflow
# but the actual exponent returned to ruby is -9223372036854775811. This is smaller than SIGNED_VALUE_MIN.

This bug does not happen when BASE_FIG=9 and SIGNED_VALUE is 64bit because SIGNED_VALUE_MIN + BASE_FIG - 1 == -9223372036854775800 is a multiple of BASE_FIG == 9

BigDecimal's exponent should not overflow/underflow long.
To check, BigDecimal restricts the range of vp->exponent.
The calculation is complicated and not correct when BASE_FIG is 4.
@tompng tompng force-pushed the addexponent_strict_minmax branch from 1b570e0 to 02b9f64 Compare July 11, 2025 16:40
@tompng tompng merged commit 15b7e0a into ruby:master Jul 11, 2025
78 checks passed
@tompng tompng deleted the addexponent_strict_minmax branch July 11, 2025 16:45
@tompng tompng added the bug label Jul 12, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant