Skip to content

Fix wrong converge check in VpSqrt#353

Merged
tompng merged 1 commit intoruby:masterfrom
tompng:fix_sqrt_precision_bug
Jun 13, 2025
Merged

Fix wrong converge check in VpSqrt#353
tompng merged 1 commit intoruby:masterfrom
tompng:fix_sqrt_precision_bug

Conversation

@tompng
Copy link
Member

@tompng tompng commented Jun 13, 2025

Fix #338

x/y == y does not mean y converged to sqrt(x) if the precision of x/y is not high enough.
Perhaps there is a more complicated way to fix this convergence check, but #323 or #343 is better than doing it.

@tompng tompng merged commit 95bf47a into ruby:master Jun 13, 2025
123 of 124 checks passed
@tompng tompng deleted the fix_sqrt_precision_bug branch June 13, 2025 13:31
@tompng tompng added the bug label Jul 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Wrong sqrt calculation

1 participant