Improve numerical stability and domain coverage of gamma_approx and ln_gamma_approx#105
Improve numerical stability and domain coverage of gamma_approx and ln_gamma_approx#105jzeuzs wants to merge 5 commits into
gamma_approx and ln_gamma_approx#105Conversation
ln_gamma_approx using Euler's reflection formulagamma_approx and ln_gamma_approx
|
I reimplemented the integer fast-path for The fix prevents |
Axect
left a comment
There was a problem hiding this comment.
Thanks for the improvement. The pole handling and the log-reflection formula look correct, and the added tests cover the tricky sign cases well.
One thing I'd like to fix before merging.
On the integer fast-path (src/special/lanczos.rs).
if z > 0.0 && z.fract() == 0.0 {
let n = (z - 1.0) as u64;
for i in 2..=n {
result *= i as f64;
}
return result;
}- Any
f64with magnitude above2^52is already integral, soz.fract() == 0.0is true for huge values too, and they fall into this loop. resultoverflows toINFINITYaroundi = 171, but the loop keeps running up ton. Sogamma(1e16)spins ~1e16 iterations, and1e300 as u64saturates tou64::MAX, which effectively never returns.- Since public
gamma()reaches this path with no magnitude guard, this is triggerable from outside as a hang.
Could you add a magnitude guard before the loop? Something like returning INFINITY for z > 171.0 (since gamma(171) is finite and gamma(172) already overflows) would keep the exact factorial for small integers and avoid the loop for large inputs.
Everything else looks good to me. Thanks again.
We introduce domain checks, pole handling, and Euler's reflection formula to both
gamma_approxandln_gamma_approx. These changes prevent integer overflows and avoid NaN crashes on valid inputs.Previously,$z \lt 0.5$ and fails completely for negative numbers, causing severe precision loss or NaN crashes.
ln_gamma_approxblindly evaluated the Lanczos approximation for all inputs. Lanczos breaks down forAnd, at exact negative integers (e.g.,$z = -1.0$ ),
(PI * z).sin()yields a tiny non-zero float rather than exactly 0.0. This causedgamma_approxto return massive incorrect finite numbers instead of failing mathematically, andln_gamma_approxto return finite bounds instead of infinity.Further, the fast-path for exact integers utilized the factorial function. Because factorial returns an integer, this caused a hard overflow/panic for any$z > 21$ .
The following changes have been made:
ln_gamma_approx.abs()to the sine calculation in the reflection formula.gamma_approxfactorial(z_int - 1)shortcut. By falling back toln_gamma_approx(z).exp(), the function can now safely evaluate exact integers up to