Why doesn't python decimal library return the specified number of signficant figures for some inputs

一个人想着一个人 提交于 2019-12-06 02:05:00

Is it a bug? I don't know, I don't think the documentation is tight enough to make that determination. It certainly is a surprising result.

It is possible to fix your own function with a little more logic.

def to_sigfigs(value, sigfigs):
    sign, digits, exponent = decimal.Context(prec=sigfigs).create_decimal(value).as_tuple()
    if len(digits) < sigfigs:
        missing = sigfigs - len(digits)
        digits = digits + (0,) * missing
        exponent -= missing
    return str(decimal.Decimal((sign, digits, exponent)))

The precision of a context is a maximum precision; if an operation would produce a Decimal with less digits than the context's precision, it is not padded out to the context's precision.

When you call to_sigfigs(0.000001, 5), 0.000001 already has some rounding error due to the conversion from source code to binary floating point. It's actually 9.99999999999999954748111825886258685613938723690807819366455078125E-7. Rounding that to 5 significant figures gives decimal.Decimal("0.0000010000").

On the other hand, 1 is exactly representable in binary floating point, so 1.0 is exactly 1. Since only 1 digit is needed to represent this in decimal, the context's precision doesn't require any rounding, and you get a 1-digit Decimal result.

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!