Precision, Scale, Sum, Divide.. Truncation

后端 未结 1 1956
無奈伤痛
無奈伤痛 2021-01-19 18:20

I have the following code:

SELECT -701385.10 -- -701385.10
SELECT SUM(-701385.10) -- -701385.10
SELECT -701385.10/2889991754.89 -- -0.000242694498630
SELECT          


        
相关标签:
1条回答
  • 2021-01-19 18:32

    Your working is wrong

    Precision: p1 - s1 + s2 + max(6, s1 + p2 + 1)
    Scale: max(6, s1 + p2 + 1)
    

    Gives

    Precision: 38 - 2 + 2 + max(6, 2 + 12 + 1) = 53
    Scale: max(6, 2 + 12 + 1)                  = 15
    

    Which is greater than 38 so you are getting truncation as covered here

    0 讨论(0)
提交回复
热议问题