Sql Server Precision Crazyness

后端 未结 2 1862
野趣味
野趣味 2020-12-11 19:28

I am having an issue with sql server precision.

I have the following queries:

DECLARE @A numeric(30,10)
DECLARE @B numeric(30,10)
SET @A = 20.225
SET         


        
2条回答
  •  醉梦人生
    2020-12-11 20:23

    This is important at the bottom of the link that you pasted:

    • The result precision and scale have an absolute maximum of 38. When a result precision is greater than 38, the corresponding scale is reduced to prevent the integral part of a result from being truncated.

    For all of the results where you have a precision of 30, the resultant calculated precision is 61. Since the maximum precision possible is 38 the resultant precision is being reduced by 23. Thus, all of the scales are being reduced as well to avoid truncating the integral parts of the result any more than absolutely necessary.

    The 2nd to last value, where the precision of each value is 20, the resultant precision is 41, which only needs to be reduced by 3, leaving a might lighter reduction in the scale portion.

    (30,15) works because the resultant scale is 30, so, when it gets reduced it's still large enough to hold the value you want.

    Lesson: Don't make precision and scale any large than you need them to be, or you'll get odd results.

提交回复
热议问题