c# string to float conversion invalid?

后端 未结 3 2097
天涯浪人
天涯浪人 2021-01-24 04:41
var x = dr[\"NationalTotal\"].ToString();

gives me 333333333

var xxx = Convert.ToSingle(dr[\"NationalTotal\"].ToString());
3条回答
  •  面向向阳花
    2021-01-24 05:44

    The floating point specification says that the 32 bit representation of a floating point number is

    enter image description here

    Thus, the largest integer that can be represented without loss of (integer) accuracy is 16777216 (0x1000000). A simple test program to convince you that this is so:

    #include 
    
    int main(void) {
      float x;
      unsigned long j;
      j = 0x00FFFFFC;
      int i;
      x = j;
      for(i=0; i<10;i++)  printf("%ld + %d = %f\n", j, i, x+i);
    }
    

    Output:

    16777212 + 0 = 16777212.000000
    16777212 + 1 = 16777213.000000
    16777212 + 2 = 16777214.000000
    16777212 + 3 = 16777215.000000
    16777212 + 4 = 16777216.000000
    16777212 + 5 = 16777216.000000  <<< from here on, adding one more doesn't give the right answer
    16777212 + 6 = 16777218.000000
    16777212 + 7 = 16777220.000000
    16777212 + 8 = 16777220.000000
    16777212 + 9 = 16777220.000000
    

    EDIT Based on the comments underneath the question, we slowly converged on the fact that you had two questions, not one.

    The first: "Why is this happening?" is answered with the above. A single float simply is not able to represent 333333333 exactly, so you get the nearest representable value, which is 333333344.

    The second: "How do I fix it?" was initially answered by me in the comments - I will reprise my answer here:

    Your database floating point number is not typically single precision - in fact, by default it is double precision. Thus, you solve your problem by converting the string to double, and assigning it to a double precision variable:

    double xxx = Convert.ToDouble(dr["NationalTotal"]);
    

    I would like to refer you to http://floating-point-gui.de/ - "what every programmer should know about floating point". There are actually many guides out there with similar names. Essential reading if you ever stray from using just integers (and most people will, at some point in their programming career).

提交回复
热议问题