VB 6 & CSng()

Friday, January 23, 2004 2:39:00 PM
Here's a trick question hopefully someone can shed some light on. Why does this piece of VB6 Code:

For x = 8 To 10 Step 0.1
    sngVar = CSng(x)
    newSngVar = sngVar * 100
    Debug.Print newSngVar
Next

Produce the output:

800
810.0001
820
830
839.9999
850
860.0001
870
880
889.9999
900
910.0001
920
930
939.9999
950
960.0001
970
980
989.9999
1000

Is there some sort of rounding/floating point issue that is happening which low-lifes like me are not aware of?

The fix for this is pretty obvious (cdbl() or val()) but the reason why it happens would be a "nice to know".

3 Comments

  • Dave Rothgery said

    Did you snip out declarations of x, sngVar, and newSngVar as Single?

    If not, there's a lot of implicit Single<->Double converstions going on, and it's easy to lose some percision in there.

  • Rob Chartier said


    Here's some C# examples of the same behaviour:


    double x=8;
    while(x<=10) {
    System.Single s = Convert.ToSingle(x);
    System.Single s1 = s * 100;
    Console.WriteLine(s1);
    x=x+0.1;
    }

    Console.WriteLine("--------------------");

    float y=8;
    while(y<=10) {
    float s2 = (float)y;
    float s3 = s2 * 100;
    Console.WriteLine(s3);
    y=y+(float)0.1;
    }

  • denny said

    it's called "floating point" for several reasons.....

    1/3 => .333333333333333333333333333333333333333333333333333333333333

    and many other fun bits lurk in the land of the less than perfect number game.

    never trust the fractions in fp math unless you test them or force them into whole numbers and then back to fractions.

Comments have been disabled for this content.