rounding off errors

[lpok]

When I add certain numbers together, I sometimes get a number back that is real close to the correct answer, but not the exact answer. For example, when I do something like this:

alert(5.5+1.3+.1);

The answer I get is 6.899999999999995.


Anyone else experiencing problems like this? This happens in all browsers I tested on, IE4, IE5, and NS6.

[joh6nn]

that happens in every programming language, due to the way fractional numbers are stored. i don't think there's anything you can do about it.

[lpok]

Originally posted by joh6nn
that happens in every programming language, due to the way fractional numbers are stored. i don't think there's anything you can do about it.

You say this effects fraction numbers, what if I used integers to store all my variables, example, instead of my three variables being 5.5, 1.3, and 0.1, I save them as 55, 13, and 0.1. Then whenever I need to display on the screen, I add them and divide by 10. Will this work? Does the round off error effect adding intergers and dividing by multiples of 10?

[joh6nn]

i think storing them as whole numbers, and dividing by 10, would work, but i've personally never tried it, so don't hold me to that.

[Lee Brenner]

Javascript makes no distinction between integers and floating-point numbers.

[Zvona]

Originally posted by Dave Clark
The only thing you can do is "fix it".

For NS4:

alert(Math.round((5.5+1.3+.1)*10)/10);

For IE and NS6:

alert((5.5+1.3+.1).toFixed(1));

Hey, what is that method? I've never seen or heard it before?

Hmm..time to go read some tutorials ;)