Enjoy an ad free experience by logging in. Not a member yet? Register.


Results 1 to 5 of 5
Thread: decimal calculation accurately

05242012, 05:04 PM #1
 Join Date
 Mar 2012
 Posts
 60
 Thanks
 65
 Thanked 0 Times in 0 Posts
decimal calculation accurately
Hi
In javascript how do you these calculations accurately
The number is imputed in a form and does not always have the same amount of digits after the decimal.
12345.67891234567 X 3.1234 then round it off to 4 decimal places
123123445.678912 X 3.1234 then round it off to 4 decimal places
123123445.678123443434343912 X 3.1234 then round it off to 4 decimal places
thanks for the help
05242012, 05:08 PM
#2
etcCode:(12345.67891234567*3.1234).toFixed(4)
Users who have thanked xelawho for this post:
codeintime (05302012)
05242012, 05:10 PM
#3
Javascript does all calculations using the builtin hardware math processor. ANSI double precision.
That means that any one number will be accurate to ROUGHLY 15 decimal digits and the result of any ONE operation should be at least accurate to ROUGHLY 14 decimal digits. The more operations you do, the less accuracy that can be guaranteed. Generally, though, you can count on 14 digits in most cases.
And that's it. If you need more accuracy, you will have to find a computer language that supports "double double" precision.
An optimist sees the glass as half full.
A pessimist sees the glass as half empty.
A realist drinks it no matter how much there is.
Users who have thanked Old Pedant for this post:
codeintime (05302012)
05252012, 03:23 AM
#4
 Join Date
 Sep 2005
 Location
 Sydney, Australia
 Posts
 6,472
 Thanks
 0
 Thanked 634 Times in 624 Posts
Note that JavaScript like all programming languages does its calculations in binary  that means that any fractional portion will be adjusted to the nearest binary fraction. So unless your fractions have an exact binary equivalent your calculation will not be exact.
If you need an exact answer to a set number of decimal places then shift the decimal point in all your numbers that many places to the right before doing the calculation and then shift it back after the calculation. For example with currencies that use two decimal places you should always multiply them all by 100 at the start and divide by 100 at the end or the answer might not be exact.
Stephen
Learn Modern JavaScript  http://javascriptexample.net/
Helping others to solve their computer problem at http://www.felgall.com/
Don't forget to start your JavaScript code with"use strict";
which makes it easier to find errors in your code.
Users who have thanked felgall for this post:
codeintime (05302012)
05252012, 05:38 AM
#5
That won't work in any of his examples.
Even the one with fewest digits:
123123445.678912 X 3.1234
Using your suggestion (he wants it accurate to 4 places):
1231234456789.12 X 31234
I count 13 digits multiplied by 5 digits.
That exceeds the accuracy available even with integer values when using an ANSI 64 bit floating point value.
An optimist sees the glass as half full.
A pessimist sees the glass as half empty.
A realist drinks it no matter how much there is.
Users who have thanked Old Pedant for this post:
codeintime (05302012)