View Full Version : decimal calculation accurately

code-in-time

05-24-2012, 06:04 PM

Hi

In javascript how do you these calculations accurately

The number is imputed in a form and does not always have the same amount of digits after the decimal.

12345.67891234567 X 3.1234 then round it off to 4 decimal places

123123445.678912 X 3.1234 then round it off to 4 decimal places

123123445.678123443434343912 X 3.1234 then round it off to 4 decimal places

thanks for the help

xelawho

05-24-2012, 06:08 PM

(12345.67891234567*3.1234).toFixed(4)

etc

Old Pedant

05-24-2012, 06:10 PM

Javascript does all calculations using the builtin hardware math processor. ANSI double precision.

That means that any one number will be accurate to ROUGHLY 15 decimal digits and the result of any ONE operation should be at least accurate to ROUGHLY 14 decimal digits. The more operations you do, the less accuracy that can be guaranteed. Generally, though, you can count on 14 digits in most cases.

And that's it. If you need more accuracy, you will have to find a computer language that supports "double double" precision.

felgall

05-25-2012, 04:23 AM

Note that JavaScript like all programming languages does its calculations in binary - that means that any fractional portion will be adjusted to the nearest binary fraction. So unless your fractions have an exact binary equivalent your calculation will not be exact.

If you need an exact answer to a set number of decimal places then shift the decimal point in all your numbers that many places to the right before doing the calculation and then shift it back after the calculation. For example with currencies that use two decimal places you should always multiply them all by 100 at the start and divide by 100 at the end or the answer might not be exact.

Old Pedant

05-25-2012, 06:38 AM

If you need an exact answer to a set number of decimal places then shift the decimal point in all your numbers that many places to the right before doing the calculation and then shift it back after the calculation.

That won't work in any of his examples.

Even the one with fewest digits:

123123445.678912 X 3.1234

Using your suggestion (he wants it accurate to 4 places):

1231234456789.12 X 31234

I count 13 digits multiplied by 5 digits.

That exceeds the accuracy available even with integer values when using an ANSI 64 bit floating point value.

Powered by vBulletin® Version 4.2.2 Copyright © 2016 vBulletin Solutions, Inc. All rights reserved.