View Full Version : Problem in arithemetic conversion

vijiruba

09-05-2006, 08:08 AM

Hi,

I have problem in basic arithemetic which makes me hectic in major issues.

i have to compare two numbers "a" and "b". And the values is as follows:

a = 18446744073709552000

b = 1.844674407371E+19

Both the values are equal.

But if i compare in javascript as if(a < b), sometimes it works well. But often there occurs problem since the value of "b" is in E power value. itz comparing as 1.844674407371 leaving E+19.

How to fix this?

Regards,

Viji.

Philip M

09-05-2006, 08:48 AM

A Google search brough me to:-

http://www.irt.org/articles/js069/index.htm

One of the interesting features of JavaScript is, that the precision it uses to judge the accuracy of its calculation algorithms is noticeably lower than that of other programming languages. Despite it now being at revision 1.2, this standard has not yet been improved.

To give you an idea of the nature and depth of the problem, although for instance Authorware, Turbo Pascal, Excel and even Word6 will calculate the following correctly, multiplying:

0.119 by 100 in JavaScript evaluates to 11.899999.

0.14 by 100 in JavaScript evaluates to 14.0000000000002

Normally in JavaScript a value has 16 digits of precision, and this just seems to be the limitations inherent in the routines used.

Hope this helps.

liorean

09-05-2006, 10:44 AM

Actually, JavaScript has the same accuracy and the same mechanics as most other programming languages using standard double precision floating point numbers. What JavaScript lacks, however, is a functionality that hides accumulations of errors such as the one VBScript has.

See Eric Lippert (http://blogs.msdn.com/ericlippert/)'s blog entry Why does JScript have rounding errors? (http://blogs.msdn.com/ericlippert/archive/2003/09/15/53000.aspx) for more on this.

For more on understanding why this is so, have a look at:

Floating Point Arithmetic, Part One (http://blogs.msdn.com/ericlippert/archive/2005/01/10/350108.aspx)

Benford's Law (http://blogs.msdn.com/ericlippert/archive/2005/01/12/351693.aspx)

Floating Point And Benford's Law, Part Two (http://blogs.msdn.com/ericlippert/archive/2005/01/13/352284.aspx)

Fun With Floating Point Arithmetic, Part Three (http://blogs.msdn.com/ericlippert/archive/2005/01/17/354658.aspx)

Fun with Floating Point Arithmetic, Part Four (http://blogs.msdn.com/ericlippert/archive/2005/01/18/355351.aspx)

Fun with Floating Point Arithmetic, Part Five (http://blogs.msdn.com/ericlippert/archive/2005/01/20/357407.aspx)

Fun With Floating Point Arithmetic, Part Six (http://blogs.msdn.com/ericlippert/archive/2005/01/26/361041.aspx)

Powered by vBulletin® Version 4.2.2 Copyright © 2017 vBulletin Solutions, Inc. All rights reserved.