View Full Version : Problem in arithemetic conversion
09-05-2006, 07:08 AM
I have problem in basic arithemetic which makes me hectic in major issues.
i have to compare two numbers "a" and "b". And the values is as follows:
a = 18446744073709552000
b = 1.844674407371E+19
Both the values are equal.
How to fix this?
09-05-2006, 07:48 AM
A Google search brough me to:-
To give you an idea of the nature and depth of the problem, although for instance Authorware, Turbo Pascal, Excel and even Word6 will calculate the following correctly, multiplying:
Hope this helps.
09-05-2006, 09:44 AM
See Eric Lippert (http://blogs.msdn.com/ericlippert/)'s blog entry Why does JScript have rounding errors? (http://blogs.msdn.com/ericlippert/archive/2003/09/15/53000.aspx) for more on this.
For more on understanding why this is so, have a look at:
Floating Point Arithmetic, Part One (http://blogs.msdn.com/ericlippert/archive/2005/01/10/350108.aspx)
Benford's Law (http://blogs.msdn.com/ericlippert/archive/2005/01/12/351693.aspx)
Floating Point And Benford's Law, Part Two (http://blogs.msdn.com/ericlippert/archive/2005/01/13/352284.aspx)
Fun With Floating Point Arithmetic, Part Three (http://blogs.msdn.com/ericlippert/archive/2005/01/17/354658.aspx)
Fun with Floating Point Arithmetic, Part Four (http://blogs.msdn.com/ericlippert/archive/2005/01/18/355351.aspx)
Fun with Floating Point Arithmetic, Part Five (http://blogs.msdn.com/ericlippert/archive/2005/01/20/357407.aspx)
Fun With Floating Point Arithmetic, Part Six (http://blogs.msdn.com/ericlippert/archive/2005/01/26/361041.aspx)
Powered by vBulletin® Version 4.2.2 Copyright © 2016 vBulletin Solutions, Inc. All rights reserved.