If you ask a programming language to calculate 0.6 + 0.7, you’ll get something like 1.2999998, and that’s because of how floating point numbers are represented in computers. But if you ask a calculator, or even the calculator app on your phone, to do the same thing, it will give you the correct answer: 1.3. Why? Do calculators not represent floats the same way that other computers do?
4
-
1$\begingroup$ Different languages use different internal representations for floating point numbers. Some languages will definitely get the right value here. $\endgroup$– Thomas AndrewsCommented Dec 16, 2023 at 0:18
-
2$\begingroup$ Good question, though it’s not really a good fit for this forum. I’d guess that some calculators do some checks to see if you have entered a short decimal number and if so they do exact arithmetic rather than floating point addition. I am just making this up though. Perhaps the scientific computing stackexchange would be a better fit. $\endgroup$– littleOCommented Dec 16, 2023 at 0:21
-
1$\begingroup$ If they can give you 12 digits, they might calculate to maybe 16, then round. $\endgroup$– Empy2Commented Dec 16, 2023 at 0:26
-
2$\begingroup$ It may be that the calculator is using some version of Binary Coded Decimal instead of binary floating point in its internal representation of numbers. $\endgroup$– awkwardCommented Dec 16, 2023 at 14:00
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
1
The short version is, they're cheating. Calculators or calculator apps are, if anything, less capable than computers and programming languages in general. That said, they're also specialized for doing exactly that sort of thing, so they're more inclined to take 12.9999998 and round it for you to 13. Programming languages will do the same thing, but depending on which language and how precise the number, you might have to ask them to.
Take any calculator and try something like adding $1+10^{-100}-1$, and in my experience, you'll get $0$, not $0.0000\dots 001$.
-
$\begingroup$ Typically there is no "magic" rounding involved: Many calculators simply perform computations with more digits internally than are being displayed. Presumably this can lead double-rounding issues (first round to internal precision, then round to display precision). $\endgroup$– njuffaCommented Dec 16, 2023 at 23:44