Python: poor printing of floating point

See how Python prints a simple decimal number:

>>> 94.8
94.799999999999997
>>>

[edit on 2010-01-26: allegedly this is all fixed in Python 3.1. Please skip article if that’s okay with you]

In an attempt to dispell comments on the lines of “floating point doesn’t produce exact answers”, I should point out that I know:

• Floating point is tricky; and,
• 94.8 is not exactly representable as a floating point number.

Without delving into the arcane details of floating point let’s lay out some basics. Each floating point value is some exact real number, in fact a rational number. I’m ignoring NaNs and infinities because they’re not important for this discussion. There are only finitely many floating point values.

With regards to input and output of floating point numbers what is reasonable?

Reasonable input requirement

Every decimal number has some floating point number that is nearest (possibly two if it lies midway between two adjacent floating point numbers). It seems reasonable to me that when converting decimal numbers to floating point numbers (as Python does when it reads 94.8 in the program) it should pick the nearest floating point number to represent it internally (and if there are two such nearest, then pick one). (For very large magnitude numbers you might want to return infinity or throw an exception; doesn’t matter for this article.)

Reasonable output requirement

When converting a number from internal floating point format to decimal (as Python does when printing) it seems reasonable to print one of the decimal numbers whose nearest floating point number is the one we are printing. In other words we would like to print a number that when read according to the reasonable input requirement (above) we get the same internal floating point number. Note that there are infinitely many decimal numbers with a given nearest floating point number (because there are only finitely many floating point numbers); once you’ve printed enough decimal digits to zoom in on the desired floating point number you can keep on printing whatever digits you like, generating lots of different decimal numbers all of which share the same nearest floating point number.

Python certainly meets our reasonable output requirement. 94.799999999999997 and 94.8 share the same nearest floating point number.

Desirable output requirement

Consider the two number 94.799999999999997 and 94.8. By our reasonable output requirement above, they’re both good numbers to print out, but the shorter one is easier to read and think about for humans. It also satisfies an elegant principle: Don’t print out any more digits than you need to.

So a desirable requirement for printing out numbers is that we print out a number that meets the reasonable requirement, and is the shortest possible of all such numbers.

If you need a bit of persuading that shorter is better, then this argument might help you: Consider π (pi). An approximation to 18 decimal places is 3.141592653589793238. Now imagine you have written a Python program to compute π as accurately as it can. It prints 3.1415926535897931. That last digit is wrong. It’s not so much wrong as unnecessary, 3.141592653589793 represents the same floating point number:

>>> 3.141592653589793 == 3.1415926535897931
True

By printing that extra garbage digit Python is misleading you as to how many digits of precision it has.

Can we be desirable?

So how easy is it to generate reasonable output and desirable output? Well it turns out to be a bit trickier than you might think on first blush, and it requires bignums. On the other hand, this stuff was all sorted out about 30 years ago, and every reasonable machine is large enough to do bignum arithmetic without busting a gut. Unfortunately the gurus Steele and White didn’t publish the algorithm until 1990 (see [SW1990]). To quote them: “Why wasn’t it published earlier? It just didn’t seem like that big a deal; but we were wrong.” Boy were they wrong. Even 17 years after publication, modern implementations of modern languages running on modern hardware still get it wrong.

[Political rant: I’m inclined to think that one the contributory factors is that the article was published by the (“we love dead trees”) ACM. It just doesn’t seem helpful to lock all this knowledge away and charge for it. I happen to have a photocopied version of this article on paper (I guess that makes me lucky, and able to write this blog article), but in 2007 I can hardly believe that I have to resort to paper copies to do research.]

Bottom line: 94.8 should print out as 94.8.

Python’s excuse will no doubt be that it relies on the platform’s C library. C has an excuse. C targets tiny platforms where it would be ridiculous to suggest that the printing routines use 320 bytes of RAM to store intermediate computations (on a platform I recently had the pleasure to program, this would’ve been about 10% of the entire RAM). On the other hand any platform where I’m actually likely to use double I almost certainly have enough space for both for code and the bignum arithmetic it requires to get the right answer. Certainly, on my MacBook the C implementation has no excuse.

It’s no coincidence that Steele had a hand in writing the 1990 paper and also a hand in designing two of the languages that actually bother to say something sensible about converting internal numbers to external format. Scheme says that reading what you print will yield the same number, and this shall be done using the minimum number of digits. Java says “as many, but only as many, more digits as are needed to uniquely distinguish the argument value from adjacent values of type double”. Curiously, Common Lisp doesn’t seem to actually state anything like that, but possibly I just can’t find it. Common Lisp implementations, of course, get it right, because they’re invested with the spirit of Do The Right Thing. I had a look to see what Haskell said on the matter, but apart from noting with amusement that a literal number did not denote an internal number object I couldn’t find anything. ECMAScript also specifies the Right Thing, but then, Wikipedia reckons that Steele edited the first edition.

Python has automatic memory management, bignums, strings. It feels like a big language, abstracted and distant from the silicon. It’s time for Python to grow up and specify that repr for float returns a decimal string the mathematical value of which is closer to the argument than any other float, and it returns a string with the fewest number of digits.

The trickiest thing about Steele and White’s algorithm is the bignums. Python has bignums. It turns out to be very easy indeed to implement Steele and White’s algorithm in Python. We can then use this algorithm to create conversion routines in the style of ECMAScript or Java. And I have done so. The implementation isn’t very large, but it is too large to contain in the margins of this article. If you want the code then go to, yet another dead before it began sourceforge project, flopri.

Appendix A – Other Stuff I Found On The Way

Comparison with other languages

Compare Python with similar codes in shell, awk, C, and Lisp:

$ printf ‘%.20f\n’ 94.8
94.80000000000000000278
$ awk ‘BEGIN{printf”%.20f\n”,94.8}’
94.79999999999999715783
$ echo ‘#include <stdio.h>
int main(void){printf(“%.20f\n”,94.8);}’ > t.c
$ cc t.c; ./a.out
94.79999999999999715783
$ sbcl –noinform –eval ‘(progn(format t “~,20f~%” 94.8d0)(sb-ext:quit))’
94.80000000000000000000

Bit of a mixed bag isn’t it? At least Lisp gets it right.

Blind Alleys

Python has a module called fpformat which sounds like it might be the sort of place to find optimal floating point formatting routines, but its useless. “unneeded” as its own documentation avers. pprint is no good either; pretty, but no smarts.

Python’s Tutorial

The Python Tutorial, good in many ways, has what I think is a very weak appendix on this matter: Appendix B – Floating Point Arithmetic: Issues and Limitations (warning: fragile looking URL). It reads more like an apology than justification and varies between patronising, annoying, wrong, and helpful. It contains mistakes like:

Stop at any finite number of bits, and you get an approximation. This is why you see things like:
>>> 0.1
0.10000000000000001

As discussed above, 0.1 is a perfectly good approximation to the floating point value that 0.1 becomes on input, so approximation itself is not the reason that Python prints so many digits (don’t believe me, then try evaluating «0.1 == 0.10000000000000001»). The appendix goes on to recommend str if you want prettier output. I think this is the wrong fix for the problem; if Python used a different printing routine for repr then people wouldn’t be driven to str.

“You’ll see the same kind of thing in all languages that support your hardware’s floating-point arithmetic”. No I won’t. Java, ECMAScript, Lisp all get it right.

REFERENCES

[SW1990] “How to Print Floating Point Numbers Accurately”; Guy L. Steele Jr., Jon L White; ACM SIGPLAN’90 Conference on Programming Language Design and Implementation; 1990-06