Archive for the 'maths' Category

Minus times a minus is a plus


My response to the blog wars about multiplying negative numbers. Mostly inspired by Eric’s comment on Mike Croucher’s Walking Randomly.

Big image, links to a PDF (of vector goodness).


I wanted to put the Inkscsape SVG source inside the PNG image. But it turns out “optimises” the image and means my klever hack doesn’t work. Bad

Multiplication, Addition, Counting, and Python


Jason Dyer’s teasing post got me thinking. About how Python could be used to give some insight into the meta-cognitive aspects of whole number multiplication. Natch.

When children solve a multiplication problem by correspondence, the objects in the multiplier set are mapped over for each object in the multiplicand set (hmm, or is it the other way around?). A typical procedure for multiplying 4 cakes by a price of £2 per cake might be to point to a cake with the left hand and then count up the 2 pounds using the right hand, then move the left hand to the next cake and repeat the count with the right hand, with the oral count continuing up; this is repeated until the left hand has exhausted all cakes.

We can model this in Python with the following program:

def mul(multiplicand, multiplier):
    count = 0
    for i in range(multiplicand):
        for j in range(multiplier):
            count += 1
    return count

Whereas children using repeated addition do something more like this:

def mul(multiplicand, multiplier):
    count = 0
    for i in range(multiplier):
        count = add(count, multiplicand)
    return count

In this case, add is a subroutine. You could define one easily enough in Python or else you could go «from operator import add».

Clearly the second program is more efficient than the first, both as a Python program and as a manual procedure performed by children; it’s more the latter that I’m interested in, I’m using Python to describe the procedures. However, the second procedure requires that add is already adopted as an internal procedure. Of particular note is that, apart from count, the second procedure uses only one state variable, i; the first procedure uses two.

At the stage where multiplication is introduced, many children will not yet be performing addition accurately without counting. Effectively add is not yet available to them as an internal procedure. This is likely to be a problem because this type of learner is unlikely to be able to accurately add the multiplicand to the count without getting confused about where in the multiplication procedure they were.

As an example of what might go wrong, imagine that a learner starts by using the left hand to maintain the i variable in the second procedure (above); this hand will count from 1 to multiplier (hmm, there’s a small sleight of hand going on here, the Python counts from 0 to n-1 whereas most learners will prefer count from 1 to n). The count will be maintained orally (in other words by speaking out the successive multiples of the multiplicand). Begin by raising one finger of the left hand and uttering the multiplicand (the initial count). Now we need to add the multiplicand to the oral count. Maybe the learner can do that without using their fingers, maybe they can’t; in any case depending on the parameters chosen, at some point some learners may need to use their hands to perform the addition. So then the procedure for addition kicks in as the learner adds the multiplicand to the current count. Many learners will have personal procedure for addition that requires both hands. The addition may be performed accurately, but the i state variable will be lost. We lose track of where we were in the multiply procedure.

If the learner can accurately add the multiplicand to the count mentally, then they stand a much better chance of performing the second procedure. This is what I mean by having add available as an internal procedure.

The first procedure can be thought of as a way of simultaneously arranging to perform the multiplication and the additions required by the multiplication without having any state variables overlap. Thereby minimising the chance that confusion will result. Most learners will be capable of keeping track of the required state variables to perform the multiplication, but if left to their own devices may choose methods where the state variables overlap (in other words, they run out of hands). Thus, they can benefit by being guided towards a procedure which they can manage.

Another way to think about this is that at the sort of age where children begin to learn to multiply, their procedure for addition is leaky. It is not fully abstracted; performing addition may require the use of hands (for state variables), making the addition leak over any other use of the same hands to maintain state.

It seems to me that only when a learner can perform a typical count + multiplicand addition accurately in their head are they ready to perform multiplication as repeated addition.

Oh yeah, the original research on the whole “multiplication is/ain’t repeated addition” debate. It sucks. They test children at times t0 and t1 with a randomly chosen math related intervention in between. It seems to me that a more carefully designed study should have also included a non math-related intervention such as giving the subjects fish-oil pills or teaching them to conga. After all, if I was being tested on one day and then told that I was going to sit a similar test tomorrow, I would bone up on the material before the second test, regardless of what I was being taught. Wouldn’t you? They make no attempt to account for this effect.

Appendix for Python hackers

The first definition of mul that I give is of course completely worthless as a practical implementation in Python. However, note the following: «10*10L» is 100L but «mul(10,10L)» is 100; in other words mul returns an int if it possibly can.

Ofsted: satisfactory doublethink


Maybe you’ve read the BBC article “how maths teaching is not good enough”? Perhaps you should read the Oftsted report. Perhaps I should.

41% of the maths teaching (in secondary schools) is satisfactory. The tone of the news article is that this is not good enough.

This is characterised by the section headline in the Ofsted report (section 26): «What is not good enough about ‘satisfactory’ teaching?»

I have news for Ofsted. “satisfactory” means almost the same thing as “good enough”. If you’re not satisfied with “satisfactory” teaching, then you set your assessment criteria incorrectly. How unsatisfactory.

BBC: remove errors bars for better headline


In this article from the BBC Richard Black claims “This year appears set to be the coolest globally this century”. There is no basis for this claim, and moreover the very notion of picking warmest and coolest years amounts to bickering about global warming.

Black appears to be making this claim on the basis of looking at column 2 of the HadCRUT data. Here’s a graph, freshly minted from the Google Chart API:

The data is taken from HadCRUT, here’s a relevant extract:

2000  0.238  0.249  0.227  0.333  0.144  0.238  0.233  0.334  0.143  0.334  0.143
2001  0.400  0.411  0.388  0.495  0.304  0.400  0.394  0.495  0.304  0.495  0.303
2002  0.455  0.466  0.445  0.553  0.358  0.455  0.450  0.553  0.358  0.553  0.357
2003  0.457  0.468  0.447  0.556  0.359  0.457  0.452  0.556  0.358  0.556  0.358
2004  0.432  0.444  0.421  0.530  0.335  0.432  0.426  0.530  0.334  0.530  0.334
2005  0.479  0.490  0.469  0.580  0.378  0.479  0.473  0.581  0.378  0.581  0.378
2006  0.422  0.432  0.412  0.517  0.327  0.422  0.416  0.518  0.326  0.518  0.326
2007  0.404  0.414  0.394  0.501  0.307  0.404  0.398  0.502  0.307  0.502  0.307
2008  0.281  0.292  0.270  0.428  0.134  0.281  0.275  0.429  0.134  0.429  0.134

The format of the data is described here, by Hadley.

In the graph the red line is the best estimate, the pink lines shows the combined 95% uncertainty from all sources. You can get more, or possibly just different, graphs from Hadley.

The first thing to notice is that Black’s claim is false if you include the year 2000. Okay so technically I know that “this century” starts in XX01 but I also know we all celebrated the beginning of the millennium in 2000 and we accepted then that although 2001 was technically the beginning of the millennium (and hence the century) it was much hipper to celebrate 2000. So that deserves a mention at least.

But really my gripe is about not observing the error bars. The uncertainty in the data is such that the error bars all overlap! The data does not support the claim that 2008 is warmer than 2005 for example; if we take as our null hypothesis that these two years are the same temperature then we cannot reject it with any confidence. The same is true about any other pair of years (except possibly for 2005 versus 2000; we might be able to claim that 2005 was warmer than 2000).

Neglecting 2000, as Black obviously does, the data are consistent with a constant anomaly of +0.4°C. That’s just an example, many other temperature series would be consistent with the data, including ones which make 2005 the coolest year.

And that’s the problem with trying to “rank” years. The uncertainties in the data are all so large compared to the yearly changes that it’s totally meaningless to talk about the warmest year or the coolest year. We just don’t know.

Of course, if Richard Black had thought about the uncertainties in the data then he would’ve had to say “latest HadCRUT data shows 2008 about as warm as any other year this century”, and that’s not a very controversial thing to say. All this dramatic concentration on the yearly, monthly, daily ups and downs of global temperatures, greenhouse gas levels, what-have-you is nonsense. It’s just talking about the weather while the planet burns.

The trouble with matrix multiplication


is that it’s not commutative. That means that when the SVG spec says in the documentation for the multiply method in the SVGMatrix DOM “this matrix is post-multiplied by another matrix” it really does matter which way round you do the multiplication of the two matrices.

In the call «A.multiply(B)», A is this and should be post-multiplied by B. In the usual mathematical convention the result should be AB.

Safari computes BA, Firefox computes AB. Oopsy.

This is a confusing area; when I was first getting the turtle motion in Curly Logo going (on Firefox initially) I remember a bit where the turtle’s motion was wrong, and I realised that I had some matrix multiplication the wrong way round. Happens all the time regardless of how much thinking I do beforehand about which way round matrices ought to be multiplied, so I thought nothing of it and simply swopped my arguments. As I was writing the first draft of this article I thought Firefox had it wrong and Safari had it right, turns out that’s not right, it’s the other way round.

Regardless of who is right, anyone wanting to use the routine has to do something to work around it.

So here’s what I do. I perform a trial matrix multiplication using SVGMatrix.multiply and determine whether the multiplication was performed the correct way round or the wrong way round.

Consider the matrices L, a flip, and R, a projection:

L = \begin{pmatrix}0&1&0\cr 1&0&0\cr 0&0&1\end{pmatrix}, R = \begin{pmatrix}0&0&1\cr 0&0&0\cr 0&0&1\end{pmatrix}

LR, that is «L.multiply(R)» as it should be:

LR = \begin{pmatrix}0&0&0\cr 0&0&1\cr 0&0&1\end{pmatrix}

What happens if the implementation swops its arguments:

RL = \begin{pmatrix}0&0&1\cr 0&0&0\cr 0&0&1\end{pmatrix}

In JavaScript (well, SVG DOM really) the entries of a matrix are assigned to properties like this:

\begin{pmatrix}a&c&e\cr b&d&f\cr 0&0&1\end{pmatrix}

Note that the two result matrices (above) only differ in their e and f entries. Specifically when «L.multiply(R)» is computed correctly (following the specification) the result’s e property is 0; when computed with the matrices swopped (incorrectly) the result’s e property is 1. How convenient.

Here is some JavaScript to detect if SVGMatrix.multiply is wrong:

matmulwrong = function() {
  var g = document.createElementNS('', 'g')
    'matrix(0 1 1 0 0 0) translate(1 0)')
  var l = g.transform.baseVal.getItem(0).matrix
  var r = g.transform.baseVal.getItem(1).matrix
  return l.multiply(r).e

There is a lot of faffing around creating a pointless g element just so I can attach two matrices to it. That really was the most convenient way I found to create two matrices.

We can use matmulwrong to select one of two definitions for our own matmul function to multiply two matrices:

// matmul(A, B) computes AB
matmul = matmulwrong() ?
    function(a, b) { return b.multiply(a) } :
    function(a, b) { return a.multiply(b) }

Now we should only use matmul and avoid using SVGMatrix.multiply. Defining it like this means we decide once, when the JavaScript is loaded, and not every time that we call matmul. Notice how this simply tests whether SVGMatrix.multiply is swopped or not and acts accordingly; it doesn’t care whether you’re using Safari or Firefox, it’s more in the spirit of object detection, not browser sniffing. As soon as bug 16062 gets fixed in FirefoxSafari the code will detect it as being fixed. Life is good again and Curly Logo works on both browsers.

PS: Opera agrees with Firefox.
PPS: Guess which WordPress feature I used today that I thought I’d never find a genuine use for.

Python: poor printing of floating point


See how Python prints a simple decimal number:

>>> 94.8

[edit on 2010-01-26: allegedly this is all fixed in Python 3.1. Please skip article if that’s okay with you]

In an attempt to dispell comments on the lines of “floating point doesn’t produce exact answers”, I should point out that I know:

• Floating point is tricky; and,
• 94.8 is not exactly representable as a floating point number.

Without delving into the arcane details of floating point let’s lay out some basics. Each floating point value is some exact real number, in fact a rational number. I’m ignoring NaNs and infinities because they’re not important for this discussion. There are only finitely many floating point values.

With regards to input and output of floating point numbers what is reasonable?

Reasonable input requirement

Every decimal number has some floating point number that is nearest (possibly two if it lies midway between two adjacent floating point numbers). It seems reasonable to me that when converting decimal numbers to floating point numbers (as Python does when it reads 94.8 in the program) it should pick the nearest floating point number to represent it internally (and if there are two such nearest, then pick one). (For very large magnitude numbers you might want to return infinity or throw an exception; doesn’t matter for this article.)

Reasonable output requirement

When converting a number from internal floating point format to decimal (as Python does when printing) it seems reasonable to print one of the decimal numbers whose nearest floating point number is the one we are printing. In other words we would like to print a number that when read according to the reasonable input requirement (above) we get the same internal floating point number. Note that there are infinitely many decimal numbers with a given nearest floating point number (because there are only finitely many floating point numbers); once you’ve printed enough decimal digits to zoom in on the desired floating point number you can keep on printing whatever digits you like, generating lots of different decimal numbers all of which share the same nearest floating point number.

Python certainly meets our reasonable output requirement. 94.799999999999997 and 94.8 share the same nearest floating point number.

Desirable output requirement

Consider the two number 94.799999999999997 and 94.8. By our reasonable output requirement above, they’re both good numbers to print out, but the shorter one is easier to read and think about for humans. It also satisfies an elegant principle: Don’t print out any more digits than you need to.

So a desirable requirement for printing out numbers is that we print out a number that meets the reasonable requirement, and is the shortest possible of all such numbers.

If you need a bit of persuading that shorter is better, then this argument might help you: Consider π (pi). An approximation to 18 decimal places is 3.141592653589793238. Now imagine you have written a Python program to compute π as accurately as it can. It prints 3.1415926535897931. That last digit is wrong. It’s not so much wrong as unnecessary, 3.141592653589793 represents the same floating point number:

>>> 3.141592653589793 == 3.1415926535897931

By printing that extra garbage digit Python is misleading you as to how many digits of precision it has.

Can we be desirable?

So how easy is it to generate reasonable output and desirable output? Well it turns out to be a bit trickier than you might think on first blush, and it requires bignums. On the other hand, this stuff was all sorted out about 30 years ago, and every reasonable machine is large enough to do bignum arithmetic without busting a gut. Unfortunately the gurus Steele and White didn’t publish the algorithm until 1990 (see [SW1990]). To quote them: “Why wasn’t it published earlier? It just didn’t seem like that big a deal; but we were wrong.” Boy were they wrong. Even 17 years after publication, modern implementations of modern languages running on modern hardware still get it wrong.

[Political rant: I’m inclined to think that one the contributory factors is that the article was published by the (“we love dead trees”) ACM. It just doesn’t seem helpful to lock all this knowledge away and charge for it. I happen to have a photocopied version of this article on paper (I guess that makes me lucky, and able to write this blog article), but in 2007 I can hardly believe that I have to resort to paper copies to do research.]

Bottom line: 94.8 should print out as 94.8.

Python’s excuse will no doubt be that it relies on the platform’s C library. C has an excuse. C targets tiny platforms where it would be ridiculous to suggest that the printing routines use 320 bytes of RAM to store intermediate computations (on a platform I recently had the pleasure to program, this would’ve been about 10% of the entire RAM). On the other hand any platform where I’m actually likely to use double I almost certainly have enough space for both for code and the bignum arithmetic it requires to get the right answer. Certainly, on my MacBook the C implementation has no excuse.

It’s no coincidence that Steele had a hand in writing the 1990 paper and also a hand in designing two of the languages that actually bother to say something sensible about converting internal numbers to external format. Scheme says that reading what you print will yield the same number, and this shall be done using the minimum number of digits. Java says “as many, but only as many, more digits as are needed to uniquely distinguish the argument value from adjacent values of type double”. Curiously, Common Lisp doesn’t seem to actually state anything like that, but possibly I just can’t find it. Common Lisp implementations, of course, get it right, because they’re invested with the spirit of Do The Right Thing. I had a look to see what Haskell said on the matter, but apart from noting with amusement that a literal number did not denote an internal number object I couldn’t find anything. ECMAScript also specifies the Right Thing, but then, Wikipedia reckons that Steele edited the first edition.

Python has automatic memory management, bignums, strings. It feels like a big language, abstracted and distant from the silicon. It’s time for Python to grow up and specify that repr for float returns a decimal string the mathematical value of which is closer to the argument than any other float, and it returns a string with the fewest number of digits.

The trickiest thing about Steele and White’s algorithm is the bignums. Python has bignums. It turns out to be very easy indeed to implement Steele and White’s algorithm in Python. We can then use this algorithm to create conversion routines in the style of ECMAScript or Java. And I have done so. The implementation isn’t very large, but it is too large to contain in the margins of this article. If you want the code then go to, yet another dead before it began sourceforge project, flopri.

Appendix A – Other Stuff I Found On The Way

Comparison with other languages

Compare Python with similar codes in shell, awk, C, and Lisp:

$ printf ‘%.20f\n’ 94.8
$ awk ‘BEGIN{printf”%.20f\n”,94.8}’
$ echo ‘#include <stdio.h>
int main(void){printf(“%.20f\n”,94.8);}’ > t.c
$ cc t.c; ./a.out
$ sbcl –noinform –eval ‘(progn(format t “~,20f~%” 94.8d0)(sb-ext:quit))’

Bit of a mixed bag isn’t it? At least Lisp gets it right.

Blind Alleys

Python has a module called fpformat which sounds like it might be the sort of place to find optimal floating point formatting routines, but its useless. “unneeded” as its own documentation avers. pprint is no good either; pretty, but no smarts.

Python’s Tutorial

The Python Tutorial, good in many ways, has what I think is a very weak appendix on this matter: Appendix B – Floating Point Arithmetic: Issues and Limitations (warning: fragile looking URL). It reads more like an apology than justification and varies between patronising, annoying, wrong, and helpful. It contains mistakes like:

Stop at any finite number of bits, and you get an approximation. This is why you see things like:
>>> 0.1

As discussed above, 0.1 is a perfectly good approximation to the floating point value that 0.1 becomes on input, so approximation itself is not the reason that Python prints so many digits (don’t believe me, then try evaluating «0.1 == 0.10000000000000001»). The appendix goes on to recommend str if you want prettier output. I think this is the wrong fix for the problem; if Python used a different printing routine for repr then people wouldn’t be driven to str.

“You’ll see the same kind of thing in all languages that support your hardware’s floating-point arithmetic”. No I won’t. Java, ECMAScript, Lisp all get it right.


[SW1990] “How to Print Floating Point Numbers Accurately”; Guy L. Steele Jr., Jon L White; ACM SIGPLAN’90 Conference on Programming Language Design and Implementation; 1990-06

Learning my multiplication tables in J—again


I like learning new programming languages. It’s like a whole new way of thinking. I thought I’d have another look at J (an ASCII descendent of APL). APL and J clearly enshrine a way of thinking that is unusual amongst programming languages, and that alone makes them worth learning. I last looked at J about 10 years ago. My head exploded. This time the tutorials seem to have a tiny bit more explanation to them.

I learnt something that I wasn’t expecting (invaluable). I learnt that if you take two single digit numbers and multiple them and take the last digit of the answer, then you get the same last digit if you subtract each of the starting number from 10 and do the same thing. 3 × 4 = 12; 7 × 6 = 42. Note that 7 is 3 from 10 and 6 is 4 from 10; we could say that 7 and 6 were the complements in 10 of 3 and 4.

Learning this new thing happened as I was bashing away at the J prompt. One of the examples that comes with J produces an addition table with code like this: (i.5) +/ (i.5). And you can produce a coloured quilt pattern with viewmat (i.5) +/ (i.5) (or at least you can once you’ve remembered to type «require ‘graph’» first):


So I thought that a multiplication table mod 10 would be a good idea. It shows the last digit of a multiplication. The code for that is easy: viewmat 10 | (i.11) */ (i.11):


The first thing I noticed was that there was a reflection symmetry along the main-diagonal. That’s easy to explain, multiplication is commutative so x × y = y × x.

The next thing I noticed suprised me. There was a reflection symmetry along the other diagonal (the counter-diagonal). I wasn’t expecting that at all. I thought about this for a bit. So what I seem to be seeing is that:

a ⊗ b = (10 – a) ⊗ (10 – b)

where ⊗ denotes multiplication in the ring of integers modulo 10. Well that’s easy to see with a bit of algebra:

(10 – a) ⊗ (10 – b) = (-a) ⊗ (-b) = a ⊗ b

or without using a ring:

(10 – a) × (10 – b) | 10 = 100 – 10×a – 10×b + a×b | 10

by multiplying out; and we can see that 100 and 10×a and 10×b are all 0 | 10, so the right hand side reduces to:

a×b | 10

Another way of thinking about this is that adding 10 to a number obviously isn’t going to affect the last digit of a multiplication. Nor is taking away 10. So subtracting a number from 10 is just like negating it; and negating both numbers obviously doesn’t change the answer when you multiply them together. We can see this repeating pattern in J with viewmat 10 | (i:10) */ (i:10) which gives the table for integers from -10 to 10:


Aside: I find the above image a bit disturbing to look at. Some part of my brain wants to believe that it has left-right and top-bottom mirror symmetry (which it almost does have) and is fighting with the bit of brain that can see that it doesn’t have mirror symmetry.

A larger repeating version can be seen with the code viewmat 10 | (i:40) */ (i:40):


Further aside: I produced these pictures by saving them as BMP files using savemat_jviewmat_ in J and converting them to much smaller PNG format using the excellent netpbm suite. In the BMP files all the blues got swopped for reds and vice-versa. Someone got confused as to whether the colours were specified RGB or BGR, so I had to channel swop them back again. Of course that’s pretty easy to do using netpbm. Bottom line: BMP sucks.

So the thing that impresses me about J was that I was just playing around with it casually and discovered for myself this little fact about numbers that I hadn’t known before. This is how it should be when we’re using a tool for learning. It reminds me of the way you can discover things about geometry by playing around with Logo. It makes me want to use computers as an exploratory tool for teaching children maths through discovery.

J itself still makes me think my head is about to explode.

If J intrigues you from this taste, perhaps you ought follow along with me: Learning J part I.