Archive for July, 2009



When I was a young man in my first job (implementing garbage collectors for dynamic languages) we developed an informal policy of reviewing a paper a week (a paper, as in learned journal, but anything similar would be okay). It was good, I read a lot of interesting stuff, and as a result of writing something about each one, I think some of it may even have stuck.

Of course it was mostly memory management, hardware architecture, and language implementation in those days. Little has changed. My first review is Lambda: the ultimate GOTO.

Let’s hope making it public keeps me regular.

Food Chain Emissions


Friends of the Earth have sent our household a postcard. It says «The meat and dairy industry produces more climate-changing emissions than all the planes, cars and lorries on the planet.» They don’t quote a study, or any other source. Just a bold assertion which, on the face it, seems implausible. Even if you eat a gargantuan 250 g of meat a day (in other words, the typical US diet; Europeans eat about half that), does that really compare to all that driving round? It also seems a little bit mean to exclude trains and ships on the “transport” side. Is the balance between transport and food really so close that those 2 modes make all the difference? In the UK, rail and water account for about 4% of the total transport energy budget, so I would hope that the question isn’t so close that adding them back in tips the scales the other way. For one thing, any reasonable quantification of errors is bound to swamp that.

I think the FoE statement is false, here’s my homework.

David MacKay stacks up the UK’s energy consumption (Sustainable Energy – Without the Hot Air, Chapter 18, page 103), he has (per person): car 40 kWh/d, plane 30 kWh/d, food 15 kWh/d. So with 70 kWh/d (82 if we add the other transport modes) on the side of transport, and 15 kWh/d on the side of food then it does indeed seem implausible that food chain emissions would be higher. Note that we have all food production on one side, I can’t be bothered separating out meat from the rest, clearly meat forms the bulk of the energy consumption anyway. But wait…

As well as emissions related to the energy required to maintain the animals, they produce carbon-dioxide and methane all by themselves. In other words the food industry has emissions not related to its energy inputs (even if all the energy was produced sustainably, there would still be emissions). Non-energy related emissions show a weakness in David MacKay’s book; he neglects them completely. That’s okay, because his focus is Sustainable Energy, but be aware that it’s not the whole picture. Food, concrete, deforestation all have non-energy emissions. For animals I think we can neglect the CO2 emissions because the carbon originally came from the atmosphere anyway (respiration forms part of a close carbon cycle). Methane however is not negligible.

I reckon 1 kg of lamb produced between 60 g and 180 g of methane when it was walking about in the Peak District. That’s equivalent to about 2.4 kg of CO2. Let’s say I eat 100g of lamb a day. That’s (methane emissions equivalent to) emissions of 240g CO2, or about 1kWh of diesel. That’s roughly 0.1 litres; if you fill up 40 litres (about the size of my small car’s tank) every two weeks then that’s 3 litres a day. How often do you fill up? From a personal perspective, It looks like food-related methane emissions are not even close (to transport emissions).

Okay. So much for the ovine. What about the bovine, porcine, and, er, chickens? Well, I’m no veterinarian so this will take a lot of piecemeal research. Bugger that, lets go to a (competent?) summary: The UK’s Fourth National
Communication under the United Nations Framework Convention On Climate Change
. In 2004 UK agriculture (note: not just meat and dairy) emitted 13.8 MtC (megatonnes of carbon equivalent); transport emitted 37.4 MtC. Just what are these Friends of the Earth smoking that makes them think they can claim “The meat and dairy industry produces more climate-changing emissions than all the planes, cars and lorries on the planet” when it is so out of line with the UNFCCC GHG inventory. Is the UK really so atypical?

I suspect that what’s really happening is that the FoE are doing some clever accounting. There’s probably a little bit of double accounting (example, counting transport of feed on both sides), and I suspect some land use change. Perhaps they include chopping down ancient forest to grow soya beans for animal feed as an emission on the food change? I just don’t know, because they don’t show their homework. But I have a couple of points to make anyway. The first is that it’s not at all clear that the beef industry is too blame. If there was less demand for beef (and hence soya beans to feed the cows), then I think it’s likely that the same companies would have chopped down the same forest to grow something else. Miscanthus perhaps. The second is that while this land use change will be an emission (the UNFCCC recognises land use and land use change as a carbon source / sink), this emission occurs only once. Once the forest is cleared to grow soya, there will be no land use change emissions. So the emissions from the single land use change should be amortised over all future soya bean seasons. I think.

So FoE, how do you make the sums add up?

Appendix for the pedantic

«250g of meat a day … the typical US diet»

A quote from USDA Agriculture Factbook 2001-2002, Chapter 2, “Profiling Food Consumption in America”, :

“In 2000, total meat consumption … reached 195 pounds … per person”. That’s 242 g per person per day (2000 was a leap year).

«rail and water account for about 4% of the total transport energy budget»

Department for Transport, TSGB Chapter 3:

«1kg of lamb produced between 60g and 180g of methane»:

One 60 kg ewe produces about 20 litres methane a day (see below). Boned and trimmed meat is about 2/3 of the animal’s weight, so 0.5 litres / kg (boned). Lamb is generally defined as less than 12 month’s old or less than 18 month’s old for export. 360 days × 0.5 litres = 180 litres. × (the density of methane gas) 0.717 g/l = 129 g. 60 g to 180 g gives a range around this (to account for younger and older lambs, for one thing).

«one 60 kg ewe produces about 20 litres methane a day»

See Proceedings of the Nutrition Society, Volume 41, page 9A, meeting of 1981-07-17, “Methane production in lambs fed high- and low-roughage diets”. It depends on their diet: about 23 litres for high roughage; about 9 litres for low roughage. Two things: 1) when did you last see sheep being fed lucerne hay? 2) using 20 litres per day favours the FoE case anyway.

«equivalent to about 2.4 kg of CO2»

In terms of greenhouse gas warming potential, per kilo, methane is 20 times more potent than CO2. So 120 g methane equivalent to 2.4 kg CO2.

Unusable Train Reservations


Returning from EuroPython I got the 1730 from Birmingham New Street to Sheffield. The seat reservations in this class of train appear above each pair of seats on a little illuminated dot matrix display that shows 2 rows of text. Each row displays 16 characters (it was quite tricky to count, but it was close enough to 16 that surely, if there is any god, it must be 16).

Each row displays the seat reservation information for one seat. As a little message that scrolls along. Most of these messages were of the form: “20 This seat is not reserved”. This message is 28 characters long. Which means it needs to scroll to fit on the display. Some genius decided to pad the scrolling message with 16 blanks, so that the end of the message scrolls off completely before the message begins again. So, for the display of this message, there are 44 states that a row can be in. Each state corresponds to a position in the 44 character string (each position in the string can be identified with the display state that has that position at the extreme left-hand end of the display).

The seat number is 2 digits long. It is only displayed for 15 of the 44 states. Meaning it is only visible for 34% of the time. It’s actually kind of important to display the seat number. Especially as they made the mistake of putting the larger of the two seat numbers on the top row. The two rows display the reservations for a pair of seats: N (bottom row), and N+1 (top row). Nuts. In fact it’s not necessary to display the seat number on the display itself. Adjacent to the display (on either side) are stickers showing the numbers for the two seats and whether they are window or aisle. It would be a trivial design change for these stickers to point to the appropriate row of the display.

Some of the time the display will show t reserved or eserved or something else that could reasonably be mistaken for reserved. So if you just glance at the display then you have about a 7% of interpreting it as “reserved” (when in fact it’s not reserved).

The shorter the message is, the more likely we are too be able to comprehend it instantaneously without having to wait for it to scroll. So it would be good to get rid of unnecessary text. “This seat is” is totally unnecessary. We’re on a train, we can tell that the little display refers to a seat reservation. So all it needs to say is “20 not reserved”. And that’s only 15 characters long, so it can be displayed permanently, without needing any anti-assistive scrolling.

The case where the seat is not reserved is a bit special, but it’s worth concentrating on, because those are the seats that people will want to find when they are boarding the train. At least, people without reservations. People with reservations don’t need the overhead displays at all because they can just look at their ticket to find the seat number. To recap: The reservation displays are only useful for people without reservations, so they should be organised around making it clear which seats are free.

The remaining cases, where the seat is reserved for some of the remaining journey, should probably be handled with text that is something like “free until chesterfield” or “reserved until york”. Possibly the “free” and “reserved” can be “stuck” at the left-hand side of the display while the remainder of the display scrolls to show the whole message. Dunno. But I bet a day of trying out a dozen ideas would be a vast usability improvement on how it works now.

After Copper


We gather in the dim space afforded by the overgrowing ash (Fraxinus excelsior) and cow parsley (Anthriscus sylvestris). The trees provide some shelter from the rain, after all, it is July in the Peak District. The key is brought forth. The old iron gate swings open. We step down into a small stream, and through the gate, passing under a stone reading “DEEP ECTON: DRIVEN 1774”. A tunnel arch leads inward, towards the heart of the hillside.

My wellingtons are in muddy water. The pavement beneath, which I cannot see, is uneven. The daylight fades, but my eyes have not yet adapted to the dim torchlight. Stumbling forwards, reaching out for the walls. Tracing my hand along the wall to steady myself, missing my footing as my hand discovers crumbling gaps in the walls. The dwarves that once worked this place left over 100 years ago; they must have been dwarves, for sure they were smaller folk than me, for I keep banging my helmet on the odd out of place stone in the ceiling. We are in Deep Ecton Mine, once the source of the Peak District’s copper.

After a short while, plaster gives way to (limestone) brick. Little nooks appear, presumably once filled with candles. Then the brick gives way to a rough unlined tunnel through the bedrock. The horizontal tunnel we are walking, stumbling, along would have been called an adit by the dwarves. Its upward slope is imperceptible save for the fact that we are walking through a stream; running downwards and outward to rejoin its brethren waters of the Manifold. My eyes are adapted now, and my foot and inner ear give me mostly steady passage.

We pause at the first “chamber”, a swelling of the rough tunnel. A crawl above a pile of rock debris, “deads” as the dwarves called them, leads to a much older, smaller, adit. Its roof has fallen in, now blocked and unsafe. A narrow shaft leads upwards, presumably once connecting to the surface. The surface, and all evidence of the outside world, seems very distant now.

Passages leading off into the gloom. Stepping over discarded ironwork, occasional rocks (fallen from the roof?), and… tramway sleepers. The adit would once have been busy with minecarts. I see a little metal tag marked with “A6” (a modern survey tag). One dwarf has mezzotinted his initials, IB, into the wall with his pick. The whole thing reminds me of, well, what else but Colossal Cave (and damnit, why didn’t I think to try saying “PLUGH”?).

The “IB” chamber has a big rectangular hole in the floor. Full of water. We are on the lowest dry level of the mine (the entrance that we used is only a few meters above the River Manifold). Just around the corner we come to the main attraction. A huge vertical cavern, or pipe. Big enough to contain a house and extending upwards in a complex series of natural shafts, platforms, and other connecting chambers. The house would have to float, for the “floor” of the cavern is a gigantic pool filled with crystal clear water. No plant infiltrates, and no animal stirs the mud. A tiny waterfall chirps and trickles its way down into the pool from some higher cavern. Downwards, through the water, we can see that the pipe continues. Occasionally we can see massive wooden props, essentially whole mature oaks trimmed to a rough rectangle, fitted across where they once would’ve supported platforms. The mine continue downward, as does the pipe, for at least another 300m. But the underwater areas remain unexplored since a diver’s death here in the 1960’s.

This pipe is where the copper was. Formed when the bedrock flexed and cracked, allowing copper carrying water to seep in and deposit its lode. Of course the dwarves took all the copper, but we can see occasional spots where copper has remineralised and formed a greenish colouration on the walls (copper carbonate?). All around we can see places where the bedrock has bent into huge curves, cracked, and the cracks filled with worthless calcite (by contrast, the limestone bedrock nearer the entrace has no calcite veining).

This pipe was formed by hacking out valuable copper ores. The next cavern is 10 metres across and big enough to stand in (and in parts up to about 8m high). It was dug out not for copper, for it never contained any, but to house an engine. It is the engine chamber. Housing engines for pumping water out of the depths. Engines powered not by steam, but by horse, and by water. The amount of effort involved is quite incredible: all that worthless rock removed to create this large room, rivers diverted, engines installed, a vertical 300m oak beam (bolted in sections, naturally). All just to remove water so that the lower sections of the mine could be worked for their copper.

Then the tour was over. We had to make our way to the endgame before our batteries ran out and the cave collapsed (just kidding, another text adventure reference). Whilst we had been underground for quite a while, it didn’t take us long to go back along the adit and reach daylight and the smell of fresh air, a smell you really appreciate when you’ve been underground.

Thanks to the staff of the Peak District National Park who used their own time to give us the opportunity to see the mine and benefit from their experience.

My EuroPython talk: Python Sucks!


On 2009-07-01 I gave a talk at EuroPython called “Python Sucks!”. They made me change the title of the talk, but the first slide sticks, so “Python Sucks!” it is.

It was a bit of a misleading title. As I did actually mention some things that I like about Python.

The slides (updated in blue to add useful things that people said in the talk itself) are available in PDF. I’m not sure the slides are particularly useful without a transcript; it’s not always clear if the point illustrated on the slide is something that I think is a good thing, or a bad thing.

I was a bit overwhelmed with the talk actually. I was thinking that, as I am not as famous as Bruce Eckel or Tony Hoare, about 30 people would turn up; and I think I could probably wrangle about that many people. The Recital Hall holds about 150 people, and it was pretty much full. *gulp*.

The audience included Tony Hoare (Man of Science); when I spotted that I sort of thought “oh no, Tony Hoare is in the house, I’d better behave now”. He usefully (and somewhat embarrassingly on my part) suggested that Occam be added to the cloud of “languages I don’t know enough about”. And it should.

One of the points of the talk was to get the audience talking; I think I did okay at providing a very lively forum for people, not just me, to get their python gripes off their chest. The contributions from members of the audience were well appreciated, and often informative. Certainly I felt that the audience provided a useful contribution, which of course made my job easier.

Later on in his keynote Tony Hoare said something like “from what I’ve seen here today you are doing a good job” [of being scientific engineers, or of steering Python, or something]. I hope he wasn’t referring to me.

Note to self: do not show a slide with “distutils” on it to 150 people. Unless you have nothing to say.

Tony Hoare, man of Science


At EuroPython, on 2009-07-01, Tony Hoare gave a very interesting speech on the opposition between science and engineering (video here). In many ways it reprised the themes of Laura Creighton’s keynote from PyCon UK 2007, but from a science perspective rather than a history of science perspective. In the middle of the talk, some wit twittered “Stand back! Sir Tony Hoare is about to do science!”; shame it wasn’t on the big twitter wall.

Tony Hoare is clearly old skool. His slides had the calm and aged patina of the OHP era, and I thought they were all the better for that. If you have a message, then that message can be conveyed without all the flash and shine that PowerPoint tempts you with (although, being a Microsoft man, of course his slides were in PowerPoint). As Andrew Kuchling says: “(good talk, plain slides) > (bad talk, fancy slides)”

I was particularly impressed with this slide from Tony’s talk outlining a few “special interests” of the scientist and engineer respectively:


On the scales on the side of Science we have things like “long-term”, “perfection”, and “originality”. Balancing the scales for Engineering we have “short-term”, “adequacy”, and “best practice”.

What I liked about this slide is that many of the things on the Science side would be seen as defects in an engineer, and many things on the Engineering side would be seen as defects in a scientist. We have all seen scientists attacked for relying on intuition or merely amalgamating best practice. And what engineer has not been barracked (by their manager) for attempting solutions that were too perfectionist or wasting time on long-term goals?

Tony Hoare’s insights are clearly the product of long and hard work. He seems very optimistic about the possibilities of a virtuous feedback between the engineering and scientific sides of computing. Perhaps we have every right to look forward to the day when “Software will be the most reliable component of every product which contains it.” (the last slide from his talk). But right now… it seems a long way off.

Python: dictionary of functions


On twitter recently I was wondering what the best way to create a dictionary of functions in Python was. On a suggestion of Paul Hankin’s I looked into classes and metaclasses. The most direct way I discovered is to use a class; metaclass not required:

class foo:
  def bar(): pass
  def zon(x): return 1+x
fundict = foo.__dict__

This is direct, but not at all obvious. Hint to people that are mystified already: the __dict__ attribute of the class is a dictionary containing everything defined in the class.

fundict is now a dictionary that contains the two functions bar and zon.

I made this discovery a few days before EuroPython, and was fortunate enough to bump into Bruce Eckel in the hallway at EuroPython just before he gave his metaprogramming talk, so I showed it to him in what I call my “naughty decorator” form:

def asdict(x):
  return x.__dict__

class baz:
  def bar(): pass
  def zon(x): return 1+x

(By the way, one of the things I learnt at EuroPython is that the crappy decorator syntax gives Java refugees a warm fuzzy feeling).

Bruce pronounced this “not as naughty as you imply” and “worth showing” (this blog post was half-written before I bumped into Bruce, but he is definitely encouraging me).

It’s worth showing for the following reason:

The functions you get from the class’s __dict__ dictionary are not the same as the methods you get by accessing the attributes of the class. In other words « is not foo.__dict__['bar']». I was surprised by this, and so was Bruce.

As well being a little weird, it makes a compact example to show the different between a function, an unbound method, and a bound method.

I hope I don’t need to introduce a function. It’s just a thing you call with some arguments. Where it differs from a method is that a method is regarded as a message sent to an object, and receives that object as its first argument.

In Python, an unbound method is a method that isn’t associated with any particular object; it requires an object as its first argument (and Python loses big here, by requiring the object to be an instance of the class that defined the method). A bound method is a method already associated with an object; that object becomes the method’s first argument when the bound method is invoked with the remaining arguments.

So if we define a class foo (as we did, above), then: is an unbound method;

foo().bar is a bound method (bound to the object we just created by invoking the foo class); and,

foo.__dict__[‘bar’] is a function.

This last fact was a great surprise to me. I had expected it to be an unbound method, and thought that my naughty decorator would have to have some hacky code to dig the function out of the unbound method. But it doesn’t.

Tiny problem: Using the asdict gives a dictionary that contains __module__ and __doc__ keys. Solution: another decorator:

def cleandict(x):
  for k in ('__module__', '__doc__'):
    del x[k]
  return x

class baz:
  def bar(): pass
  def zon(x): return 1+x

EuroPython 2009


Overall I had a really enjoyable time, met lots of interesting people, some new and some renewed friendships, and learned some good stuff.

Too many web frameworks. Too many VMs.

What I like is the diverse range of applications to which people put Python. The Guardian use it so we can inspect our MP’s expenses. Gregor Lindl uses it and Papert’s turtle graphics to teach. The Deutsches Zentrum für Luft- und Raumfahrt use it to manage the extraction and shuffling of petabyte (that’s 10**15!) datasets across the supercomputing networks used by climate scientists. The talented Stani Michiels creates the new Dutch Euro coin designs using Python. Weather trading derivatives. Video workflows. Collaborative mathematics. Games. Academic document archives. Billing. System Administration. The list goes on.

Naturally I managed not to go to most of those talks (apart from keynotes (like Tony Hoares) and lightnings, I went to 2 talks). That’s partly because going to 4 or 5 talks a day (which would easily have been possible given the packed schedule) makes you dead tired and causes leaky brain; partly because there were timetable clashes! But mostly because I was writing the materials for my two talks.

Writing your talk at the conference itself is… exciting. And intense. And it probably gives the conference organisers the willies (as in, “where are your slides?”). It did mean that I was able to include some stuff from the conference itself in my talks. A war story I picked up in hallway chat about having to use 6 year old versions of Python on a government IT project made it into my “Loving Old Versions”” talk. Dr Sue Black’s talk about Bletchley Park was in the timetable, and that prompted me to put a reference to Turing’s coffee mug into my “Python Sucks!” talk (Sue Black got there first with the same anecdote; ah well). The “Python Sucks” talk also got a couple of references to Bruce Eckel’s keynote which covered some of the same ground (just using a lot more space, according to Thomas Guest).

But it’s way stressful (Rob Collins’s massage to raise PSF funds was very helpful in that regard). Next year I’ll leave the laptop at home, and give a presentation using a marker pen (well, I will if they accept it!).