Not in 2010

Things I predict we won’t be seeing in 2010:

Perl 6

Latex 3

1:25,000 scale Ordnance Survey vector data made public

the hottest November

iTunes for Linux

reduce in Python 3.x

A Green Party MP (you know, the UK parliament, not some cheap plastic Euro knock-off)

What is IsItHotOrNot?

This is the landing page for the Twitter account IsItHotOrNot. I’ll probably move it to a proper home at some point.

IsItHotOrNot displays the global temperature anomaly, month by month. Currently the data is sourced from GISTEMP, specifically the “Global-mean monthly land-ocean temperature index”. The file, GLB.Ts+dSST.txt, also has annual and seasonal values, but I don’t currently use those.

What does the number mean in “2009 Nov +0.680, hottest November on record”? It’s November’s temperature anomaly (in degrees C). The anomaly for November is the difference between this November and the average November temperature between 1951 and 1980, the base or reference period. This year’s November was 0.68 degrees C warmer than the average between 1951 and 1980. Obviously the anomalies for other months are calculated in a similar way. Note that this method takes account of the fact that Novembers are, on the whole, colder than Junes, by using the average November temperature between 1951 and 1980, not the average temperature. In other words it does seasonal correction.

It would be nice to say “about 1.0 degrees C” warmer than pre-industrial levels, like the EEA do, and I can do that by simply adding 0.3 to all the values, but since the errors for the 1850 to 1899 (which is what is usually used when you see the phrase “pre-industrial”) average are larger (on account of having fewer records), I decided not to do that. You can wing the error and add 0.3 to all the values if you like.

There are other temperature indexes available: NCDC do one; there’s the HadCRUT3 diagnostics; the Japanese Metereological Agency have their global average surface temperature anomalies. The different indexes use different reference periods (NCDC use 1900 to 1999, HadCRUT3 use 1961 to 1990, JMA use 1971 to 2000), the only effect of which is to move the graphs up and down the page (it does mean that if you want to compare the time series from two different providers, you need to rebase them to give them a common reference period).

Where does the data come from? Well, from GISTEMP, but the land and ocean index is a data product synthesized from other sources:GHCN (land), USHCN (USA land), HadSST2 (ocean), Reynolds SST (ocean), SCAR READER (antarctic land). The other indexes use more or less the same data, the differences are mostly in how it is processed.

Why GISTEMP? All the indexes are basically the same, so it doesn’t really matter which one I pick. Because of my work on Clear Climate Code I happened to be familiar with their datasets, so that was simpler from a programming perspective. The differences between the indexes are typically very small, but it does mean that the statements like: “second hottest September on record” are very sensitive to the exact dataset used. The JMA had September 2009 tying for first place, whereas GISTEMP have it second. I regard the “hottest November on record” stuff as nothing more than a popular discussion point, not an absolute truth.

Java Implementation of Lua Language

Just a quick note for those that don’t follow me on twitter:

I have recently announced the open sourcing of Jill: Java Implementation of Lua Language. It’s a little thing I (mostly) wrote 3 years ago, and it’s taken far too long to get it open sourced, but it’s open now.

No doubt it will prompt me to write one or two blog posts on JME type things.

Python: slicing with zip

Wherein I feel compelled to write some more on Python code that I find more amusing than clear.

The more I use zip the more I love it. I’m thinking about writing a tutorial on how to (ab-) use zip, but for now just this recent discovery.

Say you have two iterators that each yield a stream of objects, iland and iocean (they could be gridded temperature values, say), and you want to get the first 100 values from each iterator to do some processing, whilst not consuming any more than 100 values. You can’t go list(iland)[:100] because that will consume the entire iland iterator and you’ll never be able to get those values past the 100th again.

You can use itertools (probably my second favourite Python module):

land100 = list(itertools.islice(iland, 100))
ocean100 = list(itertools.islice(iocean, 100))

It seems a shame to mention islice and 100 twice. One could use map with a quick pack and unpack, but this is not clear:

land100,ocean100 = map(lambda i: list(itertools.islice(i, 100)), (iland,iocean))

(a simple form of this, which I do sometimes use, is x,y = map(int, (x,y)))

What about giving some love to zip? It turns out that zip will stop consuming as soon as any argument is exhausted. So

zip(range(100), iland, iocean)

returns a list of 100 triples, each triple having an index (the integer from 0 to 99 from the range() list), a value from the iland iterator, and a value from the iocean iterator. And as soon as the list produced by range(100) is exhausted it stops consuming from iland and iocean, so their subsequent values can be consumed by other parts of the program.

And yes, this seems to work by relying on a rather implementation specific feature of zip that I’m not sure should be set in stone.

That zip form above is all very good if one wants to go for n,land,ocean in ..., but what if we want the 100 land values and 100 ocean values each in their own list (like the code at the beginning of the article)? We can use zip again!

_,land100,ocean100 = zip(*zip(range(100), iland, iocean))

zip(*thing) turns a list of triples into a triple of lists, which is then destructured into the 3 variables _ (a classic dummy), land100, and ocean100.

Don’t worry, the actual code use the islice form from the first box because I think it’s the clearest.

Carbon into Trees

The BBC report that the Forestry Commission want to afforest 4% of the UK. And thereby get us 10% of the way towards our 80% emissions reduction target. Their wording is slightly odd, but see paragraph 12:

It is hoped the latest plan would absorb 10% of the UK’s target of slashing its emissions of greenhouse gases by 80% by 2050.

Alarm bells ringing. 1 million hectares (4% of the UK land) can sequester 8% (10% of an 80% emissions reduction) of the UK’s current CO₂ emissions? No. My earlier article on coppicing willow suggests that an optimistic estimate for sequestration is 18 tonnes CO₂ per hectare. So with 4% of the UK land, we could sequester 18 million tonnes, or about 3% of our (600 million tonnes of) emissions. I think my 3% figure is a really top end estimate. It’s not like willow grows particularly well in this country (but it is one of the best crops for sequestration) and with 4% of the UK covered, we may have to afforest some sub-optimal sites; short rotation coppicing is also different from growing mature forest, but I have a hard time believing that growing mature forest pulls down more carbon (yeah yeah, soil, nitrogen).

So where do the Forestry Commission get 8% from? I have no idea. And as usual the clueless journalists at the BBC fail to use the power of hyperlinking (welcome to the 1990’s) and they don’t have a link to the Forestry Commission research. Or even their press release (I suppose that would let everyone know they copied their homework).

Oh wait, here’s the first paragraph of the Forestry Commision press release: (ewgh Lotus Notes)

If an extra four per cent of the United Kingdom’s land were planted with new woodland over the next 40 years, it could be locking up ten per cent of the nation’s predicted greenhouse gas emissions by the 2050s.

Oh. So they mean 10% of our 2050 emissions. Which, as you know, are going to be 80% less than our current emissions. So 10% of 20% of our current emissions. Or 2%. Yeah, I buy that (just about, but at least it’s biologically plausible).

So the BBC mangled the press release. Does the BBC version seem very unclear to anyone else?

Minus times a minus is a plus

My response to the blog wars about multiplying negative numbers. Mostly inspired by Eric’s comment on Mike Croucher’s Walking Randomly.

Big image, links to a PDF (of vector goodness).

I wanted to put the Inkscsape SVG source inside the PNG image. But it turns out wordpress.com “optimises” the image and means my klever hack doesn’t work. Bad wordpress.com.

Natural History Museum: Butterflies

On Friday popped into the Natural History Museum and went to the Butterfly Jungle. I’m a member, and entry to the for-money exhibitions is free (already paid for). It makes me feel terribly middle class.

Before entering into the “jungle” (it’s a temporary hut made of out polytunnel) I thought we could go see the insect gallery so we could learn something about butterflies before seeing them. Well, there is no insect gallery, there is the creepy crawlies room. Where’s the long room full of display cabinets crammed with dead insects pinned to neatly labelled pieces of cardboard? Needless to say the creepy crawly room sucks.

So we sort of wandered about at random. Hey, did you know the toilets have bacteria zapping UV on the hand dryers. Cool. But no interpretation. Not Cool.

Tania Kovat’s TREE is very good. It’s a slice of a 200 year old oak set into plaster panels in the ceiling; the pieces are arranged more or less how they would have been on the tree, in other words: in the shape of a tree. I find the connexion to Darwin a bit lame. The inspiration is Darwin’s now famous “I think” cladogram from his Transmutation Notebook B (ain’t the Darwin online project great?). The cladogram, you know, looks like a tree. And so does Tania’s TREE. Cunning. TREE is displayed in a rather nice gallery at the top of the splendid staircase in the Central Hall. Behind the statue of Darwin, and between the statues of Hooker and Owen. A holy place.

In the same gallery is Ida, apparently the world’s most complete fossil primate specimen. She’s a beautiful little bush-baby-like creature, Darwinius masillae. She lived 47 million years ago. Of course, I know the vast majority of species (well over 99%) become extinct, so it is, statistically speaking, unlikely that Ida is our ancestor. Nonetheless it is difficult to dispel the romantic notion that Ida could be our ancestor. Certainly she will have shared a lot in common, looks, behaviour, social grouping, with our actual ancestors. Ida’s cabinet featured something that I think the NHM should have a lot lot more of. A cladogram.

After wandering past the primate gallery (now quite aging) and the Sequoiadendron giganteum we found the entrance to the Minerals collection. I didn’t actually know the NHM did rocks. And this is awesome. A big gallery full of oak cabinets (original 1881!), stuffed full of… rocks! We didn’t want to spend much time here (we were getting hungry), but I thought it would be interesting to see what the NHM had to say about the alexandrite effect and birefringence. It was a simple pleasure to use the alphabetic mineral index to find the cabinet displaying alexandrite.

Alexandrite appears to be different colours under different lighting conditions. One colour under natural sunlight, and a different colour under incandescent light. I was slightly disappointed to find that cabinet didn’t have a button to press to illuminate the alexandrite with different lights. Oh well. I suppose every mineral is special in its own way, so I can’t expect every one to have a cute interpretation. Of some local interest to me was spotting the enormous Blue John specimen, about as big as my chest. Blue John is a fluorite variety local to Castleton. Of course, I’ve seen far better examples in the shops in Castleton.

I knew quartz was a birefringent material, so I popped over to the quartz cabinet. No birefringence here. As we were ambling out of the room, I luckily found a fine quartz crystal ball on display in the jewellery cabinet next to a rather fine jade box on loan from the Queen. Gazing into the crystal ball gives the birefringent double image effect (this is deliberate, there is an interpretation sign to explain the effect). Nice, but I think Wikipedia’s image is more impressive.

After lunch and a quick trip round the wildlife garden (impressive use of a small urban space, and I expect it to keep improving; didn’t see the foxes though) we did eventually make it to the Butterfly Jungle.

Which I thought was a bit disappointing. However, there’s something intrinsically delightful about having lots of butterflies flapping about, and it hard not to enjoy that rather pleasant experience. And they are pretty to look at. As for science though, there was precious little to be found (not none, but not a great deal). It wasn’t all butterflies, there was an amusing collection of slightly exotic creaturees in glass cages. Giant african millipede, death’s head cockroach, Charlie the Iguana iguana (who I last saw in the Darwin exhibition!); that sort of thing. And a kiddies playground. Which looked quite good, but no use to me.

The Natural History Museum is such a large museum and with so much on display, I find that it’s impossible to do anything but see a small sample of it in any of visit. I’ve been three times recently and I’ve still only seen a small fraction of what it has to offer. There are still things to discover in the Central Hall: I was pleased to see a Glyptodon that I had missed on my previous visits.

I did learn one thing in the Butterfly Jungle. Butterflies taste with their feet.

Windy isn’t it?

Damn hippies think we can just sprinkle a few wind mills around, and because Europe has “huge wind resources” we’ll be okay.

This silly web article claims that europe’s wind energy potential is “huge”, and “equivalent to almost 20 times energy demand in 2020″.

O RLY?

YA RLY, according to the European Environment Agency’s report, Europe’s onshore and offshore wind energy potential.

O RLY?

YA RLY: It’s hard to miss this sentence from the executive summary: “Europe’s raw wind energy potential is huge. … it may be equivalent to almost 20 times energy demand in 2020″.

“energy demand”, that’s the problem. Their assumed energy demand is between 3537 TWh and 4078 TWh. (By the way, notice that the EEA cover their backs with a “may” when they use the lower demand figure to get the “20 times” headline-grabbing numbers, but the web article referencing somehow manages to drop the “may”). So, Europe has 271e6 people (according to Google); that’s 15.3 kWh per person per day. Oops. They must have meant…

Electricity demand.

Twats.

The electricity demand, in Europe, in nothing like our energy demand. In the UK we travel around by burning oil, and we heat our houses and food by burning gas. That hugely swamps our electricity usage.

Energy and Electricity are not the same thing.

Double twats for the people who ignorantly repeated them. Of course the European Environment Agency know the difference. There are two occurrences of the phrase “energy demand” in the document; 7 occurrences of “electricity demand”. Both the “energy demand” phrases related to the “20 times” sentence. One is in it, the other is in the footnote of the table of data on the same page as the “20 times” sentence. Before I did the textual analysis (by which I mean I used the PDF search feature; it’s abysmal, but it’s what I have available) I put the use of “energy demand” down to sloppy practice. Now I think it’s mischievously deliberate. I think they used “energy demand” in that “20 times” sentence in the executive summary because they knew people would make a headline of it.

I have to say that apart from this headline grabbing glitch, the report is well worth reading. Map 6.1 is particularly interesting (apologies for the pixelly rendering, partly their fault, partly mine, but mostly the fault of STOOPID PDFs):

Basically the British Isles is the only place in Europe (not quite, but nearly so) with cheap on-shore wind. And we’re full of NIMBYs.

Screw Hydro!

The Archimedean screw. A venerable machine for lifting water. You can run it in reverse to generate power. How much?

New Mills, where the Sett meets the Goyt, has a community owned Archimedean screw. From their blog the energy generated for each month is:

September: 11108 kWh
October: 25356 kWh
November: 24232 kWh
December: 29513 kWh
January: 19512 kWh
February: 9185 kWh
March: 20330 kWh
April: 3091 kWh
May: 4436 kWh
June: 1389 kWh

Somewhat arbitrarily, but giving them some benefit of the doubt, I’ll replace September’s figure with October’s (perhaps the low September output was mostly teething troubles), and for the missing July and August figures I’ll use May’s.

So the total is: 25356 + 25356 + 24232 + 29513 + 19512 + 9185 + 20330 + 3091 + 4436 + 1389 + 4436 + 4436 = 171272 kWh per year.

or 19.6 kW. This is considerably lower than the 31 kW quoted by one of their investors.

Nice rule of thumb I discovered whilst writing the post: 1 kWh per year is 0.1 W.

The people who built it give it a plate rating of 63 kW (it’s capacity, or maximum power output). So it’s load factor is a little less than 1/3 at 0.31. They also quote a flow rate of 2860 l/s with a drop of 3m. Neglecting the water’s kinetic contribution (which I’m not sure is reasonable), the water has a power of about 86 kW (2860 litres of water is about 28600 Newtons, dropping 3m every second). So the extractive efficiency is about 73%. Quite impressive. I wonder if it can really be that high? Perhaps at high flow rates the kinetic energy is a more useful contribution.

The seasonal nature of the power is clear from the graph:

(the empty bars are missing data, not zero generation)

Basically, you only get power in winter, when it rains. The rest of the load factor gets eaten away by maintenance (oiling, fishing, that sort of thing), high water flow (!) and HSE requests (which I take to mean noise complaints).

David MacKay, in his book “Sustainable Energy – without the hot air” has a cute chapter about hydro. He analyses the total energy of the rain falling on our land and concludes that we can only ever produce about 1.5 kWh per person per day from hydro. After that there’s not much to say, and the chapter is correspondingly short. His figures for actual UK production (page 56) suggest a load factor of 0.29 for large scale hydro, and 0.16 for small scale hydro. So Torrs Hydro is doing atypically well (or I’ve been overly generous in filling the data).

The thing that surprises me is that the Archimedean screw produces a solution that is comparable, in load factor and efficiency, to large scale hydro.

Reviews

When I was a young man in my first job (implementing garbage collectors for dynamic languages) we developed an informal policy of reviewing a paper a week (a paper, as in learned journal, but anything similar would be okay). It was good, I read a lot of interesting stuff, and as a result of writing something about each one, I think some of it may even have stuck.

Of course it was mostly memory management, hardware architecture, and language implementation in those days. Little has changed. My first review is Lambda: the ultimate GOTO.

Let’s hope making it public keeps me regular.