hacking habits
Just a quick note for those that don’t follow me on twitter:
I have recently announced the open sourcing of Jill: Java Implementation of Lua Language. It’s a little thing I (mostly) wrote 3 years ago, and it’s taken far too long to get it open sourced, but it’s open now.
No doubt it will prompt me to write one or two blog posts on JME type things.
Wherein I feel compelled to write some more on Python code that I find more amusing than clear.
The more I use zip the more I love it. I’m thinking about writing a tutorial on how to (ab-) use zip, but for now just this recent discovery.
Say you have two iterators that each yield a stream of objects, iland and iocean (they could be gridded temperature values, say), and you want to get the first 100 values from each iterator to do some processing, whilst not consuming any more than 100 values. You can’t go list(iland)[:100] because that will consume the entire iland iterator and you’ll never be able to get those values past the 100th again.
You can use itertools (probably my second favourite Python module):
land100 = list(itertools.islice(iland, 100)) ocean100 = list(itertools.islice(iocean, 100))
It seems a shame to mention islice and 100 twice. One could use map with a quick pack and unpack, but this is not clear:
land100,ocean100 = map(lambda i: list(itertools.islice(i, 100)), (iland,iocean))
(a simple form of this, which I do sometimes use, is x,y = map(int, (x,y)))
What about giving some love to zip? It turns out that zip will stop consuming as soon as any argument is exhausted. So
zip(range(100), iland, iocean)
returns a list of 100 triples, each triple having an index (the integer from 0 to 99 from the range() list), a value from the iland iterator, and a value from the iocean iterator. And as soon as the list produced by range(100) is exhausted it stops consuming from iland and iocean, so their subsequent values can be consumed by other parts of the program.
And yes, this seems to work by relying on a rather implementation specific feature of zip that I’m not sure should be set in stone.
That zip form above is all very good if one wants to go for n,land,ocean in ..., but what if we want the 100 land values and 100 ocean values each in their own list (like the code at the beginning of the article)? We can use zip again!
_,land100,ocean100 = zip(*zip(range(100), iland, iocean))
zip(*thing) turns a list of triples into a triple of lists, which is then destructured into the 3 variables _ (a classic dummy), land100, and ocean100.
Don’t worry, the actual code use the islice form from the first box because I think it’s the clearest.
The BBC report that the Forestry Commission want to afforest 4% of the UK. And thereby get us 10% of the way towards our 80% emissions reduction target. Their wording is slightly odd, but see paragraph 12:
It is hoped the latest plan would absorb 10% of the UK’s target of slashing its emissions of greenhouse gases by 80% by 2050.
Alarm bells ringing. 1 million hectares (4% of the UK land) can sequester 8% (10% of an 80% emissions reduction) of the UK’s current CO2 emissions? No. My earlier article on coppicing willow suggests that an optimistic estimate for sequestration is 18 tonnes CO2 per hectare. So with 4% of the UK land, we could sequester 18 million tonnes, or about 3% of our (600 million tonnes of) emissions. I think my 3% figure is a really top end estimate. It’s not like willow grows particularly well in this country (but it is one of the best crops for sequestration) and with 4% of the UK covered, we may have to afforest some sub-optimal sites; short rotation coppicing is also different from growing mature forest, but I have a hard time believing that growing mature forest pulls down more carbon (yeah yeah, soil, nitrogen).
So where do the Forestry Commission get 8% from? I have no idea. And as usual the clueless journalists at the BBC fail to use the power of hyperlinking (welcome to the 1990’s) and they don’t have a link to the Forestry Commission research. Or even their press release (I suppose that would let everyone know they copied their homework).
Oh wait, here’s the first paragraph of the Forestry Commision press release: (ewgh Lotus Notes)
If an extra four per cent of the United Kingdom’s land were planted with new woodland over the next 40 years, it could be locking up ten per cent of the nation’s predicted greenhouse gas emissions by the 2050s.
Oh. So they mean 10% of our 2050 emissions. Which, as you know, are going to be 80% less than our current emissions. So 10% of 20% of our current emissions. Or 2%. Yeah, I buy that (just about, but at least it’s biologically plausible).
So the BBC mangled the press release. Does the BBC version seem very unclear to anyone else?
My response to the blog wars about multiplying negative numbers. Mostly inspired by Eric’s comment on Mike Croucher’s Walking Randomly.
Big image, links to a PDF (of vector goodness).
I wanted to put the Inkscsape SVG source inside the PNG image. But it turns out wordpress.com “optimises” the image and means my klever hack doesn’t work. Bad wordpress.com.
On Friday popped into the Natural History Museum and went to the Butterfly Jungle. I’m a member, and entry to the for-money exhibitions is free (already paid for). It makes me feel terribly middle class.
Before entering into the “jungle” (it’s a temporary hut made of out polytunnel) I thought we could go see the insect gallery so we could learn something about butterflies before seeing them. Well, there is no insect gallery, there is the creepy crawlies room. Where’s the long room full of display cabinets crammed with dead insects pinned to neatly labelled pieces of cardboard? Needless to say the creepy crawly room sucks.
So we sort of wandered about at random. Hey, did you know the toilets have bacteria zapping UV on the hand dryers. Cool. But no interpretation. Not Cool.
Tania Kovat’s TREE is very good. It’s a slice of a 200 year old oak set into plaster panels in the ceiling; the pieces are arranged more or less how they would have been on the tree, in other words: in the shape of a tree. I find the connexion to Darwin a bit lame. The inspiration is Darwin’s now famous “I think” cladogram from his Transmutation Notebook B (ain’t the Darwin online project great?). The cladogram, you know, looks like a tree. And so does Tania’s TREE. Cunning. TREE is displayed in a rather nice gallery at the top of the splendid staircase in the Central Hall. Behind the statue of Darwin, and between the statues of Hooker and Owen. A holy place.
In the same gallery is Ida, apparently the world’s most complete fossil primate specimen. She’s a beautiful little bush-baby-like creature, Darwinius masillae. She lived 47 million years ago. Of course, I know the vast majority of species (well over 99%) become extinct, so it is, statistically speaking, unlikely that Ida is our ancestor. Nonetheless it is difficult to dispel the romantic notion that Ida could be our ancestor. Certainly she will have shared a lot in common, looks, behaviour, social grouping, with our actual ancestors. Ida’s cabinet featured something that I think the NHM should have a lot lot more of. A cladogram.
After wandering past the primate gallery (now quite aging) and the Sequoiadendron giganteum we found the entrance to the Minerals collection. I didn’t actually know the NHM did rocks. And this is awesome. A big gallery full of oak cabinets (original 1881!), stuffed full of… rocks! We didn’t want to spend much time here (we were getting hungry), but I thought it would be interesting to see what the NHM had to say about the alexandrite effect and birefringence. It was a simple pleasure to use the alphabetic mineral index to find the cabinet displaying alexandrite.
Alexandrite appears to be different colours under different lighting conditions. One colour under natural sunlight, and a different colour under incandescent light. I was slightly disappointed to find that cabinet didn’t have a button to press to illuminate the alexandrite with different lights. Oh well. I suppose every mineral is special in its own way, so I can’t expect every one to have a cute interpretation. Of some local interest to me was spotting the enormous Blue John specimen, about as big as my chest. Blue John is a fluorite variety local to Castleton. Of course, I’ve seen far better examples in the shops in Castleton.
I knew quartz was a birefringent material, so I popped over to the quartz cabinet. No birefringence here. As we were ambling out of the room, I luckily found a fine quartz crystal ball on display in the jewellery cabinet next to a rather fine jade box on loan from the Queen. Gazing into the crystal ball gives the birefringent double image effect (this is deliberate, there is an interpretation sign to explain the effect). Nice, but I think Wikipedia’s image is more impressive.
After lunch and a quick trip round the wildlife garden (impressive use of a small urban space, and I expect it to keep improving; didn’t see the foxes though) we did eventually make it to the Butterfly Jungle.
Which I thought was a bit disappointing. However, there’s something intrinsically delightful about having lots of butterflies flapping about, and it hard not to enjoy that rather pleasant experience. And they are pretty to look at. As for science though, there was precious little to be found (not none, but not a great deal). It wasn’t all butterflies, there was an amusing collection of slightly exotic creaturees in glass cages. Giant african millipede, death’s head cockroach, Charlie the Iguana iguana (who I last saw in the Darwin exhibition!); that sort of thing. And a kiddies playground. Which looked quite good, but no use to me.
The Natural History Museum is such a large museum and with so much on display, I find that it’s impossible to do anything but see a small sample of it in any of visit. I’ve been three times recently and I’ve still only seen a small fraction of what it has to offer. There are still things to discover in the Central Hall: I was pleased to see a Glyptodon that I had missed on my previous visits.
I did learn one thing in the Butterfly Jungle. Butterflies taste with their feet.
Damn hippies think we can just sprinkle a few wind mills around, and because Europe has “huge wind resources” we’ll be okay.
This silly web article claims that europe’s wind energy potential is “huge”, and “equivalent to almost 20 times energy demand in 2020″.
O RLY?
YA RLY, according to the European Environment Agency’s report, Europe’s onshore and offshore wind energy potential.
O RLY?
YA RLY: It’s hard to miss this sentence from the executive summary: “Europe’s raw wind energy potential is huge. … it may be equivalent to almost 20 times energy demand in 2020″.
“energy demand”, that’s the problem. Their assumed energy demand is between 3537 TWh and 4078 TWh. (By the way, notice that the EEA cover their backs with a “may” when they use the lower demand figure to get the “20 times” headline-grabbing numbers, but the web article referencing somehow manages to drop the “may”). So, Europe has 271e6 people (according to Google); that’s 15.3 kWh per person per day. Oops. They must have meant…
Electricity demand.
Twats.
The electricity demand, in Europe, in nothing like our energy demand. In the UK we travel around by burning oil, and we heat our houses and food by burning gas. That hugely swamps our electricity usage.
Energy and Electricity are not the same thing.
Double twats for the people who ignorantly repeated them. Of course the European Environment Agency know the difference. There are two occurrences of the phrase “energy demand” in the document; 7 occurrences of “electricity demand”. Both the “energy demand” phrases related to the “20 times” sentence. One is in it, the other is in the footnote of the table of data on the same page as the “20 times” sentence. Before I did the textual analysis (by which I mean I used the PDF search feature; it’s abysmal, but it’s what I have available) I put the use of “energy demand” down to sloppy practice. Now I think it’s mischievously deliberate. I think they used “energy demand” in that “20 times” sentence in the executive summary because they knew people would make a headline of it.
I have to say that apart from this headline grabbing glitch, the report is well worth reading. Map 6.1 is particularly interesting (apologies for the pixelly rendering, partly their fault, partly mine, but mostly the fault of STOOPID PDFs):
Basically the British Isles is the only place in Europe (not quite, but nearly so) with cheap on-shore wind. And we’re full of NIMBYs.
The Archimedean screw. A venerable machine for lifting water. You can run it in reverse to generate power. How much?
New Mills, where the Sett meets the Goyt, has a community owned Archimedean screw. From their blog the energy generated for each month is:
September: 11108 kWh
October: 25356 kWh
November: 24232 kWh
December: 29513 kWh
January: 19512 kWh
February: 9185 kWh
March: 20330 kWh
April: 3091 kWh
May: 4436 kWh
June: 1389 kWh
Somewhat arbitrarily, but giving them some benefit of the doubt, I’ll replace September’s figure with October’s (perhaps the low September output was mostly teething troubles), and for the missing July and August figures I’ll use May’s.
So the total is: 25356 + 25356 + 24232 + 29513 + 19512 + 9185 + 20330 + 3091 + 4436 + 1389 + 4436 + 4436 = 171272 kWh per year.
or 19.6 kW. This is considerably lower than the 31 kW quoted by one of their investors.
Nice rule of thumb I discovered whilst writing the post: 1 kWh per year is 0.1 W.
The people who built it give it a plate rating of 63 kW (it’s capacity, or maximum power output). So it’s load factor is a little less than 1/3 at 0.31. They also quote a flow rate of 2860 l/s with a drop of 3m. Neglecting the water’s kinetic contribution (which I’m not sure is reasonable), the water has a power of about 86 kW (2860 litres of water is about 28600 Newtons, dropping 3m every second). So the extractive efficiency is about 73%. Quite impressive. I wonder if it can really be that high? Perhaps at high flow rates the kinetic energy is a more useful contribution.
The seasonal nature of the power is clear from the graph:
(the empty bars are missing data, not zero generation)
Basically, you only get power in winter, when it rains. The rest of the load factor gets eaten away by maintenance (oiling, fishing, that sort of thing), high water flow (!) and HSE requests (which I take to mean noise complaints).
David MacKay, in his book “Sustainable Energy – without the hot air” has a cute chapter about hydro. He analyses the total energy of the rain falling on our land and concludes that we can only ever produce about 1.5 kWh per person per day from hydro. After that there’s not much to say, and the chapter is correspondingly short. His figures for actual UK production (page 56) suggest a load factor of 0.29 for large scale hydro, and 0.16 for small scale hydro. So Torrs Hydro is doing atypically well (or I’ve been overly generous in filling the data).
The thing that surprises me is that the Archimedean screw produces a solution that is comparable, in load factor and efficiency, to large scale hydro.
When I was a young man in my first job (implementing garbage collectors for dynamic languages) we developed an informal policy of reviewing a paper a week (a paper, as in learned journal, but anything similar would be okay). It was good, I read a lot of interesting stuff, and as a result of writing something about each one, I think some of it may even have stuck.
Of course it was mostly memory management, hardware architecture, and language implementation in those days. Little has changed. My first review is Lambda: the ultimate GOTO.
Let’s hope making it public keeps me regular.
Friends of the Earth have sent our household a postcard. It says «The meat and dairy industry produces more climate-changing emissions than all the planes, cars and lorries on the planet.» They don’t quote a study, or any other source. Just a bold assertion which, on the face it, seems implausible. Even if you eat a gargantuan 250 g of meat a day (in other words, the typical US diet; Europeans eat about half that), does that really compare to all that driving round? It also seems a little bit mean to exclude trains and ships on the “transport” side. Is the balance between transport and food really so close that those 2 modes make all the difference? In the UK, rail and water account for about 4% of the total transport energy budget, so I would hope that the question isn’t so close that adding them back in tips the scales the other way. For one thing, any reasonable quantification of errors is bound to swamp that.
I think the FoE statement is false, here’s my homework.
David MacKay stacks up the UK’s energy consumption (Sustainable Energy – Without the Hot Air, Chapter 18, page 103), he has (per person): car 40 kWh/d, plane 30 kWh/d, food 15 kWh/d. So with 70 kWh/d (82 if we add the other transport modes) on the side of transport, and 15 kWh/d on the side of food then it does indeed seem implausible that food chain emissions would be higher. Note that we have all food production on one side, I can’t be bothered separating out meat from the rest, clearly meat forms the bulk of the energy consumption anyway. But wait…
As well as emissions related to the energy required to maintain the animals, they produce carbon-dioxide and methane all by themselves. In other words the food industry has emissions not related to its energy inputs (even if all the energy was produced sustainably, there would still be emissions). Non-energy related emissions show a weakness in David MacKay’s book; he neglects them completely. That’s okay, because his focus is Sustainable Energy, but be aware that it’s not the whole picture. Food, concrete, deforestation all have non-energy emissions. For animals I think we can neglect the CO2 emissions because the carbon originally came from the atmosphere anyway (respiration forms part of a close carbon cycle). Methane however is not negligible.
I reckon 1 kg of lamb produced between 60 g and 180 g of methane when it was walking about in the Peak District. That’s equivalent to about 2.4 kg of CO2. Let’s say I eat 100g of lamb a day. That’s (methane emissions equivalent to) emissions of 240g CO2, or about 1kWh of diesel. That’s roughly 0.1 litres; if you fill up 40 litres (about the size of my small car’s tank) every two weeks then that’s 3 litres a day. How often do you fill up? From a personal perspective, It looks like food-related methane emissions are not even close (to transport emissions).
Okay. So much for the ovine. What about the bovine, porcine, and, er, chickens? Well, I’m no veterinarian so this will take a lot of piecemeal research. Bugger that, lets go to a (competent?) summary: The UK’s Fourth National
Communication under the United Nations Framework Convention On Climate Change. In 2004 UK agriculture (note: not just meat and dairy) emitted 13.8 MtC (megatonnes of carbon equivalent); transport emitted 37.4 MtC. Just what are these Friends of the Earth smoking that makes them think they can claim “The meat and dairy industry produces more climate-changing emissions than all the planes, cars and lorries on the planet” when it is so out of line with the UNFCCC GHG inventory. Is the UK really so atypical?
I suspect that what’s really happening is that the FoE are doing some clever accounting. There’s probably a little bit of double accounting (example, counting transport of feed on both sides), and I suspect some land use change. Perhaps they include chopping down ancient forest to grow soya beans for animal feed as an emission on the food change? I just don’t know, because they don’t show their homework. But I have a couple of points to make anyway. The first is that it’s not at all clear that the beef industry is too blame. If there was less demand for beef (and hence soya beans to feed the cows), then I think it’s likely that the same companies would have chopped down the same forest to grow something else. Miscanthus perhaps. The second is that while this land use change will be an emission (the UNFCCC recognises land use and land use change as a carbon source / sink), this emission occurs only once. Once the forest is cleared to grow soya, there will be no land use change emissions. So the emissions from the single land use change should be amortised over all future soya bean seasons. I think.
So FoE, how do you make the sums add up?
Appendix for the pedantic
«250g of meat a day … the typical US diet»
A quote from USDA Agriculture Factbook 2001-2002, Chapter 2, “Profiling Food Consumption in America”, http://www.usda.gov/factbook/chapter2.htm :
“In 2000, total meat consumption … reached 195 pounds … per person”. That’s 242 g per person per day (2000 was a leap year).
«rail and water account for about 4% of the total transport energy budget»
Department for Transport, TSGB Chapter 3: http://www.dft.gov.uk/pgr/statistics/datatablespublications/energyenvironment/
«1kg of lamb produced between 60g and 180g of methane»:
One 60 kg ewe produces about 20 litres methane a day (see below). Boned and trimmed meat is about 2/3 of the animal’s weight, so 0.5 litres / kg (boned). Lamb is generally defined as less than 12 month’s old or less than 18 month’s old for export. 360 days × 0.5 litres = 180 litres. × (the density of methane gas) 0.717 g/l = 129 g. 60 g to 180 g gives a range around this (to account for younger and older lambs, for one thing).
«one 60 kg ewe produces about 20 litres methane a day»
See Proceedings of the Nutrition Society, Volume 41, page 9A, meeting of 1981-07-17, “Methane production in lambs fed high- and low-roughage diets”. It depends on their diet: about 23 litres for high roughage; about 9 litres for low roughage. Two things: 1) when did you last see sheep being fed lucerne hay? 2) using 20 litres per day favours the FoE case anyway.
«equivalent to about 2.4 kg of CO2»
In terms of greenhouse gas warming potential, per kilo, methane is 20 times more potent than CO2. So 120 g methane equivalent to 2.4 kg CO2.
Returning from EuroPython I got the 1730 from Birmingham New Street to Sheffield. The seat reservations in this class of train appear above each pair of seats on a little illuminated dot matrix display that shows 2 rows of text. Each row displays 16 characters (it was quite tricky to count, but it was close enough to 16 that surely, if there is any god, it must be 16).
Each row displays the seat reservation information for one seat. As a little message that scrolls along. Most of these messages were of the form: “20 This seat is not reserved”. This message is 28 characters long. Which means it needs to scroll to fit on the display. Some genius decided to pad the scrolling message with 16 blanks, so that the end of the message scrolls off completely before the message begins again. So, for the display of this message, there are 44 states that a row can be in. Each state corresponds to a position in the 44 character string (each position in the string can be identified with the display state that has that position at the extreme left-hand end of the display).
The seat number is 2 digits long. It is only displayed for 15 of the 44 states. Meaning it is only visible for 34% of the time. It’s actually kind of important to display the seat number. Especially as they made the mistake of putting the larger of the two seat numbers on the top row. The two rows display the reservations for a pair of seats: N (bottom row), and N+1 (top row). Nuts. In fact it’s not necessary to display the seat number on the display itself. Adjacent to the display (on either side) are stickers showing the numbers for the two seats and whether they are window or aisle. It would be a trivial design change for these stickers to point to the appropriate row of the display.
Some of the time the display will show t reserved or eserved or something else that could reasonably be mistaken for reserved. So if you just glance at the display then you have about a 7% of interpreting it as “reserved” (when in fact it’s not reserved).
The shorter the message is, the more likely we are too be able to comprehend it instantaneously without having to wait for it to scroll. So it would be good to get rid of unnecessary text. “This seat is” is totally unnecessary. We’re on a train, we can tell that the little display refers to a seat reservation. So all it needs to say is “20 not reserved”. And that’s only 15 characters long, so it can be displayed permanently, without needing any anti-assistive scrolling.
The case where the seat is not reserved is a bit special, but it’s worth concentrating on, because those are the seats that people will want to find when they are boarding the train. At least, people without reservations. People with reservations don’t need the overhead displays at all because they can just look at their ticket to find the seat number. To recap: The reservation displays are only useful for people without reservations, so they should be organised around making it clear which seats are free.
The remaining cases, where the seat is reserved for some of the remaining journey, should probably be handled with text that is something like “free until chesterfield” or “reserved until york”. Possibly the “free” and “reserved” can be “stuck” at the left-hand side of the display while the remainder of the display scrolls to show the whole message. Dunno. But I bet a day of trying out a dozen ideas would be a vast usability improvement on how it works now.
