Saturday, 15 February 2020

Visualising ecological footprints

A very short post today on something slightly different. I was discussing the issue of how to visualise a city's ecological footprint with Dan Raven-Ellison last year and I made a very simple interactive map of it. The ecological footprint of an area can be expressed in relation to amount of land required to sustain an area's use of natural resources. It's usually expressed in a hectares per capita way, so for London I took the Greater London population (about 8.8 million) and multiplied it by a conservative value of 4.4ha per capita to get an area of about 38,720,000 hectares or 387,200 sq km (about the size of Japan). Since the shape of Greater London is quite well known, I used this to visualise it, as you can see below.

Black = actual boundary | Pink = ecological footprint equivalent

I did put this on an interactive map, although it doesn't actually do much apart from show you the size in relative terms and the wide area it covers. Click the map for a little bit of info on the area, as in the image below. The land area of the UK is about 242,000 sq km. That's about 94,000 sq miles. Or about the size of Oregon, or half the size of Sweden, or just a bit bigger than Uganda, or twice the size of North Korea.

Note that the figure above is in sq km 

Notes: I have seen a number of different figures per hectare for London, the UK and different places online but I used the number I did so that I could be sure I wasn't over-estimating it. This report on London puts the per capita figure for the city at 6.63ha per head (see p. 11). That would give an area of 488,680 sq km for London's ecological footprint - almost exactly twice the land area of the UK (yep, the size of Turkmenistan, 80,000 sq km larger than California, and a little bit smaller than Spain).

Wednesday, 8 January 2020

Land, people and political maps

This is a final map wrap-up following the UK General Election at the end of 2019, but also a follow-on from my last blog post: Land doesn't vote but it does matter. I'll explain more below, but let's start with a little gif, which fades in and out between the new political map of the UK at the start of 2020 and a different version of the same map, but showing only where there are buildings (in an attempt to scale the data to the underlying population more closely). I've included some interesting facts about people and land below, so do keep reading. Teaser: only 4.4% of the UK land area is Labour constituencies, in contrast to the 32.8% of the population who live in Labour constituencies, which is very close to their 32.2% vote share at the election.

Land matters, but it's good to see both

Here are the individual frames from the gif, below, in case you want to look at them a little more closely. It's a bit of a balancing act deciding upon what line width to use for the buildings-only map - too thick and it's just massive blobs of colour. Too thin and everything disappears, so what you see here is a kind of compromise that is supposed to reflect the pattern of the underlying urban fabric that would be visible on a satellite view, for example.

Buildings file available here

The political map of the UK in 2020

How many people live in areas with a Conservative, Labour, SNP or Lib Dem MP?
This is an interesting question, but not one I came up with by myself. I was asked for an answer to this question, and because I'd compiled all the data already it was a relatively quick bit of analysis to arrive at some answers. So, here we go - below - based on the latest UK mid-year population estimates from 2018.

  • 55.3% of the UK population (36.7 million people) live in areas with a Conservative MP. The Conservatives have 56% of the seats (365 out of 650). The Conservatives won 43.6% of the UK vote in the 2019 General Election.
  • 32.8% of the UK population (21.8 million people) live in areas with a Labour MP. Labour have 31% of all UK seats. Labour won 32.2% of the UK vote in the 2019 General Election.
  • 7.4% of the UK population (4.5 million people) live in areas with an SNP MP. But of course that's a bit of a silly statistic because the SNP only stand in Scotland, obviously. So, the relevant figure here is shown below. The SNP won 3.9% of the UK vote in the 2019 General Election. Note: obviously, the % population and % seat shares will be quite similar owing to the sort-of-equal population per constituency. For Scotland, both figures are 7.4% of the UK in terms of seats and population living there.
  • 82.7% of the Scottish population (4.5 million people) live in areas with an SNP MP. The SNP won 45.0% of the Scottish vote in the 2019 General Election and have 81.4% of all Scottish seats.
  • 1.7% of the UK population (1.0 million people) live in areas with a Liberal Democrat MP. They also have 1.7% of all seats. The Liberal Democrats won 11.5% of the UK vote in the 2019 General Election.

You can see the full spreadsheet here if you like - it includes all parties and has separate tabs for England, Scotland, Wales and Northern Ireland and it has a map of the results. It looks like this (below). I've used total population here rather than electors because that was the question I was given and of course MPs are representative for all people.

More interesting than you may imagine, perhaps

None of this is of course particularly profound or surprising but I'm thinking about it in the context of the maps above and in relation to overall vote share, so I find it interesting. 

How much of the UK land area does each party 'hold'?
Describing this correctly is a bit tricky, but what I mean here is what percentage of the UK's land area does each party 'hold' or 'represent'? That is, what proportion of the new political map of the UK is shaded blue, red, yellow, orange, green and so on? I do like the different kinds of political maps we see these days (including the now-ubiquitous hex cartograms) but I also like to see things mapped in a more traditional manner, so long as we also have a different way of looking at it and are aware of the underlying numbers and settlement pattern (hence the gif at the very top of the page).

Okay, prepare to be blown away, or not, by this geographical trivia.

  • 62.4% of the UK land area is covered by Conservative constituencies.
  • 4.4% of the UK land area is covered by Labour constituencies - yes, 4.4% (but of course that's because they are mostly urban and therefore geographically small, but still this low figure surprised me).
  • 19.5% of the UK land area is covered by SNP constituencies.
  • 60.4% of the Scottish land area is covered by SNP constituencies.
  • 5.6% of the UK land area is covered by Liberal Democrat constituencies (with thanks to Caithness, Sutherland & Easter Ross, clearly).
  • The full spreadsheet above has the rest of the data, including the individual UK country breakdowns.

The mid-2018 population estimates from the ONS put the UK population at about 66.5 million, with 56 million in England, 1.8 million in Northern Ireland, 5.4 million in Scotland and 3.1 million in Wales.

For land area, the UK as a whole is about 244,000 sq km (about the same size as Oregon, or almost exactly the same as the total area of the Great Lakes in North America). England covers 130,000 sq km, Northern Ireland 13,600 sq km, Scotland 79,000 sq km and Wales 21,000 sq km. The figures are in square miles as well as sq km in the spreadsheet.

What was that? You want more gifs, but different speeds and different sizes. Okay then, see below. 

More seriously - and there is a rationale here - switching relatively quickly between the two maps in this way helps highlight the ways in which the standard map view can, if we're not careful, give a distorted view of political representation. That's why I think in political mapping a mix of methods and numbers works best. Also, where we can use different kinds of approaches to explain this (like gifs) we probably ought to.

Fast enough for you? 

A mini version

This is a bigger version - click to zoom


Okay, one last stat. 

What percent of the UK population lives in Constituencies where more than 50% of the votes went to the Conservatives? 

By my calculations the answer to that is 28,258,422, so 42.5% of the UK population which, it so happens, is not so far away from the 43.6% share of the vote. But that is definitely not a defence of first past the post!

Bye for now. 

Tuesday, 3 December 2019

Land doesn't vote, but it does matter

A short post today on the well-work topic of election mapping and, specifically, how to represent the results of elections in a way that reflects the true proportions of who voted for who. This isn't an attempt at a solution or a definitive answer, just some ideas and maps to provoke further thinking on the topic. Let's start by looking at two election maps (below). The one on the left colours in all constituencies according to the party of the MP at the end of the most recent UK parliamentary session. The one on the right is coloured in the same way but only where there are buildings. People like to use these kinds of maps with accompanying comments like 'land doesn't vote' or 'because sheep don't vote' (although I would like to see some polling on how they'd vote if they were so enfranchised). Scroll to the bottom to see the method for this in QGIS.

Standard map vs 'dasymetric' map
Some obvious stuff first... There's a lot of blue on the map to the left above. There is also quite a bit of yellow, and not much red. So, when we look at the map on the left it might give the impression of Tory dominance, when in fact they are not as far ahead as the colour share would suggest and  things are nowhere near as bad for Labour as it might appear from the colour share. Reasons include population density, urban voting patterns, etc, etc. But, but, but, when we get to this point, I always think "yes, true, but the vast majority of people know that". But then again maybe the vast majority of people have more exciting lives and don't think about this kind of stuff at all and end up being inadvertently misled my maps. Perhaps. SO, ENTER THE HEXAGON!!! Yes, it's our old friend, the hex map - example below from a new book. This definitely helps shift the story of the conventional election map from 'wow, the Tories are crushing the competition' to 'it looks quite close'. I quite like this approach, but with only a small quite and probably because of my own biases.

Hex for the win?

Yes, this hex map is better in many respects, but I'm not a huge fan of using these on their own and wouldn't really mind not seeing them again. But I do agree it provides a more useful representation of the vote share and as a compromise it's a fairly good one if we're interested in displaying vote share. I just think in this case a bar chart might be better if we're most interested in vote share, given how hard it is to get the hexagons arranged in a way that matches the underlying geography and what it does to my brain. That's why I've been experimenting recently with the dasymetric map style you see above. Talking of which, here are a few zoomed-in versions, with place labels, and a bit more discussion below.

What colour is your town?

This is a bit more informative I think

But I wouldn't use this approach on its own

Although I do think there is a place for this style

The colour balance is a bit more satisfying here

The last thing to say in this bit is that I also like the buildings-only maps with labels as it allows us to say a bit more about specific places. Let's imagine for a moment that there was a human being out there who didn't know the precise boundaries of the Berwickshire, Roxburgh and Selkirk constituency. I know it's hard to believe but I'm told such people do exist. They know it's a blue constituency and on the first map above they can actually see Selkirk (to the bottom right of the first labelled map) in blue, rather than a giant block of blue covering the entire area. 

Of course, this overlooks the fact that not everyone in the constituency voted for the Conservatives but that's another issue, and in a first-past-the-post electoral system I think shading by winning party is quite logical. Anyway, my point here is really that we can focus in on individual towns and how they voted without being distracted by giant wodges of colour that strictly speaking we do not need. I think the North of England map above does a good job of this - so you can just look at the east coast  of England and see that Bridlington or Scarborough are represented by the Conservatives and that Blackpool has a red wedge between two blue blobs on the west coast.

If you did want to colour areas by the strength of vote for one party then you could use what I call a 'staunchmap' approach, like the example below that I've shared previously. The deeper the colour the higher the vote share - e.g. see Merseyside and its deep Labour reds.

Who will win in 2019? I predict nobody will win.

So, what's my final verdict on how best to map elections? I think a mix of approaches is best, and because most of what we view is on multi-media platforms I think some use of animation might be useful, as in the gif I made already and shared on Twitter. In print or static format this could just be a simple side-by-side version, like the image at the top of the page. Actually, having just written this, I think broadcasters and print media should do both - perhaps standard map + hex + dasymetric or standard map + one other. Not that it matters that much but if we've got the data and tools seems like it would be a good idea to use them here and, maybe, pass on a little more information with this slightly different encoding.

Ooh, fancy - but it does add a bit of value I think

("land doesn't vote, but it does matter" is my usual thought on this, so I'm adding it here, and that's why I'm using a technique that does at least show the land at some point, before switching to a view that more closely reflects the numbers we're probably interested in. Plus, we can't have the Highlands zapped from the map, or at least not permanently.)

Data and method: a cumbersome way to do this would be to use a buildings layer to clip the constituencies but that would take ages. I did this in QGIS using a simple blend mode but it's possible with other tools as well. The data I used are from the spreadsheet I put together and shared on Github previously. You'll also find a constituencies file there to to join it to. You can grab the buildings layer from this page and then you've got all you need to replicate what you see here. Just take a look at the screenshots below to see how I did it. Just make sure your map canvas background is black (via Project > Properties > General).

This is the layer order you need

The constituency colours come from the attribute table

The buildings are all white, with a thin outline

Friday, 8 November 2019

Amazing 3D rendering with Aerialod - a tutorial

One of the more exciting developments in recent years for those of us into geospatial things is the arrival at the end of October 2019 of Aerialod by ephtracy. What am I talking about? I'm talking about being able to create the kind of images you see below in only a few minutes using free software and open data. Scroll past the images for some tips on how to do this, and note that this write up was completed on 8 November 2019 and refers to v0.0.1.

The Cuillin ridge in Skye

Part of the City of Belfast in Northern Ireland

This is indeed a little bit of London

A little circular bit of London

This is also London, for Millwall fans

The Cuillins in Skye, featuring Sgùrr Alasdair

This is Sheffield, around the area where I work

Okay, so how do you get up and running? First of all, go to the Aerialod website and download the package you need. It's Windows 64- or 32-bit only for now and you just download and unzip and then run the .exe to launch Aerialod.

You do get some sample raster data in the zipped download (in the 'map' folder) but if you also download the 'Sample Maps' archive next to the software download button you'll get a central Manchester Lidar png and a Mars (yes, the planet) png. This is a nice reminder that Aerialod is able to handle different formats, including .asc, .png. and .tif for example. I haven't tried any other formats though I think you'll be okay with .jpg too.

They don't look like much here, but wait and see!

When you launch Aerialod you'll see something pretty much like the image below - and it will have that blocky sample layer in there. This is useful for playing around with so you can get to grips with navigation etc. Just note that when you zoom or move around Aerilod may briefly look pixelly/fuzzy as it re-renders, so don't worry about that. It sharpens up perfectly once it's done, although with more complex layers it takes a bit longer.

You may be a bit bewildered at first, but it won't last long

Before I forget, be sure to look in the config folder and open the hotkey.txt file, which I've shown below. That's really useful. But I find mouse navigation easier, so read on. Also note that the second section of hotkey actions below combine a left mouse button click with keyboard actions too.

Hit D and be amazedx

If you are finding the interface too small and can't actually see the icons easily (e.g. if you have a 4k monitor or something like that) then you can use CTRL + or - to scale the UI but you could also just edit this bit of the config.txt file (in the config folder) so instead of 1.0 it says 2.0, like below:

view :
// 0.5 ~ 3.0
ui_scale : '1.0'

Okay, we're all set now so here are the basics of moving things around:

  • Scroll wheel/middle mouse button - you can scroll forwards and backwards to zoom in and out and with the button pressed down you can position the layer wherever you want. 
  • Right mouse button - tilt/pan/rotate etc. Just have a play and you'll see what I mean.
  • And of course with keys, as above, W to zoom in S to zoom out, D to rotate clockwise and A to rotate anti-clockwise.

You loaded the software, figured out navigation with the sample data but now want to render some real world stuff. See below for how to do that.

There are tons of sources for this, including things like NASA's 30 metre SRTM but really it's going to look best with high-resolution DSM or DTM data and for this Lidar is ideal. On the Aerialod page they link to two potential sources of this - the UK's Defra Lidar page where you can download a variety of 25cm, 50cm, 1m and 2m Lidar data for England or get NASA's HiRISE data for elsewhere in the universe.

To get data into Aerialod, the easiest method in my opinion is just to drag and drop a raster file straight in, so that's what I just did with the Manchester.png layer you can see in the screenshot below. To be clear, what you see below is just the result of me dragging and dropping a png file into Aerialod. I haven't done anything else yet.

That was easy!

Before I say more about settings, a word on getting data into Aerialod in other ways. In the top right of the window you'll see a save icon. That will save the raster layer (not the 3D render) to a location of your choice. The next button (the open folder icon) will let you open a new single layer instead of dragging and dropping into the window. The next button (the closed folder icon) is a bit different, but it effectively allows you to stitch together all layers in a single folder, and it's amazing. Just click one item in the folder and it will add them all, as you can see below. The final button (blank page) just starts a new Aerialod blank view.

Stitch multiple files!

Okay, see below for where I've used the open folder button to add a single layer. This is for a single tile (of about 7MB) near the University of Sheffield. In the image below this, I've used the closed folder icon to select the first item and then all layers are added. You don't actually have to select the first item though, just any of the files in the same folder. This is really handy as when you download Lidar data it's more than likely going to be comprised of lots of little chunks as individual files.

Single layer from within a folder

All layers from within a folder

If you get too greedy and try to add a gazillion zigabytes of data, Aerialod may crash. I know, I've tried. Anyway, for the rest of this tutorial I'm mainly going to use the Manchester.png sample file provided on the Aerialod download page - and what you see below is the result of me just dragging and dropping it into the viewer. This covers a good chunk of Manchester city centre and also a bit of Salford and Trafford and to the left of the image below you might just spot Old Trafford - both the football and cricket ground.

Just another sunny day in Manchester

Okay, so the rest of this will cover the main things you need to know but I can't cover everything because a) that would take too long and b) I don't actually know enough to tell you everything and I'm kind of learning as I type here. So the two annotated screenshots are my main contribution for now. Look at the images closely though and you should learn enough to produce great visualisations in not much time at all.

Click on the next two images for a little explainer of what does what and then try it yourself.

This covers the basics, click to expand

Some more tips - including on Focus

Offset - actually quite useful for flood viz
A full screenshot so you can see some of the settings here

That's basically it for now. All that's left to say is don't forget to check the #Arielod hashtag on twitter as well as the @ephtracy Twitter feed. Last of all, don't forget that you can drop all kinds of things into Aerialod, not just terrain models and suchlike. I've experimented with adding in photos of people, which often looks sort of cool but also weird, and I've tried all sorts of other stuff but I'll end with one of the MiniScale raster relief maps of Great Britain which are part of their open data offering. It's not intended to be used this way but I think it looks quite interesting.

This actually turned out alright

Saturday, 2 November 2019

A deprivation by constituency chart

Yesterday I decided to update a little chart I made after the 2017 General Election. It was inspired by a histogram that Owen Boswarva made and the idea was very simple: put England's 533 constituencies into 10 columns, with the most deprived on the left and least deprived on the right, and then colour it by party. The image below is the result. UPDATE: I have now done a full-UK version of this - see below. Also includes an animation and the individual frames which show one party at a time. Read the notes on the UK chart for more information.

Link to slightly higher resolution version

Full size version

Easier to decipher as a gif




Liberal Democrats


Sinn Féin

Plaid Cymru


Independent (at 2017 General Election)


I did this out of curiosity the last time and then after speaking to my colleague Philip Brown about data, elections and suchlike I decided to update and try to improve the older chart, which was informative but a bit of an assault on the eyeballs. I then saw that the House of Commons Library team re-ran their analysis aggregating the English Indices of Deprivation for the 533 English constituencies, so that was all I needed. And, by the way, the House of Commons Library team are in my opinion doing some of the best data curation, manipulation and analysis out there - really great team. 

Of course, the results are hardly surprising but I didn't expect the sorting to be quite so stark. Naturally I tweeted the graphic and lots of people also found it interesting. So, I've done a slightly different version below which has the constituency names in the boxes - but you'll have to click and zoom to read those.

Full-size version here

Zoomed-in extract of the image above

It's not possible to do a full UK-wide one using a single dataset from the same time point, because the deprivation indices for each country of the UK are slightly different and cover different time period but it would be nice to have been able to - and at least in Scotland the colours would be quite spread across the deprivation spectrum. UPDATE: as you can see above, this last statement is true and I've also done the full-UK chart.

I also did one more version of the graphic, this time in a very long single column vertical one, the original of which is here, with a lower resolution one below. The tends to work great on a mobile but needs a bit of zooming in and out in most browsers. I've added the 'required swing %' figure to this one, showing what percent vote shift it would take for a constituency to change hands.

Click to see full size

Looks like this when full width on screen

There wasn't any great agenda or rhyme or reason behind this, I just wanted to see what it looked like and in particular how much the colours would be grouped and where the obvious anomalies were.

Given what I do for a living I'm duty-bound to point out the following obvious but important things:

  • Correlation is not the same as causation (yawn! but true).
  • I don't believe voting Labour makes you poor, or that voting Conservative makes you rich, as some people online seem to have implied - or even that 'the poor' vote Labour or 'the rich' vote Conservative - and I certainly wasn't trying to demonstrate either of these but understand that's sometimes how people see things.
  • There is a reasonably high amount of variation within most constituencies in relation to levels of deprivation, although at the top and bottom of the scale a lot less than you might imagine - I'm currently working on a project all about this kind of thing.
  • Colours - they are the html values for the party colours from the respective party websites.
  • The aggregation method used to derive the constituency rankings could be done a number of different ways, and therefore can produce slightly different results based on how you do it but the order of places would always be broadly the same.

That's all for now.