Saturday, 11 November 2017

Land Cover Atlas - Notes, FAQs and Data

The last post had some background information on my Land Cover Atlas of the UK. In this post I'll say a bit more about it, answer some of the questions I received, and share some of the data from the project. As I said in my previous post, this was really about me trying to understand the data better and seeing if I could produce a localised atlas of land cover for the whole UK in a way that was accessible and comparable between areas.

Corine Land Cover data for Liverpool and Manchester area

As above, but for Northern Ireland and a bit of Scotland

If you really want to know more about what I did, and the results of my analysis, I strongly recommend you see my more detailed Land Cover Atlas of the UK, including the Appendices. The piece on the BBC was a simplified version of this work, because it had to be. You just can't have giant, detailed maps on the BBC website in the same way you can on Figshare, where the full set of maps are hosted. That's why, after some discussion, we created a simplified four-class Corine map set, with one set of maps optimised for desktop (640 pixels wide) and another set optimised for mobiles (350 pixels wide). My heart did sink when I found out I'd have to do this but then again it was a good way to learn a few more tricks and the BBC graphics and data team are brilliant to work with.

On the underlying classification itself, I decided to use Corine because I had always wanted to understand it better, but it's also open data and comparable across Europe, and over time - it goes back to 1990. This means that, in theory, you could repeat this kind of analysis for the whole of Europe over 20+ years. But don't hold your breath; I'm done with Corine for now. As I note in the Atlas, the categories themselves do require a little bit more investigation if you really want to make sense of things, as you can see in the extract below.

An extract from my Atlas

On the four-fold simplified classification used on the BBC website, there was quite a bit of debate about this. We moved from using 'Artificial' (as Corine uses) to 'Built on' after some testing at the BBC suggested there was confusion about what 'artificial' actually meant. For 'natural', I think Mark Easton's blog piece did a good job of explaining that this must be interpreted with care. 

'Natural' must be interpreted with care

Ultimately, I thought that producing a localised land cover Atlas would be a good way of showing that when it comes to national averages, they don't always make sense locally. That is, the UK might be mostly farmland and semi-natural, but doesn't mean much if you are surrounded by a concrete jungle and are always stuck in traffic jams. Having said that, I did discover that Midlothian was the local authority which most closely matched the overall UK land cover figures, almost exactly. Thus, 'the Midlothian question' is born.

The UK in miniature?

Just beside Edinburgh, in case your geography is rusty

Doing something like this, if it will only be seen by a few interested boffins, is normally fine. But when it is on the BBC website millions of people might see it (and this was the case). That means you get a wide range of responses, from the 'oh cool, maps!' to the 'this is EU propaganda gone mad, the UK is basically a car park and I don't care what the numbers say'. But that's okay because most people are in the middle ground, though much closer to the first and, anyway, people are perfectly entitled to their opinions. What I thought I'd add here are some answers to common questions that were either asked directly or posted elsewhere online.

Why did you do this?
I wanted to know more about land cover across the UK. This was motivated by my desire to understand the country better in terms of land cover but also to see how much of it was actually artificial or 'concreted over' as some put it. The answer was about what I expected it to be but there is great variation locally and that's partly what I hoped to find - and to put some more precise figures on it. 

You really want to concrete over the whole of our beautiful countryside, don't you?

But you're a secret housebuilding cheerleader, right?
I do think we need to built more houses, and we do have space to build them if we want to, but the housing question was not part of this project. If it was, I would have tried somehow to look at 'developable' land or something like that. A key question in this is that for housebuilding there needs to be enough infrastructure and it needs to be in the right place. This kind of analysis could potentially help with that, but it wasn't my goal here.

Was this funded by the EU?
The data I used are created through an EU programme but neither the BBC nor I received any funding for this. I started this in my spare time (evenings and weekends) and although I used the Corine data provided by the EU (it's free and open data) this project definitely wasn't funded by the EU.

How much did the BBC pay you?
Nothing. We decided to use what I'd already done and then create a bespoke map and data set for their news website but I'd done the vast majority of the work already - Mark Easton and colleagues just thought it would be interesting and I see this as a useful kind of public engagement activity that adds to knowledge. If it gets more people interested in land and maps, then great.

What do I think it tells us?
Lots of things, but three come to mind - 1. there is more space in the UK than some of us might imagine. 2. We can find space for more housing if we want to. 3. We should probably get out and about more and enjoy the space that's out there. On 1, this is a question of where we live (i.e. 83% in urban areas) vs lots of land that can't really be lived on (e.g. the 9.5% or so that is peat bog) but I wasn't thinking it being about space for any one particular use. On 2, I didn't begin with a housing question but inevitably it comes up and when you look at it objectively you can easily find space, if you are willing. On 3, that's mainly just a message to myself but it also serves as a useful reminder that the UK is full of amazing landscapes. Maybe, just maybe, it also tells us that the way the population is distributed is something that needs to be looked at.

But England has the highest population density in Europe so this data is meaningless, isn't it?
Overall, in global terms, the UK is quite densely populated and within Europe the UK is now said to be the most densely populated. This data does not challenge that. It is about land cover, not people cover. The goal here is to show land cover, and it does show that some areas are very built up in percentage terms. Some areas may feel very crowded, but this is not just down to population numbers - it's also got to do with infrastructure, services and housing. And density is best thought of locally in my view, not at the national level. A simple national arithmetic average where you divide population by land area isn't very helpful. But I'm working on a more nuanced, meaningful metric for this that is based on a local density measure based on 1km squares. London is the most densely populated area in the UK - with an average of about 5,500 per sq km - and a high of about 25,000 - but it is much less dense than Manhattan or central Paris, for example.

If I complain that the country is full, am I wrong?
No. Yes. Maybe. Take your pick. The answer to this question depends upon people's perceptions, experience, and sometimes biases. I prefer to think about this question in relation to infrastructure, services, housing and international comparisons. But if you were born in 1930, when the population of the UK was around 20 million lower than it is now, I can understand the point of view that the country is 'full', though I don't agree with it. However, my aim was to look at land cover, not population. It's inevitable that the population question comes up but I think some data on land cover is helpful in helping people understand the UK better.

What does it all mean?
It means that someone looked at the EU's Corine data, turned it into localised maps for the UK's 391 local authorities and collaborated with the BBC to produce a simplified set of maps for the same areas. It is an attempt to take raw data and turn it into more meaningful information. The 'knowledge' bit will largely come from what people think to begin with or, in some cases, perhaps change people's perspectives. But for me it's about understanding how local areas compare to the national picture.

Can I have the data?
Yes, see below.

Last of all, if you're looking for the underlying data, I've made it available in an open Google Drive folder. Here's what it looks like if you open it up in QGIS - and there are a few more screenshots in the folder. I have also put a 'Shapefiles etc' folder in there, which contains a QGIS project file you can open and see it styled as below (there's a qml style sheet in there too). If you're an ArcGIS user, you can use the layer file included in the same folder. There is also a UK local authorities file that I made, which should come in handy in providing a bit of context.

Take a look yourself if you're a GIS user

The other thing I've added to the folder is a spreadsheet with a summary of how much of each local authority is 'built on', etc. This just provides an opportunity to see the all map data in a single place. If you want to know how the land cover categories in this table translate to the original Corine classes, well it's relatively simple. Although Corine has a 44 class nomenclature, at the most basic level it covers 5 classes - 'Artificial surfaces', 'Agricultural areas', 'Forest and semi-natural areas', 'Wetlands' and 'Water bodies'. In the simplified classification, 'Artificial' became (1) 'Built on', but without 1.4.1 and 1.4.2 ('green urban areas' and 'sport and leisure facilities'), which became their own class of (2) 'Green urban'. 'Agricultural' became (3) 'Farmland', and the rest were put into (4) 'Natural', though of course some of it is certainly not 'natural'. 

A single file with all the data from the BBC maps

Final words
I hope some of this is useful and that it makes people think. But really, of course, I'd rather you look at my more detailed Atlas because that's where all this started. I'll end with a small sample of comments from Mark Easton's blog because I think they are a reasonable representation of the kinds of reactions there have been, and they also highlight the fact that land, population and statistics can sometimes be an intoxicating combination. 

Beware positive internet comments

Nick Faldo, is that you?

There were lots of these

The lowest rated comments - a mixed bag overall

A good few people worried about how this data will be used

Right, enough on land cover for now.

Wednesday, 8 November 2017

A Land Cover Atlas of the UK

This blog post provides a bit more detail and background information about my new publication A Land Cover Atlas of the United Kingdom. It's basically a 23 page pdf accompanied by a set of 392 maps of land cover - one for each local authority in the UK, plus an overview map of the whole UK. The publication date kept getting pushed back because I was also working with the BBC on the topic and we wanted to ensure things went online at the same time. I have once again chosen Figshare to host this as it's an excellent platform for dissemination of this type of data. The Atlas uses Corine Land Cover 2012 data and was the result of a spare time summer project that I recently finished. See below for an example of one of the local authority maps, from Tonbridge and Malling in Kent, in the South East of England.

Tonbridge and Malling - a fairly fruity landscape

There are plenty of other land cover maps, and this is not new data, but I wanted to show what it looks like at the local level in a set of static maps and I hadn't seen it presented like that before; so it's kind of a new take on existing data. I did this partly because I wanted to be able to understand how areas differ but also because I wanted a better idea of the diversity of land cover at the local level across the UK in a way that was easy to digest. That's what I've attempted here. I'll leave it to the reader to decide if it works, but I quite like the results as it give me a better understanding of land cover nationwide.

The rest of this post just provides a little bit more background detail to the Atlas project itself and some of the decisions I made in producing it, including a few pictures.

First of all, I wanted this to cover the whole UK. I thought it would be interesting to be able compare all 391 local authorities in terms of their land cover. It's always a bit annoying when you see some great data but your part of the country isn't included! The Corine classification has a total of 44 land use classes but since it covers the whole of Europe there are some categories that don't appear in the UK - such as Glaciers. This leaves a total of 39, though some of these 'land cover' categories are actually water. 

Anyway, each map legend lists all possible land cover categories so that you can easily compare places and the proportion of different land types in them. I thought it was also important to be able to identify what land cover types areas don't have, that's why you'll see a legend entry for a category even if it doesn't appear on the map.

I also wanted the project to be based on open data and open source software, so I used Corine (open), Ordnance Survey data (open), Source Sans Pro font (open) and QGIS (free, open source). See the images below for a bit more on this kind of thing.

Not that it really matters, but I do like an open font

This map extract is from the Angus council area in Scotland

This was a spare time project that got out of hand

The full resolution maps are quite big - see permanent link 

I also wanted to show what land cover areas do not contain

Values <1% are shown in light font

Scale is always shown, but areas differ hugely in size

Some slight boundary line mismatches - later maps will have a fix

This all started with me wanting to investigate what percent of the UK is 'urban' and how much is 'golf course' using the new OS greenspace data. I got to the bottom of the latter but in relation to the former it really does depend upon how you measure it. I don't believe there is a definitive answer but if I was pressed, I'd always say less than 10%. Corine data suggests it is about 6%. 

Take a look at the Atlas document itself if you're interested - and then check out the map for your area. You can also see more about this in a related BBC story, for which I produced a set of simplified Corine maps that are much easier to digest because they only show four categories.

In the BBC piece, I also decided to show building footprints, as a way to add a little more detail and legibility to the maps. This meant I had to create a set of files with all UK buildings in it. For Great Britain this was pretty time consuming, but straightforward. For Northern Ireland I had to create it using open data. But the end result was quite satisfying and after a bit of to-and-fro with Tom Armitage from Digimap to confirm my figures I was ready to go. As it happens, the total area of the UK covered by buildings amounts to about 1.3%. In Great Britain overall it's about 1.4%, which is much higher than, say, the land covered by golf courses: 0.54%. I have also calculated these figures at local authority level, but that's one for another day.

Mind you, if you are stuck for something to say during a lull in conversation, you might want to interject with "I say, did you know that Islington is 30% building, and that Birmingham and Manchester are both 15% building?".

Notes: I have done my best to map this as accurately and clearly as possible. If you do spot any mistakes, please feel free to get in touch. Like I said above, there are other methods for mapping land cover and there are other data sources so don't take this as the definitive account. Having said this, I think it provides a good overview of land cover in the UK at a local level. There are some further notes on these kinds of things in my Atlas document, on page 3. The Appendices also contain data tables on land cover for each part of the UK.

Monday, 16 October 2017

Local House Price Growth in England and Wales, 1995 to 2016

On the BBC website and news today there's a great story about house price growth in England and Wales over the last decade. It uses ward-level house price data to track changes over the past decade. I wasn't aware until relatively recently that the ONS had done a lot of the leg-work with HM Land Registry price paid data and aggregated it all to smaller geographies. They publish it down to MSOA level as 'house price statistics for small areas', and very nice it is too. There's median price, lower quartile and a range of other indicators. Naturally, I wanted to see what this looked like, but I thought it would have to be simplified. What I did was take two market price points - £200k and £500k (as very rough approximations for national and London averages) and map the spread of house price growth since 1995. Here's what it looks like in a 30 second animated gif. The geography is the MSOA, with an average population of about 5,400.

Watch this on loop for best effect

I wonder if I should post all the individual frames of this below? Okay, go on then. And you should be able to click to open then in a larger size in a new tab/window. It's interesting that after 2008 there appears to be a geographical contraction and also a bit more volatility and less uniformity of spread - and by 2016 you see a big chunk of the London area in dark red (over £500k average).

Okay, so we can really tell that by 2016 in most MSOAs across England and Wales house prices are at £200k or above and in a good chunk of London and the South (and a few areas elsewhere, notably Cornwall and south Manchester) they are £500k or more. That's all for now. I'll leave you with the same gif as above, but this time at 1 second per frame, plus another earlier version with the year label in the top right.

Slightly different design, same data

Thursday, 5 October 2017

Measuring Land for Mountain Goats

Land is frequently contested and fought over. In fact, in the history of civilisation, you could say that land has been at the centre of some quite important disputes (if you really wanted to understate things, that is). Because I'm a) a bit of a nerd and b) technically some kind of planner or geographer, I decided to look into the question of how you measure it. More specifically, I looked at the question of how you measure land area in areas where land is not flat - as it mostly is not across the surface of the earth. I did this one Friday afternoon many months ago with colleagues Ed Ferrari and Ruth Hamilton, and then I wrote up the slide set below. I'm hoping someone out there finds this interesting and/or useful.

The basic problem: land is very rarely totally flat. A 50m x 50m square on a flat plane will cover an area of 2,500 sq metres. But that doesn't take into account slopes, bumps and so on. So when one of your friends asks you 'if you flattened Switzerland, how big would it really be?' - after reading this you should be able to tell them.

To find out more, click to follow the slides below. The slides explain things in more detail and also say a bit about how to do these calculations in a GIS.

If you're too busy to do this, just know that when you take into account the topography of the land, Switzerland would grow by about 7% and Liechtenstein would grow by about 8%. But of course this partly depends upon the resolution of the data you measure it with. These figures come courtesy of Ed Ferrari.

A picture of a mountain goat I found on the internet

To make it easy for you to remember, should you ever want to use these slides, will take you straight there. That's just followed by measure sloping land as one word, after the forward slash.

Notes: spot an error in all this? Let me know. Got better examples? Please get in touch. I believe land is measured most often as if it were on a flat plane. Mostly this probably doesn't matter but I did want to find out what impact there would be if you took slopes into account. There are quite a few notes and disclaimers in the slides, so take a look at those for more information. Mountain goat picture is from this blog.

Tuesday, 19 September 2017

Buildings of Great Britain

One of the great things about Great Britain is of course Ordnance Survey, our 225 year old national mapping agency. Since a lot of their data was made open in the last decade it has become easier than ever to explore and map the country (particularly if you're some kind of GIS boffin). I should point out before going any further that Ordnance Survey covers Great Britain only, not Northern Ireland. For that, you'll need to look at Ordnance Survey Northern Ireland, who also have a good selection of open data. In the past, I've created sets of building data for English cities, done some calculations on how much land is taken up by golf courses, and a variety of other things, including creating 3D building models using OS open data. Today I'm sharing some shapefiles of all buildings in Great Britain

London has lots of red buildings

Why have I done this? The main reason is that I want to share these complete files with others who might need them and don't want to - or know how to - patch together separate tiles of Ordnance Survey open data. If you need some building footprint data, you can just download the set you need and zoom to your chosen area, or extract what you need from an individual shapefile. I used the OS OpenMap - Local product for this, so the detail in the buildings is very good, as you can see below in the zoomed-in extract.

The reason I did this in the first place was because I wanted to come up with a number for the percentage of the land area of Great Britain that is covered by buildings. There has been some debate about this topic, and it was covered in an FT Fact Check piece in 2016 by Kate Allen partly in response to the claim that golf courses cover more land than housing. My calculations using new OS data revealed that Great Britain is 0.54% golf course (1,256 sq km, about the same area as Greater Manchester). But do buildings cover more than this? Yes they do. 

Click to enlarge - you can see lots of detail

Using the data I'm sharing here, I calculated that buildings in the UK cover 1.35% of the land. I reported this previously in a tweet that was quite widely shared. To my embarrassment, for the GB figure I used the UK area as the denominator so the figure reported there was a little low, though the England, Scotland and Wales figures were and are accurate. Here's the important information you need, if you're ever faced with an awkward silence at a party.

> Great Britain is 1.35% buildings

> England is 2.0% buildings
> Scotland is 0.4% buildings
> Wales is 0.9% buildings

The Great Britain figure equates to about twice the area covered by Greater London - 3,150 sq km. However, the OS OpenMap - Local product isn't the most detailed building-level data covering Great Britain. For that, you'd need to look at OS MasterMap, a much bigger job. That is, unless you are Mike Gale and Tom Armitage at Edina and you have all this information in a lightning-fast database ready to query. They very kindly went beyond the call of duty and did some calculations and confirmed that my figures are pretty much spot on. They did loads of other really cool stuff too, but more on that another time perhaps.

If you click on the link, you'll see the set of shapefiles I created in a Dropbox folder. It contains some licence information, a few sample images, plus the following shapefile sets:

  • All buildings in Wales in a single shapefile
  • All buildings in Scotland in a single shapefile
  • All buildings in the North of England in a single shapefile
  • All buildings in the Midlands in a single shapefile
  • All buildings in the South West of England in a single shapefile
  • All buildings in the South East of England in a single shapefile

The complete set, and individual files, are pretty big, since they cover large areas and have millions of individual polygon features in them. This isn't exactly the best way to view and map this data, I know. That's obviously an understatement. But I also know that it's the format many people know and love, and want to work with. So, if you want to play around with buildings or use this as background mapping, be my guest. 

Data notes
Giant shapefiles make the world go round, okay. More seriously, there are better ways to download and view this data, but that's for another blog post. On a related note, see this from Emu Analytics on a very cool project which utilises OS building data. If you download the files and look at the attribute table in your GIS of choice, you'll see that I've added an area column for each polygon showing the area in square metres. 

Using this to query the dataset can be interesting - e.g. to find the largest building. Be aware, however, that the OS OpenMap - Local data still has a degree of generalisation in it so sometimes separate buildings can be merged together - e.g. if they are very close together. But I know from what Mike and Tom did with MasterMap data that this doesn't affect the final calculation much at all, thankfully.

As for the golf courses vs. housing thing, I know that's not strictly-speaking solved definitively either. The reason for this is that we have no way of knowing exactly what area houses cover vs non-residential buildings. As far as I know, even Ordnance Survey don't know this, and probably can't know this from what data we have available to us. However, the vast majority of buildings are residential (not sure on the %) and I'm very confident that housing covers a much bigger area than golf courses, I just can't say how much.