Research with QGIS, R and speaking to people

I recently led a piece of research for the Joseph Rowntree Foundation on disconnected neighbourhoods - basically, it looked at the UK's most deprived areas and how connected or disconnected they are to their wider cities in relation to jobs and housing. You can read the brief Findings or Full Report here. This post is just about the methods we used and some of the outputs. We used open source software QGIS and R for the analysis (led by Ruth Hamilton) and we also spoke to policymakers across the country (Rich Crisp and Ryan Powell).

You can see the full report here

We looked at those areas in England, Northern Ireland, Scotland and Wales that fell within the 20% most deprived on the national deprivation indices in each nation and then explored data relating to household moves and commuting (plus lots more). We updated and developed two area typologies to help us make sense of the data - and to see how things changed we produced riverplots (Sankey diagrams) in R (this was done by the briliant Ruth Hamilton).

Created with the Riverplot package in R

I also did a little bit myself with open source software, including updating a 'divided cities' type graphic I produced in the past - looking at the spatial split between most and least deprived parts of 13 cities across the UK, as you can see below. See Sheffield in particular for a very stark divide.

Red = most deprived, blue = least deprived

Our colleagues Rich Crisp and Ryan Powell then spoke with more than 140 policymakers in cities across the UK - another nice 'open source' method. You can read more about this in Chapter 6 of the report but the bottom line is that if we want things to change for the better then we need to take a different approach to urban policy - a more inclusive approach.

As I said, there are two typologies, and we then combined these into a matrix in an attempt to understand area types a bit better and then suggest possible policy responses that might make sense. What each category means is explained in the report but in the figure below you can probably make sense of what 'Gentrifier' areas are and what those labelled 'Disconnected' are.

The shaded areas might be a good policy focus to begin with

The point of this blog post, however, was just to highlight how useful and effective open source software now is, and can be, in real-world research. Advocates will already know this but many more have yet to make the leap so hopefully this will provide just a little bit of inspiration or motivation to do so.

We produced hundreds of maps for the project (too many for the report) so you can probably find one for your area in the online folders. Two examples - one for each typology - are shown below.

Residential typology map of Birmingham

Travel to work typology map for Glasgow

For more on the methods used to develop the typology, see the Annexes in the Full report.

What can explain Brexit?

There have been enough maps, charts and infographics on Brexit already, so I'll just post three scatterplots here. I was trying to figure out what was going on, so I explored a few key variables that I thought might explain why people voted the way they did. I chose deprivation, lack of qualifications and higher qualifications, using 2011 Census data. I only focused on England here. Some of it has been done already by John Burn-Murdoch at the FT, though in a different (much nicer) way.

First, deprivation is - overall - not strongly correlated with percent voting leave at the local authority level (R-squared is 0.0369). I used 2015 Indices of Deprivation at the local authority level in the scatterplot below, where I've also labelled some areas and coloured the points by region. 

Click to enlarge

Next, I decided to look at whether the percent of people with no qualifications correlated closely with the percent voting for leave in each local authority in England. This was much more successful, with an R-Squared of 0.6197. 

A pretty convincing pattern here

Finally, I decided to see whether higher levels of education - rather than lack of education - was more strongly associated with the propensity to vote leave, and it is. In the scatterplot below I compare the percent with Level 4 qualifications or above with the percent voting to leave the EU. This produces an R-squared value of 0.8053, which is really pretty high.

The outlier to the top left is the City of London

This may not be particularly surprising, given what is known about the link between voting patterns and education but I think it's particularly interesting because of i) the historic significance of this referendum and ii) the people likely to be hardest hit by any post-Brexit economic downturn.

Data sources: Census 2011 table KS501EW via NOMIS and EU Referendum data is from the Electoral Commission. I have shared the spreadsheet (and the Level 4 vs. % leave) here in case anyone wants to look more closely or make an interactive version.

Deprivation and the 1%

There's a lot of talk these days about the 1%, or the global elite in terms of wealth. Danny Dorling's  'Inequality and the 1%' is a recent example of a powerful polemic on this issue. This post is about a slightly different kind of 1%, within England. As keen observers may know, I've been producing a series of map extracts based on the English Indices of Deprivation, so I'm now adding to this by looking at the 1% most and least deprived areas across the country.

Cheltenham features prominently at one end of the spectrum

Middlesbrough features prominently at the other end of the spectrum

It's a very simple concept, and I was particularly interested to see which local authorities came out of this with no representation at either end of the spectrum - in addition to those which feature at both ends. In total, 160 local authorities feature in the map series which looks at the top and bottom 1% most deprived in England and 10 local authorities have areas that fall within the 1% most and least deprived, as follows: Bradford, Bury, Chesterfield, East Riding of Yorkshire, Leeds, Newcastle, North Somerset, Sheffield, Stockport and Wirral. See below for more maps.

Wirral - features at both ends of the 1% most/least spectrum

Sheffield also features in the 1% most/least divide

Leeds has quite a distinctive inner/outer deprivation pattern

Look closely at the north east coast here

If you want to see all 160 maps to find out if your area is included, I've set up a separate page for that - you can see a preview below. If you're looking for other IMD maps, please take a look at my Indices of Deprivation 2015 maps and data page. I've done the 1% versions today to highlight the extent to which deprivation is concentrated spatially in the areas at the very top and bottom of the rankings - and also as an exercise to see which local authorities have areas in the top and bottom. I didn't know this before and I think the results are quite interesting.

You can explore all the areas here

Finally, a bit of map-related trivia. I couldn't help notice that in this map series Guildford looks like some kind of animal, but I can't quite think what. I'll add this to my previous observations.

I'll add this to my 'boundaries that look like animals' file

Note: I missed out the East Riding of Yorkshire in the original list. Thanks to Adrian Bailey for spotting this. 22 Oct 2017.

Children living in deprived households in England

Not a particularly upbeat post title today, but it's an important topic too often overlooked. I wanted to shed some light on the matter because there are copious amounts of data on the issue, including those released as part of the 2015 English Indices of Deprivation, which I've explored in-depth in the  past though a series of maps. A recent Twitter message from freelance writer and HuffPost blogger Shumailla Dar prompted me to re-visit this topic (thanks) and since I had most of the data set up, I thought I'd do another map series - this time with the 'Income Deprivation Affecting Children Index' (IDACI) from 2015. See below for an example of what this looks like. In all maps, I've added a little inset to show the pattern from the overall Indices of Deprivation 2015, for comparison. I also show the percentage of each local authority's small areas in each decile on the IDACI measure nationally.

Quite a north-south divide in relation to income deprivation and children

Okay, so what is 'income deprivation affecting children'? It's the proportion of all children aged 0 to 15 living in 'income deprived' families. 'Income deprived' is defined as 'families that either receive Income Support or income-based Jobseekers Allowance or income-based Employment and Support Allowance or Pension Credit (Guarantee) or families not in receipt of these benefits but in receipt of Working Tax Credit or Child Tax Credit with an equivalised income (excluding housing benefit) below 60 per cent of the national median before housing costs'. Again, not the most exhilarating topic, but given the impact this can have on young people's lives, it's such an important one. If you want more information on the details, see the Indices of Deprivation Technical Report. Beyond the technicalities, here's how it looks on the ground in Middlesbrough, the local authority with the second highest proportion of children living in income deprived households (35.7% overall).

Second only to Tower Hamlets on this index

The Indices of Deprivation 2015 Research Report found, though a user survey, that whilst 99% of respondents had used the Index of Multiple Deprivation, the figure for IDACI was 69%. Still high, but it suggests a lot of people haven't looked at it, particularly since most respondents were people already working with the data in local authorities, universities, central government and charities. Most 'normal' people have very little idea that the data exists or what the patterns are like in their area - hence today's post. There is also a similar index which reports income deprivation affecting older people, but that's one for another day. Before showing any more maps, here's the top 20 local authorities across England in terms of the percentage of children living in income deprived households.

Source: DCLG, 2015, p. 23

Now for some more maps, before I provide a link to the folder with a map of every local authority in England. First of all, here's Tower Hamlets. Remember that the bars show the percentage of small areas (LSOAs) in each local authority that are ranked in each decile within England - which sounds a bit confusing, I admit. To give an example instead - for the map below, 54.2% of Tower Hamlets LSOAs are ranked within the most deprived decile in England on the IDACI measure. Just bear in mind that the maps present a relative picture for England as a whole and that, broadly speaking, red = bad and blue = good.

This issue is very well know, but persistent, in Tower Hamlets

Chiltern's at the other end of the scale - but note the single red area

A real mix of areas in Bury

Liverpool is a city of contrasts on this measure

What does any of this tell us? Of course, we know that some places are rich and some are poor and that this will inevitably have an impact upon children in those areas but these maps reveal nothing of cause and effect. Rather, I hope they will provide local agencies, analysts and residents with an opportunity to explore patterns related to income deprivation affecting children in their area and perhaps to think about a topic they hadn't before. It's certainly not a new issue but one that, I think, we could do a lot more about tackling. But that's a step too far for today because I just wanted to share these maps after running off a new batch after being prompted to think about it. 

Click here to go to the big Google Drive folder with all the maps

Files are ordered alphabetically, by local authority name

If you want to see comparable maps for the 2015 Indices of Deprivation overall, the 20% most and least deprived and other varieties of deprivation map, see my main IMD 2015 page. If you find any of these maps useful, feel free to use them and share them. 

Notes: you can find full details of the data and method on the government's English Indices of Deprivation 2015 web page, including the IDACI spreadsheet. I'm not a member of, or affiliated with, any political party in case anyone asks about the red and blue colour scheme! The labelling is a bit wonky in places owing to the variable coverage of OSM data. I did a version with Ordnance Survey labelling too but this had too many points, but hopefully some of the labels help identify key areas you are interested in.