Friday, 28 August 2020

Automatic Knowledge

This is a short blog post about what I'm doing next, now that I have moved on from my job as a professor at the University of Sheffield. It's not going to be a self-indulgent retrospective, it's more for information and a bit of reflection. From 1 September 2020 I'll be working on my own business, Automatic Knowledge Ltd

Take a look at the website to find out more, but I'm going to be doing similar kinds of things (maps, stats, analysis) though instead of working mainly within academia I'm mostly going to be doing this with other organisations and businesses. I was doing some of this already and in fact it was taking up more and more of my time so I decided it was time to make the move. 

Oh, but since I really like teaching (and learning) I'm going to continue to offer my QGIS courses - see the Training part of the website for more on that too. I've already done these for Savills, the BBC, the FT, Regeneris and more.

The Automatic Knowledge website

What's that you say? Brand identity? Corporate logo? Well, I'm not very big on all that but if you see something that looks like this (below) on a map or graphic or chart from now on you'll know it comes from me and my team (talking of which, I'll be working with a couple of former associates from now on, as projects require it).

Why 'automatic knowledge'? Read more here

People keep asking (really) so I'm opening a small print shop as well

Anyway, that's what's next and I'm very much looking forward to it - expanding my consultancy work, building on existing relationships and making new connections. 

Get in touch via the Contact page on the website, via LinkedIn or via Twitter if you want to start a conversation about consulting work.

Free stuff
In my previous job I liked to share as many free, useful resources as I could with the wider data and spatial analysis community, and I'll continue to do that with Automatic Knowledge. It's a small gesture but I know many people have found my resources useful - because they've told me about it. So, on my new website, I have also added a Resources section where you can find the following datasets. It's all already available as open data but in all cases I have added to the original with useful extras. I will also continue to try to give back to the open source GIS world in the form of financial support to the QGIS project. I've done this before and in fact that's the only time Automatic Knowledge has been seen online previously, though in this example it was before I set it up as a formal business. I'd only used that name for anonymous donations in the past.

  • A set of geofiles (shp, gpkg, geojson) of all places in Great Britain - more than 40,000 of them. There are also some more useful goodies here, like a fully-formed QGIS project and instructions on how to style and filter the data.
  • My popular 'All buildings in Great Britain' layer. You can get this online already in small chunks but I have put it all together and also added for each feature information on which local authority it is in and the floor area of the building footprint. This is a geopackage (gpkg) only because of the size.
  • A 'UK local authorities (2020) with data' file. I've added a number of fields to the dataset, including ones to indicate what country or region an area is in, plus the latest ONS population estimates (mid-2019) for each area, broken down by single year of age. There is also a field indicating what kind of area each local authority is (e.g. London Borough, Unitary Authority, Non-metropolitan District). I've made this layer available in shp, gpkg and geojson formats.
This is a preview of the GB buildings file

Looking back, just a bit
I'm not really one for looking backwards, or for sentimentality, but it appears that I worked at the University of Sheffield for a total of 4,320 days (about 12 years). I'm also not one for deleting emails and I can tell you that I received a total of 190,020 during that time (using 71.67 GB of inbox storage space, despite deleting tons of attachments). 

Using a sophisticated mathematical formula to crunch the numbers, I can tell you that this equates to an average of 44 emails a day. Now, in the early days I wasn't getting as many as I did at the end, but I tried to answer them all, honestly. But really, why am I talking about email? I have no idea.

Anyway, if you email me now at my University address you'll just get an out of office (forever) autoreply and a picture of the Western Highlands of Scotland. 

A typical blue sky day in the Highlands

If you were a student during my time at the University of Sheffield, I just want to say thanks. Teaching and interacting with students was one of the best things about it and I feel like I learned a lot about myself, life, others, and so much more. I'm still in touch with many former students, so I hope that continues - even if it's just to ask me for a reference. I 100% enjoyed marking every piece of work you all submitted (plus or minus 75%). 

I'm so proud that so many former students have gone on to do great things, or just to figure out that they really weren't that into urban studies, planning or GIS in the first place - there are many roads, and finding the one you want to be on can take time! I can't take any credit for this, but it's good to have been able to join you for part of the journey.

Talking of students, see below for a 2009 Lille field trip photo - quite low resolution, unfortunately, but then again it was taken on one of those custom-built 'cameras' we all used to take photos with. 

I can't name absolutely everyone because the resolution is too low, but I will at least give a shout out to Rebecca (far left), Grant (front, crouching), Isaac (double wave, far right), Liz (middle-ish) and Justine (also middle). As for the small boy in the plant bit of the sculpture, I have no idea who he is but he wasn't a Sheffield planner. Talking of the sculpture, it's the Shangri-la tulips sculpture by Japanese artist Yayoi Kusama in the French city of Lille, right by the Eurostar station (the glass building in the lower background).

Click to enlarge - or click here instead

Right, that's more than enough of that.

I'm going to take a very short break and maybe do a little bit of Highlands and Islands touring before I get back to work on the projects I have lined up, but feel free to get in touch if you think I can help with any analysis, maps, stats or research.

Sunday, 2 August 2020

Rain shadow maps

Each summer for the past few years I've experimented a little bit more with mapping techniques, so here I am again with another effort. You can see earlier ones on mapping marine traffic and shaded relief elsewhere on the blog but this time I've taken a little look at mapping rainfall. Well, it's mostly rain in the UK but of course to be accurate it's about all kinds of precipitation. But the effect I was playing around with also doubles as a kind of 'rain shadow map' effect and my first go was for the UK so rain definitely fits the bill here. Okay, let's start with a map of rainfall in Great Britain, below. This is a slightly edited version of one I already posted on twitter. Below that, you can see a new one for the contiguous United States, using the same style. The UK one only covers a short time period but the US one relates to 30 year averages.

The vertical peaks are exaggerated for effect

I find the patterns here really quite fascinating - e.g. Central Valley

I've done quite a bit of this kind of 3D mapping recently and of course there are upsides as well as downsides to doing this but I think this does a pretty good job of highlighting the peaks in precipitation and also the general pattern across the areas shown. If you enlarge the images you should be able to see things in a lot more detail. The Great Britain map is based on a 1km grid and the US one is based on an 800m grid.

Here are some other versions of the same kind of thing, for the US and also for Northern Ireland.

Simplified and stylised but you get the general idea

Hey, does it ever rain in Seattle, or San Francisco? How about Bakersfield?

The data for Northern Ireland is published separately to the rest of the UK

Okay, that's basically it but see below for some information relating to data sources and software. These visuals are really just me experimenting with the method and I suppose I'd classify them as a mix of science and art (if I'm being charitable to myself) rather than useful analytical tools. But I do think they provide an interesting overview of climate patterns and there is potential here with this kind of technique to take it further both for climate data and other stuff.

Plus, I like the idea of 'rain shadow maps' that use actual shadows to show where the rain shadow effects might actually be felt.

Data sources
For the US maps I used data from the PRISM Climate Group at Oregon State University, and there are all sorts of great datasets available. The data in the map above is from the Normals section of their website and they have lots of great ready-made climate maps available to download.

For the UK (although it's provided as GB and NI separately) I used data from the Centre for Ecology and Hydrology. This is available under an Open Government Licence but I also need to cite the following source:

Tanguy, M.; Dixon, H.; Prosdocimi, I.; Morris, D.G.; Keller, V.D.J. (2019). Gridded estimates of daily and monthly areal rainfall for the United Kingdom (1890-2017) [CEH-GEAR]. NERC Environmental Information Data Centre.

You may not be surprised to learn that these were produced in Aerialod. I did a bit of pre-processing in QGIS and some file compression in IrfanView but the visuals were rendered in Aerialod. I outputted the US ones at 5760x3240 pixels in about half an hour, once I got all the lighting and everything else just as I wanted. If you want to learn more about Aerialod, you can check out some of my earlier blogs on how to get started.

Thursday, 18 June 2020

Where is Scotland's pole of inaccessibility?

The first question you might be asking is 'what is a pole of inaccessibility'? Well, it typically refers to the furthest point from the coast - either at sea or on land. In this case, I've calculated Scotland's pole of inaccessibility using the coastline (Mean High Water mark) and arrived at a single, fairly accessible point a few miles from Braemar, in Glen Quoich. It's just over 41 miles from the sea and at an elevation of about 470 metres above sea level. It's almost slap bang in the middle of the Cairngorms National Park, as you can see below. This all began when Andy Arthur asked me an entirely different question on Twitter last night about dividing Scotland in two, but that's a different story with a different answer

UPDATE, 25 June 2020: a Braemar resident very kindly got in touch, went to the pole of inaccessibility (I provided accurate coordinates), and has taken some photos of it. Scroll to the bottom to see these.

Actually quite an accessible point, if you're keen

Where's the nearest coast to this point? Well, Dundee comes close and so does Montrose, but by my reckoning the nearest coast is just off Inverness (near the A96 and the big Tesco, since you asked). Nairn beach comes close as well though. In reality, as Colin Angus has pointed out, it's going to be more than one point but I chose Inverness because it's surely one of them, plus home town bias (EDIT: see bottom of post for the three closest points). From the centre of Braemar it's about three and a half miles to the point of inaccessibility. Here's how that looks on a national map (below) and some zoomed-in maps too. It should go without saying that this calculation is based only on what might be called 'mainland' Scotland for the purposes of this exercise - i.e. the calculation is based on the big landmass you see highlighted below in the first image.

The point furthest from the sea

Looks like you can walk pretty close to it

OS 20km tile N008, since you ask

To calculate this, all I did was load a detailed shapefile into QGIS, then I divided the Scotland file into separate polygons and extracted the 'mainland' polygon. Once I'd done that I just ran the 'Pole of inaccessibility' tool from the Processing toolbox and got the point you see here. I then did some buffering at single mile intervals to explore it a bit further, as you can see below - and above in the zoomed-in one.

The shading is 1 mile buffers - from the coast

The white lines are single mile buffers

By now, you might be thinking something like 'but how does this compare to the location of the geometric centroid of mainland Scotland'. Admit it, you definitely were thinking this. Well, the answer to that question is that it's just over 42 miles away to the south west, in between Loch Ericht and Blackwater Reservoir near Kinlochleven. See below for a couple of maps on this - and as always if you spot something iffy here or think this doesn't add up, feel free to get in touch. 

Amazing knowledge to impress your friends

The beating hearts of Scotland

I suppose the amazing thing about all this is that you didn't now how much you needed to know it until you knew it. I would say it's pointless information, but there are at least two points on the map above that provide evidence to the contrary. A little bit of map knowledge can't do us any harm, and you might even be able to use this to visit Scotland's pole of inaccessibility and then tell all your friends about it.

If you need a map of how to get there, see below.

There you have it - white lines are a mile apart

A short addendum here because I couldn't leave this without knowing where the closest coastal points were, based on the Mean High Water definition. It turns out it's by Inverness, the tidal part of the Tay to the north of Perth Harbour, and the tidal parts of the South Esk just by Montrose Basin. The circle in the map below has a 41.25 mile radius from the pole of inaccessibility shown above.
Yes, Perth has a harbour

Obviously, this is 'coast' by the official definition

This is the most 'coasty' of the coastal points

I'd begin from Inverness if I were you

Technical post-script
Despite spending literally quite a few minutes on all this, I couldn't figure out a way to calculate a pole of accessibility for Scotland that didn't treat the border with England as coast. Not that this really matters anyway because you can eyeball it and see that it's not in the south. I also manually checked it. But, this does mean that the internal buffer map above should be read in this context - i.e. the buffers from the border are obviously not from the coast. In order to show a bit more of this, I've taken the circle in the maps above and centred it roughly where I think the pole of inaccessibility is in the south of Scotland - somewhere around Hawick I suspect. If I had spent a lot of time on this, I might have been able to solve it but for the time being a manual check on this is all I need.

The border is not the coast, obviously

Here are some photos of Scotland's 'pole of inaccessibility' - with a pole marking the spot!
As I said above, these were sent to me by a very kind local resident, so I didn't travel here to take them but I hope to do so one day in the future. The pole of inaccessibility is at the edge of the Glen Quoich Caledonian pinewoods and birchwoods, as you should be able to see from the photos. It is surrounded by expanding woodlands, so the view will be very different in a few decades' time! (words provided by our friendly on-the-ground Braemar resident who lives less than 5km away from this point).

Looking south to Morrone
Looking southwest to Creag Bhalg

Looking northwest
Small Scots Pine

Here's the pole - looking at Caledonian pinewoods

Friday, 12 June 2020

City street patterns

This is a post about streets. Or roads. Or Avenues, Lanes, Gardens or Places, or even Ways. Giuseppe Sollazzo did this kind of thing already in an extremely clever and efficient way, and he's very much the inspiration here. But I wanted to revisit the topic for a variety of reasons, though mostly just curiosity - and sometimes that's enough. 

So, let's begin by looking at one of the 451 maps I made, of the UK's biggest new town (population now about 270,000).

Quite a few Close and Court and Hill ones 

As you can see, all I've done is colour the roads according to their designation as 'Road', 'Street', 'Way' and so on. The 'Other' category includes things like 'Close', 'Court', 'Hill', 'Crescent', 'Brae' and a few more, but there aren't enough colours for all of them. In Milton Keynes, and some other places, you also get streets with no second name, like where they are just called things like 'Downland' or 'Cavenham' or 'Garston'. I think that's because the local road namers refused to work pro bono. Okay, I'm really struggling to make this U2 joke work, sorry.

Anyway, I've done maps for 451 different locations across Britain using Ordnance Survey open data (the OS Open Roads layer). I then added some place names using a possibly-annoying whimsical font (Nanum Pen) and then added a colour bar legend along the bottom so you can tell what's what. Here's a map centred on Liverpool, which is a good example of the way in which 'Street' tends to be in more central, older locations. 

You can see Queens Drive here easily

The scale is the same on all maps - I used a 1:50,000 scale and you can see how far one mile is by looking at the scale bar in the bottom left of the images. they are mostly focused on individual towns and cities but in the case of London, for example, there are maps centred on specific areas, such as Kensington.

It's all Roads beyond here

Edinburgh (with its New Town, for example) and Glasgow (with its central grid pattern) are actually very interesting, so I've added them below.

Spot Holyrood Park

Look closely to see the M8 and M74, M77, M80

What about some other big towns and cities? The full set is in this online folder, but I've posted some more below for the time being. I've started with York, because it's not fair that it always ends up last on the list.

That's all there is to it, really. 

But I do think it gives us an interesting glimpse into lots of other things, like how the new towns were laid out, how some of our older and bigger towns and cities developed, and even things like density, connectivity, topography and so on. 

I'm fascinated by the shapes of towns and cities so that's how I got here, and if you've scrolled this far maybe you are too.