Thursday, 9 September 2021

10km City Squares

In 2020 I wrote a piece on here along with some maps showing 10km by 10km (that's 6.21 miles) city squares for places across Great Britain. I'm fascinated by the different forms and types of urban development we see on the ground, so I'm back for more - this time adding in blue and green to the grey of building footprints. You may even have seen my Twitter thread on this, but here I'm sharing some final versions of these grey-green-blue maps and also saying a bit more about what they actually are - and are not. But first, let's look at a few of the final map outputs for some interesting places. There are currently maps for 199 different places across Great Britain in the project folder. Sadly I don't have the data for Northern Ireland, otherwise I'd have included that too.

























What are all the different colours?

The grey bits are buildings, roads and railways. The blue bits are surface water (lochs, lakes, ponds, rivers and the like) as well as coastal water. The green bits are less straightforward in that they do not represent every bit of green in a place - and are not intended to. They are the areas included in the OS Open Greenspace dataset from Ordnance Survey. You can read more about it on the dedicated webpage for it but in short it includes public parks, playing fields, sports facilities, play areas, allotments and more. So, things like some large green areas (e.g. national parks, gardens, and the like) are not green on these maps. What I am trying to do here is compare like-with-like across the country to examine density, the patterning of urban fabric, water coverage and a particular kind of green. Nothing too profound but something I find particularly interesting.


My town isn't included - you scoundrel!

I can in theory do this for about 43,000 places across Great Britain so if the place you want to see has been overlooked and is not included in the web folder then feel free to get in touch and I might be able to add it. Alternatively, strike me off your Christmas card list and consider me a non-person from this point onwards.


Can I print one of these and hang one on my wall?

Be my guest. I have outputted them at 300dpi (png files) so they should print at a reasonable size. I may yet add some to my print store but not sure yet.


How did you make these?

I downloaded the map data from Ordnance Survey and then put the layouts together in QGIS. I then automated the exporting of each map using the QGIS Atlas function. If you're desperately keen to learn how to do such a thing then I also do training on that. Once you get one layout set up, it's fairly straightforward to export a big batch, so long as your computer is up to it. See below for how they look in QGIS just before I export them.

This is how it looks in QGIS Print Layout

Why did you choose 10km?

Partly because it's a nice round figure within which a lot of towns and central cities fit, and partly because 20km was too big and 5km was too small. I chose it through trial and error and it works reasonably well for most places.


That's all for now. Happy browsing.



Wednesday, 1 September 2021

A village called Trumpet (and how to find it using open data + QGIS)

This post has two main purposes. The first is to highlight the often weird and wonderful place names of Great Britain, including the village of Trumpet in Herefordshire, recently visited by my colleague Philip Brown - hence the title. The second purpose is to show anyone who is interested how to filter and find them using freely available Ordnance Survey data and free GIS software (QGIS). I'll start with a little bit of information about the data, then some maps of interesting place names, and then at the bottom I'll share more for any aspiring data boffins who might not yet know how to do this. I use these kinds of datasets in my QGIS training sessions, so that's the real reason I've been playing around with this stuff lately.

Trumpet, Herefordshire

The first step is to grab Ordnance Survey's OS Open Names dataset - a free download, available in several formats, and no registration required. There are nearly three million points in this dataset (2,952,321 as of the August 2021 file) and for each point we have the name of the feature in English, plus Gaelic or Welsh where applicable. You can read all about it in the official user guide, but for now all you need to know is that for each point there is a column which tells us whether it's a populated place, transport node, landform, some kind of landcover (e.g. a forest), hydrography (e.g. a loch) or 'other' (e.g. a primary school). For this little blog post, I filtered the data so it relates to populated places only, and this gets us a total of 43,120 places across Great Britain. This is how the places are broken down by type in the OS Open Names classification

There are four places called 'City' in Great Britain - two of these are hamlets and two are suburban areas. there is also one hamlet called 'Hamlet' and one suburban area called 'Hamlet'. 

Maps of interesting place names

Right, let's try some maps of interesting place names now - I'll leave the labels on and messy, so you won't be able to see them all on every map but that's okay, this is just a bit of trivia.

Place's, right?

Note 'London Apprentice'

Lots of Saints

Map-on-the-blog

The only exclamation in the nation


Rhymes with scone

Nicely lined up


I detect a pattern

England has cornered the market

Cat places

Such vital, vital work

Hello Romans

Yes, you get the idea. I could go on forever but I won't - I'll just add a few more and then say a little about how to structure queries for this in QGIS.




How much longer caan I go on?

I thought this should go at the bottom


How to filter to find different places in QGIS

If you are a regular reader, or if you follow me on twitter there is more than a tiny chance that you might already be a bit of a GIS nerd, so this bit may be surplus to requirements. If not, and you want to know how to filter layers in QGIS, read on.

In the final map above I have shown places containing the word 'bottom', because I am very mature. To do this in QGIS, all I did was right-click the layer in the Layers panel, click Filter... and then enter the correct query into the Provider Specific Filter Expression box, as you can see in my screenshot below.

This is using the OS Open Names points layer

Here's the filter text, if you want to copy and paste it:


"TYPE" = 'populatedPlace'

AND 

(

"NAME1" LIKE '%bottom%' 


Explanation? There is a "TYPE" column and a "NAME1" column in the OS Open Names dataset so I've used these to filter the data to show only those places that are categorised as populated places, and only where the name contains the word 'bottom'.

On the last point, the LIKE operator looks for anything between single quotes - the % signs just ensure that QGIS finds anything with bottom, including anything with text before or after it. If you put in 'bottom' it would find only places called 'bottom', if you put in '%bottom' it would find only places ending in 'bottom' and if you put in 'bottom% it would find only places that start with 'bottom'. Note that you may read that the LIKE operator is case sensitive, but in my case I was using QGIS 3.16 and it was not case sensitive (I believe this is because only certain data types will require you to take case sensitivity into account - and in those cases you can just use the ILIKE operator).

Where you have variations of spellings, etc. then you might need to be a bit more creative - e.g. in the case of the 'st.' / 'st' / 'saint' map above. This was my query for that.

"TYPE" = 'populatedPlace'

AND 

("NAME1" LIKE 'st. %' OR
"NAME1" LIKE 'st %' OR
"NAME1" LIKE 'saint %' )

One thing that can be confusing is how to search for text that contains an apostrophe, because if you just put one apostrophe between single apostrophes (i.e. single quotes) and % symbols then it won't work. The solution is to add two apostrophes, as below.

"TYPE" = 'populatedPlace'

AND 

(

"NAME1" LIKE '%''%' 


How about the hyphen-hyphen-hyphen one? Well, that goes like this:

"TYPE" = 'populatedPlace'

AND 

("NAME1" LIKE '%-%-%-%')


If you just wanted to filter to show only places of your choosing, rather than filter to find some that contains certain bits of text, then it's just a case of using the IN operator and adding places in a comma separated list, in brackets, as shown below.

"TYPE" = 'populatedPlace'

AND 

(

"NAME1"  IN ('Aberdeen','London','Southampton','Middlesbrough','Liverpool')


Slightly different example below, where I've filtered the OS Open Names dataset using a different data type so that I can show all the 'lakes' in Scotland.

"TYPE" = 'hydrography'

AND 

(

"NAME1"   LIKE '%lake%' AND "COUNTRY" = 'Scotland'

Lots of these are tiny, but they do exist


Hopefully you get the idea - hours of fun, limitless possibilities for silly blog posts, but ultimately all this is actually very useful (as well as simple) and it's the kind of thing I do day-to-day in the course of my work.

Thanks for reading.

Friday, 11 June 2021

Arterial Cities

I was recently experimenting with some circular city layouts in QGIS in preparation for my Automatic Knowledge QGIS training sessions. After a bit of playing around, I ended up creating a set of city models in Aerialod. I'm calling them 'arterial cities' because they show cities criss-crossed by major arterial road and rail routes but also because they're kind of art as well. I've now done this for 50 towns and cities across Great Britain - using a 5 mile diameter circle on top of a 10x10km plinth. Here's a close-up of one of the London ones (below). I've posted a selection of the others further down, but if you want the original high resolution images for 50 towns and cities you can find them all in this folder. As one of my earlier tweets said, this might be a useful way to show how cities are divided by road and rail infrastructure and highlight things like urban permeability. If you want to learn about the method, scroll to the bottom of the page.

London in a new light

Some examples

The final set of 50 arterial city models include the major cities plus a selection of other places, chosen for a variety of reasons such as location, urban form, size, and a bit of personal preference. For London, Cardiff, Birmingham, Leeds, Manchester, Glasgow and Edinburgh I also created some large, re-coloured, and also zoomed-in renders and you can find them in the shared folder as well. Ok, so let's take a look at some of them, starting with Leeds, because it's really quite fascinating. I'll say a bit more about specifics below, but in all the graphics the rail lines are the highest features, followed by major roads and then minor roads, with buildings being slightly higher than the minor roads. This 'major' / 'minor' thing isn't perfect of course but that can always be tweaked.

Lots of big roads and rail lines (I've recoloured this one)

Glasgow has lots of roads and rails

Spot those motorways

Quite a pleasing ring road here

A real cross-cross of routes

No hills are shown in any of these

Lots of barriers

A very interesting one I think

Interesting compartments here

A real mix of old and new patterns


Hopefully there is something here that you find interesting. Let me also post a few of the zoomed in versions I did, focusing on the lettering in the top left of the plinths. I've changed the sun direction on these ones so it's coming from the north.





Little details that I find interesting
I've done quite a lot of visualisation with Aerialod before and written up a few tutorials but this is the first time I've made this kind of graphic and it wasn't planned. Even so I think the approach can be quite useful at highlighting city structures and there are a number of elements in the design that interest me. The first is when the text of the city name overlaps the city fabric, like in the Newcastle & Gateshead example below.

I just like the way it looks

I also like the way the railway lines in some cities appear to fray at the end, as the lines diverge to different platforms - like in the example below showing Euston, St Pancras and King's Cross in London. 

I just like the way these look

Then of course we have other features like roundabouts and grid patterns, for which we can use Milton Keynes as the pre-eminent example. I think the light and shadow does a good job here.

There are more roundabouts than you can see


In quite a few of the coastal cities it looks like there is only half a city, just because when I extracted the images from QGIS I did so using a distance of 2.5 miles from the city centres. So for places like Hull, Brighton and Bournemouth you'll see half a circle. But in other places it shows how they are connected to places across the water, as in Liverpool or in Dundee (below) and I quite like this tethering effect.


Tethered across the Tay

Also quite interesting, given that Euro 2020 is now in progress, is the way you can see many football stadiums and this can often help us get our bearings when we see them. So, for example, in the snapshot below we can see Celtic Park to the east of the image and Hampden to the south of it. This one below has also been recoloured, like the Leeds example above, but they are both available in the original gold tones in the shared folder.

Hampden: home of the Euro 2020 champions

There are loads more interesting details, so I'll leave you to look for yourself. For now, I'll leave you with Oxford because I think the urban structure here is both unusual and fascinating. 

Take a look at the real place on Google maps


Okay, so that's the graphics and what I think about some of the details. The rest of the post is about how I made these so read on if you want more information on that.

If you want to make your own prints out of any of the graphics on the shared folder, be my guest.


Stage 1: prepare your images in QGIS
For these graphics I used OS Open Zoomstack data and specifically the five layers below - they are layered in this order in the QGIS Layers Panel and the greyscale colour I used for each of them is shown in brackets. The greyscale colour values determine how much or how little Aerialod extrudes features because that's just how it works - white (#ffffff) will be the highest and black (#000000) shows up as a hole, just like the little Ordnance Survey credit in the bottom left of my images above. I quite like using black to give a kind of cut-out effect with text.

  • roads_national (0.35 line thickness, #4d4d4d - this is 30% greyscale)
  • rail (0.25, #595959 - this is 35%)
  • roads_regional (0.25, #404040 - 25%)
  • roads_local (0.1, #262626 - 15%)
  • local_buildings (#2b2b2b - 17% and with a 0.05 mm stroke width on the building polygons)
I prepared these in the QGIS Print Layout using the Atlas tool so I could automate it (see my previous tutorial on that) and I added text labels for the city, my twitter handle (bottom left) and the OS credit (bottom right). The background page colour in the Print Layout is #1f1f1f (12% greyscale) rather than black because if it's black it won't show up in Aerialod - although this is good if you want to make these without a square plinth. 

What they look like before adding them to Aerialod


Then I just exported the images at 1200dpi as png files straight out of the QGIS Print Layout. I have put all the images in this shared folder for anyone who wants to try making the graphics in Aerialod but doesn't want to do the QGIS bit.



Step 2: load the images into Aerialod and go wild with the settings
Aerialod is an interactive path tracing renderer for height maps in Windows and that's what I used to create the graphics on this page. It is pretty easy to get started but it can also be a bit baffling and there are so many options - many of which may not be obvious to the casual user. So, rather than mess about telling you what I did and making all kinds of suggestions I'll just show you the settings I used to create these images, in the screenshot below.

I exported these at 3840x2160 pixels

The only thing you need to be aware of is that it could take hours to render just a single image at this size - it all depends upon your computer. I have quite a powerful machine with 64GB of RAM, an i7-10700KF CPU running at 3.80GHz and a NVIDIA GeForce RTX3070 graphics card. Without the latter piece of equipment in your machine - if you just have a basic graphics card - the rendering could take hours, or maybe never.

You can spend hours, days, months messing about with the settings in Aerialod and there are an almost infinite range of possibilities in terms of light, shade, colour, tone, extrusion and so on. You can of course achieve similar things using tools like Rayshader or Blender but Aerialod is deceptively simple, yet impressively powerful in my opinion (once you get past the initial steps of what does what).

One thing that is not immediately obvious, or indeed obvious at all, is how to achieve the blurred/focal point tilt-shift effect you see in the example below for London. This is done by clicking on the area of the map you want to be in focus. When you do that, the Focus setting on the right of Aerialod changes and you see the blur effect everywhere you didn't click.

You can go round in circles with this kind of thing


Note the Focus value on the right

Want to achieve a totally different, less glowy kind of aesthetic? Well here are some other settings you might like.

The key difference here is the 'SKY' setting on the left

And one more, like above but slightly tweaked.

Intensity (left) and Exposure (right) adjusted in this one

And if you want to remove the plinth completely, you just need to reduce the Offset value on the right in Aerialod, as shown below.

I also quite like the plinth-less approach

This is the big plinth-less version, rendered as above


This is my way of doing it although if you see the original twitter thread mentioned above you'll see loads of different kinds of examples. 

Hey, could you do it with buildings extruded more realistically in 3D? Yes, but that's for another day.

In this example I've used building height data from Emu Analytics