10x10km City Squares

I've been thinking about little projects I could do with the Great Britain buildings GeoPackage I shared previously on here and have come up with the idea of comparing the urban fabric across the country. For this I drew inspiration from previous work by Geoff Boeing, though instead of 1 square mile of urban fabric, the images below are 10x10km squares. Look at the gif first, then I'll explain the method and share all the static images. All images are ordered roughly from north to south so we start in Inverness and end in Canterbury. Update: 1 Sept 2018. I decided to manually select 144 towns and cities and do a 12x12 version of the original city mosaic below, just to make it more interesting. if the image below crashes your browser, try this smaller one. Want all the individual tiles? I put them all here in a Dropbox folder.

Click to enlarge - huge version (12,000 x 12,000) here

65 cities, at two seconds per frame (squares are 10 x 10km)

What I did here was take some Ordnance Survey open data on places in Great Britain and then filter it so that only places with an 'Extra Large' place label were visible. See this blog post for an explanation of what that means. This left me with the 65 cities you see in the graphic above. The image below has 64 just because I wanted to do a 8x8 version. In all these images all I have shown is the most detailed buildings layer available as open data from Ordnance Survey - you can download the lot from my buildings page. 

10 x 10 km squares for 64 British cities

The centre point of each square is where the Ordnance Survey place label is located. If you moved things north or south, or east or west in some cases you'd get a bit of a different picture but the points are generally in the centre of these places.

I've posted all the individual images below in case anyone is interested. This was also a bit of a test for QGIS and I did actually do about 1,600 of them just to see how my computer would handle it so if you're really sad that your town or city isn't on here let me know and I'll share it.

Method: in case anyone is interested this was done in QGIS Atlas with Ordnance Survey open data, and ImageMagick for the layout. I tend to use IrfanView for batch resizing and so used that here. The font - Raleway - is from the free Google font pack available online. The colour is #222222 (RGB 34, 34, 34).

That's all for now.

Watching the ships go by

A bit of summer map fun today, with no real purpose apart from learning about UK shipping density and routes, plus making some shiny maps. Anyone can look at live ship position data, and I've enjoyed it in the past, but it was only when my colleague Ruth Hamilton pointed it out to me that I realised the raw AIS data was available for the UK, via the Marine Management Organisation. So here I am again.

The most recent data I could find was for 2015, so I downloaded it and mapped it on top of a bathymetry layer and then added in a land surface layer, and nothing else. In the maps below I've split the dataset up into shipping types so you can see where they go. This gives us quite a good picture of marine traffic patterns around the UK, and further afield. For technical details, including sources, see the bottom of the page. Click an image to enlarge it, or try opening in a new page/tab to see it in more detail. I've put the high resolution images (400dpi) here if you want to look at them in full size.

Note the location of North Sea oil/gas platforms

Note the loop to the north west of Shetland

Lots of activity off the east coast of England

Quite a bit of traffic to offshore wind farms, perhaps

Lots on the south coast, but also in The Minch

Kind of as you'd expect, but also quite striking

Hmm, quite a lot of cargo

There are also quite a few tankers

I really find the fishing vessels map fasinating

Quite a lot of traffic between England and France

These UK-level overview maps are quite interesting - at least to me - and you can see them pretty clearly at a more local level in the full resolution maps, but I've also done a zoomed-in set for each shipping type, as you can see below. 

Clearly a lot of non-port service craft from Aberdeen

Also a good bit of non-port service traffic around Shetland

Some port service traffic patterns for Scotland and England

Port service traffic is quite busy in the English Channel

Some dredging/underwater activity in the north of Scotland

Note the back and forth patterns off the south west coast

High speed traffic in the Irish Sea area

High speed traffic - to offshore wind farms?

A good bit of military and law enforcement traffic here

Lots of passenger traffic here, as you'd expect

Quite stylish routes I reckon

All hail Caledonian MacBrayne! (and NorthLink, obvs)

England to France: take your pick

There is cargo in UK waters, didn't you know?

There is cargo in the Irish Sea too - it's everywhere!

You want tankers? We got tankers

Fishing in The Minch, and notice the ship tracks down Loch Ness

Note the tracks of the ships compared to the bathymetry

Lots of fish to be caught here

Some nice recreational sailing off the west coast of Scotland

Like I said at the start, this is just a bit of summer map fun with an different kind of dataset. I think it's quite interesting and when you map it in QGIS and add a bit of styling plus terrain and bathymetry data you have some shiny maps. The full resolution images are the ones to download if you really want to see things in detail. There's also a zipped folder with all the images if you want to download them all in one go.

FAQs

What does AIS stand for? Automatic Identification System.

Where can I read more about all this stuff? The Marine Management Organisation have some great documents explaining the data and what it means.

Can I get data for other parts of the world? I'm not sure, but you can look at live AIS shipping data for the whole world on Marine Traffic. They also have a density maps tool, as you can see below.

This is all marine traffic, on the live web map

How did you make these maps? I used QGIS 3.2. I played around with the styling quite a lot to achieve a look that I was happy with and in particular the feature blending modes (that's what makes the lines look shiny where there are higher densities) and I also applied hillshades to the terrain and bathymetry data, as well as some colorization.

Why did you add the bathymetry layer? I wanted to see if it in any way explained any of the traffic patterns, particularly for fishing, and I think it does. Look closely at some of the maps to see what I mean.

Why did you make these maps? Because I like making maps and find this data interesting.

Can I use these maps? Yes, feel free.

Why did you not include any maps with place name labels? I wanted to keep them super simple. Plus it is of course great fun trying to figure out what's what.

Data sources

The AIS data I used is available under an Open Government Licence (UK) and can be downloaded for several different years, with 2015 being the most recent. The terrain data can be downloaded for free on the EU's Copernicus/Land Monitoring Service website. The bathymetry data can be downloaded from the EMODnet interactive web map via the 'Download products' link. 

The AIS dataset for the full year is pretty big, more than 3 million records, and it's a little unwieldy. But it is also very rich, with data on the start and end time and date for each track, ship length, ship width, ship draught, plus lots of other time and date variables and of course the vessel type. There is an 'unknown' vessel category but I didn't map that here.

Visualising Bay Area Commutes with Kepler.gl

A couple of months ago I saw on Twitter that Uber launched a new visualisation tool called kepler.gl. Apart from the fact that it looked seriously amazing, it also looked very useful but I hadn't managed to have a play with it until this week, so here I am with some visuals and a few words.

I've already looked at commuting in the San Francisco Bay Area quite a bit in the past (including this in Wired) so I had the data already and when Bay Area urbanist Alex Schafran got in touch about this very topic a couple of days ago it prompted me to dust off my Bay Area commute data and have a play around. These are the results. 

If you want to have a play yourself, I've put the csv in a shared folder, alongside a ready-to-upload json, some images, and a bonus csv of commuting and image folder for England and Wales as well, which is good fun.

Each line connects two Census Tracts, which have a resident population of just over 4,000 people. 

Commutes to San Francisco (100+) coloured by origin county

Bay Area commutes (colour = destination county)

Commutes flows (50+) within Alameda County 

Commute flows (150+) within Alameda County

Same as above, but with a different base map

Bay Area commutes (100+), coloured by destination county

As above, but showing the Filter tools in Kepler.gl

As above, but showing a tooltip for one flow line

Commute flows (50+) in Contra Costa County

Commutes to Santa Clara County (100+)

Commutes to San Francisco County (50+)

Commutes to San Francisco County (100+)

San Francisco to Santa Clara County commutes (all)

The final four images are just to show you the styles and settings I've used with the data, so that you can understand how it works a little better if you want to have a play around with it yourself.

You can see that lines are based on orig, dest x and y coords

I've applied a single filter here, but you can add more

These are the fields used in the tooltip - very useful

You can choose different base maps and turn layers on or off

That's all for now. Like I said, you can play around with this data if you want or even try the England and Wales dataset I created, which has some useful fields for filtering (e.g. local authority name, distance, flow volume).

Notes: Kepler? Presumably named for Johannes Kepler, the German mathematician. The Bay Area commute data includes commutes within and between the nine Bay Area counties of Alameda, Contra Costa, Marin, Napa, San Mateo, Santa Clara, Solano, Sonoma, and the City and County of San Francisco. The dataset is one I put together for another project, and comes from the American Community Survey. The England and Wales file is from the 2011 Census and includes all commutes of 25 people or more. I've found that you can't always export images from Chrome directly but it worked in Firefox. I haven't tried on Safari but I'm guessing image export works on that. Have a go yourself with the data I used to create the images above.