One degree of population

Here I am at the end of the year with another bit of map fun, this time inspired by Bill Rankin's world population histograms from 2008 (check out his website if you haven't already - it's amazing). For no good reason, other than curiosity, I decided to do some mapping of the world's population by single degrees of latitude and longitude. Actually, curiosity is I think quite a good reason, so see the first graphic below, making sure to read the caution on the map (tl:dr not all cells are the same size). Update 28 December 2018 - I added a version doing it by longitude as well. There's a gif at the bottom of the page and the mp4 files are in the repo I link to below. I also did a backwards-forwards looping gif - see below.

I think this backwards and forwards version works well

It makes more sense to compare areas at the same latitude

In the image above, I've picked out the ten most highly populated 1 degree cells but of course they are not all the same size - they get smaller the further away from the equator you are, because the lines of longitude narrow towards the poles. But even so, I think it's probably safe to say the cell where New Delhi is has a lot of people in it, as do the other nine featured above.

Now, we all know that most of the world's population live in the northern hemisphere (I make it about 87%), which is hardly a surprise because that's where most of the land is (68%) but I thought it would be interesting to do an animation of the above map, and that's exactly what I did. Here are some stills from the animation (below) and here is a direct link to the repository where I've put a 15, 30 and a 60 second animation. You'll probably have to download the files before viewing them.

Before that, I've embedded a version from YouTube, with an audio track.

If you live below 30 degrees south you are quite special

People seem to like living where the land is

Above 30 degrees north is where it's at 

I'm one of the 5% - are you?

And by longitude? I did that too and those files are at the GitHub repo link above but here's a fast gif of it for now.

How I did this

The data I used for this is the GHSL world population dataset that I've written about before. I've been using it recently in a paper so thought I'd do something a little different with it. This is the latest data, for 2014. I used the 1km gridded population dataset and re-aggregated it by lat/long so it's not a precise match with lines of latitude and longitude but it's close enough to be a reasonable estimate. I mapped it in QGIS and exported a few thousand frames in QGIS Atlas then patched the video together using FFmpeg.

How to create an interactive 3D landscape model in QGIS 3

It's been a while since I've posted a QGIS tutorial so with the release of QGIS 3.4 I thought I should do another one. This post is about how to create an interactive 3D model in QGIS 3.4 which you can then export and view in a web browser, like the one you see below. You can see the interactive web version here - it's great fun to play around with, particularly when you press R and watch it rotate. This kind of thing was possible in previous versions of QGIS but I think there are some great improvements with the new version of the Qgis2threejs plugin that you create this with (thanks to Minoru Akagi).

This is Kauai, an island in Hawaii

Make sure you read the instructions - very useful

The basics

All you need to make this work in the way that I have is a) QGIS 3.4, b) a DEM layer (that is, a raster layer with elevation data in it), c) some kind of satellite or aerial imagery, d) the Qgis2threejs plugin which you install via the Plugins menu in QGIS and e) a bit of time.

There are many sources of digital elevation model data but for this I used NASA's 30-meter resolution elevation data from the Shuttle Radar Topography Mission, which is very easy to download thanks to Derek Watkins' tile grabber. You need to register (for free, takes only a few minutes) to get the 30 metre resolution data but if you can't be bothered doing that you can get the coarser 90m SRTM data without registration. As is normally the case with this kind of thing, the area I was interested in was split between two tiles so I patched them together in QGIS (Raster > Miscellaneous > Build Virtual Raster). I chose Kauai in Hawaii because of its size and varied terrain.

Derek Watkins, I salute you

For satellite or aerial imagery, the usual approach would be to add a layer in QGIS using the QuickMapServices plugin (e.g. Google Satellite layer) but in this instance I used a Landsat 15m image from the State of Hawaii's Office of Planning GIS pages. This is a bit better quality than those available via QuickMapServices so I used this.

When you have the DEM data and the Landsat imagery in QGIS 3.4 it will look something like this:

Make sure your image fills the whole map view in QGIS

The preview window is a great addition

The process

You then activate the plugin by clicking the icon (see above) and it's a case of playing around with the options until you are happy with what you see. Since qgis2threejs has excellent documentation, I won't go over all the steps and options here but I would just add the following points - and see point 5 in particular.

1. Set the level of vertical exaggeration via Scene > Scene Settings... In my case the vertical exaggeration is 3.0.
2. In Scene Settings under Material you may be baffled by the options - Lambert, Phong, and Toon. I've exported three versions of Kauai, one for each - you can see the difference by opening them in a web browser but basically the lighting will look different depending upon which one you choose.
3. Right click individual layers to set the Properties. On the DEM settings you'll get a much crisper visual with the resolution set to 400% but the file size will be bigger and it will most likely load slower in the browser.
4. If your preview looks wrong, with weird, impossibly high extrusion, change the projection in the bottom right of the QGIS window, as below.
5. In previous versions of qgis2threejs, when you exported the view from QGIS it would automatically open the visualisation in a web browser. When you try this from QGIS 3.4 it will attempt this but if your default browser is Chrome or Internet explorer then you'll get an error message. This is a known issue so you'll have to view in Firefox or Edge (I haven't tried other browsers) to overcome it. This is only a local problem - it's fine viewed over the web in any browser.
6. If you look at the style panel in the image below, you'll see that I've colorized the raster layer. I did this to give the image a kind of 'morning glow' look to it. See below for a comparison between a colorized and non-colorized image.
7. Via Scene > Decorations you can now add in a north arrow (very cool, since it is dynamic) and a footer label to add text (which can be html, like I used to style the font and add the QGIS image).
8. If you want to add labels to a point layer, you can still do this by right clicking the layer in the preview and setting the options. It's no longer Times New Roman font either, so that's an improvement too. You can see an example with labels in a previous map I did of Madeira. I've added another screenshot below to show you how to set this up.
9. You can also now set the view to be in Perspective (the default) or Orthographic, which is another nice advance.
10. There are many things you might want to tweak manually that you can't do from within the plugin options but they can be done by editing the html and the js and css files (see below for more).

Don't worry, just change the CRS

Subtle difference, but I like the effect (this is a looping gif)

Setting labels relative to the DEM surface will usually look good

When you export the scene, qgis2threejs will create a number of files and folders which you can then put on a web server so that the world can see how amazing you are. Or, you can just view them locally on your own computer (but not in Chrome or Internet Explorer, like I said above).

Making it look even nicer

You may now want to change the background colour, font size and other bits and bobs. To do this you need to edit different bits of the .html file, the .css file and the .js file created by qgis2threejs. These files will be named as below.

• index.html (this is the default name, you may have changed it to something else) but it is here you can change the page name and footer text, for example.
• Qgis2threejs.css - you can edit this to change the north arrow position (I put it in the top right, but it's not there by default) and the colour of the sky.
• Qgis2threejs.js - you can edit this to change the font size (edit the value of the MinFontSize under label, where you can also edit the leader line colour) and the colour of the base sides and the overall lighting colours. 

This is how you edit the north arrow size and position (css file)

This is how you edit the sky colour (css file)

This is how you edit the footer text (html file)

This is how you edit the side colour and lighting (js file)

This is how you edit the font size for labels (js file)

If you want to edit the font, search for "font-family: arial, sans-serif;" in the css file and change it to something else. You can see some different options for that on this page. In the example below I just changed the font size to 24 and added font-family: "Comic Sans MS", sans-serif; to the css, as below.

You can use lots of different fonts here

This is the bit you want to edit

Last words

The last thing to say is that you should take some time to figure out how to control the visualisation. On mobile, it works really nicely too and upon export there is now a mobile option. It's worth playing around with the different export options to get a sense of what can be done. My favourite thing is to click on a point and you'll see the lat and long plus elevation and then you have two options - Zoom in here or Orbit around here - self explanatory but the orbit option is really cool. You can of course make it rotate just by pressing R.

Comic Sans has its uses, but this isn't one of them

Okay, that's it. 

A Kepler tutorial (plus data)

Earlier this year I wrote about kepler.gl and how I visualised some Bay Area commute flow data with it. It's an amazing tool from the fantastic Uber Visualization team. Take a look at their website for more info, plus nice group photos. But this is about Kepler so to give you an idea of what it can do, see below for the first example, which shows daytime population density in the London area for 1km grid cells. We'll get to how you do this in the rest of the post, which includes the data I used. I've also shared the config file and data in a single json file that you can upload to replicate the image below.

This shows daytime population for 1km x 1km cells

Before going any further, I should say a few things. First of all, you should scroll down all the way and see all the examples on the Kepler home page, or you're missing out because Kepler can help you make all kinds of maps. Obvious, but easy to miss. Second, it seems your data should be in WGS84 format otherwise it won't display. At least that is what I found. Third, when you try to export images using the export tool in a web browser you might not see any output until you clear your cache. I had this problem but it was solved by clearing my cache. Finally, just be aware that you can put labels on top of your map content in the layer options - which I've also screenshotted below. All data and images are shared here. Have fun!

This is the home page - but scroll down too

Clicking Get Started takes you here - I uploaded geojson

The data appears on the map - you can see the tooltip popup

On the left, I'm choosing which column to map

Now we can see daytime population density

Now I'm making it 3D, by clicking the button, top right

Now I set the column to be used for the 3D effect 

In this case I've used the maximum value of 100

I've clicked 'Show wireframe' to make it look a bit clearer

Here's where you can change the contents of the tooltip popup

And now how you put the place labels on top

Some place labels appear - different for each basemap type

See, you get more labels with this lighter basemap

Now I've used some filtering options to show the high values

Only the highest value shown now, with tooltip data

I've added a legend using the button on the right

Now you can share/export your viz

If you see nothing here, clear your browser cache

This has the legend too - best to change the layer name first usually

How to export the whole lot - or just the config

Doing this will export everything, including the data

Okay, so that's about it really. I've just added some more images below that I extracted previously. There is sample data for quite a few different parts of the UK, as you can see.

This was me just playing with the 3D extrusion slider

Central Scotland

Liverpool and Manchester

Belfast

The Greater London Pizza Region

Cardiff and Bristol area

Data source: I've used the CEH's UK Gridded population data for this. It's available under an Open Government Licence, with the requirement to add the following statements: Contains data supplied by Natural Environment Research Council and © NERC (Centre for Ecology & Hydrology). Contains National Statistics data © Crown copyright and database right 2011. 

Data note: if you download the data (registration required, but open and free) you will get a residential daytime grid for the UK and a daytime population grid for the UK. I combined these into one file. The only problem was that the original data has the population data stored as text so I converted it to integer and added a column showing the difference between daytime and residential population, plus a ratio field too. I then extracted data for some areas of the UK, exported them as geojsons from QGIS (using WGS84 projection) and put them in a shared folder.