Showing posts with label map. Show all posts
Showing posts with label map. Show all posts

Thursday, 19 January 2023

Animating a proportional symbol map in QGIS

Many people who read this blog will be familiar with Karim Douïeb's fantastic 'transition between surface area of US counties and their associated population' map, shown below.

I wanted to see if I could create something a bit like this in QGIS, using a geometry generator approach, so that's what this post is about. My end result is shown below, and read on for more information about my method. See the notes at the bottom of the page for a bit more detail. The version embedded below will be a bit small, so here's the full size, high-resolution mp4 if you want to take a closer look at it.



What's a geometry generator in QGIS?

In QGIS you can style layers in the normal way, as in any GIS software - e.g. by using a blue fill on a polygon and a white outline - or you can use the geometry generator option to style a layer in a different way. For example, if you had a set of points representing populations, you could turn them into spikes with the height based on their populations. 

You could display lines as a series of points, and all kinds of other options, all without having to create any new files. Or you could convert areas into proportional symbols, as I've done above.

Okay, so you may have read this and still be like 'what the what? what the heck does this mean?' and that's pretty normal actually because it's not necessarily an easy concept to grasp without seeing examples. That's why I created a sample file where you can drop it straight into QGIS and play with different geometry generator styles, as in the tweet/ thread below.


You can get really pretty fancy with geometry generators in QGIS and do all sorts of interesting things, some of which you can easily find if you search for geometry generator qgis on Twitter. 

The thread below is another example of what I'm talking about here - it ends up with me making a tartan t-shirt based on a geometry generator style and some point data in QGIS!


Okay, fine, but could we create something like Karim's animated map in QGIS? Well, I tried and I think I got reasonably close, so read on for more about it.

Polygons to proportional symbols with QGIS geometry generator + Atlas

Okay, first things first - if you want to replicate this style, you can find the QGIS project file plus the individual map layers in my training data folder - directs links to each file are provided below. Whereas Karim did this for the entire US, my example only includes the lower 48 states. If you already know what you're doing with QGIS this will be fairly straightforward but anyone with a bit of experience should be able to download the files and replicate my steps. Here are the files you'd need to replicate my work.

If you want to replicate my project exactly, you'll also need to have the Comfortaa font installed because that's what I used for the labels and in the Print Layout. You'll also need to open the Print Layout in the project and make sure you activate the QGIS Atlas (via the Atlas menu then Preview Atlas). I did this in QGIS version 3.24 so if you're having any issues try using the same version, or a later one. Last of all, if you want to add the QGIS logo like I have, you will need to change the file path for the Picture in the print layout. I put the QGIS svg logo file in the same folder as the files above.

As you move through the Atlas page by page, each of the counties changes from a polygon to a proportional symbol, from west to east, with the size of the symbol based on the number of votes in each county. This is not perfect, but it's a more proportional representation and it's all about testing the concept here. This is what it looks like half way through the set of QGIS atlas frames.

Half way through the animation

Once I got everything set up, I exported each page of the Atlas as a single png and then combined it into an mp4 as an animation, using ffmpeg. I've written about that process in a previous post. For the final version I added a pause at the end, then reversed it so it loops back and forth.

So what about the QGIS project file itself? Well, if you download it you'll see it contains:
  • A US states layer, to show the state outlines - simple black colour outline
  • A US cities layer, filtered to show a selection of cities across the US, and also labelled
  • A grey US county backdrop layer so that when the polygons turn to proportional symbols we still have some background behind it
  • And on top of the grey US county layer we have the red/blue county layer showing which party won in each county at the 2020 US election. This is the layer that has the geometry generator style in it. 
  • Plus the QGIS logo svg in the bottom right of the print layout. 
If you want to look at the all-important geometry generator style, just go to the Layer Properties for the red/blue us-election-2020-lower-48 layer and then double-click on the blue colour patch symbol then Geometry Generator to see the geometry generator style for the blue areas and then do the same for the red areas, as shown below. 

You can also access this via the Symbol dropdown in Layer Properties


This is the geometry generator style I used - it's pretty basic, but works well:

CASE

WHEN x($geometry) <  @atlas_pagename 

THEN  buffer(centroid( $geometry),7000+ "total_votes"/30 ,50)

ELSE

$geometry

END

What does it do, and how does it work? Well, the first bit after the WHEN uses the x coordinate of county centroids to change the shapes from west to east - this is because I used the longitude of each county as the Atlas page name. That's why @atlas_pagename appears in the expression. You'll need to think about this for a while if you're not used to how the Atlas tool works in QGIS but if you look at the Print Layout you'll see each page is named using x($geometry).

The next bit draws a buffer around the centroid of each county, to 7000 metres, then scales it using the total vote count, divided by 30 (otherwise the circles are HUGE, but I did experiment with this and scaled it loads of different ways). The 50 value just makes the circle smoother. Why 7000 metres for the buffer? Well, if it's much smaller then you can't actually see it at the scale I'm mapping at. 

The ELSE $geometry bit just says to QGIS to draw the polygon as usual until the Atlas page name (which contains the longitude) meets the WHEN condition above.

Confusing? Yes, probably if you've not used this before but feel free to download the project and have a go.

You'll also notice that in the Atlas tab I've done some other things, like set the page name using the format 0001, 0002, 0003, 0004 and so on. 

This is what the Atlas layout looks like at frame 2000, below.

What it looks like in the QGIS Print Layout


Okay, so can we create an animated proportional symbol map in QGIS? Yes, we can. 

Is it as cool as Karim's? No, but it does demonstrate the power of QGIS geometry generator styles and shows the kinds of things that can be done - and that's what this post is all about. 

This is example is fairly simple really but if you've not used QGIS in this way before it can be quite confusing - that's why I shared the simple geometry generator file plus the more complex project file shown here.

Want to replicate this but can't quite get it all to work? Feel free to get in touch.

Notes

If you want to actually create new geomtries (e.g. a new shapefile or geopackage) using a geometry generator expression, you can do it via 'Geometry by expression' in the Processing Toolbox.

Are the proportional symbols sized exactly to match the population? No, because the size is based on the buffer radius, but if you use a bit more clever maths in the expression you can get it spot-on. But if you do this, the smallest symbols will likely disappear.

What's the point of all this? The point is to test what kind of things are possible with geometry generator styles in QGIS and to share this with people who might be interested.

That's all for now. I may add further notes if there are any questions but I'm putting this post here as a more permanent resource for anyone interested in how to do this kind of stuff.

Sunday, 9 October 2022

Cometmaps

In one of my many 'playing around with map stuff and then posting it on twitter' adventures, I recently posted a kind of comet map - see below. This started off as something completely different but once I had the basic idea I then wondered what kind of data I could apply it to, and that's why I did it to the UK political map of 2019. Read on below for how I did it, as well as how you can replicate it pretty easily in QGIS. Scroll straight to the bottom if you're just looking for the 'how-to', as well as a bit more on how this kind of thing might work on a US county-level election map.

First of all, this was done in QGIS. I'm sure it could be done lots of ways, but I did it in QGIS using something called 'geometry generator', which is basically just a little bit of code that can be used to style things more fancily than just clicking a button and selecting a colour and shape. 

My starting point was actually just to see if instead of displaying things as polygons I could display them as lines that went from one corner of an area's bounding box to another - in this case from the far south-west to the far north-east corner of a shape's bounding box. That was easy enough - and the result is shown below. I did it with UK local authority areas, then moved on to doing it with constituencies - which you can grab from my Automatic Knowledge resources page.

A basic line, within each area's bounding box

Then I thought about applying a random degree of rotation to each line, just for the sake of it, and that's what you see below. This is probably not that useful unless you want to make a map that looks like a pile of sticks but all the same it's interesting to see that it's possible and perhaps I'll find a use for it one day. 

Big pile of red sticks drifting into space

So then I went back to no rotation and thought about buffering the lines, but using a tapered buffer so that one end was thick and the other wasn't. I applied this to each area and also used a gradient fill so that the colour faded along each shape. The result of this is shown below. This one was also done on local authority boundaries rather than constituencies because I was playing around with both at the time - but then I wondered about applying it to a political map and doing something more interesting, so the cometmap came to life. 

A cometmap emerges from the darkness


The result was that I decided to use geometry generator in QGIS to do the following, based on the results of the 2019 UK General Election - and my goal here was to use the comet trail as the main device in the map to show who came second and how far behind they trailed. I realise some people look at the maps and perhaps see the opposite but for a little map experiment I thought it was a) interesting and b) potentially useful (probably with some further development).

  • Use a circle in the centroid of each constituency, using party colour to indicate who won that seat.
  • Use a tapered buffer behind each coloured circle, with the colour of the party who came second.
  • The length of the comet trail is dictated by the winning margin - so if a party is WAAAAAY behind the winner then it gets a longer trail. So this map draws attention more (deliberately) to the second placed party. But of course any of this can be edited.
  • The size of the circles and buffer is a few thousand metres but of course that can be adjusted too. See the results below.
Dot with no trail? Means it's very close

Loooong blue trails? Conservatives waaaay behind

Mostly short trails = mostly quite close results

THE LIBERAL DEMOCRATS ARE COMING!!!!

Okay, so that's about it. 

Read on if you're a map nerd and you want to replicate the style yourself.


How to replicate this style in QGIS

  1. First of all, you'll need to grab the UK General Election 2019 results file from my Automatic Knowledge resources page. Use the GeoPackage because it's easiest.
  2. Add the file to QGIS. 
  3. Optional step. For exact duplication of my maps, make the QGIS background very dark grey (e.g. #111111) and make duplicate the UK general election layer and make it dark grey, both fill and stroke colour (e.g. #333333) so that when you make the comets they look brighter and have a dark backdrop and you can also get your bearings from the plain dark backdrop. 
  4. Download the cometmap layer style file to your computer. This is just a standard QML file and you'll apply it in the next step.
  5. Go the Layer Properties for the layer you want to apply the cometmap style to and then via the Style button at the bottom of the Layer Properties window, and then it's Load Style... browse to the cometmap-ukge-2019-example.qml file on your computer, then it's Open, Load Style and OK - and that should be it!
  6. Once you've done this you can inspect the properties of the style yourself and edit them if you want to - see screenshot below.
How the trail is generated - rotation set to zero right now


Note that I am using a geometry generator twice in the screenshot above - once to create the circle and once again to create the trail. You'll see that the colours are set using an expression - if you click Simple Fill you can see this when you click the little Expression button (see below). Note that this is done both for the trail and the circle.

The '97' in the html colour values sets the opacity


Could you apply this to, say, a county-level US election map? Yes, but I think it would probably require a good bit more thought otherwise you'd end up with a really wild map. But here are a few examples.

File these under 'hmm' for now.

Done very quickly, just to test the method

You can see where this needs work - but it could work

Just a few tweaks needed here!

That'll do it for now

Needlemaps - the next frontier

I had a bit more of a think about it all and for the US I wanted to see if I could create a kind of needle or dial that showed which way a county leaned. using the 2020 US election results file. Well, turns out that this works reasonably well. Not perfect but it's a good start. See below for the results and then if you want to replicate it yourself and then play with my settings, download the qml needlemap style file for QGIS and then apply it to the election results file and have lots of fun.

Lots of ways this could be edited/improved


Yeah, not surprising

Big 'lean left' energy going on here

Actually quite interesting here I think

Lots of interesting stuff here, for sure


Note that in the needlemap style above, size = scaled to population, roughly, but of course I had to make some a minimum size otherwise they would be invisible, so I also added opacity by population to make them super-faint if they are tiny. Colour = blue for lean Dem, red for lean GOP. Lean angle is set from the Dem/GOP ratio from 2020.

Thomas Gratier on Twitter suggested we could call these pinball maps, and that's also a good fit as the needles look very much like flippers on a pinball machine.


Tuesday, 28 December 2021

Map animations with ffmpeg

This post is about how to create video animations from a sequence of individual images, using a simple but extremely powerful command line tool called FFmpeg. If you're scared of code, command line stuff and weird computer language in general then this post is for you. Part of my sneaky plan here is to lure you in with enticing visuals so that you overlook a little bit of the complexity - which is really not that complex, I promise. If you're already a Jedi-level coder but maybe haven't used ffmpeg, welcome too. If you're somewhere in the middle, also hi.

I'll try to demystify the mysterious and show you how to do some interesting things without too much pain. To begin with, here's a video I made using ffmpeg, from a series of map frames I exported from QGIS Atlas. But you don't need to know anything about QGIS to complete this tutorial - I've already exported all the individual map frames for you and shared them down below. We're going to fly from London to New York today, and also from London to Rome.


There are a few parts to this tutorial, as follows. If you found this post somehow randomly online and were just looking for how to install ffmpeg and how to run a couple of commands to turn a series of images into a video, then this post is also for you - please read on!


Sections


1. How to install ffmpeg on Windows, Mac or Linux (this can sometimes feel like the hardest part, but it's not very difficult at all, I promise). [Okay, let's be honest for a moment. It can feel a bit painful on Windows, but once you've done it once (in under 15 minutes) you're all set!]

2. Check ffmpeg is actually installed. This is important, but very quick.

3. Get some image frames that we can turn into a video. I've added the two flight-based image sets I refer to in this blog, as well as some additional bonus ones you can play with.

4. Use ffmpeg in Windows Command Prompt or Mac/Linux Terminal and run some commands to create videos.

5. Experiment with different settings to produce different results.

6. Some tips, such as how to name files if you're using QGIS Atlas to export the individual frames, and how to do a few other cool ffmpeg things.


1. Let's install ffmpeg 

I have ffmpeg installed on my Mac, on Linux (I use Ubuntu) and on Windows. I mostly work on Windows but when it comes to ffmpeg the syntax we'll use for entering commands is the same. I've covered each install method below, so you can just skip straight to the bit that matches your operating system. The Windows install may feel a bit complex if you're not used to it but it really doesn't take long.

To install ffmpeg on a Mac, the easiest way - and what I do - is to first install Homebrew and then type in a command into the Terminal (see below) wait a little while and then ffmpeg is installed.  

The command is brew install ffmpeg

On my Linux machine (on Ubuntu), I just use the standard sudo apt install ffmpeg command in the Terminal (see below), enter my computer password and then wait a short while until ffmpeg is installed.

Windows user? You can ignore this one too

Okay, so you're on Windows and you want to know how to install ffmpeg? Here we go.

This can actually be pretty confusing as there are loads of different instructions online, and lots of different ways you can do it. And then the website and install links can be confusing too if you're new to it. So, if you are new to all this, here's what I recommend. 

  • Go to the ffmpeg download page and look for the Windows logo (probably in the 'More downloading options' section). As I write, the Windows logo is the blue and white squares one but this may change in future so, to be clear, it's in the 'Get packages & executable files' section of the download page. 
  • At this point things can get confusing because all you might actually want is a familiar .exe file you can click on to download and then run - it's not like this with ffmpeg though. BUT DON'T WORRY! Even though the ffmpeg project only provides source code (you can just ignore the Download Source Code link) there are ready-made exe files to download - see the next step. 
  • Now you can go to one of the options for downloading a Windows exe file - and for this you may see a couple of options - I went to the 'Windows builds from gyan.dev' link so I recommend you do too.
  • What now? Help! Which one do I download? Why are there so many download options!? What the heck does 'git master builds' mean? Don't worry about all that, just read on.
  • Scroll down the page a bit and you should see the release builds section and then one downloadable file called ffmpeg-release-essentials.zip (this is what it was called as of 27 Dec 2021) - when you click to download it I recommend saving it on your C drive in a new folder called ffmpeg, so the save location would be C:\ffmpeg - it doesn't actually matter too much where you save it, just so long as it's on the hard drive of your computer. I just find it easier to save it in a dedicated ffmpeg folder directly in the root of my C: drive.
  • Okay, almost there - once it downloads (it's about 77MB) you need to unzip the .zip file and then when you do you'll have a folder called something like ffmpeg-4.4.1-essentials_build. The numbers tell you what version of ffmpeg you have - currently (as of 27 Dec 2021) it's version 4.4.1. 
  • Look inside this folder and you'll see some more folders - and if you look inside the bin folder you'll see ffmpeg.exe, ffplay.exe and ffprobe.exe. How do we make ffmpeg actually run though? Well, not by clicking these exe files. See below, we're almost there.
  • Since we'll be running ffmpeg from the Windows Command Prompt (see below for an example of what this actually looks like) we first need to tell the Windows operating system where the ffmpeg binary files are (that's why the folder is called bin) so that when we type text into the Command Prompt later on it runs ffmpeg for us. We're almost done - I promise! Read on below.

  • Now we need to tell Windows where the ffmpeg.exe file is on our computer by setting the Environment Variable. Enlarge the image below to see screenshots of the steps you need to follow now, as described here: - 1. Start typing environment variable into the Windows search box and then once you see the 'Edit the system environment variables' you can click it. 2. in the System Properties window click Environment Variables... towards the bottom and then 3. With Path selected in the list of System variables, click on the Edit... button at the bottom (this will allow any computer user to use ffmpeg - if you just want to be able to use it on your own Windows user profile you can click the other Edit... button in the User variables section of the window - also making sure Path is selected in the list of variables) then 4. we need to click New and then we paste in the folder location of the ffmpe.exe file on our computer (i.e. the path, hence why we're putting it in the Path section), as in 5. in the image below where I have right-clicked the address bar in Windows explorer and used Copy address as text so that I could paste it in at step 4 (so, I pasted in C:\ffmpeg\ffmpeg-4.4.1-essentials_build\bin as my New Environment Variable path because that's where I put my ffmpeg.exe file). Once we do all this we can click OK until all the windows are closed and then the Environment Variable for ffmpeg will be set and we don't have to do this again! Woo.

We only need to do this once!


Windows Command Prompt (we'll get here soon)

How to set the Environment Variable (on Windows)


2. Check ffmpeg is actually installed

Now we need to open a Terminal or Command Prompt on our computer (in case anyone isn't clear - the thing we type ffmpeg commands into is called the 'Command Prompt' on Windows and the 'Terminal' on Mac or Linux - for some this will be obvious, for others it may be entirely new). On Windows this is easily done - I do it by typing cmd into the search tool in the bottom left of the taskbar and then click where it says Command Prompt - and voila! I see the Command Prompt window - all black and shiny and empty. It's also very easy on Mac and Ubuntu.

To check whether ffmpeg is actually installed on your computer just type the following text into the Command Prompt or Terminal window (note the dash before version here - this is a common feature of typing ffmpeg commands).

ffmpeg -version (there is no space between the '-' and 'version')

If you've installed ffmpeg correctly, you'll see some results telling you the ffmpeg version on your computer and lots of other stuff that may mean nothing to you. If you're on Windows and you get an error at this stage, just double check that you set the correct folder path for the Environment Variable at the step above. You can see that on one of the computers I did this on, I'm using version 4.2.3.

May I ask when the fun begins?

Okay, ffmpeg is installed so that's quite enough of that. Let's have some fun now!


3. Let's grab some image files we'll use to make the videos

I've created and shared some image sequences we can use to make videos from. There are two folders of images at the link below that we'll use here (I've added other image sets for you to play with afterwards). One is individual frames of a flight from London to New York and one is from London to Rome. In both folders you will just see a series of png image files. I made both of these using QGIS Atlas and you can find out more about how to do that in this QGIS training workbook I put together. 

Content folder

The lhr-jfk folder has 758 individual map frames so it's bigger in terms of file size and will produce a longer animation, whereas the lhr-rome folder has 161 images and is smaller and will make a smaller animation. But I have shared both of them so you can play around with them as you please. You can use either, or both - it's up to you. 

The easy way to get the files is just to click on the lhr-jfk.zip and download it and then the lhr-rome.zip and download it. Note: I mean it's quickest to download the already-zipped folders rather than trying to download the unzipped image folders.

Once you download the folders (and unzip them), you'll see lots of individual map files - I made these in QGIS and have named all the files sequentially with padded zeros - e.g. 001.png, 002.png and so on. This makes the animation process with ffmpeg a bit simpler.


A snapshot of some of the lhr-jfk png files

I've also shared a txt file with ffmpeg commands that you can use to copy and paste stuff.


4. Let's run some commands in ffmpeg

At this stage I normally like to use a blank text document (e.g. Notepad on Windows) as a kind of intermediate copy/paste text editor so that I can be sure I've typed things correctly and can edit stuff there before pasting it or typing it into the Command Prompt or Terminal window to run a command. But of course just do what suits you best - this is just how I like to work.

Okay, so let's make a video in a couple of simple steps.

4.1. We need to make sure that the Command Prompt/Terminal is working in the correct folder. In Windows we can just type cd followed by the path of where our image files are - like in the example below where I have navigated to my lhr-jfk images folder

Copy the folder path with a right-click

cd C:\Users\XPS13\Downloads\lhr-jfk (I'm using this file path because that's where I saved the lhr-jfk image folder on my computer - you'll just need to type cd followed by the file path of the location you saved the images to)

Note that if you downloaded images to an external hard drive with a drive letter like G or F or something else, then you don't use cd to change hard drive location, you just type in the drive letter followed by a colon (e.g. f:) and then within an individual drive you can used the cd command followed by a file path to navigate to the correct folder.

Now we're ready to roll

On Mac or Linux we can just right-click a folder and choose to open a Terminal at that folder's location and we're all set. 


4.2. Look at the command below and then read the explanation. After that you can just copy and paste it into the Command Prompt/Terminal on your own computer and it will create a video from the png file I've given you. This may look utterly baffling right now, but that's totally fine. It will become clearer.

ffmpeg -r 30 -i %03d.png -c:v libx264 -s 1920x1080 -r 30 -pix_fmt yuv420p flight-30-30.mp4


Explanation

The ffmpeg bit just means we're telling the computer to use ffmpeg.

The -r 30 bit means the input frame rate is 30 frames per second.

the -i %03d.png bit tells ffmpeg what the inputs are - in this case it is a series of sequentially-named png files with numerical names padded with zeros. That's why it says %03d.png - if we had files name 00001.png, 00002.png and so on then we'd use %05d.png

the -c:v libx264 bit tells ffmpeg to use the libx264 encoder. What? Just know that this will work and it works well.

the -s 1920x1080 bit is about the size of our video. Here I've used the standard 16:9 aspect ratio screen size of 1920x1080, even though the files I gave you are only 1020x583 pixels in size. But as you will see this doesn't matter - you could replace this with 1020x583 and see if it looks different. Just note that if you put in a dimension that isn't divisible by 2, you'll probably get an error message. Best to use even numbers here. You can just add or subtract pixel values by 1 so they are both even.

the -r 30 bit is the output frame rate. You don't have to enter this - ffmpeg will default to 30 if you don't but I like to include both input and output frame rate. This can be useful for low frame rate videos when, for example, you want Twitter to play them properly.

the -pix_fmt yuv420p bit relates to pixel format and all you really need to know is that this works and is a good choice. If you don't include it, ffmpeg will still produce a file but it may not actually play.

the flight-30-30.mp4 bit is the output file name - what our final mp4 file will be called. I've just called it flight-30-30.mp4 because this tells me the video is a video of a flight with an input and output frame rate of 30 frames per second. Call it whatever you want, but it's a good idea to give it a descriptive name that has some clue to what's in the file. I output the file to the mp4 format but you can export to all sorts of video formats, including mov, avi, wmv and many more. To do that you'd just write flight-30-30.mov instead of using flight-30-30-mp4.

Okay, so go ahead and paste the command below into the Command Prompt or Terminal and hit enter and then look in the same folder as your image files for the resulting video. See below for my screenshot before I hit enter. 

ffmpeg -r 30 -i %03d.png -c:v libx264 -s 1920x1080 -r 30 -pix_fmt yuv420p flight-30-30.mp4

I've highlighted the different parameters here


Once you hit enter it will run for a little while, anyone walking past your computer will see complicated text scrolling down and think you're a coding genius, and then a few seconds later you'll see a mp4 video file in your working folder (this should be the same folder where all the images are). 

Is this The Matrix, or what?

And here's the video it produced - note that instead of the hundreds of megabytes of images we originally had, the video file size is very small - 693KB for mine.



5. Experiment with different settings

What now?

Well, I think it might be enjoyable to experiment a little, particularly because an input frame rate of 30 gives a pretty slow flight from London to New York (less so with London to Rome). If you're using the London to New York files, try this command to increase the flight speed (note that it's the input frame rate that determines the speed)

ffmpeg -r 120 -i %03d.png -c:v libx264 -s 1920x1080 -r 30 -pix_fmt yuv420p flight-120-30.mp4

Adjust the input -r frame rate (that's the -r parameter just after ffmpeg) and just experiment with the video size settings too if you like - but ideally keep it in the 16:9 aspect ratio and make sure you use even numbers for the -s size parameters. 

Let's say you want to have the flight arrive in New York or Rome and then have the video pause for two seconds. How might you do this? Well, once we have our initial video, the command below will do the trick - and you can adjust the number to increase or decrease the pause at the end. Just replace input-video.mp4 with the video you want to add the pause to and then replace output-video-with-pause.mp4 with your new file name.

ffmpeg -i input-video.mp4 -vf tpad=stop_mode=clone:stop_duration=2 output-video-with-pause.mp4

Want to reverse your video? Fine, try the command below (again, replace the input and output file names with your own). The -vf reverse command is the bit that reverses the video. In this instance, vf stands for video filter and you can always search for other ffmpeg video filters and try them too.

ffmpeg -i original-video.mp4 -vf reverse reversed-video.mp4

Planes don't generally fly backwards of course, but this technique can come in handy.


Finally, let's say you want to convert one of your new videos to another format, you can do this:

ffmpeg -i original-video.mp4 new-video-format.mov


How about a gif? Well, that's a bit more tricky. If you just do the same as above but use .gif as the output file type then ffmpeg will create a gif but it won't look good at all. Then if you search the internet on how to do it, you'll probably end up lost for days. 

So, to avoid any further pain, try the command below - replacing the input video file name with the one you want to convert - and see below for the resulting 1MB gif. Note I have used 960 as the image width in the scale= parameter below but you can always experiment with another value - bigger or smaller.

ffmpeg -i flight-90-90.mp4 -vf "fps=30,scale=960:-1:flags=lanczos,split[s0][s1];[s0]palettegen[p];[s1][p]paletteuse" -loop 0 flight-90-90.gif

Making gifs with ffmpeg can be quite confusing!


6. Some tips, notes etc

How to get good at this kind of stuff? Well, a lot of Googling and reading things on Stack Exchange and random blogs is the way to go mainly. But I'd also recommend trying to get to grips with one or two basic commands - like the ones above - by changing the settings and seeing what happens when you edit different parameters.

I'll end with a list of tips that I find useful.

  • If you're making a map and then plan to make it an animated mp4 or gif, then it's a good idea to think about where the file will end up so that you can plan ahead - e.g. does it need to be screen size like 1920x1080, does it need to be square, does it need to be a specific colour? Will the play and pause controls obscure any important bits of the video? I'd really just recommend thinking backwards from the end use before you begin.
  • If you're exporting map frames from QGIS Atlas then you can easily export the files with numeric names, padded with zeros. That's what I normally do, by using an expression-based file name on export - as shown here, and see screenshot below: lpad(@atlas_featurenumber,3,0)
  • Save your ffmpeg commands in a text file and make a note of what they do. Use ffmpeg as a video converter if you ever need to convert, say, a mov file to an mp4. I've added a text file with commands to the Content folder so feel free do use this too.
  • Remember that there are loads of ways of doing things in computerland. There isn't normally one single, best way. There may be better ways than those I have shown you here, but these methods work well. However, from time to time things change with ffmpeg and sometimes you may need to do a bit more Googling.
  • Want to learn more about ffmpeg? Then why not take a proper course on Udemy - I like this one.
  • Want to extract a single frame from a video, as a png, at 8 seconds into a video? This will work: ffmpeg -ss 00:00:08 -i flight-30-30.mp4 -frames:v 1 out1.png (as with all examples here, you can replace the name of the input file after the -i parameter with the name of your own input file, and the name of the output file to whatever you want to call it)
  • Want to extract a 4 second section from a longer video, starting at 2.5 seconds into the video and ending at 6.5 seconds? Try this command: ffmpeg -i flight-90-90.mp4 -ss 00:00:02.500 -to 00:00:06.500 -c copy cut-4-sec.mp4
  • Note that in the example above, the format for time is hh:mm:ss.sss where there is a dot rather than a colon between the seconds and milliseconds units.


This makes the process much smoother

  • What else? Probably the most important things in all of this are a) making sure you have ffmpeg installed properly and b) that you take some time to play around with a command that works and then change the settings to see what it does.
  • You don't have to use a sequentially-named list of files, but if you don't then ffmpeg may not include files after a gap - e.g. if you run the command to create a video on a folder with files 001.png, 002.png, 003.png and then 007.png you'll just get a video with the first three files. You can search online for how to get round this.
  • Want to add a bit of visual 'noise' to a video? Try this (may be a bit slow) ffmpeg -i flight-30-30.mp4 -vf noise=c0s=60:c0f=t+u noise.mp4
  • How about when you want to add some audio to a video you've made - like if you have an mp3 file you want to add as music? Well, that's fairly simple and you can do it like this: ffmpeg -i input-video.mp4 -i input-audio.mp3 -map 0 -map 1:a -c:v copy -shortest video-with-audio.mp4 - and note that -shortest will trim the video to the duration of the shorter file - so if you have a 5 second video and only 3 seconds of audio, the video file will only be 3 seconds long. You can remove -shorten if you don't want this to happen.
  • Other tips? Just spend some time searching, but this is a good place to start. 
  • Sometimes animated maps are nice, sometimes they are a bit weird and sometimes they can be confusing - so just use your judgement and have fun!

This ended up a bit longer than expected but I'm hoping you may find it useful - I know I will when I inevitably forget some of this and have to come back here to remember. If you spot any errors or have any questions, just get in touch.

Read more About FFmpeg.


Postscript for geo nerds

Why do this? Well, it can be useful if you want to show how things change over time or space or when you are looking to emphasise movement or growth or change of any kind. Also, since we are not restricted to flat, paper maps like in the olden days it's sometimes interesting - and useful - to use animation. People have been doing this for decades and probably most famously Waldo Tobler in the 1960s and early 1970s, as in this video put online by Matthew Wilson. 

And, of course, the 1970 Tobler paper in which the famous 'first law of geography' is stated is actually called 'A Computer Movie Simulating Urban Growth in the Detroit Region' so I thought it would be useful to help demonstrate one way to make a 'computer movie' today - hence this ffmpeg post. I met Waldo in sunny Santa Barbara back in 2013 (and before that he was a reviewer on one of my papers) and he explained some of how they used to do things - so ffmpeg is incredibly easy by comparison!

Anyway, about the 'first law of geography' thing - in a follow up collection of papers more than three decades after the original piece, Tobler wrote 'I was just having fun' in relation to some of the work reported in the original 1970 paper when he stated the 'first law'. This is the final section of text from the 2004 paper and I think it's quite interesting what he says, including the bit on diversity of viewpoints, and also on having fun with animations. 

It's also a good way to learn stuff, in my opinion, which is partly why I do this blog and tweet map stuff. For me, despite all the noise, acrimony, animosity and general shenanigans on twitter, the contributions of so many people in geo, data, viz and other fields have led me to contemplate many ideas and references with which I was previously unfamiliar so my hope is that in sharing some of what I know, others may find it useful in ways that I couldn't contemplate myself. But yeah, I was mainly just having fun making an animation.

Tobler, 2004, p. 308


Sunday, 28 November 2021

World Population by Latitude

If you search online for 'world population by latitude' you'll quickly find quite a few results, including the great analysis by Bill Rankin on his Radical Cartography blog from 2008, which uses population from 2000, and also includes population by longitude. There's also a nice interactive version on Engaging Data, plus similar things on my One Degree of Population piece on here a couple of years ago and my global population density spike maps. In this post I look at world population by single degree of latitude using data from 2020. There are maps and stats below, but let's begin by looking at what appears to be some kind of alien eyeball but is in fact world population by single degree of latitude, where redder areas = more people at that latitude and bluer areas = less people. I used 2020 WorldPop data to do the calculations and all the maps were done in QGIS, as usual.

Weird alien eyeball? Or population map?

It's better viewed as a flat map, obviously, so I've posted that below. I've labelled quite a few places across the world and of course there are places at some highly populated latitudes that have very few people (notably the Sahara Desert) but you get the idea here: redder = more people at a given latitude.


Note the numbers on the left - including population data

If you view this image in full size you should be able to read the population numbers for each single degree of latitude - I've posted a zoomed in extract of this below. The most highly populated single degrees of latitude, according to my analysis? Here are the top 10 that I get:

  1. 25-26° North: 278.6 million people
  2. 26-27° North: 271.7m
  3. 23-24° North: 244.5m
  4. 24-25° North: 237.4m
  5. 22-23° North: 235.3m
  6. 30-31° North: 234.8m
  7. 31-32° North: 226.2m
  8. 34-35° North: 215.8m
  9. 35-36° North: 214.6m
  10. 27-28° North: 198.3m
And then down in 28th place we have the first entry in the southern hemisphere with 6-7° South having a population of 117.5 million people, according to my calculations. 


These are numbers I calculated myself

It's difficult to do anything with this colour scale these days without bringing to mind Ed Hawkins' warming stripes, but I think it's useful to use the red/blue colour ramp here because the most highly populated places are generally the warmer ones and the colder latitudes on the whole have fewer people. But of course there's another factor here, and that is about where all the land is. So first let's look at land vs population and then we can look at a density version of the above map - i.e. population density by latitude that presents the stripes based on the population density of land at all the different latitudes.


This is what I get for land by latitude


And population by latitude - people don't like living in the sea

You can do a little crossfaded gif to get the comparison between these two different elements, so that's what I did below. You can see it in the still images above, and in the gif below I have highlighted the single degree of latitude with most people and most land. 


Finally, we have a population/land by latitude gif

Let's hit pause on the gif, mid-fade, and then add some labels so we can make a bit more sense of what's going on. That's what I've done below - click to enlarge in order to read the labels.



Here's another globe-style view of the population data but this time from a different perspective.


Some surprises here perhaps

Okay, so the next three images are similar to the 'world population by latitude' image above but this time they are actually population density by latitude. That is, the redder latitudes are the most densely populated. I did this so that it only takes into account land, which is kind of where people like to live for some reason. You'll note in particular here there is a dark red stripe in the southern hemisphere that cuts across Santiago, Buenos Aires, Cape Town and Sydney, among other places. Note that I experimented by dimming and then removing the latitude colours over the sea in the second and third images because it kind of makes sense to do that.


Density makes a bit more sense

Dimmed the sea because people don't live there

Removed the sea latitudes, but I like this less


Other odds and ends

Once I'd done this I experimented with different map projections and views, hence the weird population latitude eyeball at the top of the piece. I also experimented with views from above and from below, so you can see them below in the first two images. The first one in particular looks like a bloodshot eyeball to me.


Where do most people live? Just eyeball it

Antarctica is pretty big

What else? Well, I did a few other versions, including just coloured lines of latitude, then adding only land, adding a chart and that kind of thing so I've posted them below too. These are more experimental versions - some I like more than others but the one with just colours and land I think is quite interesting.


Population stripes, with land

Just the population stripes

I took away only the land here

In this one I just added a chart of population by latitude

This is the original, as above, but without a white border

Does the final result on all this depend upon what dataset you use? Do you get a different result by using, say GHSL data vs WorldPop data? Or what about NASA's GPWv4, or even another source? Well, probably a bit, but the difference between GHSL and WorldPop wasn't huge and WorldPop is also available for 2020 and GHSL isn't so I used WorldPop. But you can see the difference between GHSL and WorldPop below anyway.


GHSL

WorldPop

GHSL WorldPop GHSL WorldPop GHSL WorldP

This all basically makes sense and is not particularly surprising but then again it's nice to be able to put some numbers to all this and make some maps of it. Plus it's interesting for training data and for working on map methods and techniques in QGIS.

People live here

Here's a screenshot of what this looked like in QGIS before I exported some of the final images. The font is Righteous, by the way.


Print Layout on left, map view on right


So there we have it.