Wednesday, 20 January 2016

The spatial dataviz web

So far on this blog I've covered visualising data and how lots of what we see today has a long history. Today I thought I'd write about the 'spatial dataviz web' or, more accurately, the sites and resources that I find useful when doing spatial data analysis and visualisation. So, it's really my spatial dataviz web but that didn't sound as good. This is really just the tip of the iceberg. If you're reading this then the chances are you might know all of it already but hopefully there's something new for someone. I've organised this around websites, data and software with a good bit of overlap between the categories. If there are any great tools I've missed I'd love to hear about it (@undertheraedar on Twitter). I'll add these at the end of the post.

An example of a map created using open data and open source GIS

Websites

QGIS Tutorials and Tips - this website, from Ujaval Ghandi provides a really good introduction to learning QGIS and includes stuff which total beginners, intermediate and advanced users alike will find useful. I really like the fact that it's all screenshots and words rather than screencasts (though for this, Steve Bernard at the Financial Times has you covered with this QGIS Uncovered video series).

Note the drop-down menu for selecting your language

Andy Kirk's 'Resources' - Andy Kirk is one of the world's most famous dataviz experts and the Resources section of his website is a real treasure trove of dataviz insight. Quite a few of the examples here are relevant for spatial dataviz, so it's well worth a look as he has created a really comprehensive list of examples here.

Go on, try to find something that isn't here

The Spatial Blog - I also really like Nicholas Duggan's The Spatial Blog as it's regularly updated, packed with useful information and easy to read.

CartoGroup at Oregon State University - this group's website is full of technically impressive, beautiful mapping. It's broken down into different section (Atlases, Demos, Research) and the people behind it have won a ton of awards.

Some lovely work here

Spatial.ly - James Cheshire's site is on most people's radar and his pages are packed with useful resources and great visualisations.

ESRI data dictionary - don't know your TIN from your Tobler? Confused about the difference between topography and topology? Have an exam question on run-length encoding? The ESRI data dictionary has you covered. A real gem but not all that widely known about in some quarters.

Ordnance Survey - cartographic design, via their blog. Great Britain's mapping agency has recently added a series of posts on 'cartographic design principles' to their blog, written by experts such as Charley Glynn.

Some great tips from Ordnance Survey


Data

Natural Earth - a very obvious choice, but if you haven't heard of it or used it, it's basically the place to go for global geodata. If you're looking for a shapefile of US states, or of the sub-national regions of Botswana, this is the place to go. They have cultural, physical and raster data covering the whole world and it's all free to download. In fact, the extremely generous Terms of Use state that 'No permission is needed to use Natural Earth. Crediting the authors is unnecessary'.

The brilliant Natural Earth

GEOFABRIK - we all know that a good chunk of the world has now been mapped by OpenStreetMap but sometimes it can be difficult to know where to turn to get the data in Shapefile format. That's where GEOFABRIK from Karlsruhe in Germany come in. They have compiled, archived and regularly update Shapefiles for all areas of the world, downloadable in national subsets.

A free service, from Geofabrik GmbH

Mapshaper - if you're making interactive maps for the web, there's a good chance that you'll want to reduce the level of detail (and file size) in any spatial data you use. For example, if you download data from somewhere like OS OpenData in the UK, you'll get really high quality data but also big file sizes. There are loads of ways to simplify geodata and reduce the file size, but mapshaper's online interface and simplification algorithms are so simple to use. Created by Matthew Bloch of the New York Times.

Still my go-to tool for most simplification tasks in GIS

NASA - they have a ton of data, but unless you know exactly where to look it can be a bit bewildering for some users. The NASA link here is actually to Derek Watkins' 'SRTM Tile Grabber' which allows you to easily select an area of the earth and download elevation data for almost anywhere on earth. There are also loads of socioeconomic datasets available via NASA's SEDAC centre at Columbia University. NASA also have the amazing Visible Earth image series, which I used to create the little animated globe below.



Project Linework - this is less well known and is a 'library of handcrafted linework for cartography, each designed in an aesthetic style, as it says on the website. Downloads come in ai, geojson, shp and topojson formats. It's really cool.

A really nice initiative, via Daniel P. Huffman

These are the ones I use most often or like this best. But the best GIS data list on the internet is curated by Robin Wilson at the University of Southampton in the UK. It's fantastic and includes everything from glacier outlines to global terrorism.


Software

I've tweeted and blogged a lot about QGIS over the past few years, and with good reason. It's free, open source and constantly improving. It's not the only GIS I use (I still dabble in MapInfo from time to time and use ArcGIS for some geoprocessing) but most of the time I use QGIS now so this is my go-to tool for 80% of tasks. But this section is about free software, so here's an overview of the main free software I use.

QGIS - it might sound mysterious and/or odd to outsiders, and possibly a little threatening to some non-open source advocates but QGIS has over the past five years in particular established itself as a leading geographic information system. Certainly - in my opinion - the best of the open source GIS tools and many see it as a serious 'rival' to proprietary packages. But I prefer not to think of it in terms of rivalry. Instead, I am just in awe of what we can now do with it and grateful to people like Nyall Dawson, Nathan Woodrow, Anita Graser and Tim Sutton.

GIMP - a very dodgy name, for obvious reasons, but a great, free image manipulation programme. I often use this for post-GIS image processing but somewhat less these days because QGIS can now do quite a lot in the Print Composer on its own.

Blender - I'm a real novice at this but am posting it here because I've seen what it can do. For example, Steve Bernard's recent experiments with 3D animated globes.


IrfanView - this is not a spatial tool at all, but a free image viewer and manipulator, developed by Irfan Skiljan, a Bosnian graduate of the Vienna University of Technology, originally from Jajce. It's simple but powerful and really great for things like re-sizing large batches of image files or converting from one image format to another. I also use it a lot for basic editing of screenshots and photos. There are many more powerful tools but I love this for its simplicity and efficiency: two attributes I value highly.


Okay, so not exactly the 'spatial dataviz web' in its entirety but it's my version. If anyone wants to suggest an addition - beyond the more obvious stuff like ColorBrewer and ESRI's GIS Tools for Hadoop, I'd love to share it. Also, you'll find that pretty much everything you could want in terms of data is covered by Robin Wilson's list and for tools and software by Andy Kirk's Resources. I've also had a few suggestions for more, so see below.


Addendum

CartoDB - didn't put this in at first because most people are aware of it but now added at the suggestion of Maarten Lambrechts. I've used CartoDB quite a bit and have, more recently, experimented with different projections.

Mapbox - another one I filed under 'too obvious' but it's made the cut after Maarten's suggestion. Given that Prof Alex Singleton said the following, I think it's worth putting in here: "The new @Mapbox Studio is one of the slickest outputs I has seen from the GI industry in a while! Stunning work and a great barrier remover!"

Any more you want on the list? Let me know.