Thursday 31 December 2015

Playfair's Statistical Breviary, 1801

I recently managed to get myself a copy of William Playfair's Statistical Breviary, from 1801. The whole thing is online but the fold out pages with the charts haven't been scanned properly so it's only with the hard copy that you get the full view. Why is the Statistical Breviary of interest anyway? Well, because it's one of the most important texts in the history of statistical graphics and where the pie chart first appeared (below) - though many wish it never had! Playfair had already come up with the line graph and bar chart in The Commercial and Political Atlas so he's rightly held up as a pioneer in his field - even if this work was largely overlooked at the time. 

At least he didn't try to make it 3D

Fast forward over two hundred years and you can see that his ideas on graphical display are still pretty potent - and many analysts use them daily without realising. For example, the chart options in Excel (below) contain four types which he invented (line, pie, area, bar). The pie chart gets a bad rap, and Edward Tufte says 'the only thing worse than one pie chart is lots of them', but it is useful for showing parts of a whole. It's just that it's so often abused. This New York Times magazine piece from 2012 does a good job of setting things straight - and the comments are pretty interesting too. But today's blog isn't about pie charts. Instead, I thought I'd share some interesting nuggets from the Statistical Breviary.

Microsoft Excel - Playfair looms large

As is so often the way with old books, much of what is written could have been penned today. For example, when Playfair says that 'making an appeal to the eye when proportion and magnitude are concerned, is the best and readiest method of conveying a distinct idea' (p. 4). This definitely applies to the best of today's data visualisations. He also said that 'geographical knowledge has long been considered as necessary for persons of both sexes who wish to acquire any tolerable degree of general information; in so much that, next to ignorance of one's native language, nothing betrays a want of information so soon as ignorance in matters of geography' (p. 4-5). Also potentially useful as advertising copy for a university geography department...

Translation: 'dataviz rules' - note the long s

You can also sense Playfair's frustration with the data he used, perhaps most tellingly in his use of the phrase 'scanty gleaners' (p. 10). Perhaps he was one of the early advocates of open data when he said that 'were the aid and support of public men obtained in collecting statistical knowledge, great progress might be made in it at little expense ... but so long as that is not the case, individuals will find themselves reduced to the situation of scanty gleaners, not that of men carrying home an ample harvest' (p. 10).

Are you a 'scanty gleaner'?

Why on earth was Playfair motivated to do any of this? Well, there were probably many reasons but the most obvious is that he was trying to make data engaging and useful. He says as much in the Preface and Introduction but also on p. 16, where he notes that 'no study is less alluring or more dry and tedious than statistics, unless the mind and imagination are set to work'. Indeed, that's part of the motivation behind this blog.

Playfair wanted to inspire students here
The Statistical Breviary, as the name suggests, then proceeds though the reporting of various trade and other data for a number of empires and dominions - from 'The Empire of All the Russias' to the 'small states' of Genoa and Parma. His descriptions of some of the nations are - from today's vantage point - rather amusing!

The Turkish Empire - 'the finest portion of the world is in possession of the Turks' (p. 20).

Denmark - the people are 'still of a very brave nature' (p. 28).

Poland - 'this extensive and fruitful country, better peopled than any of the neighbouring nations, and with a brave race of inhabitants' (p. 30).

France - 'the French are violent, quick, generous, and enthusiastic' (p. 32) - note that Playfair lived in Paris from 1787 to 1793.

Britain and Ireland - I was interested to note here that Playfair, despite being a Scot, writes in this section that 'England is now the first commercial and manufacturing nation' (p. 36). It's unclear whether he's conflating England with Britain and Ireland, or just referring to England alone, but it seems like the former. Surely the kind of thing to set off vexed comments if he was writing on the Guardian DataBlog (being a Scot myself, I'm sympathetic to this). The population is listed as 14 million and the 'Chief towns' as London, Dublin, Edinburgh, York, Liverpool, Bristol and Newcastle.

Portugal - 'may be considered as Spain in miniature' (p. 42), which I thought was quite amusing. 

It's also interesting to note that Playfair decided for some reason to cover 'Hindoostan' in the Statistical Breviary, and this final section also has circle charts and data - but it looks like a late addition to me. When we look back on texts like this it makes me think that there isn't really that much new in the world and that genuine innovation ('shewing, on a principle entirely new' in Playfair's lingo) is actually quite rare. If you're interested in reading more, take a look at an online version. If you want to go and look at an original Breviary, you might have a library near you with a copy, though access can be a little complicated because it's so old. For now, I'll finish with a little pie chart joke that's been doing the rounds since last year - though nobody seems to know the original source so if you find out do let me know.

Original source seems to be this tweet

Did all his best work in London, naturally

Saturday 26 December 2015

Arctic sea ice, 1978-2015

During the recent Paris climate talks, I began to think more about the topic of arctic ice melt because I really didn't know too much about it, beyond the general knowledge that it's disappearing fast. So, naturally, I decided to see what data I could get my hands on to explore it further and understand it better. I discovered that the 2015 Arctic sea ice minimum was the fourth lowest ever and that NASA's Goddard Media Center have produced some amazing visuals on it. More excitingly for me, the National Snow and Ice Data Center (NSIDC) publish richly detailed data and images on sea ice, going back to 1978. This seemed like a good topic for my first post on the new blog. So, to kick off, here's the extent of Arctic sea ice in November 1978, the first month covered by the dataset (click any of the images to enlarge).

The first month for which data are available
The NSIDC do have their own image animation tool which is derived from monthly images, but I wanted to do my own versions which include only images from September and March - the months of minimum and maximum ice extent respectively. If at this point you're wondering how the data are derived or other questions, check out the FAQ section on the NSIDC website. If you're looking for more on sea ice in particular, check out the NSIDC sea ice news section on their website - it's a great resource. Also, to provide a little context, Arctic sea ice extent in November 2015 was just over 10 million square km - just larger than Canada. It normally peaks in mid-March at about 15 million square km and is at its minimum in September - with a mean of about 6 million square km (twice the size of India). However, you'll see in the animation below that it bottomed out at only 3.4 million square km in September 2012. 

Every September from 1979 to 2015 (animated)

I've also pulled out the static image from September 2012 as it's easier to see this way. If you're planning to go all the way from the Atlantic to the Pacific in your container ship, be careful - it's not plain sailing yet, as this Washington Post story notes well.

Arctic sea ice, record seasonal minimum in 2012

It's also really interesting to look at the month when sea ice is at its maximum extent. In March each year the ice is more than double the area compared to September, though there has been some significant variability in the minimum lately. You can see the extent of Arctic sea ice for every march from 1979 to 2015 in the animated GIF below.

Every March from 1979 to 2015

The data are a great resource for looking at trends, but of course not everyone is interested in trends. For example, the Daily Mail ran a story on this kind of thing in 2013, noting the growth in ice cover between August 2012 and 2013. It also grew in extent between March 2011 and March 2012 (see below) but of course this isn't consistent with the long-term trend of sea ice loss documented by NSIDC. The overall story is one of ice melt.

March 2011 maximum was 14.6m sq km / 5.7m sq miles

March 2011 maximum was 15.2m sq km / 5.9m sq miles

There are so many powerful visuals on this topic, including NASA's own great small multiple showing all months from 1979 to 2014 and a recent Goddard video on the 2015 minimum. One thing I've seen less of is a single year in a simple animation, so that's what I've done below for 2014 in another animated GIF. 

Full year animation for 2014

That's all for now. I'll be returning to my normal urban data topics in future but I thought I'd kick off the new blog by doing something a little different and learning something new. In the spirit of the title of the blog (see the About section for more on that), I wanted the first post to have some 'stats, maps and pix', and this seemed like a good fit - plus I also learned something new about a phenomenon which has occurred entirely during my lifetime. Finally, here's the September and March animations again, but this time at half the speed.

September ice extent - at 1 frame per second

March ice extent - at 1 frame per second

Data: I used data from the National Snow and Ice Data Center, which is a fantastic resource. I've barely scratched the surface of it but I now have a much better understanding of what's going on thanks to their work. Here's a direct link to the FTP directory containing the Shapefiles. The NSIDC is based at the University of Colorado in Boulder and is affiliated with the US National Oceanic and Atmospheric Administration.

Software: I used QGIS 2.10 for the mapping and GIMP to create the animated GIFs.

Citation: Fetterer, F., K. Knowles, W. Meier, and M. Savoie. 2002, updated daily. Sea Ice Index. Boulder, Colorado USA: National Snow and Ice Data Center.