An academic collaboration between Cambridge and Oxford universities has used Big Data to analyse the final checksum of the effects of a major London tube strike – and found that it brought net benefits by forcing commuters to experiment rationally with how they use London’s tube system, and to find more optimal alternatives which they adopted thereafter. Additionally the research brings into focus both the inadequacy of the famous ‘circuit-diagram’ tube map and the shortcomings of existing travel planners for the city.

Finally the study not only contains a vivid example of the insights that Big Data can offer productivity workflows and systems, but also a strong indicator that entropy – in the sense of the tendency to stop at the first ‘workable’ solution to a problem, regardless of whether it is necessarily the best – is costing government, business and individuals significant amounts of money and inconvenience.

The Benefits of Forced Experimentation: Striking Evidence from the London Underground Network [PDF] by Shaun Larcom, Ferdinand Rauch and Tim Willems is a particularly readable and lucid account of the analysis of the usage of 18,113 Oyster cards both before, during and after a major London Underground strike which closed 171 out of 270 tube stations for two days. The large data set, which covers the period of January 19th 2014 to February 15th 2014, contains millions of data points and was anonymised before being provided to the researchers by London Transport.

The scientists restricted the study to a baseline of Oyster-using commuters who ‘forged on’ through the service disruption, and whose movements could be studied both before and after the strike. The characteristics of the strike itself made the study meaningful, since enough of the London tube stations and routes were left operational to provide auto-piloting commuters with some alternative travel choices that they would actually have to think about, evaluate and try out. Since the data set was confined to Oyster-based travel within the underground network, complete closures would have yielded no meaningful evidence of commuters ‘adapting and evolving’ to the problem facing them.

The results, the researchers contend, lend increased credence to the often-criticised ‘Porter Hypothesis’ as posited by Harvard Professor Michael Porter. The hypothesis, which deals with environmental regulations, argues at a deeper level that systems coalesce and retrench significantly before they reach their optimal configuration; that, in essence, disruption is a necessary imperative towards improved systems – whether of infrastructure or of travelling to work.

The report concludes that ‘despite the fact that a substantial share of travelers is likely to have received help from online journey planners, from previous disruptions to the network (calling for earlier experimentation), as well as from the experiences of others, they were still not maximizing. Given that the challenges faced by businesses are arguably more complex than the commuter-problem analyzed in [this paper] it seems likely that many firms are not operating efficiently either. Consequently, the Porter-hypothesis…might be less implausible than its critics, such as Palmer, Oates and Portney (1995) and Schmalensee (1993), have argued.’

Three attempts to understand the underground: Top: the Evening News’s effort from 1910; Middle: Harry Beck’s ‘circuit’ map as evolved from its roots in 1933; bottom: the geographically-logical version proposed by user ‘Sameboat’ at http://tinyurl.com/svgtubemap

Three attempts to understand the underground: Top: the Evening News’s effort from 1910; Middle: Harry Beck’s ‘circuit’ map as evolved from its roots in 1933; bottom: the geographically-logical version proposed by user ‘Sameboat’ at http://tinyurl.com/svgtubemap

The inadequacies of the ‘circuit-diagram’ London tube map

The report additionally posits that the faulty initial configuration of many commuters’ journeys is likely ascribable to bad input data in the form of the 84 year-old ‘circuit diagram’ designed by electrical draughtsman Harry Beck in 1933. The map seemed an ingenious replacement to a series of increasingly confused efforts to visualise the pell-mell growth of the London Underground in a useful manner, but has come under criticism in recent years for its low (0.22) accuracy relative to the actual geography it traverses. The Oxford and Cambridge researchers additionally note that neither the existing publicly-provided maps nor official route-planners such as Transport For London take account the disparate speeds or crowding conditions between the various tube lines:

‘Next to commuting time, travelers are initially also uncertain on many characteristics of the various available alternatives. How crowded is a particular line at the preferred time of travel? Is the route from the exit station to the final destination convenient (is there for example a supermarket along the way, or does it happen to take you past a place that serves good breakfast)?

‘An important way in which these various uncertainties can be reduced, is by actually trying the available alternatives i.e. through experimentation. ‘

The tendency to stop at the first workable solution

In an uncharacteristically philosophical flourish, the report concludes by posing the question ‘when was the last time that you did something for the first time?’. It is an extraordinarily interesting question in terms of both the practical implications of Big Data – which seeks to identify, characterise and classify trends in much the same spirit as marketing companies, and for many of the same commercial imperatives – and of economics in general.

In terms of the philosophy of free market economics, the study brings to light two competing models for the practicality of gaining economic benefits from Big Data: the identification of what people actually do, which brings with it the possibility of increasing the value and exploitation of the most popular trends observed; and information about what they don’t do, and why they don’t do it – which hints at untapped markets, new governmental and infrastructural economies and ‘painless’ rationalisation.