From sensors gathering climate information, to posts on social media sites, from purchase transaction records to cellular location data, and from digital media such as images and video to machine data recorded by factory and plant instruments, most organisations are drowning in data.

We’re creating around 2.5 quintillion bytes of information every day, according to IBM. As much as 90 percent of the data in the world right now was created in the past two years. Financial market data, server logs, medical imaging files, metrological data, and genomic research information – there’s just no end to it.