Big Data – the need for integration.

Sandy Pentland of MIT points out some of the key issues about Reinventing Society in the Wake of Big Data here. Data ownership is important because it concerns the main driver, as well as one of the main barriers, to developing Big Data services.

The main driver for developing Big Data services is value creation – not just commercial value for firms and the value that customers get from great services – but also the possibility of sharing some of the value that personal data generates with the person that generated it. It’s the main driver because people commonly do things because they get something in return that they value.

One of the main barriers to developing Big Data services is fragmentation and the fragmentation of ownership is critical. Big Data is the personal diary of everyone and everything. This personal diary is sometimes called the ‘data exhaust’ . It is co-created by customers consuming a service and the service provider as it produces the service. I.e. as all the service provider’s staff and machines produce the service.

But who owns this flow of bi-products, this data exhaust? Is it the service provider? Is it the service provider’s partners in its supply chain, the telecommunication company, the content providers, the payment services partner, the security and authentication partner or others? Or is it the customer?

What makes matters more complicated is that data is always copied rather than consumed. So potentially all stakeholders could use the information – for different purposes. And maybe here lies the solution.

If the value of data depends on its use and it is infinitely copyable then we should be asking who owns the right to use it for specific purposes rather than just asking who owns it? Maybe the problem of ownership fragmentation is a licensing problem. Framing data ownership as a licensing problem may partly help to solve the problem of controlling the use of customers’ personal data as well all the problem of sharing out the value that it can create.

Another Big Data fragmentation problem that Sandy Pentland mentions is that Big Data rarely has one source. It’s nearly always generated on different parts of the firm and partly bought-in from partners – and even in a single firm there are database and business process silos.

Database fragmentation makes it hard to integrate the vast volumes of data even when you are aware of what data is available. It’s another main barrier to developing Big Data services because data services are emergent – they only exist as a whole and they can be combined in many different ways.  So its difficult to judge which services will be a success.

It is much easier to start with a good description of customers’ service-needs and then work backwards. But that’s difficult for the firms in the supply chain with tonnes of data and little direct customer knowledge.

That’s why the supermarkets are taking over the world – direct customer contact gives them deep customer knowledge, which helps them to build-up current and new business areas. Then the new business areas give them new insights into different aspects of their customers’ lives – which enable them to build up further new business areas. Look at how Tesco and Sainsbury’s went from groceries to clothes, consumer electronics, then financial services and more services.

If your firm has tonnes of data that you are sure can be leveraged to create new value, but it’s fragmented and you do not know how to use it or when more data  needs to be bought in, then I’d suggest using a customer journey analysis approach, like PIGs. Fragmented product and service components always come together at the point of consumption. And anyway, if you want to assess service quality, value is ‘in the eye of the beholder’ – the consumer.