Common threads running through three recent Big Data roundtables


Source: Wikimedia commons

In the last few weeks I’ve been to three roundtables that were full of experts on Big Data and Big Data analytics or business users of the insights from such analytics. There were also a few Members of Parliament and senior industry regulators.

The first roundtable was at Econsultancy, the second was at the think tank Reform and the third was a Personal Big Data roundtable. These round tables addressed three very different aspects of Big Data but there are some common threads that stretched through all of them.

The first common thread was the richness and variety of the topics that we discussed. Big Data is a new and emerging set of technologies and right now we are at Big Data 1.0 not Big Data 2.0. The discussion at these roundtables, just like Big Data articles on the web, was as unstructured as Big Data is itself. When a structure forms we will call it Big Data 2.0.

Some people were focused on the hardware, some liked to talk about the data it handles. There was a huge amount of discussion about ‘data’ and less about what data or which data. Indeed there was a general thirst for examples, case studies and illustrations of uses of Big Data.

There were also lots and lots of metaphors like ‘Data is the new oil’ or my own biased favourite ‘Big Data is like the minute-to-minute personal diary of everyone and everything’. When it is unclear what ‘something’ is, a something that is emerging as we develop how we use a these technologies, then metaphors are very useful. They help use to generate potential forms that we can check and test for usefulness.

Even the experts have different points of view and many questions about what form, or forms, Big Data will take. However, there are a few rough characteristics starting to take shape and this is what I hope to describe here.

The second thread that ran through these roundtables was that there was more talk of the hardware and the data themselves rather than of the actual services that Big Data analytics could create.

It is relatively easy to deconstruct a service after it has proved highly popular. But thinking up that highly popular service in the first place is very hard. Right now we have some new hardware, access to vast amounts of raw material data and a complicated range of analytical tools but it is unclear how to combine all these into specific configurations that produce the ‘killer apps’.

One way around this might be to start out with some commonly valued objectives and work backwards to try and connect them to the outputs that we know that our new analytical techniques can produce.

For example, both government and industry are perennially keen to [1] increase services or sales, and [2] make savings. And we know that a key role of these emerging analytical techniques is to help us accurately understand the needs of people – on a more personal and individual basis.

So we should be looking for analytical techniques that suggest the unmet needs of citizens and customers – because knowing unmet needs helps us to increase services or sales. And more precisely tailoring the services that we already provide could reduce wasted resources and make savings.

These analytical techniques are based on analysing the individual interest graphs and contexts of peoples’ lives, e.g. here, and they are the foundation of Big Data services.

The third thread was about balancing the societal and individual privacy aspects of Big Data. Economic growth from new Big Data firms and services depends on consumer trust. But these services depend on organisations sharing consumers’ data between themselves.

Few organisations share enough of a person’s life to understand their needs very deeply. But sharing data for good or for profit generates questions like How do I control my data? How do I share in the value that it is used to create? and How do I fix it when my data is hacked or stolen?

The Personal Big Data roundtable in March brought together some of the leading experts in data analytics, retail, healthcare, financial services and some key industry regulators. These questions were at the top of our agenda but they were also touched on in the other two roundtables.

The point is this: consumer trust depends on regulation, which depends on legislation, which in turn depends on policy. But current regulation, legislation and policy are inadequate for handling the opportunities and dangers that Big Data presents society – they are not so much out of date, it is more that they have been made technologically irrelevant.

From my research I am starting to see how the regulation and legislation could be developed in order to support the societal benefits that we hope to gain. To do this we need to help legislators and regulators to start this change process – stories and case studies will help but there are no case studies for some of the more complicated inter-relationships and business models that are yet to emerge.

The forth thread concerned the people, citizens and customers, that we are describing increasingly accurately with these new technologies. People do not only vary in terms of their needs for different services, which is why we analyse their data. They also vary in their attitudes to privacy – people exist on a spectrum of sensitivity with some not caring about data privacy and some being highly sensitive.

Also, people rarely read through user agreements – they do not have the time nor the training, e.g. when they download an app that will give the app firm access to the content of their mobile phone and their location on a 24 hour a day real-time basis.

But most interestingly of all, when you spend a lot of time surrounded by experts, it is worth noting that most people lack an awareness of just how the technologies that I talk about here are changing their personal and work lives right now. There is a huge need for education and awareness if people are to get the most out of these new services and use them safely.

There are two main implications from the discussions that I have had the pleasure of being part of in the last few weeks.

The first is that there seems to be some vacant niches in the Big Data ecosystem, to use another metaphor. There are some unfilled roles, like a broker that would manage a person’s data and deal with firms on their behalf; a defender of a person against harm; a fixer of such harm; an educator that teaches people what they need to know about our unfolding Big Data society; a new form of regulator to uphold public interest; or even a third party ‘dating agency’ for firms and their data.

These roles might not exist within the same organisation. Indeed some of these roles may be taken up by regulators or they may fall to multiple competing Third Parties rather than a single organisation.

The second implication is that there is a huge and complex gap between the raw material Big Data, on the one hand, and the consumer needs that it could be used to satisfy on the other. We know that we have lots of data and we know that we can buy-in, swap or access more data. We know that we have some sexy, fast, new hardware and unbelievably clever analytical software.  We even know that we want to hit the same old organisational targets of doing more with less.

But we do not know which particular data to use; which particular software to install and learn to use; which specific way of using the software, which analytical services to produce out of all those that we could; which consumer needs to target, even which consumers to target. The huge and complex gap is made up of all the dependencies in the last sentence and we are only now starting to come up with Analytical Strategies that can bridge it.

Mobile Big Data: how to link very large scale analytics with very small scale personal needs.

Finger of God

[Source: Wikimedia Commons]

Retail is being drastically changed by new digital technologies like Big Data analytics and the services that mobile smart phones can deliver. Big Data has been discussed a lot but there is little analysis on what it can specifically do for firms and their customers.

Also, apps that run on Mobile devices, like smart phones and tablets are revolutionizing multi-channel retailing and customer relationship management. But mobile apps are usually just catalogues and directories when they could be personal shopping ‘sat navs’like your best friend owned the store.

Most pressingly of all, there is very little useful thinking, or practical advice, available on the overlap between Big Data analytics and personalised services that are delivered by your phone, i.e. the links between the outputs of very large scale analytics and the very small scale personal needs of individuals.

Firms are still working out what they can do with these technologies. Customers are still deciding how they want to use them.

We need a roadmap

New digital technologies are producing a bewildering number of options: new shopping and customer relationship technologies, new Big Data information resources, new analytical possibilities and new business strategies.

Firms need an underlying roadmap for taking advantage of these new tools and resources as they unfold and develop. We need to connect back to the fundamental business objectives of commercial success and amazing customer service in the face of a chaotic digital landscape.

This is the first of three linked posts that show the significance of these technology-driven changes; explain the underlying processes at work; point out the business challenges on the horizon; and map out the strategic options that are now possible for retailers, their customers and the brands that they partner with.

Part One covers how new Big Data and mobile technologies are changing marketing, retail and the rest of business from a business strategy perspective rather than from a technological perspective.

Part Two maps out and explains the confusing new options and approaches that these new digital technologies are now making possible. From the retailer’s perspective, from the customer’s perspective and from a business strategy perspective.

Part Three explains some ideas on how to deal with the emerging possibilities described in the first two parts – from a strategic and analytical perspective, then from an implementation perspective and then with a consideration of how these fascinating changes will continue to unfold.

Part One: A mobile app is like the finger of God

Mobile apps are location-here, segment-of-one, stage-of-now, and downloaded by YOU.

Every app can potentially give you incredibly personalised recommendations, suggestions and advice based on knowing where you are, who you are, being with you every minute of the day and being trusted by you (you downloaded it).

These dimensions are revolutionising how firms communicate with you, learn about you and produce services for you. Everyone you know and most people that you don’t know also has a phone so you could multiply the previous sentence’s possibilities by most of the human population.

But neither firms nor consumers have fully worked out how to use these dimensions. Using mobile phones to figure out a person’s location has grabbed a lot of headlines and initiated a few start-ups and service features like geofencing.

Services that run on smart phones naturally segment customer populations down to a single individual because we rarely share our phones. So phones are platforms for personalisation, i.e. precise learning about individual needs as well as giving customised advice and information.

Also, we carry our phones around everywhere and a lot of people never switch them off – so services can potentially be real-time and anytime, whenever customers need them. Most importantly of all, customers choose to download the app and then give the app the permissions it needs to work.

So services that are delivered by phone apps [or mobile sites] have the capability for complete personalisation in terms of ‘where’, ‘who’ and ‘when’ plus they are a potential bridge for two-way exchange of information.

What goes around comes around

Complete ‘who’, ‘when’ and ‘where’ personalisation is great but an app [and the huge supply chain behind it] needs to know which service options, product variants, SKU number, model, user experience configuration or other permutation of what it could potentially provide is best for each particular ‘who’, ‘when’ and ‘where’ customer at the moment that they tap the screen to say they want it.

This need to decide which service would best fit the immediate needs of a specific consumer will be even more pressing when apps suggests useful things without an actual request from the consumer. Like Google Now is starting to do.

The absolute best thing about mobile phone apps and services is not their in-build sense of ‘who’, ‘when’ and ‘where’ – it’s the information that the consumer gives to the app owner or service provider [and the huge supply chain behind it] plus the permissions to use it.

Mobile apps are not just a bridge to God-like services – they are a two-way bridge to God-like services.

‘God-like’ means not quite omniscient, i.e. you have to give the app some clues as to what you need from it.  Mobile apps know who you are [you registered], when you ask for something or contextually might need something [they are always on] and where you are [they are in your pocket]. So if the app does not abuse the data permissions that you give it [an unresolved issue] and keeps being indispensible then it can potentially act as the ultimate loyalty card.

Introducing the ultimate loyalty card – the Mobile + Big Data version

Big Data is the minute-to-minute personal diary of everyone and everything. Mobile apps have the potential to be the pen that writes your Big Data diary.

All our on-line transactions, communications and surfings are recorded and increasingly stored, analysed and used. But there are still vast gaps in our personal Big Data diaries. For example, a huge retailer like Tesco with a tremendously sophisticated loyalty programme, like Clubcard, only directly knows about the part of your life that is your shopping-life. It can buy-in shared data from its partners to get a better insight into your specific needs. But it can never know your full Big Data diary.

But a mobile app [or collection of apps and mobile services] – that you trust enough to give the right permissions to about collecting your personal data and that you interact with as you go through your life events – could potentially know your whole Big Data diary. The Big Data diary of your on-going minute-to-minute life.

Just think of how helpful [or dangerous] this could be. Just think of the services, new services and benefits to society that this could be the platform for. Nobody could accuse those people behind Siri and Google Now of aiming too low.

Linking smart phones and Big Data means everything in the cloud delivered to you, right here and now. The intimacy, immediacy and relevance of smart phone apps is combining with the vast scale and power of Big Data and the cloud.

This generates many confusing and unfolding possibilities. A massive aggregation of information and other resources is combining with a highly specific understanding of customised requirements. Very large scale Business Intelligence is colliding with very small scale human needs.

The scale and variety of Mobile Big Data is both an enabler and a barrier. For firms and government it changes the problem from ‘we would like to do’ to ‘which should we do?’ plus ‘are we sure that’s the best thing we could do?’ There are too many new possibilities in too many new areas.

Part Two maps out and explains the confusing new options and approaches that Mobile + Big Data is now making possible. It does this from the retailer’s perspective, from the customer’s perspective and from a business strategy perspective.