The flipside of the personal data coin: useful suggestions on one side and ownership, control and responsibility on the other

Think about how Tesco Clubcard and many other firms are finding out more and more of our needs, interests and other personal data – in fact every time we use a phone or a web browser to make a transaction or to communicate with friends we generate data that firms collect and can then analyse. These are the firms that we buy things from or that we ask for information to help us buy things. Or they are the firms whose services help us in other ways.

Whatever you do online generates data – when you search, when you click through web sites, when you register for information or when you ask questions you are generating data for those firms that are able to see what you are doing. Your broadband supplier, your web browser supplier and the owners of the web sites that you visit all know what you do because it is their service that helps you do it.

Firms can also get very detailed location information and timing information – especially when you use your phone for browsing or on the web. And that’s great when they use it for suggesting really useful things that you would never have thought of – like Amazon’s ‘People who bought these books also bought these’.

But what happens when firms lose your data or when they get hacked? (And many will) So how can you be protected? What happens when you say ‘its my data, I want to control how it is used!’ And what about sharing the value that this data generates with the person it is about, i.e. me?

So, on one side of the coin there’s the increasingly detailed personal information (Interests, Location, Time) that firms can use to help people a lot and create immense value. But on the other side of the coin there are unresolved issues of control, share of value and fixing damage when its lost or stolen. This general lack of clarity about control, value and fixing damage also means that the role of the regulators is also unclear because these issues are what the regulators’ roles are based on.

I think I have a solution to most of the problem of fixing damage, and the control problem, but I’d be interested to hear of any ideas that help with the share of value problem. I know how to create it, read my blog posts, but sharing value is different.

For example, what if I buy tomatoes from Sainsbury’s and then Sainsbury’s sells data on my buying behaviour to its tomato supplier so as to help it produce ‘better’ tomatoes. Then what if the tomato supplier makes more profit from ‘better’ tomatoes, e.g. because they are more popular. Sainsbury’s might have a clever deal to get part of that value. Like a reduced price on he supplied tomatoes or a percentage of the tomato supplier’s increase in profit.

But how can I get part of that increased profit? Should I get part?

I’m looking for different ways to look at this issue and for some underlying logic to work out how to share the value that is created. Any suggestions?

Socialising your data: Social data needs intermediaries

Any single retailer gets a very limited view of its customers’ lives – just the part of their lives that it helps them with. But deep insights need to be based on very broad views of customers’ personal contexts, which are understood by looking at lots of different aspects of their lives.

The social data that one firm generates by its own interactions with customers can be massively enriched by using data intermediaries that are gateways to other firm’s social data.

Let me explain. If you want to make relevant suggestions to your customers then you need to personalise your suggestions. But personalisation requires a deep knowledge of the personal context your customers.

Unfortunately, what you know about your customers is limited by what your relationship with them is about. Each aspect of your relationship is like a telescope that observes from only one angle. You can buy-in data but it needs to be linked to your actual customers which is hard for third parties to do. Also data from customers’ relationships with other firms will rarely be focusing on the precise things that interest you. It might also be aggregated or anonymized. For example, consumer classification services, like Mosaic, are really useful for getting a general profile of the people. But they cannot tell you what a specific customer is like. They enable you to take a bet with good odds, but it is still a bet.

So how can a firm with a limited view of their customers lives really understand them enough to make on target and relevant suggestions that are more than just repeat purchases? I’m talking about cross-selling and selling for different uses of their products. For financial services firms the relationship data that they generate is like looking at their customer lives through a key hole. So how can they learn enough about a specific customer to suggest different products or to suggest different uses for the same products? For example, how can an insurance firm know when a specific car insurance customer also needs pet insurance or even when they want to buy a new pet?

The answer is not scale, i.e. huge customer populations. Learning about your customers’ lives through their grocery buying habits has the same fundamental limits for Tesco as it has for a medium sized grocer. Tesco gets to learn a bit more because of the range of its offer – you cannot understand how a customer feels about electronic goods and music unless you sell a wide enough range for them to tell you their interests by what they buy and how they do so.

Customer buying data is like the tests that you go to the doctor for – they only answer the questions that you ask. If you want to check for a specific disease then you need to do that particular test. If you want to know a customer’s taste in music then you need to stock a detailed enough range of DVDs so that they can tell you what they like by buying specific DVDs. The granularity of the question enables the granularity of the answer, not the other way around.

The diversity of the different services that you provide for customers, and what you talk about with them, drives what you can learn from them. The more of their lives that you take part in then the more of their lives you can help them with – in terms of information, services and products.

One great way to get highly specific and up-to-date insights is to use social data. Social data can help with assessing sentiment, targeting communications, product and service innovation, identifying influencers and detractors, dealing with complaints and many other ways to support your marketing strategies.

Social data can be used on an individual level to communicate with specific customers.  And even better, it can be used in a bottom-up way to build groups or segments to interact with at higher and higher levels. This bottom-up method is the complimentary but opposite way to segmenting top-down from you total customer population.

Best of all you can take individual level social interactions, analyse them on a segment level and then take action back down on an individual level with the personalised manifestations of a segment-level strategy.

But you only get into deep and meaningful conversations about specific subjects and specific products. Usually this means your own products, and usually it is either some way down the buying funnel or after purchase. So what you can figure out about a customer still depends on your own product range.

Which is why social data needs intermediaries – firms that help other firms to collect and most importantly of all to share social data. With the proper permissions and safeguards, of course.

Social data intermediaries can help firms to get many more observation angles on their customers’ lives. They are a way of sharing the perspectives that have been gleaned from more than one product range, which means from more than one customer relationship, i.e. from more than one angle of observing each customer.

Of course, for several brands to share data in this way there needs to be things like methods of introduction, frameworks for generating trust, ways of enforcing good behaviour, and platforms for orchestrating efficient collaboration. But intermediaries are good at these as well. And there are plenty of precedents in older areas of analytics – like the data sharing clubs that Experian and Equifax host to help manage risk for member firms.

What retailers want – and how to get it (3/3)

New Analytic Strategies: lots of new toys plus Big Data – what to do now?

The key objectives of Retail remain the same. Measures like recency, frequency and value are what defines the profitability of the different retail ‘entities’ such as SKUs, lines, ranges, single stores, regions, chains, brands, campaigns, seasons, segments, channel and even individual customers.

But the increasing availability of Big Data information resources makes it hard to figure out what to analyse for and how to go about doing so. The many new shopping technologies and new supply chain technologies mean that there are too many new data sources and associated methods of using them to make it a simple choice.

The business outcomes are the same. But how can you connect these outcomes to the appropriate analytics approach, and the analytics approach to the vast choice of data that could be used?

This problem is related to part of the fragmentation problem below. When I work with retailers on customer data projects a really common barrier is fragmentation of data sources and multiple versions of the truth. I think that’s what’s behind some of the issues in Econsultancy’s report on what is holding back social marketing.

But I don’t get it. OK, there are lots of sources of data, internal and bought-in, but they are mostly known and its not a new problem so there are lots of IT integration platforms.

So what’s the problem? I think its because some firms do not really understand what the actually use of the data is – i.e. its not well specified. Then the integration (in all senses of the word) problem becomes a moving target. Which makes the problem complex.

You cannot take data at face value. Before you can use it you need to know its context – its history, provenance and the circumstances of how it was generated. You need to have permission. But permission depends increasingly on what you want to use it for and there are so many different things that a single piece of data can be used for. Right now and in the future.

The challenge is not a technical issue. It’s a matter of knowing what data you actually need and have, what analytical approaches are available to you and what strategic questions you can ask. The pattern of data, analytical processes and strategic questions can be configured in so many different ways that it makes it hard to choose. It also makes it hard to know you have made a good enough choice.

Its like the ‘chicken and the egg’ question – which came first? If you have this data what analytical process can you feed it into and what will this tell you? Or, starting at the other end, if you have these strategic questions then what analytical process will give you an answer and what data will that require?

I think the problem is about getting to grips with choosing New Analytic Strategies. A lot of confusion is caused by too many new analytical approaches that can now be used to deliver the usual business objectives. It can be unclear which approach to use and what data to feed into it.

Solution: as with all complexity problems its always best to start with the outcomes that you want and then work backwards. Working backwards damps down unmanageably large numbers of options. Here, it can systematically map out the flows of cash, transactions, customer decisions, queries, attention and awareness in a sort of ‘reverse funnel’.

Don’t start with what data do you have – you don’t know what you want it for yet and you can always manufacture more or buy it in. And certainly do not start in the middle with sexy new analytical fashions that are just begging to be used.

Use a ‘reverse funnel’ to investigate what your analytical processes needs to end up looking like. You can then compare what different analytics methods and software are able to deliver – and what data they need to do so.

Think of your analytical strategy as an array of Business Intelligence telescopes that tell you different things about your customers, your business and your environment. Here’s how Google+ and Facebook are starting to configure their very different arrays of Business Intelligence telescopes on their members.

What retailers want – and how to get it (1/3)

What retailers want – and how to get it (2/3)

What retailers want – and how to get it (2/3)

Indispensable Customer Experience

How many loyalty cards do you have in your pocket? How many loyalty schemes are you a member of? … That’s not very loyal is it? Most loyalty programmes are just bribery in return for customer data.

But if each customer’s experience of a brand, through several cycles of awareness to consumption, was that it was utterly indispensable and an absolute joy to use then they would probably keep coming back. And spend more. And tell their close and trusting friends.

Experience, usefulness and ease are much more valuable to customers than vouchers or loyalty points. Although they mean different things to each customer, so they need to be pre-specified and then configured at the same time as the service is consumed by each customer. Which involves knowing a lot about the customer and producing your service in a very agile manner.

Most importantly, experience, usefulness and ease must be supported across all customer touch points. So they must be a foundation of your omni-channel retail strategy.

Solution: design your omni-channel retail strategy in terms of the customer journey – not just in terms of your sales funnel but by using all that you know about that customer. Use their diverse and changing personal circumstances not some static, aggregated, averaged and frozen history. What they need now and how they need it. Not what some people a bit like them needed at some time in the past.

You also need to segment as low as you can go – it all comes down to segments-of-one in the end because even an off-the-shelf product is consumed according to personal convenience. Every time you segment lower you get a more specific understanding of some customers’ needs and they will value it more.

What retailers want – and how to get it (1/3)

What retailers want – and how to get it (3/3)

What retailers want – and how to get it (1/3).

Retail is being disrupted by lots of new technologies right now. The social media that networks shoppers together in lots of different ways. The mobile devices that people use anywhere and everywhere as well as the apps that run on these devices. And the Big Data that shoppers and supply chains continuously generate.

But the technology is not the problem. The really difficult thing to figure out is how people will use these new technologies. Remember Web 2.0? It only got a name when the different ways that people used it started to settle down a little bit into common patterns.

Now this technological tsunami has reached retail – online and off-line.

What retailers want – E-commerce strategists, CRM Directors, Heads of Marketing, CIOs, Analytics and Insight staff, User Experience teams and Social Media managers  – can be boiled down to solving three key business problems:

  • Dealing with Fragmentation – multiple data sources, retail channels, mobile devices, business process silos, brand partners and other variables make it very hard to even choose appropriate strategies, never mind action them.
  • Creating an Indispensable Customer Experience – bribes are not the true drivers of loyalty. But usefulness, ease and enjoyment are. How then can we be indispensible?
  • Designing New Analytic Strategies – retailers know the business outcomes that they want. But the vast range of new multi-channel shopping technologies, and their associated new analytical technologies, makes it difficult to say which analytical approach will generate the strategic plan to deliver them. What’s possible, not just what to do and how to do it?


Nobody – customers as well as supply chain partners and competitors – has settled on their ‘usual’ ways of using disruptive new technologies like mobile devices, social communications, multi-channel synergies and Big Data analytics.

New hand held devices and web services launch all the time, things keep changing. Everyone is still figuring out how to use them – there is no norm, no ‘usual’. People are still getting used to the features of the devices, apps, sites and new information resources. They are still figuring out what works best for them and what new features they can offer their customers.

This has led to a stage of growing fragmentation – multiple data sources, retail channels, mobile devices, business process silos and brand partners are all variables. They present many different strategic options and opportunities for shoppers, as well as for retailers, and they are so new that there are no obvious or accepted patterns of use yet.

So shoppers, retailers and brand partners are confronted with too many different strategies to comfortably choose between. They also get the slightly disconcerting feeling that they are missing out on some even better ones.

Shoppers have way more sources of advice and new ways to shop than they have needs to satisfy. Retailers have to deal with shoppers who keep changing their customer journey and with brand partners who now interact directly with shoppers at scale, using social media. Also, brand partners now have to deal directly with shoppers’ obscure queries and frank reviews – and be watching out for dissatisfied rants as well as random image-building opportunities.

So in reality, retail is not fragmenting. It’s just that the links that connect shoppers to retailers and the rest of the supply chains are fragmenting. Complexity comes from the number of ways things connect up not from the number of things.

The destination is the same but there are lots of new ways to get there.

Solution: use the vast expertise that comes from your position in the supply chain to give customers and brand partners the advice that’s right for them, i.e. personalised content marketing. Not just what products suit them the best but how these solutions fit into their lives, and the tools and information they need to make the buying decisions themselves. Your position in the supply chain potentially gives you the total knowledge of all your staff, brand partners and past customers.

And also, if its going to fit into your customers’ lives then they need to tell you about their lives…so you get to know them even better.

What retailers want – and how to get it (2/3)

What retailers want – and how to get it (3/3)

They are looking at you: Google’s telescope versus Facebook’s telescope.

I love the way that Google+ is an even bigger reason to login and give Google a token to link together all my other interactions with Google products. Each of which tells Google a little bit about what I’m interested in.

Google products are not there to get customers to use the web, they’re there to watch customers use it. I work with loyalty card and customer data firms and Google’s array of products give a much better cumulative view of each customer’s interests than any single big retailer’s loyalty scheme, even those of Tesco Clubcard or Boots Advantage Card in the UK.

Think  about those lines of small radio telescopes that you see in the desert. Astronomers combine the data from each of the perspectives of the individual dishes into one big view – and the wider the telescopes are spaced out then the broader the perspectives have access to and the more insight they can gain.

In digital marketing and ecommerce insights are about ‘who’ is interested in ‘what’ products and services, ‘when’ and ‘where’ – even if they do not know it themselves.

Loyalty programmes do a great job of helping to figure this out but their insights are limited by the actual transactions and the relationship that generate the data. For example, an insurance company knows a lot about a customer’s ‘insurance life’ but that’s just like looking through a key hole at the rest of the customer’s life.

Supermarket chains get much broader perspectives than insurance companies because they sell customers things that help them in more diverse parts of their lives. But even that is a small part of their whole lives. Truly indispensible, personal and timely suggestions need to be in the context of large parts of who each customer is and what they do (and what they want to do). Especially if they do not know themselves.

Sure, you could buy-in data but bought-in data is generally more indirect and aggregated than the data that comes from your own relationship with that customer. The more removed from the particular relationship that you want to influence then the less relevant and understandable it is – bespoke always fits better than off the shelf.

Helen Taylor’s post on Econsultancy got me thinking about how Google has developed a very broad array of perspectives on each customer’s life and how it is using Google+ to glue them together and to dig deeper. The +1 button is the simplest way to tell Google what is interesting. But all of Google+’s features help to generate deeper insights and each one gives a subtly different perspective on customers’ interests:

Streams – tells Google different things that the member might be interested in. On timeline to enable insights about trends at the person and group levels.

Circles – tells Google which members might be interested in these different things. Members can segment by some preset categories (Friends,Family,Acquaintances, Following) and define more categories themselves. Analysis of these user-defined categories will give valuable insights into how members thing about their different interests in terms of interest-to-interest associations and higher level groupings of interests (like analysing the category structures of folksonomies and socially generated tag clouds).

Brands can segment by global versus local because its useful for them. So brand partners can also signal to Google what type of members interest them.

Hangouts – helps Google to get the sort of deep insights that only come from closely monitoring small groups of people talking openly. As Helen said, these are panel sessions. The Hangouts On-Air feature enables panel session content to be broadcast, stored and edited. The members and brand partners who choose to view this content are telling Google about their own interests.

The other features of Google+ (and other Google products) are designed to cumulatively generate live and updating ‘process Interest Graphs’, i.e. very wide arrays of perspectives on each member’s life.

Each perspective is a key hole on a person’s life and together they give much more diverse, and deeper, insights for Google’s brand partners than a loyalty programme can – maybe more than Hunch or Gravity can as well. So Google can partner with more brands and do so in more actionable ways.

Facebook, LinkedIn and Twitter have very different ‘arrays of telescopes’ to Google and these give them very different arrays of perspectives to look at their members’ interests from. First, each of these social networks focuses its arrays on different general aspects of their member’s lives. Although they do overlap – and Google+ seems to overlap the most:

Facebook – entertainment and social life, short-term issues, life curation

LinkedIn – work life, short-term issues and long-term projects, network curation

Twitter  – all your life, immediate issues, bare bones content

Facebook and LinkedIn have much tighter feedback loops between members – in terms of more levels of connection (ways to directly exchange content) and some features that enable actual two-way conversations.

Twitter is a bare bones way to connect with people who you think might have interesting things to say. Mostly its about broadcasting with some ability for loose two-way communications.

Second, each of these social networks uses different features to get the data that gives them their arrays of perspectives. Google use product-based features outside of their social network, as well as inside like the others.

The bottom line is that they all try to be really useful in their chosen aspect of their member’s lives because they know that being really useful help requires clarification and clarification leads to much deeper customer knowledge than bare transaction data.

Update: Facebook is looking at combining information across its other services here.