Websites have a big advantage over bricks and mortar shops – they know who you are. You might be a loyal customer who has accumulated vast numbers of points on your store card, but no-one in the shop knows that until you're paying for your purchase and ready to walk out the door.
But thanks to ubiquitous tracking technology and ad networks, websites know exactly who you are as soon as you show up. They can look up your previous purchases, work out what you might be interested in and give you a tailored view of things.
Amazon is especially good at this, although many other sites lose customers by getting it wrong. In a recent OrderDynamics study, 74% of sites were promoting irrelevant products. If you bought men's fleeces, you'd be offered details about women's sweaters. If you looked at fishbowls, you'd get offers for cat and dog food.
If you need help with improving how you target offers, you could use the recommendation engine algorithm in Azure ML, Microsoft's machine learning service. Or you could do more traditional click stream analysis to understand your visitors (and then feed the same information into the recommendation engine to generate the offers). "That's a common scenario for HD Insight," says Microsoft general manager Eron Kelly (referring to Microsoft's Azure-based Hadoop solution). "You can throw the click stream into Hadoop and use Power Map to analyse it."
Power Map, one of the data visualisation tools in Excel, lets site owners see on a map who's clicking on their site, and animate to fly through a day's worth of information (a good way of spotting if you pick up more overseas customers out of your own business hours). You can see how long people spend on each page they visit and add in demographics, like income groups, helping you find which items appeal to customers with more money.
But how can you do that for the products in a physical shop? Microsoft has been working with retail chain Pier One (which you'll also find in the UK as The Pier) on a system using Kinect sensors and Internet of Things services like Azure Stream Analytics to collect the data.
"These are proof of concept ideas," Kelly told us, "they haven't implemented all of them in production but they're looking at them. The CIO is really interested in, how can I use big data technology to transform how we do business, to change the shopping experience in the store and to provide better analytics?"
The Kinect sensors track the flow of customers around the store and measure how much time people are spending in front of particular displays. The data is loaded into Power Map and overlaid on a custom map of the store layout (Power Map recently added the ability to include custom maps and position data using coordinates as well as its built-in Bing maps of the world with standard geolocation information).
You can see users moving around the store as an animated heat map, making it obvious which products are attracting the most attention. "You can look at this to see the behaviour in just one store, or you can pull in the information from multiple stores and compare by region or by time of day." That would let you compare customer behaviour in your most profitable store with other locations that don't have as many sales, to see if you can spot what the problem is. Is it an issue with the product not being relevant or a problem with the way the store is set up?
Pier One is also looking at Azure ML to understand what customers are likely to buy next. They picked some typical customers who bought a few things at the same time (not just one product and not more than eight) and analysed the data with a standard machine learning algorithm called a decision forest, and with a new algorithm from MSR UK called a decision jungle (that combines a lot of decision trees), which was more accurate.
The drag and drop Azure ML makes that kind of experimentation easy to do, plus you can publish the model as a web service that you can analyse in Excel with tools like Power View. "Now I can manage my stock more effectively. I can see what products are most interesting to my loyal customers," says Kelly.
Mobile apps could use the same web service to give recommendations. That could be a loyalty app that suggests products to customers, or it could be an app for sales staff to use when they're helping customers in store that notifies them if a customer is walking back and forth, so they're probably trying to find something – and suggests other products to mention after they've found it.
With the loyalty app, Kelly says: "When I walk into the store, the model can predict that I'm interested in beer mugs, cocktail glasses and pillows, and show me a store map of where they are on the shelf. That's machine learning models running in the cloud driving an experience that's personalised for me on the phone, backed up by a pretty rigorous business process."
Will customers find that kind of tracking creepy? Kelly believes they won't if it's done responsibly and sticks to useful suggestions. "I can't count how many times I've been to the DIY store twice in one day, because I didn't get everything and I had to go back for one part. If the app tells me 'people who buy this pipe and this valve also need this wrench to fit them; do you have one?' That's really useful."
Putting sensors in stores isn't cheap, but it's an alternative to effectively cutting prices. "The way Pier One thinks is 'I want to recommend a product I think you're going to love without having to discount it'. Right now the retail industry is in this conundrum where everything has to be discounted because that's the call to action – get it cheap. Can I have a call to action that's 'check out this great product, this rug, this chair?' Can I get you excited about the product? Let me show you the latest things you find interesting."