Predictive Analytics: The Power to Predict Who Will Click, Buy, Lie, or Die was called by Tom Peters, “the most readable big data book I’ve come across, By far, great vignettes and stories.“ Author, ERIC SIEGEL, PhD, is the founder of Predictive Analytics World and Executive Editor of the Predictive Analytics Times. Eric makes the how and why of predictive analytics understandable and captivating.
Download PDF Transcript of Podcast
Note: This is a transcription of a podcast. It has not gone through a professional editing process and may contain grammatical errors or incorrect formatting.
Related Podcast: The Power to Predict Who will..
Transcription of Podcast
Joseph Dager: Welcome everyone, this is Joe host of the Business901 podcast with me today is Eric Siegel, PhD, he is the founder of Predictive Analytics World and Executive Editor of the Predictive Analytics Times. Eric makes the how and why of predictive analytics understandable and captivating. His new book, Predictive Analytics was called by Tom Peters: “the most readable big data book that I’ve come across.” Eric, I would like to welcome you and wow, what a recommendation.
Eric Siegel: Yes, thanks Joe. It’s good to be on the show – thanks for reading Tom’s quote.
Joe: So what is meant by Predictive Analytics, your subtitle reads The Power to Predict Who Will Click, Buy, Lie, or Die? Is it all that?
Eric: Sure that does count as a high level definition, the subtitle of the book there. This really is a technology that learns from data to make predictions per person. That’s what distinguishes it from forecasting, where you’re looking at the aggregate overall – how many sales are we going to have next quarter in this region or something like that. This is which individual person is likely to actually make the purchase that level of detail per person, and learning from data part is where we’re leveraging data technology, leveraging technology that’s come out of research labs especially in the last decade, in research it’s called “machine learning”, and we call it predictive analytics or predictive modeling and commercial deployment and data is this great resource and all this hype over data because data is essentially an encoding of experience, it’s a long, long list of things that have happened and so it is literally the aggregate organizational experience from which to learn and then the most actionable thing to learn is how to make those millions of per person predictions.
Joe: Could you tell me a little about the book, the intended audience and so forth and who should read it?
Eric: Sure, you know rather than writing a business reader for a manager to ramp up on the technology and make use of it, although it certainly counts as that, I took it a step beyond that as far as accessibility, and I wrote this in the mode of pop science books like Freakonomics. It’s got that kind of flow and anecdotes and sort of accessibility and entertainment factor once you get down to it. It’s just that much more enjoyable to read and so that it will not only go through the business world where there’re so much value and so much reason for people to learn about predictive analytics, but even to everybody’s uncle or cousin who’s not in a business at all but is somehow a science enthusiast. It’s very much a wide scale book. However, as a former academic, it’s certainly conceptually complete and serving as an introductory textbook for certain business level predictive analytics courses at universities such as UC Irvine. It is like an introductory textbook but written in a more entertaining mode.
Joe: How much of a company’s success anymore is driven by data, the first thing I always think of is Amazon, are we all going to be driven by data sooner or later?
Eric: I think so. I think that deriving decisions from data especially with prediction is something that has taken hold in all large organizations. All are making use of it in some way or another, and the kinds of business applications include targeting marketing, making decisions of whom to give a loan to for credit scoring, fraud detection and all kinds of business applications. As in the title of the book, the power to click, buy, lie, die – click, buy, lie, die we’re talking about all sorts of other applications and industries including law enforcement and healthcare. As far as what’s the importance of data driven decisioning, I see the go-to quote from Tom Davenport, a very well-known person in the analytics space who co-authored Competing on Analytics, he’s got a new book called Keeping up with the Quants and he also wrote the preface to my book Predictive Analytics. What he said in an article, and also in Competing on Analytics, and I’m paraphrasing – in a world today where so many companies offer very similar services and products and have so much in common, one of the last remaining points of differentiation are the internal organizational processes, and that is to say that making those more intelligent in a data informed manner.
Joe: But what about companies like Apple, was it all Steve Job’s intuition or was all good data research?
Eric: Well that’s a great question, although that certainly counts as an anomaly outside of the premise to the quote I just gave from Davenport, where he’s talking about where so many companies are offering largely similar things. When Steve Jobs designed the iPhone, I think we all agree that it was largely out of intuition; he would say things like “hey, we know better what the end consumer wants than the consumer until they actually get in their hands” creating a new product that’s very original. Beyond making a new groundbreaking product, which is certainly something we’d all like to do, one of last remaining points of differentiation is making your processes more effective and efficient in terms of marketing, sales, distribution, inventory all the things that drive decisioning especially on a mass scale over millions of marketing operations – who to give a loan to, how to price applicants for health insurance etc.
Joe: Well I have this saying that I always use that it’s not marketing’s job anymore to get the message out so much as it is to get the message in – so that’s what you’re kind of saying with the data and the analytics.
Eric: One of the prime examples is a longstanding use of modeling often called response modeling when you decide these are the people we should keep on our direct mail list because they’re much more likely to make a purchase if we mail them an average, so you still play a numbers game; you still send out lots of mail that just gets thrown directly in the garbage and ignored by the end consumer. It’s just that you’ve tweaked that numbers game in a way that ends up being really significant and increasing the bottom line. Marketing is a numbers game where there’s a sort of a small sliver where you’re going to get positive responses relative to the whole landscape of consumers out there, very much the same thing with the last Presidential campaigns for the 2012 election where everyone is scrambling for this small sliver of voters within the swing states who could potentially be swayed. Not that they’re swing voters because that’s an ill-defined concept but that they’re persuadable, and actually the Obama campaign did use predictive analytics to find who amongst the many voters are more likely to be within that really small but precious sliver of persuadables.
Joe: When we look at this, data certainly plays a part but people would refute the fact that data can tell the future. I always use this Einstein quote “Logic will get you from A to B, but imagination will take you everywhere.” Can numbers be creative, can numbers be imaginative?
Eric: That’s a great question; I would say yes because as I’ve mentioned earlier – the core method here is predictive modeling, academically known as machine learning. It is literally looking at data, the history of many transactions of how things turn out in the past in order to learn how to predict under new circumstances for a consumer who has a new profile and new history of behavior that’s never been seen before and to robustly be able to apply what’s been learned. That is you’ve actually not just discovered a pattern that shows up in this particular data set, but that actually holds in general. There is an art to that; it is amazing kinds of things that come out of it; it can’t be visualized ultimately by human thought process because computers can do things in a multidimensional way. It’s all about finding that model that looks at all the different factors about an individual, both demographic and behavioral, and consider them together in concert to come up with the best prediction for that individual. The means; the mechanism to do that is the model is the thing that predicts is the thing that’s learned or output from the predictive modeling process.
I would say yes, there’s definitely creativity, I devoted a chapter to how amazing the results ended up being as far as the IBM Watson computer that learns from Jeopardy, the TV quiz show Jeopardy, questions and how to answer new ones – that’s an amazing story. However, unlike that story, usually it’s not about accuracy, so you premise your question just now by saying, “some people say you can’t really predict very well,” the fact is in general, especially with human behavior and the weather for that matter; there’s a real limit to how far ahead and in what way we can accurately predict. It turns out that the use of this technology on all of these different operations and getting value from it does not hinge on the accuracy. Predicting better than guessing, often significantly better than guessing, is what makes the difference and what provides value in running mass operations more effective.
Joe: In the sales and marketing world, we’re furnishing forecasting for operations, should predictive analytics improves our sales forecast?
Eric: Sales forecast is a tough thing. It’s going to depend on factors that are way outside of your knowledge. It has to do with the overall world. What’s happening in the economic climate etcetera, sort of macro scale factors. What predictive analytics is good at is ranking all the consumers relative to one another. Who is relatively more or less likely to exhibit behavior? As far as how many are going to buy, that’s prone to go up and down in ways that we’re all susceptible – we’re all going for a ride in the rollercoaster that’s known as the economy, so there’s a difference between macro level forecasting. On the other hand, as far as the core technology, there are ways in which making millions of per person or per consumer or enterprise client if it’s B-to-B rolling all of those millions of predictions up. In B-to-B maybe more like thousands, but rolling all of them up for macro level forecasts, there are technical ways to do that in which it can help, but in general there’s no sure fire solution to the difficulty of forecasting.
Joe: One of the items that you talk about in the book is persuasion modeling; can you go a little deeper into that and explain that and how you’re looking at influencers?
Eric: Sure, well persuasion modeling is actually looking for who’s influenceable where the only influencer in that equation is you as an organization, you’re trying to influence people to buy more, or in the case I mentioned a moment ago, a Presidential campaign. The Obama campaign did this last year. Looking to persuade voters and in that case it’s a marketing campaign to sell a Presidential candidate but it’s the same principles and the difference here is a little bit subtle, but it’s all the difference in the world. Instead of predicting behavior, which is what most projects and predictive analytics do – predicting what you’re going to do, what each individual is going to whether they’re going to take this action commercially, fraud behavior, medically whether they’re going to get better, whatever the outcome is going to be yes/no. Instead of that, it’s predicting will this treatment, whether it’s a marketing treatment or medical treatment, have a positive impact on the outcome, will it convince them to change their action. Are they influenceable or persuadable with respect to the choice we’re making about whether or not to contact them or with which message or in what manner to contact them. It’s driving decisions in a direct way there about how they can be influenced. I left it for the final chapter of the book. This topic persuasion modeling or also known as uplift modeling because it is advanced, more complex in terms of the underlying analytics, but it’s a fascinating area. It has a lot of potential, and there’s a lot of great case studies, not only in the Presidential campaign last year, where they reported gaining more votes by employing this technology but also with enterprises targeting their marketing and their retention initiatives by way of uplift modeling.
Joe: How does a normal company start making all this data meaningful to them? Do we all need statisticians on board?
Eric: That’s a good question, how to get started; there’s a certain amount of overhead to get started with predictive analytics and at the end at some point in the project you do need an expert. The main thing to figure out, if you’ve not used predictive analytics then it’s a first initiative; what’s the lowest hanging fruit, what are the mass operations, whether it’s in marketing, fraud detection if you’re an insurance company it may be augmenting the actuarial practices as far as how to do pricing and selection. What are the mass scale operations that could be rendered more effective by following the guidance of per person predictions and amongst those, where do you actually have data from which to learn. If you’re trying to retain in terms of marketing, although the longer standing application is what I’ve mentioned targeting direct marketing or direct mail, the probably hotter or lower hanging fruit in many cases is targeting retention offers. Trying to keep customers and slow down the attrition rate by predicting who’s leaving, who’s at risk of defection, churn, attrition, these are basically synonyms it is often called churn modeling or attrition modeling so that each individual that you’ve got in your wheelhouse is given a score as to the risk if they’re on the way out. That score is used to much more effectively determine where it’s worth expending the cost of a retention offer, like a discount, so if that makes sense you do a back-of-the-napkin arithmetic on a macro scale and what potential you could get, and that’s where you want to start. Beyond that decision, you can take some training and ultimately, probably if you haven’t done a hire, need some consulting services to help with certain parts of the project.
Joe: Well that brings me to the next question, what do you look for in an organization or person to help me predict the future?
Eric: In terms of prediction, to predict the future, again predictive analytics is a little more specific than that to differentiate from forecasting but if you’re talking about that kind of mass number of predictions to more effectively drive these types of operations like in marketing the main thing you need is a track record. You need to work with somebody at some point in the project who has actually implemented predictive analytics in the past. The biggest part of the project technically is that data preparation, getting your data into the right tabular format, it’s a pretty simple concept, but it can be elusive, and it’s a big database programming task. The right form and format so that you can then feed it into predictive modeling software, but that part of the process requires some strategic guidance from an expert. Potentially with a relatively small amount of training could be conducted by your internal resources. Then it’s the push and on to predictive modeling software where you need an expert. Even though we’re talking about something that is fully automated, at the core the predictive modeling learning process over the data, how you kind of set it up and run it and look at the results and iterate on that does require expertise. It’s not recommended to do all that for your first project entirely by your lonesome. Then the last part of the project is taking those predictive scores and actually using them which can be really simple in some cases. This part is called deployment; it can be really simple if you’re trying to decide who should I bother mailing and I’ve got this list of 10,000 or 100,000 or 1,000,000 people. Once you’ve got the predictive model it can just put a number next to each person’s head or on each row of data and then you can just order the list by that number and then draw the line. That part can end up being relatively simple, again needs some strategic guidance but can be executed without an expert.
Joe: If I’m not ready for it, but I want to look into the future. Is there a way to organize my data or be aware of my data, what I’m doing now to be able to condition it for the future? So many times when I’m looking at a company’s data, it’s so un-segmented, it’s just been put into a CRM and there has not been anything that is organized it in the past, that it’s really difficult to use some of these tools because the original data isn’t organized correctly.
Eric: There’s a lot you can do if you’re looking ahead as far as what predictive modeling you’re planning or considering doing today as far as what data is being collected. When it comes down to preparing that data to learn from the data that you need to learn from is all historic, it’s all cases where you know what turned out to happen, this customer did make this purchase, this customer did cancel, this customer did commit fraud and those cases have nothing to predict are cases from which to learn. To make them a valid learning case what you have to do is put that outcome, yes or no, positive or negative, for each one next to all the stuff you knew about them back before you found out how it turned out, so you have to roll the clock back, roll your data back to how it looked back then which is a lot easier in some cases if you’re actually snapshotting now and then later you put the results, the outcomes next to the snapshots.
Joe: Since writing the book, what have you’ve learned from it that you didn’t know before?
Eric: The writing process does elucidate, and you get a broader vision of the overall subject you’re writing, my case was a little bit of an exception actually though because as far as my major epiphanies about the industry came as the organizer of the conference Predictive Analytics World, which has been running since early 2009. It’s in that process that I realized just how much more than I even would have imagined that is actually going on now in the commercial deployment of predictive analytics and in some of the advanced topics that had not yet been on my radar before that such as persuasion modeling, which we were just discussing.
Joe: What do you see happening to predictive analytics in the future?
Eric: Predictive analytics is growing quickly. The most exciting thing even in the immediate future is just seeing how quickly it’s being adopted across organizations in these well-trodden business application areas that I’ve mentioned; marketing fraud, finance and these things but also beyond that where you see companies discovering new things to predict. For example, Google uses predictive analytics in the ways you might expect such as to improve their search engine results but they also do new things to predict about consumers that might help an organization in a way you hadn’t anticipated unless you were working there and spent a lot of time thinking it through. Google predicts which new ads are going to be ill received and are going to receive a high bounce. Where you click on the ad but then you click right back on behalf of the advertisers to warn advertisers “hey, this new ad that you’re wanting to post has a high risk of getting a bad bounce rate.” There are all kinds of ways it’s being used now. The thing I mentioned in the appendix of my book where I said, “what is it going to look like by 2020” that’s from the other advantage; that’s from the advantage of the consumer. As we see our smartphones being integrated more with our car and where we’re using Google glass and all these mobile things, the number of ways in which predictive modeling affects your everyday experience, which you know it already does with product recommendations such as Amazon on books, Pandora and music and Netflix and movies and all these other ways in which it affects you in terms of recommendations and such. This really amplifies and multiplies as computing becomes more mobile and things become more greatly integrated
Joe: You have an active speaking schedule. You have your conference, can you tell me about them?
Eric: Sure, yes, Predictive Analytics World taking place seven times this year and next year probably will take place more than that including two in Europe and one in Canada. We have a government focused event September 18th to the 19th in Washington D.C. Our next business focused event is the last day of September first day of October and some workshop days before and after that in Boston. You can check out predictiveanalyticsworld.com to see the schedule of the conferences
Joe: Who should go to the conference?
Eric: The conference actually divides evenly between hands-on practitioners who are experts and people who are newcomers or business users of the technology who aren’t so much the hands-on technical practitioners. We have to tracks that accommodate that. The conference focuses on case studies and brand name organizations’ stories and speakers from those organizations. We invite everyone to check out the agenda just to see how many different topics and company stories are presented at the conference.
Joe: What’s the best way for someone to contact you and find out more about the book?
Eric: Well the website for the book is thepredictionbook.com and there’s excerpts, videos, interviews and all kinds of stuff on that website, there’s also links directly to the conference there so all our information is housed right there thepredictionbook.com
Joe: What is the best way to contact you? Is it through the website?
Eric: Yes, you can find my contact information there, or you can email me at [email protected]. I’m the founding chair, and there’s more contact information under the about page on the book’s website.
Joe: I would like to thank you Eric; I appreciate it. It was a great book and not one that’s going to be outdated soon. I think the stories will live on.
Eric: Great thanks for your positive feedback and good questions Joe, I enjoyed it.
Joe: This podcast will be available in the Business901 ITunes store and on the Business901 blog site. Thanks again everyone.
Lean Sales and Marketing: Learn about using CAP-Do
Special Marketing with Lean Book and Program offers on Facebook