Sanjog Misra, Charles H. Kellstadt Professor of Marketing and Neubauer Family Faculty Fellow, teaches the courses Digital and Algorithmic Marketing and Marketing with Big Data.
Knowing when to stop looking at data comes up constantly in my Algorithmic Marketing class. In this class, one of the main goals is to be able to develop tools that would help someone make better decisions. Building these tools relies on knowing what exactly the decision is or what the question is. Very often people don’t specify their question in a precise enough form. You need to write down a specific question—it can’t be a vague goal or a vague statement. It’s important to thoroughly articulate your question and your research plan. The more precise your question, the easier time you will have looking for an answer.
The question in itself isn’t enough, though. We also need to specify the exact parameters of an acceptable answer. It doesn’t occur to people to write down specs of an answer, but that’s another thing that needs to be done before you get started.
“You need to give yourself some set of parameters to help you understand when you’re going to stop even before you start.”
You need to give yourself some set of parameters to help you understand when you’re going to stop even before you start. These parameters could be a set of rules you have to satisfy. For example, if I’m looking at how advertising impacts sales, it might be that I am looking for a set of parameters in the context of a particular model. Knowing that helps you look in the right direction. You have to chart out what the ideal answer would be, and you have to chart out what you’re going to be satisfied with in the findings.
This isn’t a totally new idea, just new to analytics. In the business world, one wouldn’t want to put out a call for proposals with no details about what he or she is looking for. We wouldn’t want to wade through a million proposals to decide what suits our needs. That would be silly. Instead, when you put out a call for proposals or a purchase order, you typically outline a very detailed specification of what you want. Similarly, it’s worthwhile speccing out the “answer” you are looking to find. You don’t go around aimlessly.
One of my interpretations of Peter Kennedy’s 10 commandments about data analysis is, “Thou Shall Not Fish.” That’s something I emphasize in my classes. Of course, sometimes mining for data is actually what’s required. So if the objective is to fish, then you should be fishing. If it isn’t, the commandment applies.
Charles “Chase” Carpenter is an Evening MBA student and the assistant director of advanced analytics for the Chicago Cubs.
It’s my job to build a roadmap for data-driven decisions for the business operations for the Chicago Cubs. That means that I need to decipher what the solution might be, and whether we are dealing with too much data or too little. The balance of when to shut off analysis can be challenging. On a personal level, it’s the most fun part of the work that I do.
For me, it starts with a clear project plan before any analysis is completed. I need to have a detailed discussion with my stakeholders about what the desired end goal is and what sort of business problem we’re facing. I also make sure that I’m actually dealing with an analytics problem and try to enumerate the specific outcomes. If I can’t come up with a bullet point list, then we may need to start over. Not all problems require you to make your best guess without all of the data; however, much of the time, that is the biggest challenge.
“Whether it’s a $100 problem or a $10 million problem, you can avoid the trap of, ‘Have I found the perfect answer?’ through some pretty diligent planning.”
After this sort of preplanning activity, you’ll hopefully have boxed your analysis into the right-sized problem. Whether it’s a $100 problem or a $10 million problem, you can avoid the trap of, “Have I found the perfect answer?” through some pretty diligent planning. Ultimately, the goal is to conduct an appropriate analysis given the size of the problem.
As you are working through your problem, you can always compare the best-case scenario to the worst-case scenario. Compare the difference between those answers. If they are vastly different, you should continue analyzing. But if the two answers are similar, you may be getting close. As a data analyst, you always want to have more data and more time. But with the right data and enough time, there aren’t too many questions that you can’t answer.
A lot of effort goes into model development and model building, but what often gets shortchanged is how you communicate those findings to your stakeholders. More emphasis needs to be placed on communication the further you move away from people who live and breathe predictive analytics every day.
Don’t discount the element of human intuition. As an analyst, I bring a perspective about what the data tell us, but business experts can also provide a great perspective. It’s about layering in that human element as a helpful step to find actionable solutions.
In most business situations there’s no one right answer, but there are a variety of answers that meet the business objectives. You need to be comfortable with embracing that uncertainty and embracing that there may not be a perfect answer. I’ve also learned that explaining the analysis to stakeholders means I’ve got enough information to stop. If left unchecked, I would happily analyze data until there was nothing left to analyze.
Wei Sun, ’11, is a partner at Digital Craftsman Venture Partners, an investment firm focusing on data and enterprise solution ventures in Beijing.
It takes me two to six months to understand whether we, as a firm, want to invest in a business. So I have a limited amount of time to learn about the company and the quality of the team. Sometimes, the data that I collect is already out of date. In a few months the company can grow to 20 people from just three or four. For me, that’s the key.
In order to pull away from the data later, I need to understand the timeline at the beginning of the process. A lot of the time, this so-called timeline is invisible. A VC firm may not even know its competitors—let alone have a stated timeline. In many cases, there’s a fine line between needing more information and completing the analysis in time. Wait too long and the opportunity can disappear. There’s a timed window for the investment, and if I don’t know the answer then someone else will make the investment—that adds to the pressure.
“My mission isn’t to have all of the data—I just look to have enough data to defend my decision.”
My mission isn’t to have all of the data—I just look to have enough data to defend my decision. While I was a consultant, for example, I would try to piece together all of the critical data as a business analyst. I could never get a big chunk of the data or all of the data, but I needed to figure out the minimum of what was enough. For me, avoiding the kind of paralysis that comes from staring at a problem for too long and the constant feeling that I need more information comes down to having a personal sense of urgency.
It gets easier with practice. You can consider VC investment as a practice of pattern recognition; the more you do it, the simpler it gets to pull away with the available data and make a decision. Venture capitalists have their own secret patterns of how they combine specific pieces of information to help them understand whether a company will be successful—they never have all of the information at once. But even when I’ve made a decision, I don’t always break away. Rather, I may work in the background and kind of follow along with more data to make sure I made the right call.
Mo Bhasin, ’15, pushes himself to the limits with the exhilarating sport of kiteboarding.
Booth 101 Kiteboarding
The Stevens Doctoral Program has trained researchers to share across disciplines, challenge ideas, and transform our understanding of business.
The Booth PhD at 100: A Century of Discovery
Three Booth experts tell us what they think can get the engines revving again.
Why Is Productivity Stuck in Neutral?
