Strong business case could save your project
Like anything in corporate culture, the project is oftentimes about the business, not the technology. With data analysis, the same type of thinking goes. It’s not always about the technicality but about the business implications. Data science project success criteria should include project management success criteria as well. This will ensure smooth adoption, easy buy-ins, room for wins and co-operating stakeholders. So, a good data scientist should also possess some qualities of a good project manager.
[ DATA SCIENCE Q&A]
Q:You have data on the durations of calls to a call center. Generate a plan for how you would code and analyze these data. Explain a plausible scenario for what the distribution of these durations might look like. How could you test, even graphically, whether your expectations are borne out?
A: 1. Exploratory data analysis
* Histogram of durations
* histogram of durations per service type, per day of week, per hours of day (durations can be systematically longer from 10am to 1pm for instance), per employee
2. Distribution: lognormal?
3. Test graphically with QQ plot: sample quantiles of log(durations)log?(durations) Vs normal quantiles
Yes, you read it right. It is a light title for a serious problem. I spoke with big-data scientists in some fortune 100 companies and tried to poke them to learn their strategy on how they want to tackle big data & how they are figuring out the method/tool that works best for them. It was interesting to hear their story, to learn all the options that are available to them and how they ended up picking the tool. I was trying to understand/resolve the problem and then, one night I saw my 2 year daughter cry non-stop. We all huddled to find what is troubling her. Then it occurred to me that, it is the similar situation that companies are facing today.
First, let me explain what happened, and then I will try to make the connection on why and how it is relevant. On one blue moon, my daughter who has just turned two, started acting fussy compared to her normal state. There were some guests at home, so as a normal parent we started figuring out what is bothering her to calm her down, but nothing seems to be working. One of guest put forward some suggestion for the reason for her fussiness, and then there were other theories that got added. All of us were trying to find the right reason for her fussiness from our individual experience and soon, a collaboration of various tricks worked and she found her peace. Not sure if the reason for the fussiness is any important here but the good part is that she became relaxed.
Now, this is the problem that most of the companies are facing today. Like my daughter they all are fussy as they all have a big-data problem, they have lot of unknowns hiding in their data. They all can barely understand how to find them, let alone the way to put them to use. And if we compare visualization tool to guests, parent and everybody around my daughter trying to figure out their own version of what is happening- Itâs a chaos. If you let one of the many figure out their version of what it is, they may be off for quite some time that could be painful, discomforting and wrong for some time. On the other hand, a model of collective wisdom worked best as everyone gave their quick thoughts which helped us collaborate and iterate on the information and figure out the best path.
Now consider companiesâ using multiple tools on their problem, and babysitting for days/months/years costing time, money and resources. These tools could end up becoming the best nanny there is or the worst one. Outcome is anyoneâs guess, but if you get a good tool, will you ever find out if there is a better or best tool out there. That is the problem big-data industry is facing today. Unlike their other traditional appliances/tools, big-data tool requires considerable cash influx and time/resource commitment, so going through long sales cycle and marrying a single tool should not be high on their charts.
Before you get onto your hunting, make sure to create a small data set that best defines your business chaos. The data should contain almost every aspect of your business in a way that it could work as a good recruiting tool for data discovery platform. I will go a bit deeper into what entails some good preparatory steps before you go shopping. But for this blog, letâs make sure we have our basic data set ready for testing the tools.
Now, the best approach in recruiting best visualization framework should go through one of the three ways:
1. Hiring an independent consulting, like we consult pediatrics for their expertise in dealing with baby problems, we could hire a specialized shop that could work closely with your business, and other data visualizations vendors. These consultants could help companies recruit those tools by acting as a mediation layer to help you filter out any bias, or technological challenge that restricts your decision making capabilities. These consultants could sit with your organizations, understand itâs requirements and go for tool fishing recommending the best tool that suits your needs.
2. Maximizing the use of trial periods for platform. Just as we quickly turn around things and validate which method could pacify the kids quickly and not get into long cycle of failures, we could treat It is the same. This technique is painful but still does relatively less damage than going full throttle with one tool on long journey of failure. This approach prepares you to have a mindset, tactical and strategic agenda to hire/fire tool fast and pick the best tool that is delivering maximum value per dataset. This technique is relatively expensive among the three and it could introduce some bias in the decision making.
3. Go with platform plays: Similar to pediatric clinic, you could find almost everything that could help pacify the situation. Similarly, vendors that provide you with platform system to help you experiment all those methodologies and let you pick the best combination that will work for your system. These vendors are not stuck to any visualization techniques but they make everything available to clients and help them get stuck with best package out there. Having locked at such system you could make sure that your business interest should get the highest precedence and not any specific visualization/discovery technique. For keeping the blog clean from any shout outs, I would keep the company name out of the text, but do let me know if you are interested to know which all companies provide platform play for you to experiment with.
And by that you could make the baby stop crying in fastest, most cost effective and business responsive manner.