A data driven leader, a data scientist, and a data driven expert is always put to test to help their teams by using their skills and expertise. Believe it or not, but a large part of that decision tree is derived from intuition that adds a bias in our judgement and makes it flawed. Most skilled professionals do understand and handle these biases but in some cases, we give into tiny traps and could find ourselves trapped in those biases which impair our judgement. Here are some examples of the biases and a good leader must understand and keep these under check:
- Analysis Paralysis Bias:
Every data science professional who has spent a good amount of time with data understands the problem with over analysis as well as under analysis. Many times âunder analysisâ leaves things undiscovered and leaves the results susceptible to failures. Imagine the âTargetâ debacle when a pregnant teenâs dad ended up receiving deals on maternity items. Such problems often occur when the analytical models are not completely thought through. Similarly, consider a case when one has spent way too much time on a task that requires little attention. Such shifting requirement for attention between little or more analysis is often not very clear upfront. So, as a leader one should build the right mechanism for helping data analytics team understand the shifting bias towards spending appropriate time analyzing and not falling into the trap of under or over analysis.
- Framed Bias:
When a leader is fed information, there are often the figments of framed bias. Algorithms and models are susceptible to take the bias of the designers. The more complicated a model is, the more it can learn and get influenced from its designers. Such a model or methodology is often tainted with the framed bias. Take a simple example of guessing a team size when options are 0-100, 100-500, 501-1000 viz a viz 0-2, 3-7, 8-12 etc. Both options (the span and volume of the values) when presented to a subject, influences the outcome and induces the framed bias. As a leader, it is important to understand that the data analytics teams are not working under any framed bias.
- Sunk Cost Bias:
Along the lines of analysis paralysis, consider a case when a team is working on a high profile task and has spent a lot of resources (effort, time and money). Many times invested resources often induce a bias of sunk cost. There is always a temptation to invest more time due to the fear of letting the past resources go to waste. This is one of the toughest bias to beat. This is clearly seen when you see a team trying various ways to tame the outcome when you know that the outcome is dependent on random variables beyond the control of the team. The only less painful way to deal with such bias is to understand the odds, let probability be the judge and have stage gates in your teamâs analysis.
- Anchored Bias:
This is another interesting pitfall that taints the judgement and the reason could be our own anchors or supplied by our teams. Anchors are often the biases that we strongly believe in. They are the most visibly picked assumptions in the data or analysis that stick with us and find their way into influencing subsequent judgement. One of the easiest example of such a bias is any socio-political analysis. Such analysis is often anchored with our pre-conceived bias/ information that closely satisfies our understanding. And subsequently, we try to influence the outcome. In many ways there is a thin line of differentiation between Anchored and framed bias. Besides the point that framed bias is influencing our judgement based on how something is framed viz-a-viz anchored bias leverages our pre-conceived notions to influence decision making. The easiest way to move around such bias is by keeping an open perspective and always staying within the bounds of data or analysis.
- Experience Bias:
This is one of the most painfully ignored bias that any leaders have. We often think that we are the best judge of our actions and we have the most adequate knowledge of what we do. Being an expert does come with an advantage that helps handle a task with great speed and comfort. However, experience bias tricks an individual in believing that an experienced judgement is often the right judgement. A typical example that I have come across is when a team is using obsolete models and techniques without realizing the problem that there is something better out there. Such a bias limits our capabilities and restricts decisions to our limited understanding about the subject. A leader must be swift in understanding such a bias and work around it. One of the easiest way to work around such bias is by asking questions. Many times such biases fade away when one questions their own knowledge and discovers cracks and pitfalls in their decisions. This is a critical bias to fix for success of data analytics teams.
- Effort & Reward Fallacy Bias:
We as human are designed to work smartly and we get our dopamines with every success that needs minimal effort. This definition of success has been engraved in our genepool and taints our judgement. When one sees a reward early in the process they often stop thinking beyond and get fixated on the outcome. This problem was briefly mentioned in the book: innovator’s dilemma. Normally we are designed to treat bigger reward with less effort as success. This is one of the most difficult bias to overcome / fix. When we meet a major breakthrough, we stop looking around the corners for something better or more effective. Such a bias could be easily treated by providing clear directions/goals. A clear roadmap to success often overpowers our quick wins and help the team maintain a vigilant eye on the bigger goal.
- Affinity towards new Bias:
Another bias that is engraved in people is our openness and friendliness towards the new. New TV, new house, new gadget, new way to think etc. Such a bias also influences data analytics teams. The most prominent occurrence of such a bias is when a new solution/model is proposed for a currently existing model and we give it a green light. Many times such decisions are tainted with the fact that we love to try new things and many times forego analyzing the hidden and undiscovered drawbacks. As a leader it is most important to be free from such a bias. One should have a clear scaled measure for substitution. It is important to complete SWAT analysis and understand the difference between the new viz-a-viz old ways. The team should be clearly instructed and trained to identify such pitfalls as many times this bias makes its way through untrained or relatively newer analysts and often gets unnoticed. Keeping the team from such a bias helps the team in siding with the best models for the job and helps improve the quality of data and analytics.
- First Thought Bias:
In many strange experiments humans are always seen making judgements early in the process when adequate information is not known to help derive a hypothesis. We are constantly seen using the judgement to help form the basis for future outcomes. A typical example is seen when you hear a new process and a quick thought appears in your mind that you wish to share, and once shared with public, you spend the rest of the time to prove your initial reaction. Many opinionated leaders often suffer from such a bias and find it difficult to eradicate from their decision making process. One of the quick and easy ways to keep this bias in check is to always have an open perspective, never let any bias get formed early in the process, and use the flow of the conversation to understand the situation before coming to any conclusion. Many good leaders do it in a great way and handle it well by keeping the decision process at the end of the conversation. With practice this is one of the easiest and most beneficial bias to get fixed.
So, it is important for a leader to keep their analysis and their teamâs practices bias free so that the company could enjoy the benefits of bias free data driven outcomes.
I am certain, there are many other biases that are not covered here and I would love to hear from the readers about those. Please feel free to connect and email your suggestions and opinions.