For many B2B companies, exponential increases in product lines and customers, large sales forces, globalization and M&A activity have created massive complexity in their businesses. The challenges stemming from such complex business environments manifest themselves in the millions of small decisions made by front-line employees every month — decisions that ultimately add up to a company’s financial performance. While the outcomes of those decisions are clearly important, the most important decision a manager can make is the decision about how the organization is going to make decisions.
Even the most well-intentioned employees cannot accurately and consistently make the best decision that aligns with company strategy each and every time. They are only human. While that may sound like a cop-out, humans are, in fact, persistently inconsistent in their decision-making and predictions. Our reliance on memory of data, reliance on subjective inferences and reliance on our subjective inferences on our memory of data are to blame.
In 1954, psychology professor Paul Meehl published the book Clinical vs. Statistical Prediction: A Theoretical Analysis and a Review of the Evidence, in which he made the claim that mechanical methods of data combination, such as using algorithms, always outperform clinical, or “subjective,” methods when making predictions.
The book caused significant controversy among psychologists, but time and again since then, it’s been scientifically proven that simple algorithms make better predictions than humans. Predictive models have proven more reliable than people at predicting everything from the success of electroshock therapy, criminal recidivism, academic performance, progressive brain dysfunction, the presence, location and cause of brain damage, and proneness to violence. A model can take into account any type of data, including experts’ input or judgment, but once given, the model makes the prediction. In other words, expert intuition is a valid input in a model, just not the only input.
Despite more than 50 years of proof, most companies still make the vast majority of decisions by relying on experience and intuition, or by simply viewing data and drawing conclusions. Predictive models are rarely used to guide the decisions. [Read more...]