I think my favorite part of this book was the reference to the W. Edwards Deming quote “In God We Trust, All Others Bring Data”. Davenport and Harris take a survey approach to what companies are doing with statistical analysis of data in order to base their decision making. What I do wonder, and I think its the Taleb influence, is how much of the statistical analysis that the authors site are misappropriating Gaussian distributions to non-Gaussian phenomena, and thereby increasing profitability in the short to medium term, but exposing the companies to great losses in the long term.

That aside, I do agree with the concept of looking at data to help guide what the best course of action would be. One of the topics they bring up (though not nearly as entertainingly as Moneyball by Michael Lewis) is the current usage of statistics in determining the value of players in sports. This certainly appeals to me, as a proponent of a meritocracy.

One of the things that is employed at my company is a ranking system where each person at a specific level is ranked against all other people at that level. In order to accomplish this, first each person is ranked via their contributions on their project, versus the contributions of their peers on the project. This is then done again at the client and industry levels, and ultimately across the firm in order to determine compensation. While the supervisors who are grading the work and reviewing the contributions are good people trying to make objective decisions, when the larger ranking discussions are held, the rankings are based on subjective analysis of which people are thought to have performed better.

How about a mathematical formula with some room for subjective evaluation, but rooted primarily in objective evaluation, after the calculation of which everyone is ranked just as they are today and compensated accordingly? I would propose that we utilize a formula such as the one below that could be adjusted based on level (Managers may be more heavily evaluated on Project Financials and Subordinate Reviews, versus Analysts who may be more heavily evaluated on Earn/Burn ratios and Peer Reviews)

15% Earn/Burn Ratio (the expected task duration over the actual task duration)
15% Project Financials Variance (eg. how far from the expected margin)
10% Peer Reviews
10% Subordinate Reviews
10% Supervisor Reviews
15% Client Feedback
15% Maintaining a healthy work-life balance (as judged by the amount of overtime and vacation incurred during the year)
10% Extracurricular Activities

While obviously these criteria could still be subject to some possible scrutiny (IE, one project was way over-estimated, while another was way under-estimated and therefore causing the Earn/Burn ratios to vary), one should start from sort of objective basis for personnel evaluation, and I think the more transparent, and the more aligned with the goals of the company, the better the results.

Competing on Analytics at Amazon

Moneyball at Amazon