Big Data: Help or Hindrance?

 Tumin and Want in their upcoming book, Precisely, highlight a necessary discussion regarding “big data” and its analysis. While they tout the many benefits of “precision systems”—reliable, reproducible, accurate output from data analysis—they do not overlook the past failures or forthcoming issues if imprecision is not exorcised from our data sets and algorithms. In this regard, this book builds on others like Weapons of Math Destruction. The authors discuss key successes in sports, criminology and business.

They warn against people who want to take action on analysis results before asking the tough, probing questions regarding the collation of data, the assumptions behind algorithms and so on. They caution that the “action vanguard” who blindly trust the data engine outputs will “fire, aim, ready” in that order. Part of the ready, which needs to be first, is ensuring high quality data integrity and freedom from corruption or misapplication.

There are some examples that we can find success by utilizing simpler tools. Many business processes (not just manufacturing) have benefited from timely Statistical Process Control analysis where outliers quickly instigate an investigation to determine if “something” has changed—or other patterns. Like any good business prospectus’ caveat, historical success does not guarantee future success, extrapolation of data should be “taken with a grain of salt.” Too many have been burned assuming projections are linear, only to discover there’s a performance plateau (e.g. market saturation). Whenever we’re dealing with markets and other phenomena of human behavior, it becomes less predictable; behavior changes when it’s observed, sort of like the quantum Heisenberg Uncertainty Principle. I’ve seen results often improve by ten percent just because people knew it was being measured. Consumer trends can run hot and cold in an instant and they become less predictable. Thus, precision systems might have their place in static processes and environments. I wish the authors would address this more completely.

For example, how would the authors build a precision system for most of our endeavors which can be characterized as an Infinite Game (Carse, 1987 and Sinek, 2019). In an infinite game, the rules may change, the competitors may change, the boundaries may not exist and there’s no time limit. In business, I always asked my staff to revisit policies, processes and procedures every six months—and I advise other business owners similarly—because what used to work may not work still: competitors, suppliers, service providers, regulations, customers, team members’ behaviors and skills, community resources, etc. have all changed. (As a Greek philosopher said many centuries ago, “You cannot step in the same river twice.”) 

In addition, we’ve probably all experienced that our metrics were leading us to the wrong behaviors, decisions and goals…and businesses go bankrupt when this happens. So Tumin’s and Want’s precision systems need to ensure that we’re not “putting the ladder against the wrong wall.” Their chapter on transformation, red zones and watching out for misleading correlations is worthwhile.

This is good addition to the data analysis conversation to move us toward reliable, reproducible and accurate results.