11 April, 2017
A Big Fat Big Data Failure Illustrates Its Problems
Big Data is huge. Everybody wants it. If you’re not doing it, you’re hopelessly antiquated. But it has serious flaws. The high-profile role played by Big Data in the recent election provides an excellent example. Calling those efforts a “face-plant” is kind. In addition to illustrating many of the glaring flaws I have previously enumerated, this face-plant clearly and explicitly demonstrates the corrosive effects of bias: the experts weren’t seeking the truth — they were rooting for an outcome. Given the undeniable predictive failure, you’d think a little self-reflection might be in order. This post uses the recent election Big Data failure as an example. The flaws it illustrates, and others, are common in Big Data efforts, and are the reason why so many of the much heralded efforts result in no substantial benefit.