18 July, 2014
The Industrial Revolution Of Data
We are now at the nascent stage of what Joseph Hellerstein at Radar calls “The Industrial Revolution of Data.” The first Industrial Revolution was characterized by a series of rapid changes in the manufacturing industry, transforming the way people traveled, worked and produced products.
Likewise, the coming revolution in public data may prove to be just as significant and far-reaching as the invention of the steam engine.
The data deluge
In recent years, the number of tools available for accessing public data has skyrocketed. As developers move towards more open source platforms, there will continue to be an explosion of public data on the scene. In addition, the number of new web APIs that give public access to data has soared during the big data epoch, with a spike in the last five years.
The open data movement is a key driver of the industrial revolution of data. Proponents of open data argue that data should be freely available to the public to use and reuse without copyright or patent restrictions.
Consequently, a number of open data sources have emerged that aim to give the public access to the information that is available. The U.S. government launched Data.gov, an open data project that makes government data available for free. Microsoft’s Windows Azure Marketplace has hundreds of free datasets that can be mined.
In addition to open data sources, open data tools like Hadoop have made possible the impossible: processing and storing large-scale data sets.
The commodification of data
Public data is slowly flooding the marketplace. Sometimes data is useful but it hasn’t yet been turned into something that has commercial value. As such, analysts predict that data will become a commodity that can be bought and sold freely in the market. As the market for a particular resource expands and demand rises, there will be more pressure on price, resulting in the commoditization of the resource.
That isn’t necessarily a bad thing, argues a Gartner article. When data becomes free or inexpensive, people are more likely to collaborate on data sets and build innovative new business models.
In his book The Structure of Digital Computing: From Mainframes to Big Data, Bob Grossman describes the commodification of time in the seventeenth century. Because of the advances that took place in clock-making centuries ago, we take our watches, phone timers and digital clocks for granted. Likewise, Grossman predicts that ready access to data very well may become the norm for future generations.
This observation is nothing new. Several years ago, Gartner analysts predicted that by 2014, “organizations which have deployed analytics to support new complex data types and large volumes of data in analytics will outperform their market peers by more than 20 percent in revenue, margins, penetration and retention.”
If the past several years have proven anything, it is that companies that recognized the value of big data early on are reaping its benefits today. As more tools are developed that can digest and analyze massive amounts of that data, we very well may witness another Industrial Revolution.