I wrote about consumer transaction-generated information in the 1990s. Companies collected and analyzed data from sales points and loyalty programs. But it became sexy only recently. Why?
It should not be too surprising that a Google-created entity should have this bent. Google, along with Web pioneers like Yahoo and Amazon, was crucial to the creation of the emerging Big Data industry. By tracking things like consumer clicks and the behavior of thousands of computer servers working together, they amassed large volumes of data at a time when collapsing prices for data storage made it attractive to analyze. They also captured information from nontraditional sources, like e-mail, leading them to create so-called “unstructured” database software like Hadoop and MapReduce. Versions of those are now used to store and analyze other kinds of data.
The ways in which Google analyzed Web traffic to predict patterns are increasingly applied to other fields, as that data moves online. Mr. Ghobarah believes that biology is an increasingly attractive area.