We’re playing around with some ideas about connectedness. We want to use big data to see what real (as opposed to administratively mandated) communities are. Using Facebook’s analytics page, did some surface analysis of SAARC and ASEAN. It is very clear that India is the center of SAARC, being the country that most Bhutanese have friends in (value of 5 given) and the country with the second-largest number of friends for Bangladeshis, Maldivians, Nepalese, Pakistanis and Sri Lankans (value of 4). I guess the only surprise there is Pakistan.
LIRNEasia is sending its Senior Research Manager, Sriganesh Lokanathan to participate in a short course on Visual Analytics at Middlesex University in the United Kingdom. The course if of high relevance to LIRNEasia’s 2012-2014 research agenda.a A big part of the upcoming research cycle involves the use of Big-Data analysis, partly to try and answer some social science research questions. Middlesex University is part of the UK Visual Analytics Consortium (UKVAC), which is a partnership of UK universities with a shared interest in establishing a multi-disciplinary scientific community in the UK dedicated to promoting and contributing to the visual analytics research and development agenda in the UK. The UKVAC Visual Analytics Summer School 2012 (VASS2012) is jointly funded by the US Department of Homeland Security via the US Centre for Visual Analytics for Command, Control and Interoperability Environments (VACCINE) and the UK Home Office.
WSJ has a piece on big data. “It’s not unlike a microscope—taking something you can’t see and bringing it into the scale of perception,” Aaron Koblin, 30, told me at lunch in Google’s San Francisco office. He’s head of the company’s Data Arts Team. Mr. Koblin’s work sits right on the line between art and information.
Acxiom does a lot more than just analyze streams of transaction-generated information (our definition of big data). But TGI is an important element of what does into Acxiom’s machines. Few consumers have ever heard of Acxiom. But analysts say it has amassed the world’s largest commercial database on consumers — and that it wants to know much, much more. Its servers process more than 50 trillion data “transactions” a year.
How can peer review be effective when the underlying data cannot be shared? When scientists publish their research, they also make the underlying data available so the results can be verified by other scientists. At least that is how the system is supposed to work. But lately social scientists have come up against an exception that is, true to its name, huge. It is “big data,” the vast sets of information gathered by researchers at companies like Facebook, Google and Microsoft from patterns of cellphone calls, text messages and Internet clicks by millions of users around the world.
One thing we know about “big data” in developing countries is that the only data stream that covers the poor is that which is generated by the mobile operators. Here is an account of an interesting application of mobile big data: There’s a vast market of consumers in countries like Brazil, China, India, and the Phillipines who want access to financial services like credit cards, loans, or insurance,” says Jonathan Hakim, Cignifi’s chief executive. “But while they may have jobs, and some have bank accounts, there really is no credit history for them.” One thing they do have? Mobile phones.
I wrote about consumer transaction-generated information in the 1990s. Companies collected and analyzed data from sales points and loyalty programs. But it became sexy only recently. Why? It should not be too surprising that a Google-created entity should have this bent.
First there was searching. Then there was ego surfing, where one spent time and energy looking to see how big a profile one had on the web. Of course, there was help, with Google alerts and such. Now, as we venture into big data (also known as business analytics), it is no surprise that the introspection angle is coming up. Here’s a nice little piece by Stephen Wolfram documenting what he’s done to analyze his personal big data of the past 23 years.
US government gets behind big data. We agree, we’re getting into big data too. Difference is that in our countries there are not that many big data streams. Big data refers to the rising flood of digital data from many sources, including the Web, biological and industrial sensors, video, e-mail and social network communications. The emerging opportunity arises from combining these diverse data sources with improving computing tools to pinpoint profit-making opportunities, make scientific discoveries and predict crime waves, for example.
We thought it would only be social science. But all science? The trend of looking for commonalities and overlapping interests is emerging in many parts of both academia and business. At the ultrasmall nanoscale examination of a cell, researchers say, the disciplines of biology, chemistry and physics begin to collapse in on each other. In a broader search for patterns, students of the statistical computing language known as R have used methods of counting algae blooms to prove patterns of genocide against native peoples in Central America.
We’ve been thinking about the potential of big data (large, continuing streams of computer-readable data) for development applications. There is nothing about development in the marketing campaign below, but can any zealous privacy advocate identify a problem with it? A mobile campaign by Blue Chip Marketing Worldwide, which is based in Chicago, places the ads for the thermometer within popular apps like Pandora that collect basic details about users, including their sex and whether they are parents, and can pinpoint specific demographics to receive ads. But not all mothers will see the ad on their smartphones. Rather, the ads will be sent only to devices that, according to Google, are in regions experiencing a high incidence of flu.
We’ve been saying the new, new thing is big data. According to the NYT, the US government is getting behind big data in a big way. The question we have is how to mobilize this momentum for the BOP in our countries. So far there have been only scattered examples of the potential of mining social media. Last year HP Labs researchers used Twitter data to accurately predict box office revenues of Hollywood movies.
Big data was what caught my attention at the talk by IBM Fellow C. Mohan at WSO2Con. The talk was good. But the exhibition would have been better. It turns out that the initial “data wall,” as it is called, offers a series of displays culled from “live data streaming,” some from sensors around Lincoln Center.