Today, our CEO Helani Galpaya was on a panel “Harnessing the power of convergence and big data for enterprise success” at a Sri Lankan summit called “Enterprise 2.0: building future ready enterprises” (full video of the panel session is available HERE). I thought some of the ideas she proposed about were worthy of further discussion. LIRNEasia is curently working on utilizing telecom network Transaction Generated Information (TGI) to conduct public interest research using big data. One of her comments was about how companies are not fully appreciating the value of the data that they have.
For those who worry about their privacy being harmed by transaction generated data, here’s more to worry about: sensors in the sky. These systems generate so much data that they do require big-data analysis. Just as important, he shepherded research and development of new kinds of satellites that made digital pictures of objects on the ground as small as five inches across and then transmitted the images to earth for analysis almost instantly. The aerial reconnaissance programs, most done in conjunction with the Air Force, were highly classified, and many remain so. In a 1967 speech that he asked not be quoted, President Lyndon B.
LIRNEasia was recently asked to sign a declaration on Internet rights. We do not normally sign declarations unless we have done the research to back up our signature. Another reason for declining at this time was my wariness about right-based approaches. I prefer the Deng Xiao Ping approach of crossing the river by feeling the stones. We need to figure out, for example, what is practical with regard to big data before imposing rights and policy solutions imported from other contexts to something that we are all unfamiliar with.
When asked about emerging trends of relevance to those picking research topics at the recent CPRsouth conference, I pointed to the growing importance of the badly named “big data” or its more analytically satisfying subset of transaction generated data (TGD) or information (Thomas McManus coined TGI back in 1991; TGD is more accurate). It’s going to be big data in everything. Even the shift to MOOCs is driven by the need for TGD, according to the NYT. There are potential advantages to this shift. When students are logged on, educators can monitor their work in ways that are otherwise impossible.
The New York Times carried a story on “big data for development” that featured Global Pulse, the UN initiative seeking to harness the potential of data to address development questions, much like what we are doing in our current research. The efforts by Global Pulse and a growing collection of scientists at universities, companies and nonprofit groups have been given the label “Big Data for development.” It is a field of great opportunity and challenge. The goal, the scientists involved agree, is to bring real-time monitoring and prediction to development and aid programs. Projects and policies, they say, can move faster, adapt to changing circumstances and be more effective, helping to lift more communities out of poverty and even save lives.
LIRNEasia and WSO2 are jointly looking for a software programmer with skills in statistical analysis. The position can be full or part-time till September 2014 with potential extensions. Requirements Responsibilities The selected candidate would work with Big Data technologies such as MapReduce as well as with statistical methods using real world Big Data whilst conducting cutting edge research. Specific responsibilities include: Provide technical support in collecting, cleaning, organizing, managing and analyzing data Assist with the large-scale analysis using Big Data tools such as T-Cube (http://tcube.autonlab.
We were not quite ready to start talking about the privacy issues surrounding the massive amounts of data generated by telcos in the course of making it possible for people to communicate, but recent news events are accelerating the schedule. I thought it might be useful to start with this quote from someone I used to work with in the 1990s: “American laws and American policy view the content of communications as the most private and the most valuable, but that is backwards today,” said Marc Rotenberg, the executive director of the Electronic Privacy Information Center, a Washington group. “The information associated with communications today is often more significant than the communications itself, and the people who do the data mining know that.” Full report.
The Economist talks about how New York and Chicago are using different approaches to the analyze big data generated from within their operations. Sadly, no such activity can be reported from our part of the world. Many cities around the country find themselves in a similar position: they are accumulating data faster than they know what to do with. One approach is to give them to the public. For example, San Francisco, New York, Philadelphia, Boston and Chicago are or soon will be sharing the grades that health inspectors give to restaurants with an online restaurant directory.
Returning to the privacy field after a break of more than 10 years, I was struck by how inappropriate the old notice and consent approaches would be for what was actually happening on the ground. Here is an attempt to evolve new principles. Not had time to fully digest yet. Traditional approaches are no longer fit for the purposes for which they were designed, for several reasons: • They fail to account for the possibility that new and beneficial uses for the data will be discovered, long after the time of collection. • They do not account for networked data architectures that lower the cost of data collection, transfer and processing to nearly zero, and enable multiuser access to a single piece of data.
Alex Pentland of MIT has been working on mobile big data (as are we at LIRNEasia). Here is a snippet of an interview in the NYT: The phone tracks our movements, as well as our calls and texts, so it can reveal a lot about our daily lives. What did you learn about yourself by studying your own cellphone data? That I’m very predictable. We tend to pay attention only to the new things in our lives.
I’ve been thinking about big data and privacy these days. I used to think about this subject a lot in the early 1990s. Back then I did not have a lot of company. But now, there is plenty. But as I read what is being written, I worry.
I have always considered Disney to be operating at the cutting edge of service delivery and crowd management. According to the NYT, it appears that they are planning to transform both using location-sensitive technologies and big data. The ambitious plan moves Disney deeper into the hotly debated terrain of personal data collection. Like most major companies, Disney wants to have as much information about its customers’ preferences as it can get, so it can appeal to them more efficiently. The company already collects data to use in future sales campaigns, but parts of MyMagic+ will allow Disney for the first time to track guest behavior in minute detail.
For too long, the Internet of Things was something that was talked about in hyperbole at conferences and then forgotten. Now, finally, it is being operationalized. This particular account is of developments at GE. At Mount Sinai, patients get a black plastic wristband with a location sensor and other information. Similar sensors are on beds and medical equipment.
In the old days one needed supercomputers to analyze big data. American Express was the second largest customer for Cray after the NSA. Then you could do analysis on normal computer computers but with fancy software like T Cube. Now Microsoft plans on building these capabilities into Excel. Next year’s version of the Excel spreadsheet program, part of the Office suite of software, will be able to comb very large amounts of data.
In 1992, I wrote parts of a report for the National Regulatory Research Institute in the US on privacy and competitive implications for transaction-generated information (a term that has been eclipsed by the less informative “big data” in recent times). We covered all utilities, including electricity. Burns,Robert; Samarajiva, Rohan & Mukherjee, Roopali (1992) Customer information: Privacy and competitive implications, NRRI 92-11 . Columbus OH: National Regulatory Research Institute. Now, 20 years later, the issue is hot, the subject of a BBC story: The EDPS report voices concern over the “potential intrusiveness” of smart meters, which it says can track what members of a household do in the privacy of their homes.
Behavioral economics has brought to the fore the power of the default. As big data makes it easier to understand people’s actual behaviors and guide their choices, the power of the default is beginning to be fought over. Interestingly, it’s Microsoft versus the rest. Next came an incensed open letter from the board of the Association of National Advertisers to Steve Ballmer, the C.E.