As part of electricity work LIRNEasia has made recommendations on the importance of DSM in Sri Lanka. Effective DSM is not possible without smart meters and that was an important part of the message, when we were invited by the Colombo Electricity Board (CEB) to share our research with their senior management. So it was with great interest that I perused the research of one of the winning finalists from a Big Data Challenge conducted by Telecom Italia (and partners) with data pertaining to the territories of Milan and of the Autonomous Province of Trento in Italy. The datasets covered telecommunications, energy, weather, public and private transport, social networks and events. The researchers utilized smart meter data and behavioral data extracted from the Telecom Italia’s transaction generated data to predict peak daily energy consumption and also the average daily energy consumption for each line through the electrical grid of the Trentino Province.
The work that we have been doing using mobile network big data over the last year, has been challenging on many fronts. I’ve spent some time reflecting on some of the analytical challenges that are faced in the Big Data paradigm and the common fallacies that I sometimes find in the broader discussion I see on the subject. What is below are some of my preliminary thoughts, which I am working up into a paper. Still a work in progress. Comments welcome.
We are in the big data for development space, but we keep an eye on what is happening in the big data for profit space. And IBM is a company we watch. Since 2005, IBM has invested $24 billion in the data analytics business, including $17 billion on 30 acquisitions. In 2013, the business generated nearly $16 billion in revenue. So if IBM makes less money in the future selling hardware, software and services for corporate customers’ data centers, it plans to make more money helping its customers make sense of data — to cut costs, increase sales, innovate and personalize product offerings.
Is their ability to generate massive transaction-generated data streams that will yield insights into human behavior. Packed with sensors and software that can, say, detect that the house is empty and turn down the heating, Nest’s connected thermostats generate plenty of data, which the firm captures. Tony Fadell, Nest’s boss, has often talked about how Nest is well-positioned to profit from “the internet of things”—a world in which all kinds of devices use a combination of software, sensors and wireless connectivity to talk to their owners and one another. Other big technology firms are also joining the battle to dominate the connected home. This month Samsung announced a new smart-home computing platform that will let people control washing machines, televisions and other devices it makes from a single app.
Prof Hal Abelson of MIT recently shared his thoughts on privacy in the digital realm, at a online alumni webcast. Amongst the noise that one hears on this topic these days, his thoughtful comments resonated. Partly for sharing and partly for my own memory, I felt it justified a blog post and I capture his main points below: People don’t really know what they want when they think of privacy. They describe their privacy needs through use-case scenarios for e.g.
I recently participated in a panel on “Big data in the telecommunications industry” at the 11th World Telecommunication/ICT Indicators Symposium (WTIS) held in Mexico from 4-6 December 2013. Going by the feedback from the Q&A session, two aspects rose to the front: Firstly the issue of “privacy” is on everybody’s mind going by the number of questions that came from the audience. Everybody seems to have his or her own viewpoint. UN Global Pulse, whilst acknowledging there are valid concerns that must be addressed (and they have a set of privacy guidelines for their own work) clearly doesn’t want the concerns to derail the efforts to utilize telecom network big data for social good. Telefonica, as an operator, was quick to point his or her own set of privacy guidelines that inform their big data work.
Today, our CEO Helani Galpaya was on a panel “Harnessing the power of convergence and big data for enterprise success” at a Sri Lankan summit called “Enterprise 2.0: building future ready enterprises” (full video of the panel session is available HERE). I thought some of the ideas she proposed about were worthy of further discussion. LIRNEasia is curently working on utilizing telecom network Transaction Generated Information (TGI) to conduct public interest research using big data. One of her comments was about how companies are not fully appreciating the value of the data that they have.
The New York Times carried a story on “big data for development” that featured Global Pulse, the UN initiative seeking to harness the potential of data to address development questions, much like what we are doing in our current research. The efforts by Global Pulse and a growing collection of scientists at universities, companies and nonprofit groups have been given the label “Big Data for development.” It is a field of great opportunity and challenge. The goal, the scientists involved agree, is to bring real-time monitoring and prediction to development and aid programs. Projects and policies, they say, can move faster, adapt to changing circumstances and be more effective, helping to lift more communities out of poverty and even save lives.