As part of our big data work, we have been looking at sources other than mobile network big data for socio-economic monitoring. Night lights images from satellites was the favorite. But I’ve been always skeptical, partly because of looking down from the New Delhi-Colombo flight which flies through the middle of India and then right down the island from Jaffna. The intensity of the lights is so much higher in southern India, than in Sri Lanka. This story is about daytime images taken by satellites.
That’s title of a report Sriganesh Lokanathan and I completed for the New Venture Fund. Here is an extract from the executive summary. Much of the discussion of the socio-economic implications of behavioral data has focused on the inclusion of more citizens and more aspects of their lives within the sphere of control enabled by pervasive data collection. Effective public policy rests on good information about problems and the efficacy of the deployed solutions. Governments obtained such information through National Statistical Organizations (NSOs) in the 19th and 20th Centuries.
Big data is sexy these days but still it’s a big deal to get coverage in the New York Times for research conducted in Rwanda. Josh’s work is complex and involves training data sets and also the use of multiple kinds of data. He and his colleagues relied on anonymized data on billions of interactions, including details about when calls were made and received and the length of the calls. The researchers also looked at when text messages were sent, and which cellphone towers the texts and calls were routed through in order to get a rough idea of geographic location. “So it’s the who, where and when of the call, but not the what or the why,” Dr.