Last year we conducted research to explore the possibility of leveraging online job portal data for economic analysis in 13 Asia Pacific countries, as a part of a project for the Asian Development Bank. We examined the types of information available on major portals across the region, to discern the nature and format of available data. We also tested and refined methodologies to analyse a dataset comprising online job vacancies sourced from a Sri Lankan job portal, to demonstrate use cases for exploring the impacts of shocks on the labour market. The first step in this exploration was to review where in practice online job portal data has been used, to identify the methods and techniques available along with their strengths and limitations. The full review is published below. 
Last year we conducted research to explore the possibility of leveraging online job portal data for economic analysis in 13 Asia Pacific countries, as a part of a project for the Asian Development Bank. We examined the types of information available on major portals across the region, to discern the nature and format of available data. We also tested and refined methodologies to analyse a dataset comprising online job vacancies sourced from a Sri Lankan job portal, to demonstrate use cases for exploring the impacts of shocks on the labour market. The first step in this exploration was to review where in practice online job portal data has been used, to identify the methods and techniques available along with their strengths and limitations. The full review is published below.
Firms have always had an interest maintaining the loyalty of their customers. This has also involved knowing more about the customers. In a discussion of subscription models, the Economist, refers to what may happen because of restriction on data that may emerge because of the Cambridge Analytica imbroglio. Subscription models are becoming more popular, in part because technology has made it easier to rent rather than own assets. Instead of buying software, for example, users can get access to it as a cloud-based service.
Privacy is a subjective thing. Some of it is from the inside of the individual; some is social. It’s not immutable. It’s not the same across societies. Now after Yudhanjaya’s reflection on the Chinese social credit system, we are more interested than ever in what is going on in China.
For more than a year, we at LIRNEasia have been working on the analysis of images. The NYT story on Stanford researchers working on Google Street View describes the potential well. For computers, as for humans, reading and observation are two distinct ways to understand the world, Mr. Lieberman Aiden said. In that sense, he said, “computers don’t have one hand tied behind their backs anymore.
It is natural to think of state entities as the key actors in south-south cooperation (SSC) for improving public-service delivery. But as the highlighted example of Bangladesh’s Union Digital Centers (UDCs) shows, non-state actors can play important roles in public-service innovation. If true innovation is the objective, it would behoove the UN Office for South-South Cooperation and other interested parties to cast the net wider to include innovative organizational mechanisms as well as government innovations.
I will be participating in a panel on using technology for governance at the Global Technology Summit on 7-8 December 2017 in Bengaluru. This is an annual event organized by Carnegie India. Problems associated with policy implementation can be potentially solved through partnerships with the private sector and the use of technology. But to gain maximally from such efforts, both policy makers and executors have to realign their vision and understand the technology space for what it is: a vibrant zone of activity, willing to test, experiment, fail, and learn. This fundamental shift in approach from existing governance models presents both huge opportunities and challenges, as this panel probes.
by Keshan de Silva and Yudhanjaya Wijeratne One of the most useful datasets we have is a collection of pseudoanaonymized call data records for all of Sri Lanka, largely from the year 2013. Given that Sri Lanka has extremely high cell coverage and subscription rates (we’re actually oversubscribed – there’s more subscribers than people in the country; an artifact of people owning multiple SIMS), this dataset is ripe for conducting analysis at a big data scale. We recently used it to examine the event attendance of the annual Nallur festival that happens in Jaffna, Sri Lanka. Using CDR records, we were able to analyze the increase in population of the given region during the time of the festival. A lengthy writeup describes it on Medium, explaining the importance of the festival and the logic for picking it.
Perera-Gomez, T. & Lokanathan, S.
As part of our big data work, we’ve been thinking about the opacity of the algorithms we use. The pretty picture and tables that result from the research are persuasive, but if people wanted to know how they were derived, it would not be easy to explain. But then, we have to always think about the alternative. A method may be familiar and may have been used for decades if not centuries. But that does not necessarily make it fair. 
The digital world is exploding with uncountable data. Millions of users generate information via thousands of sources every day. This data is then consumed for a number of purposes from business to entertainment. Is there a purpose and potential for big data beyond business and entertainment? The big data team at LIRNEasia is trying to answer this question.
Preparing for a session of the Privacy Advisory Group of UN Global Pulse and the UN Privacy Policy Group on 17-18 April, I had cause to reflect on some moves to develop new definitions (sensitive data, meta data and micro data). I may change my mind after listening to the deliberation, but here’s my starting position: Definitions are developed with some purpose in mind. A definition that is appropriate for one purpose may not be useful for another. Definitions embody assumptions and agendas. I believe that personally identifiable information (PII), a venerable category of data deeply embedded in privacy theory and practice is the only category of data requiring hard protection.
LIRNEasia research fellow, Dharshana Kasthurirathna, Ph.D. presented a paper, ‘Detecting Geographically Distributed Communities using Community Networks,’ at the International Workshop on Mining for Actionable Insights in Social Networks that was held in conjunction with the Tenth ACM International Web Search and Data Mining Conference in Cambridge in February 2017. The paper was co-authored by three LIRNEasia research fellows (Dharshana Kasthurirathna, Madhushi Bandara, Danaja Maldeniya) and Mahendra Piraveenan from the University of Sydney. Based on the presentation, there was an invitation to extend the paper to be submitted to a special issue of the Elsevier Information System’s journal, with a draft journal paper due in April 2017.
Lokanathan, S., Perera-Gomez, T., Zuhyle, S.
The UN Data Innovation Lab invited LIRNEasia to share our experience in entering data partnerships and the challenges associated with the same, at a workshop held in Cape Town on the 19-20 January 2017. The workshop, co-hosted by UN Global Pulse, centred on designing data capacity within the UN system. The session conducted by LIRNEasia was attended by representatives from a range of UN agencies including UNICEF, UN WTO, UN Women and UNAIDS. In addition, other participants at the session included representatives from Flowminder and Facebook. I had the opportunity to share LIRNEasia’s experience in building relationships with the government and private sector data providers, particularly in terms of leveraging mobile data for urban planning and traffic management in Sri Lanka.
LIRNEasia is currently hosting Dr Ayumi Arai from the University of Tokyo’s Center for Spatial Information Science. She is also a Research Fellow with LIRNEasia collaborating on our big data for development research in Sri Lanka. We took the opportunity to organize a lecture for her yesterday (14th July 2016) for the senior staff of the Department of Census and Statistics (DCS) Sri Lanka, as preamble to a longer discussion with the department to collaborate with LIRNEasia and our partners on big data and official statistics in Sri Lanka. Dr Arai’s talk was on her ongoing Dynamic Census research work in Bangladesh which utilizes mobile network big data and official statistics to provide spatio-temporal insights on the socio-economic and demographic characteristics of the population at high granularity and high frequency. The slides from her talk are available HERE.
LIRNEasia was a core partner for Sri Lanka’s first national summit on “Foresight & Innovation for Sustainable Human Development” that was convened by UNDP and the Ministry of National Policies and Economic Affairs. Held in Colombo from 24-25 May 2016, the summit brought together more than 300 people from government, private sector, and civil society from all over the country. Developing foresight and fostering innovation is a priority for the government and underscored by the Prime Minster’s attendance at the event. I spoke on the first day after the opening. My talk was on the leveraging both new and traditional data if the goal is to get towards real-time responsiveness and enhanced resilience.