How evaluations get reported

Posted on March 14, 2012 / 3 Comments

Yesterday I heard a speaker at the ICTD 2012 conference in Atlanta say that 80% of the USD 4.2 billion spent by the World Bank on ICTs had been labelled as a failure by the Independent Evaluation Group. I had read the study in detail, and had blogged about it. I still wonder how language such as that below, taken from the report, can be interpreted thus:

In other priority areas, including ICT applications, the Bank Group’s contributions have been limited. Targeted efforts to increase access beyond what was commercially viable have been largely unsuccessful. Support to universal access programs was largely superseded by the roll-out of phone services by the private sector, in some cases supported by World Bank sector reforms. Access for the poor has been more effectively supported through general, non-targeted interventions focused on the enabling environment and direct support to private investments. The World Bank’s record in ICT applications has been modest, despite their significant role in Bank projects.

This to me suggests that general reforms were successful. In some cases, they contributed to failure of the “targeted efforts.” Unless 80% of the World Bank’s disbursements were for “targeted efforts” how could one claim 80% were failures?

3 Comments

Abu Saeed Khan
March 15, 2012, 1:18 pm
IEG has introduced itself in the report as follows:

“The Independent Evaluation Group (IEG) is an
independent unit within the World Bank Group. It
reports directly to the Board of Executive Directors, which
oversees IEG’s work through its Committee on Development
Effectiveness. IEG is charged with evaluating the
activities of the World Bank (the International Bank for
Reconstruction and Development and the International
Development Association), the work of the International
Finance Corporation in private sector development, and
the guarantee projects and services of the Multilateral
Investment Guarantee Agency.

The goals of evaluation are to learn from experience, to
provide an objective basis for assessing the results of the
Bank Group’s work, and to provide accountability in the
achievement of its objectives. It also improves Bank Group
work by identifying and disseminating the lessons learned
from experience and by framing recommendations drawn
from evaluation findings.”

Either World Bank or IEG or both of them have serious communication problem.
Rohan Samarajiva
March 15, 2012, 3:35 pm
If Abu can elaborate on the failure?

The IEG must identify failures. It has. It must communicate them. It has. It has not said that all World Bank initiatives, especially the sector reforms, were failures. But it has said that the targeted efforts, including universal service funds, were failures. What is the problem?
Rohan Samarajiva
March 19, 2012, 7:52 pm
The presenter of the paper seemed very upset by my comments. I promised to look at it again since my intention was not to upset or hurt. I looked over the text of the paper (I do not have access to the slideset which was the basis of my comment). The paper suggests that all of the USD 4.2 billion was a failure. Wrong. I guess my 80% claim above could possibly be wrong. Sorry about that.

The real issue is how we understand and report the specialized language of evaluations. On that I stand my ground.