External evaluation recommends that LIRNEasia not be used as yardstick


Posted on November 25, 2010  /  2 Comments

An external evaluation of the Pan Asian Networking program under which LIRNEasia was funded since 2006 has just been published on the IDRC website.  There are many references to LIRNEasia, one of the larger projects funded by PAN, but I found the para below the most intriguing:

Influence on telecommunications policy reform has been one of the strongest areas of the program’s outcomes, at least in terms of explicit causality, specifically from the work of LIRNEasia.  According to many informants, however, LIRNEasia, is a special case given the organizational culture, the numbers of people devoted to working almost exclusively on policy issues, the specific policy arena in which they work, and the strong personality at the center of the group. While LIRNEasia successes are notable, the external review panel urges the program not to set LIRNEasia as a standard for outcomes, since their achievements would be difficult to replicate elsewhere.

The quotation has been taken from the Findings Brief, prepared by the IDRC Evaluation Unit, though the same sentiments are also found in the External Review Report.

2 Comments


  1. At LIRNEAsia we have always managed to see action or research ideas in all types of situations. This is no different. I see some interesting possibilities for action research by IDRC or anybody interested.

    Take the four reasons cited by the panel to conclude that the LIRNEAsia cannot be replicated. I see in those a valuable distillation of observations that deserves what I call Strong Inferencing – i.e. not go for the first inference that comes to mind but consider all possible inferences. for example:.

    ALTERNATIVE INFERENCE:
    “We can find and nurture such a combination of attributes through a new funding mechanism”

    CRUCIAL EXPERIMENT:
    An open-ended call for proposals for think tanks requiring (1) a focus on a policy arena with a higher possibility of impact (2) a strong leader with credibility in the arena (3) an action plan to develop a team of policy intellectuals within a conducive organizational culture ”

    LIRNEasia with its regional experience in developing capacity can even be asked to help IDRC in carrying out the crucial experiment!

  2. I think you should see the glass half filled.

    Many think policy changes are about policy briefs with fancy graphs and flashy presentations that makes Accenture cry. These fools don’t achieve anything in real life. All such policy briefs are duly filed (in a folder or a computer server) and forgotten in time.

    Few think policy changes are about acting smart at the right moment, and hitting hard. They make their mark.

    While fancy graphs and flashy presentations can be replicated, the real efforts cannot. They need distinctive skills. Such skills are extremely rare and can never be replicated.

    You are lucky to have evaluators who not only recognise your true strengths but also make a great compliment (perhaps unintentionally) by rightly admitting their irreplicability. That is something I would like to hear from my own evaluators.