Development Sweden #43. Sweden Stops Aid to the Georgian Government

A newsletter on Swedish development cooperation and policy

Dear readers,

It happens a lot in Sweden and the world that have an impact on the Development Cooperation (ODA), so we will keep publishing Development Sweden more frequently to keep you updated. Let us know what you think! In this issue the discussion on the role of evaluators continue. Read the opinion article that is outside of the pay wall. And if you haven't already, we encourage you to subscribe for full access to all our content and to ensure you don’t miss any important insights during this dynamic period.

Welcome to a new issue of Development Sweden.

David Isaksson
Editor-in-Chief, Global Bar Magazine

Monika Gutestam Hustus
Editor, Development Sweden

What do you think we should write more about? Please give us feedback and suggestions.

Write to:
david@globalreporting.net
mgutestam@aol.com

For subscription related issues, contact:
lisa@globalreporting.net

If you would like to read the full newsletter you are most welcome to upgrade to a paid subscription.

Our headlines

Opinion: How Sida’s Evaluation System Can Be Improved

Sweden Stops Aid to the Georgian Government

Will the Government Stop the Olof Palme International Center?

---

Opinion: How Sida’s Evaluation System Can Be Improved

It is welcome that Sida is now changing its evaluation unit. This should mean that Sida takes a holistic approach to evaluation activities, which has been lacking for many years, writes Jocke Nyberg, independent senior evaluator who has previously worked for Sida in Colombia.

This is an opinion piece. The views expressed are the author's own.

Two managers of Sida’s evaluation activities express criticism that the evaluation unit in Sida’s new organisation has “been transformed into a team far down in the organisation.” The change is said to risk threatening its independent foundation. The chair of the OECD-DAC evaluation network draws the same conclusion. Sida responds that the evaluation team has not been placed in an operational department but remains a support function for management.

It is evident that something needed to be done, given the weaknesses that the evaluation unit at Sida has exhibited for many years. Better oversight of partners’ evaluations, more focus on reviewing results and accountability, developing and communicating systems for learning from and the effects of evaluation activities. These are improvements that can now become a reality if Sida’s management and board act appropriately.

Placing the evaluation team within a unit that falls under a support department for methods and results does not necessarily lead to reduced independence. Sida’s managers have professional integrity, and evaluations will continue to receive the same, or even higher, priority.

The National Audit Office’s Review

The new organisational changes at Sida align with the recommendations from the National Audit Office’s review, which states, among other things:

• Aid governance, follow-up, evaluation, and reporting must be more clearly connected.

• The government needs to conduct a review of aid follow-up, evaluation, reporting, and governance.

• It is difficult to assess the aid results against the goals set by the parliament based on the government’s reporting of aid results.

Therefore, the National Audit Office argues that authorities that evaluate should:

• Work more strategically and selectively with evaluation as a tool to gather information about the long-term effects of interventions, portfolios, support areas, and strategies.

• Develop support for Sida’s employees regarding when it is relevant to conduct evaluations of interventions and strategies and what those evaluations should focus on.

Poor Oversight of Partner-Led Evaluations

One of the weaknesses that Sida’s evaluation unit has had for many years is that it has not considered itself to have the resources to keep full track of so-called partner-led evaluations, which are evaluations led and conducted by entities other than Sida. These amount to at least a hundred per year and are by far the most common.

Within Sida’s aid, there are four categories of evaluations:

• Partner-led – commissioned and led by Sida’s partners, estimated by Sida to be about 100 per year.

• Decentralised – commissioned and led by Sida and project managers, mainly at embassies, about 25 per year.

• Central evaluations of strategic importance – commissioned and led by Sida.

• Internal reviews, 3-5 per year.

The partner-led evaluations are special. Partners formulate the terms of reference, decide which consultants can submit bids, evaluate bids, select evaluators, and are then recipients of the evaluation reports. Whether this relationship itself can be a form of conflict of interest has attracted the interest of some of Sweden’s leading anti-corruption researchers to investigate. Regardless of whether there is a conflict of interest, Sida’s evaluation unit has not prioritised the task of mapping partner-led evaluations for many years.

In 2019, Sida commissioned NIRAS, a consulting firm, to map the partner-led evaluations. NIRAS’s report, Meta-study of Partner-led Evaluations 2019, is available here.

In the report, one can read:

“Sida lacks a system for creating a comprehensive picture of the evaluations conducted by partners. Sida, therefore, has an insufficient understanding of the number, type, credibility, quality, and usability of partner-led evaluations. This information is important for Sida to help strengthen the credibility, quality, and use of evaluations.” (My translation from English, which was the report’s language).

Lack of Transparency in Procurements

Another systemic issue regarding evaluations is Sida’s long-standing inability to require that partners apply the same transparency and guidelines in the procurement of evaluators as Sida itself so commendably does. The agency is a model when it comes to transparency. Among other things, there are always clear criteria for how bids will be assessed/evaluated in procurements. There are often different percentages for qualitative aspects in bids (for example, methodology, relevant experience, work plan, understanding of the assignment, etc.) and how these are weighted in relation to the offered price. It is true that many partners are not authorities and thus fall outside the Public Procurement Act (LOU). However, Sida requires that partners demonstrate effective governance systems for, among other things, follow-up, evaluation, and procurement. In some cases, assessment criteria should be included in the procurement documentation, but they are rarely present. We are a few consultants who have collected more than a hundred so-called Terms of Reference where such criteria are missing.

As everyone knows, the greatest risks of corruption are associated with procurement, and this also applies to the aid sector. I am not saying that Swedish civil society organizations are corrupt, but the knowledge among their project managers about procurement is often inadequate. Therefore, Sida should ensure that assessment criteria are always included in partner-led procurements, with percentages for quality and weighting of quality/price.

Jocke Nyberg
Evaluator