Philippines: evaluation of ADB assistance to water supply services in Metro Manila

The Asian Development Bank (ADB) has had more than 3 decades of partnership experience in supporting the development of water supply services in Metropolitan (Metro) Manila. During this period, ADB provided nine loans (worth US$ 425.3 million) and seven technical assistance (TA) grants (worth US$ 3.6 million).

The ADB conducted a special evaluation study (SES) to assess the performance of their assistance to draw lessons for future partnership operations. The findings were used as input to the Philippines Country Assistance Program Evaluation completed in 2008.

Overall, the SES rates ADB’s assistance to improving water supply services in Metro Manila as “partly successful”. All projects are rated relevant [...] and were well coordinated with other development partners. With the exception of the Manila South Project, they were generally effective in achieving their immediate objectives, efficient in resource use, and likely sustainable from the financial and economic perspectives, which was eventually strengthened following upward tariff revision. The Manila South Project experienced a shortage of raw water and did not utilize the full loan amount; hence, it could not meet its immediate objectives.

Key lessons identified from experience with implementing water supply projects in the Philippines relate to the need (i) for greater attention to demand-side issues in project design; (ii) to reduce the high levels of NRW [non-revenue water] as an integral way of responding to rapidly increasing demand; (iii) for water utilities to give priority to adequate maintenance of existing water supply facilities and investment for rehabilitation to reduce NRW over the investment in new assets; (iv) for better readiness for project implementation; and (v) to improve transparency and accountability in preparing concession agreements.

Read the full evaluation study report (Sept 2008) here.

Project Performance and the Project Cycle: lessons from the Asian Development Bank

Despite a general trend to higher numbers of projects being rated successful or better over the past few years, about 30% of ADB’s project portfolio of the Asian Devlopment Bank (ADB) is still rated less than successful during the evaluation phase.

In order to better understand why this happens, a business practices study was undertaken to look at project performance in the project cycle.

The retrospective analysis was able to determine 17 common factors which affected project success. The most frequently occurring of these factors were deficient capacity-building and ownership measures during project design, insufficient or deficient supervision by ADB during project implementation, less than rigorous ADB internal review, and inadequate technical analysis or inappropriate project design.

Based on an analysis of each stage of the project cycle, three main issues were highlighted:

1. Overloading of the Project Team Leader
Recommendation. Project teams have been less than effective in adequately supporting the project team leaders. Management should continue to explore modalities for strengthening team work (including through appropriate incentives) to have better burden sharing in teams.

2. Budgetary Support not Commensurate with Expanding PPTA (project preparatory technical assistance) Scope
Recommendation. Management should review PPTA funding requirements in light of the current and future scope of PPTAs and ensure the necessary PPTA funds.

3. Need for Greater Project Supervision to Avoid Implementation Delays
Recommendation. Current resource allocation for project implementation supervision should be reexamined for adequacy. Start-up delay, which is a common feature for most projects, should receive special attention, and adequate resources should be made available including thorough flexible use of TA and loan funds and/or greater internal human resource allocation.

Read the full ADB report (September 2008)

Lessons for out-scaling and Up-scaling from DFID’s RNRRS Studies and Research

Key lessons for up-scaling and out-scaling of DFID research

Authors: W. Richards; Research Into Use Programme, DFID
Publisher: Department for International Development, UK, 2008

Full text of document

This report consists of a series of short syntheses which bring together key lessons for up-scaling and out-scaling research based on 19 key reviews, summaries and reports detailing DFID natural resources research. Each synthesis provides background information and key points, while the lessons learned are illustrated using examples and case studies.

Key lessons learned include:

• Start from what’s on the ground: most successes in spreading the uptake of research findings came about when the people involved understood  what was happening on the ground, including indigenous  knowledge on practices and processes, and worked with it. This is true at the local level, in communities, and at administrative and government levels.
• Understand the key players and build relationships: starting from what is on the ground leads to an understanding of who the key players and local champions  are, and how they operate. This mapping of institutions and understanding of their roles in a network are pre-requisites for deciding who we need to invite to join ‘coalitions’ (ie the groups of people and organisations who get
  together to out-scale or up-scale research findings). Understanding what they want and where they are coming from also helps build trust and relationships with them.
• Be realistic about what can be achieve: sometimes one has to face up to the fact that the conditions for uptake of research findings just aren’t  conducive. For example, experience shows that any innovation in developing countries where innovation frameworks and infrastructure are weak is tough. Outscaling and up-scaling research findings have a better chance of success where governments have encouraged
• Partner, communicate, advocate and build capacity strategically: an understanding of the local context, the key players and what can realistically be achieved will help develop clear strategies for ‘how’-partnering, communicating, advocating and building skills-out-scaling and up-scaling will be achieved. adoption of new technologies by, for example, supporting producer prices, subsidising inputs and credit for new technologies, and investing in irrigation, roads and marketing systems.
• Capture your own learning and learn from others: it is essential to capture  opportunities to learn and share learning with others. Learning what works and what does not is going to be vital in out-scaling and up-scaling. A good way to capture lessons learned is through a formal system of monitoring and evaluation that emphasises learning rather than accountability.

However, one size does not fit all. The lessons learned will be worked, reworked and adapted by practitioners to different  situations.

Source: ELDIS

Aid effectiveness in the water sector: lessons from Bangladesh, Uganda and Ethiopia

A study by the Overseas Development Institute (ODI) reviewed the extent to which the five Paris Principles (PPs) on Aid Effectiveness (AE) as set out in the Paris Declaration (PD) on AE: ownership, alignment, harmonisation, managing for results and mutual accountability, are being applied in the water and sanitation sector. Comparisons were made between the water, health and education sectors.

The approach included in-depth case study research in three countries (Bangladesh, Ethiopia and Uganda) and a broader document review. From this evidence the study aims to identify ways in which external support to the water sector can be delivered more in the spirit of the PD.

ODI found that the water sector was not consistently underperforming in the three country case studies. Rather, situation is one of ‘fluid dynamics’ – the sector is moulded by the surrounding political-economic context in which it is situated, i.e. predominantly by the national governance environments in the countries examined.

The following conclusions and recommendations emerged:

1. The broader governance environment is a more important influence on progress against the Paris Principles than sector characteristics. Aspects of the governance context beyond the sector, rather than sector characteristics alone, are a key influence on progress.

2. The perception that the water sector is lagging behind is not supported if the spirit rather than the mechanics of the PD is considered. SWAPs or other instruments should not be seen as a one-off step but rather as an ongoing dynamic process, a platform for learning for both donors and recipients. The pace and level of engagement depends on the opportunities at hand in a given situation provided by the prevailing politicaleconomic context.

3. Some dynamics are specific to particular (sub-) sectors and require a more targeted approach. The type and number of actors and type and levels of financing differs between
sectors. The water sector is likely to be more affected for instance by new donors such as China with a particular emphasis on infrastructure development while the health
sector stands out for receiving high levels of aid from private foundations and multilateral funding initiatives.

4. Some aspects of system alignment such as PFM and procurement cut across sectors. Engagement at a higher level than the sector may be more effective than trying to find solutions within a sector.

5. Paucity of data prevents the measurement of progress against the PP for AE at sector level. There is
also currently no evidence that the fulfilment of the PP leads to better development outcomes e.g. increased access to WSS.

Welle, K. … [et al.] (2008). Fluid dynamics? : achieving greater progress on aid effectiveness in the water
sector – lessons from three countries : final report to the Department for International Development. London, UK, Overseas Development Institute. 57 p. Read full report

A shorterned version of this paper was published in 2009 in Water Alternatives:

Welle, K.; Tucker, J.; Nicol, A. and Evans, B. 2009. Is the water sector lagging behind education and health on aid effectiveness? : lessons from Bangladesh, Ethiopia and Uganda. Water Alternatives ; vol. 2, no. 3 ; p. 297‐314. Full article

Global Water Partnership: joint donor external evaluation

The Global Water Partnership (GWP) was established in 1996 as an international network promoting an integrated approach to water resources management. The Performance Assessment Resource Centre (PARC) carried out an evaluation of the GWP for its Joint Donor Group of six European bilaterals (Denmark, UK, Netherlands, Germany, Norway and Sweden). The evaluation focuses on the 2004-2008 strategy period.

One of the main achievements of the GWP was that it has expanded from 28 Country Water Partnerships (CWPs) to 71 and from 9 Regional Technical Advisory Committees (RTACs) to 12 Regional Water Partnerships (RWPs) during this strategy period. The number of members has tripled from 600 to over 1800. There is also a greater awareness of IWRM across the network.

A major challenge is that GWP’s international policy advocacy has not kept pace with emerging issues. Whilst the central message of IWRM is still valid, the global advocacy role needs reinvigoration to remain relevant and demonstrate that GWP is abreast of topical issues that affect its members. The financial sustainability at any level of the partnership was said to be doubtful.

A key recommendation is for the GWP to re-position itself as a globally recognised advocate of IWRM that clearly articulates the relationship between IWRM and emerging global challenges. The regions should become stronger and technically support the countries. Most resources should go to the country, then regional, then HO level.

Read the full evaluation report here

Watershed development: evaluating social impacts in India

Research from Oxford University, in the UK, evaluates the social impacts of watershed development in Madhya Pradesh, India. ‘Propensity score matching‘ was used to compare the impacts from a control micro-watershed, where no interventions had taken place, and a treated micro-watershed.

The analysis focuses on changes in agricultural income and access to domestic water following several development interventions in the treated watershed. These include pond construction, tree planting and water harvesting terraces.

One of the findings was that there was a reduction in domestic water collection times for those households that had previously spent the longest time collecting water. However, despite these benefits, the majority of people were still without basic access to water.

To achieve more effective development impacts, there is a need for more rigorous and objective evaluation methods.

Read more: id21, 5 Apr 2008