Arrangements for project management, along with appropriate supervision of, and support for, project implementation present major difficulties which appear to arise from a combination of factors. The first of these is project design; the second, UNDP management capacity; and the third, the poor definition of the role of country offices and resident representatives. These three primary issues will be examined in this section of the report. More specific issues regarding project execution, the role of monitoring and the character of backstopping and technical support will be considered in the following section.
With few exceptions, the projects reviewed do not have (either in the project documents or in the projects as practised) clearly defined management structures that specify roles and responsibilities and indicate accountability for carrying out and supervising project activities in order to achieve objectives and results.
Beyond this, there is a characteristic vagueness in indicating precisely the responsibilities of the executing agency and those that are to be assigned to the implementing organization. In fact, judging from the experience of the projects reviewed here, the distinction between execution and implementation is problematic.
In addition, the role of UNDP headquarters (regional bureaux and other headquarters units) is unclear. As a result, in many cases, there has been inconsistency, uncertainty and slowness to act concerning the provision of backstopping support, regular monitoring and project decision-making. Whatever is specified on paper (for example, in corporate agreements between UNDP and the United Nations Office for Project Services (UNOPS)), in practice, for individual projects, there is a lack of understanding about what is to be done and who is to do it.
There is a demonstrable need for much more careful specification of roles and responsibilities in project documents. A clear specification of who is responsible for what as well as a detailed explanation of which responsibilities are assigned to each of the partners in the project will also create a basis for better communication and understanding among all the participants in the project. It is also important that the resident representatives be included in this process and that they be proactive in ensuring that they have a role in major intercountry programmes that will be implemented within their territories.
There are many complaints from country offices concerning regional bureaux and UNOPS. In part, this is to be expected as normal in field/headquarters relationships, but there appears to be the need for some sustained work in clarifying mutual expectations among the regional bureaux, the resident representatives and UNOPS. All of this may very well be covered in formal corporate documents, but there seem to be some major, practical problems that require a more collaborative strategy if they are to be resolved.
Consideration of roles and responsibilities must also extend to taking in a broader range of stakeholders. While national, regional (and local) partners may be involved in some form of overall steering or advisory committee, their roles in relation to project implementation strategies along with assigned responsibilities are often poorly spelled out. Associated with this is a characteristic failure to define the roles of national focal points or coordinators or to set out criteria and procedures for their selection. This is the case even with otherwise exemplary project documents and design frameworks, such as those for the Mediterranean Environmental Technical Assistance Programme, Phase III.
In UNDP, there appears to be a general problem of weakness in dedicated management capacity at the centre in supporting inter-country programming activities. On the basis of the data collected in this evaluation, the problem is found in all the regional bureaux as well as the divisions responsible for global and interregional projects*. This holds true regardless of the arrangements for project execution.
* The best practice was found to be in the Bureau for Arab States, which appeared, in the judgement of the evaluation team, to be the best organized in providing direction and support to regional prgrammes
While some large projects may be over-administered, on a general basis, there is a failure to build in sufficient resources - both managerial and technical/professional - to undertake essential supervisory and monitoring activities. As a result, many projects suffer from inadequate supervision and guidance from the centre.
Even where execution responsibilities are assigned to another organization, it is UNDP that remains responsible for ensuring that the project maintains its overall direction, that it is on the right road to achieve results and that project partnerships are working effectively. As matters stand, for the majority of projects considered here, the degree of involvement from New York and the quality of that involvement are insufficient.
By contrast with country programmes, intercountry programmes are more complex to manage and actors in the field are more distant from the responsible UNDP management unit. Because of the dispersion and character of project activities, the consequences of insufficiently attentive management, poor monitoring and/or inadequate backstopping are less immediate and stakeholders are less likely to bring pressure to demand immediate change. For all these reasons, UNDP must take a more proactive role in undertaking its broad management responsibilities.
In many cases among the projects examined, the human resources and the budget to support these forms of supervision and monitoring were inadequate, if dealt with at all. Participation in an annual project management meeting or a tripartite review (or its equivalent) is an inadequate substitute for active supervision and guidance on an ongoing basis.
It should be noted that in undertaking preparations for new phases launched in l996, two multiphase projects have made specific provision in the project design and budget to support a substantial effort in undertaking project monitoring. These are the Urban Management Programme and the Mediterranean Environmental Technical Assistance Programme.
In principle, country offices and resident representatives have a central role to play in ensuring that projects stay on track in each of the countries where they are implemented and to provide support and advice to national and local partners in intercountry programmes and feedback to headquarters. Unfortunately, in practice, the role of the country office is often unclear.
In many cases, PRODOCs assign responsibility to the field without developing a prior agreement with the resident representative concerned, without ensuring that resources exist to support the required level of activity, and without providing adequate briefing or substantive training to national officers, who will be assigned responsibility for project monitoring. As a result, the role of country offices and resident representatives in practice is haphazard at best. This state of affairs reflects badly on UNDP and brings credit to no one.
A small number of projects have successfully addressed this problem. All of them are examples of intercountry programmes that are implemented through a network of country-based operations. In such cases - the Local Initiative Facility for Urban Environment, External Resources Management and the Sustainable Development Networking Programme are the best examples - the projects are regarded as though they were country programmes. There is strong ownership for such projects by resident representatives and country offices. Where such a decentralized approach is combined with regular monitoring missions and active supervision and support from headquarters along with appropriate support from the executing agency, as appears to be the case with the three projects noted above, a close-to-optimal result may be achieved in terms of management and implementation practice.
The otherwise exemplary Alternatives to Slash-and-burn Agriculture (ASB) project, which is funded by UNDP but otherwise dealt with in a completely hands-off way, lacks any connection to the UNDP country offices in the countries where it operates. The responsibility for this state of affairs lies both with UNDP headquarters and the responsible division and, in some cases, with the country offices themselves. The same situation also applies in the case of the Special Programme for Research and Training in Tropical Diseases (TDR) project. Both ASB and TDR have extremely successful country operations. However, any benefits of synergy with other UNDP programmes that might be gained are not obtained because of the isolation of these projects from UNDP’s country portfolio.
Many difficulties in project execution and implementation were identified during the course of the present study. The majority can probably be traced back to weaknesses in design and in the specification of roles and responsibilities, as noted earlier. In this section of the report, attention will be drawn to some particular concerns, with detailed illustrations from the study. One concern, mentioned earlier, is the weakness in monitoring. Associated with this, although distinct as an element of project design, is the issue of the quality, relevance and timeliness of backstopping and technical support. These topics will be discussed in detail below.
The question of project execution is one that was investigated in some detail during the evaluation. A number of general concerns were noted earlier in the report. Otherwise, especially where the issue of executing agencies is concerned, the findings were rather inconclusive. As mentioned earlier, there were complaints from the field about UNOPS, yet across the sample of projects examined, no systematic pattern of deficiencies was identified relating to inadequate performance by UNOPS. Indeed, it should be noted that UNOPS is the executing agency in the case of several projects selected as best-practice cases, including the Local Initiative Facility for Urban Environment and the Sustainable Development Networking Programme.
There were no complaints from respondents questioned on project execution in Europe and the Commonwealth of Independent States, where UNOPS is heavily involved, or in Africa. The most frequent criticism raised elsewhere - and particularly in Latin America - related to slow disbursement and major delays in the transmission of essential information. While there is undoubtedly substance to these complaints - and the project documented several cases of excessive delay and poor communications - these were not echoed to the same degree in other regions.
In the judgement of the evaluation team, difficulties in this regard are, in part, a symptom of larger problems regarding poorly defined PRODOCs, under-budgeted projects, and a failure to attend to a proper specification of roles and relationships within the framework of the management, accountability and governance structure at the design stage.
It should be noted that UNOPS is undergoing a major effort at organizational change. It is moving to redirect itself to client service and is addressing many of the issues raised in this section of the report (i.e., issues regarding management structures and capacities as well as project execution, monitoring and backstopping/technical support). The transition will be difficult. A majority of officers and support staff are permanent employees of UNDP, on secondment. These individuals were trained over the years to perform different roles from those required of them in the reformed UNOPS. Despite this, it is to be hoped that in clarifying its own role as a businesslike provider of services, UNOPS may be of assistance to UNDP in addressing some of the major problems raised here concerning project design, management and supervision.
As matters stand, UNOPS is often brought into the picture very late, at a stage where the project design and many of the practical arrangements for implementation have already been finalized. This leaves little room for adjustment to ensure workable management and operational structures.
Weaknesses in monitoring and technical support (backstopping) were apparent across the range of interregional and regional programmes. This deficiency in project design and implementation has implications in setting limits to the effectiveness of any effort to identify lessons learned and build them into either the current project or its successors. The projects examined tended to lack any framework for systematic monitoring and evaluation. The absence of baseline information, milestones and performance indicators also renders proper monitoring extremely difficult.
Interestingly, among the exceptions to this trend are the two global projects examined, Special Programme for Research and Training in Tropical Diseases (TDR) and Alternatives to Slash-and-burn Agriculture (ASB). Alongside these should be placed one major interregional project, the UNDP/World Bank Water and Sanitation Programme. All of these projects are designed around a set of well-defined tasks, with a clearly specified division of labour and assignment of responsibilities for task performance at the global, regional, national and (where applicable) local levels.
All three projects are supported by organizations with strong professional/technical competence. In the case of the Water and Sanitation Programme, it is the World Bank, which has a strong technical capacity in the sector, along with the regional expert panels constituted by the project. For the ASB project, central support is provided by the International Centre for Research on Agro-Forestry (ICRAF) and other members of the Consultative Group for International Agricultural Research (CGIAR) family of institutions. TDR is supported by the World Health Organization (WHO) and a strong international network of researchers.
Satisfactory arrangements for monitoring and technical support derive from a well-structured organizational model for the project and strong stakeholder involvement in all components of project management and decision-making. The global projects rely very heavily on a number of steering groups organized at the global, regional, national and local levels, each with a set of responsibilities for project execution, implementation and coordination.
In the case of the Alternatives to Slash-and-burn Agriculture (ASB) project, ICRAF is the coordinating institution and the global coordinator of the programme operates from ICRAF headquarters. At the regional level, the regional steering groups ensure regional coordination, set priorities and assign institutional responsibilities. At the national level, the national steering groups monitor the participation of government agencies, NGOs and research and extension institutions. Technical and professional inputs are provided by national research institutions, international agricultural research centres and NGOs.
The Special Programme for Research and Training in Tropical Diseases (TDR) has a similar arrangement. The operational strategy is based on a network of existing research institutions and laboratories, with management responsibilities decentralized to national institutions and regional networks. Monitoring of the performance of each aspect of the programme is undertaken by panels of external reviewers from the scientific community. Performance of the monitoring function is facilitated by the computer-based management information system for tropical research (MISTER). Monitoring is also undertaken through periodic review of the programme by the various steering committees and the tripartite review meeting, which approves the work plan of the TDR project. What role UNDP plays in monitoring is not clear from the project documents or from interviews. Technical and professional inputs are provided by WHO as well as by bilateral donors and collaborating research institutions, such as the National Institutes of Health and the Centre for Disease Control (both United States-based institutions).
Other well-integrated projects with excellent support systems in terms of monitoring and technical/professional support include: the Urban Management Programme/Arab States Regional (UMP/ASR); the External Resources Management (ERM) project (Europe and the Commonwealth of Independent States); the Human Development and Governance project (Latin America); and the Sustainable Development Networking Programme (SDNP) and the Local Initiative Facility for Urban Environment (LIFE) (both interregional).
The arrangements to support the operations of the LIFE project are straightforward but worthy of attention. UNOPS is the executing agency. The country office is responsible for implementation at the country level, backstopped by the global coordinator in BPPS. Although the regional bureaux are mentioned in the PRODOC as backstopping the country offices and implementing the project, there is no indication, in practice, that they perform this role. The responsibility for day-to-day project management, coordination and monitoring is shared between the national project coordinator and the country office. These arrangements appear to have worked well in all countries where the project was assessed. For the most part, technical support is provided by the national coordinator and the national steering committee within the country of implementation.
One recommendation by the evaluation team is that the project could make more use of its experienced national coordinators in providing technical support to their counterparts in countries where the project has only recently begun. More effective use could be made of this mechanism in the Sustainable Development Networking Programme as well.
Of the three types of projects examined, in general terms, the global projects have the most effective and broadly based governance structures. In the two projects examined, Special Programme for Research and Training in Tropical Diseases (TDR) and Alternatives to Slash-and-burn Agriculture (ASB), governance structures revolve around a series of steering groups organized at the global, regional and national levels. In the case of TDR, the World Health Authority, a governing council of WHO, approves general guidelines for the programme. TDR's Joint Coordinating Board (JCB) is responsible for project governance. The various scientific steering committees (e.g., malaria, river blindness) review the state of the art, design strategies and make recommendations for project funding.
Similarly, the global steering group, chaired by ICRAF, sets general policy guidelines for the ASB project. The regional, national and local steering groups are responsible for the implementation of activities in support of project goals at their respective levels.
The governance structure for regional projects varies substantially from one project to the next. In Asia, none of the projects assessed had made provisions for a steering committee. The nearest equivalent to a steering committee was the Advisory Council of the Statistical Institute for Asia and the Pacific (SIAP), a highly representative body that meets once a year. The Council considers project performance evaluation reports (PPERs), provides advice and guidance to the Institute, and functions very much as a steering committee. There was a consensus among those interviewed that the Advisory Council was a very useful mechanism and that it had proved to be of considerable assistance in the proper functioning of the SIAP.
The Asia regional HIV/AIDS project, Strengthening Multisectoral and Community Responses to the HIV Epidemic in Asia and the Pacific, also provided for a similar mechanism: the Consultative Group, consisting of UNDP representation from headquarters (RBAP and the HIV/AIDS and Development Unit) along with the New Delhi country office*, WHO representation (WHO/Global Plan of Action (GPA) and the regional office) and UNOPS. The Group held two apparently useful meetings in l994 (the PRODOC actually required it to meet quarterly), but the mechanism was abandoned thereafter, seemingly without any consideration of replacing it. This appears to have been a contributory factor in the difficulties that the project encountered and the later decision to abandon the project.
*Both the project`s regional coordinator and the principal project resident representative (PPRR) were based in New Delhi.
The ASEAN ASP-5 Subprogramme on Capacity-building does not provide for a separate governance mechanism, presumably because ASEAN has its own standard arrangement of a joint management committee for projects funded by its donor partners. Unfortunately, however, it has not proved possible to activate the Joint Management Committee as a result of the delay in having its TOR approved by the appropriate ASEAN bodies. Consequently, the project is well into the second half of its lifespan of three years without there having been a single tripartite review and without the review of a single PPER. No action has been taken by UNDP to address this issue.
As a one-year programme of preparatory assistance, the Asian subregional project, Institutional Development at the Grass Roots for Poverty Alleviation, did not provide for a separate governance mechanism. A more elaborate structure is planned for the medium-term project, which has been developed as a result of the preparatory assistance programme.
In Africa, various arrangements have been put in place. These appear to have been reasonably effective in most cases. The Trade Development Promotion Programme (COMESA) is governed by UNDP and the two cost-sharing partners. The governance structure of the National Long-term Perspective Study (NLTPS), by contrast, involves a national steering committee in each country composed of representatives from NGOs, the private sector, the UNDP Resident Representative and the Government itself. The Assistance to the Africa Capacity-building Foundation (ACBF) project has a three-tier structure: the Board of Governors, the Executive Board and the Secretariat.
In Latin America, the arrangements for governance and accountability appear to be rather haphazard and often ineffective. In the case of two of the projects examined, namely, Regional Strategies for the Conservation and Sustainable Management of Natural Resources in the Amazon and Poverty Alleviation and Social Development, the project process as well as the results obtained would surely have been better had effective governance structures (allied with proper monitoring arrangements) been put in place.
In Europe and the Commonwealth of Independent States, decisions for the External Resources Management (ERM) project are taken at the Project Steering Committee meetings, which allow for a wide participation at all levels. Final decisions, however, are taken at RBEC in consultation with the resident representatives and county offices. This procedure seems to be effective. The country offices are in constant consultation with the national coordination authorities, which in turn are closely connected to the line ministries.
The Project to Support the Development and Enhancement of Democracy, Governance and Participation (DGP) has no formal governance structure, with decisions taken at RBEC. There is a consultative task force; however, it has no formal role. The plans for the next phase of the project include a proposal for a project steering committee with ten members. It is intended that there will be two resident representatives, two resident project coordinators from selected countries and representatives from UNOPS, the Management Development and Governance Division of BPPS, UNDP and the Organisation for Economic Cooperation and Development (OECD) as well as other donor organizations.
With the "Beijing Express", a project planned and implemented in the space of a few months and for which implementation itself took place over a matter of days rather than weeks, overall governance was necessarily in the hands of RBEC. However, practical decision-making took place in a participatory manner on a day-to-day basis on the train to Beijing.
In the Arab States, the Centre for Environment in the Arab Region and Europe (CEDARE) vests responsibility for governance in the hands of the Sponsors Committee, consisting of UNDP, the Arab Fund for Economic and Social Development (AFESD) and the Government of Egypt. Broader-based consultations with participating countries also take place and consideration is being given to developing a forum for a regular meeting of national focal point representatives.
On paper, the Mediterranean Environmental Technical Assistance Programme, Phase III (METAP 3) has developed a rather democratic governance structure, with decision-making shared among donors, national institutions, recipients, NGOs and beneficiaries. However, it appears that, in practice, major decisions are taken in Washington at the World Bank, as they were in previous phases of the project. Consultations are held with UNDP in New York and with the European Investment Bank (EIB), the major sponsors of the project. However, considerable work will be required to ensure that the proposed structure for decision-making actually works. Otherwise, the habits adopted in earlier phases of the project with its top-down decision-making style will continue to prevail.
As for the interregional projects, arrangements for the Local Initiative Facility for Urban Environment (LIFE) and the Sustainable Development Networking Programme (SDNP) seem to have worked reasonably well, with annual meetings of national coordinators and project management as well as national steering committees and, in the case of LIFE, a global steering committee. In several of the larger projects, matters have been less satisfactory. A number of such projects are characterized by a lack of effort to involve the participating countries and institutions in the management of projects.
In the case of the Urban Management Programme (UMP) and the interregional UNDP/World Bank Water and Sanitation Programme, the executing agencies involved, namely, UNCHS and the World Bank, have occupied most of the space available in determining the general direction and content of substantive programmes. In the particular case of UMP, the Programme Review Committee, comprised of the major donors, takes important management and operational decisions, which are binding rather than advisory. Participating countries have no means by which to have their views heard. On the other hand, at the regional level, the Africa region Water and Sanitation Programme attempts to elicit the views of client governments and institutions during the tripartite review meetings.
Probably the most important finding of the evaluation on this topic is that, by and large, projects are seriously underfunded. This state of affairs holds true even where there is cost-sharing with other donors. A consequence is that projects are often unrealistic in setting their objectives and try to do too much.
The weakness noted earlier in poor focusing of projects is compounded by underbudgeting of activities. The lack of provision for proper monitoring and technical support in many projects may, in part, originate in the lack of project finances.
No doubt this situation has been exacerbated by the dramatic reduction in programme funds available to UNDP. However, this apart, the trend is also associated with what appears to be an aspect of an organizational culture that rewards - or regards as good practice - a lengthy portfolio of projects, even where those projects lack adequate financing and where quality and results may be placed in question.
As to cost-sharing, there has been a reasonable degree of success in the case of the intercountry projects examined here in obtaining additional funding from other donors and, in a few cases, national governments. Given the difficulty in securing cost-sharing for intercountry, rather than country, programmes, UNDP performance in this area may be regarded as quite positive.
Although UNDP has succeeded in leveraging funds for interregional projects, it is the executing agencies in such cases who reap the primary benefits in terms of visibility and the building of core capacities. Thus the financial contribution of the World Bank towards the UNDP/World Bank Water and Sanitation Programme and the Urban Management Programme has been negligible in light of the fact that these programmes have done a great deal to raise the profile of the World Bank as a leader in these two sectors. By contrast, UNDP's role is not always visible. To balance this perspective, it must be said that without the ability to access the critical mass of technical and professional expertise available only outside UNDP, it would have been impossible for these projects to proceed. This issue will be looked at in more detail later when the question of the value added by UNDP to its projects is reviewed.
Moving to a different subject, very few of the projects examined have looked seriously at the issue of financial sustainability of activities beyond the period of UNDP/donor support. One exception to this is the Sustainable Development Networking Programme (SDNP). SDNP has been quite explicit about how it is going to use its limited funds and in limiting expectations of project partners with regard to the extent of the resources and support that will be made available through the project. SDNP has been rather successful in providing assistance to its national coordinators and the networks developed in-country in working towards financial self-support. For this reason, it stands out as a best-practice case in terms of using limited finances and addressing long-term financial-support issues.
The issue of under-budgeting noted above has another serious consequence. It means that activities begun by projects that are underfunded are unlikely to be completed satisfactorily. This is particularly critical in the case of Africa, where UNDP is supporting a number of extremely important regional initiatives. The lack of resources and the entitlement of all Member States within a region to participate in the project make it extremely unlikely that these projects will achieve the desired results. This has important implications for sustainability of project investments. The issue will be addressed in detail later when efforts are made to draw on the results of the evaluation to inform the best practice for focusing regional projects.
Mention must also be made of one other significant finding regarding financial issues. It was determined, to the surprise of the evaluation team, that most of the principals involved in the projects assessed knew very little about the financial status of the project concerned. The knowledge and sharing of knowledge on financing and project budgets are very poor. There is great uncertainty and fuzziness on this issue, which leads to many misunderstandings.