FIFTH CYCLE SPECIAL PROGRAMME RESOURCES (SPR)
Midterm Evaluation
Submitted by:
Richard Huntington & Elena Marcelino
with Assistance from:
Rasheda Selim
August 1996
Table of Contents
SECTION ONE: OVERALL SPR EVALUATION
I. Summary
II. Fifth Cycle SPR Programme
III. SPR Critical Issues
IV. Fifth Cycle Management Issues
V. Implications and Recommendations for Sixth Cycle Arrangements
SECTION TWO: SUMMARY PROGRAMME EVALUATIONS
THEMATIC AREA PROGRAMMES
SELECTED NEW AND SPECIAL PROGRAMMES
DISASTER MITIGATION (A 1-4)
I. SUMMARY
A. Evaluation
This evaluation of overall SPR programming is a desk study that reviews and summarizes the completed mid-term evaluations of the individual SPR components, and presents an overall meta-evaluation of the Fifth Cycle SPR experiment. The purpose of the evaluation is to examine to what extent the SPR programme has succeeded in meeting its goals of rapidly providing a flexible, innovative, and catalytic mechanism for strengthening national capacity, especially in the thematic areas of emphasis of UNDP. At the same time, the evaluation reviews the experience of SPR, positive and negative, in light of the successor funding arrangements established for the Sixth Cycle.
The evaluators carried out extensive interviews with all SPR programme managers, many sub-programme managers, as well as other key players in the Bureau for Policy and Programme Support (BPPS), the Office of Evaluation and Strategic Planning (OESP), the regional Bureaux, and headquarters visitors from country offices. Additionally, the evaluators reviewed the mid-term evaluations, annual progress reports, and other project documents, reviews, and products. (See Annex A: List of Persons Interviewed; and Annex B: List of Documents Consulted.)
The evaluation of SPR Component A, Disaster Mitigation, is not yet complete, but the SPR evaluators have relied on interviews with the Component A evaluators as well as on their substantial evaluation progress report. The evaluators are not reviewing SPR Component F, Assistance to the Palestinian People, or Component G, Contingencies.
The first section of this evaluation provides an overall assessment of the SPR endeavor, according to the unique requirements of the SPR programme. We emphasize the experiences of the larger programmes, since they represent the predominant resources expended; and refer to selected smaller programmes in instances where they provide examples of interesting or important, positive or negative, evaluation findings.
Section Two provides brief summary evaluations for SPR Category A (Disaster Mitigation), Category B (Six Thematic Areas), and selected programmes from Categories C (New and Special Programmes), D (Aid Coordination), and E (Programme Development). The selections are made to provide more in-depth information for the examples that were given in Section One. No inference is to be made regarding the quality of programmes not here summarized. The reader is referred to the 1995 Annual Report on SPR for summaries of activities and accomplishments of all 27 lines of the SPR Programme.
B. Overview of SPR Programme
The Fifth Cycle SPR Programme, with a final allocation of $284.3 million for its 27 components, was a significant departure from previous modes of funding at UNDP. For the first time, a relatively large amount of funding was available for programming outside of the normal IPF entitlement structure, country offices were required to formally compete for programme funds, and BPPS (BPPE) moved from its limited policy role into a more intimate involvement with project operations.
Despite the late start in the Fifth Cycle SPR, it is almost fully programmed, with $264.4 million programmed by December 1995. The SPR resources are intensely fragmented into over 950 separate projects. Over 60% of the funds are executed by UNOPS, with only 17% executed by national entities. The remainder is executed by specialized agencies.
C. Assessment of Special SPR Programming Criteria
The SPR Programme is intended to be innovative and catalytic, with the main focus on building national capacity in the six thematic areas of special UNDP programme focus. More broadly, it was intended that SPR contribute overall to improving and sharpening UNDP's delivery of development and humanitarian assistance.
Innovation. The independent evaluations of the major SPR components and programmes report an impressive pattern of innovation. The innovations include new programme ideas, new methodologies for involving participants, new approaches to delivering assistance, and creative utilization of new technologies. Not all innovations begin at headquarters, and a number of SPR projects have recorded, disseminated, and adapted innovative ideas that were first tried out in communities in developing countries. Some programmes, however, especially in the areas of poverty and NGOs, have been criticized for continuing to use their many small grants to support very familiar types of interventions.
Catalytic Programming. The central criterion of the Fifth Cycle SPR is that it must be "catalytic", dynamically linked to country programmes, other international development agencies, and to national programmes. The evaluators find that the Fifth Cycle SPR is a gregarious set of programmes, in broad dialogue with a wide range of entities concerned with human development.
Catalytic Resource Mobilization. Many SPR projects and activities are in one way or another co-financed or parallel financed with non-SPR resources, including IPF funds, funds from other donors, national resources, and in-kind and financial contributions from private sector corporations and foundations. According to BPPS records, the value of the formal cost-sharing arrangements with non-UNDP funds is approximately $105 million on over 100 SPR projects, equal to almost 40% of the total programmed SPR funds. This number, however, does not nearly capture the total of significant parallel and subsequent financing of SPR activities.
Catalytic efforts to build national capacity. Many SPR activities are designed to function as institutional catalysts. Much of the funding supported workshops, public sector management reform, networking, and training, activities that in theory should add to local and national capabilities. For many of these activities, it is difficult to assess whether these activities actually lead to strengthened national capacity. Most notably in the NGO and Poverty components, many of the small workshop and training support grants are one time events, without resources for significant preparatory consultation or follow-up. Among the six thematic components, those with fairly high levels of resources and/or sharply defined technical and conceptual focus can demonstrate significant impact on overall national capability in their area.
Catalytic new programming processes. A closely related aim of SPR is to mainstream the UNDP thematic emphasis into the national IPF programming process, as this process increasingly becomes more independent through the application of the Programme Approach and National Execution. The results in this area are mixed. The delay in funding SPR, meant that the Fifth Cycle of IPF programming was well under way in most countries by the time the relevant SPR activities had the opportunity influence the process. In reality, many of the Fifth Cycle SPR programmes, operated quite independently of the IPF programmes in most countries. Also, it is difficult to assess to what extent the Fifth Cycle SPR programme is assisting in the mainstreaming of its emphases into Sixth Cycle IPF programming.
Sustainable Human Development. Mid way through the Fifth Cycle, in 1994, the SPR programme in support of Sustainable Human Development (SHD) was established in order to promote the new emphasis of UNDP, including four areas of concentration (poverty, employment, women, environment). Under this new SPR initiative, 81 countries have received grants that support national institutions and consultants to develop national human development reports, formulate national sustainable development strategies, or to carry out training and information programmes on the concept of sustainable human development. In all instances, with a catalytic focus on utilizing "national entry points", the small grants directly address the building of national capacity for SHD analysis and implementation, although in most regions (with the positive exception of RBEC) it is too early to demonstrate results.
D. Management Issues
Programme Management. SPR, being new, lacked an established and practiced system of management. Programmes were well-managed or not, largely as a function of the individual programme manager, in the case of the smaller components, or as a function of the adequacy of staffing and/or previous establishment for the larger programmes.
At first, the Fifth Cycle SPR represented a significant shift of authority and resources to BPPS. This shift, combined with increasing programming authority in the field left the role of the regional bureaux uncertain and reduced. Gradually in the course of SPR implementation, many SPR programmes developed an effective partnership with the regional bureaux. None-the-less, there continues to be a somewhat bewildering variety of relationships, as some SPR components respond directly to requests from country offices, other divide the SPR resources like entitlements to the regional bureaux, and other SPR components emphasize a supply-driven approach.
Dual Systems. SPR and IPF operated largely as separate realms, with different systems of financial allocation, project monitoring, execution, and financial reporting. In most country offices, SPR activities are viewed by staff as an additional item, outside their regular country programming. Country office ownership of many SPR activities is reported to be weak.
Project Selection Processes. SPR, for the most part, began as a user-unfriendly programme. The multiplicity of requirements and programming modes were confusing to country office programme officers and regional bureau staff alike. Also, country office personnel were not used to the idea that they needed to compete for UNDP funds, and were resentful when their efforts at preparing an SPR proposal were not successful. Given the limited resources of many SPR programmes, the disappointed applicants by definition outnumbered the selected grantees. Many SPR selection processes were unclear, especially at the beginning. The competitive nature of the process was not always cost effective, especially considering the small size of the grants. Over-stretched country office personnel had cause to resent the SPR hoops through which they were led in efforts to secure small grants. However, most often proposals were rejected because they failed to meet the criterion of being innovative or catalytic, and as such were properly rejected.
Reporting, Monitoring, and Evaluation. Each of the SPR programmes submits an annual progress report to BPPS, which then prepares an omnibus SPR annual progress report for submission to the Executive Board. Reporting from many of the small field projects and grants is uneven, and many SPR programme managers report that they have a difficult time assessing the fragmented activities for which they are responsible. Since many of the activities are small, one time events, it is likewise difficult to monitor their outcomes.
SPR Programmes with budgets in excess of $10 million, new phases of continuing subprogrammes not previously evaluated, and programmes under $10 million are each subject to specific evaluation requirements. Compliance with the evaluation requirements is strong, with 17 of the 27 programmes implementing independent evaluations, and 8 implementing internal self-evaluations. SPR programmes have shown a tendency to make mid course corrections in response to these evaluations. Since these evaluations were performed in most instances after a brief interval from project initiation, they of necessity focus more on process and conceptual issues than on measurable impact.
Financial Management. BPPS established a special financial unit for the SPR programmes. This unit, beginning operation after SPR programming was already in progress, struggled to sort out the finances of the multitude of small projects and grants that make up SPR. At this point the funds appear to be adequately accounted for and tracked by this independent parallel system that needed to be erected from scratch in a very short space of time. The task of tracking and accounting for the funds in over 950 separate SPR projects is made easier by the fact that 62% of the funds are executed by UNOPS. Although the information system of the special SPR financial unit provides adequate financial accountability, it does not easily provide aggregated information on inputs such consultants (international & national), training, subcontracts, and so forth.
E. Overall Assessment of the Fifth Cycle SPR
Despite the inefficiencies of its managerial structure and the "unfriendliness" of its many project and grant selection criteria, SPR provides a flexible response mechanism for highlighting thematic UNDP concerns and developing new mechanisms for delivering development assistance. Perhaps most important, the availability of SPR funds aided in the transformation of UNDP from simply a funding agency toward becoming an international organization able to provide conceptual, policy, and technical leadership in important fields. In areas such as environment, public sector management, HIV/AIDS, post-war economic rehabilitation, gender analysis, human development reporting, and others, SPR resources, flexible mechanisms, and support for innovation have moved UNDP into leading roles.
F. Implications and Recommendations for the Sixth Cycle Arrangements
In general, a close review of the experience of the Fifth Cycle SPR reveals that the arrangements adopted for the future conserve and expand some important successful elements of SPR, namely, flexibility, competition, and improved technical capability. Equally important the new arrangements remove some of the more troublesome aspects, namely: fragmentation; dual systems of programming, management, and accounting; and excessive competition. The new arrangements also redress the Fifth Cycle imbalance in roles between BPPS and the Regional Bureaux. However, three issues emerging from the evaluation of the Fifth Cycle SPR should be addressed as part of the implementation arrangements for the Sixth Cycle.
In light of these experiences, three recommendations are offered for Sixth Cycle operations:
Recommendation 1: Maintain a Role for Innovation. The operational guidelines for TRAC 2 allocations should specifically include "innovation" as one of the performance criteria. Regional bureaux should be rewarding and recognizing countries' programme creativity, in addition to their delivery. Global programming should stress the criterion of "innovation" as an absolute requirement, for at least a significant proportion of activities. The creative and experimental process has done much to make programmes such as SEED, MDP, HIV, and Gender dynamic and successful. In the important areas of poverty elimination and employment, much remains to be learned through experimentation regarding programming for significant impact.
Recommendation 2: Improve Substantive Monitoring. The Project Documents under the Global Programme (Line 1.3) should include practical monitoring mechanisms to assure BPPS's ability to track in a timely fashion the accomplishments of field activities. This information will be required for evaluating the impact of the global programmes.
Recommendation 3: Evaluations Should Focus on Impact. Evaluations of well-established global programmes planned for the third year should focus on providing quantitative evidence of positive and broad impact (and document the limits or failures to produce a broad impact). There is now an adequate body of varied programme experience to allow for and require more rigorous evaluation.
II. FIFTH CYCLE SPR PROGRAMME
A. Origins and Purpose
The origins of SPR, and of some of its important components, are found in UNDP's Fourth Cycle when a small contingency fund was established under the authority of the Administrator to cover the costs of special activities outside the normal IPF framework. The Fourth Cycle SPR was utilized to support several special programmes such as TCDC, and MDP, to carry out ex-poste evaluations (since the IPF funds for completed projects had generally already been expended) and to strengthen the headquarters evaluation function. This Fourth Cycle SPR fund was small (approximately $30 million) and its utilization largely ad hoc, at the discretion of senior UNDP management.
With the arrival of a new Administrator, planning for the Fifth Cycle included a greatly expanded SPR fund that could support a broad range of activities outside of the normal IPF allocations for recipient countries. The establishment of this much larger special fund was the subject of strong debate between the UNDP administration and the Governing Council, as well as among donor and recipient countries on the Governing Council. The resulting Fifth Cycle SPR Programme was a multi-faceted compromise between the interests of donor and recipient nations, of central administration (BPPS) and the country offices/regional bureaux, and between the UNDP administration and the Governing Council, especially the more powerful donor nations.
Donor nations, led by the United States, were most interested in stressing issues such as environmental protection, improved management, and control of drug production. Southern priorities stressed the need to support programmes that reduce southern dependency on northern expertise and technology, give southern countries a greater role in the provision of technical assistance, and provide better access to sophisticated technologies.
The resulting SPR for the Fifth Cycle ushered in a new landscape of UNDP funding options. Coinciding with the accelerated efforts to introduce National Execution and the Programme Approach in the field, the establishment of SPR at headquarters rounded out the triad of new initiatives intent upon transforming UNDP in the mid 1990s. The new SPR presented four important departures from previous practice. For the first time in UNDP history:
B. Overall Structure and Arrangements
A Diverse and Fragmented Set of Programmes. The resultant compromise SPR programme was a composite representing many different, and in some instances, divergent interests. Twenty-seven separate programmes comprise the Fifth Cycle SPR portfolio, a few quite large activities plus many tiny special initiatives. The following chart indicates the resources (including the Fourth Cycle carry-over) allocated to each SPR programme line.

Fig. 1: Allocation by Programmes
These 27 programmes address widely divergent functional areas, from improving UNDP's central evaluation function to strengthening UNDP's role in responding to natural and complex disasters. In the effort to forge a compromise among many competing interests and priorities, the Fifth Cycle SPR became splintered beyond the expectations of any of its framers. Furthermore, many of the twenty-seven programmes themselves became further fragmented as they responded to demands from the field and pressures at headquarters. For instance, one of the larger of the 27 programmes - environment - is itself divided into 26 projects, many of which operate in multiple countries.United by a Common Set of Purposes. Despite the diversity and fragmentation of SPR activities, they share several important common goals. SPR programmes are intended to be innovative and catalytic. And, although innovative at first, they are intended to mainstream their new ideas or structures into the IPF country programmes, and thus transform the way the larger structures of UNDP-funded assistance operate. And SPR programmes are supposed to build capacity at the national level.
The rational for these strictures is clear. Since the SPR programmes operate outside the IPF structure, they need to demonstrate that their activities are different from those likely to be funded through the usual programming process, and since the resources are relatively small, they need to mobilize additional financial support in order to have an adequate impact.
Perhaps most important, the SPR programmes are intended to assist with the operationalization of the broad themes of the UNDP Fifth Cycle, assuring that areas such as poverty eradication and grass-roots participation, environment, management development, women and gender issues, TCDC, and technology transfer, become more than just ideals and slogans, but are translated into active implementation programmes.
Main Components of SPR. The 27 SPR programmes are grouped into seven components as follows:
C. Pattern of Fifth Cycle SPR Allocations
Budget and Delivery. The final level of funding for all 27 SPR programme lines was $284.3 million, which was made available beginning in 1993. Of this, a total of $264.4 million had been programmed as of the end of December, 1995--leaving a small balance of about 7% as yet un-programmed SPR resources. Despite the delay in SPR allocation, "delivery" is not a problem, and most programmes are on track with their levels and rates of expenditure. To some extent, the across the board 30% reduction in Fifth Cycle funding seems to have added to the pressure on SPR resources as country offices seek ways of supplementing their reduced IPFs.
Allocation of SPR Programme Resources. Approximately one third of the total SPR is utilized for inter-regional and global programmes and projects, while two-thirds of the resources are directly channeled through regional bureau SPR projects and programmes or utilized for direct country level programmes.
Fig. 2: SPR Approvals by Year

(Source: BPPS Finance Unit SPR Ledger as at December 1995)
Allocation by Region. According to the records of the BPPS Finance Unit, the largest share of SPR resources goes to Africa, followed by Latin America, Asia and the Pacific, Eastern Europe and the CIS, and Arab States. The following table shows the allocation of the SPR resources (including approved, hard shadow and pipeline projects or programmes) directly to the regional programmes and projects (excluding the SPR inter-regional global projects, and excluding direct country level SPR projects).
Figure 3: Allocations by Regions and SPR Categories
(US$ in millions)
|
|
Africa |
Asia/Pacific |
Arab States |
Latin Amer. |
Europe |
|
Disaster Mitigation |
$6.4 million |
$8.7 million |
$3.8 million |
$4.5 million |
$2.2 million |
|
Thematic Activities |
$18.7 million |
$15.4 million |
$4.7 million |
$11.7 million |
$3.8 million |
|
Special/New Activities |
$6.8 million |
$0.05 million |
$0.2 million |
$20.4 million |
$1.0 million |
|
Aid Coordination
|
$12.2 million |
$6.0 million |
$0.7 million |
$3.0 million |
$2.5 million |
|
Programme Development |
$3.2 million |
$3.1 million |
$1.4 million |
$1.7 million |
$1.5 million |
|
Totals
|
$47.3 million |
$33.2 million |
$10.8 million |
$41.3 million |
$11.0 million |
(Source: BPPS Finance Unit SPR Ledger as at December 1995)
Allocation by Thematic Areas. Approximately 42.3% ($120.4 million) of the net available SPR ($284.3 million) for the Fifth cycle supported the thematic lines represented in the Category B. The following graph shows the relative allocation of this net available SPR to the six themes. The relative allocation of resources among the six thematic areas is the result of considerable negotiation at the onset of the Fifth Cycle, and the results were skewed toward public sector management and reform, and environmental issues. This skewing, to some extent, is the result of pressure from donor nations, and, to some extent, represents the strong interests of two UNDP administrators.
Figure 4: Allocations by Thematic Areas

None the less, with the exception of the strong SPR emphasis on governance (MDP), the distribution of SPR funds is similar to the pattern represented by all UNDP programme funds.
Execution of SPR. UNOPS executed the majority of the SPR funds, amounting to approximately $160 million, or 61%. UN specialized agencies, development banks, economic commissions and other entities executed projects with a value of approximately $59 million, or 23%, of which only DDSMS had a significantly large share (over $13 million). Not surprisingly, given the nature of SPR, National Execution accounted for the smallest share, about $45 million, or 17% of the programmed total.
Fragmentation of SPR Funding. According to the records of the BPPS finance unit, the SPR components and subcomponents are further subdivided into over 950 separate project accounts, each with its own project number (i.e. ALB/93/005/D/15/31) and budget. During a period when overall IPF programming has been moving in the direction of consolidation with the implementation of the Programme Approach, SPR moved in the opposite direction toward intense project fragmentation.
III. SPR CRITICAL ISSUES
A. Innovation
The independent evaluations of the major SPR components and programmes report an impressive pattern of innovation. Although a few programmes are criticized for their "business as usual" programming, the bulk of the work carried out with SPR funding is a clear departure from or new development of previous UNDP programming modes, in substance, focus, or style.
The emphasis on innovation in SPR programming was not intended to suggest that all innovation in UNDP comes from the center, rather than from the field, or that innovative programming does not also take place in IPF programmes. But SPR, as a "special" programme, was given a mandate to focus on new programming initiatives and on new management and delivery techniques. Some of the early conceptual papers on SPR stressed that it could be sort of an "in-house research and development" operation, helping UNDP to experiment and adapt its activities to changing times and opportunities.
There are a number of facets to innovation, including new program ideas, new methodologies for involving participants, and creative utilization of new technologies; including activities that may be innovative for UNDP but common at other institutions, and activities or processes that are new to the developing country but common to developed countries.
Learning to Use New Technology. One important area of innovation relates to UNDP efforts to assist recipient countries to put the newly emerging global communications technology in the service of addressing development problems. An example of successful efforts in this area is the Sustainable Development Networking Programme (SDNP) supported by the SPR component for Environment and Natural Resource Management (B2). To date, SDNP has assisted 16 developing countries to connect to the Internet and utilize international databases and bulletin boards to access useful environmental information in a timely manner. SDNP stresses not simply the technology, but also the establishment of national networks of governmental and non-governmental bodies and individuals concerned with environmental issues. An important innovative aspect of SDNP pertains to the way the networks cut across the line between government and non-governmental entities. Such linkages are new to many countries, and they also help provide a concrete realization of UNDP's commitment to work with the broader "civil society" beyond government.
The Internet and Environmental Clean-Up One success story for SDNP is from Pakistan, where information was provided through the Internet regarding the proper disposal of two and a half tons of meta-dinitrobenzene that had been dumped in the river near Karachi. An SOS message posted on the Internet brought in valuable information from all over the world within a week, allowing local groups to prove that the chemical was indeed dangerous and that therefore the company should not be permitted to dump it in the river. Further dumping was blocked, and the information from the Internet provided further instructions on how properly to clean up the dump site in the river. It should also be noted that we evaluators found the information on this case study, not in the evaluation report, through interviews, or in the project files at UNDP headquarters, but by surfing the World Wide Web. |
The HIV and Development programme (C5) adopted the stance that communities, each in its context, have developed their own mechanisms of response to the HIV crisis, and that this response needs to be understood, accepted, changed and mobilized from within. Since the AIDS epidemic is a relatively new and changing phenomenon, a process of adaptive, participatory, and active research is required. Among the grants awarded by the programme, was one to the Salvation Army to document innovative community responses to the epidemic. In this sense the SPR fund is being utilized, not to impose trial and error innovations from the center, but to discover and share proven successful innovations already independently initiated by communities.
In other respects, the overall thrust of the HIV and Development Programme is innovative in that as most governments and world bodies are concerned with finding a cure for AIDS and providing medical treatments for its opportunistic infections, UNDP is assisting nations and communities to draw on their cultural and social resources to map out more effective ways of helping those infected, their families, and communities, to address the crisis directly. A network of persons with HIV, while no longer an innovative activity in the developed world, is a new and forceful concept when operationalized in developing countries.
Innovative Consultative Processes. SPR programmes have in many instances developed new ways to involve a wider range of participants, beneficiaries, and stakeholders in development processes. The Management Development Programme (B3), for example, has effectively adapted and introduced a "Process Consulting Approach" into the sensitive area of civil service reform and public sector restructuring. Although the general approach is not new, its successful implementation is a new and positive experience for most of the developing country participants in MDP-supported activities. The practical introduction of transparency into a restructuring process potentially fraught with mistrust helps build the broad base of support (including at the highest levels of government) necessary to push through fundamental reforms. In this example, a "soft" participatory approach, often associated with NGOs, community development, and social services, has been sharpened and utilized at the highest levels to facilitate necessary administrative reform.
New Types of Partnerships and New Roles for UNDP. SPR programmes, partly due to scarcity of resources, and partly due to the emphasis on "catalysis", have reached out in innovative ways to work with a broad range of new partners and new roles. The most important instance of this is the role UNDP was able to play in support of the peace process in Central America (See box).
Practical Support for Peace Initiatives and Post-War Rehabilitation The Development Programme for Displaced Persons, Refugees, and Returnees (PRODERE) was a $148 million programme initiated within the context of the International Conference for Central American Refugees, through the use of SPR funds as seed money, and then implemented by collaborating agencies of the UN system. The funding was provided by a number of sources, most notably $115 million financed by Italy. PRODERE is lauded an innovative concept whereby the UN Agencies as neutral parties assist in establishing the juridical, economic, and socio-cultural conditions necessary for the success of the peace agreement. Most important, PRODERE facilitated the reintegration of displace populations into their countries of origin, and monitored the countries' compliance with this and other terms of the peace agreement. PRODERE's success has encourage the replication of similar initiatives in other post-trauma situations, especially, Cambodia and northern Somalia. |
As different types of example, the Environment SPR (B2) now jointly produces the prestigious annual report on the state of the world's environment with its originator, World Resources Institute; and the Technology Transfer and Adaptation SPR (B5) has established review panels of leading international experts in each field of endeavor, bringing to the programme an unparalleled network of business and scientific resources.
New Administrative Structures. Under SPR, BPPS has created two new divisions in order to provide more concentrated technical resources to address important issues. The establishment of the Sustainable Energy and Environment Division (SEED) of BPPS as a center responsible for all UNDP activities in the sector has proven to be an innovative and effective administrative reorganization which has provided UNDP with a strong and coherent voice in environmental issues. BPPS has also upgraded the status of its unit responsible for disaster response by creating the Division for Humanitarian Response (DHR), to coordinate UNDP's growing responsibilities in this area. SPR resources are not to be utilized to cover salaries, however, the availability of SPR programme resources combined with the administrative consolidation in some of these key areas, allows for a more focused technical approach and in-house capability.
Individual Visions. Innovative ideas begin with people, and the SPR programme, despite the many strictures placed upon its operation, have provided in several important instances a vehicle through which either a single dedicated person or a small group of like-minded professionals were able to pursue a somewhat personal and occasionally idiosyncratic development ideas. Not all large bureaucracies have a significant mechanism that allows for some dynamic and creative people to implement and pursue a professional or program goal. Some of the in-house criticism of SPR has been directed against what are perceived as personal empires pursuing what are sometimes controversial programmes. Area officers, for instance, complain that their proposals for a science and technology project were rejected because although they met the general criteria for the programme, they did not fit into the more focused and controversial programming ideas of the unit director. From the point of view of the technology unit however, the programme must focus narrowly on selected types of technology, or with its limited budget and world-wide mandate it will have no lasting impact.
The Technology Programme and the Human Development Report, for example, each provided freedom for a single dynamic and thoughtful person, along with his immediate colleagues, to pursue an essentially new programming direction, despite controversies, budget problems, in-house resistance, and administrative strictures.
In summary, "innovations" succeeded most effectively when the "innovation" was well grounded in a conceptual framework for the relevant discipline. For instance, the innovative process consulting approach of MDP is a classically successful methodology in private industry, here modified and adapted to developing world public sector administrative structures. The catalytic process of searching for "unique products" that is stressed by the Technology Transfer SPR is rooted in basic approaches to research and development. The community response focus of the HIV and Development SPR is innovative within the framework of much international AIDS work, but it is conceptually rooted in 20 years of literature and experience in community development.
Lack of Innovation in Certain Small Programmes. Not all SPR programmes have received praise for their innovations. The Poverty Programme and the NGO Programme, for instance, are both criticized for their failures to promote new types of activity or to develop new conceptual approaches in their areas of concern. These programmes continued to dole out small grants for workshops and conferences, and have not, according to the evaluators, contributed new approaches for addressing the important issue of poverty eradication, or provided new concepts for the expanded involvement of NGOs. This does not necessarily invalidate their activities, which given their small budgets, may have been the most prudent use of the resources. Similarly, the TCDC Programme has fined tuned its programme and made improvements, but it has, apparently correctly, held to the basic style of action already developed.
B. "Catalytic" Programming
A central aspect of the Fifth Cycle SPR is that it is supposed to be "catalytic". As discussed in Governing Council documents (DP/1991/64, 20 May 1991;DP/1992/7, 27 November 1991), "catalysis" relates to three areas:
Resources. Many SPR projects and activities were in one way or another co-financed with non-SPR resources, including IPF funds, funds from other donors, national resources, and in-kind and financial contributions from private sector corporations and foundations. The BPPS financial unit tracks the formal cost-sharing arrangements, but is unable to track the many less formalized instances of parallel, or more important, subsequent funding of SPR activities. According to BPPS records, the value of the formal cost-sharing arrangements with non-UNDP funds is approximately $105 million on over 100 SPR projects, or over one third the value of the total SPR. And this doesn't include the many instances of informal parallel funding.
From a resources perspective, the Fifth Cycle SPR is rarely a go-it-alone operation. The concept of "catalysis" implies a cause and effect sort of relationship rather than merely cost sharing and co-financing, and there are many instances in which the SPR contribution was perhaps not always the critical catalytic element mobilizing outside resources to support a UNDP-devised activity. The $10,000 added by SPR to the $4.2 million drug programme in Lebanon, may have been useful, but not catalytic. By contrast, a similar size contribution to the emergency response programme in Northern Iraq was catalyic, not only in generating further resources but in enabling a large programme to meet its objectives.
|
Timely SPR Inputs in a Complex Disaster Under the A2 SPR - Emergency Activities, SPR contributed to the "Coordination of UN Humanitarian Assistance Plan for Iraq, a project of UNDP/Turkey. A timely input of $12,000 resulted in the successful establishment of a Convoy Coordination Unit (CCU) in Silopi. The corridor through Turkey enabled the UN Humanitarian Programme in Iraq to assist trucks carrying humanitarian supplies to Northern Iraq. The CCU is still operational with separate UN/DHA and ODA funding. |
The evidence in the evaluations is overwhelming that SPR funds often encouraged and led other resources into the arena. Similarly, SPR funds assured that the enthusiasm and ideas from major international conferences on Environment, Education, and Social Development, received catalytic follow-up on a broad, if modestly funded, scale.
The necessary condition for "catalysis" is that UNDP and the SPR programmes be involved and conversing with other actors in their areas, rather than operating isolated programmes. Clearly, the Fifth Cycle SPR is a gregarious set of programmes, in broad dialogue with a wide range of entities concerned with human development.
|
Guyana: SPR's Catalytic Role in Generating Resources for Economic Recovery Several relatively small MDP initiatives generated a total of $78 million of resources and investments, which along with technical assistance to improve public sector management greatly contributed a stimulus to Guyana's economic recovery.
|
Catalytic Efforts for Capacity Building. The emphasis on utilizing SPR programmes to build national capacity stems in part from the notion that these "headquarters projects" must not become isolated empires. Strengthening national institutional capabilities, especially the ability to integrate the six thematic areas into government policy and planning, was stated as an immediate and ultimate goal for SPR. Although the next section will take up this theme more fully, it should be noted here that there are many examples of SPR activities that were designed specifically to function as "institutional catalysts". The most notable examples are the following:
Whether or not these efforts were successful is not the issue here, but it is clear from the evaluations of the SPR programmes that most activities were catalytic in their intents and in their functioning.
Catalytic New Programming Processes. A closely related aim of SPR is to mainstream the issues of the six thematic areas into the IPF planning cycle, and for SPR to develop and improve processes in this light. In part, this concern with mainstreaming is related to the development of National Execution and the Programme Approach. To the extent that UNDP supports truly national programmes, executed by national governments, it risks losing the automatic leverage of insisting upon its own chosen themes and issues. SPR is in part an effort to develop the capability and mechanisms within UNDP to influence the increasingly national IPF programming by providing seed money, demonstration projects, networks, and linkages to international expertise. Since UNDP wishes to emphasize certain themes to give a focus to its overall programme and increase its developmental impact, SPR provides the resources for UNDP to convince governments that these are issues that have an important place in their national programmes and policies.
Overall, the verdict is not yet in as to whether the Fifth Cycle SPR programmes have contributed significantly to a dialogue with IPF country programmes in catalytic processes. One serious constraint was the delayed funding of SPR, so that by the time most SPR components were truly functioning, IPF Fifth Cycle country programmes were already formulated and being implemented, and so it was too late for SPR programmes to contribute significantly to the upstream dialogue.
SPR funding supported the Round Table, Consultative Group, and NatCAP processes relating to aid coordination. SPR D1 was used to support projects assisting governments to prepare for the donor meetings, round tables and consultative group. The evaluation reported that these funds were used rather narrowly, assisting the government to prepare the necessary documentation for the meetings. Rarely did the SPR project develop a programme framework for building national capacity in aid coordination. Regarding the NatCAP process, the evaluation reported that the traditional system of bringing in external experts to prepare national strategy papers, identify priorities, and so forth, has continued. The evaluation concluded that in many LDCs where round tables, consultative groups, and/or NatCAPS have been operating for some time, "aid coordination is as dependent on the donor support services as it was twenty years ago."
In response to this 1995 evaluation, the SPR D1/D2 programmes changed their modus operandi to address capacity building for programme development and aid coordination, and in the process, build the type of catalytic enhancement of programming dialogue originally envisaged for SPR. The programme manager organized a retreat for both country office and headquarters staff to discuss improving capacity building for aid coordination. As a result they constituted a task management team which is in the process of producing a guide book on conceptual issues, tools, levels, and approaches to capacity building. The focus is on creating national capacity both for aid coordination and for integrating key thematic programme areas into national policy. This new initiative appears to be well received by the parties involved and appears to be moving fairly rapidly to achieve some of the SPR goals for institutionalizing catalytic dialogue into the upstream programming process.
Limitations of the Catalytic Approach. With a strong emphasis on catalysis, the SPR programme succeeds in leveraging other resources and institutions in support of its goals, but, at the same time, SPR partially relinquishes exclusive control of its and activities. Follow-up becomes problematical as other institutions with other resources are responsible to carry on the activities. An example of these limitations is often faced in the TCDC programme, for instance. TCDC organizes Capacities and Needs Matching (CNM) workshops bringing together experts from a number of developing countries who then sign accords of cooperation on specific activities. Sometimes, the promise from the successful CNM workshop is not fulfilled as the cooperating countries fail to follow through and devote the necessary resources to carry forth the accords of cooperation. TCDC's role is a catalytic one aimed at starting the process of cooperation; it cannot (and should not) fund the subsequent cooperation.
The catalytic approach also limits the degree of monitoring and evaluation of SPR activities. Much of the SPR funding consists of small grants to NGOs that neither UNDP nor the NGO has the capacity to track and monitor in terms of impact and results. The evaluations of SPR programmes are full of reasonable descriptions of what should result, and what evidence there is that the expected results may follow. But there is little systematic monitoring of many of the 950 SPR sub-projects, partially because the monitoring of such fragmented small grants would cost almost as much as the grants themselves.
C. Building National Capacities
Governing Council decision 91/3 (22 February 1991) stressed "the importance of Special Programme Resources as a catalyst in assuring that the other resources of the programme achieve the objective of building the national capacity of developing countries, especially in the six thematic areas...." Throughout the decisions and related documents setting up the Fifth Cycle SPR, the emphasis is clear that these resources are to be used to build national capacity. In part one might speculate that this repeated emphasis was required as a means of assuring that these resources which had in a sense been withheld from the IPFs, would none the less primarily serve the recipient nations. SPR was not to be seen as a redistribution of limited UNDP resources from the recipient countries to the headquarters, but rather as an additional (to the IPF) channel of assistance to developing countries.
The evaluations of the SPR components present much evidence that most SPR activities were clearly aimed at increasing national capabilities, both governmental and non-governmental. Much of the money supported workshops, public sector management development and administrative reform, networking, training, and local experimental technical assistance activities - all ostensibly aimed at strengthening the institutional capabilities of developing countries.
For the NGO Programme (C9), for instance, the activities concentrated on building and strengthening organizational capacity of NGOs and CSOs, particularly in undertaking strategic planning in areas of their concerns. The concept behind most of the SPR supported NGO activities was to enable NGOs to shift from their limited roles as deliverers of project-bound services to become viable independent organizations actively taking part in policy dialogue and national development planning activities. Through a number of regional, international training seminars, workshops, and networking activities, involving a number of NGOs and CSOs from different countries and regions the NGO programme provided training on strategic planning exercises in general and in specific sectors such as micro-enterprise development.
While these activities have the potential of strengthening capacity, it is difficult to assess whether they have actually led to the strengthened capacity of NGOs and CSOs to influence policy decisions at the national, regional, or inter-regional level. Most of the activities were one time events, and as many of these do not appear to have been prepared on the basis of a very lengthy consultative process. Also with limited planned follow-up activities, the resultant capacity-building dimension remains unclear. It is claimed in the that the networking efforts and training given to the NGOs for their participation in the international conferences on Social Development and Women enabled many southern NGOs to effectively voice their concerns and contribute to the policy. Quite likely, but the question remains as to whether the SPR NGO projects and sub-projects have resulted in many institutionally-strengthened NGOs.
There is the emphasis that SPR is to help build capacity in the six thematic areas that form the focus of the Fifth Funding Cycle. In this sense, SPR is also seen as a resource that supports the programmatic focus from UNDP headquarters in the midst of the many national agendas competing for IPF resources under the national execution modality. The unstated concern, as mentioned above, was that with the increased programming freedom of the IPF under national execution, the distinct concerns of UNDP, as expressed in the decisions of the Governing Council, might become lost. Although the programmes for the six thematic areas will be discussed later in this report, it is worth briefly highlighting the national capacity building issue for each of them here:
Management Development. The SPR component that most directly and substantially addresses the issue of capacity building is the Management Development Programme (MDP), which has, as a result of earlier evaluations, developed a comprehensive strategy for capacity building. With relatively large resources, and a several year head start on most other SPR programmes, MDP has developed an inter-linking set of activities that it applies to a select number of countries. As the first necessary component of capacity building, MDP works only in countries where there is an initial government commitment to improving public sector management. Next, MDP positions itself strategically in a key ministry and establishes close links with the highest levels in government. Simultaneously an MDP project works to establish a wider base of support through information dissemination.
An MDP activity progresses by the process consulting mode, working closely with the client institutions from the stage of needs assessment to the implementation and follow-up. Process consulting has been crucial in developing national capacity at the programme development and reform management level.
Although MDP applies considerable high-powered international expertise to its efforts, it does so through the use of the mixed team approach with an increased use of national consultants. Additionally, the MDP approach of using international consultants is one where the same experts who participated in the needs assessment exercise are those who return to work with their national colleagues on successive MDP activities. This continuity of personnel on repeated short-term visits improves communication and allows a fuller participation of national consultants.
Public sector management improvement requires a large training programme in country, to train those civil servants who are being re-deployed within and outside of government. MDP also focuses on assisting national training institutions. MDP makes full utilization of national training institutions, providing them assistance, to perform this large training task.
Finally, MDP, especially in recent years, is making increased use of the national execution modality for its projects. This is in contrast to the overall trend among SPR components to rely heavily on UNOPS and specialized agencies for project execution.
The MDP Programme embodies a comprehensive approach to national capacity building, and has the level of resources (and these are supplemented by significant cost sharing) to carry out the approach and follow-up on the progress. This situation contrasts strongly with the fragmented and lightly-funded nature of many SPR activities that clearly address capacity-building issues with the faith that their chosen intervention is contributing to a larger process.
Each of the other five thematic areas has capacity building as a key element of its programme:
In summary, all six thematic components focused their efforts on national capacity building, but only those with fairly high levels of resources and/or a very sharp technical and conceptual focus can demonstrate significant impact in this area.
D. SPR and the Transformation of UNDP
Although the emphasis of the Fifth Cycle SPR was on national capacity building for developing countries, the programme contributed importantly to the transformation of UNDP from being largely a funding organization to becoming an organization that plays a stronger and often leading role in important technical and policy arenas.
In the area of environment, it is arguable that due in large measure to the SPR resources (combined with GEF and Capacity 21), UNDP today is a major player. UNDP now has a consolidated in-house technical capability allowing it to participate and even lead international initiatives in this increasingly important and challenging area. UNDP was clearly not in such a position a decade ago, or even five years ago.
This new UNDP capability does not eclipse the important role of specialized agencies such as UNEP or FAO in environmental issues. Although UNDP has developed a capability to engage in policy dialogue and shape programmes and responses to environmental problems, it does not attempt to develop a technical capability to deliver direct technical assistance. The available soil scientists, foresters, water resource experts, range management personnel, wastewater engineers, and so forth reside with the specialized agencies (and with governments, universities, and NGOs), not with UNDP. But UNDP now has the capability to help shape the dialogue on environmental issues as the next millennium approaches. The availability of approximately $17 million in SPR funds, as well as the SPR imperative to be catalytic and innovative, and the institutional consolidation of SEED, have helped UNDP establish a leadership role and maintain the momentum of the Rio de Janeiro Conference.
A similar case can be made in the important area of public sector management development. After a long period of funding standard training and institution-building programmes for public sector line ministries, UNDP took the lead in addressing the basic problem of efficiency of public sector operations. Borrowing and adapting private sector management tools, UNDP has built up an in-house capability to address and even lead the implementation of management development programmes. At this point, UNDP is prepared to use its successes with management development as a stepping stone to address the complex issues of governance, bringing UNDP into a potentially catalytic and complementary relationship with IMF and World Bank efforts at reform, adjustment, and stabilization. Although success in the area of governance is not assured, SPR funding has created the capacity at UNDP headquarters to be a player and attempt to provide leadership in its own niche of this important development problem.
These two sectors have in common the fact that both have been strongly favored by the highest level of UNDP administration, both strongly funded and relatively well-staffed.
The Poverty Eradication and Gender components, by contrast, are modestly funded and lightly staffed in relation to the resources. The evaluations of these programmes do not indicate that there is a significant net gain in headquarters capability.
The Disaster Mitigation component, despite its large level of resources and a positive administrative up-grading to the level of division, remains lightly staffed in relation to its responsibilities. UNDP's capability in this unfortunately growing problem area has not increased as a result of SPR. The Emergency Response Division still struggles to respond to what often appears as an avalanche of ad hoc demands.
The TCDC component continues the programme established earlier, progressively adapting its approach and maintaining its commitment to improving South-South cooperation. Fifth Cycle SPR funds have enabled the TCDC programme to continue upon its path, but cannot be credited with greatly strengthening UNDP's capacity in this area, or in providing a new prospective or conceptual framework.
The Private Sector and Technology Transfer programmes have built up a headquarters technical capability, despite limited funding and staffing. Changing UNDP priorities in the direction of Sustainable Human Development seem to islolate these programmes, which now appear to have a tenuous place at the UNDP table.
One can complete the checklist of 27 SPR programmes noting those where headquarters capacity has or has not grown as a result of SPR, and where this strength may or may not persist. As a quasi-experimental, very varied programme, SPR should expect a variety of results. However, there is no question that a large portion of the resources has had the effect of stimulating an important institutional transformation at headquarters, even though these SPR programmes were clearly directed toward capacity building in recipient countries.
E. Effectiveness and Impact
Not surprisingly, most of the evaluations of the SPR programmes report on "work in progress" since the SPR funding was relatively recent at the time of evaluation. Many promising activities have begun, many workshops and training sessions have prepared people and institutions to accomplish important goals, many follow-on agreements have been signed, and so forth. If not always measurable, the accomplishments of many SPR activities are evident.
|
Administrative Reform in Venezuela An MDP Project for Development of the Management Capacity of the State had a significant impact on decentralization, the professionalization of the public service, and the simplification of administrative procedures.
Additionally, the program has generated broad and positive support from all levels of government and society. Communities, mayors, and governors are pressing for the increase of local authority and the freedom to initiate innovations in local programmes. |
In the area of the utilization of communication and information technology, the task has only begun, and the broad impact is yet to be achieved. For example, the Sustainable Development Network Project (SDNP) of the Environmental SPR can provide impressive case studies of effective utilization of the new international communications technology, and few would doubt that wider utilization of the Internet has an important place for environmental management in developing countries. But to date the number of developing countries connected remains limited. In a related situation, TCDC's efforts to improve a database on "southern" expertise, shows promise, but the improvements are not yet complete, and the utilization of the previous database has been minimal. The potential for full utilization of information and communications technology is still unfulfilled, and the development impact of these activities is yet to be felt across the board.
But it is almost impossible at this time to demonstrate the overall concrete impact and effectiveness of many of the 950 projects and sub-projects. Most of the activities are too recent to allow assessment of real impact. By their catalytic nature, many SPR activities are relatively small contributions to a larger set of multi-funded activities, making the assessment of the impact of SPR difficult to separate from the impact of other inputs. Overall, the SPR programme, with its intense fragmentation into almost a thousand pieces does not lend itself, either in the aggregate or in its discrete activities, to a controlled measured assessment.
However, the independent evaluations of the major SPR programmes give ample evidence that many SPR programmes are well conceived and ably implemented. Such assessments especially characterize the evaluations of the Environment, MDP, HIV/AIDS, TCDC, and Technology Transfer components. The following example is one of many instances where the impact of the SPR project is significant.
IV. FIFTH CYCLE MANAGEMENT ISSUES
A. Introduction
The verdict has already been declared that the management structure and processes embodied by the Fifth Cycle SPR were unwieldy and in many respects problematical. As a result of this perception, the Sixth Cycle arrangements disband SPR as a distinct programme. Overall, there are three aspects of the structure and operation of SPR that contributed to discontent with its operation:
B. Programme Management
SPR, being new, lacked an established and practiced system of management. BPPS established a position of SPR Coordinator and assigned a manager for each of the 27 SPR components. In a number of instances the managerial structure already existed. But for many other SPR activities, a new programme manager was selected who then had the responsibility to formulate the programme's operating principles. Programmes were well-managed or not, largely as a function of the individual programme manager, in the case of the smaller components, or as a function of the adequacy of staffing and/or previous establishment for the larger programmes. MDP and TCDC, for instance, were existing programmes, and had already established management procedures and styles of operation. The Disaster Mitigation programme, although not new, was clearly short staffed given that it is the largest SPR component, operating in the face of unpredictable demands.
At first, the Fifth Cycle SPR represented a significant shift of authority and resources to BPPS. This shift, combined with increasing programming authority in the field left the role of the regional bureaux uncertain and reduced. Gradually in the course of SPR implementation, many SPR programmes developed an effective partnership with the regional bureaux. None-the-less, there continues to be a somewhat bewildering variety of relationships, as some SPR components respond directly to requests from country offices, others divide the SPR resources like entitlements to the regional bureaux, and other SPR components emphasize a supply driven approach to spreading their "gospel" to cooperating countries.
C. Project Selection Process
Many of the SPR programmes operated as facilities that country offices could access for additional funding for activities within the mandate of the particular SPR programme. As stated earlier, the concept of country offices competing for central funds by submitting proposals was a new one within the system. Many country offices, keen to augment their reduced IPFs, looked to the SPRs as an important source of additional resources, only to be very disappointed. In attempting a access this source they confronted a bewildering variety of requirements, schedules, and criteria from the 27 separate SPR programmes. The variety was all the more bewildering since many of the SPR programmes, themselves new, were in the beginning uncertain and unclear as to exactly what their procedures would be.
Country office staff, unused to having to prepare proposals to Headquarters for special funds, found the process to be time consuming considering the small size ($50,000 in most instances) of the grants. Even more distressing was the experience of making the effort and being rejected, often for reasons that were not clear to the field staff.
Requests were turned down for three reasons:
Limited Resources. The resource constraints in most SPR areas almost guaranteed that the number of disappointed country offices would outnumber those who succeeded in obtaining a small SPR project. In some instances, perfectly good proposals were turned down simply because the SPR programme lacked adequate resources to respond to all of the demands from all of the countries. Even with the small size of the SPR grants, a small programme such as the HIV programme or the gender programme could sponsor activities in only a minority of the countries. Even a large programme such as Environment could not possibly provide resources for a significant environmental programme in all of the recipient countries of UNDP.
Failure to Meet General SPR Criteria. Often, the request was turned down because it did not meet the minimal SPR criteria that the activity be "catalytic" and "innovative". Country offices looking simply for a bit more funding of their normal activities were often rebuffed, and resented that someone at headquarters should be determining what was best for their country. But from the perspective of the SPR programme managers, if SPR were simply to fund "business as usual" it would defeat the very purpose of its establishment as a separate activity.
Failure to Meet Specific Programme Criteria or Foci. Finally, some SPR programmes were more "supply-driven" than others. The Technology Adaptation and Transfer Programme, for instances, in an effort to concentrate its limited resources, focused its activities narrowly on a process of finding and developing "unique products", and on developing some "flagship" technologies, focusing on solutions to environmental problems. More advanced countries, such as those of the former Soviet Union, with their very limited IPFs, saw the area of technology transfer and transformation to be a natural field of action. In many instances, their desired technological areas did not match the SPR unit's somewhat arbitrarily chosen fields of emphasis. There was resentment that perfectly strong proposals in the area of technology transfer were rejected because they did not constitute the discovery of a "unique product".
Other SPR programmes carefully developed a focus of activity or a set of criteria for selection that was not always easily understood or acceptable to supplicants from the country offices who had not participated in the programme development process. MDP, for instance, was not simply funding any effort at "capacity building for the public sector", but had determined to focus on management development in situations where there is strong government commitment to significant reform, and fertile ground for its chosen "process-oriented" approach (rather than simply a training approach) to succeed.
Many SPR programmes gradually developed less cumbersome and more transparent systems of allocating their resources. Some programmes, such as Sustainable Human Development, simply divided its funds equally among the five regions, with the region simply dividing the resources equally among all its countries for the stipulated activities. MDP developed an effective process of working with each bureau to determine which countries would be eligible and most likely to gain the maximum benefit from an MDP activity. These two approaches avoided the situation where country offices of a region were spending time writing proposals in competition with one another for resources that were very limited, or for a programme whose aims were not clear.
D. Reporting, Monitoring, and Evaluation
Programme Monitoring. Each of the programmes submits an annual progress report to the SPR Coordinator at BPPS, who then prepares an omnibus SPR annual progress report. However, reporting from many of the small field projects and grants is spotty and uneven, and SPR programme managers themselves report that they have a difficult time assessing what is happening in the fragmented activities for which they are responsible. Since so many of the funded activities are small, one time events, it is likewise difficult to monitor the outcomes of these small catalytic efforts, especially in a cost effective manner.
|
Partners in Development Programme II (PDP II) PDP adopted a minimalist approach to reporting requirements in recognition of the small size of most of its grants. Even so, project reports are submitted sporadically and with varying qualities of information presented. In general, there is a lack of benchmark data for evaluation purposes on these many small activities. In some cases, the local PDP has provided potentially useful information for which there is no framework for its utilization. For instance, the village committee of a forestry project in Malawi meticulously maintains records on the numbers of seedlings, trees planted, sales, and so forth which could be useful for a cost impact analysis, but no one is prepared to carry out such an analysis. Several factors account for the weak M & E, in addition to the small size of the grants. There is no clear delineation of M&E responsibilities; PDP II has no national or regional coordinators other than the overburdened officers at the country offices. In some instances the national PDP selection committee delegated the M&E function to local NGOs, themselves lacking the resources or know-how. The M&E constraints inherent in a small grants programme are compounded by a lack of a strategy to address them. As a consequence, the absence of relevant information greatly hampers the possibility of learning from the PDP experience as UNDP continues its search for effective means of eradicating poverty. |
SPR Evaluation Framework. As part of the original negotiations setting up the Fifth Cycle SPR, a special evaluation plan was developed and adopted. According to the plan, programmes with budgets in excess of $10 million are subject to independent mid-term evaluations; and smaller programmes, especially new programmes and new components of existing programmes not previously evaluated, are required to perform an internal evaluation.
SPR EVALUATIONS
|
Programme |
Evaluation Type |
|
A. Disaster Mitigation |
|
|
A1 Disaster Preparedness Management |
Independent |
|
A2 Emergency Relief |
Independent |
|
A3 Reconstruction and Rehabilitation |
Independent |
| A4 Refugees,Displaced Persons, and Returnees |
Independent |
|
B. Thematic Activities |
|
| B1 Poverty Eradication & Grassroots Participation |
Independent |
| B2 Environmental Problems & Nat. Resources |
Independent |
|
B3 Management Development |
Independent |
|
B4 TCDC |
Independent |
|
B5 Transfer and Adapt. of Technology |
Independent |
|
B6 Gender in Development |
Independent |
|
C. Other Special and/or New Activities |
|
|
C1 Human Development Report |
Internal |
| C2 Special Plan of Economic Ass. to Central America |
Independent |
| C3 African Economic. Recovery & Development |
Independent |
|
C4 Drug Abuse Control |
Internal |
|
C5 HIV/AIDS |
Independent |
|
C6 Social Dimensions of Adjustment |
No Report |
|
C7 Education for All |
Internal |
|
C8 Private Sector |
Independent |
|
C9 NGOs |
Internal |
|
D. Aid Co-ordination |
|
|
D1 Round Tables & Consultative Groups |
Independent |
|
D2 NATCAPs |
Independent |
|
D3 Needs Assessment., etc |
Internal |
|
D4 Country Programming Initiatives |
Internal |
|
E. Programme Development |
|
|
E1 Programme. Development. & SHD |
Internal |
|
E2 Evaluation |
Independent |
|
E3 Research |
Internal |
|
F. Palestinian People |
No Report |
The Guidelines for SPR Evaluations provided a general evaluation framework, stressing the importance of evaluating the unique SPR requirements of "innovation", "catalysis", and so forth, in addition to examining the usual parameters of performance. It is the responsibility of the SPR Programme manager to develop, with assistance from OESP for large programmes, specific terms of reference for the external evaluation his or her programme.
Evaluating the SPR programmes presented some difficulties in many instances. Although each SPR programme has defined objectives and responsibilities, SPR programmes had not been as rigorously formulated as most IPF programmes with their detailed specifications of inputs, activities, outputs, and immediate objectives. The evaluation of the large SPR programmes is not an easy task, since most of these programmes support large numbers of very small activities in numerous countries.
Assessment of the Quality of SPR Evaluations. Many of external evaluations, designed with the assistance of OESP, were creative in addressing the problems inherent in evaluating such programmes and forward looking and constructive in their criticisms and recommendations. The HIV evaluation, for instance, adopted a highly participatory approach that matched the tone and structure of the programme itself. The evaluation thus became a strong feed-back mechanism for the innovative programme. The evaluation of the Environment programme developed an effective rating system for independently evaluating the extent to which each activity met the special SPR criteria.
Overall, the evaluations are friendly evaluation, focusing on process, and providing anecdotal evidence of impact, since the activities being evaluated are relatively recent. Most evaluations focused more effectively on documenting compliance with SPR criteria such as innovation and catalysis then on the impact of the programme activities. The one relatively harsh evaluation pertained to the thematic area of poverty alleviation, where the evaluators stressed that, despite the importance of the theme, UNDP has not yet developed a successful, comprehensive, and operational strategy in the area.
Utilization of Evaluation Results. The fact that many of the evaluations were somewhat co-opted by the programme had the positive effect that what criticisms and recommended changes were presented were more likely to be incorporated productively into programme operation. SPR programmes have shown a strong tendency to make fairly rapid mid-course corrections in response to these evaluations, to the clear benefit of the programmes. The MDP programme, for instance, greatly sharpened its focus on "process consulting" as a result of its 1992 evaluation, and produced useful and important guidelines and manuals as a result of its 1994 evaluation. As mentioned earlier, the NatCap/Round Table programme manager organized a retreat after the 1995 evaluation, where the concerned parties redesigned the activity as a more dynamic approach to capacity building. SPR programmes, it seems, have the flexibility and independence to quickly make mid-course corrections in response to constructive analytical criticism.
E. Financial Management and Monitoring
At the beginning of the operation of the Fifth Cycle SPR, BPPS had no mechanism for financially tracking this almost $300 million operational programme. The task of tracking and accounting for the funds in over 950 separate SPR projects was made somewhat easier by the fact that 62% of the funds are executed by UNOPS. None the less, BPPS quickly decided to establish a special financial unit for the SPR programmes to enable BPPS to report accurately to the Executive Board. This unit, beginning operation well after SPR programming was already in progress, struggled to sort out the finances of the multitude of small projects and grants that make up SPR. With little in place as a standard financial reporting system, several of the SPR programmes' financial records of the first year were a shambles. A system was established whereby each of the 950 projects prepares an annual Project Delivery Report (PDR). These PDR detail expenditures according to category of inputs (personnel, training, and so forth), but this information on inputs is not aggregated at a higher level. The operational decentralization of many SPR programmes to the bureaux and country offices, however, often leaves the BPPS programme manager with financial responsibility for funds over which he or she has little or no control, and often, little information on their expenditure.
At present the funds appear to be adequately accounted for and tracked by this independent parallel system. The system is able to track the expenditures for all programmes, sub-programmes, and projects, and to track formal co-financing expenditures. As the Fifth Cycle SPR winds to a halt, the SPR finance unit will be able to fully account for all funds before it shuts down its operations to make way for the Sixth Cycle successor arrangements.
F. SPR is a Transitional Structure
The SPR programme is a transitional management and funding structure within UNDP. From a small contingency fund in the Fourth Cycle, it became a relatively large and complex set of programmes lodged largely within BPPS. With these resources, BPPS's role within UNDP expanded exponentially in the course of just a few years, and the bureau, as we have seen, had to create its own system of financial monitoring and control in order to handle and account for the almost $300 million.
For the Sixth Cycle it has been decided that the activities currently funded under SPR will be folded into the new successor arrangements into various appropriate budget lines. The transitional nature of SPR is perhaps, after all, its most salient characteristic. SPR began as a largely new programme, finding its way and creating its systems and organizational cultures while simultaneously carrying out a challenging range of substantive programmes. No sooner has SPR established itself in the workings of the organization, that it is to be dismantled and rearranged within a new structure of resource allocation.
V. IMPLICATIONS AND RECOMMENDTIONS FOR SIXTH CYCLE ARRANGEMENTS
A. Successor Arrangements Respond to SPR Problems
The complex successor arrangements for the Sixth Programming Cycle are an appropriate development from the combined Fifth Cycle emphases of SPR, the Programme Approach, and National Execution.
|
Sixth Cycle Programming Country Programmes: The Sixth Cycle allocates UNDP programme resources to countries through a three-tier TRAC (Target Resource Assignment from the Core) scheme.
Global Programme: Budget line 1.3 is designed to provide focus for UNDP global activities in five broad areas of concern:
|
A close review of the experience of the Fifth Cycle SPR reveals that the arrangements adopted for the future conserve and expand some important and successful elements of SPR, especially:
At the same time, the new arrangements remove some of the more troublesome and controversial aspects of the Fifth Cycle SPR, namely:
The new arrangements also redress the Fifth Cycle imbalance in roles between BPPS and the Regional Bureaux. The following points are noteworthy:
Flexibility and Competition. The Successor Arrangements maintain and expand the flexibility of UNDP programming through the TRAC 1 and TRAC 2 modes, building on the SPR experience of reserving a portion of UNDP programming resources for which there will be a measure of competition among countries. Additionally, the Successor Arrangements regularize the competition to some extent by relating it more closely to the IPF entitlements, and by reducing the amount of needless competition for scarce resources that often characterized many Fifth Cycle SPR programmes.
Strengthened Role for Regional Bureaux. The Sixth Cycle role of the regional bureaux in determining and approving TRAC 2 allocations, builds upon the successful pattern of resource allocation developed by MDP and other Fifth Cycle SPR programmes. BPPS returns to its more traditional role of policy and programme support, but with enhanced technical capabilities in several key areas.
An End to SPR Dualism. Programmes of a universal nature (SHD, Environment, Management Development & Governance, and so forth) are folded into the Global and Inter-Regional budget line, thus ending one form of SPR dualism. All programmes will report through the same financial accounting unit, and follow the same rules for programme monitoring and evaluation.
A Coherent Global Programme at BPPS. BPPS will oversee a comprehensive and consolidated "global" programme, divided into five broad categories:
The Director of BPPS will manage Category A. The Director of the Division for Social Development and Poverty Elimination (SEPED) will manage categories B and D. Category C will be the responsibility of the Director of the Sustainable Energy and Environment Division (SEED); and Category E will be the responsibility of the Director of the Management Development and governance Division. Each division director will provide an annual progress report to the Director of BPPS. Programmes in each category will be fully formulated and reviewed by the Bureau PAC, and will include measurable indicators of accomplishment in the programme designs. Evaluations, scheduled for the third year, will be based on the quantifiable indicators of impact established in the Programme Documents.
B. Recommendations for Sixth Cycle Operations
Recommendation 1: Maintain a Role for Innovation
TRAC 2. The operational guidelines for TRAC 2 allocations should specifically include "innovation" as one of the performance criteria. Regional bureaux should be rewarding and recognizing countries' programme creativity, in addition to their delivery.
Global Programme. Line 1.3 programming should stress the criterion of "innovation" as an absolute requirement, at least for a significant proportion of activities. The creative and experimental process has done much to make programmes such as SEED, MDP, HIV, and Gender dynamic and successful. In the important areas of poverty elimination and employment, much remains to be learned through experimentation regarding programming for significant impact.
Discussion. The most important strength of the Fifth Cycle SPR, its emphasis on innovation, risks being lost or diluted under the Sixth Cycle Successor Arrangements. Fifth Cycle SPR activities were required to be innovative and different from the usual IPF programming that was developing with National Execution and the Programme Approach. This requirement for innovation assured a vitality to new UNDP programming and played an important role in helping UNDP adapt to new opportunities and changing situations. It also caused a degree of consternation in the selection process, as proposals from the field were rejected at headquarters because they proposed all-too-familiar activities.
The division of the "IPF" into TRAC 1 and TRAC 2 provides programme funding flexibility and a theoretical incentive to innovation, but does not necessarily guarantee that a share of the funds will be utilized for truly new or experimental efforts. Indeed the language of the documents establishing the new system stress "conformity" with SHD principles, rather than "innovation" as criteria. Additionally, the system of allocation which will involve both the regional bureaux and national authorities to a larger extent than SPR, is probably not conducive to innovative programming.
Similarly, the Global Programme (Line 1.3), in its efforts to provide strategic focus, may not encourage the types of experimentation in programme delivery that characterized important Fifth Cycle successes. Partially the changed emphasis reflects the situation of reduced resources wherein innovation is a luxury. There is an understandable pressure to focus resources on activities with proven successful track records. This pressure needs to be balanced with a continued emphasis on an appropriate role for less certain experimentation.
Recommendation 2: Improve Substantive Monitoring
The Project Documents under the Global Programme (Line 1.3) should include practical monitoring mechanisms to assure BPPS's ability to track in a timely fashion the accomplishments of field activities. This information will be required for evaluating the impact of the global programmes.
Discussion. Although there was in place for the Fifth Cycle SPR a system of annual reporting, it was often difficult for the SPR programme managers to report in an informed manner on the multiple small field activities under their responsibility. Reporting from the field to BPPS was often uneven and spotty. Grants to the field under the Sixth Cycle global programmes must include resources to support the transmission of basic information on the performance of activities and delivery of outputs. Furthermore, a mechanism to assure timely reporting (withholding a portion of the grant payment) and a later follow-up report needs to be established, especially for small activities.
Recommendation 3: Evaluations Should Focus on Impact
Evaluations of well-established global programmes planned for the third year should focus on providing quantitative evidence of positive and broad impact (and document the limits or failures to produce a broad impact).
Discussion. The evaluations carried out under the Fifth Cycle SPR focused on process issues and on the degree to which the programmes conformed to SPR requirements regarding innovation and catalysis. Since many of the SPR programmes got underway late, the evaluations could not address the issue of impact. However, most of the global programmes and subprogrammes in Line 1.3 have now been operating long enough so that evaluations can and should focus on the difficult matter of impact. The programmes and subprogrammes of SEED and MDGD, for instance, have expended significant levels of resources over a number of years in a wide range of countries. It is time to provide more rigorous assessments. Other programmes under Line 1.3, still in exploratory stages, however, will benefit from the type of process-oriented evaluations that characterized the Fifth Cycle SPR.
SUMMARY PROGRAMME EVALUATIONS
THEMATIC AREA SPR PROGRAMMES
A. Poverty Eradication and Grassroots Participation (B1)
Programme Resources Management. The magnitude of SPR funds allocated to poverty eradication and grassroots participation is relatively insignificant. As of the end of 1995, Fifth Cycle entitlement for this area totaled $14 million. It accounts only for 11.6% of the total SPR for the six thematic areas. Of this amount, $11.2 million has been approved for a portfolio of 20 inter-regional, 2 regional, and 10 country projects, or a total of 32 projects. Resources are centrally managed by BPPS with the exception of the biggest component that was decentralized to the country offices.
Programme Content and Issues. The Partners in Development Programme - Phase II (PDP II) is the biggest component under B1, with a $7.5 million budget, representing about two-thirds of the total SPR for poverty eradication and grassroots participation. The experience with PDP II will have a significant effect on the overall effectiveness or impact of SPR support for poverty eradication and grassroots participation.
As a brief background, PDP II was approved in July 1992 to provide small grants to NGOs, particularly community-based organizations (CBOs). It has three-fold objectives, namely, income-generation, capacity-building, and NGO networking. Supplemented by UNCDF funds of $0.5 million for the 1992-1993 period to support income-generating activities, PDP II covers 73 countries, with allocations averaging $60,000 per country.
The mid-term evaluation of PDP II in June 1994, while pointing out the programme's potential, observed significant weaknesses in the programme's small grants framework. PDP II's outstanding feature is its ability to deliver direct support to NGOs and poor communities, especially those with restrictive, rather than enabling, environments. It has the potential to reach vulnerable groups of society, mainly through the authority of the National Selection Committee (NSC) in each participating country to establish priorities and grants criteria within the context of local demands. However, the microprojects have been sporadic and dispersed, of short duration, and with no common sectoral or geographic focus. These factors have tended to negate the programme's potential to be innovative and catalytic, and to be an effective instrument for capacity-building. For instance, they do not provide a suitable environment for developing and testing new approaches to address the poverty problem and to promote grassroots participation. PDP II's "minimalist" approach to monitoring and evaluation does not also encourage efforts to determine useful lessons from experience. With very few exceptions, SPR poverty projects have not generated additional support from other donors at a significant level. The concentration of projects on income generation and social welfare (almost 80%) reflects a neglect of the programme's capacity-building and networking objectives. Practical training and operational networking among NGOs and CBOs in planning and project implementation are generally lacking, thus, making sustainability even a greater concern.
The remaining one-third of SPR support for poverty eradication has been allocated for: 1) a project similar to PDP II, the Local Initiative Facility for Urban Environment (LIFE), piloted in eight countries to promote dialogues among municipalities, NGOs and CBOs to improve the urban environment in low -income settlements; 2) the formulation of poverty alleviation policies, programmes, and strategies; and 3) the documentation and sharing of experiences through research, workshops, and conferences.
Under type 2, support was provided for a meeting of local NGOs to generate inputs to a joint UNDP/IDB strategy on survival and organizational strategies for the poor in Latin America. SPR funding in this particular case proved catalytic in terms of broadening the scope of partnership and bringing the civil society into policy discussion. Assistance was likewise extended to: 1) Rwanda and Gambia, in formulating their national poverty alleviation programmes; 2) Zimbabwe, in devising an implementation strategy for its Social Dimensions of Adjustment Programme; 3) China, in developing its poverty alleviation policy and strategy; and 4) a Joint Poverty Alleviation Programme of Government and JCGP Agencies in the Philippines that aims to improve coordination between the government and JCGP agencies and to introduce interventions at the macro and micro levels. The potential benefit from these initiatives is the establishment of the policy or institutional framework for developing specific measures to combat poverty in the countries concerned.
Type 3 projects include case studies of the impact of environmental innovations on the urban poor in nine mega-cities, success stories on community-based partnership efforts in Africa, follow-up research on the poverty situation in the Republic of Korea, study of the social impact of the structural adjustment policy on vulnerable groups in Mexico, and participatory monitoring and evaluation of the effects on the rural poor of land redistribution in Sao Tome. In addition to these studies, workshops and conferences have been funded by SPR to deliberate on issues such as grassroots participation, social mobilization and organization of the poor, poverty and social development, and emerging role of NGOs in African development. The 1994 mid-term evaluation of UNDP's Fifth Cycle Poverty Alleviation Programme noted this tendency for the exchange of international experience among researchers rather than for activities that are directly tied in to national programmes. It added that SPR poverty-related projects primarily reflect the headquarters' concerns and links with research institutions and conference organizers. This seems to suggest that such projects are centrally-driven and not identified from the field, and may not therefore necessarily reflect the genuine needs of target constituencies.
Overall Assessment. Due to the lack of a clear, unified approach and monitoring mechanisms especially for the biggest programme component, it is difficult to establish whether SPR-supported interventions relating to poverty eradication and grassroots participation have produced a conclusive or definitive impact.
There are, however, some indications showing gaps in promise and performance. The level of SPR funds allocated for this programme does not reflect the high priority attributed to it. Most projects, the greater bulk of which are under PDP II, are very small and targeted at very specific needs. Given their micro nature, these projects are not expected to produce a significant impact outside their immediate setting. Despite their potential to be catalytic, innovative, and effective particularly in terms of building the capacity of NGOs and CBOs as development partners, PDP II projects seem to have fallen below expectations. Participatory approaches initiated under PDP II as well as LIFE are far from being integrated into the mainstream of UNDP's programming but might offer valuable insight for future activities relating to the focus on governance.
The other projects are more of input interventions by nature, and in most cases, need follow-up activities. For instance, the production of case studies must be complemented by systematic dissemination of their results to relevant users in government and the NGO sector. Workshops and conferences must be followed by implementation and monitoring of agreed action plans. These activities can only be meaningful within the context of the overall process of establishing the policy and institutional framework to fight poverty. Beyond this, it is difficult to establish a direct causal relationship between studies and workshops, on the one hand, and poverty eradication, on the other.
Finally, some project outputs are fairly recent and have not been tested yet. A good example of this is UNDP's policy paper, Poverty Eradication: A Policy Framework for Country Strategies, published in 1995. The extent to which this document will be used in developing poverty strategies at the national and local levels remains to be seen.
B. Environmental and Natural Resources Management (B2)
Programme Resources Management. The total Fifth Cycle entitlement for this programme is $17.6 million as of the end of December 1995. It represents 14.6% of the total SPR for the six thematic areas. Of the said amount, $16.4 million has been approved for 27 projects.
In most instances, projects are demand-driven with demands coming directly from the country offices rather than the regional bureaus at headquarters. In consultation with the regional bureaus, country offices, and UN specialized agencies, the Sustainable Energy and Environment Division (SEED) of BPPS manages all the projects.
Programme Content and Issues. The 27 projects are grouped in four sub-programmes as follows:
| Country Planning and Policy Support | 12 projects | $3.4 million
| Training in Environment | 4 projects | $4.9 million
| Support to Global/Regional Activities& | 10 projects | $6.2 million
| Improving UNDP Quality and Effectiveness | 1 project | $1.9 million | |
National capacity-building is the primary and direct objective of many of the most important projects under B2. Among others, the Sustainable Development Network Programme aims to develop in each participating country a consultative process that will facilitate the access of a broad range of public and private institutions to national and international information pertinent to sustainable development. The Environmental Management Training Project is also judged to exhibit strong national capacity-building components. It provides training in a procedure called "Environmental Overview" which is a type of environmental impact assessment. The procedure is specifically designed for use in the up-stream formulation process for a wide range of projects other than construction projects. Almost 3,000 participants from over 111 countries have attended the training and there is great potential for generating a significant multiplier effect through this large group. Capacity 21, initially an independent programme, but now a unit within SEED, also takes national capacity-building as its principal mandate.
Catalytic aspects have been observed in programme activities in varying degrees. The mid-term evaluation of environmental and natural resources management, as a component of SPR, found six projects that clearly have as their primary concern the provision of seed money to enable the initiation of larger activities with additional funding sources. These projects include the following: