PART ONE
CHAPTER 16. STAKEHOLDERS

Who are the Monitoring and Evaluation Stakeholders?
All of the groups that have a role and an interest in the objectives and implementation of development activities are the stakeholders in the monitoring and evaluation process. The key stakeholders are:

• target groups or those sectors of the population targeted to benefit ultimately from the results of programmes and projects;

• direct beneficiaries, usually institutions and/or individuals who are the direct recipients of technical cooperation aimed at strengthening their capacity to undertake development tasks that are directed at specific target groups;

• those who are responsible for ensuring that the results are produced as planned: programme managers and staff of the Government (under the national execution modality) or of the United Nations executing agencies;

• those who are accountable for the resources that they provide to the programmes and projects: national policy-makers and budget authorities, UNDP, donors and otherdevelopment partners.

Several associated parties could also be added to the list:

• external consultants, suppliers and other persons or organizations providing inputs to the programmes or projects; and

• other institutions (private-sector entities, CSOs) in the programme or project environment that may also be affected by or interested in the results of the programmes or projects.

Why Should Stakeholders Be Involved in Monitoring and Evaluation?
The involvement of stakeholders in appropriate monitoring actions and evaluation exercises furthers the objective of promoting participatory development. Stakeholders have the right and the responsibility to know what is happening in the programme or project, which aspects need corrective action, what the results are, and which lessons can be learned and shared with one another, but they should not simply be recipients of monitoring and evaluation reports.

One effective way for stakeholders to contribute to the achievement of programme or project objectives is to be directly involved in the monitoring and evaluation process - in the formulation of critical questions and in the collection and analysis of data. This enables them to participate directly in the assessment of the relevance, performance and success of the programme or project and in recommending how to improve the quality of current and future interventions.

How Do You Involve Stakeholders in Monitoring and Evaluation?
The first step is to identify the key stakeholders who in some significant way are affected by, or involved in, the programme or project during its lifetime and beyond.

The second is to provide for mechanisms that will allow stakeholders to interact with each other in a meaningful way not only in monitoring and evaluation but earlier, starting at the pre-formulation stage and continuing during formulation and implementation. For instance, the active participation of stakeholders in the selection of indicators is emphasized (see chapter 12). Field visits and stakeholder meetings are monitoring actions that are aimed at actively involving stakeholders (see chapter nine). Participatory evaluation, as a methodology, allows stakeholders to be the question-makers and not simply the objects or targets of evaluations.

Participatory Evaluation
Participatory evaluation is an evaluation approach or methodology that has evolved from such earlier approaches and schools of thought as participatory action research, rapid rural appraisal (RRA) and participatory learning and action (PLA), farming systems research, self-evaluation and beneficiary assessment. Its main characteristics are presented in box 9.

BOX 9. PARTICIPATORY EVALUATION

Purpose/Function
To help to build the capacity of stakeholders to reflect, analyse and act

To contribute to the development and feedback of lessons learned that can lead to corrective actions

To help to ensure accountability to stakeholders

Selection Criteria/Timing
Projects or programmes that have a clearly identified group of end-users and beneficiaries

Timing: usually mid-term but also towards or at the end; ex-post, too, but since not all stakeholders may be involved after project com-pletion, the level of participation may vary con-siderably

Focus
Relevance, performance and success, specific elements of which depend on the timing and scope of the evaluation

The process itself, to involve stakeholders actively and directly in the evaluation

Agents/Participants
Project or programme stakeholders

Participatory evaluation facilitator acting as a catalyst or stimulator, managing but not directing the evaluation

Outputs
Depend on the scope and timing of the evaluation

Generally, these should include stakeholder views as reflected in the analysis of issues and in recom-mendations to resolve them

Source: Adapted from UNDP, OESP, Who Are the Question-makers? A Participatory Evaluation Handbook, 1997.

PART ONE
CHAPTER 17. PLANNING FOR MONITORING AND EVALUATION IN THE CONTEXT OF THE COUNTRY COOPERATION FRAMEWORK (CCF)

The concepts presented in the preceding chapters have little value as isolated elements. Rather, their usefulness depends on how well they are integrated into a specific context. That context will dictate the monitoring and evaluation exercises that are needed to satisfy the requirements of decision-making, accountability, learning and capacity development. At the country level, the context is the CCF prepared by the Government of the programme country in collaboration with UNDP.

The CCF defines the areas of cooperation between UNDP and the Government of the programme country. It is essentially a matching of the UNDP mission and comparative advantage in specific areas of technical cooperation on the one hand with the development goals of the country on the other. As such, the CCF must provide the framework for planning the monitoring and evaluation activities at the country level.

The CCF is the culmination of dialogues between the Government and UNDP, beginning with the preparation of an advisory note by UNDP. (In some cases, the CCF also evolves from the country strategy note that covers areas of cooperation between the Government and the United Nations system in general.) The advisory note presents views on the best way to channel UNDP assistance to support the programme country's national development goals, consistent with the UNDP mission and comparative advantage.

To help to ensure effective management of the CCF, the prerequisites of effective monitoring and evaluation must be considered as early as the preparation of the advisory note. For instance, the advisory note should already incorporate the following: baseline data in presenting the development situation of the country, tentative indicators for assessing the progress and results of UNDP support in the proposed areas of thematic focus, the monitoring and evaluation approach to be adopted and preliminary plans on the monitoring actions and evaluation exercises that may be required. At this stage, the foundation for effective monitoring and evaluation of UNDP's cooperation with the programme country must be established.

That foundation must be strengthened during the formulation of the CCF. The baseline data should be refined or developed in greater detail in order to present a clearer picture of the develop-ment problems that UNDP cooperation will help to address. The UNDP country office should assist the Government in arriving at a consensus among the main stakeholders on the indicators to be used for assessing the overall progress and results of interventions supported under the CCF.

Monitoring and evaluation plans must be pre-pared as an integral part of the CCF. These plans should be detailed enough to contain essential information such as the types of monitoring actions and evaluation exercises that are needed to have an adequate basis for assessing the overall progress and achievements of the CCF. They must be reviewed once a year in light of the requirements and/or results of the annual review of the CCF.

See chapter 22 for steps in preparing a country-level evaluation plan.

PART ONE
CHAPTER 18. FUNDING

Why is Funding for Monitoring and Evaluation Critical?
Monitoring and evaluation are essential to the effective management of development inter-ventions. It is therefore crucial that adequate funds be provided to carry out these functions.

Previous experience has shown that monitoring and evaluation actions have been limited by budgetary constraints: either costs are underestimated or funds originally allocated to monitoring and evaluation are transferred to other activities. In the face of a financial shortfall, and perhaps more than at any other time, those responsible for monitoring and evaluation must ensure that those functions are not neglected. Both can generate critical information that can serve as a basis for making decisions on adjust-ments in the magnitude, scope and focus of programmes or projects that may be necessitated by an unfavourable financial situation.

Within UNDP, the increased delegation of authority to country offices for programme or project approval necessarily demands a higher level of accountability and responsibility on the part of the country offices to ensure the quality of UNDP-assisted interventions. Monitoring and evaluation are management functions that can support the country offices in responding to these demands.

The shift from execution by United Nations specialized agencies to national execution likewise has direct implications for the responsibilities of country offices, especially with respect to monitoring. Country offices should assist governments in programme and project monitoring, a responsibility that used to be carried out primarily by United Nations specialized agencies. There must be adequate funds to allow UNDP country offices to undertake field visits and hold stakeholder meetings involving representatives of the Government at various levels, target groups and relevant development partners.In particular, funds are needed for a wide range of monitoring- and evaluation-related activities, from the collection of baseline data for situational analyses to monitoring and evaluation exercises that are carried out after the completion of programmes and projects.

It must be reiterated that evaluation is mandatory for programmes and projects with budgets of $1 million and above. Therefore, securing an adequate budget provision for such evaluations is necessary.

When Should Funding Arrangements Be Made?
Funding requirements must be identified when the major reviews and evaluation exercises for the CCF are being planned.

At the programme or project level, the costs of monitoring- and evaluation-related activities must be estimated at the formulation stage in consultation with the key stakeholders. They must then be included in the programme or project budget.

What Sources and Mechanisms Can Be Used to Provide Funds for Monitoring and Evaluation?
First and foremost, programme and project budgets must include allocations for monitoring and evaluation. In addition, two support-cost facilities under the successor programming arrangements may be tapped for funds: Support Services for Policy and Programme Development (SPPD) and Support for Technical Services at the Project Level (STS).16

How Can Monitoring and Evaluation Costs Be Realistically Estimated?
The most important factors to consider when estimating the amount of money that will be needed for monitoring and evaluation actions are their scope, methodology and participants. Cluster evaluations requiring the participation of a large number of stakeholders would, of course, require more funds than the evaluation of an individual programme or project.

Given the diversity in the types of interventions and the development context in each programme country, it is not practical to prescribe the percentage of the total budget of every programme or project that should be allocated for monitoring and evaluation actions. However, the following figures may provide some guidance in this area:

• for monitoring:
  one to five per cent of the programme or project budget;

• for an evaluation of one project:
  $35,000 to $40,000;

• for a cluster evaluation:
  $50,000 to $100,000.

There is room for considerable flexibility in this regard, but the guiding principles are that the careful planning of monitoring actions and evaluation exercises requires a great deal of selectivity and that once a selection of the activities that will be undertaken has been made, sufficient funds for those activities must be ensured.

16For details, see UNDP, Overview of Support Cost Arrangements under the Successor Programming Arrangements (issued 23

December 1996, effective January 1997).

PART ONE
CHAPTER 19. RESPONSIBILITIES AND CAPACITIES

MONITORING AND EVALUATION RESPONSIBILITIES
Under the national execution modality, the Government, as the executing agency for programmes and projects assisted by UNDP, bears primary responsibility for monitoring and evaluation. However, UNDP and United Nations agencies, as development partners of Governments, should assist them in developing their capacity to fulfil their responsibilities.

STRENGTHENING UNDP CAPACITY TO ASSIST GOVERNMENTS IN MONITORING AND EVALUATION
UNDP must prepare itself for the task of assisting Governments in strengthening their capacities for monitoring and evaluation. The regional work-shops on monitoring and evaluation sponsored by OESP for UNDP and national programme staff are an important step towards this end. Information on previous monitoring experience will be easily accessible through the Executive Information Management System (EIMS), for which a prototype has been developed, while CEDAB is a source of lessons learned from evaluations. Other initiatives that could help UNDP country offices in this area need to be explored, at least on an experimental basis, in selected countries.

Role of OESP
OESP serves as the central evaluation office within UNDP. It performs monitoring- and evaluation-related functions in the following areas:

Monitoring and Evaluation Policies

• Develops policies and guidelines on monitoring and evaluation;

• Monitors compliance with evaluation requirements.

Development of Methodological Innovations

• Develops methodological innovations, such as the evaluation of interventions under the programme approach, participatory evaluation, performance measurement andimpact assessment.

Conduct of Evaluations

• Initiates and manages policy evaluations, including strategic evaluations;

• Assists regional bureaux, country offices and funds in association with UNDP in organizing evaluations.

Lessons Learned

• Assesses trends in programme and project performance;

• Maintains and updates CEDAB, which contains lessons and other relevant information from evaluation reports, and provides other UNDP units at headquarters and in the country offices with access to CEDAB;

• Produces a publication series on lessons learned.

Linkages with Other International Bodies

• Links with United Nations agencies in the harmonization of monitoring and evaluation guidelines as chair of the Inter-Agency Working Group on Evaluation and active participant of the JCGP;

• Links with the OECD/DAC Expert Group on Evaluation;

• Links with professional evaluation associations.

Collaboration and Coordination between OESP and Other UNDP Units
Effective performance of the functions described above requires a close working relationship between OESP and other units such as:

• the Division for Operational Policies and Procedures (DOPP), to ensure that monitoring and evaluation policies and those pertaining to other aspects of the programme or project cycle are consistent and mutually supportive;

• the Division for Audit and Management Review (DAMR), to effect a coordinated approach to using monitoring, evaluations, audits and management reviews as complementary management tools;

• the regional bureaux and the country offices, to plan monitoring and evaluation actions, particularly the major evaluation exercises for the CCF, and to monitor compliance with mandatory evaluations;

• the Learning Resources Centre, to develop and conduct appropriate training programmes and workshops on concepts, methodologies and practices relating to monitoring and evaluation.

Two major areas where collaboration and coordination between OESP and other UNDP units are particularly important are evaluation planning in the context of the CCF and the development and selection of indicators.

Evaluation Planning at the Country Level
Since OESP is mandated to report periodically to the Administrator on the compliance rate for mandatory and non-mandatory evaluations, country offices, through their regional bureaux, must:

• submit to OESP their triennial evaluation plans and

• inform OESP of their compliance with evaluation requirements, clearly communicating reasons for any non-compliance.

Selection of Indicators
OESP may provide assistance to country offices and programme and project managers in the following areas:

• facilitating the selection of performance indicators by programme and project managers and other key stakeholders through sharing access to current views on the subject among other United Nations agencies and donor organizations;

• identifying resource persons who can facilitate the selection and use of performance indicators at various stages of the programme or project cycle (as well as the compilation of baseline data on the four areas of thematic focus under the SHD framework);

• organizing and conducting training, in collaboration with other relevant UNDP units such as the Learning Resources Centre, to engender a positive attitude towards, and provide the knowledge and skills required in, the selection and use of performance indicators.