PART TWO
CHAPTER 23. PLANNING AND MANAGING AN EVALUATION

Effective planning and management are crucial to the conduct of a successful evaluation. Several principles have proven to be particularly helpful in increasing effectiveness:

• adequate preparations should be made prior to conducting the evaluation;

• terms of reference (TOR) should be designed with the involvement of key stakeholders;

• a system to monitor the evaluation exercise substantively should be established.

ADVANCE PREPARATION
Before the TOR are drafted, a few key questions should be asked and answered:

• Who initiated the evaluation?

• Why is the evaluation being undertaken?

• What products are expected from the evaluation exercise?

• Who are the stakeholders?

FORMULATING THE TOR
Adequate time should be invested in formulating and refining the TOR. Key stakeholders must be involved in the process as much as possible so that their views and interests can be reflected in the TOR.

To ensure the effective conduct of an appropriate evaluation, the TOR should contain the following elements.

Introduction
The introduction should consist of a brief description of the programme or project and its objectives within the context of the development needs and priorities of the programme country and identification of the key stakeholders.

Objectives of the Evaluation
In this section, the answers to the following questions must be clearly stated:

• Who initiated the evaluation?

• Why is the evaluation being undertaken?

• What will the evaluation try to accomplish?

• Who are the main stakeholders of the evaluation?

Scope of the Evaluation
The evaluation sample should be described in terms of:

• the type(s) of programme(s) and/or project(s) that will be evaluated;

• geographic coverage of the programme(s) and/or project(s); and

• time frame of the programme(s) or project(s).

Issues to Be Addressed by the Evaluation
The TOR should include a description of issues pertaining to the relevance, performance and success of the programme(s) and/or project(s) covered by the evaluation. The specific aspects of those dimensions will vary by type of evaluation. For example, issues relating to impact as a criterion of success are more appropriately addressed by an ex-post evaluation than a mid-term evaluation.

The TOR should also indicate that recommendations will be made and that lessons learned will be drawn from the programme or project experience.

Products Expected from the Evaluation
This section should contain a description of the products that the evaluation manager wants to obtain from the evaluation, e.g., a particular strategy, recommendations on the best practices in a certain area or on the appropriate niche for UNDP interventions in a specific programme country.

Methodology or Evaluation Approach
The methodology that will be used by the evaluation team should be presented in detail. It may include information on:

• documentation review (desk study);

• interviews;

• field visits;

• questionnaires;

• participatory techniques and other approaches for the gathering and analysis of data.

Evaluation Team

• Number of evaluators and areas of expertise;

• Responsibilities.

Implementation Arrangements
This section should consist of details about the following:

• management arrangements, specifically the role of UNDP country offices (consultation with the offices and their prior approval when the evaluation is being initiated by headquarters);

• realistic time frame for the evaluation process
  • desk review
  • briefings of evaluators
  • visits to the field, interviews, questionnaires
  • debriefings
  • preparation of report;

• resources required
  • logistical support needed.

NB: While funding arrangements for the evaluation must be considered at the

planning stage, they are not to be reflected in the TOR itself.

Chapter 24 provides guidance on planning an evaluation of a subregional programme. Chapter 25 presents a sample outline for a mid-term evaluation report on a capacity development project.

SUBSTANTIVE MONITORING OF THE EVALUATION EXERCISE
The UNDP country office assumes a critical role in managing the evaluation exercise being conducted in the field. The staff designated for this purpose must be thoroughly familiar with the TOR and work with members of the evaluation team to arrange effective monitoring of the evaluation exercise to ensure that the TOR are fully satisfied. Such arrangements may include:

• briefings on the progress of the evaluation;

• preparation of an issues report midway through the evaluation to indicate the preliminary findings of the evaluation;

• validation of preliminary findings with stakeholders through circulation of initial reports for comments, meetings, and other types of feedback mechanisms;

• formal debriefings with evaluators in the field and at headquarters.