Evaluating HIV and development

Evaluating HIV and development: guidelines and suggestions
Evaluating HIV and AIDS: Why capacity development is central to assessing performance

Preface

Almost since its inception, the HIV and Development Programme (HDP) recognised the potential contribution, which could be made by evaluation to improved programme and project effectiveness. In the early nineties, HDP worked in partnership with the Salvation Army in designing and facilitating a workshop intended to build the capacity of programme implementers to identify, document and disseminate lessons learned. In 1996, HDP took on the role of co-ordinating the donor group response to the draft report of the team of consultants tasked with evaluating the International HIV/AIDS Alliance, an initiative in which UNDP had been involved almost since the beginning.

Taking advantage of the opportunity presented by an evaluation of its own UNDP Global Programme-funded activities, HDP worked with colleagues inside and outside UNDP, particularly those working in the area of evaluation, in designing an evaluation methodology consistent with the capacity-building emphasis of the Programme.

A number of events occurring more or less simultaneously led HDP towards a more consistent involvement in the issue of the evaluation of HIV-related initiatives. First among these was HDP’s own history in relation to involvement with evaluation and the specific challenges posed by evaluating initiatives relating to HIV and development. Second, the experience of co-ordinating the donor response to the Alliance evaluation highlighted some of the challenges of working simultaneously on evaluation with large groups of stakeholders and beneficiaries. Third, the experience and process of the HDP programme evaluation pointed to the possibility of evaluation methodologies that could respond to the needs of a broader range of stakeholders instead of focusing narrowly upon the needs of donors. Finally, in addition to the above mentioned evaluation reports, a number of other significant evaluations of HIV-related projects, programmes and organisations had been conducted and reported on in the course of the previous eighteen months.

It was decided by HDP to form a ‘virtual’ working group which would include HDP personnel and a group of consultants with experience of working with HDP, UNDP, UNAIDS or other relevant agencies, such as NGOs. Group members were tasked with reviewing (according to a set of key questions) one or more of the above evaluation reports. These reviews and the ensuing deliberations formed the basis of a meeting of the working group in September 1999. In response to the presentation of the reviews and the ensuing deliberations, it was decided to form two smaller groups focusing respectively upon issues of evaluation methodology and capacity building.

The papers enclosed in this publication are the results of the initiative. Bruce Parnell and Des Cohen, the respective focal points for the two ‘theme’ groups, are the authors of the papers. Cate Hankins, Peter Gordon and Tang Tuong (members of the working group) together with colleagues from the UNDP Evaluation Office, reviewed drafts of the papers and made a number of suggestions which (to the extent that it has been possible to do so) are reflected in these final versions.

Time constraints prevented the kind of exhaustive consultation process that might have led to institutional ownership of the papers. Therefore, they should be understood as reflecting the views of the individual authors rather than the views of the entire working group, HDP or UNDP. The purpose of these papers is not to present the definitive opinions of ‘experts’: instead, it is to share more widely current thinking on evaluation as it relates to HIV and development and capacity building approaches, together with existing constraints and possibilities, in recognition of the potential of evaluation to enhance learning among donors, and stakeholders and beneficiaries.

Mina Mauerstein-Bail
Manager
HIV and Development Programme

Evaluating HIV and development: guidelines and suggestions
by Bruce Parnell

Introduction

Addressing HIV as a development issue requires responses which are broader than the more ‘traditional’ focus on health promotion and service provision which is characteristic of a ‘public health response’. Epidemiology, clinical and behavioural research, in isolation, are seldom adequate to the task of measuring subtle, complex changes in culture and socio-economic conditions that play such an integral role in the HIV epidemic. Clearly, HIV surveillance is a useful activity for tracking the epidemic, and behavioural research can make important contributions to our understanding of the extent and role of factors associated with specific sexual or drug using behaviour as they relate to the epidemic. However, neither can account for the varying levels of susceptibility among different groups living in similar circumstances: the very factors that a ‘development’ approach to the epidemic seeks to address.

Many countries and regions are implementing responses to the challenges of HIV and development in order to:

• change the socio-economic factors that create the conditions in which the epidemic thrives
• facilitate provision of care and support for people living with HIV and their dependants, in situations where present institutional arrangements are unable to meet those needs
• build the capacity of people and organisations in different sectors to respond to the challenges of the HIV epidemic, including their capacity to analyse current and future situations, and their capacity to work across sectors in responding to issues of importance.

Processes and mechanisms adopted toward these ends have included broad programmes and targeted projects, policy development and strategic planning. But how can these best be evaluated? In other words, how can we maximise the learning from these initiatives? This is an important question, because effective evaluation can provide the information necessary to improve the design, scope and methodology of many policy and programming responses to the HIV epidemic. Asking the right questions is a first, important step in ensuring that the required information and lessons are captured through evaluation.

The purpose of this paper is to articulate the kinds of questions which might enhance our capacity to understand and mitigate the impact of the HIV epidemic upon sustainable human development and to demonstrate the importance of evaluation as a way of generating learning in the field of HIV and development.

UNDP, HIV and Development

As the UN agency with the specific mandate of promoting sustainable human development, UNDP, with some foresight, initiated organisational responses to the HIV epidemic in the late 1980s in anticipation of the devastating impact of the epidemic upon the social and economic fabric of developing countries. In the early 1990s this response was institutionalised in the form of the HIV and Development Programme, based in New York, which was charged with working throughout the UNDP system in support of co-ordination of its HIV-related activities. By then, UNDP had a significant body of relevant human and technical resources, for example, in relation to poverty, gender and governance - all of which were recognised as critical dimensions of the epidemic. Moreover, UNDP had an established team (the Evaluation Office) working on the monitoring and evaluation of development projects and programmes

Evaluating HIV and Development Initiatives

Drawing on a number of sources, this paper suggests some issues to consider in relation to evaluating initiatives on HIV and development. These include:

• Results-Oriented Monitoring and Evaluation: A Handbook for Programme Managers.¹ (UNDP Evaluation Office);
• Field experience of practitioners involved in HIV and development processes (including the deliberations of a 1999 meeting held in New York and bringing together a range of development practitioners from different regions to consider these issues);
• Drawing upon a number of project evaluations, undertaken in recent years, with particular relevance to HIV and development.

This paper is intended to articulate some of the questions that might most usefully be explored through the evaluation of HIV and development initiatives and to identify potentially appropriate methods for this work. The paper does not offer a set of "how to" instructions for the evaluation of HIV-initiatives. This kind of advice is already available from a variety of sources. For instance, information relating to precorders for the evaluation of UNDP supported programmes and projects can be found in the UNDP programming manual (Chapter 7) and further guidance on organising and conducting evaluations is to be found in the UNDP Evaluation Office’s Results-Oriented Monitoring and Evaluation: A Handbook for Programme Managers.

These guidelines focus more specifically on some of the key issues that should be considered in evaluating initiatives on HIV and development. It is hoped that the use of these guidelines will lead to the identification of a broader range of issues to be addressed by future evaluations and that these, in turn, will clarify a more extensive set of programming and policy options. The ultimate goal is to achieve more effective, extensive and sustainable responses to the challenges of HIV and development.

The nature of evaluation

Evaluation is a process used to enhance understanding. Within the context of HIV and development, there is no ‘single issue’ that should be understood. Rather, evaluation processes have to be responsive to specific needs as they arise during particular initiatives. So, what makes a suitable starting point?

A primary consideration is that an evaluation framework should be developed which leads to processes that are useful:

Evaluation findings are used to make decisions. Evaluation is different from monitoring or auditing of programme implementation in that it asks questions that go beyond those that might have been the focus of programme designers. Monitoring and auditing are processes that measure the extent to which a project has done what its designers said it would do. In contrast, evaluation involves reflection on what has been done, and learning about the value of what has been done, in order to feed into the next stage of planning for what should happen next.

UNDP’s Results-Oriented Monitoring and Evaluation: A Handbook for Programme Managers notes that monitoring and evaluation are closely related, and that they are mutually supportive. It also notes the distinction between these two functions and other functions of auditing and research, summarising the relationships as follows⁴:

Another useful way of considering the same relationships is to focus upon the purpose of the three functions:

Given these distinctions, an emphasis on learning has been stressed for development programmes:

Thus, evaluation can be considered as part of an ongoing learning process, through which people will find the solutions to specific problems relating to HIV and development. This is summarised in the diagram below. Evaluation occurs when people:

• consider what is needed in the current situation
• reflect on what has happened so far
• collect data to check whether their initial impressions are accurate
• analyse that data
• use the data to decide how a situation has changed, or what actions should happen next.

A circle of learning: the general processes of learning

Some possible uses of evaluation

Within the learning cycle, what is the particular role of evaluation? This depends on what is considered most important to learn at any stage of the process. UNDP (1997b) suggests that three interrelated dimensions of programmes and projects must be assessed as the substantive focus of monitoring and evaluation: relevance, performance and success⁶. The handbook suggests the following considerations for each of these areas of focus:

Given this range of options, decisions must be made about the focus of any particular evaluation. These choices will depend on the purpose of the evaluation: i.e. what it is intended to be used for. Patton (1997)⁷ suggests three distinct categories and recommends that evaluation should have a primary purpose, based on choosing just one of these categories:

• to judge merit or worth
• to improve programmes
• to generate knowledge

Planning the evaluation

Ideally the time to begin planning an evaluation is during the project design phase. In this way, realistic monitoring and evaluation mechanisms can be identified and put in place from the outset. This will assist project management and staff and volunteers to assess the extent to which they are achieving what they set out to do, and if necessary recognise when and what changes need to be introduced. This approach is preferable to the usual practice of planning the evaluation at the end of the project since this timing can seriously constrain the options of the evaluation exercise. With early planning and the use of project staff, volunteers and beneficiaries as key ‘researchers’, evaluation can become a critical and valuable aspect of project culture, instead of an unwelcome, externally imposed requirement.

While the integration of evaluation from the outset may be an ideal, nonetheless there are certain conditions that need to be in place for an evaluation to have significant impact on programming and policy development. For example, those who are in a position to implement recommended changes and those who will be directly affected by any recommendations must have a minimum level of ‘buy-in’ to the evaluation process and willingness to implement its recommendations. There is a requisite level of relevant capacity (or commitment to building it) in order for people and institutions to implement or respond to any necessary changes. Adequate financial, logistical, facilitation, and managerial support are also essential if an evaluation is to proceed in a timely and effective fashion.

Once the purpose of the evaluation has been clarified together with the specific aspects of programme quality that have been defined as its focus, the next step is to consider how the actual evaluation process will be conducted. Two sets of issues are important at this point:

a) Which issues will be considered? Deciding upon the primary purpose of the evaluation, is only part of the process of determining the specific issues to be considered. The next section lists a number of key issues that might usefully be included in the evaluation.

b) Who will conduct the evaluation? Who will design the process, articulate the questions to be answered, collect and analyse the data, and apply the lessons learned through the evaluation?

Ideally evaluation should be conducted in ways that are designed to build the capacity of people and institutions by supporting them in undertaking their own data collection and analysis, drawing conclusions and making changes. In this way, sustainability can be promoted together with the ability to reflect and adapt in response to changes in local and national circumstances. This process can facilitate meaningful participation in the identification of the critical questions to be answered by an evaluation by those whose lives are most affected by a programme.

Key Evaluation Issues

The following tables contain questions that might usefully be explored in relation to the evaluation of initiatives on HIV and development. Some tables are considerably more detailed than others and simply reflect the extent of relevant discussion to date.

EVALUATING HIV AND AIDS
WHY CAPACITY DEVELOPMENT IS CENTRAL TO ASSESSING PERFORMANCE

by Desmond Cohen

1. UNDERSTANDING CAPACITY DEVELOPMENT

The core of UNDP’s mandate is the achievement of sustainable human development, and the instrument for doing this is capacity development. The term capacity development can be defined as follows:

Capacity is the ability of individuals and organisations to perform functions effectively, efficiently and sustainably. The term "capacity development" is preferred to "capacity building"; while capacity strengthening is important so are the retention of existing capacity, improvements in the way in which existing capacity is being utilised, and the retrieval of capacity which has been eroded or lost. Thus capacity development does NOT take place only through the training of additional staff or the creation of new organisations, but requires an enabling environment to ensure that people are used effectively, are retained within organisations and structures that need their inputs, and are motivated to perform their tasks.

All projects and programmes should develop a capacity development strategy and be evaluated on the basis of whether they address the key issues that this involves. Central to evaluation of the project or programme is whether in the formulation of a capacity development strategy a capacity assessment was first undertaken and whether the results of this assessment informed the strategies for capacity enhancement that would ensure that the project or programme was both feasible and sustainable.

A capacity assessment involves an analysis of capacity issues such as:

• The overall context: an examination of relevant economic, social and political conditions. These include political commitment to capacity development, the functioning of labour and asset markets, the standards of tertiary and professional training institutions, existing cultural and gender barriers, and the extent of the HIV epidemic. Unless a supportive overall environment is available resources will inevitably be wasted; identifying and addressing the policy and programme constraints to capacity development are critical first steps in ensuring that objectives will be achieved.
• The task network: a mapping of the array of organisations engaged in the performance of the function in question and the inter-organisational relationships involved. The network will generally include non-government as well as government organisations, users as well as providers of services, training institutions, and relationships with regional and overseas institutions. The assessment would examine the extent to which interactions among organisations constrain or enhance the capacity to undertake activities and achieve objectives.
• The organisation and its management: the organisational structure, corporate culture, management systems and processes that affect the recruitment, utilisation, recognition, motivation and retention of staff.
• The availability of human resources: the quantity and quality of personnel available to the organisation, the quality of training available and provided, the degree to which existing human resources are utilised effectively, and likely constraints on the future availability of essential human resources.

On the basis of a capacity assessment it is possible to examine the strengths and weaknesses of the existing arrangements for undertaking the project or programme, the reasons for weaknesses in existing capacity and why these have persisted over time, and the factors which are undermining existing capacity such as losses of skilled labour due to HIV-related mortality. Such an assessment would also generate insights into past failures and successes in regard to capacity strengthening and how to build on the latter in implementing activities.

What would be the main areas of focus of capacity development strategies in the light of the capacity assessment?

These might include at each level the following:

• The overall context: sustained investment in human resource development, the strengthening of institutions that mediate and prevent social conflict and build social capital, increased participation of communities in the design and implementation of development programmes, and mechanisms for ensuring that public policy and programmes are delivered in ways that are efficient and respond to the needs of sustainable development.
• The task network: targeted improvements to those parts of the network which constitute the most important constraints on performance; strengthening mechanisms for co-ordination, and enhancing approaches for systematically involving beneficiaries in the task network.
• The organisation and its management: open competitive recruitment and promotion procedures, performance review, recognition and reward structures, strengthening problem solving procedures and skills, and development of external support structures that improve organisational performance.
• The availability and utilisation of human resources: management training, processes and reward systems that induce commitment and effective performance by all personnel, identifying and responding to performance constraints, training opportunities related to performance and organisational needs, and responding to changing organisational objectives through human resource investment strategies and programmes.

What is the role of donor organisations in capacity development?

It is self evident that donors have a clear interest in supporting the capacity development strategies of national governments through the full integration of donor programme activities with those already underway or planned at country level. This makes it essential that donors formulate their activities for capacity development within coherent frameworks that ensure consistency with what each other is planning to do and consistency with national needs and capacities. It may be necessary for donors to support the establishment of co-ordination machinery and the strengthening of national capacity to co-ordinate donor activities effectively.

Donors should avoid activities that are destructive of national capacity, such as the recruitment of key staff away from national uses and the distortion of national pay scales through excessive payments to some nationals. The objectives of donors should be to support national capacity development through human resource investment and to strengthen the policy environment for a more effective delivery of projects and programmes that are relevant for sustainable development. Achieving these objectives may in some cases require that donors initially seek through their own activities to strengthen their own capacity and motivation to undertake capacity development as a prior first step towards more effective general performance.

How does one ensure that capacity development is integral to projects and programmes?

It is clear from research to date that putting in place a capacity development strategy and activities for its implementation at the design stage of projects is a sine qua non for more effective performance. It follows that unless the capacity development issues noted above are addressed at the design stage, and relevant activities identified and implemented as integral to projects and programmes, then only very partial and weak overall performance will result.

Monitoring and evaluation benchmarks should be developed at the design stage of projects and programmes. These benchmarks should reflect the priorities that have been selected for interventions based on an analysis of critical constraints and should take into account the fact that project and programme objectives and modalities will require adjustment during the life of the project and should not be left to be modified as a result of an end of project evaluation. The review and evaluation processes should be iterative and should examine the quality, timeliness and quantity of services provided in relation to organisational goals and client needs. These should be examined within the broader context of capacity assessment and follow-up, organisational management, structures and performance, and in terms of the availability and utilisation of human resources.

It follows that capacity development for undertaking relevant project and programme evaluation may itself have to be supported through specific activities in order to ensure effective monitoring and evaluation. This is a familiar dilemma: capacity may have to be strengthened first so that it will be adequate to achieve the iterative monitoring and evaluation required for better development outcomes. It also follows that once such capacity has been developed, commitment and mechanisms to ensure effective utilisation are essential.

2. CAPACITY DEVELOPMENT, THE HIV EPIDEMIC AND EVALUATION

Section 1 sets out the reasons why capacity development is the critical instrument for achieving development objectives and reviews those factors relevant to its full integration in projects and programmes. The important question is not whether the principles and criteria for effective capacity development, as described earlier are changed in a world of HIV and AIDS. If anything these guiding principles remain pertinent and the need for capacity development in the context of HIV and AIDS is enhanced rather than diminished. Rather, the epidemic changes the context within which capacity development strategies are expected to function and magnifies the difficulties of sustaining capacity.

It is useful to review the factors that change the context and feasibility of capacity development and the ways in which these affect the criteria that are important in evaluation of projects and programmes. The issues are complex and the particular situations are very diverse, therefore the following discussion should be seen as illustrative rather than definitive. The aim is to increase understanding of the effects that the HIV epidemic has on capacity development strategies, on programme implementation and on evaluation of projects and programmes.

How is the stock of Human Capital affected by HIV and AIDS?

HIV prevalence is concentrated amongst those in the key social and working age groups – between the ages of 15-45 years. In general, young women become infected at earlier ages than do young men and thus lose more years of healthy life. There is evidence that in mature epidemics in many countries of sub-Saharan Africa more women are infected than men, with an overall ratio of 6:5. Often in the early stages of the epidemic, rates of HIV acquisition are greater for those in higher educational and occupational groups whose members have greater mobility and spending power. This has important implications for the maintenance of both the stock and the flow of those human resources that require substantial social investment in education and training. While HIV prevalence may be highest in urban areas it is nevertheless the case that in absolute terms the largest numbers of those infected reside in rural areas in sub-Saharan Africa.

Rates of HIV prevalence amongst adults in many countries in sub-Saharan Africa are now in the range of 15-35%, with even higher rates in some locations and cities. The severity of the epidemic affects all social and occupational groups, including both men and women who have higher level skills and experience and those who are supposedly "unskilled". Amongst the latter category is most of the rural population which, in fact, has very valuable and hard to replace task-specific experience and skills.

Since HIV is concentrated amongst the core of the working population who have important social roles, and in particular the support and socialisation of children, there are bound to be effects both on the current generation and on subsequent generations. It follows that not only is the structure within families, including gender roles, changed by the experience of the epidemic but that there are also important issues to address with respect to maintaining households as productive enterprises.

What can be concluded from the losses of human resources that are critical for capacity development?

Most obvious are the losses of human capital due to the epidemic – skilled, educated, and unskilled men and women, in both urban and rural locations. One important issue is how to sustain production in circumstances of high morbidity and mortality across wide swathes of the active labour force. All programmes and projects have to deal with this fact: how can production be maintained in the face of ongoing and often severe losses of labour? Losses may be disruptive precisely because they are not confined to categories of labour that may be "easily" replaceable, but also affect many categories of more specialised labour including supervisory and managerial components.

It follows

• that capacity development strategies have to address the maintenance of productive capacity across many sectors of production in both formal and informal productive organisations – in both urban and rural locations.
• that because households are integrally affected both as economic units as well as ones with important social functions there is a need to sustain household capacity to function in the face of erosion of the economic base through losses of labour and reduction of productive assets.
• that since households are the primary organisation for the socialisation and care of children anything that erodes the ability of families to perform these roles will have effects that are inter-generational – on levels of poverty and on the skills and education of the future labour force.

But the effects on capacity are unfortunately not limited to the above and there are at least three other main channels through which capacity is undermined by the epidemic. These are as follows:

• The public sector in all countries is the main supplier of essential goods and services, such as education and health, transport and communications infrastructure, police and military security, law and order, welfare services, etc. But since public administration and public industries will also lose human resources due to HIV and AIDS then their capacity to sustain normal activities will be undermined. The question becomes in part that of how to maintain production of essential goods and services under conditions of losses of organisational and human resource capacity?
• Equally important are the general effects of the epidemic on fundamental conditions relating to security and law and order, including the maintenance of political authority and the functioning of associated governmental and para-governmental organisations. The effects will be felt both directly through the loss of human resources and indirectly through changes in the capacity to deal with intensifying economic, social and political problems. In other words, one of the outcomes of the epidemic will be a loss of political authority and a further undermining of the capacity to address mounting problems – in part due to the losses of personnel in a large number of state organisations.
• There is widespread evidence that the HIV epidemic has significant effects on the internal matrix of relations within social and economic organisations – at the level of households, private formal and informal enterprises, and the public sector. In part this is due to disruptive effects on productive relations due to morbidity and mortality, which mean losses of experience, management capacity and task-specific skills. In part it is the result of changes in morale within organisations resulting from the mounting level of mortality. The resulting outcome for many organisations – social and economic- is greater instability and dysfunctionality allied to a greater inability to address and solve problems.

Finally, there is the aggregation of all of these effects on development capacity that will in total be more than the sum of the individual components. This is the simple result of applying the concept of synergy in understanding and measuring the impact of the epidemic in total. Parts of the social and economic system depend for their efficiency on other parts working as "normal", but the HIV epidemic undermines this capacity so that feedback from one part of the system disrupts the planning and production in other parts. In the aggregate the impacts are multiplied because of these "systemic" effects - inducing further reductions in capacity throughout the social and economic system.

What does this mean for Evaluation of Projects and Programmes?

There is perhaps no need to detail the self-evident implications of the foregoing for capacity development and for the evaluation of projects and programmes. Not only is it necessary to adapt the kinds of projects and programmes that need to be implemented so as to meet the new priorities of those countries that are deeply affected by the epidemic, but attention has to be paid to issues of how to strengthen capacity and maintain it in the face of multiple forces that are systematically reducing existing capacity. This is occurring in conditions, where as noted above, the capacity to address and solve problems is itself diminished by the losses of human resources and the associated reduction in problem-solving capacity within social and economic organisations.

It follows that it is even more essential that all of the factors and principles identified in Section 1 in respect to the design and implementation of capacity development strategies be integral to all projects and programmes. It is not that the principles for effective capacity development are intrinsically changed by the epidemic but rather that sustaining capacity becomes an even more central objective for all projects and programmes, along with the aim of ensuring that they address how best to achieve their objectives under conditions in which both direct and indirect factors are undermining capacity. This must also consider the capacity of other organisations, including relevant networks, and of public administration and other state institutions involved in the public production of goods and services.

What are the Strategic Evaluation Questions when the Objective is Capacity Development?

What follows is intended to provide some guidance to those undertaking evaluation of projects and programmes in countries with high rates of HIV prevalence. It is not intended to be complete but instead to generate some insight into the kinds of strategic questions that need to be addressed by evaluators if capacity development as described above is to be realised.

Thus:

• Did those designing the project/programme undertake a capacity assessment? Did the capacity assessment take into account the effects of the HIV epidemic? If not, why not, and what were the consequences for the success or failure of the project/programme? What lessons can be learned for future projects/programmes from the failure to undertake a capacity development assessment?
• Where an appropriate capacity assessment was undertaken at the design stages of a project/programme what were the main conclusions relevant for the successful completion of the activity? To what extent did the assessment take into account the effects of the HIV epidemic on capacity development?
• What capacity development objectives were set for the project/programme, and what activities were established for the project in pursuit of these objectives? Were objectives and activities appropriate, and if not why not?
• What activities relating to capacity development were undertaken by the project/programme? Did these achieve their objectives and if not why not?
• How far were capacity development constraints external to the project/programme important for its success or failure in meeting its overall development objectives?
• Did the project/programme from the initial stages address the issues of sustaining its own productive and organisational capacity in the face of HIV/AIDS? What activities were undertaken to achieve this objective, and with what success or failure?
• Performance of the project/programme in part depends on the capacity of beneficiaries [e.g. of households in the case of poverty alleviation projects, or of small farmers in the case of rural development projects]. Were attempts made to assess their capacity, and what activities were proposed/undertaken to ensure that their capacity was sustained?
• Were those activities aimed at sustaining capacity, for example, of networks of NGOs and CBOs that were external to the project/programme, successful or not and were appropriate lessons drawn from this experience for the operational success of the project/programme?
• Did those responsible for the project/programme undertake regular reviews of performance relative to the attainment of objectives and did these include specific attention to capacity development both internal and external to the activity? What actions were taken in the light of periodic assessment of capacity development aims relative to performance, and with what outcomes?
• At the end of project/programme were the lessons of capacity development identified, and what attempts were made to ensure that these were transmitted to those who needed to absorb and use these lessons for future development activities?
• Did the project/programme put in place the development capacity essential for sustaining this activity? If not then why not, and what lessons can be drawn from the project/programme about the determinants of the success or failure of such activities in ensuring sustainability under conditions of HIV and AIDS?

CONCLUSION

Capacity development in the circumstances of the HIV epidemic is both significantly more difficult to achieve and is critical to the successful outcome of projects and programmes. Thus important evaluation criteria focus on measuring the extent to which projects and programmes at the design stage take account of the many ways in which the HIV epidemic undermines capacity in all of its dimensions, both directly and indirectly, and respond to this challenge in ways that lead to effective outcomes. Whether the project or programme sets out to achieve capacity development, and whether it was successful in doing so in a sustainable way, become perhaps the most important questions to answer in evaluating performance.

Select Bibliography

There is a large literature on the HIV epidemic and on generic issues of evaluation but relatively little that addresses issues of capacity development and HIV in the context of evaluation. The following are some suggestions for follow-up and web sites locations are identified as the most easily accessible way of finding sources.

Capacity Assessment and Development –In a Systems and Strategic Management Context [Technical Advisory Paper No.3, Management Development and Governance Division, UNDP Jan 1998].

Capacity Development-Lessons of Experience and Guiding Principles [UNDP, Dec 1994].

Human Capital and the HIV Epidemic [http://www.hivdev.org.uk]

Responding to the Socio-Economic Impact of the HIV Epidemic in sub-Saharan Africa – Why a Systems Approach is Needed [http://www.undp.org/hiv]

The HIV Epidemic and the Education Sector in sub-Saharan Africa [http://www.undp.org/hiv]

The HIV Epidemic and Sustainable Human Development [http://www.undp.org/hiv]

The Economic Impact of the HIV Epidemic [http://www.undp.org/hiv]

Assessment of the Socio-Economic Impact of HIV/AIDS [http://www.hivdev.org.uk]

Endnotes

1. This publication is currently undergoing review and it is anticipated that it will be accompanied by sets of guidelines on monitoring and evaluation in relation to specific thematic areas and subjects.
2. Patton 1997:23
3. UNDP Evaluation Office Results-Oriented Monitoring and Evaluation: A Handbook for Programme Managers.1997. p.10
4. UNDP 1997b. p. 19
5. UNDP 1997 p.26.
6. p.76.
7. These questions will potentially be relevant to the evaluation of any project, not just those focused on capacity development