How do we know the systems change we work for is happening?

Bill & Melinda Gates Foundation teams up with UNDP to find out

February 6, 2023

Authors

Andrea Bina

Monitoring, Evaluation and Learning Specialist

Simon Cooper

Communications Specialist, UNDP Food and Agricultural Commodity Systems

How do we know the systems change we work for is happening?

“System change not climate change” is a central call to action everywhere from climate marches to government policy negotiations like CoP27. But what do we really mean, in practice, when we talk about “systems change”? And how do we know whether change is really happening, and who is benefitting from it? What can we learn to help us design and implement our work effectively?

“Systems approaches” have been adopted as the most adequate way to frame and guide the design and implementation of interventions aimed at addressing complex problems, especially in food and agriculture¹. International development donors, UN agencies, governments, companies, and civil society alike have been increasingly targeting food systems transformation as their ultimate goal.So if everyone says they are aiming to transform food systems, how do we know whether they are succeeding? And if so, by how much? These critical questions are at the heart of UNDP’s investigation on systems monitoring, learning and evaluation (MLE), supported by generous funding from the Bill and Melinda Gates Foundation.

UNDP Food and Agricultural Commodity Systems (FACS) Global Head Andrew Bovarnick said:

“We need new ways of catalyzing change in food systems. Interventions that address the root causes and not the symptoms. Intervention strategies though that go after root causes in systems, no matter how well thought through, are based on assumptions on how a system operates and what it takes to change it. So what we need to complement systemic interventions are tools to assess, measure and generate insights on how well the interventions are actually catalyzing change in a given system and to what extent, why and how. Only with the development and application of such tools, which we want to make easily accessible to everyone, can we evolve technical assistance to truly and effectively transform systems.”

The initiative addresses two core questions²:

How do we know if we are making progress, generating results, and contributing to positive change in a complex system?
How do we use MLE processes and the insights they provide to continuously learn and adapt when engaging with complex systems?

These questions will be explored in general terms by looking at existing evidence and insights coming from all sectors, but the initiative will focus on food systems as the primary space for experimentation.

The current state of knowledge will be gauged by reviewing existing literature on the topic and developing case studies. The team will also devise experiments and hands-on pilots aimed at generating new insights and learnings around the key learning questions.

The initiative team will broaden the search for ideas and understanding by convening a movement focused on shedding light and generating insights about the key learning questions. Practitioners from different sectors will be invited to discuss and co-create alongside the core team, and will have access to a series of tailored workshops and events, including through the UNDP Food and Agricultural Commodity Systems community and M&E Sandbox.

The initiative builds on the experience built over 10+ years of operation by the UNDP FACS team, and supports the Strategic Innovation Unit’s M&E Sandbox – a broad collective effort to rethink monitoring, evaluation and learning with a systems lens.

As UNDP’s Strategic Plan 2022-2025 puts it:

“Traditional, linear results-based management has not evolved to measure new ways of working and the outcomes of a portfolio approach. Work is under way to develop systems of monitoring, evaluation and results measurement better suited to these new ways of working, that value learning as results, and can track transformative change and its enablers, like changing social norms, over longer timespans.”

If you would like to contribute your ideas to this initiative, please reach out to andrea.bina@undp.org who will send further details on collaborating and/or accessing our digital space and resources.

_{¹These are not limited to climate change but include other negative externalities of food systems such as poverty, inequalities, food insecurity and the diseases caused by unhealthy diets or the use of toxic substances in agricultural production.}

_{² Please refer to table 1 at the end of this article for the complete, though not definitive, list of key learning questions under each of the two main questions.}

_{Table 1: List of core questions and sub-questions under investigation by the UNDP Systems MLE initiative.}

List of core questions and sub-questions
1.How do we know if we are making progress, generating results, and contributing to positive change in a complex system?	Measuring system change: How can we document / measure change in complex real-world systems?
	Progress tracking: How do we do we know if we are on track when working systemically to address complex problems?
	End-beneficiary relevance and impact: How do we know change is really occurring and we are impacting the people we care about – e.g. farmers and their communities?
	Attribution and contribution: How to capture/interrogate our contribution to system change?
	Aggregate results: How do we measure progress, value, and results in the aggregate across a portfolio of our own interventions or across what many different actors are doing?
2.How do we use MLE processes and the insights they provide to learn and adapt when engaging with complex systems?	Learning, adaptation and (re)design: What MLE frameworks, approaches, and practices are useful for allowing us and others to continuously learn from, adapt, and accelerate our efforts to transform complex real-world systems?
	Accountability: How do we design and implement accountability frameworks, metrics and/or practices that allow for emergence, flexibility and adaptation and that shifts focus from individual interventions/projects as primary unit for accountability towards portfolios of interventions/projects?
Cross-cutting themes
Diversity and inclusion: How do we best empower and nurture perspectives, capabilities, practices, and peer-exchange among stakeholders (globally and locally) on these issues? How do we ensure inclusive processes and diverse voices?
Approaches, methods and data: What methodologies and approaches can be used to i) measure system change, ii) learn and adapt, and iii) monitor progress and results? When should different methodologies be used in what context? What are the different approaches and tools that can be applied at different scales (e.g. projects vs programmes vs portfolios vs whole ecosystem; scale of budget and resources, small to large)? What counts as good (enough) evidence and data?
Implementation: How do we implement these approaches? What capacity is required to implement? What are the obstacles and challenges for implementation and how do we address these? How can we increase capacity in projects/programmes (in food and agriculture) for working more skillfully with complexity?