The Issue

In 1995, the Beijing Conference crystallized a shared commitment among experts, practitioners and funding agencies to integrate gender equality across the broad world of aid. Most fundamentally, it introduced a tool, gender mainstreaming, meant to turn that commitment into real practice. This created a system where every aid project would ask a set of questions addressing gender inequalities and their effects at every stage from planning and design to implementation, monitoring and evaluation. The idea was that formal commitments could contribute to substantive change by asking “who is affected differently, why, and what can be done to change that result” (Miller & Razavi, 1995).  

Thirty years later, gender mainstreaming has become a pillar of development aid with a system based on the Gender Equality Markers (GEMs), assigned by donors to indicate how gender equality is incorporated into a project’s objectives and design:  

• GEM0: The project “does not target” gender equality.
• GEM1: Gender equality is a “significant objective”, but not the principal objective.
• GEM2: Gender equality is a “principal objective” of the project.  

However, integrating gender mainstreaming in throughout projects’ lifecycle, from ideation to completion, is a long difficult process because it requires consistent attention, relevant expertise, institutional support and follow-through at each stage. Given such complexity, a fundamental question arises: does a gender equality marker on a project actually mean that gender considerations are built into a project’s design?

What We Did

To answer this question, we developed a two-layered methodological approach to systematically capture the level of gender integration in development aid projects.

We first created a 23-point gender equality integration scorecard building on guidelines from the OECD, international funding agencies and existing research (FAO, 2024; OECD, 2025). The scorecard has two sections. Section A reflects the minimum requirements determined by the OECD and includes questions such as whether the project has at least one gender-specific indicator or includes sex-disaggregated data. Section B goes deeper and asks questions on gender analysis integration throughout the different phases of a project, such as asking whether the project accounted for intersecting identities or involved women in decision-making.  

As a pilot, we then collected 14 health-related projects implemented by UN agencies in Indonesia with different gender equality markers (GEMs) from the International Aid Transparency Initiative (IATI) database. Using the 23-item scorecard, we developed and refined prompts to instruct two Large Language Models (LLMs) to assess project documentation and assign 0 or 1 point per item based on the presence or absence of supporting evidence. Finally, after evaluating each project with both LLMs, we cross-validated their results to improve evaluation quality.  

The main contribution of this methodology is the testing of LLMs use to systematically assess a large body of documentation and evaluate them against a carefully developed scorecard. This methodology not only shows the advantage of LLM use in similar systematic evaluations but also underscores the potential for expanding this methodological approach to wider development aid databases across sectors, agencies, and countries.  

What We Found  

First, the same marker can hide enormous variation. Among the six projects carrying the highest possible gender marker (GEM2), where gender equality is the principal objective, total scores ranged from 94% to 37%. The highest performing programmes carried out a thorough gender analysis by examining how gender intersects with disability and geography and building concrete mechanisms for women's participation during project implementation. At the other end, however, we found projects that consistently framed women as beneficiaries but offered little in the way of actual gender analysis and integration in the project design. Despite carrying the same marker, these projects encompass forms and depths of gender analysis that are worlds apart.

Figure 1: Projects' scores on section A, based on OECD minimum requirements.

Figure 2: Projects' scores on Section B (deeper gender integration). Only 12 projects had sufficient documentation for Section B assessment.

Second, most projects meet the formal requirements of gender marking, but only a handful go the full distance. If we zoom in on the results by section, the gap between formality and substantive integration becomes evident. On section A, which tracks integration of minimum OECD requirements, 10 of 14 projects scored above 50%. Conversely on section B, testing for more holistic gender equality considerations across project lifecycle (e.g., examining root causes of inequality, planning for unintended consequences due to the project implementation), only 3 of 12 projects scored above 50%.

Lastly, the results also suggest that markers may sometimes overlook meaningful gender integration within project design. The KOICA-UNICEF maternal health project (GEM0), for instance, scored higher than several GEM1 and GEM2 projects, suggesting that markers and project substance might not always travel together. Importantly, however, this needs to be read with caution as this was the only case of its kind in our dataset.  

What This Means  

These results offer important insights on the state of gender mainstreaming practice in development aid, as well as new methodological possibilities for assessment.

• Insight 1: The limits of the GEM box. Our findings show that projects carrying the same gender marker can differ in their respective levels of gender analysis integration. Particularly, the way gender analysis is embedded in the understanding of the problems, used to design interventions, and address needs can differ widely in content and quality (Moser and Moser, 2005). This is a fundamental issue because these markers are one of the main ways we track progress in closing gender gaps. If a marker doesn't reliably mean what we take it to mean, then its value as a monitoring tool becomes limited. While reconsidering the marker itself may appear to be one solution, restructuring an established institutional tool would require considerable time and resources which may reproduce the same problem if weak gender integration in practice is not addressed. A more practical and pragmatic solution would be to put more attention and effort into training personnel and reinforcing the importance of well-performed gender analysis and its incorporation in projects’ lifecycle (Kuusi, 2025).  
• Insight 2: Moving beyond formality. Our results reveal an important gap between meeting the minimum requirements and crossing the bridge to real integration. Irrespective of the gender marker, projects perform substantially better on Section A than on Section B. In other words, most projects can speak the language of gender mainstreaming at the surface level, but the commitment thins out as they move from the broader talk of gender inequality to the deeper integration (Kalbarczyk et al., 2025). This matters because the road to a sustainable development that provides equal opportunities to all genders is paved with obstacles, and we are not addressing them. Creating projects that can articulate the problem is no longer enough. To enable practitioners and beneficiaries to challenge and change the structures keeping inequality in place, we need to ask why inequality is there in the first place. We need to stop circling the roundabout of gender awareness and move towards the next destination: gender transformative practice.  
• Insight 3: Road to systematic assessment. This research also highlights the advantage of using LLMs to review large volumes of documentation and identify blind spots or consistently overlooked components in project design. As emerging research in other fields has shown, LLMs can reduce the time and cost required for this type of analysis, as well as maintain consistency in applying the same criterion across many documents (Gilardi et al., 2023; Ornstein et al., 2025). These approaches are just emerging, and this study is limited to projects from one sector, one country and one type of agency. Nonetheless, it shows the potential that the field of development studies stands to gain by scaling this method to other sectors, countries and funding agencies to reveal patterns and trends that are currently obscured by fragmentated aid information. Similarly, used alongside expert validation, this approach can also be expanded beyond evaluating gender equality markers to assessing other policy commitments, such as climate and environmental objectives.

To conclude, this research shows that (1) LLMs can play a useful role in project evaluation, and (2) gender markers remain useful for tracking commitment but should not be treated as stand-alone evidence of meaningful gender integration. Effective gender mainstreaming cannot be limited to simply including women as numbers in a results framework. The BERANI I project is the clearest example from our sample. The project not only specified participatory mechanisms to engage girls and boys through community-based learning and adolescent-friendly health education but committed to involve boys and men in the prevention of gender-based violence as key strategy, so that all beneficiaries could be active agents of change rather than passive recipients. In general, gender mainstreaming should unpack the layers of inherent gender inequality, build systems that translate commitments into practice, and bring us closer deconstructing the systemic gender-related imbalances across the social, economic and political spheres.

This blog post draws on the research project "Looking at Gender-targeted aid through the lenses of Large Language Models" co-authored by research residents Francesca Gentile and Damla Tas under the KU Development Futures Lab Student Research Residency Programme, supported by the UNDP Seoul Policy Centre.  

This article represents the views of the authors and does not reflect the views of UNDP. 

About the United Nations Development Programme

UNDP is the leading United Nations organization fighting to end the injustice of poverty, inequality, and climate change. Working with our broad network of experts and partners in 170 countries, we help nations to build integrated, lasting solutions for people and planet. Learn more at undp.org or follow at @UNDP.

About UNDP Seoul Policy Centre

UNDP Seoul Policy Centre is a facilitator of innovative development cooperation to catalyse the achievement of the Sustainable Development Goals. Through its SDG Partnerships programme and other South-South and Triangular Cooperation initiatives, the Centre supports countries by sharing innovative, tested-and-proven practices and policy tools on strategic development issues globally. Learn more at undp.org/policy-centre/seoul or follow at @UNDPSPC.

References

• FAO. (2024). Guide to mainstream gender in the FAO project cycle. https://openknowledge.fao.org/server/api/core/bitstreams/11bc3142-0997-47be-b900759e3670093a/content
• Gilardi, F., Alizadeh, M., & Kubli, M. (2023). ChatGPT outperforms crowd workers for text-annotation tasks. Proceedings of the National Academy of Sciences of the United States of America, 120(30), e2305016120. https://doi.org/10.1073/pnas.2305016120
• Kalbarczyk A, Saksena K, Carras MC, Gomes I, Zon J, Zhou X, Agarwal S. Developing a Gender Framework for Responsive and Adaptive Design in Digital Health (FORWARD) from a review of reviews. BMC Digit Health. 2025;3(1):91. doi: 10.1186/s44247-025-00231-y.
• Kuusi, I. (2025). From Norms to Numbers: The OECD DAC Gender Marker in Finnish-Afghan Development Practices. Forum for Development Studies, 52(3), 457-478. https://doi.org/10.1080/08039410.2025.2452488
• Miller, C., & Razavi, S. (1995). From WID to GAD: Conceptual shifts in the women and development discourse.  
• Moser, C., & Moser, A. (2005). Gender mainstreaming since Beijing: A review of success and limitations in international institutions. Gender & Development, 13(2), 11-22. https://doi.org/10.1080/13552070512331332283
• OECD. (2024). Aid (ODA) activities targeting gender equality and women's empowerment (database accessed on 11/02/2026)  
• OECD. (2025). The Handbook on the OECD DAC Gender Equality Policy Marker Retrieved from https://one.oecd.org/document/DCD/DAC/GEN(2025)2/en/pdf 
• Ornstein, J. T., Blasingame, E. N., & Truscott, J. S. (2025). How to train your stochastic parrot: large language models for political texts. Political Science Research and Methods, 13(2), 264–281. doi:10.1017/psrm.2024.64