Public discussion about AI still tends to begin in the same place: with larger models, larger data centres and larger spending plans. Large language models helped reinforce that picture. They pushed expectations higher and made cloud infrastructure feel like the natural home of advanced AI. That view captures only part of the story. 

The current AI debate often treats scale as the main measure of progress. The next phase may depend just as much on fit. 

The more AI becomes part of everyday systems, the more value will depend on whether those systems can operate at scale for institutions, and for people whose access to infrastructure is uneven. 

In a March 2026 paper, James Evans, Benjamin Bratton and Blaise Aguera y Arcas argued that intelligence may develop less like a single supermind and more like a social infrastructure made up of many interacting parts. Their focus was agentic AI, but the idea is useful beyond that debate. Once AI moves into the physical world, architecture starts to matter as much as model size. Some systems can rely on the cloud. Others need part of their intelligence to sit close to the device, sensor or user. 

 That is the promise of edge AI: deploying AI models on local or nearby devices so data can be processed where it is generated, enabling real-time responses with less dependence on constant cloud connectivity. 

This is different from what gets most of the attention today: cloud-based systems, always-on connectivity and centralized processing. Edge AI shifts at least part of that intelligence closer to where data is produced. In practice, this means a phone can recognize a wake word, a car can react to what its sensors see or a wearable can flag an abnormal signal—all locally and almost instantly. That makes AI more useful in settings where speed, resilience, bandwidth, privacy or cost matter.  

Edge AI is not a new idea. It has been taking shape across several fields for years. In 2020, researchers behind PlantVillage Nuru described a smartphone-based crop diagnosis tool that works in the field without requiring internet access. Or Arm's work on TinyML, which helped draw attention to another part of the same trend: machine learning models that can run on microcontrollers within strict power limits. 

These examples sit in different sectors, but they share a common design choice. They place intelligence close to the setting where a decision needs to happen. 

LLMs bring new urgency to the Edge AI discussion 

Large language models expanded what people expect from AI systems. Users now assume that digital tools can summarize long documents, help draft text, translate, answer questions and support decision-making in natural language. In public institutions, this creates a new opportunity. It also creates a new design problem. 

Many of the settings where development actors, municipalities and public systems work do not match the ideal conditions of a cloud-first product. Connectivity is uneven. Budgets are constrained. Trust is fragile. Sensitive data may not be suitable for routine transfer to external services. In many cases, the people who could benefit most from better digital tools are also the least well served by an architecture that assumes constant, high-quality access to the network. 

LLMs do not remove those constraints. In many cases, they sharpen them. Consider a field worker using an AI tool to query guidance in an area with weak connectivity. A cloud-only model may be slower, unavailable, or too costly for repeated use, while a smaller model running locally can still handle routine language tasks on the device. The implication is that progress is not only about bigger centralized models: it is also about placing enough intelligence close to the user for systems to remain useful under real conditions. 

Major technology companies are already moving toward more distributed model strategies, which shows that even the firms best positioned to benefit from centralized cale are investing in local inference for language tasks. 

This does not mean that LLMs suddenly become easy to run everywhere. Google also notes that deploying gigabyte-scale language models across diverse edge hardware remains a major technical challenge. The future of language AI will be mixed: some capabilities will stay in the cloud, while others move closer to the user. 

So what difference does it make for public sector development projects? 

Access. A cloud-only service assumes stable, affordable connectivity every time a user needs it. ITU figures for 2024 show that 83 percent of urban dwellers were using the internet that year, compared with 48 percent of rural populations. AI that can run partly offline, or with limited connectivity, can reduce dependence on constant network access and make digital/AI services more accessible to people and institutions in lower-connectivity settings. 

Resilience. Classic cloud infrastructure can struggle to meet growing demands for bandwidth, low latency, fast response and security. If an AI-enabled service supports local government operations, field diagnostics, environmental monitoring or community health work, part of the system may need to keep functioning when the network is unstable or absent. 

Trust and governance. Public institutions work with sensitive information about people, communities, infrastructure and local conditions. Local processing can reduce the amount of raw data that needs to leave a device or facility. It can also help institutions make more deliberate choices about data residency and oversight. 

Cost. Many public-interest use cases involve frequent, small interactions rather than occasional premium queries. In those settings, the economics of repeated cloud inference can become a constraint. Google's recent description of on-device LLM deployment is notable for this reason. It presents offline availability and cost efficiency as practical reasons to run language models locally for recurring tasks. 

A few examples  

In conservation, Rainforest Connection has deployed solar-powered acoustic monitoring systems in remote forest environments, generating alerts for illegal activities across 22 countries as of 2022. This points to an expectation that monitoring would work better when the first layer of sensing and analysis stays close to the environment being monitored. 

PlantVillage Nuru’s proposed  low-cost smartphone tool can help identify crop disease symptoms in the field without internet access. The value of the system depends on whether a farmer can use it at the point of need. 

In our own UNDP work, the discussion on what kind of AI and the infrastructure we are building is becoming more central. For instance, in the realm of sustainable urban development, we’re examining the use of in-orbit Edge AI on satellites. By processing environmental and urban data directly in space, these systems bypass ground-based latency and bandwidth issues entirely, delivering faster, more resilient insights to city planners. 

A policy conversation worth having 

For development organizations, the policy conversation on AI often concentrates on regulation, ethics, and high-level opportunity.  Those questions remain important, but often don’t go deep enough to infrastructure design. If intelligence is becoming more distributed, public institutions will need to decide how.They will need to ask which services should continue during outages, which kinds of data should stay local by default and which public functions can benefit from language models without inheriting the full cost and dependency structure of constant cloud inference. 

This also creates an opportunity to think differently about public digital systems. A more distributed AI architecture can support local autonomy while still benefiting from central coordination. It can allow municipalities, clinics, schools, or field teams to use AI tools under local constraints, while still connecting to shared cloud systems for training, evaluation, analytics and oversight. 

Edge AI has been here for a while. It matters now, especially in sustainable development contexts, because of how it works with systems that operate under physical, legal or economic constraints. 

Note: The core thesis, framework, and original ideas in this piece are entirely my own. I used AI tools as a collaborative partner to assist with background research and structural editing.