Can we actually agree on indicators to measure urban development?

A month before Habitat III, there’s no resolution on how to align monitoring of the New Urban Agenda and the ‘urban SDG’. In part that’s due to the complexities of using global frameworks to guide local development — and of measuring progress in cities.

Klanarong Sansuk/Shutterstock

Over the course of 2015 and 2016, the world’s governments will have agreed on a series of landmark new frameworks seeking to guide the development of cities around the world, perhaps for decades to come.

A year ago this month, national governments agreed on a first-ever cities-focused goal in the new 2030 Agenda for Sustainable Development, the framework that will guide global anti-poverty and sustainability efforts for the next 15 years. That goal, one of 17 Sustainable Development Goals (SDGs) known as the urban SDG, is the product of a World Urban Campaign involving civil society, governments and the United Nations.

Implementation of the 2030 Agenda and the SDGs formally began in January and is expected to run through 2030. However, debate has continued on how exactly national governments should be implementing the urban SDG, given that much of this development work is to be carried out by and in cities. Discussion likewise continues on how best to track implementation and measure progress — comparatively, impartially and accurately — within and across cities.

Many now hope that some of these issues can receive new attention within another multilateral process that will conclude next month. This is the Habitat III conference in Quito, where national governments will officially adopt a new 20-year vision of sustainable urbanization — the New Urban Agenda. This agenda is the platform on which the SDGs and more will be implemented in cities in the years to come.

[See: Cities turn to implementing the Sustainable Development Goals]

Yet it remains unclear, even as we go into final discussions in Quito around the New Urban Agenda, how these two frameworks and their monitoring and review processes might align. In particular many discussions are taking place on how “indicators” — metrics that aim to track progress on implementation — can be used to bring about the urban SDG’s aspiration towards inclusive, safe, resilient and sustainable cities. How these metrics fit into the New Urban Agenda thus becomes an important question.

Overall, sticky questions persist around indicators: How useful are they as a practical planning tool for improving cities? Can pushing performance-based monitoring help bring coherence, focus and a more data-driven approach to city problems? And how can globally negotiated indicators be made useful and relevant across such a wide diversity of cities?

Political, scientific tools

Currently, the urban SDG includes 10 detailed targets and 15 indicators to track progress and guide action; these latter metrics, however, have not been finalized and remain the focus of an ongoing debate. While it is important to appreciate the strengths that indicators can bring to improving urban areas, we also need to address the limits and challenges of promoting and instituting global indicators.

“How useful are they as a practical planning tool for improving cities? And how can globally negotiated indicators be made useful and relevant across such a wide diversity of cities?”

Numerous practical challenges exist around using globally decided indicators in towns and cities. Such areas are extremely diverse, after all, varying in economic, demographic and geographic conditions with a wide range of density, environments, cultures and governance structures.

Capturing the dynamics of cities is also difficult. Cities are complex, perhaps even “overcomplicated”, each with their own metabolisms, nested interactions and patterns of activities and politics, consumption and production. Impacts of city processes, both positive and negative, also are felt well beyond their physical boundaries. This makes any set of indicators seem rigid and unable to fully reflect the pulsing vitality of cities and their connectivity to broader dynamics.

[See: Inextricably interlinked: The urban SDG and the new development agenda]

Unsurprisingly, indicators themselves also can be complicated, often playing multiple roles. On the one hand, these metrics are developed for scientific purposes, to further our understanding of how cities operate. On the other hand, they also can be selected or negotiated for policy reasons, to highlight and transform certain aspects of urban development — and hence, to direct attention and resources to this end.

Indicators also can be used to monitor performance for political reasons, to help mobilize resources for interventions and even to legitimize actions. Priorities, interventions and public dialogue all can be shaped by indicator selection, by who is doing the selection and subsequent monitoring, and by how data gets interpreted. In short, indicators are political as well as scientific tools.

Another complication with indicators arises around the availability and quality of data in cities, especially within those that have high levels of informality. In part because of this informality, readily accessible urban data simply does not exist in numerous cities, especially in Africa and Asia. Further, what data does exist often privileges formal systems, marginalizing the systems people actually rely on.

In many cities, high levels of informality mean that much is from household surveys that form the basis of national statistics. For instance, a significant part of the transport system in many cities consists of quasi-formal minibuses, which tend to be ignored by planners, who do not even collect data on them. As a result, even directly measuring whether populations are within a convenient distance of public transport — as required by the urban SDG — is a challenge.

[See: Developing countries face a catastrophic lack of urban planning capacity]

In some cases, data does exist but is inaccessible because scarce data is a valuable resource — one that can be tightly controlled and commodified, thus preventing data sharing. However, creative use of new technologies — what’s commonly referred to as the “data revolution” — can help such situations, depending on who creates and controls the data and analytics. The Digital Matatus project in Nairobi, for instance, has shown how the use of cell phones and dedicated efforts by universities can create valuable open data on informal bus routes and stops.

The lack of accessible, standardized and open data for key indicators even at the national scale is reflected in the current official categorization of indicators in “tiers”. Tier 1 indicators have established methodology and data that is widely available, Tier II means that methodology is available but data is not widely available, while Tier III indicators have no data available or have no generally accepted collection or creation methodology.

[See: Are we ready to implement the SDGs?]

A group known as the Inter-Agency and Expert Group on SDG Indicators, whose mission is to develop the indicators and monitor the 2030 Agenda, is expected to meet for the fourth time in October to finalize this tier system and to “review work plans for Tier III indicators”.

Balancing localization

The framework of the urban SDG enters a complex arena in which indicator systems have already proliferated and numerous challenges exist to their utility for change. However, the strength of this new framework is that it can help channel global resources to city improvements such as improving poorer neighborhoods, urban transport, access to green public spaces, public health and resilience to shocks.

“Numerous practical challenges exist around using globally decided indicators in towns and cities. Such areas are extremely diverse, after all, varying in economic, demographic and geographic conditions with a wide range of density, environments, cultures and governance structures.”

By leveraging these goals and targets along with measurements and data to show the state of progress, diverse urban networks can argue for badly needed investments and actions at the local level. After all, indicators tied to financial resources tend to get attention.

[See: Local linchpins: Mayors commit to the SDGs]

Yet urbanists who wish to use this new tool in their advocacy efforts are likely to face a number of challenges. Three obstacles in particular stand out: the practicality and usefulness of the indicators, the persistent problem of data availability, and the uptake of the framework by local governments and coalitions. This last point, potentially the most critical, is linked to how creatively and concretely this goal can connect to local needs and the political coalitions behind them.

This “localization” challenge is daunting. While national governments have negotiated the details of the urban SDG, the goal and its targets will need to be realized at the urban/city scale. Thus, the urban SDG raises the question of the relationship and coordination between cities and national and even other sub-national governments in implementing the goal and then monitoring related progress. As cities are often in complex and even contentious relationships with higher levels of government, how the urban SDG will filter through these layers of politics is a key issue that can only be resolved politically.

The uptake of the goal and its indicators at any level will depend on how powerfully supporters of this new framework can communicate, collaborate and gain allies who find the new framework useful within their local struggles. Thus, the global networks behind the goal will need to stretch further and deeper into cities globally. Yet in so doing, they will need to do so without displacing or disrupting local movements for change that may have different frameworks and approaches.

[See: The SDGs don’t adequately spell out cities’ role in implementation]

This is the localization challenge. These indicators should not crowd out other local measures of change but rather compliment and strengthen them, especially because each indicator on its own is extremely limited.

Consider, for example, the urban SDG’s Target 2, “to provide access to safe, affordable, accessible and sustainable transport systems”. The proposed indicator for this target is the “proportion of people within convenient distance of public transport”, disaggregated by sex, age and persons with disability.

This indicator would indeed offer an idea of who has proximity to public transport. But it would not offer insight into a range of other issues, including affordability, service quality, safety or even physical access for vulnerable groups. And certainly it would have nothing to say about whether the population is then linked to employment centres, education, health care and all the opportunities the city has to offer.

Thus, the urban SDG indicators will need to be understood as guides for evaluation, not definitive measures of progress on complex issues such as access in the city. For this reason, we need to continue to refine contextually sensitive approaches and analysis when working to gauge progress on the urban SDG and the urban components of the other SDGs, especially to address the specific conditions of the poor and other vulnerable populations in varied cities.

Local relevance

Meanwhile, some cities have their own development plans, complete with local goals and targets. In these situations, localizing the SDGs becomes an issue of reconciliation and integration or of simple validation. For example, New York City’s current plan, OneNYC, has 17 vision-level goals, 27 additional goals, 55 targets and 199 related initiatives, all based on numerous consultations with citizens that occurred prior to the urban SDG. Many of these goals and targets mesh well with and even surpass the new global goal and targets.

“By leveraging these goals and targets along with measurements and data to show the state of progress, diverse urban networks can argue for badly needed investments and actions at the local level. After all, indicators tied to financial resources tend to get attention.”

In recent progress reports, New York officials unsurprisingly do not refer to the urban SDG itself, because the city government is speaking to its own political constituency rather than to a new global framework. Still, the mayor of New York has pledged his support to the SDGs, as have many other mayors. After all, the goal itself is useful in raising the profile of cities in the global arena — and hence to national governments that provide critical funding.

[See: First step for cities on the Sustainable Development Goals: Making people aware of them]

In cities more dependent on global development flows, we can imagine more instrumental adoption of the urban SDG within planning as a means to access global resources tied to the new framework. This can create openings for local advocates to raise questions about where resources are going and to what end. In the Habitat III context, this also meshes well with the New Urban Agenda’s aim of “improving transparency of data on spending and resource allocation as a tool to assess progress towards equity and spatial integration.”

Within the global urban campaign, recognition exists that for any indicator system to be adopted at a city scale, it needs to be “relevant, acceptable and practicable” for “generally overstretched and under-resourced local authorities in most parts of the world”, according to a 2015 study by Mistra Urban Futures.

Yet even in developed countries, the use of indicators at a sub-national level faces many challenges. Last year, a team led by Mistra’s David Simon tested a set of proposed indicators for the urban SDG in five cities (Bangalore, Cape Town, Gothenburg, Greater Manchester and Kisumu). They discovered that all cities struggled to access adequate data, and all proposed changes to make the urban SDG indicators more locally relevant. (Note: Mistra Urban Futures supports Citiscope.)

[See: Cities respond: Testing the urban SDG indicators]

Another 2015 study, conducted by Yale University using Delhi and Atlanta as test cases, found “data variability and lack of comparable methods hinders implementation of the urban SDG on a global scale”. However, this study also noted that, “ultimately, the SDGs are a political process and the current push to select 100 indicators to measure and monitor sustainable development arbitrarily confines the integration of science.”

There is an incompatibility between what is useful at the practical and political level of city politics, and what is useful for the scientific goal of better characterization and understanding of the complexity of cities. In the end, it will be the process of city politics and the tactics used within struggles for improvements that will determine whether any indicators or broader science get integrated into planning and urban action, and how this will occur.

Habitat III guidance?

Where does the Habitat III process stand on these considerations? The New Urban Agenda does not offer specific methods for follow-up and review, an issue that has been a key point of tension in recent months.

[See: Proposal would kick Habitat III’s main sticking point to U. N. General Assembly]

Nonetheless, the document does stress the need to track progress to “ensure its effective and timely implementation”. Furthermore, the agenda acknowledges that follow-up and review “must” (rather than “should”, as in earlier drafts) have effective linkages with the follow-up and review of the 2030 Agenda “to ensure coordination and coherence in their implementation”.

The New Urban Agenda does not discuss indicators, leaving the use, type and value of these metrics open to interpretation and further discussion. However, it does aim to “strengthen the data and statistical capacities at national, sub-national, and local levels to effectively monitor progress.” This will be critical for the success of monitoring the urban SDG and New Urban Agenda progress, as well as for accountability more generally.

The New Urban Agenda should thus help address but not determine the processes and steps needed to achieve the urban SDG and the more radical goal of the “right to the city” for all. Indeed, the urban SDG and the New Urban Agenda reflect and support the critical role of cities, along with civil society and national governments, in crafting local strategies and actions for achieving inclusive, sustainable and just urban development.

[See: Cities shut out of U. N.’s first SDGs review, advocates say]

In the end, the troubles with urban indicators are an intrinsic part of the realignment of power that places cities in a more elevated, strategic position in the struggle for global change. Continued conversation about indicators will no doubt occur in Quito and elsewhere; ideally, it will catalyse critical rethinking of priorities and creative ways to collect new data and see the city, its problems and opportunities more clearly.

We do not need to agree and resolve all the problems with indicators — an impossible task — for concrete progress to be made on pressing urban problems. Creative and tactical use of indicators, however imperfect, along with other tools and approaches might help make a real difference in the struggle to hold governments and civil society accountable for promised and urgently needed improvements in urban life.

Stay up to date on all Habitat III news! Sign up here for Citiscope’s weekly newsletter. Citiscope is a member of the Habitat III Journalism Project; read more here.

Subscribe to updates on the U. N.’s global cities conference

Back to top

More from Citiscope

Latest CitiSignal

Comments Policy

Citiscope is a place for the world’s urban leaders — mayors, councils, business, civic, neighborhood and independent observers — to exchange ideas and learn from each other. Comments are most welcome. Participants must first sign in to Disqus. (Not registered? It’s easy: Sign up here or connect with a social media account.) We ask that you use your real first and last names and say what city you’re from. Comments that do not follow Citiscope’s comments policy will be removed.

Jacqueline M. Klopp

Jacqueline Klopp is an associate research scholar at the Center for Sustainable Urban Development at the Earth Institute in New York.

Danielle Petretta

Danielle Petretta is a PhD student affiliate at the Center for Sustainable Urban Development at the Earth Institute in New York.