How do UN agencies measure impact on sustainable development in countries that lack data to begin with? Mythili Sampathkumar reports
After several rounds of negotiations and several late nights last September, over 190 countries came together at the United Nations headquarters in New York and agreed the Sustainable Development Goals (SDGs). The 17 goals are meant to be the framework for the world to address pressing social issues like poverty, climate change, and inequality, among others. It was a monumental document, one which will be in place until the next review in 2030. However, one key question remains: how do you measure the SDGs and their impact when so many developing countries lack the necessary data collection capacity?
General consensus in the halls of the UN and in the development arena is that the SDGs are a vast improvement on its predecessor negotiated and set in 2000, the Millennium Development Goals (MDGs). The SDG negotiating process was far more transparent than the MDGs and as a result, encompasses a wider range of issues.
According to many development practitioners the MDGs were a failure for a variety of reasons, chief among them was that the goals could never actually be accomplished. For instance, Goal 1 was “eradicate extreme poverty and hunger,” with only a few targets attached.
The targets were meant to serve as checkpoints on the path to achievement, but Goal 1’s target of “achieve full and productive employment and decent work for all” is problematic for several reasons. How does the world define “decent work?” How do you measure productivity? What about the unpaid work often done by women around the world managing households and farms?
The criticisms could be applied to several of the MDGs and their targets in a similar manner. This is one of the reasons the SDGs are so numerous, wide-ranging, and now contain 179 targets under the 17 goals. The first step in answering the measurement question – having more concrete, detailed targets – has been addressed adequately.
The next step is the more complicated one and took another round of negotiations in March 2016 by the representatives of UN agencies and 27 member countries of the Inter-Agency and Expert Group on Sustainable Development Indicators (IAEG). The group operates under the aegis of the UN Statistics Division (UN-SD) and decided on a list of 231 ‘indicators.’
These indicators are supposed to serve as the micro-level checklist under the 179 targets and help countries figure out where they are in terms of achieving each of the 17 goals.
The set of indicators are not without their flaws, however.
Dr Keiko Osaki-Tomita, chief of the Democratic and Social Statistics Branch of the UN-SD says that after the March meeting “not everyone was happy, but we had to move on.” She explains that the unhappiness stemmed from political agendas, ill-defined terms, and a general tendency to want to continue to add more indicators rather than take them out.
Osaki-Tomita says “at this stage we’re trying to figure out what we’re measuring” rather than the capacity or method of measuring it. Questions arose as to what countries should measure under the education goal, for example. Some countries need to focus on literacy rates while others wanted an indicator on measuring quality of education already provided.
A further problem is one of basic definitions. Osaki-Tomita explains that “some of the concepts were not defined enough,” meaning specific terms within the indicators have different definitions in different countries’ contexts. This was especially true with indicators under the environment-focused SDGs because, Osaki-Tomita says, “the Paris Agreement [on climate] change was more about political will and commitment [rather than] the technical” aspects of coming up with indicators. Adaptation to a changing climate looks vastly different in a place like the desert of Namibia than it does in the Fijian coastline.
She also comments that some of the indicators were “not fully reflective of the SDGs” because the SDGs were agreed upon based on what should be done rather than a country’s ability or willingness to do it. As Charles Kenny, a senior fellow at the Center for Global Development in Washington, DC, says, “the process to come up with indicators for the SDGs was “stick more in, rather than take them out…but there is no agreed measurement standard” so though the list is extensive, the indicators are not necessarily curated to fit specifically with each SDG.
The underlying problem to all this is that most developing countries do not have the capacity or political will to collect the necessary SDG-based data even if definitions are internationally agreed upon.
Kenny notes that “world statistics organisations need a lot of help” and adds that the “first step in fixing them is fine grain data at local and national levels.” National statistics organisations in developing countries, particularly in Africa, often have outdated numbers on issues like infant mortality, maternal health, literacy, and even crop yields because the processes for collection were most likely formed during the MDG phase, explains Kenny.
The data gaps have been traditionally filled by non-profit organisations or international aid agencies conducting regional or local surveys. However, in places like Tanzania even this is an issue as the government moves towards banning unofficial, non-government collected data. It goes hand in hand with the fact that statistics offices are “not the sexiest thing” for a politician or government official to promise to shore up. The results of it could actually hurt them politically since, as Kenny says, “the data national organisations produce can be quite scary.”
One example Kenny gives is education in India. The focus during the MDG phase was increasing the number of inputs, in this case schools. However, the reality of independent data collection shows that though India has a larger number of schools than it did 15 years ago, the quality of education as measured by literacy rate has not increased proportionally. Any politician is likely to highlight the former and sweep the latter under the rug.
Perhaps though, as Kenny points out, “the SDGs are not a good prioritisation mechanism” for national statistics agencies to follow. Osaki-Tomita also says that though the UN would like to see data collection capacity increased in developing countries and there is “huge political pressure” for those countries to do so, it may be more beneficial to have their national indicators be parallel to the SDGs but not the same.
Kenny says the “ability to evaluate at a very bespoke level” is more of a feature rather than a flaw of the SDGs. The framework needs to be agreed upon internationally, but the what and how should not necessarily have to be in order to have the greatest value.
Traditional aid agency-funded programmes may not suffice in order to achieve some of the SDGs. This is where the private sector could play a critical role in measuring the SDGs. Not just in the straightforward sense of lending statisticians and expertise to developing countries’ governments to assist in the what of data collection but also in diversifying the how.
Amit Bouri is CEO of the Global Impact Investing Network (GIIN), which seeks to connect investors who want to put their money towards projects and businesses that will have a strong social impact. He describes the connection between his work and the SDGs as: ‘the SDGs…are ambitious, important goals…and to achieve those goals we can’t leave it alone to the government and philanthropic sectors. Impact investing can be a cornerstone of the solution.” He explains that there is “tremendous opportunity” to marry untapped private capital with public sector needs on a longer term basis, without the silos of knowledge and impact produced by aid agency projects.
Of course, even within the private sector however, the underlying problem is the lack of data collection capacity in these developing markets. Couple that with the development arena’s criticism that private investors only have the financial bottom line in mind and the private sector’s role in measuring and achieving the SDGs is difficult, at best.
Bouri and the GIIN are however working on a metric for measuring the results of impact investing beyond the financial returns, which may also entice more private investors and public interest. The group surveyed 30 of their network members, a combination of investors and investees, and found that the ability to use existing data in developing country markets to improve on their operations like human resources management and accounting, presenting that data as an opportunity rather than a hindrance to foreign investment, and the quality of investment decisions because they took the time to study a country’s business environment and needs were all common measurements the survey participants shared beyond just revenue growth. All of these could be tied to indicators under the SDGs on poverty and economic growth but could help achieve other SDGs depending on the industries in which investors are entering whether that is health, green infrastructure or renewable energy.
The data on these may not be included in the 231 indicators but could be indicative of a country’s pathway to achieving an SDG. Osaki-Tomita says the status of measuring the SDGs is at a “practical standing point…but there is room for improvement.” Underlying all of it is shoring up data collection capacity, but in order to avoid the fate of the preceding MDGs, the 17 goals, 179 targets, and thus far 231 indicators cannot be so vague as to be unreachable.
As Kenny puts it, the SDGs are not legally binding but “social compliance” of countries might be necessary so the SDG indicators cannot be so rigid as to not give countries space to tailor their work within the framework. The private sector, and impact investing in particular, could be part of that tailored solution that helps measure and make progress towards the SDGs.