The Principled Agent

Thoughts on development economics and impact measurement

Acumen and the impact learning agenda

with one comment

While progress has been made with outcome reporting standards, e.g., IRIS, much work remains to establish impact measurement practices that will allow Acumen staff to credibly assess the effectiveness of different products and services. Once developed, these practices will support both greater internal learning and external understanding of Acumen’s social return on investment.

Last February, ANDE’s Randall Kempner wrote a blog post very much related to this subject that likely caused some pulses to quicken:

Organizations join ANDE because they share the common mission of increasing the prosperity of poor people in emerging markets, believing that supporting entrepreneurship and market-based solutions is part of a sustainable approach to poverty reduction. … But that’s a hypothesis; there isn’t sufficient evidence to back the assertion that small and growing businesses are a key part of poverty reduction. That’s why I was scared… I was standing there thinking “I don’t want to be sitting in a room like this 10 years from now, hearing how my industry, my movement, my life’s work had ‘an average impact on poverty of zero.’”

ANDE is collaborating with researchers to improve the robustness of the existing evidence base on the impact of small and growing businesses on poverty. But what does this mean for Acumen? Acumen now finds itself in the same position as impact-driven MFIs in the 90s. The additional social value of the products and services supplied by Acumen portfolio companies accords the organization a competitive advantage, but the less rigorous and robust the impact measurement, the more difficult it is to communicate this added value. Indeed without more robust impact measurement, the impact component of impact investing will be treated as a commodity. The story of microfinance offers a clear example, as institutions with marginal impact were perceived to be the same as those with significant impact. The impact-driven MFIs benefited less than they should have in the short term, while then sharing fully in the bust prompted by their less noble counterparts.

Outside of external communication, Acumen also has much to gain from impact measurement with regards to refining portfolio strategy and service offerings.

As Kempner noted in his piece, impact investing is a nascent field and there is not yet sufficient evidence to validate its theory of change. Given the heterogeneity of the impact investing field, there will never be a study that will show that it does or does not reduce poverty. Every country context, every industry, every combination of capital, assistance, and facilitation will ultimately result in a “new” program with a different set of causal mechanisms. There will likely be some combinations that result in successful enterprises with limited impact, and others that produce less successful enterprises with more impact. Acumen already rightly understands that the point of entry in an enterprise’s lifecycle is one such crucial factor. The Head of Impact will need to help systematize and deepen this learning to better understand and validate the mechanisms by which Acumen is achieving impact. While monitoring and evaluating alone won’t lead to dramatic changes overnight, in time, the work produced by the Head of Impact should undoubtedly shape Acumen’s portfolio strategy.

So what does this mean in practical terms? Acumen’s already worked to help establish a common IRIS lexicon. The next step is increasing the robustness of impact measurement through embracing the counterfactual (more here) and developing strong internal controls. This doesn’t mean turning Acumen into nothing more than a research lab for RCT advocates. It does mean taking concrete, transparent steps toward estimating the incremental impact catalyzed by Acumen’s work, by creatively utilizing the myriad techniques available that require less in additional resources than they do a change in mindset. And yes, it means engaging with the empirical researchers in order to ensure the research agenda of initiatives like IPA’s SME initiative reflect the research interests of Acumen. It means advocating for small steps that will result in stronger learning and understanding of what works and developing processes and tools to help make this vision a reality.

This undertaking will require an academic’s sense of rigorous impact estimation, a cutting-edge NGO’s creativity in devising cost-effective methods, a market researcher’s sense of turning data into useful managerial information, and a controller’s insistence on sound internal controls to maintain the needed integrity and objectivity of the results. The high-impact MFIs of the 90s lacked this combination of skills and saw their stock and reputation vacillate and ultimately fall with that of low-impact MFIs. Acumen is well-positioned to avoid this fate. By developing the infrastructure and processes needed to ensure it is successfully managing to impact, Acumen can at the same time set the bar for other impact investors, supporting the development of a true market for impact and catalyzing the philanthropic support needed for the impact investing field’s Blueprint to Scale.

Relevant posts:


Written by Chris Prottas

April 1, 2012 at 2:42 am

Posted in measurement

One Response

Subscribe to comments with RSS.

  1. Just wanted to note that Kevin Starr made a similar point about paying attention to the counterfactual in evaluating impact investments in this SSIR post:

    Alexander Berger

    August 1, 2012 at 3:44 pm

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: