President William Davis is focusing on investors—mostly big institutions—that can bring their impact investing portfolios onto the platform with them. At launch, Davis says, GATE Impact will list available impact investment deals of between $500 million and $1 billion. By comparison, the social enterprises on Asia IIX’s Impact Partners platform have sought to raise a total of about $70 million. [Source]

But do impact investments need “hot money”? It’s hard to know what the trading behavior would be like on such an exchange, but there is an advantage to being a private company, the ability to look long- rather than short-term chief among them.

I do think a secondary market will be very important to the industry, especially given the fear of many that there will be limited opportunities for exits. At the same time, I wonder if what the industry needs is an active private equity secondary market, with secondary buyers purchasing stakes in existing investments or funds themselves, allowing the initial investors to achieve at least partial exits.

Given the concern about exits, I would think that a secondary buyer could play a catalytic role in the industry, increasing the confidence of concerned potential investors and providing exits for initial investors that will allow them to make additional investments in the space. The role of the secondary buyer is indeed to provide an exit for the initial investor before the investment itself achieves an exit (e.g., M&A, IPO, etc.), and given the patience required for “patient capital” impact investing, the off ramp provided by secondary buyers could prove crucial to getting mainstream institutional investors on board. Unlike the proposed stock exchange, it won’t expose the businesses themselves to hot money.

In addition to a secondary private market, capital could be channeled through a trading exchange that listed social investment funds rather than the investments themselves. There are legal concerns given the non-profit status of many social investors, but it’s an idea worth considering.

The $400 billion to $1 trillion question is how to translate investor interest into active investments. An active secondary market and a platform for funding the funds may help (responsibly) tap those resources.

If you have any thoughts or comments, please send me an email at: cdp283 at nyu dot edu.

Previous Posts on the challenge of Social Performance Measurement:

• Cost-effective Impact Assessments for the Impact Investor
• Separating the Gold from Pyrite: Analytical Rigor and the Potential of Impact Investing
• Impact Investing in Rigorous Analysis

Impact investors can better estimate their impact on growing businesses and, in turn, the impact of supported businesses on communities without a significant increase in evaluation cost. While impact investors rightly regard randomized controlled trials (RCTs) as inappropriate for day-to-day performance measurement, impact investors can use low-cost alternatives (specifically, control groups and Best Alternative Pro Formas) to construct counterfactuals in order to estimate what would have occured in the absence of their investment. While these methods will not meet the standard of RCTs, they can offer a significant and meaningful improvement in the measurement and management of social performance.

To evaluate impact, the impact investor must make “a comparison between what actually happened and what would have happened in the absence of the intervention¹.” RCTs represent the most rigorous method available to approximate the counterfactual, which in any case can only be estimated. Presently, most impact investors do not utilize a counterfactual, and this omission introduces both bias and noise into social performance measurement.

This document will address the role of RCTs, the necessity of a counterfactual estimate, and practical alternatives for the impact investor to construct a realistic counterfactual and thereby understand and maximize impact. These principles apply both to estimating the impact of an organization on a community, and the impact of an investment on an organization, the latter of which is the subject of this analysis.

Randomized Controlled Trials and Impact Investing

In a recent presentation at DFID, Chris Blattman made two points on the proper role of RCTs particularly relevant to this topic: “Don’t evaluate [with RCTs] things that you cannot generalize,” and, “Test ideas not programs².”

The impact investing community should work with evaluators to establish an RCT agenda that will test industry assumptions and mechanisms (i.e., “ideas”) that underpin the industry at large. However, while RCTs can play a crucial role in validating (or calling into question) the theoretical underpinnings of the impact investing model, they are not appropriate for impact investors to evaluate the performance of individual investments (i.e., “programs”) in their portfolio.

Motivation for a Counterfactual: Why Changes in Outcomes are Not Enough

In order to understand and then maximize the impact – value-added – of impact investing, the investor needs to assess the performance of its investments based on the incremental performance improvement that is due to their support. There are a number of reasons why a change in outcome indicators will not reliably provide an (even roughly) accurate picture of the investor’s impact on the firm or community. Three of these will be addressed below:

a) Maturation: Organizations may demonstrate improvements over time that are not due to any intervention, but due to the passage of time which allows for growth and thereby improved outcomes.

Implication: If you assume that changes in your investment’s outcome indicators are attributable to your investment, your perception of your impact will be greater than the reality.

b) Selection: Impact investors are likely to invest in organizations that show particular promise, some aspects of which are difficult to capture in any metric (such as, management quality, industry connections).

Implications: Compounding the maturation bias, selecting investments based on positive characteristics makes it more likely that your investments would have matured without your investment. Therefore, if you assume that changes in your investment’s outcome indicators are attributable to your investment, you will overestimate your impact.

If you choose as a counterfactual the change in outcomes of another organization with similar observable characteristics (though less impressive in some way that preempted investment) your perception of impact will be greater than the reality, as your investment would likely do better than the comparator with or without your support.

c) History: External events, such as changes in commodity prices, may positively or negatively affect how well your investment does.

Implications: If you assume that changes in your investment’s outcome indicators are attributable to your investment, you may over or underestimate your impact, depending on whether external conditions have become more or less favorable for your investment.

If you choose as a counterfactual the change in outcomes of another organization that was not affected by the same external events, you may again over or underestimate your impact, depending on which organization’s external events were relatively more favorable.

There are a number of qualitative and quantitative methods for constructing counterfactuals, which – while not reaching the RCT’s standard of internal validity – can improve impact investors’ understanding of an investment’s impact. This document will focus on quantitative methods with particular attention paid to addressing the aforementioned threats to the internal validity of the counterfactual estimate.

Estimating the Counterfactual: Alternatives to RCTs

In theory, Control Groups can address all three biases, albeit imperfectly, depending on how well the comparators match your investments.

The impact investor should have strong contacts in the areas and industries in which it operates and have relationships with organizations that it does not assist. The impact investor could “match” other firms to their portfolio companies, and interpret the performance of the comparison organization as the counterfactual. At the same time, if the portfolio companies differ systematically from these comparators (e.g., management team quality) then these firms will not provide appropriate counterfactuals. In addition, depending on the industry, it may prove quite difficult to find an appropriate comparator.

Weaknesses: The greater the impact investor’s focus on finding high-potential individuals or companies, the more difficult it will be to find adequate comparators. In addition, it may be difficult to acquire performance data from privately owned comparator companies, especially as they may be competitors of your investments.

Best Alternative Pro Forma Forecasts are Pro Formas that project future performance of the organization had they not received your investment. For example, if you were proposing an equity investment of $200,000, the best alternative might be financing growth with a series of loans from a local bank or microfinance institution. The Best Alternative Pro Formas would provide an estimate of the counterfactual at the time of the investment.

Pro Formas are a natural component of the due diligence process. With minimal incremental cost, an additional Pro Forma could be constructed for each prospective investment that reflects the enterprise’s forecast without the impact investor’s involvement. In some cases, this might assume additional capital from a local source. In all cases, it would amount to the best estimate of future performance based on the resources available outside the impact investor.

In addition, the impact investor can compare the original Pro Forma (with investment) with the actual performance of the investment. Investments that under- or over-perform their forecasts may have similarly under- or over performed their Best Alternative Pro Formas had they not received an investment. In this case, the impact investor could adjust it’s counterfactual estimate up or down by the amount that the investment under- or over-performed its Pro Forma with investment. 

The forecasts can and should incorporate the flow of economic and social benefits to different stakeholders. The Asian Development Bank, among others, has produced a number of guidelines on this subject³.

Weaknesses: Foresight is limited: the enterprises make forecasts with imperfect information at a point in time, after which company and market conditions change. It’s worth differentiating the error introduced through:

a) Unforeseen external conditions with a predictable impact on company growth
b) Unforeseen external conditions with an unpredictable impact on company growth

To the extent the impact on growth is predictable, the impact investor can create a Pro Forma that links key external condition variables to company growth and thereby improve the accuracy of the estimate (by updating the values in the future when the values become known).

There are two additional weaknesses alluded to above. First, forecasts are often optimistic. To some extent, organizations can control for overly optimistic forecasts by accounting for the extent to which the organization under-performed its forecast with investment.

Second, a forecast is more subjective than a control group comparison. It introduces a greater possibility for gaming, and requires internal controls.

Conclusions

• Changes in outcome indicators cannot be attributed solely to your investment due to maturation, selection, and history biases.
• Your investments are special: it will be difficult to find comparator firms because these comparators did not receive your support for a reason –likely a lower growth trajectory or less appealing intangibles.
• Forecasts are difficult to do well: it will take thoughtful financial modeling to construct forecasts with and without investment (prior to investment) that will provide a realistic and objective estimate of the counterfactual.

Recommendations

• Pilot the construction of Best Alternative Pro Formas and discuss utility of incorporating key external condition variables (to be updated in the future) in the financial model and adjusting the Best Alternative Pro Forma based on the firm’s under- or over-performance of its Pro Forma with investment.
• Explore the viability of matching investments with comparators.
• Discuss challenges to ensuring the objectivity of the impact assessments through internal controls or external contracting.

If you have any questions, comments, or feedback, you can reach me at cdp283 at nyu dot edu.

References

1.  White, H. (2006) “Impact Evaluation: The Experience of the Independent Evaluation Group of the World Bank,”  World Bank, Washington, D.C.
2. Blattman, C. (2011) “Impact Evaluation 3.0?” DFID Presentation. http://www.chrisblattman.com/documents/policy/2011.ImpactEvaluation3.DFID_talk.pdf
3. ADB. (1993) “Guidelines for the Economic Analysis of Projects.” http://beta.adb.org/documents/guidelines-economic-analysis-projects

Acumen Fund’s Sasha Dichter recently responded to Dean Karlan’s critique of the lack of rigorous analysis of social investments by differentiating between the social investment and the “purely philanthropic intervention”:

No matter what scale a pure philanthropic intervention reaches, the total marginal cost of delivering the nth “thing” (any intervention) is always positive, so you’re in the business of figuring out how the impact relates to that cost and how the impact relates to other similar interventions.  Not so if you find the “next cellphone” – except it’s not a cellphone, it’s safe drinking water or a bio-mass powered light on a mini-grid or a safe and affordable place for a mother to give birth.

Sasha seems to be differentiating between the two because the philanthropic venture’s product or service always requires some subsidy, while the social enterprise does not. Similar to the cell phone manufacturer, the social enterprise will (at least partly) find its profit and social incentives aligned, with healthy growth and adoption signaling that it’s having a social impact without the need for a rigorous impact evaluation. Continuing with the cell phone example, Sasha notes, “Since we’re not using up grant money to pay for it, it stops being the purview of development economists to fret about this.”

From my perspective, this last point is crucial. If you are providing investors with market-rate returns without direct or indirect subsidy, I completely agree that your burden of proof is fundamentally different than that of a philanthropist.

If you are providing submarket risk-adjusted rates of returns, however, or accepting subsidies of one type or another, then you are leaving the world of cell phones and entering the world of bed nets. I believe that the majority of impact investors fall in this category, with the subsidy coming in a number of different forms.

Now if JP Morgan is correct, impact investing has the potential to be a $1 trillion market, and if just one-percent of that total is a subsidy in one form or another, we’re talking about a potential $10 billion philanthropic market. 

Given this outsized potential, the fact that a large number of purely philanthropic ventures with clear-cut social value propositions and a 100-percent focus on social returns have shown  virtually no impact on the lives of the poor should give pause to impact investors who are now laying the foundation for their own high-potential philanthropic market.

In addition, the microcredit industry demonstrates the damage that can be done to an industry with financial and social ambitions that allows all the players to sell a significant social impact not based in rigorous analysis but solely in the supposed benefit of providing a service “that make[s] a material positive impact on the lives of poor people.”

Microcredit surely has benefitted the poor, but would the poor have been better off if those hundreds of millions of dollars of subsidy had been put to microsavings? Or would it have been better spent subsidizing capital equipment loans for producer associations? Or, perhaps most importantly, could it have been better spent on pro-poor microcredit itself if its allocation and social investment criteria were informed by more rigorous social performance measurement before the IPOs?

Like microcredit, I’m not worried about whether impact investing will have a benefit greater than zero. I am worried that the social performance data collected will not allow impact investors to separate the gold from the pyrite, which will reduce the effectiveness of the budding industry.

Setting aside the methodological question of how to assess impact, I think the most important take away from the RCT movement is that there’s a lot of good looking pyrite.

Within the impact investing sphere, there are a huge variety of industries (e.g., irrigation systems, medical care), business models, and markets, all of which will affect the ultimate impact of a financially-successful social enterprise. That is, impact will likely vary across industry, market type, and business model (among other variables), and this presents a number of questions:

Based on the emerging impact analysis and reporting best practices…

• Are impact investors able to assess the relative social impact of these different interventions and to shift their investment focus and criteria to ensure that they are maximizing their social impact (within the constraints of their financial goals and requirements)?
• Can an impact investor who is very adept at separating the impact gold from the impact pyrite distinguish herself from the impact investor who can’t?
• Can impact investors shift the basis of “competition” for social investment from the investor’s social value proposition (e.g., financial services for the poor, rural business development) to social impact? In other words, are we seeing the birth of an impact investing industry or just the maturation of the socially-responsible investing model into a more proactive “good intentions” investing model?

The social entrepreneurship world is in a weird spot, to be honest with you. It’s a world full of rhetoric about impact investing, yet I have very rarely seen an investor actually take that seriously. When you look at the actual analysis it lacks rigor.

Ouch. Some readers will surely dismiss Karlan as a ‘randomista’, but his critique is fair, and one that I have made before.

Take microcredit. What they do is they track how many borrowers, what their repayment was, and whether the businesses grew over time. That is not telling you your impact at all because you don’t know what would have happened if you didn’t lend.

…

They push toward collecting data that doesn’t necessarily answer the question. Now you’ve burdened your organization with data collection so you can feel rigorous.

He is exactly right. The emerging industry standard, the Global Impact Investing Network’s “Impact Reporting and Investment Standards,” presents a case study. While the standards represent a dramatic step forward for the standardization of data reporting, these standards require organizations to invest a lot of time answering a lot of questions, e.g. volunteer hours worked by employees, square feet of community facilities financed. The goal is clearly to be comprehensive, with organizations and investors able to evaluate their respective areas of interest. However, while the breadth of the data collected is impressive, “[it] is not telling you your impact at all because you don’t know what would have happened if you didn’t lend.”

Like with Karlan’s microcredit example, these standards only permit a “monitoring exercise.” The standards opt to monitor a little bit of everything, rather than concentrating attention and resources to try to estimate the social impact of the organization’s core social value proposition (e.g., increased income for low-income employees, reduced water pollution).

More broadly, none of the emerging ‘best practices’ for impact reporting in impact investing even attempt to estimate what would have happened without the intervention. Because of this, the use of the term of ‘impact’ is incorrect and misleading. That said, I fear that this is partially due to the failure of the rigorous evaluation camp to sign off on non-randomized methods of estimating the counterfactual, that is, what would happen in that investee firm, community, etc. without the intervention. For sound practical reasons, RCTs will never be the impact investing standard for measuring impact, as Karlan himself accepts. So what now?

The challenge is to find a middle ground that allows for the best possible estimate of social impact given the resource constraints of impact investors. First, impact investors must believe there’s value in investing a bit more time and energy estimating social impact (perhaps reallocating some of that time and treasure spent monitoring volunteer hours worked, for example). After all, if you’re investing in a company with the goal of creating jobs or cleaning the environment, wouldn’t you rather have a good estimate of your investee’s impact on those core goals rather than a lot of monitoring data on things that wouldn’t otherwise justify your investment? Sure, you won’t have an RCT, but that doesn’t mean you can’t estimate the counterfactual in order to greatly improve your understanding of your impact and inform your allocation of resources across competing investment opportunities.

Since this isn’t new ground for this blog, I thought I’d add a few ideas on how an impact investor might estimate the impact of their investment in a firm. The principles are similar for estimating the impact of a single firm on a community, employees, pollution, etc.

a)      Create a “living” forecast of the firm’s performance with best alternative financing at baseline and plug in actual future economic variables (e.g., commodity prices) to update forecast estimates over time

b)      Performance of businesses that operate in the same industry and of a similar size

c)      Performance of similar businesses that apply for financing but do not receive it

d)      Triangulation of counterfactual estimation based on multiple estimation methods

Each option has its own drawbacks, and I only briefly note these ideas as examples of relatively cheap methods of estimating counterfactuals. (Perhaps an RCT could shed light on the relative accuracy of these different estimation techniques?)

Back to the big picture, it’s difficult to create a standard by which investors or firms would estimate the counterfactual, given the different types of impact sought by investors. However, it’s not impossible, and it certainly is possible to create standards for a) employing a counterfactual, b) reporting the process of estimating the counterfactual, c) reporting how potential positive bias was mitigated, d) reporting on the justification for this method of counterfactual estimation, to include the marginal cost of a more rigorous counterfactual estimate.

These are just some ideas, but given impact investing’s great potential for social impact, it would be a shame to ignore this opportunity to better understand this impact and to guide investors to the opportunities for greatest social impact in the future.

The authors candidly admit that their analysis only assesses performance on these “best practices”, rather than aid effectiveness. When reading the report, I was struck by the degree to which academia and the general public have been made to be like a drunk looking for his keys:

A policeman sees a drunk man searching for something under a streetlight and asks what the drunk has lost. He says he lost his keys and they both look under the streetlight together. After a few minutes the policeman asks if he is sure he lost them here, and the drunk replies, no, that he lost them in the park. The policeman asks whey he is searching here, and the drunk replies, “this is where the light is.”

While we know it’s outcomes that matter, not overhead, for example, we look to overhead, just as the drunk looks to the curb by the streetlight. It may be better than nothing, or it may actually be detrimental (as many, including myself, would argue the fixation on overhead is), but it’s all we’ve got!

Disclaimer: I consider the firms mentioned in this report to be pioneers in the field – some of the best in class – which is precisely why I chose them for this post.

While their attempts should be applauded, I argue that none of the guidelines or reports (e.g., Oxfam/Unilever, Standard Chartered, SEAF) that I have come across actually measure impact, or business contribution to society. Further, if the existing indicators are used to drive decision-making, they will systematically lead businesses away from opportunities where their impact is greatest and towards activities that would have been successful without their influence or even existence.

The bottom line is that whether an organization is non- or for-profit, it’s contribution to society (or “impact”) is incremental. That is, it is the difference between what occurred with the organization (the reality) and what would have occurred without the organization (the counterfactual). Measuring incremental impact isn’t easy, but an estimate, at the very least, is essential.

If a non-profit provided an advisement session to the 1,000 best students from a low-income area, could it accurately state that it’s economic and social impact encompassed the students’ future wages and college admission? Clearly not. This is an acknowledged bad (albeit common) practice in the non-profit world, yet it is simultaneously being promoted as best practice in the CSR and impact investing communities.

Case 1)  The socially-conscious business: Standard Chartered

In a report titled “The Social and Economic Impact [emphasis added] of Standard Chartered in Indonesia”, the authors claim that “added together, the direct, indirect and induced impacts of Standard Chartered’s operations and onshore financing in Indonesia amounted to USD 4,504 million of value-added in 2009, or about 0.8% of Indonesia’s GDP. … [With a] total impact of 1,029,000 jobs.”

The authors do note:

…the results derived in this way give a prudent picture of the economic activity [emphasis added] that is associated with Standard Chartered in Indonesia. … Most of the jobs and value-added created with the help of Standard Chartered Indonesia and Permata Bank would most likely still exist if the banks were not in the country [emphasis added], as their role as financiers, purchasers and employers might be filled by other financial institutions.”

Standard Chartered conflated their social and economic impact with economic activity.

This isn’t to knock Standard Chartered. There are certainly incremental social and economic benefits generated by their operational activities in addition to the financial value they capture and distribute to equity holders and the government via taxes.  Further, if you read the report to the end, you will find non-operational activities, e.g., spending $4 M to help with natural disaster recovery, that clearly have a social impact, but receive minimal attention or quantitative analysis in the report.

Case 2) The social investor: Small Enterprise Assistance Funds (SEAF)

If the history of results-based management has taught us anything, it’s that you get what you measure. Current non-incremental measures add to the existing financial incentive to fund businesses that would have succeeded anyway.

For example, “[Small Enterprise Assistance Funds] targets entrepreneurs active in industry sectors with price/earnings  or other acquisition multiples that SEAF believes are likely to grow substantially given the stage of development of a given economy.”

Yet SEAF then measures it success by the growth of these firms! While trying to measure its impact, SEAF is really measuring its ability to select firms that will be successful. To return to the non-profit analogy, I am not going to judge the impact of an NGO that targets high-performing students by the future performance of those high-performing students.

Furthermore, SEAF’s impact measures create an incentive for mission drift.  If our hypothetical NGO measured their success by the future performance of their students, they’re going to move upstream out of the market where they can make the most incremental impact in student performance, and toward the market where students are more likely to be successful anyway.

I’m not cynical about CSR or impact investing. I think that in practice SEAF, for example, does attempt to serve businesses that would not be able to achieve scale through other financing vehicles. The problem is that this isn’t captured in the current measures! If we want to empower individuals to manage or invest for social performance, it’s essential to systematically estimate the counterfactual. Otherwise, both CSR and impact investing risk the same crisis of faith that microfinance has experienced, when the crowds stop accepting claims of social impact with no supporting evidence.

With the majority of my work experience in the private sector, I went to grad school to make a professional transition into the development space. I’ll sidestep the question of the relative merits of hard vs. soft skills, but my emphasis was on hard skills. As a Master in Public Administration candidate at NYU Wagner, I was most excited at the flexibility I would get in choosing my coursework, and most concerned about the degree’s competitiveness in a field where I might be competing against MBAs. While I can’t comment on the latter, I now feel like I can provide some preliminary thoughts on the former.

1) It does help get an internship/job
It’s my personal experience that the “.edu” helps you get call-backs, especially if you’re making a professional transition. It signals committment. If you’re a prospective student, what’s the local market for internships relevant to your career goals? I’ve found it hugely beneficial to work while in school. Do free labor if you can’t get a paid position. Ask questions. Learn as much as you can about what they do and how it differs from what you’re learning. If you’re an M&E person, get their data and offer to analyze it as you work through your advanced quant courses. If you’re looking for international development job advice, talk to Alanna Shaikh. If you’re looking for resume or cover letter help, talk to your Career Services folks!

2) Your hard-skill mileage will vary by your program and your preparedness
Like most other programs, NYU has a number of core courses, e.g., statistics, microeconomics, public policy. If you have taken these classes before, waive them. If you can self-study and pass a waiver exam, do so. You’re paying a lot of money for your credits, make the most of them. If you’re a prospective student, find out what the requirements are and what you can waive. Talk to students about how easy or difficult it is. You can make your own curriculum, but your mileage will vary by program and your preparedness. (NYU Wagner has done well by me!)

3) There are resources outside your program to sharpen your skills
Given my interest in statistics and finance, I’ve taken a number of courses at Steinhardt’s Center for the Promotion of Research Involving Innovative Statistical Methodology (PRIISM) and the Stern business school. Prospective students should know the school’s policy and the student reality in taking courses in other programs. If you’re interested in international development, a course in causal inference, for example, might serve you well, and you may find the coursework in another program.

4) It’s a great opportunity to become a statistical software ninja
If you’re interested in M&E, grad school is a great opportunity to learn the different software (e.g., STATA, SPSS, SAS, R): know their limitations, find professors who can serve as resources beyond school, and find other resources to support your post-grad learning (e.g., www.ats.ucla.edu/). If you’re a prospective student, it’s worth checking out syllabi to see what software you’ll be exposed to in class. You can learn these programs on your own, but you’ll find free resources (including software) at school and useful academic support.

5) With or without grad school, become an Excel ninja
The coursework is there to develop Excel expertise, and learn how to validate data, calculate what-if scenarios, goal seek, etc. You’ll probably need to work on your own to learn Visual Basic, for example, but the more you can pack into your time, the better. While my experience is limited, everywhere I’ve worked Excel ninjas have been highly valued. That said, grad school clearly isn’t necessary for this.

6) With or without grad school, do informational interviews
For prospective students, I do think that having an “.edu” e-mail address increases responsiveness, but it’s not necessary. Yes, it’s painful to ask, but do it! Prepare for them. Often the interviewee will happily share information they might not otherwise broadcast about your industry that will deepen your understanding in ways that no coursework nor social enterprise conference nor even internship can. Sadly, it took grad school for me to learn the value of the informational interview.

7) With or without grad school, read the stocks, manage the flows
With all this emphasis on skills, where’s the content? As Dave Algoso suggests, if you’re a prospective or current student planning to learn your field just through course readings, you’re doing it wrong. Identify the relevant stocks of knowledge and work through them. Find a way to filter the large flow of information in a way that you can extract meaningful information without getting too caught up in the daily back-and-forth. If you’re like me, you’re taking method courses rather than content courses, which puts the onus on you to do the content on your own. Twitter, Google Reader, and journals are your best e-friends. Your job and informational interviews should ground this internet intake in reality. Warning: it’s easy to drown in the flows.

This advice won’t be relevant to everyone. (Hopefully useful to some.) However, I think it’s well worth the time of current graduate students (and, preferably, the recently graduated) interested in the field to make an effort to share their experience, so that folks considering making a large investment in their education have an understanding of what value students found in their grad education and what they did not. As far are current resources go, Chris Blattman has some great advice on the MPA/MPA-ID/PhD decision.

I’m afraid that the catalytic class may already be studied under a different name [ed note: capitalists!]. In fact, this guy might argue that this research might be coming some 144 years after the definitive study of this group. I’ve been talking for a while about how they’re a crucial but neglected part of the development process.

This is a curious riposte. Yes, the emergent bourgeoise are capitalists, just as they are humans, and doubtlessly countless other things, but what makes them (potentially) a special ally of the poor is that they economically support a new set of public interests to compete with those of the existing elite.

To the extent this new class’ power benefits from public goods that benefit the poor, the emergent bourgeoisie may indeed catalyze positive change that would not otherwise occur.

Mr Dissanayake might object that entrenched economic elites and their progeny don’t qualify as capitalists, and therefore the catalytic class is no different than capitalist.

Yet I’d object once more. The catalytic class are a special subset of bourgeoisie: bourgeoisie that happen to have interests that align with the poor. It is a happy accident that they benefit from the same public goods. It is a happy accident that their source of wealth increases demand for labor from the poor. To call them simply capitalists is to lump them with the bourgeoisie that make a fine living virtually detached from the poor. It is not even sufficient that the emergent bourgeoisie are emergent to guarantee that their interests align with the poor in any meaningful way.

Still it may be that we acknowledge the catalyst class as distinct from capitalists,  or even the emergent bourgeoisie, but find that we can’t target assistance to the catalytic class. We might not know who they are ahead of time. Or we might not be able to help them outside of more general support for capitalism, in which case, Mr Dissanayake’s conflation of the catalytic and capitalist terms is operationally correct.

Indeed the challenge of identifying the catalytic class is complicated by the fact that there is no single catalytic type of enterprise or group of people. Catalysts aren’t universal. There is no law to be found that dictates a new agricultural producer  association always creates catalytic change. It would be folly to think that a study of catalytic classes would produce a formula for industrial policy.

At the same time, there’s no reason to suppose that catalysts are entirely idiosyncratic. It’s not naivete to suggest that certain types of enterprises might increase demand for the labor of the poor more than others, or require well-maintained roads that connect rural areas to urban hubs, or create a large amount of wealth in the hands of a few new tycoons who might see profit in the liberalization of the airwaves and have the independent power to demand it. These are different types of catalytic change, and they are more or less likely to emanate from enterprises with different dependencies on capital, land, and labor, and which produce different distributions of wealth.

While we should not suppose that there is a magic catalyst class or industry, we should be eager to study how entrepreneur/enterprise characteristics make them more or less likely to create different kinds of catalytic change in different environments. As international organizations are already subsidizing capitalism in developing economies, they might as well incorporate an estimate of the positive externalities for the poor, rather than simply supposing that financial results in non-functioning markets are adequate signals of societal value.

Amidst this flurry of experimentation there is a crucial component missing. Social impact data. Just as the microfinance industry is belatedly working to create standards for social performance reporting, SME services remain a black box. The recent SME initiative at Innovations for Poverty Action is a step in the right direction, but it will take more than a select number of RCTs to learn how social dollars can best be allocated across industries, products, and services. Standards for reporting changes in income, jobs, and net assets of SMEs, as well as the cost of service, would allow donors and social investors to compare estimated social returns as they do financial. If an organization doesn’t invest in estimating its social impact (and its economic cost-effectiveness) with the same seriousness it brings to financial analysis and accounting, it’s not taking a business-like approach to social venture, it’s just doing business. Doing business in underserved markets is itself good, but it is not necessarily the best bang for your philanthropic buck.

In addition, the lack of quality data reporting preempts comparison of the relative social impact of financing, technical assistance, integration into corporate supply chains, etc. Is the social impact of subsidizing a granary and introducing a corporate procurer to a local trade association consistently high? How does the social impact of providing a dollar of working capital to a mid-sized firm compare to providing that same dollar to smallholder farmers in the form of equipment? Surely, there would be a host of analytical challenges (namely estimating the counterfactual) were this data consistently reported, but at this point it is even difficult to ascertain how the approaches differ in terms of orders of magnitude. Unfortunately, there’s little incentive for pioneers to poke their head out and say that they spend $100,000 per job-paying-$1,000 created.

The Global Impact Investing Network has launched the Impact Reporting and Investment Standards, which have been adopted by a number of organizations. At the same time, the standards fail to address the question of impact. Take their fictitious example of Free-to-bee honey, a company that “purchases honey from approximately 6,000 smallholder farmers and operates a packaging facility in rural Eastern China where it produces organic honey for export. F2B purchases exclusively from smallholder farmers and provides technical assistance and group based training to suppliers in order to improve farmers’ honey yields.”

What was the farmers’ income before and after F2B? What was the change in yield? You would not know from IRIS. But the answers to these question are important to understanding the extent to which this venture is social.

Furthermore, if F2B honey was a development initiative, the answers to those questions would be essential to understanding if it should exist.

To conclude, I am confident that for-profit firms will find a way to cost-effectively serve SMEs, just as Compartamos and SKS did so well in microfinance. There most certainly will be economic benefits to the consumers in these underserved markets and likely second-order effects which will never be captured but will be positive nonetheless. At the same time, if mission-driven organizations fail to collect and disseminate cost and impact data from their efforts to catalyze enterprises that create jobs and increase income for the poor there will be a gross misallocation of scarce development dollars to SME services that have little impact on people in need.

As I push this post into the Twittersphere, I’m curious to know which organizations, initiatives, or reports do the best job at reporting the social impact and cost-effectiveness of SME development services? Tweet @ChrisProttas and I will post them here.

Eubank posits that because the Somaliland government did not benefit from aid revenues, it had greater incentive to reconcile with the local commercial interests, which, in turn, had a vested interest in peace and stability that served the country well. Somaliland indeed appears to have taken major steps forward since its decimation by civil war, rebuilding cities and towns, and increasing schooling and commercial activity. A UNDP/World Bank survey finds that Somaliland has significantly higher average income than Somalia proper, a reversal of the prewar distribution, with superior health statistics as well.

Folks like Dambisa Moyo have received recent attention for their criticisms of state-directed aid, and Somaliland serves as an excellent, if not wholly generalizable, case study in development without state aid.  The theoretical advantage of aid-less development is that the government is dependent on its citizens for its revenues, and because “citizens and businesses can resist taxation, and by doing so increase the costs of tax collection, decreasing its efficacy, dependency on tax revenue creates a de facto mechanism of accountability which can be employed even in the absence of effective de jure accountability arrangements.”

Somaliland stands in stark contrast to most of sub-Saharan Africa, where, “in 2005 there were 16 sub-Saharan countries in which the ratio of foreign assistance to government expenditures is greater than 50%.” If Somaliland received an amount of state aid proportionate to the median sub-Saharan country, the government would have had control over approximately $94 million from foreign sources, triple its annual revenues.

Eubank argues that the absence of this external funding has allowed Somaliland to develop:

First, a lack of outside financial support increased the influence of the business community, which provided all government financing. Because pastoral economies are dependent upon stability and management of public goods like water and grazing lands, the influence of the business community has proved to be supportive of political reconciliation.
…
Second, with a large number of political actors with relative parity in terms of resources, a lack of outside support has forced compromise and co-option of opposition groups.
…
Somaliland government did not have an independent revenue base, making it dependent upon the continued support of its constituents.

The post-secession history of Somaliland is certainly not neat and orderly. The clans that came together to win their independence were unable to sustain their coalition, and the postwar transitional government broke into warring factions. The government and its opponents fell into a bloody struggle for power, with the government attempting to cease the Port of Berbera in order to fund its fighting. The government, however, was not strong enough to wrest control of the port from the opposing clan. With little hope to sustain itself, the government was forced to the bargaining table.

The peace conferences that followed were funded by local communities and the agenda largely reflected their interests, leading to broader government representation and a president favored by the port businessmen. Those early years saw additional fighting, but the government was again forced to reconcile with its opponents because it did not have the power to combat them. Civil society, thanks largely to the vested interests and relative power of commercial stakeholders, successfully pressured the government to preserve peace and stability.

Government decision-making and financial power have since remained relatively decentralized under a constitution constructed to prevent state predation. Somaliland is now “one of the only governments in Africa with ‘cohabitation’ between rival parties in the executive and legislative branches.” As an analyst put it in a  2009 Human Rights Watch report: “Some in the government don’t believe in our democratic process, but no one has enough power to destroy it.”

Since the civil war, the government has been dependent on business both for loans and tax revenues. The businessmen, for the most part, had a personal stake in “freedom of movement; freedom of trade; access to common grazing areas; access to common water sources; and the return of private property.” This is not to say their influence was wholly benevolent: livestock owners saw an opportunity to secure a monopoly of the market through politics, and did so, driving out smaller competitors. But by and large, business men positively influenced development: pressuring politicians to maintain stability and economic freedom, paying alms for social services, and conducting development projects.

As Eubank allows, the case of Somaliland is not necessarily generalizable:

Somaliland lacked natural resources that would have reduced government dependency on local revenues even in the absence of aid; it had some degree of parity among different clans, which prevented a single group from dominating others; commercial actors had an interest in promoting peace because of the nature of pastoral economic production; and the country’s relatively strong traditional institutions were able to mediate equitable agreements once resource constraints forced parties to the negotiating table.

While all of these characteristics are important, I would focus on the nature of dominant economic activity (e.g., pastoral production rather than natural resource extraction). The interests and power of the local businesses allowed Somaliland to follow a path similar to any student of European economic and political development:

Researchers studying state formation in medieval Britain and the Netherlands have argued that the modern, representative state emerged as the result of negotiations between autocratic governments who needed tax revenues in order to survive inter-state conflicts on the one hand, and citizens who were only willing to consent to taxation in exchange for greater government accountability on the other. In these historic cases, government dependency on local sources of revenue provided those in control of economic assets with significant leverage over the government which they were able to use to demand the development of more accountable and representative political institutions though a process generally referred to as “revenue bargaining.”

This revenue bargaining was only possible by the absence of state-directed foreign aid. Somalia provides an illustrative example of the impact of aid:

Somalia was Africa’s largest per capita recipient of international aid in Africa (with the exception of micro-states such as Gambia) [during the 1980's]. These huge inflows of aid money, especially from the U.S., made it possible for the Siad Barre government to establish a patrimonial system wholly disproportionate to the productive economy. Indeed, it was the aid flows that made possible the strategy of assaulting the productive sectors such as agriculture and livestock. (de Waal)

While Somaliland is but one case study, it should set off some alarm bells. Beyond questions of state-directed aid effectiveness, second-order effects on the natural political development process suggest that aid might be better directed at the citizens themselves, or at least at the local communities. Rather than reinforcing existing state structures, outsiders might better serve inhabitants by helping the individuals increase their economic and political power in order to secure a better deal at the bargaining table with government.