What SROI measures, and what it misses — the methodology of impact evaluation

The SROI formula itself is not complicated: divide the social value created, discounted to present value, by the investment made, also discounted to present value. What is complicated is what goes into that numerator. Someone who was isolated makes a friend, someone's depression eases, someone becomes able to work — none of these changes has a market price of its own. SROI handles this with what it calls a 'financial proxy': it borrows the price of an existing service or programme with a comparable effect and uses that price as a stand-in for the social value created. Easing isolation, for instance, might be proxied by the cost of a comparable counselling service. The moment that substitution happens, the evaluator's judgment enters the number.

Day one of this series noted that goodwill alone no longer draws in funding. SROI was built precisely to answer that demand. It was popularised in the mid-1990s by REDF, the Roberts Enterprise Development Fund, a US foundation evaluating social enterprises that employ the long-term unemployed, and later systematised by UK practitioners. Today it is standardised by Social Value International, formerly the SROI Network, which sets out seven Principles of Social Value. Today's article looks one level deeper at how this tool is actually used, where it can be trusted, and where it should be questioned.

An abstract explanation only goes so far, so consider one real example. A published evaluation by Skills for Justice, a UK sector body, reported £7.67 of social value for every £1 invested in an emergency-services project supporting vulnerable people. Read on its own, that number looks like a return of more than sevenfold. What it actually represents is the sum of the financial proxies the evaluator chose, divided by the amount invested. Choose different proxies and 7.67 becomes a different number. That is not fraud or error — it is a property built into the method from the start. The problem is that a reader who sees only the final figure, '7.67,' in a report has no way of knowing that.

The official SROI Guide, published in 2009 and updated in 2012, does not hide this property. It requires evaluators to subtract 'deadweight' — the change that would have happened anyway — and 'attribution' — the share of the effect caused by other factors — before arriving at a final ratio, so a figure like 7.67 is meant to be what remains after those deductions. The methodology itself, in other words, has procedures built in against overstatement. The problem is that a reader of the finished report has almost no way to verify how carefully those procedures were actually followed. The next section looks at how peer-reviewed research has documented that subjectivity.