Payment by results

Bill Puddicombe: The approach necessarily produces a simplified, time limited set of interventions in order that contractors can redeem their investment

Payment by results (PbR) in the delivery of public services has the capacity to make them more efficient and, possibly, more effective. The implementation of the approach in the wrong places and in the wrong ways, however, will lead to a serious drop in quality and the stilling of innovation. There are better ways to manage the commissioning of services that lead to less definable outcomes.

Currently one of the most heavily criticised applications of the approach is the 2011 Department of Work and Pensions’s Work Programme. In this programme, broadly speaking, contractors are paid according to the number of unemployed people that they find work for. The programme is arranged around a number of different tariffs, based on the perceived ‘difficulty’ of the case. The arguments that have been made against the programme largely relate to the way in which the contractors have dealt with their charitable, smaller partners and some of the methods that have been used to achieve remunerated outcomes.

It is too early to tell whether the Work Programme will be successful in achieving its overall goal; the reduction in unemployment (and the resultant strain on the welfare budget). The complaints laid against it are, broadly speaking, about methods used by the contractors. It could be argued that, if these unintended consequences were to be specified out by commissioners, a PbR approach could work for the relatively straightforward outcome of changing people without jobs into people with jobs.

The key here is a ‘black box’ approach in which there is a minimal specification of method and contractors can use any legal means (more or less) to achieve their outcome.

In contrast, the PbR pilots that are currently underway on drug and alcohol rehabilitation appear to be more problematical. The original approach, vigorously encouraged by central government, was to remunerate the delivery of ‘recovery’ for people dependent on alcohol and drugs. The problem that this raised (I was present at some of the early discussions of the method and saw this first hand) is that recovery is a multi-dimensional concept which, despite many attempts, has evaded a satisfactory definition. It is clear that some of the facets of it are housing, employment, improved social functioning, ending criminal activity and stopping using drugs. The discussions that produced the eventual scheme found that many of these essential elements of recovery were impossible to measure within a scheme. Consequently the eight schemes in place are using the simple, but incomplete, outcome set of freedom from drugs of dependence, improvement of health and wellbeing and reduced offending.

If these pilots ‘work’ it will be on these very limited outcomes and the eventual effect of the schemes will only be rewarded over a 12-18 month period, when an outcome of ‘recovery’ would take much longer to judge.

Experience says that the most successful commissioning of services in this kind of area is often achieved by diversity of approach, flexibility in commissioner/provider relationships and the ability for innovation to be recognised quickly and followed up in a well developed market. A ‘black box’ approach, with outcomes defined by narrow parameters is likely to be less successful in achieving the end result of ‘recovery’.

The Payment by results approach necessarily produces a simplified, time limited set of interventions in order that contractors can redeem their investment.

Bill Puddicombe is the Chief Executive of Equinox, a Third Sector organisation working with socially excluded people with multiple needs

Reference

Department for Work and Pensions (2011), The Work Programme, London: The Department for Work and Pensions.

Dan Corry: It does not really solve the problems of social policy – it is no silver bullet – and the dangers have to be watched and managed well or it will all end in tears

Money is tight these days – and that is unlikely to change in the near future. So if we want to improve what we do and what we achieve with our money in the criminal justice system then we must look favourably at Payment by results (PbR) – where the provider is only paid if they really do reduce re-offending. PbR sharpens up the incentives for providers - for-profit or not-for-profit - to deliver the outcomes that commissioners want. It forces the commissioner to be precise and stops the provider wasting time and effort unnecessarily. What is not to like?

In fact, some voluntary organisations are suspicious of such a clear and efficient model of commissioning. Though they may be reluctant to admit it, some worry they may be ‘forced’ into adopting value or practices or pursuing outcomes they do not feel comfortable with. While government commissioners are right to reflect the priorities of politicians, they should also be careful not to completely ignore good ideas and insights of organisations that have in-depth knowledge of how services are delivered on the ground. Commissioners should pay attention to what providers say about what are desirable and achievable results.

But there are other problems. There are issues with having a relevant outcome that can be properly measured and many complexities in deciding what the outcome has to be to be worthy of outcome payments (a 7 per cent reduction or a 10 per cent reduction in crime against the relevant control group?) and how it should be priced. Should it be no re-offending (or rather re-conviction since that is all we can really measure) after two years or does that fail to give incentives to providers to work with offenders so they never re-offend rather than just avoid re-arrest for a couple of years?

One of the strengths of PbR, that the provider will go the extra mile to try and hit the outcome targets desired can also lead to undesirable behaviours. One such behaviour is the temptation to ‘park and cream’. In other words you work hardest with those that you think you can get over the line and so achieve your outcome targets and associated payments. If a person is going to reoffend whatever you do then it is pointless putting much effort into them. This is a natural and at times a desirable response to the incentives faced, but it means those who are harder to help get ignored (even with complex incentive payments that are supposed to be weighted by the degree of difficulty of helping that person). It also means that prime contractors (often for-profit organisations) will pass the toughest cases on to the charity sub-contractors.

In a similar vein, all the effort goes into the specified outcomes - often hard ones with cashable savings attached - and the softer but crucial outcomes like self-esteem and wellbeing become ignored.

The other bad consequence is the temptation to fiddle the figures. If head office puts too much pressure on the organisation to hit targets, then don’t be surprised if people are tempted to make sure the data says what head office wants to hear, whatever the truth. All those involved in PbR will need to watch very carefully to avoid this - be they the providers (for-profit and not-for-profit), commissioners or indeed public watchdogs. The more that data is open and publicly available the less room for this sort of thing to go on, as people can monitor and pick up on fraud more easily.

A final potential drawback of PbR approaches is that they can squeeze out good but small voluntary sector providers who can’t manage the cash-flow consequences of only being paid in arrears and if successful. Complex social impact bonds will work for some to overcome this but so far these are very much a minority sport.

Payment by results is a good idea and will teach us a lot. But it does not really solve the problems of social policy – it is no silver bullet – and the dangers have to be watched and managed well or it will all end in tears.

Dan Corry is Chief Executive of New Philanthropy Capital and former adviser at the Treasury and Downing Street

Chris Fox and Kevin Albertson: A radical new approach to commissioning or the ‘Emperor’s new clothes’?

PbR seems an intuitively appealing concept and elsewhere we discuss its advantages (Fox and Albertson, 2011). However, there are a number of questions which must be settled for successful implementation.

What is a ‘Result’?

The Ministry of Justice notes there are several measures of re-offending which might be taken as indicators of the success of a PbR scheme. However, to date a simple, binary measure of proven re-offending seems to be their preferred option. This ignores the complexity of individuals’ journey to desistance, which might involve initially moving to less serious or less frequent offending.

There is evidence the use of simple ‘headline’ targets can be counterproductive (there are many examples in the education and health sector for example). Similarly, the potential exists for PbR providers to ‘game the system’, for example by intervening only in easier cases. Despite government assurances to the contrary, shifting the emphasis from inputs to so-called outputs will not address this problem.

Is a ‘Result’ a profit?

PbR schemes will require a sufficiently large improvement over existing provision to ensure significant savings. However, the formal evidence base and the personal experience of practitioners suggests modest reductions in re-offending are more likely. These may be too small to realise cashable savings. If large improvements are not observed, PbR interventions will require large cohorts of offenders to demonstrate changes with statistical confidence and to achieve required efficiency gains. This will limit the organisations able to become involved and the range of interventions offered. Thus, it is likely PbR providers will be more risk averse and less innovative than the state.

Where is the profit?

Home Office estimates of the social and economic costs of crime describe the potential savings arising from a reduction in crime. However, a commissioning body cannot recover reductions in private sector costs. Neither can the state recover the intrinsic savings of reduced victimisation. Further, there is no mechanism for a commissioner to access cashable savings which accrue to a different government sector, for example, the NHS. Because these savings cannot be realised, the value to the commissioner will be less than the societal value. Hence, interventions will be under-funded.

Is it PbR or bust?

The expectation of improvement rests on the assumption that private sector delivery of services are more efficient that state provision. In some cases this will no doubt be the case – it is not guaranteed. There is little evidence to support the proposition that the private sector is necessarily more efficient (see for example the equivocal evidence from the US prison system set out in Lundahl et al., 2009). It is important that we don’t lose effective public sector alternatives before PbR evaluation, otherwise PbR may persist simply because there will then be no alternative. The case for PbR should be proven before it becomes relied upon.

References

Fox, C. and Albertson, K. (2011), Payment by results and social impact bonds in the criminal justice sector: New challenges for the concept of evidence-based policy?, Criminology and Criminal Justice, 11(5), pp. 395-413.

Lundahl, B., Kunz, C., Brownell, C., Harris, N. and Van Vleet, R. (2009), ‘Prison Privatization: A Meta-analysis of Cost and Quality of Confinement Indicators’, Research on Social Work Practice Vol.19(4), pp. 383-394.

Chris Fox is Professor of Evaluation and Kevin Albertson is Principal Lecturer in Economics, Manchester Metropolitan University