Comment

The dubious 'science' of risk prediction in criminal justice

By 
Deborah Drake, Catriona Havard, and John Muncie
Wednesday, 22 October 2014

The prediction, assessment and management of risk have become central to processes of criminal justice administration in many countries around the world.

Indeed, the ‘risk of offending’, along with how it is assessed and measured, has become taken-for-granted and absorbed into routine criminal justice practice.  However, there are significant weaknesses in the methods and approaches that inform risk-based measurement when it comes to predicting human behaviour.  Human beings are inherently and inevitably social, changeable and, let’s be honest, highly unpredictable. With this in mind, we wish to raise a couple of questions in relation to risk prediction and individual ‘offenders’:

  • What is the scientific evidence on which predictions about the risk of involvement in criminal activity are based?
  • What assumptions have been made in the development of measures and methods of risk prediction?

In 1961, the Cambridge Study of Delinquent Development, led by David Farrington, aimed to identify the ‘risk factors’ that supposedly underpin crime and, in particular, juvenile delinquency. The study included a sample of 411 working class boys, selected from six primary schools in Camberwell, South London. The boys were contacted nine times – at ages 10, 14, 16, 18, 21, 32, 46, and 48 (Farrington et al., 2006) – to determine which of them had developed a ‘delinquent way of life’ and why some had continued a ‘life of crime’ into adulthood. The study found that around one-fifth of the sample had been convicted of a criminal offence during adolescence and over one-third had a criminal record by the age of 32. However, just 23 of the young men in the sample (less than six per cent) were responsible for half of the total number of criminal convictions. According to the researchers, most of these ‘persistent offenders’ shared common characteristics in relation to various individual, family and environmental factors.

Based on these data, Farrington and colleagues have argued that there are several ‘risk factors’ that predict future criminality. Amongst the individual-level factors identified in the study were personality characteristics, low intelligence and high levels of impulsiveness; the strongest family factors identified were criminal or ‘antisocial’ parents, poor parental supervision and chaotic families; while the strongest environmental indicators of future criminality, according to the study, were  deprivation and high delinquency rate schools (Farrington and Welsh, 2007). Moreover, Farrington maintains that similar longitudinal research, particularly in the USA and the UK, has established that the ‘risk factor prevention paradigm’ has global (western) reliability and practical application:

A key advantage of the risk factor prevention paradigm is that it links explanation and prevention, fundamental and applied research, and scholars and practitioners. …The paradigm avoids difficult theoretical questions about which risk factors have causal effects.

(Farrington, 2000).

That seems to be a powerful argument, but let us consider the methodological decisions and assumptions that have been made and applied in this study. Perhaps the first issue to consider with the Cambridge study is the sample of participants that was chosen. This was a group consisting almost entirely of white males (only 12 boys in the sample were from ethnic minority backgrounds) and all the boys in the sample were from working class families and lived in a specific location in London. It could be suggested that this sample is biased because it is not representative of the general population, and doesn’t take into account other groups in society such as women, those from middle or upper class backgrounds, or those from rural areas.  Choosing a representative sample is extremely important when trying to make predictions about issues that can affect the population as a whole.

A second issue to bear in mind is that whilst risk factor research may reveal some correlations with statistical records of offending, this should not be confused with having discovered the causes of such offending. It is misleading to place too much emphasis on correlations between particular variables because correlation only tells us that the variables change together (i.e. as one thing goes up another thing also goes up, or as one thing goes up another thing goes down). Correlation does not tell us why the variables change together. It might be that there is a causal link between A and B.  For example, the further you run (A) the more exhausted you will get (B), because running takes energy and tires you out. It might be that there is some other variable that affects both A and B.  For example, ice cream sales (A) and the number of shark attacks (B) are correlated, not because one causes the other but because of a third variable (C): both increase in warm weather (more people buy ice cream and more people go swimming in the sea).  Variables can even be correlated without there being any connection – direct or indirect – between them. For example, there is a negative correlation between the number of pirates (A) and the mean global temperature (B): the number of pirates has decreased over the past 200 years and the mean global temperature has increased over the same time period. However, no-one assumes from this that pirates were preventing global warming! (For more on this example, see www.forbes.com/sites/erikaandersen/2012/03/23/true-fact-the-lack-of-pirates-is-causing-global-warming/)

One of the dangers of ignoring the questionable theoretical assumptions of risk-factor prediction research and proceeding with it as though it is, at least, telling us ‘something’ is that it can result in a ‘self-fulfilling prophecy’. A self-fulfilling prophecy often occurs when people’s response to a prediction makes that prediction come true. For example, if you think you probably won’t get a job that you’re applying for then you might come across as unenthusiastic or uninterested in the interview, thus ensuring that you don’t get the job. In the context of crime risk prediction, if certain groups are labelled as being ‘at risk’ then they may either live up to that stereotype (e.g. “if I’m going to be treated as a criminal anyway, I might as well commit offences”) or they may be treated as if they embody the risks they are thought to pose (e.g. they may be treated harshly by the criminal justice system, even for minor offences). Even allowing for the impact of self-fulfilling prophecies, the type of research carried out in the Cambridge Study can only tell us what factors might be linked to (known) offending, not how and why such factors might be linked.

The narrow range of factors considered through the quantitative measures of the ‘risk factor prevention paradigm’ also necessarily ‘misses’ valuable information about the way perceptions of risk can be gleaned from juvenile justice practitioners or from young people themselves (Armstrong, 2004; Case, 2007). For example, longitudinal research carried out by the University of Edinburgh in Scotland (McAra and McVie, 2007) has revealed that the key ‘risk factor’ propelling young people into and through juvenile justice systems is not their ‘dysfunctional families’, but targeted policing based on discriminatory determinations about those children and young people who appear to be ‘respectable’ and those who do not. A core problem with the ‘risk factor prevention paradigm’, therefore, is its presentation of specified individualised ‘risks’ as if they comprise uncontroversial facts, truths and scientific realities.

In short, the Cambridge Study of Delinquent Development has focused on a specific group of individuals, in a specific location, recording specific acts. In so doing, it tells us little or nothing about the wide range of criminal and harmful behaviours which are not routinely recorded. For example, it has no application to understanding corporate crime (e.g. tax evasion, occupational health and safety violations, eco-crime); other crimes of the powerful (e.g. the expenses scandal of MPs in the UK) or state crime (e.g. deaths in custody, police brutality or war crimes). What the Cambridge study has accomplished is essentially to identify the ‘usual suspects’. In doing so, it presents an uncritical listing of the complex social situations that (some) white working class male youth live with in industrialised western societies. What it cannot do, despite the ways the findings of the study have been applied, is provide a reliable method for determining when a crime will happen or which individuals will or will not break a given law. At any given moment, social life and human decision-making can be altered in ways that are entirely random. Whilst it might be comforting to imagine that there are ways of identifying ‘risky’ individuals, the fact is that human beings just cannot be counted on to behave in predictable ways.


This article is one of a series originally published on the website of our partner organisation, the International Centre for Comparative Criminological Research.