I’m not sure when the name “living systematic review” was coined, but it was fleshed out in an article in 2014 by Julian Elliott…
In a courtroom, the full power of the state comes down on an individual. No one should have to face that on their own.
A criminal defense lawyer was making this argument to me after a long day in the court we were both working in. I’d asked him, how could he defend that man? After what he had done.
Without this, he said, without a system to temper power, we don’t have civilization.
That was in Australia, back in the 1970s. I was a court reporter for four years or so – the stenographer that takes down the evidence and reports legal argument. I’d started when I was 17.
I was often on circuit – judge, court staff and lawyers from Sydney going to country towns to deal with whatever serious offenses had surfaced and navigated the police system there.
Being away from home in a small group gives people time to talk at leisure. But I never heard discussions about the need to empirically test issues related to aspects of justice. The wide-ranging rise of randomized experiments in this area by criminologists, psychologists and other scientists was still ahead.
I took this photo of the courthouse in Bourke in 1978.
It’s stunningly beautiful country out there – very deep in the outback. “Back o’ Bourke” is one of Australians’ expressions for describing just how remote some place is.
Bourke is an amazing, complicated, rough town. The legacy of colonial genocide hits you hard there. A race riot in 1997 branded it the most racist town in Australia. It was only last year that the first shop owner in Bourke took down the security shutters that had become normal after that riot.
There had been rioting not long before our plane landed too. Feelings were running high because of what we were coming for: the trial of a white police officer who had shot a young indigenous man in the leg.
The police didn’t want us to set foot outside our hotel without their protection. As soon as they dropped us off there from the airport, though, the judge headed out on a long walk around town, including the parts with charred houses. People needed to see that the court was independent: “without fear or favor.” I went with him.
I felt like I’d walked into To Kill a Mockingbird that week. A version where Atticus Finch was the judge, but the outcome was the same. The all-white jury acquitted the policeman.
Watching juries day-in and day-out for a few years made me doubt that they can rise much above the racial, gender and other prejudices of their communities. It’s heart wrenching. What happens in a court is at once the source of stigma and the enforcer of it.
It’s hard to know objectively, though, how biased or effective juries are: people would be unwilling to put the jury institution itself on trial. For many, it’s an expression of democracy that brooks no doubt.
So scientists work hard to do the most unbiased research possible within these constraints. There have been lots of studies with mock juries and simulated court cases. But it remains far from the reality of a deliberating jury.
There are good data on how often the presiding judge in a criminal trial in the U.S. disagrees with the jury’s verdict: it’s about a third of the time. When it gets complex, judges and juries interpret the strength of evidence and the law differently.
It’s hard to argue, from what we know, that a jury is very effective at “diagnosing” the truth – or that all the other components of trials achieve it either. It’s unclear whether a jury of 12 is better than another number, or whether unanimous vote is more likely to be “right.” And we don’t know for sure how to quickly pre-select jurors who will be less prejudiced or less easily swayed.
Some other democratic countries, like the Netherlands, don’t have juries. The Netherlands is one of the countries whose justice systems were shaped by the Napoleonic Code. The Dutch system relies on judicial independence and open justice – keeping court proceedings open to the public – to temper the power of the state.
Other countries, like Germany, have a tribunal of professional judges and lay people who take the job for several years at a time. The history of Germany’s judicial policy illustrates the sociopolitical stakes and drama that underlie these institutions.
Judicial systems like the Dutch and German ones are called non-adversarial or inquisitorial, because the judges are responsible for the fact-finding. In adversarial ones, including offshoots of Britain’s common law system in Australia and the U.S., judges are more like referees between prosecution and defense. But even in these countries, many matters are tried directly from the Bench.
I’ve never been in a jury. But I have seen the view from the Bench when there’s no jury. In the 1990s, after I had been a health consumer advocate and researcher for years, I was appointed to a panel of people eligible to co-adjudicate disciplinary hearings for doctors in the state I lived in.
A full Medical Tribunal there is a court case, in a court room, with a judge, two doctors and a layperson who has been accepted by both the consumer protection side and the medical profession. I ended up sitting on the Bench with a judge I’d worked with years before as a reporter.
It was a patient-to-patient HIV transmission case. Catherine Waldby and colleagues studied that Tribunal, coming to the conclusion that it was a more appropriate way to deal with the complexity of the virus and its transmission than the criminal court approach used elsewhere.
It was very high pressure. There were days you’d have to dodge TV cameras to get into the courtroom. I lost a lot of sleep during the days of that hearing. I can’t imagine how I would have made up my mind with confidence if I couldn’t get as many answers from expert witnesses as I needed, and then deliberate with legal and content experts. It gave me a lot of sympathy for the worried looks I used to see on jurors’ faces.
“I know your laws: now you can understand mine.”
Those are the words of Joe Ngallametta, an Elder of the people of Aurukun in west Cape York, Queensland, northern Australia. He passes the knowledge from the laws of The Dreaming to young people in his community, and shares them with the non-indigenous world too. Those traditions are thousands of years old, and they point to some ways that might improve our relative newcomer systems of regulating human affairs.
Traditional Australian Aboriginal laws aren’t about the state acting, and there’s a focus on restorative justice. Restorative justice is not about interrogation and punishment. It is concerned with getting towards truth with depth of understanding, the sincere apology a victim needs, making amends, and a path to a future without re-offending.
South Africa’s Truth and Reconciliation Commission, set up in 1995, is another example of drawing on traditional laws, this time from Nelson Mandela and Desmond Tutu’s Xhosa tradition of ubuntu. The post-apartheid South African constitution embedded more traditional restorative justice perspectives into law.
In 1989, a criminologist from Queensland, John Braithwaite, published a book arguing that we had much to learn from the Aboriginal peoples’ traditions of managing shame, re-integrating criminals and healing the social harm crime causes. You can read more about this in the background of this recent systematic review by Heather Strang and colleagues. (And more on systematic review methodology here and on meta-analysis – the analysis of data from more than one study – here.)
That review analyzed 10 randomized trials of restorative justice conferencing (RJC) with 1879 offenders, based generally on the Braithwaite model. Trained facilitators bring together victims, offenders and perhaps kin and community who want to come to an agreement about what the offender can do to repair the harm they had caused.
Those authors concluded that compared with conventional options like going to court, RJC improved victims’ satisfaction and reduced re-offending – with the proviso that a trial in Australia was toxic for the Aboriginal people who participated.
Their meta-analyses show impressive results, but we really do need to see more trials before we can be sure who is helped, and when it is an appropriate alternative to criminal prosecution. We still need trials for a wider range of offenses and crimes, including sexual assault. We need to be sure the comparatively negative ratings by victims in the control groups weren’t because they were disappointed about missing out on getting to safely express their feelings to the offender.
A systematic review by Nuala Livingstone and colleagues, published a few months before, covered largely the same body of evidence, but came to a different conclusion: they don’t believe the trials show a definite benefit.
I’ve written before about the problem of dueling meta-analyses, then in relation to preventing post-traumatic stress disorder. In this case, several factors contribute to the difference in conclusions. The Livingstone review is about children and teenagers, and while the overlap is major, there are trials in adults covered additionally by the Strang review.
Livingstone’s includes a trial with a negative outcome that Strang’s group rejected on quality grounds. Including it drags the results down. And there were different choices in data analysis that moved the needle.
Whether you agree with the more positive take from Strang’s team, the less enthusiastic one from Livingstone’s, or, like me, land somewhere between the two, these trials still make the case for giving this alternative model a chance to fully develop.
Strang’s review points to areas where versions are already embedded in justice systems. In New Zealand, a traditional Maori practice now called the family group conference became a standard for juvenile crime in 1989. In Australia, diversionary conferences can be available as an alternative to police prosecution. In non-legal settings like employment and education, it’s called transformative justice.
It can be important for new systems to get the kind of rigorous empirical testing that our conventional systems have not had. This image displays the compelling results of a group of trials in a forest plot that serve as a cautionary tale of how badly wrong well-intentioned ideas can go:
In 1978, a documentary called Scared Straight! won an Academy award. It advocated for confronting teenagers who had committed offenses with harsh encounters with prisoners and prisons as a deterrent. It became a movement that grew quickly.
Years later, a systematic review of randomized trials of “Scared straight” programs would find not only no benefit, but an overall increase in the rate of crime. (That review was updated last year. It’s a riveting read.)
Whether it’s a novel or conventional system, it will involve the police: one of society’s most challenging roles.
Policing – the good and the bad – was another issue at the heart of the trial in Bourke. That police officer was on trial, not because his colleagues arrested him, but because of intervention from Sydney. The force also took action after the jury’s acquittal.
I didn’t encounter the police as a force of good and protection as a child, though – a consequence of being born on the wrong side of the tracks, and into a life long shadowed by the threat of outbreaks of domestic violence.
I only ever sought police help once when I was a kid. My mother whispered to me to get help. I crept on hands and knees behind furniture, then darted out behind my father to the door, and ran to the police station as though the devil were after me. But the police wouldn’t come. Profound distrust was born of the fear and despair I felt as I walked back to my mother with the news that we were on our own. It lingers.
That story is a microcosm of a critical aspect of procedural justice. There has to be community cooperation and belief in the legitimacy of the police and the neutrality of the force. There has to be dignity and respect in authorities’ interactions with the community, and trustworthy motives – no profiling or favoring on the basis of race or anything else. A consequence of not experiencing procedural justice is a lack of respect for the laws of the state. Not surprisingly, that can increase crime. Distributive justice – who gets the bulk of police protection – affects police legitimacy, too.
A systematic review by Lorraine Mazerolle and colleagues published a year ago found and analyzed 30 studies on police-initiated interventions that intend to increase community perceptions of police legitimacy or at least one aspect of procedural justice.
Mostly they were programs in community policing – training officers, involving them in community activities, doing foot patrols. Some were programs for police-based alternative dispute resolution processes instead of taking people to court – including a couple of the trials of RJC. Plus a few specific programs had components that met the study criteria – like this one aiming to reduce illegal gun possession and fear of gun violence in New Haven, Connecticut. There was one neighborhood watch program, too.
So did any of it work? There’s still a way to go to know exactly what works best, and in what context. But there was enough impact on community confidence and reduced reports of being victims of crime to be encouraging. With effort, change can come faster.
That science is being brought to bear on these issues gives me a lot of hope. But these days, it seems somewhat ironic to me that our symbol of justice is a woman. I chose this image of her as my final for this post – “Justice Delayed, Justice Denied” – to symbolize the system’s partial blindness to the suffering of women.
When I was a teenager working in courts, the second-wave of feminism was swirling around me. It changed the course of my life and the way I looked at the world. In my work, I’d see society’s prejudices about women reflected in unfairness, especially in trials of violence against women.
I thought it was going to change more quickly. More than 30 years later, and justice for women is still too often denied. It seems to me, if the justice system worked well for women, and if our attitudes towards women had changed enough, women would be experiencing less sexual transgression, and women would be far safer from personal violence. Locally and globally.
Recently, science writers have been going through great turmoil over the issue of sexual harassment, respecting women’s boundaries and how to deal with offenses and allegations that fall outside clearly established systems for dealing with these matters. As women began to talk about what was happening in their lives, it struck me starkly just how much unfinished business we as a society still have. And how ill-equipped we are to neither sweep things under the carpet nor allow things to spiral excessively when we’re hurt or angry.
A meta-analysis in 2003 by Remus Ilies and colleagues found that work-related harassment is under-recognized, under-reported and unlikely to be exaggerated. Ilies’ review of literature suggested that women tend not to describe incidents as harassment – but whether they do or they don’t label what happens to them as harassment, the negative impact of the incidents themselves is the same.
Based on more than 86,000 respondents in probability samples, Ilies’ meta-analysis concluded that around a quarter of women in the U.S. have experienced sexual harassment at work, as defined by researchers.
What about sexual assault? Consider these conclusions in 2006 by Erin Casey and colleague: from the ’70s till then, there may have been a small, but not major, drop in sexual assault in the ’90s. However after that, although sexual assault in young girls may have decreased, there appeared to be an increasing rate of sexual assault in adolescents. And women’s perceptions of community responsiveness may have worsened slightly.
Rates of sexual assault are hard to assess reliably and often exaggerated [PDF], but it’s unquestionably far too common a part of our experience as women. Jody Raphael and colleagues [PDF] estimated that the rate of rape remains fairly constant at about 15 – 18% of women, and it’s still greatly under-reported. While only 2 – 8% of reports may be false, many still seem to treat all allegations as though they’re highly likely to be unfair on the accused.
When it comes to “he said, she said,” the woman may be discounted every step of the way in her life, and in the justice system. In the unfair world we live in, whether it’s gender, race or any other area of systemic inequity, the accused can seem to have so much more to lose by virtue of that unfairness.
Whatever we’ve been doing hasn’t worked – or it hasn’t worked enough. Women’s suffering remains too invisible. It’s hard to make it visible, though, when to do so may not protect us from people we fear or can’t avoid – or it would harm people close to us, people that we love. Perhaps making it more visible when we can, will help.
That offenses against women on such a scale aren’t being dealt with in a way that makes society respond with greater protection and support for women’s safety, indicates, I believe, systemic problems. The level of these problems are not the same in every community: it’s not an inevitable part of life. The trend towards innovation and empirical tests in justice is another chance to address them.
I started this post with an argument a lawyer made when I was a teenager. It helped me develop more confidence in a system that had failed me and was manifestly regularly failing many others before our eyes. A good system of justice is a pre-requisite for a society in which people can flourish – and not only those with power. You only have to look at a community where justice has completely broken down to be reminded of that. Justice and fairness need to be common practice – and not just in a courtroom.
Science isn’t “the” answer. But science can help edge us closer towards justice systems that work.
If you’re interested in ongoing legal activism around race and justice in America, check out the Equal Justice Initiative.
If you’re interested in systematic reviews of aspects of the justice system, a great place to start is the Campbell Collaboration.
The image of Frederick Dielman’s mosaic “Law” at the U.S. Library of Congress is from Wikimedia Commons.
The photo of The Weighing of the Heart from the Book of the Dead of Ani was taken at the British Museum by Edna Russmann (from Wikimedia Commons).
* The thoughts Hilda Bastian expresses here at Absolutely Maybe are personal, and do not necessarily reflect the views of the National Institutes of Health or the U.S. Department of Health and Human Services.