Cures Ahead of the Evidence: When Interventions Judged Beneficial Were Tested

Judged sound, and what testing found

When an intervention is judged sound by those who design and deliver it, is that a reasonable basis for assuming it helps and does no harm? The question applies in any field that works with people, including family support. How often benefit is assumed without a fair test in family support has not been established in the sources gathered for this essay. The essay takes four cases from other fields where a test was run, reports what the authors found, and notes where the practice or the claim of benefit came before the test. Figures are as reported by the original authors and reviewers.

The four cases are of two different kinds, and the essay keeps them apart. The first two are social programmes for young people, delivered by services: the Cambridge-Somerville Youth Study and Scared Straight. They were tested on whether participants later offended less or did better in life. The second two are physical treatments for injury and illness: ice for soft tissue injury, and cooling after cardiac arrest. They were tested on whether patients recovered or survived. What the four share is one question: whether a practice judged sound, or already in use, did better than doing nothing. Cold links the two treatments, and YFL's essay Cold Water and the Body: What Research Shows About Benefit, Targeted Use and Risk sets out the research on cold water more fully.

The Cambridge-Somerville Youth Study

The study was funded, designed and directed by Richard Cabot, a Harvard professor of clinical medicine and social ethics (McCord, 2003; Welsh et al., 2017; Harvard University Library, Papers of Richard Clarke Cabot). It began in the 1930s in Cambridge and Somerville, Massachusetts, in the United States, to test the hypothesis that boys at risk of delinquency would benefit from "support, friendship, and timely guidance" (McCord, 2003). Staff matched boys on age, social background, temperament and other characteristics, and the toss of a coin decided which boy in each pair received the programme (McCord, 2003).

The programme was substantial and professionally staffed. Boys were assigned counsellors, most of them trained social workers, and received tutoring, medical and psychiatric attention and recreational activities. Almost half also attended summer camp (Gottfredson, 2010). Boys in the treatment group stayed in the programme for a mean of 5.5 years, with case workers visiting about twice a month (Welsh et al., 2019).

Joan McCord (1930-2004) was an American criminologist, later a professor at Temple University, who followed the participants for decades (McCord, 2007). Follow-ups in 1948 and 1956 found no differences in criminal behaviour between the groups. A third follow-up, about thirty years after the programme, when participants averaged 47, reported that treatment-group men were more likely than controls to have committed two or more crimes, to show signs of alcoholism and mental illness, to hold lower-prestige jobs, to have high blood pressure or heart trouble, and to die before 35 (Welsh et al., 2019, describing that follow-up). Gottfredson (2010) summarises the same follow-up as finding equal conviction rates in the two groups, but more multiple offences among treatment men. McCord took the results to show that a prevention programme can have harmful effects on the people it serves (McCord, 2003). McCord's own follow-up papers have not been read for this essay.

A later follow-up. Brandon Welsh, a professor of criminology at Northeastern University in Boston and later the study's director, led a fourth follow-up that traced participants into old age, 72 years after the intervention. Of the 506 boys, 488 were located and 446 had died. There was no significant difference in mortality between the groups: relative risk 1.05 (95% confidence interval 0.99 to 1.11), with a median age at death of 69 years in the treatment group and 70 in the control group. For death before 40, the relative risk was 1.15 (0.55 to 2.43). The authors state that the mortality effects seen in middle age were not observed in old age, and that they cannot definitively explain why. They list limits: departures from random allocation, a reduced sample that weakened statistical power, a historical programme with boys only, mostly white, and no assessment of offending in old age (Welsh et al., 2019).

Taken together, the Cambridge-Somerville case is a programme that was reported to produce worse outcomes than none on several measures at one follow-up. The mortality part of that finding was not seen at a later one. It is not a case of a programme shown to shorten lives.

McCord made a second point that applies beyond this study. Evaluations typically ask only whether a programme works, and harmful results often go unreported. Unless programmes are tested for harm as well as benefit, she wrote, choosing between them stays "a dangerous guess" (McCord, 2003).

Scared Straight programmes

These programmes bring young people who have offended, or are judged at risk of offending, on organised visits to prisons. The aim is deterrence through first-hand sight of prison life and contact with adult inmates.

A Cochrane and Campbell Collaboration review included nine randomised trials involving 946 young people, almost all male. All nine were run in the United States, in eight states, between 1967 and 1992. Each trial had a no-treatment control condition and at least one measure of later criminal behaviour. The reviewers concluded that such programmes increase delinquency relative to doing nothing. For the first post-treatment measure of officially recorded offending, the pooled odds ratio across seven studies was 1.68 (95% confidence interval 1.20 to 2.36) under a fixed-effect model and 1.72 (1.13 to 2.62) under a random-effects model (Petrosino et al., 2013). The reviewers noted that the programmes remain in use despite research questioning their effectiveness, and that agencies permitting them must evaluate them rigorously to ensure they do not cause more harm than good (Petrosino et al., 2013).

Treatments used before they were tested

The next two cases are physical treatments. In both, practice was established before trials of good quality were available, and the later evidence does not show the benefit that practice assumed. Both involve cold, which has a long history as a treatment.

Cold water before testing. Cold bathing was being written about as a remedy in England by the early 1700s (Floyer, 1702), and in 1797 the Liverpool physician James Currie published case reports on pouring cold water over the body in fever (Currie, 1797). Interest grew in Britain in the 1840s, after Captain Richard Claridge published an account of the methods of Vincent Priessnitz, who treated patients with cold water at Gräfenberg in Austrian Silesia (Claridge, 1842). Bradley and Dupree (2003) report more than twenty hydropathic institutions founded in Britain before 1860, attracting many prominent Victorians, while most medical practitioners remained unconvinced. The case for the treatment rested on theory, personal observation and testimonial accounts, not on comparison with untreated patients. The same historians do not dismiss hydropathy as simply unscientific: they describe its qualified practitioners as sharing much of the medical theory of their day (Bradley & Dupree, 2003). The Lancet, a leading medical journal, condemned the movement as quackery, and critics pointed to patient deaths. Popularity and plausible theory were not tests of benefit, and the point of the history is the order of events, practice first and trials later, not that the treatment was shown to harm.

Ice for soft tissue injury. Ice is used for sprains and strains, and a review described its use as widely accepted and widely practised (Collins, 2008). A systematic review of 22 randomised trials found a mean quality score of 3.4 out of 10 and concluded that many more high-quality trials were needed (Bleakley et al., 2004). Collins (2008) found six human trials, four of them lacking randomisation and blinding, and concluded that there was insufficient evidence that cryotherapy improves clinical outcome. Collins also reported an earlier personal survey of Irish emergency consultants, in which 73% always or frequently advised ice; experience (47%) and common sense (27%) were the reasons most often given, and 17% cited scientific reasoning. These are second-hand figures from a small survey in one country. This case shows practice ahead of trials, not harm: the reviews found weak evidence of benefit and did not report that ice damaged recovery in people. Collins noted that animal studies suggested excessive or prolonged cooling is damaging, which is not evidence about people.

Cooling after cardiac arrest. Two small randomised trials published in 2002 reported better outcomes after induced cooling following cardiac arrest, and the treatment was widely adopted (as described in secondary summaries of the later trial). In 2021 the TTM2 trial randomised 1,861 comatose patients after out-of-hospital cardiac arrest to cooling at 33°C or to normal temperature with early treatment of fever. Death at six months occurred in 50% of the cooled group and 48% of the other group (relative risk 1.04, 95% confidence interval 0.94 to 1.14). Arrhythmia causing haemodynamic compromise was more common with cooling: 24% against 17% (Dankiewicz et al., 2021). The comparison group had fever treated, so the trial cannot say what happens with no temperature management at all. European guidance in 2022 concluded that evidence was insufficient to recommend for or against temperature control between 32 and 36°C, while recommending that fever be actively prevented (Nolan et al., 2022).

A counter-case. Not every established use of cold failed testing. Whole-body cold water immersion for exertional heatstroke is described in sports medicine reviews as the gold standard, and the evidence for it rests on physiological studies and case series, because trials that induced heatstroke would not be ethical (see Cold Water and the Body: What Research Shows About Benefit, Targeted Use and Risk). An absence of randomised trials does not by itself show an absence of benefit, and a physiological rationale can be well supported.

Why plausible mechanisms did not settle it

In each case a reason to expect benefit existed. Cabot's hypothesis was that friendship and guidance would help. Cooling after cardiac arrest was postulated to reduce the brain's oxygen demand (as described in a secondary summary of the TTM2 trial). Ice has a known effect on tissue temperature. The tests asked a different question: whether people did better, and whether any did worse.

For Cambridge-Somerville, the best-known proposed explanation of the mid-life findings is "deviancy training", in which peers reinforce rule-breaking talk and behaviour when at-risk young people are grouped together. Later research found that deviancy training does occur in after-school programmes, is more likely in less structured activities, and that more violent behaviour also occurs in those periods (Gottfredson, 2010). Scholars do not agree on a single mechanism, and the degree of harm varies with programme type and individual factors (Zane et al., 2016). Scared Straight involves a brief prison exposure rather than sustained peer grouping, so the same explanation does not obviously cover both cases. For the purposes of this essay the mechanisms are open questions. The documented points are the direction of the effects found in the trials, and the strength of the evidence behind each.

What the four cases do and do not show

Observed. In all four, a practice or programme was established, or its benefit assumed, before a fair test. Where a fair test was run, it found no benefit (ice, cooling's survival outcome), or outcomes worse than doing nothing on some measures (Scared Straight, cooling's arrhythmia rate, Cambridge-Somerville at mid-life). The strength of the harm findings differs. Scared Straight rests on nine randomised trials. The Cambridge-Somerville mortality finding was not seen at a later follow-up. The ice reviews report weak evidence of benefit and no harm.

Not shown. The four cases were chosen as known examples. They are not a sample, so they cannot show how often cure claims overreach, and the heatstroke counter-case shows that some do not. None of the cases comes from family support or child protection.

Hypothesis, not finding. One reading is that in helping work too, confidence among practitioners and a plausible mechanism can stand in for a comparison before one is made. No source gathered for this essay measures how often that happens in family support.

Transfer to relational practice. It would be easy to draw a lesson for helping relationships generally: that a relationship judged supportive may harm. The evidence gathered here does not allow that step. Cambridge-Somerville did include a relational aim, with counsellors offering friendship and guidance, but it combined that with tutoring, medical and psychiatric attention, recreation and camps, so the relationship cannot be separated from the rest. Scared Straight tests a single brief exposure and has no helping relationship to isolate. The cold treatments have none. Neither youth programme measured relationship quality as the variable under study.

Setting is a further limit. Cambridge-Somerville was an American programme of the 1930s and 1940s, and the Scared Straight trials, run between 1967 and 1992, were also American (Petrosino et al., 2013). The cooling and ice evidence is clinical, not social. Law, services and professional systems differ between the United States and England and across those decades, and findings cannot be assumed to hold in English family services.

What can be said is narrower. Benefit cannot be assumed from a judgement that an intervention is sound or from long use, and a fair test needs a comparison group, an outcome that includes harm, and follow-up long enough to see it. The Cambridge-Somerville follow-ups show that length of follow-up can change the picture. YFL's essay Sure Start: What Was Promised, What Was Measured, and When It Was Known makes the related point that a fair test takes time. Does the Helping Relationship Heal? What the Evidence Shows sets out the evidence that relationship quality is linked to outcomes, and Small Effects at National Scale: The Prevention Paradox and the Helping Relationship takes up what small average effects mean when applied to whole populations.

A judgement that an intervention is sound is not, on its own, evidence of benefit. Whether any of this bears on relational work is for readers to weigh, because the research gathered here does not answer it. YFL offers the information without instruction.

---

References

Bleakley, C., McDonough, S., & MacAuley, D. (2004). The use of ice in the treatment of acute soft-tissue injury: A systematic review of randomized controlled trials. American Journal of Sports Medicine, 32(1), 251-261.

Bradley, J., & Dupree, M. (2003). A shadow of orthodoxy? An epistemology of British hydropathy, 1840-1858. Medical History, 47, 173-194.

Claridge, R. T. (1842). Hydropathy; or, the cold water cure, as practised by Vincent Priessnitz, at Gräfenberg, Silesia, Austria. James Madden.

Collins, N. C. (2008). Is ice right? Does cryotherapy improve outcome for acute soft tissue injury? Emergency Medicine Journal, 25(2), 65-68.

Currie, J. (1797). Medical reports, on the effects of water, cold and warm, as a remedy in fever and febrile diseases. Cadell & Davies.

Dankiewicz, J., Cronberg, T., Lilja, G., et al. (2021). Hypothermia versus normothermia after out-of-hospital cardiac arrest. New England Journal of Medicine, 384(24), 2283-2294.

Floyer, J. (1702). Psychrolousia, or the history of cold-bathing.

Gilbert, N., Parton, N., & Skivenes, M. (Eds.). (2011). Child protection systems: International trends and orientations. Oxford University Press.

Gottfredson, D. C. (2010). Deviancy training: Understanding how preventive interventions harm. Journal of Experimental Criminology, 6(3), 229-243.

Harvard University Library. Papers of Richard Clarke Cabot: An inventory. Harvard University Archives.

McCord, J. (2003). Cures that harm: Unanticipated outcomes of crime prevention programs. Annals of the American Academy of Political and Social Science, 587, 16-30.

McCord, J. (2007). Crime and family: Selected essays of Joan McCord. Temple University Press.

Nolan, J. P., et al. (2022). European Resuscitation Council and European Society of Intensive Care Medicine guidelines on temperature control after cardiac arrest in adults. Resuscitation, 172, 229-236.

Petrosino, A., Turpin-Petrosino, C., Hollis-Peel, M. E., & Lavenberg, J. G. (2013). 'Scared Straight' and other juvenile awareness programs for preventing juvenile delinquency. Cochrane Database of Systematic Reviews, 2013(4), CD002796; also Campbell Systematic Reviews, 2013:5.

Welsh, B. C., Zane, S. N., & Rocque, M. (2017). Delinquency prevention for individual change: Richard Clarke Cabot and the making of the Cambridge-Somerville Youth Study. Journal of Criminal Justice, 52, 79-89.

Welsh, B. C., Zane, S. N., Zimmerman, G. M., & Yohros, A. (2019). Association of a crime prevention program for boys with mortality 72 years after the intervention: Follow-up of a randomized clinical trial. JAMA Network Open, 2(3), e190782.

Zane, S. N., Welsh, B. C., & Zimmerman, G. M. (2016). Examining the iatrogenic effects of the Cambridge-Somerville Youth Study: Existing explanations and new appraisals. British Journal of Criminology, 56(1), 141-160.