Trials and error

Many of us are aware of vaccine controversies. Fewer know much about clinical trials, the randomized controlled studies that test new medical treatments. Yet these crucial processes determine how we will be treated when we need medical help, since they are the “gold standard” for getting accurate data about whether treatments do any good, or not. Done well, clinical trials can transform patients’ quality of life, even save lives; done badly, they can injure or kill. Given their importance and the huge costs involved, it would be easy to believe that all trials are undertaken based on the most rigorous evidence, with participant welfare and independent oversight the priorities at all times. Not necessarily. Two new books by US-based journalists lay bare the risks, harms and perverse incentives involved.

Both books focus on treatments for dementia, notably Alzheimer’s, which is characterized by abnormal clumps of protein in the brain – amyloid-beta plaques and tau tangles – and by brain cells that malfunction, lose their connections and die. As the brain shrinks and withers, so does the psyche: dementia removes memories, personalities and eventually the ability to do even the simplest things. It takes an awful toll on the physical, mental and financial health of the sufferer and of anyone who cares about them, and it is a way of dying that can take years to claim its prey.

Jennie Erin Smith’s Valley of Forgetting: Alzheimer’s families and the search for a cure focuses on a group of people with dementia who are well known in Alzheimer’s research circles. Probably they would be better known to the public if they were American, British or European, but they hail from a poor area of a poor and troubled country: Antioquia in Colombia. They are a set of related “paisa” families whose members have for centuries been at high risk of a form of Alzheimer’s that takes effect not in old age, but in midlife, usually starting before the age of fifty. It ruins individual lives, deprives families of their most productive members and compounds the burdens of people already affected by poverty, social insecurity and violence. Valley of Forgetting tells the story of a long-running research programme and clinical trial set up in Medellín by some exceptionally determined and enterprising doctors, with the aim of finding a cure. Its first section, “Curious Doctors”, focuses on the researchers and the work it took to get the project going. The second part, “Las Familias”, is concerned with the trial’s participants, and the final segment, “Resilience”, looks to the future.

We start with the Medellín research, led by Francisco Lopera Restrepo, himself a paisa, which identifies a genetic mutation that causes the dementia. The researchers establish that the faulty gene is autosomal dominant, so a child with an affected parent has roughly a one in two chance of inheriting it and becoming ill. The researchers trace the mutation back to the mixing of cultures and genetics – European colonialists, African slaves and the paisas’ indigenous ancestors – at the time of the Spanish conquest. They collaborate with other scientists to pin down the gene and start a clinical trial of a drug, crenezumab, that should, in theory, help.

When I worked alongside researchers running studies of dyslexia, they often grumbled about the difficulty of finding participants. They were in a large English university city with a well-educated, accessible population; Lopera’s university, by contrast, has students and professors being murdered by paramilitaries. He finds his participants by heading into the hills around Medellín on horseback, braving armed and dangerous militias, then negotiating with the families. He has to explain the science to people who believe their problems are caused by witchcraft and whose solutions are herbal medicine or prayer. He has to find the funds for new equipment and for paying the families sufficient expenses to enable them to get to Medellín. Erin Smith shows us the human cost of all this for Lopera and his team: the years of hard work, the continual search for money (at times researchers have to take jobs in the US or Europe, stalling the project), the anguish of having their key gene-identifying publication trumped by another group, and the ever-present threat of violence.

Valley of Forgetting then turns from the scientists to the participants. Here the book comes alive. Its last section is aptly titled: the families’ resilience is extraordinary. Erin Smith focuses on how clinical research’s promises, perils and processes affect the people it uses to achieve its goals, describing the human interactions that underpin the crenezumab trial. Anyone who still holds a Dawkinsian view of science as abstract and logical should read Valley of Forgetting to disabuse themselves. We see Erin Smith’s admiration for Lopera gradually tinged with doubts about his moral choices, especially when he won’t tell participants whether they are carrying the mutation and fails to mention that they can get tested privately elsewhere. The paisas are desperate to know so they can plan their lives, particularly around having children.

We also see how the identification of the mutated gene, which codes for a protein called presenilin-1, brings new power and resources to Lopera. PSEN1 is a key player in the long-dominant theory in Alzheimer’s research, the amyloid cascade hypothesis, which proposes that too much amyloid-beta is what triggers the damage. The PSEN1 mutation in the paisa families boosts amyloid levels. This makes the families a good test case for drugs such as crenezumab, which act by removing amyloid, and Lopera has assembled a sizeable cohort – so he gets the funding and equipment for a clinical trial of crenezumab. The participants are filled with hope of a cure, the scientists with energy, often working long hours and/or for free. Then come the agonizing disappointments, as trials of similar drugs, and of crenezumab itself, fail or show only minimal effects.

Valley of Forgetting has a narrative arc familiar to survivors of institutional failings. At first, scientists and participants work together towards one goal: a cure, or at least a treatment. Then priorities and expectations diverge as the trial proceeds. The paisas, in a country with poor public healthcare for neurodegenerative illness, want support and comfort in return for hours of assessments and invasive procedures – including donating their relatives’ brains. When they call for help in a crisis and no one answers the phone, they feel betrayed. The scientists need data and publications, because these bring more money to keep the project going; the original dream of treating patients morphs into the goal of sustaining the institution. Funders, meanwhile, want their money spent “efficiently”: on research, not welfare. Indeed, should a drug succeed, Erin Smith observes that the paisas wouldn’t be able to afford it and are unlikely to be given it as thanks for their help. In Colombia at least, funders have the most power, researchers have less, and the participants have very little.

As Charles Piller’s Doctored: Fraud, arrogance, and tragedy in the quest to cure Alzheimer’s makes clear, power imbalances also apply in the US, despite its emphasis on individual rights and (so far) a complex regulatory system centred on the Food and Drug Administration (FDA). The author has written about Alzheimer’s research for Science magazine, but this is a more thorough investigation into how studies get funded and trials approved. Whereas Valley of Forgetting foregrounds the humans involved in a single drug trial, Doctored is about the processes, power and money defining the field as a whole. Especially the money, because so much drug development is done by biotech startups that need a lot of investment. There’s plenty of money for them to attract, since everyone knows a treatment for dementia will yield astronomical returns. But a single null result or lawsuit, or a scientific critique from rivals sent to the FDA, can have catastrophic effects on a company’s share price. This is a high-risk, intensely volatile ecosystem in which hope and the appearance of competence are as valuable as the ability to do rigorous science. Patients and participants can easily get lost in it.

As can integrity – Piller shows how the system can be gamed. Working with a whistleblower, he looks in detail at cases where integrity failures led to the publication of misleading results that prompted large, costly, ultimately unsuccessful drug trials. Had the initial science been properly vetted, millions of dollars could have been better spent. The author leaves readers in no doubt of his key claim: that wilful blindness, groupthink, misconduct and downright fraud have contributed to vast amounts of money being wasted in recent years.

How could this happen? Because, Piller says, dubious studies have supported the beliefs of top people in Alzheimer’s research, who have built their careers on amyloid being the answer and who have everything to lose (not least lucrative consultancies within the industry) if it turns out not to be. Alternative hypotheses have been pursued, but the authors of such research find it hard to get funding unless their work is framed in terms of amyloid. Like Erin Smith, Piller shows us the power of personality and social connections, as charismatic scientists – he calls them the “amyloid mafia” – exert disproportionate influence over who gets funded, whose results are published, and in some cases what their labs’ experiments show. Noting the revolving door between the FDA and biotech companies – “members of the FDA advisory committees earn hundreds of thousands of dollars in consulting fees and perks from companies whose drugs they previously voted to approve” – he lays into government, universities and journals for being secretive, inept and riddled with conflicts of interest. Tools meant to help, such as institutional investigations and journal retractions, are not being used effectively.

Scientists know that there’s a problem, and not just with Alzheimer’s drug trials. Many disciplines have found themselves beset by damaging misinformation, political hostility and funding problems; by paper mills spewing out fake data, problems getting enough reviewers and claimed results of experiments that can’t be replicated; by mental health crises, accusations of bias and discrimination, even high-profile abuse and fraud cases. Piller finds some hope in the recent development of AI tools to help assess the vast quantities of material; in the growth of organizations such as Retraction Watch, which collates data on published articles and pre-print withdrawals; and in the forum PubPeer, where scientists can call out doubtful material. That said, he is wary of “integrity sleuths” being overly aggressive about minor mistakes, a practice that verges, he says, “on trolling”. His recommendations include changing the “publish or perish” culture, which rewards publication quantity over quality, teaching more research ethics, emphasizing open science – for instance, insisting that publications include the full data, so that others can check findings – and developing better ways of spotting serial offenders.

Doctored recommends more and better self-regulation, tailored to the specific area of research. The book is almost entirely US-centric, and it takes stamina and motivation to keep track of the complex narratives. For people familiar with dementia research, it offers a horror show of big names getting knocked off their pedestals, but they are not household names and most people outside the field won’t have heard of them. Doctored is a book aimed primarily at scientists, unlike Valley of Forgetting, an easier read with more human interest. One tells us how the system generates perverse incentives, the other shows us the effects on participants.

I hope both books will prompt action, or at least reflection. From the flurry of recent features on science reform in top journals, perhaps Doctored has already done this. Its recipe of self-regulation certainly sounds good – unless, of course, you’ve been reading similar exhortations for decades. There are more radical options being discussed than those Piller mentions, such as separating journals’ roles in spreading knowledge from their use as status indicators, and shifting power away from big science publishers (whose profits can match those of big tech firms) by having academic societies take over top outlets. But these options still envisage scientists acting to reduce the forced obsession with status and money – “forced” because even tenured scientists’ status depends on continuous grant funding.

Will any of these options work? That depends on whether scientists are in fact as rational as we, and they, like to think. If they are, then they will act to reconfigure clinical medical – and other – research as a public service, in which patients and participants are contributors, not assets. If they are not, then the lesson from history is that things will get worse until investors and the public notice, become furious and demand external assessors with genuine power to drive change. That will mean real outsiders: not just other scientists, not biotech company lawyers or industry specialists or even bioethicists or doctors, but patient groups, advocates for participants and perhaps even citizens’ assemblies. Only when they have powerful voices on the decision-making bodies – ethics and advisory committees, grant review boards – can science achieve the separation of powers that it needs to serve society better.

“Was this the best that could be done for the families?”, Jennie Erin Smith asks in Valley of Forgetting. “Or was it a way to keep the research gears turning, the money and data flowing?” Her book and Charles Piller’s differ in tone and approach, but both are valuable guides to engaging with that essential question.

Kathleen Taylor’s books include Dementia: A very short introduction, 2020, and The Fragile Brain: The strange, hopeful science of dementia, 2016

The post Trials and error appeared first on TLS.