The staggering death toll of scientific lies

> One crucial question he studied: Should you give patients a beta blocker, which lowers blood pressure, before certain heart surgeries? Poldermans’s research said yes. European medical guidelines (and to a lesser extent US guidelines) recommended it accordingly.

What the guy did was clearly wrong but it’s a slightly tenuous causal chain between that and 800,000 deaths. Questions may be asked, for example, about whether the medical guidelines should have been based on studies that seemingly had a single point of failure (this one corrupt guy).

There’s an extremely toxic (and ironically very anti-scientific) culture of “study says it so it’s true” that permeates medical and scientific fields and the reporting thereof. Caveats and weaknesses in the primary research get ignored in favor of abstracts and headlines, with each layer of indirection discarding more of the nuance and adding more weight of certainty to a result that should in truth remain tentative.

Prosecuting one type of bad actor might not make a lot of difference and might distract from the much larger systemic issues facing our current model of scientific enquiry.

Independent replication is the cornerstone of Karl Popper's formulation of the scientific method. If we were really holding to philosophical basis for how science is supposed to work, then we'd be careful to consider results that have only been demonstrated by one laboratory to be tentative at best, and be cautious about making policy decisions based on it until there has been truly independent replication.

Physicists seem to be really good about this, and many other aspects of implementing the scientific method too.

I wish the other sciences would get on board. It would eliminate almost all the chronic problems that plague biological and social sciences: falsification, p-hacking, failing to notice honest methodological mistakes, outright fraud, etc.

I fear that the problem is, we can't get there from here for social reasons. The people at the top of these fields - the ones who drive culture in academic institutions, set publication standards for journals, influence where grant money is allocated, etc. - all got there by using sloppy methods and getting lucky. I think that, on some level, many of them know it, and know that fixing the rotten core of their field inevitably involves subjecting their own work - and, by extension, reputations - to a level of scrutiny that it is unlikely to survive.

This article sort of just glosses over any reasons why existing laws aren't enough. It states that existing scientific fraud is rarely prosecuted (e.g. under fraud statutes), so it seems to me the right course of action should be to prosecute it!

In the US at least, it's nearly impossible to commit this kind of malfeasance without committing federal wire fraud - faked research would nearly always be part of a grant application, at least eventually, for example.

Plus, I'm surprised some enterprising lawyers haven't at least tried some massive class action lawsuits. The actual researcher may not have much to go after, but surely their institutions would. If you can get huge class action payouts for the dubious connection of talc in baby powder to cancer, why can't you get a payout here where (a) the malfeasance was intentional from the get go and (b) the harms are unambiguously clear from follow-up meta-analysis studies.

I guess I would like to understand if there is some fundamental reason that existing statutes aren't enough before adding laws.

What is really interesting is looking at the meta analysis cited in the Vox article:

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3932762/

This reaches the conclusion that beta blockers are harmful. However, if you look at the meat analysis, specifically figure 2, you find that the conclusion is mainly driven by a single trial - the 2008 POISE trial.

If you go to the POISE trial: https://www.thelancet.com/journals/lancet/article/PIIS0140-6...

You find that they discovered fraud in at least some of the hospitals:

" Concern was raised during central data consistency checks about 752 participants at six hospitals in Iran coordinated by one centre and 195 participants associated with one research assistant in three of 11 hospitals in Colombia. On-site auditing of these hospitals and cases indicated that fraudulent activity had occurred. Before the trial was concluded, the operations committee—blinded to the trial results at these hospitals and overall—decided to exclude these data (webappendix 1). "

We have an important question - should pre-op patients be given beta blockers - and the largest, most definitive trials to answer that question have at least some taint of fraud.

Why is it that so many of the PhDs and MDs out there are unable to tell when someone's lying to them about their very own areas of expertise?

Either they're not that smart or the processes aren't very good -- though no single researcher is responsible for their field's poor processes. Either way, we shouldn't assume that any one PhD or MD recipient is an expert until something changes. Degrees, on their own, don't signify expertise or credibility.

A scientist that causes, through willful fraud, the death of people seems to be guilty of something like manslaughter. Using fake data is a pretty clear-cut example of willful fraud, and a reasearcher fudging data over such a life and death question should 100% be held accountable.

Scientists making errors in good faith should on the other hand be insulated from any kind of liability.

I would say, a better approach would be to have better checks and balances in play. Maybe he falsified research, why was it recommended as a standard procedure by European Medical Guidelines without proper evaluation? I would hold them criminally liable before I hold the researchers liable.

The nice thing about running bad studies and cooking the p is that you can usually reap the rewards immediately and for a long time because reproducing studies is lame and no one wants to do it.

Suggestions:

1) Any research institutions that receive government funding are required to spent 10% (or 15% or 20% or whatever) of their total budget on replication. If they don't, they stop receiving any government funding.

2) When citing a paper scientists are required to include any replication studies, both successful and not successful.

This would hopefully lead to more replication studies being done, even if it doesn't answer the question of what to do with a study until it's been replicated sufficiently.

The second part would help us guess the validity of a paper. Papers that base their central premise on studies with multiple independent replication would probably be a bit more trustful than papers based on unverified studies.

A close friend of mine was doing her PhD in nutrition (in Germany). She asked me to have a look at the math (statistics) and it was a monstrosity.

I told her that I cannot read that further because she either won't get her PhD or I will be morally wounded.

She asked me for some examples of errors and to each of them she was saying, with evidence, that this is what "everyone does".

This was nutrition, something that is at least to some extend innate so there won't be disasters (I hope). The same thing with pharma is a disaster hanging by a thread.

Compare the 2011-14 L'Aquila earthquake trials, when Nature magazine and most of the I-love-science claque rallied behind the scientists who gave incorrect reassurances to L'Aquila residents https://en.wikipedia.org/wiki/2009_L%27Aquila_earthquake#Pro... . Is it that the facts are very different in this case, or it more that attitudes have shifted since then?

Yes the people setting guidelines based on non-reproducible or unreproduced studies should be held accountable. But they are part of the same power structure that makes laws, so don't hold your breath.

Literally anybody can write bullshit and anybody with some cash or connections can get it published, deciding to make it a medical guideline because the text and it's metadata looks a certain way is basically as competent as just using ChatGPT.

This is a drop in the bucket. The real crime is rich corporations using “science” to enrich themselves. Examples: denying that smoking causes cancer. Demonising fat while promoting excess sugar in processed food, and look at how many organisations are denying man-made climate change…

This seems more like an institutional failure more than anything.

I recall there was a discussion on HN a few years back about the Alzheimer plaque connection being established on fake data.

People might not want to hear it but it is going to keep being an issue - we shouldn't force people to "follow the science" when it comes to medicine. Scientists - and science as a whole - do not have the moral standing that their opinions justify authoritarianism. People should make their own decisions about whether they trust the remedies involved.

It happened to agriculture in Stalin’s USSR

https://en.wikipedia.org/wiki/Lysenkoism

A big problem in today's political climate is if you question certain scientific findings, at best you get shouted down and at worst you get branded a far right fascist and have your career put at risk.

Fundamentally scientific rigor and accuracy is often misaligned with larger societal norms and values - a pharmaceutical corporation with a profit-making pill doesn't want to hear about the 1% of users who suffer catastrophic medical conditions as a result of using their product (e.g. Vioxx with 88,000 heart attacks and 38,000 deaths out of 107,000,000 prescriptions from 1999-2004). Similarly the Soviet Union's Lysenko tailored his research results to align with Stalinist ideology on adaptability, thereby securing his position in the academic structure - behavior that was remarkably similar to that of Anthony Fauci regarding the origins of Sars-CoV2 and the efficacy of the various treatments and vaccines that were so highly profitable to the corporate pharmaceutical sector. Reckless virology research that he supported caused a global pandemic that cost at least $10 trillion in economic damage and took millions of lives - but admitting that opens the door to liability, so no.

I've worked with both ends of the spectrum - fraudulent tenured PIs at leading research universities are not that rare, but highly skilled and reliable PIs are more common. The fundamental difference always seems to be record-keeping - frauds aren't interested in keeping detailed records of their activities that can be used by others to replicate their work (since their work is non-replicable). In contrast, the reputable researcher will want such detailed records for various reasons, including defense against false claims of fraud or incompetence, which is quite common if the research results are not aligned with corporate profit motives in areas like pharmaceuticals, fossil fuels and climate, environmental pollutants, etc.

If the powers that be really wanted to reduce research fraud, the easiest way is to make detailed record-keeping a requirement of federally-funded research, with regular audits of lab notebooks and comparisons to published work. This matters, because the problem is set to get worse with the spread of AI tools that make it possible to generate hard-to-detect fake datasets and images. In the past a great many frauds were caught because their fake data generation was so obvious, often just a copy and paste effort.

Have anybody thought about asking the same questions about COVID-19 prevention and treatment research?

Scientific fraud or medical fraud? Are we tarring all of science because of fraud in one sub-field (medicione)? How about social science fraud, where poor economic policies can cause millions to starve?

Before I get pilloried for whataboutism, all I'm trying to illustrate is that the title is a hyberbole. Fraud in medical research is definitely a problem leading to serious consequences for patients everywhere. Let's just call it what it is.

One solution would be bonding. If you publish a scientific result, put up some money with a bonding firm, perhaps for a specified period of time. If someone successfully identifies fraud, they get the money. The more money you put up, the more confident you are. This also provides an incentive for fraud finders.