Edzard Ernst is well known as an “insider” in the world of alternative medicine who has come to see the need for rational, scientific analysis and evidence to justify inflicting any treatment of whatever provenance on patients. His book, with Simon Singh, Trick or treatment: The Undeniable Facts About Alternative Medicine, which I have previously reviewed, is a model for how to look scientifically and CAM, and it is an indispensible resource for anyone interested in the truth about alternative approaches.
Ernst has a new short article in the oxymoronically named journal Evidence-Based Complementary and Alternative Medicine. The unfortunate title of this otherwise useful article is Winnowing the Chaff of Charlatanism from the Wheat of Science.
In his article, Ernst suggests a possible strategy for evaluating medical claims. It is not especially novel, but it’s a clear introduction to some of the key methods by which nonsense can be distinguished from real science.
The first test is Karl Popper’s time-honored concept of falsifiability. At its simplest, this just means that it is possible for a claim to be proven false. If, for example, all disease is claimed to be caused by vertebral subluxations, then showing that any disease has another cause demonstrates this claim to be false. Many CAM approaches rely on mystical “energies” that can only be intuited, and many practitioners claim that the methods or even the skeptical attitudes of scientific researchers interfere with their treatments, so these treatments can never be tested and are, in either theory or practice, unfalsifiable. Unfortunately, many CAM proponents will argue that their approaches are theoretically falsifiable but we simply do not yet have the technology or methods to test them, so they squirm out of meeting this criterion.
Ernst then suggests moving on the test of plausibility. This is one of the concepts which separates the close but not identical domains of evidence-based medicine from more generally science-based medicine. If an idea is inconsistent with well-established scientific knowledge, then we can reasonably dismiss the idea right there, without wasting resources testing it empirically. The theories of Alternative Flight are sufficiently implausible that no one has serious suggested we test them. This criterion will also not successfully contain all pseudoscientific nonsense, because it is always possible to claim, with some justification though disingenuously in most cases, that what is plausible is based on current knowledge, and this is incomplete and imperfect. Powered human flight would have seemed extremely implausible 500 years ago, and yet it is possible, so plausibility is not always by itself a sufficient basis for final judgment of a claim.
Finally, Ernst moves on to warning signs of quackery as another criterion to help separate real science from pseudoscience. I have written about these a number of times (my own partial list, and Dr. Walt’s list), and there is significant overlap between other lists of such signs and Dr. Ernst’s. He refers to advocates of quackery as Proponents of Absurd Claims (PACs), and he describes some of the red flags they often wave about:
Intolerance: Many PACs are consumed with evangelic zeal and find it hard to accept or even consider well-reasoned criticism or debate. Anyone who has tried to have a rational discussion with someone making irrational claims will have experienced this phenomenon. As intolerance can exist everywhere, its discriminating power is, of course, low and further criteria are required.
Selectivity: Most PACs tend to ignore facts that contradict their own assumptions. Instead they favor selected anomalous data or anecdotal findings which apparently support their notions. Clinical trials, for instance, are designed to overcome the many biases associated with simple observation. Whenever their results fail to confirm their belief, PACs insist that, for this or that reason, case reports, observational studies or years of experience are preferable. In arguing their case, PACs often seem to first formulate their conclusions, then selectively identify those bits of information that apparently confirm them.
Paranoia: Many PACs believe in conspiracy theories which posit that ‘the establishment’ is determined to suppress their views or findings. The world wide web, for instance, is full with suggestions that ‘big pharma’ is conducting a campaign against ‘alternative cancer cures’ such as laetrile or shark cartilage. Anyone who points out what the evidence really shows is likely to be accused of being part of the conspiracy.
Misuse: Some PACs misuse science, for instance, by using terminology like energy, chaos theory, quantum mechanics or entanglement in inappropriate contexts, devoid of their actual meanings. Accepted standards are rejected and double standards are proposed for their own area. In case this strategy fails, other means might be employed, including outright fraud.
I wouldn’t suggest Ernst’s criteria are foolproof, and neither does he, but it is a nice simple starting point, especially for those not steeped in skepticism and already familiar with the many dodges, obfuscations, and deceptive marketing techniques of the CAM world.
I’ve certainly come across all those hallmarks in forum discussions about raw food diets, homeopathy etc. Unfortunately, just recognising them does nothing to change the situation of total incomprehension which comes about when one is involved in such a discussion. My own solution is now simply to leave off at the first signs of these giveaways coming up: since I’m not a health professonal for humans or other species, I don’t have the responsibility to make sure people don’t injure their own health or the health of others: God help those who do!
in my experience,god does not help at all!sadly,even rational discussion doesn’t either.this is one reason why we should not leave it to those who have “official”responsibility to do that job.
Thank you, Dr. Ernst, for participating in this discussion, as well as for all the work you’ve done to advance the cause of science-based medicine.
Anyone struggling against irrationality of whatever kind will eventually have to make peace with the limited effects their efforts are likely too have. I don’t expect to change the world, or my profession through my efforts, but if I can retard the proliferation of irrationality even a little, or direct even a few people towards medicine that will really benefit them and away from quackery, then I figure that must be worth something.
Has anybody taken into account PSYCHOLOGY? Humans and animals all have it. And its effect on the body and every cell through thoughs/hypothalamus/endcrine/nervous system?
Yes, psychology is irrational and can be unpredictable but it exists. Its effect on the body for better or worse exists and is real, not fake.
Sorry to be the one to break the news!
Kim,
Sorry, but if you’ve read many of my posts or any of the extensive references I refer to, you would find a great deal about psychology. The problem, is that our psychology is far more likely to lead us astray, making us believe what isn’t true, than it is to relieve our illness or suffering. Try these:
Don’t Believe Everything You Think: The 6 Basic Mistakes We Make in Thinking by Thomas Kida
http://www.skeptvet.com/index.php?p=1_13_Why-We-re-Often-Wrong
http://skeptvet.com/Blog/2010/04/the-dunning-kruger-effect-why-incompetence-is-blind-to-itself/
http://skeptvet.com/Blog/2009/10/cognitive-dissonance-and-evidence-based-medicine/
http://skeptvet.com/Blog/2010/03/why-people-believe-weird-things-by-michael-shermer/
Thanks to scientists. we don’t believe in religion any more. Huge department stores have become our Mecca. Now we are told CAM is bunk and not to believe it.
What is the alternative? 5 minutes of a rushed and hassled GP’s time after waiting for an hour and a couple of pills. More often than not, it is not what we need to relieve our stresses of the 21st century life when we are all losing our jobs and pensions.
CAM gives us the time and the attention and relaxation we are craving for. 1 hour of reiki leaves a person relaxed and happy for days. Someone listening to them for an hour about their past stresses and details of their life and giving them sugar pills makes them feel heard and valid.
And we all know that stress causes disease. Surely anything that relieves it is a welcome addition to our health repertoire.
Even by GP’s?
Here is an excerpt from the Independent on clinical trials of CAM:
“Initially, the plan was to run clinical trials of alternative therapies in the “laboratory” in the basement of the large Victorian house where his department is based. One of the first was of spiritual healing. Ernst describes it as “the most interesting study I have ever been involved in”. His idea was to compare healers with actors, and to compare a healer placed behind a door, out of sight of the patient, with the effect when there was no one behind the door (a tape recording of someone breathing and shifting in their chair created the illusion that the healer was present).
All the patients in the double-blind, randomised trial reported feelings of warmth and tingling, suggesting they were experiencing the effects of healing, even when it was delivered by an actor – or a tape recorder. More remarkably, among the patients, there were five in wheelchairs, four of whom found the effect so powerful that they were able to get up and walk.
Ernst recalls going home and telling his wife how he was going to be mocked by his scientific colleagues as the man who worked miracle cures. Instead, it emerged that the four patients who had got up and walked were equally distributed between the four arms of the trial. In other words, the results were an early lesson in the power of placebo.”
I bet the four people in the wheelchairs who got up and walked as a result of the trials who would not have walked otherwise, did not mind if it was placebo or not.
They were there in the first place because the doctors told them they could not walk even when it was proven that they could by believing in it.
This was during the trials into spiritualism. Call it “spiritualism”, “quackery”, “placebo”, “psychology”, whatever, if it can make invalids walk, should it not be available on the NHS in some shape or form that everybody is happy about?
What matters is not the egos and who is right and who is wrong, but that the patient walks, surely?
Kim,
Science is about figuring out what is true about the physical world. How can it be bad to know what is true in medicine, or good to believe things that don’t actually make you better? That makes no sense. Obviously, with your talk of religion and consumerism you have an agenda far bigger that just figuring out what kinds of medical treatments work and what don’t, and your objection to my criticisms of CAM have more to do with your philosophy than any interest in what is actually true about CAM. Unfortunately, what we want to believe doesn’t change what is.
As for the alternative, the exaggerated strawman you create out of mainstream medicine is not only not the real alternative, it has nothing to do with what will really make people well and what will only fool them into thinking they are getting real help. There’s no doubt the economics and institutions of halthcare are deeply flawed, and scientific medicine rarely does as good a job as it should in treating people’s feelings and the psychological aspects of illness. This does drive people to CAM. But the answer to the flaws in conventional medicine is not to abandon it for myths and fairy tales, but to keep what works about scientific medicine and change the healthcare system to better meet people’s needs. You create a false, and dangerous, dichotomy which can only lead us back to the dark ages of ineffective folk medicine.
Finally, nobody who is paralyzed gets up and walks because of placebo effects or the power of the mind. Faith-based medicine is wrong precisely because it tells such lies and leads people away from seeking real treatments that might actually help them. Science can’t fix everything, but it has fixed more in a couple hundred years than faith has in the entire history of humanity, and anybody old enough to remember polio or the era before antibiotics can tell you why we shouldn’t be mythologizing the past or tryig to retreat from the hard-won knowledge science has given us. It is possible to live a rich, meaningful, and joyful life AND have healthcare that really works, and your suggestion that the two things are incompatible is not only incorrect but quite sad.
I was merely referring to Ernst’s own experiments (Independent, Tues 22 April 2008). Unless they are misreporting it, anybody who read the article would react the same way as I did.
I entirely agree on how important the NHS is in other respects but unfortunately they have not got the time or the money to spend on frivolous complementary therapies. If people did not find relief and relaxation and some kind of spirituality in them, they would not be flourishing.
What I object to is this dictatorial attitude of medics that we should obey them or else, and their efforts to shut down anyone who opposes them. They have become the new Popes of the scientific religion!
Maybe this might put things in perspective:
The basic timeline is a 4.5 billion year old Earth, with (very approximate) dates:
3.8 billion years of simple cells (prokaryotes),
3 billion years of photosynthesis,
2 billion years of complex cells (eukaryotes),
1 billion years of multicellular life,
600 million years of simple animals,
570 million years of arthropods (ancestors of insects, arachnids and crustaceans),
550 million years of complex animals,
500 million years of fish and proto-amphibians,
475 million years of land plants,
400 million years of insects and seeds,
360 million years of amphibians,
300 million years of reptiles,
200 million years of mammals,
150 million years of birds,
130 million years of flowers,
65 million years since the non-avian dinosaurs died out,
2.5 million years since the appearance of the genus Homo,
200,000 years since humans started looking like they do today,
Less than 100 years since evidence based medicine (Not that we are not grateful for it!)
We as the consumers only ask not to be patronised by either parties, we are on the whole intelligent enough to make up our own minds.
Again, we disagree. I certainly try not to patronize anyone, but it is not patronizing to say that one thing is true and another isn’t. It is not patronizing to tell my clients when I firmly believe they are using an ineffective, possibly harmful treatment on their pets. Arrogance is to trust one’s own judgement so much that “seeing is believing” and evidence is less improtant than faith. Humility is to realize that we all have flaws in how we make judgements, and that the reason we need science to help us is because of these flaws.
As a doctor, I don’t dictate anything to anyone. I frequently tell my clients that because they are paying me for my advice I am obliged to give it to them honestly. Nothing requires them to take it. But if you’re suggesting I shouldn’t point out what I believe are the real facts about specific medical approaches, then isn’t it you who is trying to shut me down? If you’re objecting to the very fact that I am criticizing CAM, rather than to the substance of my criticism, who is trying to control the debate?
Intelligence is not what makes sound medical judgements. It is a reliance on proven methods of evaluating treatments. The body, and its health and disease, are astonishingly complex, and far too intricate for any one person to fully udnerstand. The CAM therapists who claim their approach is the one true cause and/or treatment for all disease are deluded and it is they who are showing their lack of humility. Science says health and disease are the result of the complex interaction of many factors, a lot of which we don’t yet understand, and so medicine is an imperfect and ever developing process. A thousand years of faith and tradition doesn’t make our intuitions, gut feelings, or personal experiences any more reliable, since human flaws and weaknesses don’t disappear just becuase you get a lot of people to believe the same thing for a long time.
A thousand years? Try 2.5 million years since the appereance of genus homo! So you are saying nature got it wrong for 3.8 billion years since we evolved from the first prokaryotes?
I am not a “pac” or a practioner or believer in “alternative medicae”. However,alternative medicine IS all wrapped up in belief system, in some casese it is an independent belief system, and science has no business evaluating belief systems. Science, by its own wise and candid admission, is INCAPABLE , by and large, of evaluating the relavtive merrit or efficacy belief systems. I would like someone to tell me how to spot “scientifically” supported medical nonsense..that is, any opinion which has been scientifically evaluated, clinicly tried, endorsed by WHO and passed into law by state legislatures… and yet remains ambiguous, unproven, probably unproveable, and of little value to human health and quality of life..such chemical treatments that “may” cause remission or extend life by as much as 5%….the gerneral piling on of smoking to be a “possible” cause of everything (including hiccups)…refusals to treat pain with (demonstrably effective) narcotic drugs in favor of hormone therapies or expensive (non generic) synthetic substances ..how about the constant “abuse” of the scientific spirit if not the scientific method (not usually anyway) in order to make enormous profits? someone please tell me how to avoid this kind of “medical nonsense”
I think it’s important to separate belief systems from claims about the world made on the basis of belief systems. Science may not be a useful method to apply to what happens inside someone’s head, but if they claim Herb X can cure Disease Y, science is by far the most reliable way to evaluate the truth of that claim. The problem is not that CAM is predicated on beliefs about the world, since all of our behaviors are. The problem is that CAM advocates often consider these beliefs sufficient justification for claims about the world in and of themselves, and they reject any attempt to validate the truth of these claims or their beliefs. Those of us dedicated to science-based medicine are no different from CAM practitioners in how we form beliefs and in all the cognitive biases that afflict our ability to evaluate the truth of these beliefs. Where we are different is in accepting this and acknowledging the need for systematic methods to account for these belief-forming and maintaining systems that lead us into error.
How do we identify bad practices based on mistakes or greed in scientific medicine? Through the emthods of science, of course. We know with certainty, for example, that funding sources influence the outcome of clinical trials because the subject has been studied, and this is leading to more transparency and better controls. The great thing about the scientific method is that it doesn’t require you to see your own mistakes, because if you fail to somebody else will point them out for you! For all the legitimate problems so frequently lamented concerning pharmaceuical companies and their products, they are undoubtedly more reliable than dietary supplements or herbal remedies because the government requires at least some scientific assessment of them. More can and should be done, but the problem lies in our failure to accept the need and the lack of political will to give science its proper place in determining the facts about the world upon which we base our policy decisions, not in the methods of science themselves.
So is science perfect? No. Are scientists any better as individuals at evaluatingt he truth of their beliefs than CAM practitioners? No. Do the methods of science correct for our individual failings and give us the best chance at a true understanding of reality? Absolutely! Spotting medical nonsense, whether in CAM or in mainstream medicine, is primarily about looking at the claims made and whether they are compatible with scientific evaluation or whether science is considered no more than post hoc icing on the cake confirming what we “already know” or even as fundamentally unable to distinguish truth from fantasy. If a belief system does not acknowledge the epistemological superiority of science over tradition, personal judgement, intuition, and so on, then it is less likely to make truthful claims about the world.