Edzard Ernst is well known as an “insider” in the world of alternative medicine who has come to see the need for rational, scientific analysis and evidence to justify inflicting any treatment of whatever provenance on patients. His book, with Simon Singh, Trick or treatment: The Undeniable Facts About Alternative Medicine, which I have previously reviewed, is a model for how to look scientifically and CAM, and it is an indispensible resource for anyone interested in the truth about alternative approaches.
Ernst has a new short article in the oxymoronically named journal Evidence-Based Complementary and Alternative Medicine. The unfortunate title of this otherwise useful article is Winnowing the Chaff of Charlatanism from the Wheat of Science.
In his article, Ernst suggests a possible strategy for evaluating medical claims. It is not especially novel, but it’s a clear introduction to some of the key methods by which nonsense can be distinguished from real science.
The first test is Karl Popper’s time-honored concept of falsifiability. At its simplest, this just means that it is possible for a claim to be proven false. If, for example, all disease is claimed to be caused by vertebral subluxations, then showing that any disease has another cause demonstrates this claim to be false. Many CAM approaches rely on mystical “energies” that can only be intuited, and many practitioners claim that the methods or even the skeptical attitudes of scientific researchers interfere with their treatments, so these treatments can never be tested and are, in either theory or practice, unfalsifiable. Unfortunately, many CAM proponents will argue that their approaches are theoretically falsifiable but we simply do not yet have the technology or methods to test them, so they squirm out of meeting this criterion.
Ernst then suggests moving on the test of plausibility. This is one of the concepts which separates the close but not identical domains of evidence-based medicine from more generally science-based medicine. If an idea is inconsistent with well-established scientific knowledge, then we can reasonably dismiss the idea right there, without wasting resources testing it empirically. The theories of Alternative Flight are sufficiently implausible that no one has serious suggested we test them. This criterion will also not successfully contain all pseudoscientific nonsense, because it is always possible to claim, with some justification though disingenuously in most cases, that what is plausible is based on current knowledge, and this is incomplete and imperfect. Powered human flight would have seemed extremely implausible 500 years ago, and yet it is possible, so plausibility is not always by itself a sufficient basis for final judgment of a claim.
Finally, Ernst moves on to warning signs of quackery as another criterion to help separate real science from pseudoscience. I have written about these a number of times (my own partial list, and Dr. Walt’s list), and there is significant overlap between other lists of such signs and Dr. Ernst’s. He refers to advocates of quackery as Proponents of Absurd Claims (PACs), and he describes some of the red flags they often wave about:
Intolerance: Many PACs are consumed with evangelic zeal and find it hard to accept or even consider well-reasoned criticism or debate. Anyone who has tried to have a rational discussion with someone making irrational claims will have experienced this phenomenon. As intolerance can exist everywhere, its discriminating power is, of course, low and further criteria are required.
Selectivity: Most PACs tend to ignore facts that contradict their own assumptions. Instead they favor selected anomalous data or anecdotal findings which apparently support their notions. Clinical trials, for instance, are designed to overcome the many biases associated with simple observation. Whenever their results fail to confirm their belief, PACs insist that, for this or that reason, case reports, observational studies or years of experience are preferable. In arguing their case, PACs often seem to first formulate their conclusions, then selectively identify those bits of information that apparently confirm them.
Paranoia: Many PACs believe in conspiracy theories which posit that ‘the establishment’ is determined to suppress their views or findings. The world wide web, for instance, is full with suggestions that ‘big pharma’ is conducting a campaign against ‘alternative cancer cures’ such as laetrile or shark cartilage. Anyone who points out what the evidence really shows is likely to be accused of being part of the conspiracy.
Misuse: Some PACs misuse science, for instance, by using terminology like energy, chaos theory, quantum mechanics or entanglement in inappropriate contexts, devoid of their actual meanings. Accepted standards are rejected and double standards are proposed for their own area. In case this strategy fails, other means might be employed, including outright fraud.
I wouldn’t suggest Ernst’s criteria are foolproof, and neither does he, but it is a nice simple starting point, especially for those not steeped in skepticism and already familiar with the many dodges, obfuscations, and deceptive marketing techniques of the CAM world.