A recent editorial from Trends in Molecular Medicine does an excellent job explaining why clinical trials testing the effects of highly implausible therapies, like homeopathy and energy medicine, are not an example of evidence-based medicine but of science tricked into serving as advertising.
Gorski DH. Novella SP. Clinical trials if integrative medicine: testing whether magic works?
Trends in Molecular Medicine 2014; In Press
Clinical trials are the last in a series of steps to evaluate potential therapies, and those that fail at earlier levels, or that have not been subjected to appropriate pre-clinical testing, should not be inflicted on patients. It is unethical and dangerously misleading to conduct such clinical trials.
When poorly designed and conducted, such trials generate spurious positive findings that mislead. When properly designed and conducted, they most often generate negative findings (read Snake Oil Science for numerous examples). However these negative studies do nothing to discourage the use of these therapies among proponents. Conducting such studies in hopes of finding a rare exception in which they prove effective and long-standing scientific principles are overturned is not an efficient use of limited resources for testing novel therapies. We should focus our energy on therapies that don’t conflict with well-established principles of basic science and which are promising on the basis of pre-clinical research. Most of these will fail at the clinical trial level as well, of course, but at least the odds are somewhat better than for trials on what amounts to magic.
This is relevant to veterinary medicine as well as human medicine, though as usual the scope of the problem is smaller. For example, conducting trials of homeopathic remedies for cancer (Banerji Protocol as Adjunct Nanotherapy for Treating Canine Appendicular Osteosarcoma: A Double Blind, Randomized Study Design), despite the overwhelming evidence that homeopathy is inconsistent with fundamental and well-established laws of physics and chemistry and has consistently failed pre-clinical and clinical testing (1, 2), is simply unjustifiable ethically and scientifically. It is an irrational use of limited resources for investigation of novel treatments.
Here is part of Dr. Gorski and Dr. Novella’s conclusion from the editorial:
In RCTs [randomized controlled clinical trials] testing modalities with low pre-test probability (i.e., low plausibility), confounding effects are vastly magnified, easily producing false-positives. In these days of extreme scarcity of research funding, it is difficult to justify spending precious research dollars carrying out RCTs of treatments where the likelihood of producing a true positive trial is so low and that have real risks that can lead to outcomes like the TACT or pancreatic cancer trials. All clinical trials, not just RCTs, should be based on scientifically well-supported preclinical observations that justify them, preferably with biomarkers to guide patient selection and follow-up. Until specific CAM and IM modalities achieve that level of preclinical evidence, RCTs testing them cannot be scientifically or ethically justified. That is science-based, rather than evidence-based, medicine.
Sadly, many of the vets I talk to, even those who are sincerely committed to science and evidence-based medicine, appear to still believe that conducting clinical trials on even the most egregiously pseudoscientific therapies is appropriate and will make a difference in whether these therapies are chosen by clinicians. There was a time when I believed this. Unfortunately, this view doesn’t reflect the reality of alternative medicine research.
The problem is that the clinical trial literature, which is often questionable in mainstream human medicine and frequently poor quality in veterinary medicine overall, is consistently abysmal when it comes to alternative medicine research. I have previously reviewed the positive literature presented by the Academy of Veterinary Homeopathy last year in defense of homeopathy, for example. It took many hours of locating and reading dozens of papers in detail to determine that this evidence was thoroughly unconvincing. And yet these poor quality studies are constantly put forward as supportive of continued clinical use for these modalities, and the majority of clinicians simply don’t have the time or interest to go beyond the abstracts and appraise them critically.
Such studies create the appearance of legitimacy, which is used to make such modalities more widely acceptable even when the literature doesn’t actually support efficacy. In some cases, for example in China, negative trials of alternative therapies are virtually never published, which illustrates the uniquely unreliable nature of this literature. It is effectively marketing rather than science.
So in light of this, clinical trials of highly implausible treatments don’t accomplish their ostensible purpose. They don’t accurately evaluate safety and efficacy, and they don’t discourage the use of modalities that are pretty clearly ineffective. The work of the National Center for Complementary and Alternative Medicine (NCCAM) is another great example. This agency has spent billions of dollars testing therapies that have not met basic preclinical plausibility criteria. The vast majority of these trials have found no evidence of efficacy. And these results have had little effect on the popularity or use of the modalities studied. (3, 4, 5).
Now of course by saying “alternative medicine,” I am lumping together many very different approaches, most of which are mutually incompatible theoretically as well as sometimes inconsistent with the mainstream scientific understandings of health and disease. But I am not suggesting none of these deserve to be evaluated in clinical trials. Many herbal remedies, manual therapies, dietary manipulations, nutraceuticals, etc. are quite plausible and can and should be evaluated scientifically like any other novel therapy.
But when talking about methods which could only work if fundamental scientific principles upon which much of our technology and medicine is based is wrong (e.g. energy medicine, homeopathy, the theoretical structures of TCM, etc.), or which have effectively been disproven already (e.g. homeopathy, magnet therapy) I can see nothing to gain from clinical trials. They will almost certainly be of low quality and evidentiary value, and even if negative they will not discourage the use of such therapies.
And yes, there are plenty of different evidentiary problems with conventional medicine. Pharmaceutical companies often shamelessly manipulate data, treatments like hyperbaric oxygen and stem cell therapy are rushed into use well before sufficient evidence is available to justify this, and so on. But clinical trials examining therapies which either can’t work, unless much of established science is wrong, or which have already been extensively tested and failed to show evidence of efficacy, are simply wasteful and misleading.
And from an ethical perspective, it is truly disturbing to see veterinary cancer patients treated with homeopathy, especially when otherwise respected individuals and institutions are involved.. I think this is unfair to the patients even in the context of a clinical trial when there is so much evidence in human medicine that there have been systematic reviews of the systematic reviews of research on homeopathy (which, as I have discussed previously, have shown that the positive results come only from inadequately controlled studies).
It seems fundamentally unethical to do work such as this in the name of science when it is clear that nothing short of a miraculous overturning of established science could allow for a positive result, and it is equally clear that no one who believes in homeopathy is going to give it regardless of a negative outcome in such a trial. Such research wastes resources, creates no clarity, does not allow for real falsification, and exposes patients to therapies for which there is really no reasonable expectation of a benefit.