I have been participating in a discussion on the Veterinary Information Network (VIN) about the AVMA’s failure to take a scientific and principled position on homeopathy, and it prompted me to outline a few thoughts on the subject of whether more research on some CAVM therapies is always worthwhile, and what the role of plausibility plays in evaluating such therapies. I’m reposting these thoughts here since these subjects come up often on this blog.
I once thought the answer to separating useful CAVM from nonsense lay in more research. Having spent several years investigating CAVM and the claims and rationales made for it, I realize this is not actually that likely to be helpful, for many reasons. One is that much of CAVM is based on philosophical and ideological foundations completely incompatible with science, and the promotes of these “metaphors” don’t have any interest in applying science to their practices other than its potential value as a marketing tool for creating broader acceptance and legitimacy. (I have written about this before.)
For scientific research to be useful in separating true from false, we must be willing to accept negative results and abandon therapies even if they seem effective in our uncontrolled personal observations. There is rarely any serious will to do so among proponents of CAVM, as illustrated by the homeopathy debate and the comments you and Craig have made about the views of some acupuncture proponents.
It is also the case that the vast majority of the CAVM research done, in human as well as veterinary medicine, is of poor methodological quality and very high risk of bias. It is possible to find many positive studies on almost any practice, yet systematic reviews that base conclusions on well conducted studies with limited bias risk almost always conclude there is no effect beyond placebo. All these poor studies suck up scarce resources and create a misleading impression of the state of the evidence. Again, to help us distinguish true from false, scientific methods have to be employed in ways that generate reliable evidence that leads to meaningful conclusion which can be applied in practice, and this is seldom the case in CAM research.
So if the research done will not dissuade true believers and is generally unreliable and misleading, how is it going to clarify our understanding or inform our practices? I’m not saying no CAVM should be researched, of course, only that in many cases (such as homeopathy, energy medicine and other forms of vitalism, etc.) the answer is already as clear as it can reasonably ever be, or the fundamental premises of the methods are not amenable to science and rigorous scientific research is not welcomed or heeded by proponents. In these cases, the resources used to conduct such research are wasted and the waters only further muddied.
Which brings me to the second response evoked by your comments. Of course, plausibility is not the definitive standard for judging the efficacy of a therapy. However, the fact that plausible ideas are sometimes false and implausible ones sometimes true doesn’t mean we should jettison plausibility entirely as a tool in evaluating therapies. The pioneering epidemiologist Sir Austin Bradford Hill discussed the multiple interacting criteria for establishing a causal relationship and both the weaknesses and the utility of plausibility in a classic paper I think every clinician should read many times, and which I have also written about before.
Plausibility is a bit like our clinical index of suspicion–it doesn’t give us a definitive diagnosis, but it helps us start moving in the direction most likely to be productive. In the case of veterinary medicine, where resources for research are so scarce, do we really want to spend them on studies of ideas which can only be true if multiple well-established principles of basic science are false? Should we indefinately call for more research on homeopathy and methods for balancing Yin and Yang, or should we put most of our energies and talent towards ideas with a bit higher probability of being true? Plausibility is one of many factors to consider in weighing botht he mertis of a therapy and the value of further research.
Finally, of course all the same principles we are talking about apply equally to conventional therapies. Much of what we do in conventional medicine lacks high-quality supporting evidence, and some of it is even implausible. I challenge the pharmaceutical reps I talk to as vigorously as the herbalists, and I haven’t made any more friends in the world of stem cell therapies or hyperbaric medicine than I have among TCVM practitioners.
However, it is fallacious to suggest, as is sometimes done, that the criticism some of us are making of unconventional medicine are somehow weakened or inappropriate just because conventional medicine is imperfect. Trimming the dross from our repertoire and strengthening the quality of science and care we provide should reasonably start with the least defensible practices. And while I may respect the intelligence and genuine commitment to patient welfare of colleagues who support such therapies, their good perosnal qualities are not really evidence for the therapies they promote any more than the often venal and malicious behavior of some in the pharmaceutical industry is evidence that their products don’t work. As you say, our interventions need to stand on the merits of the evidence concerning them, and opinion is not evidence regardless of the strengths or weaknesses of the source.
All excellent points, as usual, skeptvet. That so much effort and money is wasted on homeopathy and other nonsense just defies belief – when so much more could be put to better use either in supporting current research for diseases, or improving on therapies for better safety, etc.
The thing about prior plausibility is that you are likely to be asking people who think they are drinking Jesus’ blood (or take your pick of other ancient/recently invented ritualistic belief) to question just HOW acupuncture, et al, are even SUPPOSED to work.