The SkeptVet Blog

I ran across a series of posts from a physician which I think do a thorough job of identifying warning signs of unscientific, irrational or outright quack medical treatments. They’re not organized in a particularly systematic way, but they cover a lot of important ground. Below are the specific headings grouped under the post in which they appear, with the details to be found on Dr. Walt’s page.

Warning Signs- Post One
1. Is the product or practice promoted as a “Major Breakthrough,” “Revolutionary,” “Magic,” or “Miraculous”?

2. Do the promotions try to simply elicit an emotional reaction rather than present clear information to help you make an informed decision about the product?

3. Is only anecdotal or testimonial evidence used to support claims of effectiveness?

4. Are claims made about scientific support without giving specific details?

5. Is the information about the therapy or product being provided by a professional lacking in the proper credentials?

Warning Signs- Post Two
6. Are technical words used without a clear definition?

7. Would a treatment require you to abandon any well-established scientific laws or principles?

8. Do proponents claim that a medical system is so flawless (“airtight”) that there is no need for further testing?

9. Is the treatment said to be effective for a wide variety of unrelated physiological problems?

10. Is the product a quick and easy fix for a complicated and frustrating condition?

Warning Signs- Post Three
11. Does the proponent of the therapy claim to be criticized unfairly?

12. When challenged, do defenders attack the critic instead of responding to the challenge?

13. Do proponents claim that research will prove their therapy is effective as soon as studies are conducted?

14. Is training to provide the therapy offered only at obscure private institutions instead of accredited professional schools?

15. Do proponents use expertise in other areas to lend weight to their medical claims?

Warning Signs- Post Four
16. Is a therapy encouraged simply because it’s been used for centuries by people in some remote place?

17. Do proponents use statements that are basically true but unrelated to the therapy?

18. Do proponents blame failed tests of effectiveness on skepticism or outright nonbelief of observers?

19. Do proponents claim it is too difficult for most to understand how a therapy works, or that only the “enlightened” can understand?

20. Does the proponent disguise the truth with vague and misleading statements?

Warning Signs- Post Five
21. Does the product you’re considering require advance payment?

22. Does the advertisement promise a “money-back guarantee”?

23. Is the therapy available only in other countries?

24. Are there conflicts of interest?

25. Is the term “natural” the main advantage of the remedy?

A certain “holistic” veterinarian about whom I’ve written before (here, here, here, and—well, you get the idea) has asked a couple of good questions on his own blog, though not surprisingly I don’t find his answers at all convincing. I’m interested in talking about these questions not so much as a response to the fellow himself, who is beyond even considering any weaknesses in his own philosophy or view of the world, but  because they are truly good questions, and they provide an opportunity to look at some key elements to the persistence of ineffective veterinary treatments despite their lack of efficacy.

The first question was, “If natural therapies don’t work, and I continue to market them to the public and use them in my practice, am I guilty of ripping off the public?” The short answer, is “Yes!” which is part of why I write this blog.

Of course, I have made it clear many times that I doubt most vets, or others, practicing ineffective nonsense therapies like homeopathy, energy medicine, and so on, are actually intentionally lying. I accept that most of them truly believe that what they do is helpful, for all the reasons we all make poor judgments and delude ourselves from time to time. I don’t fault them for their mistaken beliefs, only for their arrogance in refusing to consider the evidence against them.

Rather than offer any evidence for the effectiveness of his methods, the author instead asks a series of somewhat ridiculous rhetorical questions.

Why have I never been prosecuted and convicted of being a con artist?

So the failure of the local DA to prosecute him for fraud proves that what he does works? No, it simply proves that what he is doing is (probably; he rarely offers specifics) not illegal and that he likely really believes it works and so is deceiving his clients unintentionally.

But the issue of illegal claims made on behalf of herbs, nutritional supplements, medical devices, and other CAM interventions is a real one. The law does limit what providers of such products and services can claim. Claims of being able to prevent or treat specific disease conditions are subject to laws and regulations requiring such claims be demonstrated by legitimate scientific evidence. The FDA, Federal Trade Commission, and other state and federal agencies have complex and overlapping webs of jurisdiction over such claims. Unfortunately, these organizations are understaffed and underfunded, and they have neither the resources nor the political will to police individual practitioners or organization.

I once asked a representative of the FDA’s Center for Veterinary Medicine how to best go about reporting illegal claims about veterinary therapies. She sighed and indicated that apart from large companies selling compounded drugs on a large scale, or products that might conceivably be a risk to human health directly or through food production, the agency was unlikely to be able to investigate or respond to such claims. I have, in fact, filed a number of complaints about egregiously and clearly illegal marketing of unlicensed therapies to the FDA and the FTC, and I have never had so much as an automatic email in response.  There are many reasons, practical and political, why unproven therapies are not subject to control by law enforcement, but their underlying effectiveness is not one of these reasons.  

Why do my patients improve when conventional doctors can’t help them?

Of course, he assumes this is true and would like us to assume so along with him. Once again, he offers no evidence of it, and we have no reason to believe it is so since the sorts of alternative approaches he advocates range from the possible but unproven to the outright ridiculous and impossible.

In the absence of any scientific study, CAM providers can claim what they like about the results of their treatments without fear of contradiction, and they frequently do. I could certainly advertise that I save lives where alternative therapies have failed. In fact, I believe I do! But such claims are not themselves proof of anything. Rarely are there direct comparisons of the effectiveness of a CAM treatment versus a scientific treatment, since this is precisely the sort of evidence CAM providers do not seem to feel they need.

Why does the public continue to come to me, seeking the kind of care I can offer their pets that they can’t get elsewhere.

This is indeed a good question, though the implication that people seek his care proves it must work is no more reasonable than the argument that people play the lottery so they must be winning. There are many reasons why people seek alternative medical care:

1. Desperation-As I’ve discussed before, people facing an illness that cannot be effectively treated by scientific medicine will resort to anything that gives them hope or the promise of a better outcome. This is understandable, and no one can be faulted for wanting their beloved pets to be better. Unfortunately, unproven and bogus therapies offer only false hope and often come with the risk of making things worse.

2. Comfort- I don’t know this particular vet, but I do know that many CAM practitioners seem to meet the psychological needs of their clients better than many conventional doctors. A caring demeanor, asking detailed and lengthy (even if medically meaningless) questions about every aspect of the patient’s life, a non-traditional, non-clinical consultation environment, and many other factors can make the genuine caring a CAM doctor feels for their patients clear to the clients.

It is a shame that the traditional medical environment and the time pressure many vets work under, as well as the lack of effective training in communication, sometimes mask the equally genuine caring science-based vets feel for their patients. And, of course, science-based vets are limited in the comfort they can offer by their need to be honest with clients about the uncertainties and limitations of medicine, whereas believers in miracles and magic can freely claim nothing is ever beyond hope of a cure.

3. Philosophy-Many clients seek alternative medical providers because they themselves are believers in vitalistic philosophies or are suspicious of science and conventional medicine for personal or political reasons. I once saw a clients whose dog was in great pain from a bone tumor. the dog would not walk on the affected leg, and he cried piteously when I touched it. The owner had come to me out of desperation after none of the alternative therapies she had chosen to pursue first had made the disease go away. I had some therapies to offer, though not a cure, but she refused to consider amputation, radiation, or other such conventional treatments because she believed them to be unnatural and harmful rather than beneficial. What struck me the most about her beliefs, though, was that she refused to consider any pain medication for her dog and seemed surprised and offended when I offered it. She was having acupuncture done to the dog and using a homeopathic remedy, and despite the obvious pain the animal was in, she really believed these were working, and she certainly believed they were better than “toxic drugs”. People will ignore great even powerful and obvious information if it contradicts what they believe or wish to believe.

4. Control-CAM often involves changes in diet or lifestyle that the client is responsible for. This can give them a sense of control, of being able to actively do something to combat the illness their pet is facing. And CAM therapies are often given along with simple and intuitive explanations of the disease and the treatment, which is more appealing then the often complex, abstruse, and nuanced scientific descriptions of the pathophysiology of the disease and treatment. This too makes the client feel more in control and less bewildered by what is happening to their pet.

The second general question our holistic colleague asks is this:

If natural therapies don’t work, then why does the government (FDA, state licensing agencies, AVMA, etc.) allow me to use them in my practice? Why would the government, especially the FDA, knowingly not just allow but also support the use of natural therapies in the practice of medicine unless they are in on the “scam” too? Why does the FDA regulate homeopathic remedies (which the skeptics claim don’t work) just like they do conventional medicines if these homeopathic remedies are nothing more than “sugar pills?”

His answer?

The only conclusion someone can draw from this discussion is simply this: since the government allows doctors to use natural therapies, these therapies must work and have some value in the practice of medicine. Otherwise the government could easily make it illegal to use natural therapies and prevent doctors from using them. Punishment for using these unproven natural therapies would be swift and severe, such as the loss of the license to practice medicine and possibly even criminal penalties. Since this doesn’t happen, it must be that the government recognizes the value of using natural therapies in the practice of medicine.

Not even close. The reality is that government licensing and regulation is far more about politics than it is about science. As I’ve discussed before, homeopathic remedies were grandfathered in to the original laws establishing the FDA for purely political reasons. And technically, the FDA has ruled that this does not extend to the use of such remedies in animals, so they could prohibit them if they had the political will to do so, which clearly they do not.

Similarly, veterinarians are licensed at the state level, and their activities are limited by the provisions of state veterinary practice acts. These acts are written by legislators, who are clearly not scientists, and they rarely specify precisely what is or is not considered legitimate veterinary medicine. And, of course, we wouldn’t want them to since the science is far too complex and continually evolving to expect e could enshrine it in law. Specific interventions are sometimes legally prohibited, such as declawing of cats in some local jurisdictions, but this is uncommon and, again, driven by politics and public opinion, not sophisticated assessment of the scientific evidence for what constitutes effective or ineffective therapy.

And the professional organizations that represent veterinarians are reluctant to make strong statements about what is or is not legitimate medicine. They are primarily concerned with ensuring that whatever treatments are applied to animals, they are exclusively the purview of veterinarians, a position which has some legitimacy in terms of consumer protection but which also smacks greatly of simple economic protectionism.

Attempts by professional organizations to oppose quack therapies have been made in the past, and these have been met with legal and political counterattacks that have had devastating effects. The American Medical Association once tried to suppress the use of chiropractic for both scientific as well as, admittedly, less laudable reasons. The organization lost an anti-trust lawsuit as a result, and quickly got out of the business of aggressively trying to promote scientific standards where CAM was concerned. And the very existence of the Dietary Supplement Health and Education Act and the National Center for Complementary and Alternative Medicine, both political constructs that have effectively protected unproven and bogus therapies despite all scientific evidence against them, is testimony to the inability of the political process to supervise anything like rational scientific standards for medical practice.

The amount of time this particular vet spends responding to skeptical criticism (albeit without providing any facts or sound arguments to counter it) seems to me to be a good sign. It demonstrates that even the most committed proponents of alternative medicine recognize that such criticisms have weight and influence. We will never banish irrationality, but the fact that even CAM proponents attempt to claim that science is behind their approach and that they often attempt to employ the language, if not the methods, of evidence-based medicine indicates that even they recognize that the public believes and trusts science more than they would like us to think. The facts, unfortunately, do not speak very loudly or effectively for themselves, so we who are committed to scientific medicine must speak for them.

Dr. Susan Wynn, a veterinarian and veterinary nutritionist who promotes “holistic” veterinary medicine, has put up a couple of blog posts recently which I rather liked, so I thought I’d comment briefly on them. I more commonly find myself disagreeing with Dr. Wynn on questions of veterinary CAM, but she does often present a more rational and scientific approach to the subject than most proponents of “integrative” medicine, and I appreciate the few opportunities that arise to find common ground with those who have a different perspective or approach.

The more recent of the two posts consists of advice on how to critically evaluate veterinary medical information on the internet. I have also written about this subject, and I agree wholeheartedly with Dr. Wynn’s comments. The recommendations generally focus on being wary of excessively optimistic or unrealistic claims, sites with a commercial or rigid ideological bias, sites which rely on testimonials in place of properly published and relevant scientific research, and sites which refuse to disclose relevant information, such as the ingredients in the remedies they sell or the qualifications of the people offering advice. All of this is sound advice when looking for medical information on the Internet.

Dr. Wynn also published a post on the subject of evaluating the credentials of those offering veterinary medical advice, How to Avoid Phony Practitioners. For the most part, I agree with her advice on this subject as well, with an important exception I’ll get to in a moment. It is easy to make up impressive-sounding titles and initials, and not much harder to obtain a meaningless advanced degree through fake diploma mills, and this is common practice among those pushing alternative medical therapies. Not too long ago, I wrote about Primal Defense, a probiotic product marketed by Jordan Rubin, a charlatan who routinely tried to give his advice additional gravitas by acquiring fake degrees from unaccredited correspondence schools. If one is going to seek advice on veterinary medicine, it makes sense to give more weight to the advice of someone with years of training in veterinary medicine. Doctors are just as prone to cognitive errors as anyone else, of course, but we have the advantage of getting much of our information from the most reliable source available, scientific research.

My only disagreement with Dr. Wynn’s advice is that it ignores the importance of evaluating not only the quality and rigor of the training a practitioner receives, but also the plausibility and evidence for the subject matter in which they are trained. A credential from a rigorous and well-supervised program teaching astrology or witchcraft is not worth any more than a diploma mill credential since the approach is itself nonsense, and an expert in nonsense isn’t an expert in any meaningful sense of the word.

Dr. Wynn talks particularly about the various levels of credentialing in Naturopathy, but she completely neglects the fact that the discipline itself is a pseudoscientific, faith-based approach to medicine. In principle, naturopathy is a vitalist philosophy that looks to unseen energy forces to explain health and disease. In practice, it is a hodgepodge of sensible nutritional and exercise advice and loads of CAM, varying from the plausible-but-unproven to the completely bogus. The only common theme to methods used by naturopaths seems to be the notion of vital energies of one kind or another, as found in methods such as acupuncture, chiropractic, and homeopathy, which all rely in theory on mystical energies that no one can prove exist. And while naturopaths are theoretically trained to refer patients with serious medical conditions to conventional doctors, many are suspicious of conventional drug and surgical therapies. It is not uncommon for naturopaths to oppose vaccination and to promote not only supposedly “natural” approaches such as herbal medicine but also bizarre, dangerous, and manifestly “unnatural” therapies such as chelation therapy, detoxifying enemas, and so on. If the theory is nonsense and the specific practices unscientific, it doesn’t matter how rigorous the training is.

The same logic applies to Traditional Chinese Medicine, with it’s vitalist theory, idiosyncratic and unscientific diagnostic methods, and inconsistent and mostly unproven therapeutic practices. Chiropractic, likewise, contains a few bits of useful treatment for musculoskeletal pain and a load of nonsense and outright dangerous practices, and of course homeopathy is utterly worthless.

All the training in the world in a philosophy or method which has no value does not protect the patient from harmful or useless treatments. So while I generally agree with Dr. Wynn in terms of checking into the credentials of anyone offering veterinary medical advice, I would go farther and suggest that pet owners should look into the arguments and, most importantly, the scientific evidence concerning the philosophy and methods a particular practitioner offers as well.

I recently discussed, and corrected, some comments Dr. Shawn Messonier made on his blog concerning Alison Averis’ post here illustrating how personal experience can be unreliable in evaluating whether a medical treatment works or not. Dr. Messonier has responded to some of the comments I made in that post, as usual trying to warp them to fit his pro-CAM “talking points.” He begins by dredging up that vacuous marketing label of “healthcare choice” which I’ve discussed before.

While most doctors and healthcare providers are open to natural care and support your choice and the type of medicine you desire, sadly there are still many that will come up with any argument and do anything to prove to you that your choice is wrong.

This argument presumes that giving people more therapies to chose from, even if they are useless or even harmful, is somehow a good thing. It also suggests that informing people about what reliable, scientific evidence is and how they can use it to help them make good choices for their pets is somehow limiting the freedom or autonomy of pet owners.

I offer choices to my clients every day, and I expect them to be full partners in caring for their pets. I do not offer them the option of psychic diagnosis or therapy, animal sacrifice to propitiate the gods of health, or homeopathy because those would be false choices of no real value. And if there is a therapy that I believe might be beneficial but about which there is significant uncertainty, I give them that information so their choices will be informed. Adding meaningless buzzwords like “natural” to what one does, regardless of whether there is reliable evidence to support that it is safe or effective, is purely a marketing strategy, not a means to give greater autonomy or choice to pet owners.

Dr. M then goes on to quote my recent post and respond to specific bits and pieces.

1.“Doctors rely on their personal experiences, intuition and, anecdotes as evidence to justify implausible or simply “made up” theories and practices.”

This is partly true, but it’s also true for doctors who practice conventional medicine as well. In the good old days before strictly controlled, well-funded (usually by the drug companies seeking approval for their latest drug) studies, ALL doctors learned and shared knowledge based upon their personal experiences with various medications (both natural and conventional.) There is nothing wrong with learning from experience and sharing these experiences. Yes, we all would love to have well researched studies “proving” what we already know to be factually correct. However, the company funding the study determines which studies are ultimately published. While most companies are ethical, there have been some well-publicized incidents of companies ignoring negative studies and only publishing positive studies showing benefits for their medications.

*Evidence is evidence. Either a treatment works or it doesn’t. While skeptics selectively ignore evidence showing the effectiveness of natural therapies, they never seem to ignore evidence showing natural therapies don’t work.

Several problems here. First, Dr. M is unfortunately correct that many doctors practicing primarily conventional medicine rely unduly on anecdote and personal experience. This does not, of course, mean it is the right approach, and I hope that the practice of science and evidence-based medicine will come to be the predominant approach to medical knowledge among all veterinarians. It is certainly more likely to take hold among those who already practice medical approaches that are based on sound scientific reasoning and evidence than among those who, like Dr. M, confidently assert the value of therapies that are unscientific in their underlying premises or without good supporting evidence. I certainly do not claim perfection in conventional medicine, I simply advocate for all of us as veterinarians and pet owners to recognize our limitations and attempt to do better by turning to the most reliable evidence available concerning the therapies we use.

The underlying problem with the CAM approach to knowledge is explicit in this statement. Dr. M pays lip service to scientific evidence, but he clearly views it as icing on the cake, “‘proving’ what we already know to be factually correct.” The arrogance of this kind of self-certainty is breathtaking. Science and the knowledge it generates is not merely a way of better promoting and marketing what we already believe. It is a way of finding the truth despite our own limitations and cognitive biases. There can be no learning or improvement without the recognition that our knowledge is incomplete and flawed, and yet Dr. M clearly does not believe this. To him, science is simply another tool in the promotion of his beliefs, not a way to find out what is true and what isn’t.

As far as the issue of funding bias and publication bias, there is no question these are real problems. They do not, however, invalidate the whole enterprise of scientific research, as the good doctor suggests. There is plenty of research evidence available, both from industry-funded research and from other sources (academia, government, private clinical practices, etc), and acknowledging that there are weakness in some of this research is not license to ignore it and simply do what we feel like doing.

“Evidence is evidence” is nonsense. There are levels of evidence ranging from the least reliable (personal opinion and experience) to the most reliable (multiple consistent well-designed and conducted clinical trials). The problem many CAM practitioners have is reading the pyramid of evidence upside down:

Undoubtedly, we all suffer from confirmation bias, so it is true we attend to evidence that supports our beliefs more than evidence that disconfirms them. This is a major reason why scientific evidence is more reliable than anecdote and personal experience, so raising the issue simply undermines Dr. M’s own approach. If he has specific evidence for particular therapies he thinks I am ignoring, his is of course always welcome to provide it for consideration.

2.“I would be very interested in how much “objective” measuring of outcomes doctors using natural therapies actually do. They often seem to imagine they can “eliminate” subjective owner assessments in practice, but I find this hard to believe.”

Let me quickly answer this objection by sharing with you two recent cases. One involves a dog with elevated kidney enzymes that did not respond to treatment by his prior conventional veterinarian. The other case involves a cat with elevated liver enzymes and also did not respond to treatment by her conventional veterinarian. After using several natural therapies, follow-up blood tests showed that the kidney and liver enzymes returned to normal.

There is nothing subjective about this. The laboratory, which did not know what kind of therapy I used on these patients, reported the results objectively.

Even in trying to show how objective his medical practice is, Dr. M cannot help but rely on anecdotes to prove his point. Of course, the flaw here is glaringly obvious. He assumes that improvements in the clinical laboratory values measured was due to his therapy. This is the post hoc ergo propter hoc fallacy, and it ignores the fact that medical problems can improve for all sorts of reasons other than the ones we assume. Some possibilities include:

1. Self-Limiting Disease Many diseases are self-limiting. If the condition is not chronic or fatal, the body’s own recuperative processes usually restore the sufferer to health. Thus, to demonstrate that a therapy is effective, its proponents must show that the number of patients improved exceeds the number expected to recover without any treatment at all. Without detailed records of successes and failures for a large enough number of patients with the same complaint, someone cannot legitimately claim to have exceeded the norms for unaided recovery.

2. Waxing and Waning Chronic Disease (also known as Regression to the Mean) Such conditions as arthritis, allergies, and gastrointestinal problems normally have “ups and downs.” Naturally, clients tend to seek therapy during the period or greatest clinical symptoms. In this way, a treatment will have repeated opportunities to coincide with upturns that would have happened anyway.


3. Multiple Concurrent Therapies If improvement occurs after a pet has had several interventions, and probably other unremarked changes in the owner’s treatment of the sick pet, one or another of the changes often gets a disproportionate share of the credit or blame. Frequently, the latest in a series of interventions or the newest thing tried is credited with improvement even though many things were done.

The overall tone of Dr. M’s response is that I am somehow on a crusade against “natural” medicine. That cannot be true for the simple reason that “natural medicine” is a meaningless term filled with assumptions and the naturalistic fallacy. I am actually on the same crusade as Dr. M. I am dedicated to giving my patients the best, most effective care and the highest quality of life possible, and to protecting them from things that are useless or harmful. The difference is not one of intention, but one of philosophy. I believe there is such a thing as truth and that the best way to  care for our pets is to know what is true and what is false about the therapies we offer. I believe that we often don’t know what we think we know, and that our confidence in our own intuition and judgments is often excessive and unjustified. I believe that science is the most powerful tool we’ve ever invented for separating truth from mythology, and that with it we have done, and will continue to do, better in medicine that we did in all the thousands of ears we relied exclusively on storytelling, respect for tradition and authority, and random trial and error to decide what works and what doesn’t.

If this blog is controversial or challenging, it is because when I find the evidence doesn’t support the claims people make about veterinary medicine I have the temerity to say so. I try to do so in a civil way, and I try to focus on the problems of reasoning and evidence rather than personality, though I won’t claim I never give in to irritation at the persistence of mischaracterization and misinformation in some people’s arguments. But helping pet owners to take better care of their companions isn’t accomplished by ignoring the evidence or by letting unsupported claims or misleading arguments stand, and I make no apologies for trying to help people filter the information available to them and separate the strong, reliable claims from the weak or thoroughly bogus ones. I’m happy to discuss what the evidence is and what it means with anyone, but Dr. M prefers simply to insinuate and allege motives, to throw around meaningless marketing terms, and to talk about anything but the specific evidence for or against the therapies he advocates.