The fundamental belief underlying this blog is that the truth matters, and that science is the best hope we have of approaching a true understanding of health and disease. I make an effort to investigate and understand the scientific evidence behind veterinary medical approaches and then communicate those findings to my clients and the public because I believe that this is the best way to make the right healthcare decisions for our pets and to avoid the harm and waste of  time, effort, and resources involved in utilizing approaches that don’t work. While I am a realist, and I understand that no amount of objective evidence can change the beliefs of someone with a deep faith in a particular approach or a compelling personal experience that is contrary to the rational, scientific conclusions about it, I still hope that information will truly be power for veterinarians and pet owners, and that we can learn to compensate for our biases and cognitive blind spots and make better, informed decisions.

I do, however, have my doubts. Most polls show that Americans trust scientists more than politicians, but really what does that prove? I trust chiropractors more than, say, the government of Iran, but that doesn’t mean my faith in chiropractors is all that deep. And while polls show most Americans believe science has made life better, most Americans also can’t distinguish between legitimate science and pseudoscience. Purveyors of bogus medical therapies can take advantage of the public’s trust in science easily by simply borrowing or manufacturing jargon than gives the impression of scientific credentials regardless of the real evidence for or against their snake oils. The media can easily manufacture “scientific controversies,” such as those between evolution and intelligent design or between proponents and opponents of the concept of anthropogenic global warming, simply by present on an equal footing two “experts” with differing opinions, regardless of the underlying difference in plausibility or evidence behind these opinions.  So while people trust science to some extent, this is a poor safeguard against the unscientific.

And as a culture, we are deeply faith driven. The overwhelming majority of Americans believe in God, and a solid majority believe God and other spiritual powers are active in daily life, affecting health and disease among other things (for example). So the habit of believing in what cannot be scientifically evaluated is natural to many of us. This makes it difficult to convince people that the scientific evidence against a particular medical therapy should outweigh the personal convictions or testimonials of believers in it, which are generally viewed as at least as good, perhaps better a form of evidence than that provided by science.

A friend recently passed along a social psychology study which, while not definitive by any stretch, does suggest that the picture is even bleaker than this, and that the conclusions of scientists may actually influence people to believe even more strongly in something science demonstrates is not true. The study is called Social influences on paranormal belief: Popular versus scientific support (Curr Res in Soc Psych 2009;15(3)).

Briefly, the investigators gave subjects (the usual undergraduate volunteers) some information about the popular belief and scientific consensus concerning ESP according to four conditions:

Condition 1: Participants read that 25% of the American public believes in ESP and that the scientific community rejects the possibility of ESP.

Condition 2: Participants read that over 90% of the American public believes in ESP and that the scientific community rejects the possibility of ESP.

Condition 3: Participants read that 25% of the American public believes in ESP and that the scientific community is becoming more open to the possibility of ESP.

Conditions 4: Participants read that 90% of the American public believes in ESP and that the scientific community becoming more open to the possibility of ESP.

The subjects then watched a video of a person supposedly demonstrating ESP by guessing the suit of playing cards they could not see. In the video, the guesser is actually informed of the suit and so does much better than one would expect by chance, as someone with ESP would be expected to do. The subjects then completed a questionnaire evaluating their belief in ESP and whether the video was a demonstration of ESP powers.

The mean belief scores were as follows:

Condition 1 (25% of public believes; science rejects): 4.58 (SD = 1.92)

Condition 2 (90% of public believes; science rejects): 4.50 (SD = 1.60)

Condition 3 (25% of public believes; science accepts): 3.58 (SD = 1.84)

 Condition 4 (90% of public believes; science accepts): 4.80 (SD = 1.70)

Analysis of these results showed a significant effect of public belief, with subjects being more likely to believe in ESP if told that most people do. Overall, the scientific community opinion on ESP did not affect the subjects’ level of belief, a depressing hint that science doesn’t have as much influence on the acceptance or rejection of an idea as the popularity of the idea does. However, there was a statistically significant effect of scientific consensus on subject’s belief when ESP was described as not commonly accepted. In this condition, people were more likely to accept ESP as real if scientists rejected it, and more likely to reject ESP if scientists accepted it as possible.

These are not particularly strong findings easily applicable to popular tendencies towards accepting or rejecting the scientific view on implausible propositions, and I don’t mean to make too much of them. However, my personal biases are affected by my own experiences, as everyone’s biases are. I am confronted with depressing regularity by the attitude that what everyone believes cannot be wrong regardless of the soundness of the scientific evidence against it. Personal experience and faith seem in most of us to be far stronger influences on belief than scientific evidence, and even the hint that scientific rejection of an idea may be seen as a reason to believe in that idea worries me.

CAM proponents frequently dismiss scientific evidence with ad hominem reasoning, claiming that the ideological biases or financial interests of those promoting science-based medicine make their conclusions untrustworthy. It is almost automatic for those with unconventional beliefs to assume any evidence provided by commercial researchers (such as those in the pharmaceutical or pet food industries) is worthless at best, if not intentionally deceptive. Could we actually get to the point where the same suspicion was widely applied to science in general, and where scientists were assumed to have ideological or more venal biases that invalidated their conclusions on any idea that claimed itself to be outside the purview of science and more properly in the realm of faith and personal experience? Am I wasting my time trying to convince people that intuitive and traditional ways of knowing the truth about health and disease are inferior to scientific ways? Am I actually promoting the very therapies I criticize because mistrust of science and scientists may incline people to decide in favor of those propositions science argues against?

The tireless promoter of “natural” veterinary medicine (whatever that is) and himself, Dr. Shawn Messonier, has applied his characteristic fuzzy logic and love of strawmen and misleading hyperbole to a recent guest post by Alison Averis on Fooling yourself: An example of how to fool yourself and how the placebo effect can work in animals. I shall reproduce his post below, with appropriate commentary and clarification (in bracketed, boldface type).

“I recently came across a post on another blog purporting to show that there is no proof that natural/alternative therapies work. [Actually, actually reading Alison’s post will make it clear that this is not the point of it at all. Throughout his essay, Dr. Shawn seems unable to distinguish between an anecdote used appropriately as an illustration and example of a phenomenon, and his preferred use of it as proof of something. Alison’s story was a parable illustrating how easy it is for one’s expectations to color one’s judgment about the efficacy of a therapy not evaluated in an objective, scientific way, not a statement or proof that “natural/alternative” therapies do not work.] This particular blog is devoted to debunking any therapy that falls outside the mainstream. [Actually, this blog is devoted to, as it says at the top of the page, taking a “Science-Based Look at Complementary and Alternative Medicine.” I am happy to validate any non-mainstream therapy that can be demonstrated to be effective in an appropriately scientific way, and I am equally happy to condemn any mainstream therapy that cannot.] The writer of this blog constantly seeks to show that there is no proof that any natural therapy ever works or helps anyone or any pet. [strawman and flagrant demonization]

The post I read was submitted by a reader of this particular blog. Briefly, she shared the following information in her post.
An “alternative therapist” told the owner of a horse that she would “work on” the horse for free if the owner would give her feedback on how the horse performed following the “free therapy session.” Unknown to the owner, the “therapist” never worked on the horse but led the owner to believe she had done so.

The day following the supposed “therapy,” the owner was happy to report that the horse actually “rode better than normal.” The writer of the blog used this one example to prove that natural therapies don’t work and that the reason the owner thought the horse rode better was simply due to the power of suggestion, or the placebo effect. [Again, the example was never purported to prove that “natural therapies,” or even the specific therapy referred to, don’t work. It was intended to illustrate how subjective, uncontrolled observations are an unreliable basis for making a judgment on what works and what doesn’t.]

While this was an interesting post to read, and it can easily deceive people into thinking that this one simple experiment proved that all natural therapies only work by the placebo effect, a more careful examination of the post leaves a lot to be desired.

First, I don’t know what an “alternative therapist” is or what one does. I assume the writer of this blog is referring to some sort of physical therapy such as massage therapy, but that is never explained.

Second, there’s a big difference between someone who claims to be an “alternative therapist” and a licensed doctor trained in integrative medicine. [How, exactly, does one get “licensed” to practice “integrative medicine?” This is a buzzword that simply means the use of unproven therapies alongside scientifically validated ones, and there is no process by which one becomes licensed to do this.]

Third, this particular website on which the blog appears always chastises alternative medicine for the lack of proof for alternative therapies since there are so few well controlled, double-blind placebo-controlled studies. [I have rebutted this charge of relying only on double-blind placebo controlled trials before HERE, yet Dr. Shawn continues to state the falsehood that I rely solely on such evidence.] It’s interesting that this website now uses an example of a poorly designed “study” (if an observation of one patient even constitutes a study) to prove that no natural therapies ever work! [Yet again, the hypocrisy that he gleeful implies is not present, despite his inability to distinguish an example from a proof.]

Fourth, the observation in this case was entirely subjective rather than objective. Rather than using an owner’s observation that the horse “rode better” after a particular therapy, why not use a more objective measurement such as gait analysis or forced plate analysis? In my practice, I can objectively measure things such as a pet’s blood pressure or blood count and easily determine if my therapy (conventional or natural) is working. This can totally eliminate the subjective assessment an owner may make in determining if a pet simply “feels better.” [I would be very interested in how much “objective” measuring of outcomes Dr. Shawn actually does. He seems to imagine he can “eliminate” subjective owner assessments in his practice, but I find this hard to believe. He routinely claims better outcomes for his patients with cancer, for example, than those possible through scientific medicine, yet I am unaware of any formal, scientifically controlled or objective research he has published on the subject, so I wonder how exactly he makes his assessments of his interventions more objective and independent of owner history than the rest of us in clinical practice.]

Fifth and finally, the same “conclusion” could easily be reached in this case if a conventional drug had been used rather than an alternative therapy. For example, suppose a doctor told the same horse owner that he was going to give the horse a pill which would improve its performance. But suppose the doctor secretly did not give the horse the medication but the owner still reported that the horse “rode better” the next day. If we follow the blogger’s (lack of) logic, we have to draw the same conclusion-namely that this one example proves that conventional drug therapy doesn’t work! [No, as I’ve made clear, and as should be obvious from reading the post, the conclusion we would draw would be that the owner report of how the horse performed subsequent to the sham therapy, be it conventional or alternative, is not a reliable assessment of the efficacy of the therapy. The point is that we must have a better level of evidence, and my evaluations of alternative practices such as those advocated by Dr. Shawn is intended to identify whether such evidence is or is not available. He is the one who routinely relies on his personal experiences and intuition and anecdotes as evidence to justify implausible or simply “made up” theories and practices. The whole purpose of this blog is to do better than this sloppy and outdated approach, not to replicate it in favor of or in opposition to any particular therapy.]

It’s interesting how much hatred and negativity one encounters among people who don’t like natural/alternative medicine. [It is also interesting to see how reasoned and civil criticism and a call for real evidence rather than simply opinion is identified as “hatred and negativity.”] To use one example as this blogger has done as universal proof that integrative medicine doesn’t achieve successful results is ludicrous and deceiving. [And, fortunately, not at all what has been done with this example] Fortunately, it’s easy to see through all the hypocrisy in this blog and understand the real goal of people who write this rubbish-destroy your choice in determining what kind of therapy you would like for you, your family members, and your pets. [Wow, “rubbish.” There’s a polite and thoughtful critique for you. As for the question of choice, I’ve addressed this bit of misdirection before. Offering as alternatives treatments which have not been demonstrated to be safe or effective, or which have actually been clearly demonstrated not to be so, is not offering choice–it is offering false hope at best and lies at worst, and these are not what our patients or clients deserve from us.]”

I have written about the placebo effect before, but a recent update to a Cochrane review entitled Placebo Interventions for All Conditions has prompted me to revisit the subject. It has been clear for some time that the so-called placebo effect is not an example of the power of the mind to overcome disease. There is no evidence that therapies which have no measurable biological affect can truly alter the progression or outcome of an illness. No matter how you feel when given a deliberate placebo or an inert CAM therapy, you don’t live longer, your tumor doesn’t get smaller, you blood sugar doesn’t become normal, and the objective effects of your disease on your body don’t change.

What has seemed reasonable to me in the past is the idea that there might be real benefits in terms of subjective symptoms, such as pain or nausea, attributable to placebo effects. However, I am a little less confident of that in light of this recent review.

The review was a metanalysis of clinical trials which included both placebo and no treatment groups. As expected, no significant clinical effects were found for any objectively measurable variable. However, even in the case of subjective variables like pain and nausea, the best the authors could say was, “in certain settings placebo interventions can influence patient-reported outcomes, especially pain and nausea, though it is difficult to distinguish patient-reported effects of placebo from biased reporting.”

For pain, some trials reported little or no effect, and those that reported a large effect were all conducted by the same group of researchers, which raises suspicions about the reliability of the findings. Findings for most other subjective measures either showed no benefit from placebo, or effects that varied enormously from one study to another, suggesting differences in the studies were responsible for differences in the findings. If the placebo effect were a consistent, reliable phenomenon one would expect it to show up about the same regardless of who is studying it.

The metanalysis also found that how much benefit patients seemed to get from placebo treatments depended heavily on whether they thought they might be getting a placebo treatment. Inert treatments worked better in trials where the subjects were told that no placebo would be given.

These findings and others like them make it seem very likely that the placebo effect is less a feature of the human mind than it is an artifact of  clinical trial design, a conglomeration of biases and measurement errors rather than mind over matter. The better one controls for biased reporting of effects, the less effect one sees. This is not how the results look when one studies an unequivocally effective therapy, even one involving subjective symptoms, such as NSAIDs for arthritis pain. Effects from real therapies are consistently seen in different studies by different researchers, and they do not disappear when the trials become better controlled for bias.

Of course, if someone tells you they have pain or nausea, and then you give them a placebo and they say they have less pain or nausea, can you really say the effect isn’t real? If these phenomena are ultimately subjective, than regardless of the biological events underlying them the perceptions of the sufferer must be the critical determinant of benefit. Still, most of the research on placebos and sham therapies suggests that when well-designed studies are conducted, these benefits are small and do not reach anything like the level of clinical significance of true therapies for these symptoms. And, as I have discussed before, taking advantage of what benefit may occur due to placebo effects requires successfully deceiving the patient, which raises significant ethical concerns.  The lesson of this review seems to be that placebo effects may turn out to be mostly illusory, artifacts of inadequate study design, and we should be hesitant to rely on them for treatment of any clinically significant symptoms.

There are many barriers to public understanding of science and scientific research, ranging from low levels of science literacy in the general population to frequently  poor communication skills among scientists. I recently ran across a pithy summary of some common technical jargon used, in this example, in discussing vaccine safety, and the corresponding interpretation non-scientists are likely to give these terms. I have adapted it slightly from the source, an excellent book on the question of vaccine safety titled Do Vaccines Cause That?! A Guide for Evaluating Vaccine Safety Concerns by Martin G Meyers and Diego Pineda. Many of these kinds of confusion come up a lot when I am discussing the scientific evaluation of medical therapies with non-scientists, and they are a perennial source of confusion and frustration for all concerned. I’m sure you can all think of many others, so feel free to add as many as you can to the list and I will eventually try to compile something more comprehensive.

“The trouble with the world is that the stupid are cocksure and the intelligent are full of doubt.”
Bertrand Russell

“Ignorance more frequently begets confidence than does knowledge.”
Charles Darwin

“It is one of the essential features of such incompetence that the person so afflicted is incapable of knowing that he is incompetent.”
William Ian Miller

I recently stumbled across an article on the Dunning-Kruger Effect, a theory of psychology that claims, “…the skills that engender competence in a particular domain are often the very same skills necessary to evaluate competence in that domain-one’s own or anyone else’s…” What this means in ordinary terms, is that while we all overestimate our own knowledge and skills, the less competent we are at something the more will will overestimate our abilities.

The original article is Unskilled and unaware of it: How difficulties in Recognizing one’s own incompetence lead to inflated self-assessment, and it is well worth a read. The authors conducted a number of experiments on that paradigm of Guinea pigs, the undergraduate psychology student. They gave them tests of social skills (recognizing how funny jokes were compared to a standard of professional comedians’ assessment) and cognitive skills (English grammar and logical reasoning), and they evaluated the accuracy of the subjects’ self-assessment compared with their actual performance. Those in the bottom quartile were most likely to overestimate their skills by the largest amount.

Subsequent tests and analyses indicated that this inaccurate self-assessment was tied to the lack of the same skills needed to perform well on the tests. Those subjects in the top quartile consistently underestimated their performance, but when exposed to representative samples of other subjects’ tests, these top performers were able to adjust their self-assessment appropriately, whereas the bottom quartile subjects did not correct their inaccurate self-assessment based on being able to see directly how their peers performed.

How is this relevant to medicine? Well, in the obvious way that the very people who most need to improve their knowledge and skills and those who are least likely to be able to see that they need this improvement. We all rationalize our failures under the pressure of cognitive dissonance, and we all assess ourselves more charitably than we assess others. But the Dunning-Kruger Effect suggests that those of us with the weakest skills, whether it be in medicine specifically or in the kinds of critical thinking necessary to separate truth from nonsense, we are also the least likely to be able to recognize our own deficiencies.

There is some good news, however. The study also looked at whether or not the least competent subjects could improve the accuracy of their self-assessment. As it turns out, if you make them more competent, by training them on the skills they are being tested on, they also become better able to accurately gauge their own performance. That’s a strong argument for widespread teaching of critical thinking skills and the skeptical outlook, since it suggests we can do better in both our assessments of the world around us and our judgments of our own capabilities.