The SkeptVet Blog

There was a scary article yesterday in the New York Times talking about the popularity of a variety of CAM therapies for racehorses. It is a laundry list of nonsense complete with outrageous prices and weak rationalizations. I also thought it nicely complemented my last post, illustrating some of the “empirical” competition we poor vets mired in mere science have to face.

Of course, there are the usual players, acupuncture and chiropractic. The acupuncturist quoted recites the usual rationalization for unscientific therapies of all kinds, “We would like what’s best for the horse, however we get there isn’t the point.” In other words, “We are willing to try anything and decide it works based on uncontrolled observations.” The chiropractor quoted suggests “balance” is needed between mainstream and alternative therapies. The way the article is written makes this sound almost like a voice of caution and moderation, as if chiropractic and acupuncture were themselves mainstream. Of course, they may very well be in comparison with some of the other “alterative’ therapies mentioned.

As always, the “equine telepaths” used as “psychic diagnosticians” win the prize for most ridiculous and embarrassing “therapy.” One of them manages to say with (presumably) a straight face, “I speak with them telepathically…I can do that from anywhere in the world, it doesn’t matter where I am…Usually by the time they get to me, their skepticism is tempered by their need.” The last sentence is particular revealing. When people are desperate, they suspend critical thinking in favor of wishful thinking. I’ve talked before about the risks of such “last resort” use of CAM. It is understandable, but it doesn’t really help people or their animals.

I was also a bit concerned about the hyperbaric therapy mentioned. I admit I don’t know much about hyperbaric medicine generally, though there are certainly some legitimate uses of it. However, I find no evidence to suggest that it has been demonstrated safe and effective for “healing wear and tear on…muscles” as the article suggests. And there are real risks for oxygen toxicity (see also) which can seriously damage an animal’s lungs, so such a therapy is not risk free, though it appears such side effects are uncommon in properly administered and monitored hyperbaric treatment.

Finally, the article had some interesting information about the cost of such therapies. Half an hour in a hyperbaric chamber, for example, will set your horse back about $300. This is cheap, though, compared to the $500 for an in-person consult with an “animal communicator.” I shudder to think what the reaction would be if I charged $500 for a consultation! Apparently, I could have  just skipped 4 years of vet school and the six-figure debt that went with it and learned to communicate telepathically with horses instead.

I recently ran across an edition of the Journal of the American Veterinary Medical Association from 1921 which contained a few tidbits that I found interesting concerning the distinction between scientifically trained veterinarians and their “empirical” competitors. Such documents are always entertaining for the dramatic difference in tone and cultural perspective between then and now, and for the insight into how things were different and, sometimes, how they were the same. The entire issue is available on Google books.

Merillat LA. Some notes on the comparative study of veterinary medicine in France and the United States. Journal of the American Veterinary Medical Association 1921;58(11):64-73.

The author apparently spent two years working with French veterinarians during the First World War, and he has nothing but praise for the high standards of scientific training and expertise among them, which he seems to consider better than the contemporary standards in the U.S. The passage of particular interest here is the following.

There are no veterinary laws in France as we know them in our several States. Anyone may practice, but the nongraduate is under such restrictions as to the particular disease he may treat and the drugs he may prescribe that the activities of the empirical practitioner are somewhat curbed. I say “somewhat” advisedly, because quackery is not unknown in France. In my travels in the rural districts I made a special effort to gather some information in this respect, and found that almost every community supported its handy horse and cattle doctor. The fact that all of the regular veterinarians were at the front may have intensified the situation and may have made it appear more grave than under normal peace-time conditions. I fear, however, from information thus obtained and that gathered by living almost a year in the home of a leading veterinarian of Paris, that the situation as regards empiricism is far from satisfactory to the practicing veterinarian, and I am also assured that the condition in Holland, Belgium, Germany and Italy is no better. In these countries empiricism and the patent-medicine industry thrive uncurbed. Judging from advertisements in the lay press and periodicals, it is legal to sell nostrums regardless of contents or of the effects claimed for them. Thus the veterinary practitioner, after having made this splendid preparation to practice, is little protected against cheap opposition. Then it appears that medical enormities and superstitions are practiced more among ‘the European laymen than among the more intelligent American.

The one feature in which the French veterinarian differs most from us in this respect is the way he has been able to dignify his calling and separate himself from the empiric. Here we are still classed with the lowest type of charlatan, while in France everyone knows that the veterinarian and the quack belong to two entirely different categories of individuals. Here we fight the quack to rid ourselves of an undesirable companion, while in France no complaint is entered so long as the quack does not overstep his legal domain. The French practitioner looks on cheerfully as if rather satisfied to be rid of the uncanny and gruesome jobs he might be called upon to do if the quack were not in existence. This I think explains the status of the quack situation in Continental Europe.

First, of course, we note the natural and relaxed use of the term “quack,” which today would never find its way into any official publication of the AVMA. While the haughty and paternalistic attitude of the medical mainstream in the early 20th century is deservedly a thing of the past, we have perhaps become too democratic in banning any sort of pejorative characterization of useless or nonsensical treatments. I suspect the successful anti-trust litigation pursued by chiropractors against the American medical Association in 1976 may explain this to a significant degree. In any case, “quack” is now a much rarer term, for better or worse.

We also see the beginnings of the desire for American veterinarians to be seen as scientists and professionals, which of course required emulating the kind of scientific training and methods the author so admires in his French colleagues. ” Here we are still classed with the lowest type of charlatan, while in France everyone knows that the veterinarian and the quack belong to two entirely different categories of individuals.” Certainly today, veterinarians are pretty respected and trusted as a profession, but one of the dangers of not being vigilant in maintaining a high scientific standard for our clinical practices is that of becoming indistinguishable in the public eye from “charlatans.” Embracing unproven methods and a anecdote-based epistemology is a sure way to dilute the credibility we have earned as a profession through our dedication to progress and sound science.

There’s a bit of unselfconscious ethnocentrism in the statement ” medical enormities and superstitions are practiced more among the European laymen than among the more intelligent American” that we can afford to think of as quaint since it is so obviously not an attitude that would be acceptable today.

And finally, there’s the discussion of the relationship between mainstream, scientific veterinary medicine and the alternatives, unscientific folk practices, patent medicines, and so on. On one hand, there is an obvious sense of resentment at the idea of veterinarians undergoing rigorous scientific training only to be in competition for patients with untrained lay people who are allowed to sell whatever nostrums they like regardless of their usefulness. And in contrast is the picture painted of an almost idyllic division of labor between the scientific practitioner and the quack. The situation in France is described as one in which the laws allow unscientific, or “empirical” veterinary practices so long as they are confined to domains not reserved for “real” veterinarians. What sort of restrictions are in place is not stated, but one can imagine it might be a system analogous to the veterinary practice acts of the states, in which the “practice of veterinary medicine” is limited to veterinarians, and debates then ensue as to what this encompasses.

In California, for example, performing chiropractic on animals is considered veterinary medicine, so it is required to be done by licensed chiropractors but only under the “direct supervision” of licensed veterinarians. This really only amounts to an exam by the vet within the last year and a referral for chiropractic, but occasionally I am able to catch a case in which a patient comes to our hospital for chiropractic only but I find a serious disease that requires real treatment during my “pro forma” exam of the patient, so I am able to steer them towards real therapy.

Today, the position of the AVMA and other professional veterinary lobbies seems to be that they don’t care what kind of treatment is offered or whether it has any scientific legitimacy so long as only veterinarians are allowed to offer it. This seems a total abdication of any responsibility for protecting the public from quackery and a bit of crass protectionism to me. Granted, there are political considerations since a fair number of veterinarians believe in CAM treatments, so the kind of direct condemnation of them found in this old article wouldn’t be politically possible these days. Still, the article clearly demonstrates that much the same sort of conflict that exists today, between legitimate medical practices based in science and “empirical” practices based in folklore or personal intuition, existed nearly a century ago. The more things change, the more they stay the same.

The American Holistic Veterinary Medical Association (AHVMA) has a statement on their web site explaining what “holistic” medicine is. I was interested in this because it’s a slippery word, seemingly simple on the surface but hard to pin down. Superficially, the word comes from the Greek root for “whole” and simply means a perspective that looks at phenomena (organisms, systems, etc) as wholes, rather than analyzing them at the level of component parts. This is simply one of many levels on which phenomena can, and should be examined.

The problem is that this perspective is difficult to distinguish from mysticism and the notion that natural phenomena are irreducibly complex and nothing meaningful can be learned by examining their components. Reductionism and holism should be complementary, but philosophically and politically they are more often seen as incompatible. Thus “holistic” becomes less a term describing an epistemological perspective and more a philosophical or political shibboleth distinguishing those who disdain the methods and explanatory power of science and prefer mystical, vitalist explanations to scientific ones from those of us who see utility and real truth in the explanations science gives at the component as well as the system level. Let’s go through the AHVMA statement and see if we can find some clarity.

 First, it recommends looking at “all aspects of the animal’s life.” Of course, that isn’t what they really mean since that is a practical impossibility. Could the patient’s cancer have been caused by once walking across a linoleum floor? Is she diabetic because the light shining through the bedroom window is filtered by oak leaves not maple? If we truly believe everything is relevant then we are helpless to make any useful conclusions at all. What they really mean is just that they want to know about things they consider relevant and which scientific medicine often does not. The trick, then, is to demonstrate that these things truly are relevant, which often fails to happen.

Next the statement recommends “employing all of the practitioner’s senses” when evaluating the patient. I’d be surprised if holistic veterinarians make a lot more use of taste than conventional vets, so generally the idea of using all ones senses in examining a patient doesn’t distinguish scientific from holistic medicine since both use a multisensory examination. Of course, this may be referring to “senses” which science does not acknowledge as being real, such as the psychic awareness of thoughts and emotions or “bioenergetic field” techniques that have been discussed in some of the organization’s continuing education literature and its journal. In that case, it would be true that conventional vets don’t use these “senses.”

Now we get to the heart of what “holistic” is truly used to signify, when the statement recommends “the combination of conventional and alternative (or complementary) modalities of treatment.” There is nothing intrinsic in the concept of holism that requires accepting or using CAM therapies. As I’ve pointed out before, CAM therapies often have theoretical foundations that are completely incompatible with each other, though they tend to be united by vitalist tendencies. Chiropractic is not “holistic” at all according to the root sense. It focuses exclusively on supposed misalignment of the spine as the source of all disease and the object of all treatments. Homeopathy, on the other hand, couldn’t care less about the spine when diagnosing or treating an illness but does treat all diseases with whatever substances are believed to mimic the symptoms of the patient when given to healthy people. CAM therapies are not necessarily any less reductionist than scientific medicine, so identifying their use as a component of the holistic veterinary medical approach betrays the ultimately political and marketing nature of the term. It is a warm and fuzzy umbrella under which to gather methods not usually validated by science. CAM therapies are often very narrowly focused on “one true cause” of all disease or the “one true therapy” for all ailments, they just pick a cause or therapy that is unproven or disproven by science.

The statement claims that a vet who follows the holistic approach “wants to find out all about its behaviors, distant medical and dietary history, and its environment including diet, emotional stresses, and other factors.” These sound like the same sort of historical variables any veterinarian considers in gathering data about a patient’s health and disease. The specific meaning attributed to each factor is likely different between science-based and “holistic” vets, but it is a bit of dishonest marketing to suggest conventional doctors are reductionist to the point of ignoring such obviously relevant factors as diet and prior medical history. As I’ve said before, ” I’ve never actually met a veterinarian who considers the patient irrelevant to the health of the knee or the gallbladder or the white blood cell, but CAVM practitioners like to suggest that such myopia is the only alternative to embracing vitalism and faith-based medicine.”

Next we have a very broad-brush paint job illustrating how really nice holistic vets are:  

“Holistic medicine, by its very nature, is humane to the core. The wholeness of its scope will set up a lifestyle for the animal that is most appropriate. The techniques used in holistic medicine are gentle, minimally invasive, and incorporate patient well-being and stress reduction. Holistic thinking is centered on love, empathy and respect.”

All of these vague, pleasant characterizations apply equally well to veterinarians who practice science-based medicine. I might quibble with what is really meant by “love” and “empathy,” since I think these terms are often a cover for claims that alternative practitioners are psychically in touch with the spirits or souls of their patients, a claim I think is more about religion than medicine. But the fact that conventional veterinarians are interested in facts, evidence, and truth does not, as CAM vets like to suggest, imply that somehow they aren’t human beings capable of treating their patients and clients with as much kindness and compassion as anyone else. I would argue evidence-based medicine is a pre-requisite for truly compassionate care since without it we are likely to offer our patents ineffective and even harmful therapies that don’t really help them.

Now comes a summary of the holistic medicine methodology:

“The holistic practitioner is interested in genetics, nutrition, family relationships, hygiene, and stress factors. Many patients present in a state of “disease.” At this point the holistic challenge lies in the question “why?” By a series of analytic observations and appropriate testing the goal becomes finding the true root source of the pathology. A simple-appearing symptom may have several layers of causation. Only when the true cause of the ailment has been found is there the possibility for a lasting recovery.

It is at this point that the most efficacious, least invasive, least expensive, and least harmful path to cure is selected. “

This sounds like a fair description of clinical medicine in general. Nothing here distinguishes a holistic approach from a conventional approach. The implication, of course, is that this is not the method a conventional veterinarian follows because presumably we’re so busy focusing with tunnel vision on superficial symptoms and reaching for toxic drugs that we aren’t looking deeply enough. Fortunately for our patients, this is just a vapid cliché.

Finally, we have this qualifier:

” In many acute situations, treatment may involve aspects of surgery and drug therapy from conventional western technology, along with alternative techniques to provide a complementary whole. This form of treatment has great value for severe trauma and certain infections. It often outperforms other methodologies. It is also at this time that other treatment plans such as those listed below are brought into use. Once the symptoms have been treated, the task is not complete until the underlying disease patterns have been redirected. The patient, as well as the client, will be guided to a new level of health. “

I have taken the liberty of translating this for those not intimately familiar with CAM-speak:

When we have a definable disease or an urgent medical problem, we reach for scientific medicine because it works better. However, when we have vague or hard to define symptoms or we’re just trying to convince the owner that ongoing health can only be achieved and maintained by constant subtle adjustments of the unmeasurable essence of life, then we reach for alternative methods since they and their effects are also vague and hard to define.

There is no question that the concept of holism in its original form has some value. Things can be understood about complex systems at the whole system level that may be missed when looking at their component parts. And some systems may exhibit behavior that is intrinsically not predictable at the component level even though they are fundamentally deterministic (see chaos theory) while other systems may be fundamentally indeterministic. However, none of this legitimate epistemological theory has anything to do with vitalist mysticism or the embracing of unproven or even disproven medical approaches. Looking at the whole patient in its real life context is a valuable element to understanding health and disease. Using any and all therapies that sound good to us regardless of whether or not they have real value is a mistake. Unfortunately, the popular use of “holistic” to market CAM therapies confounds these unrelated approaches.

The fundamental belief underlying this blog is that the truth matters, and that science is the best hope we have of approaching a true understanding of health and disease. I make an effort to investigate and understand the scientific evidence behind veterinary medical approaches and then communicate those findings to my clients and the public because I believe that this is the best way to make the right healthcare decisions for our pets and to avoid the harm and waste of  time, effort, and resources involved in utilizing approaches that don’t work. While I am a realist, and I understand that no amount of objective evidence can change the beliefs of someone with a deep faith in a particular approach or a compelling personal experience that is contrary to the rational, scientific conclusions about it, I still hope that information will truly be power for veterinarians and pet owners, and that we can learn to compensate for our biases and cognitive blind spots and make better, informed decisions.

I do, however, have my doubts. Most polls show that Americans trust scientists more than politicians, but really what does that prove? I trust chiropractors more than, say, the government of Iran, but that doesn’t mean my faith in chiropractors is all that deep. And while polls show most Americans believe science has made life better, most Americans also can’t distinguish between legitimate science and pseudoscience. Purveyors of bogus medical therapies can take advantage of the public’s trust in science easily by simply borrowing or manufacturing jargon than gives the impression of scientific credentials regardless of the real evidence for or against their snake oils. The media can easily manufacture “scientific controversies,” such as those between evolution and intelligent design or between proponents and opponents of the concept of anthropogenic global warming, simply by present on an equal footing two “experts” with differing opinions, regardless of the underlying difference in plausibility or evidence behind these opinions.  So while people trust science to some extent, this is a poor safeguard against the unscientific.

And as a culture, we are deeply faith driven. The overwhelming majority of Americans believe in God, and a solid majority believe God and other spiritual powers are active in daily life, affecting health and disease among other things (for example). So the habit of believing in what cannot be scientifically evaluated is natural to many of us. This makes it difficult to convince people that the scientific evidence against a particular medical therapy should outweigh the personal convictions or testimonials of believers in it, which are generally viewed as at least as good, perhaps better a form of evidence than that provided by science.

A friend recently passed along a social psychology study which, while not definitive by any stretch, does suggest that the picture is even bleaker than this, and that the conclusions of scientists may actually influence people to believe even more strongly in something science demonstrates is not true. The study is called Social influences on paranormal belief: Popular versus scientific support (Curr Res in Soc Psych 2009;15(3)).

Briefly, the investigators gave subjects (the usual undergraduate volunteers) some information about the popular belief and scientific consensus concerning ESP according to four conditions:

Condition 1: Participants read that 25% of the American public believes in ESP and that the scientific community rejects the possibility of ESP.

Condition 2: Participants read that over 90% of the American public believes in ESP and that the scientific community rejects the possibility of ESP.

Condition 3: Participants read that 25% of the American public believes in ESP and that the scientific community is becoming more open to the possibility of ESP.

Conditions 4: Participants read that 90% of the American public believes in ESP and that the scientific community becoming more open to the possibility of ESP.

The subjects then watched a video of a person supposedly demonstrating ESP by guessing the suit of playing cards they could not see. In the video, the guesser is actually informed of the suit and so does much better than one would expect by chance, as someone with ESP would be expected to do. The subjects then completed a questionnaire evaluating their belief in ESP and whether the video was a demonstration of ESP powers.

The mean belief scores were as follows:

Condition 1 (25% of public believes; science rejects): 4.58 (SD = 1.92)

Condition 2 (90% of public believes; science rejects): 4.50 (SD = 1.60)

Condition 3 (25% of public believes; science accepts): 3.58 (SD = 1.84)

 Condition 4 (90% of public believes; science accepts): 4.80 (SD = 1.70)

Analysis of these results showed a significant effect of public belief, with subjects being more likely to believe in ESP if told that most people do. Overall, the scientific community opinion on ESP did not affect the subjects’ level of belief, a depressing hint that science doesn’t have as much influence on the acceptance or rejection of an idea as the popularity of the idea does. However, there was a statistically significant effect of scientific consensus on subject’s belief when ESP was described as not commonly accepted. In this condition, people were more likely to accept ESP as real if scientists rejected it, and more likely to reject ESP if scientists accepted it as possible.

These are not particularly strong findings easily applicable to popular tendencies towards accepting or rejecting the scientific view on implausible propositions, and I don’t mean to make too much of them. However, my personal biases are affected by my own experiences, as everyone’s biases are. I am confronted with depressing regularity by the attitude that what everyone believes cannot be wrong regardless of the soundness of the scientific evidence against it. Personal experience and faith seem in most of us to be far stronger influences on belief than scientific evidence, and even the hint that scientific rejection of an idea may be seen as a reason to believe in that idea worries me.

CAM proponents frequently dismiss scientific evidence with ad hominem reasoning, claiming that the ideological biases or financial interests of those promoting science-based medicine make their conclusions untrustworthy. It is almost automatic for those with unconventional beliefs to assume any evidence provided by commercial researchers (such as those in the pharmaceutical or pet food industries) is worthless at best, if not intentionally deceptive. Could we actually get to the point where the same suspicion was widely applied to science in general, and where scientists were assumed to have ideological or more venal biases that invalidated their conclusions on any idea that claimed itself to be outside the purview of science and more properly in the realm of faith and personal experience? Am I wasting my time trying to convince people that intuitive and traditional ways of knowing the truth about health and disease are inferior to scientific ways? Am I actually promoting the very therapies I criticize because mistrust of science and scientists may incline people to decide in favor of those propositions science argues against?

I have written about the placebo effect before, but a recent update to a Cochrane review entitled Placebo Interventions for All Conditions has prompted me to revisit the subject. It has been clear for some time that the so-called placebo effect is not an example of the power of the mind to overcome disease. There is no evidence that therapies which have no measurable biological affect can truly alter the progression or outcome of an illness. No matter how you feel when given a deliberate placebo or an inert CAM therapy, you don’t live longer, your tumor doesn’t get smaller, you blood sugar doesn’t become normal, and the objective effects of your disease on your body don’t change.

What has seemed reasonable to me in the past is the idea that there might be real benefits in terms of subjective symptoms, such as pain or nausea, attributable to placebo effects. However, I am a little less confident of that in light of this recent review.

The review was a metanalysis of clinical trials which included both placebo and no treatment groups. As expected, no significant clinical effects were found for any objectively measurable variable. However, even in the case of subjective variables like pain and nausea, the best the authors could say was, “in certain settings placebo interventions can influence patient-reported outcomes, especially pain and nausea, though it is difficult to distinguish patient-reported effects of placebo from biased reporting.”

For pain, some trials reported little or no effect, and those that reported a large effect were all conducted by the same group of researchers, which raises suspicions about the reliability of the findings. Findings for most other subjective measures either showed no benefit from placebo, or effects that varied enormously from one study to another, suggesting differences in the studies were responsible for differences in the findings. If the placebo effect were a consistent, reliable phenomenon one would expect it to show up about the same regardless of who is studying it.

The metanalysis also found that how much benefit patients seemed to get from placebo treatments depended heavily on whether they thought they might be getting a placebo treatment. Inert treatments worked better in trials where the subjects were told that no placebo would be given.

These findings and others like them make it seem very likely that the placebo effect is less a feature of the human mind than it is an artifact of  clinical trial design, a conglomeration of biases and measurement errors rather than mind over matter. The better one controls for biased reporting of effects, the less effect one sees. This is not how the results look when one studies an unequivocally effective therapy, even one involving subjective symptoms, such as NSAIDs for arthritis pain. Effects from real therapies are consistently seen in different studies by different researchers, and they do not disappear when the trials become better controlled for bias.

Of course, if someone tells you they have pain or nausea, and then you give them a placebo and they say they have less pain or nausea, can you really say the effect isn’t real? If these phenomena are ultimately subjective, than regardless of the biological events underlying them the perceptions of the sufferer must be the critical determinant of benefit. Still, most of the research on placebos and sham therapies suggests that when well-designed studies are conducted, these benefits are small and do not reach anything like the level of clinical significance of true therapies for these symptoms. And, as I have discussed before, taking advantage of what benefit may occur due to placebo effects requires successfully deceiving the patient, which raises significant ethical concerns.  The lesson of this review seems to be that placebo effects may turn out to be mostly illusory, artifacts of inadequate study design, and we should be hesitant to rely on them for treatment of any clinically significant symptoms.

There are many barriers to public understanding of science and scientific research, ranging from low levels of science literacy in the general population to frequently  poor communication skills among scientists. I recently ran across a pithy summary of some common technical jargon used, in this example, in discussing vaccine safety, and the corresponding interpretation non-scientists are likely to give these terms. I have adapted it slightly from the source, an excellent book on the question of vaccine safety titled Do Vaccines Cause That?! A Guide for Evaluating Vaccine Safety Concerns by Martin G Meyers and Diego Pineda. Many of these kinds of confusion come up a lot when I am discussing the scientific evaluation of medical therapies with non-scientists, and they are a perennial source of confusion and frustration for all concerned. I’m sure you can all think of many others, so feel free to add as many as you can to the list and I will eventually try to compile something more comprehensive.

“The trouble with the world is that the stupid are cocksure and the intelligent are full of doubt.”
Bertrand Russell

“Ignorance more frequently begets confidence than does knowledge.”
Charles Darwin

“It is one of the essential features of such incompetence that the person so afflicted is incapable of knowing that he is incompetent.”
William Ian Miller

I recently stumbled across an article on the Dunning-Kruger Effect, a theory of psychology that claims, “…the skills that engender competence in a particular domain are often the very same skills necessary to evaluate competence in that domain-one’s own or anyone else’s…” What this means in ordinary terms, is that while we all overestimate our own knowledge and skills, the less competent we are at something the more will will overestimate our abilities.

The original article is Unskilled and unaware of it: How difficulties in Recognizing one’s own incompetence lead to inflated self-assessment, and it is well worth a read. The authors conducted a number of experiments on that paradigm of Guinea pigs, the undergraduate psychology student. They gave them tests of social skills (recognizing how funny jokes were compared to a standard of professional comedians’ assessment) and cognitive skills (English grammar and logical reasoning), and they evaluated the accuracy of the subjects’ self-assessment compared with their actual performance. Those in the bottom quartile were most likely to overestimate their skills by the largest amount.

Subsequent tests and analyses indicated that this inaccurate self-assessment was tied to the lack of the same skills needed to perform well on the tests. Those subjects in the top quartile consistently underestimated their performance, but when exposed to representative samples of other subjects’ tests, these top performers were able to adjust their self-assessment appropriately, whereas the bottom quartile subjects did not correct their inaccurate self-assessment based on being able to see directly how their peers performed.

How is this relevant to medicine? Well, in the obvious way that the very people who most need to improve their knowledge and skills and those who are least likely to be able to see that they need this improvement. We all rationalize our failures under the pressure of cognitive dissonance, and we all assess ourselves more charitably than we assess others. But the Dunning-Kruger Effect suggests that those of us with the weakest skills, whether it be in medicine specifically or in the kinds of critical thinking necessary to separate truth from nonsense, we are also the least likely to be able to recognize our own deficiencies.

There is some good news, however. The study also looked at whether or not the least competent subjects could improve the accuracy of their self-assessment. As it turns out, if you make them more competent, by training them on the skills they are being tested on, they also become better able to accurately gauge their own performance. That’s a strong argument for widespread teaching of critical thinking skills and the skeptical outlook, since it suggests we can do better in both our assessments of the world around us and our judgments of our own capabilities.

As many of you may know, there is a libel cases in Britain that has been followed very closely by those of us supportive of evidence-based medicine, that of Simon Singh versus the British Chiropractic Association (BCA). Dr. Singh is a science journalist and author, and with Edzard Ernst he wrote Trick or Treatment: The Undeniable Facts about Alternative Medicine, which I have previously reviewed. The lawsuit stems from an article Dr. Singh wrote in the Guardian newspaper criticizing the BCA for promoting chiropractic treatment for clearly inappropriate conditions despite clear evidence against doing so. In the article, Dr. Singh says:

“The British Chiropractic Association claims that their members can help treat children with colic, sleeping and feeding problems, frequent ear infections, asthma and prolonged crying, even though there is not a jot of evidence. This organization is the respectable face of the chiropractic profession and yet it happily promotes bogus treatments.”

The BCA alleges that “happily promotes bogus treatments” implies deliberate fraud or deception, despite the fact that Dr. Singh has clearly stated his opinion that the organization and its members quite likely believe in the bogus therapy they practice. In Britain, libel law places the burden of proof on the defendant, and it can be very difficult to prove that one did not mean what the plaintiff alleges one meant. This practice makes scientists and others who criticize CAM practices on the basis of sound evidence against them vulnerable to expensive and demoralizing lawsuits intended to intimidate them, and the media which might otherwise publish their critique.

Initially, Dr. Singh faced a significant setback in the lawsuit when a judge ruled his comments were factual assertions, not expressions of opinion, which meant that unless he could prove to a high degree of certainty that the BCA intentionally promoted therapies it knew were ineffective, he would be guilty of libel. An appeals court has now reversed that ruling, declaring Dr. Singh’s statements as opinions, which carries a much lower burden of proof for defending against libel claims.

What is encouraging about this ruling is the language of the opinion, which suggests the higher court recognizes that libel laws are structured in a way to make them easy to abuse in attempts to intimidate legitimate critics of unfounded medical practices. The justice wrote:

“The opinion [expressed in an article like Singh’s] may be mistaken, but to allow the party which has been denounced on the basis of it to compel its author to prove in court what he has asserted by way of argument is to invite the court to become an Orwellian ministry of truth…[those claiming they had been libelled] cannot, by simply filing suit and crying ‘character assassination!’, silence those who hold divergent views, no matter how adverse those views may be to plaintiffs’ interests.

Scientific controversies must be settled by the methods of science rather than by the methods of litigation. More papers, more discussion, better data, and more satisfactory models – not larger awards of damages – mark the path towards superior understanding of the world around us.”

Of course, the court has no authority to alter the law. However, there is an active movement in Britain to reform libel laws so as to allow open scientific debate and the kind of vigorous criticism based on logic and evidence needed to ensure good ideas in medicine prosper and bad ideas are cast aside. It is shameful that Dr. Singh has spent £200,000 and two years of his life battling this clear attempt at intimidation, and his battle is not yet over. But perhaps his case will be the stone that starts an avalanche of real libel reform.