Without question, the most frequent comment I get in response to my articles is that people feel some of the therapies I write about must be effective regardless of the scientific evidence because they have had personal experiences that suggest this. The experience of having used a therapy and seen an improvement is very powerful psychologically, and it makes us very confident that the therapy we used caused the improvement we saw. Unfortunately, the evidence that this kind of conclusion is not reliable is overwhelming. This kind of thinking is so common, and so untrustworthy, it constitutes a unique ogical fallacy, the post hoc ergo propter hoc, or “false cause” fallacy.

Reality is complex, and our minds seek simple, direct causal explanations and satisfying narratives to explain things. Sometimes, of course, these explanations are true. But they are false much more often than we realize. The dismissal of anecdotal evidence by scientists isn’t casual, and it isn’t based on the idea that people who have anecdotal experiences must be stupid. It is based on centuries of study of the human mind, including decades of controlled research in cognitive psychology that shows things simply aren’t always as they seem.

John Stuart Mill described eloquently the problem that while we all realize we are imperfect and acknowledge in general terms that we can be wrong, we are very, very reluctant to ever admit we are wrong about any specific belief:

Unfortunately for the good sense of mankind, the fact of their fallibility is far from carrying the weight in their practical judgment, which is always allowed to it in theory; for while everyone knows himself to be fallible, few think it necessary to take any precautions against their own fallibility, or admit the supposition that any opinion, of which they feel very certain, may be one of the examples of the error to which they acknowledge themselves to be liable.

This helps to explain why I so often have to repeat, to pet owners and other veterinarians alike, that just because they have seen something appear to work with their own eyes, that isn’t really a good reason to believe it actually does work without supporting controlled scientific evidence.

A recent article on the subject both explains the problem, and discusses some of the possible solutions.

Your Brain is Primed to Reach False Conclusions by Christie Aschwanden

Here’s her conclusion:

With a lot of evidence that erroneous beliefs aren’t easily overturned, and when they’re tinged with emotion, forget about it. Explaining the science and helping people understand it are only the first steps. If you want someone to accept information that contradicts what they already know, you have to find a story they can buy into. That requires bridging the narrative they’ve already constructed to a new one that is both true and allows them to remain the kind of person they believe themselves to be.

The good news is that the fallibility of uncontrolled personal observation is well-documented, and this has been known for hundreds of years. This means it should be possible to inoculate people against excessive confidence in their own experiences early, teaching critical and skeptical thinking early in school, even before formal teaching of scientific facts. The bad news, however, is that once people reach a conclusion based on anecdotal experience, facts are not very effective at challenging that conclusion. Being given evidence that we are wrong tends to strengthen our false beliefs and impel us to build more and stronger arguments to support them.

Sadly, simple presentation of facts isn’t enough. People need to be led to reconsider their opinions through arguments that speak to their emotions and their core values, not simply their intellect. This is certainly more challenging than simply presenting the facts, especially for scientists who tend to think in terms of objective evidence and are suspicious or arguments that appeal primarily to emotions of beliefs rather than facts. And, of course, the biggest challenge in changing minds about scientific topics is that the stronger one’s belief, the less willing one is to seriously consider alternative explanations. The very people who most need to read about why anecdotes can’t be trusted won’t.

Despite all that, since I have to make the same argument over and over again, I have collected a few resources on this subject to which I refer anyone interested and open-minded enough to consider them:

Medical Miracles: Should We Believe?

Testimonials Lie

Alternative medicine and placebo effects in pets

Placebo effects in epileptic dogs

Medical Practices Once Widely Accepted that Proved Ineffective or Harmful when Studied Scientifically

Introduction

Yes, I know the title is a misquotation of the original line, but I couldn’t resist.

The idea that music might be useful in calming animals or otherwise influencing their mood and behavior is not new, but there seems to have been increased interest in it lately in veterinary medicine, specifically in the use of music to reduce anxiety and stress in hospitalized animals. Given the profound effect music has on human emotions, it is reasonable to ask the question, could animals experience the same kind of emotional effects and could music help soothe them in times of stress? As a veterinarian, a musician (well, I play mandolin a bit, anyway), and a former primatologist working in environmental enrichment, I find this an intriguing idea.

I’m sure, though, that it will come as no surprise to regular readers that I have some skepticism about the idea as well. I know from my own behavior research work and clinical veterinary practice that it is extremely difficult to consistently and objectively define and measure stress and mood in non-human animals. I also know that it is all too easy to assume that the world seems much the same to our animal companions as it seems to us, and that they share our likes and dislikes. Finally, I also know that nothing in medicine is free, and if music has meaningful benefits for our pets and patients, it likely has some risks as well.

Humans are unique in the extent to which they deliberately create patterns of sounds, often using tools made for that purpose, with the specific intent of inducing emotional responses in others of our species. Arguably, many species create music of a sort. Birds, humpback whales, and others, can create quite complex and engaging sound patterns. But I think it is fair to say that these differ in many ways from the type of music humans create and the purposes these sounds serve. So I think it fair to be cautious in assuming other animals will perceive or respond to human music as we do.

And as we all know, music is incredible in its variety, with dramatic differences in musical styles and tastes between cultures, generations, and individuals. What I find energizing or entertaining, my wife may find grating and annoying. So again, we should be careful in making confident generalizations about what other animals may like or dislike, or whether music will have the kinds of effects on them we anticipate. Still, the relationship between music and behavior, physical stress, and even health is worth exploring.

What’s the Evidence?

There are quite a few studies looking at the effect of various sounds, including music, on a variety of animal species. Many of these involve laboratory animals, such as rodents and primates. There are also a few looking at livestock species and dogs. A nice narrative review of this literature was published not long ago in the journal Lab Animal:

Alworth LC, Buerkle SC. The effects of music on animal physiology, behavior and welfare.  Lab Anim (NY). 2013 Feb;42(2):54-61.

The clearest answer that comes out of this review to the question, “Does music have beneficial effects on animals?” is, “It depends.” A wide variety of physiologic and behavioral responses to various types of musical stimuli have been observed, and they seem to vary not only from species to species but with the type of music and even with the particular study. And while many of the results are consistent with the investigator’s expectations, there are plenty of findings that were surprising. Music can have effects that might be beneficial or harmful or, in some cases, it might have no measurable effects at all.

As an example of the variety of results, one set of studies showed that playing Mozart reduced blood pressure in hypertensive rats. However, other studies found no effect of Mozart on blood pressure. This may have been due to the use of different specific pieces of music, to differ methods of measuring blood pressure, to differences in the types of rats used, or to any of a hundred other differences between the studies.

Similarly, different types of music seem to have different effects, but the differences don’t always form a clear, predictable pattern. In general, “quieter” types of music, such as classical or “New Age” music seem to reduce physiologic and behavioral responses associated with stress, and other genres either have no effect or may even worsen these measures. However, some studies looking at classical music find no effects, or effects for some composers and not others, and other studies find positive effects from other genres, or even white noise.

Finally, apart from the variety in results of different studies, there are serious problems with the way the effects of music are measured and interpreted. Most of the studies in dogs, for example, contain few of the usual controls for bias and error so important in making scientific research results reliable. Subjects are often chosen and allocated to different groups by convenience rather than randomly, and the investigators observing and recording behaviors are rarely blinded to the music condition.

If you think classical music is more likely to calm dogs than country music and you watch dogs and note their behavior while you and they listen to each kind of music, it is quite likely you will unconsciously interpret what you see and what you think it means through the filter of your own expectations. This is why, after all, controlled scientific research is necessary in the first place.

The authors of this review provide what I think is a very fair summary of what we know so far:

The studies reviewed in this paper show that music can affect the physiology, production, and behavior of multiple non-human species and provide evidence that some effects of music exposure are similar between humans and animals. The studies identify the potential for music to benefit the welfare of animals in many captive environments…As the reviewed studies show, however, the effects of music exposure are not the same for all music styles, animal species, or situations, and playing music for captive species has the potential to increase stress or impair welfare. Therefore, each situation should be considered individually and the outcome evaluated for positive and negative effects.

Bottom Line

The best summary of what we know now about the effects of music on animals comes again from the review cited above:

The studies described above support the idea that physiological and neurochemical changes can result from music exposure. Music’s ability to improve learning and memory and diminish stress suggests that music can result in positive physiologic effects. Other studies, however, suggest that music has no observable effect or can have a detrimental effect on animals. There is not yet sufficient evidence to suggest that animals react physically to music in a way that can be manipulated predictably and consistently.

This means that while it is very likely some kinds of music can be beneficial to our pets and veterinary patients under some circumstances, the devil is, as usual, in the details, and we don’t know much about the details yet. It is reasonable to experiment with music in veterinary environments, especially with more quiet, instrumental genres, but we must try to develop objective measures of the effect to be sure we are not imply wasting our effort or, even worse, actually adding stress for our patients.

As a proponent of evidence-based medicine (EBM), I often emphasize that in the absence of good-quality controlled research evidence, we cannot reliably assess the safety or efficacy of medical treatments. And as a skeptic, I am often forced to clarify that skepticism is not the automatic rejection of the unfamiliar but the position that conclusions about whether specific claims are true or false should be based on reliable evidence. Again, in the absence of such evidence, I do not assume claims to be false–I classify them as unproven. A popular aphorism to express this position is, “The absence of evidence is not evidence of absence.”

However, despite a degree of truth in it, this aphorism is frequently and widely misused. Proponents of unproven therapies often use it to suggest that such therapies should be given the benefit of the doubt, assumed to be innocent (that is, safe and effective) until proven guilty (that is unsafe or ineffective). When I point out on this blog that specific practices have no good evidence to support them (and no, anecdotes and testimonials do not count as “good” evidence), the response from advocates of these practices is often, “You can’t prove X doesn’t work!” That entirely misses the point in several ways.

For one thing, the burden of proof is always on those making a claim, not those asking for evidence to support it. If I say there is an invisible, man-eating dragon in my garage and you doubt me, it is not your job to conclusively prove the dragon doesn’t exist. There is no need for you to take this, or any other claim about the world, seriously until the person making the claim provides evidence for it.

This is especially true for claims that contain within them assumptions that violate well-established facts. Since invisibility and dragons have never been proven to exist, and there is good reason to think they do not, my claim is not only unproven but unlikely. In statistical terms, my hypothesis has a low prior probability.

Prior probability is, in a sense, an exception to the principle that “unproven” does not mean “false.” It is technically true that my guard-dragon is only unproven, not definitively disproven. But in practical terms, the prior probability of its existence is so low that it makes more sense to behave as if it does not exist than to behave as if it does. Would you choose never to enter my garage no matter what just in case the dragon might eat you? Would this be sensible?

The same principle applies in medicine. Many therapies, both conventional and alternative, are unproven in the sense that there is not robust research evidence to characterize their safety and efficacy in all possible situations. One misconception about evidence-based medicine is that such an absence or weakness of evidence means we are to refrain from making any decision about these therapies. EBM does not require that we stand idle with our hands in our pockets whenever there is no systematic review or large, high-quality clinical trial evidence available concerning the therapies we are considering using. We must quantify and acknowledge the uncertainty associated with weak evidence, but this is only one part of the job of balancing the need to intervene with the degree of uncertainty about the consequences of our interventions.

Unfortunately, some proponents of EBM do fall a little ways into this trap, often concluding that in the absence of perfect evidence, “no conclusion can be drawn” and “more evidence is needed.” More evidence may often be desirable, but a clinician working with actual patients must always draw a conclusion, however tentative. That is our job, to guide and care for patients as best we can using the evidence we have, not the evidence we wish we had.

On the other hand, proponents of implausible or unproven therapies often make the opposite error, assuming that the absence of evidence frees them to do as they like without taking into account the uncertainty of not having good scientific evidence. While clinicians may often be forced to rely on clinical experience alone, we should never forget how deeply unreliable a guide this is. As I’ve pointed out before, the three most dangerous words in medicine are “In my experience….” The absence of evidence should not reassure us that anecdotes, personal experience, historical or cultural tradition or any other information with low reliability and high risk of bias can be sufficient to support or recommend a therapy. In medicine, “unproven” may not mean “false,” but it absolutely means “risky!”

Because there is always a chance of doing harm, of making a patient worse when we intervene, it is incumbent on doctors to be wary of interventions with a great deal of uncertainty or a lack of evidence about safety and effectiveness. This is reflected in another popular aphorism in medicine Primum non nocere (First, do no harm). And just as the benefits of a therapy remain unknown when there is no strong scientific evidence, only anecdote and uncontrolled observation, so the safety is uncertain in the absence of good-quality evidence. There needs to be a very urgent need to act, and very clear disclosure of the uncertainty to clients and patients, before we use a therapy when we can’t know the true safety or efficacy.

The necessity to avoid making things worse is a major reason why we generally avoid using therapies without good evidence for their effects. It is widely accepted that if a pharmaceutical company invents a new drug, they don’t start selling it to patients on the basis that it hasn’t yet been proven not to work! These companies are required to go to great lengths to identify the risks and benefits before we are willing to give new medicines to our patients.

A low prior probability makes such a precautionary approach even more appropriate. Even if a therapy is “unproven,” in the sense of there not being much reliable research evaluating it, if the theories and assumptions behind the therapy contradict established scientific understanding, then the burden of proof is even higher, and the principle of avoiding such therapies in order not to unintentionally do harm is even more appropriate.

As an illustration, here are a few “unproven” practices that, nevertheless, most of us would follow despite the lack of controlled scientific evidence because they have a high prior probability of benefitting us:

1. Wearing a parachute when jumping out of an airplane in flight
2. Looking both ways before crossing the street
3. Taking an ambulance rather than a taxi to the hospital after having been shot in the chest

There are no high-quality clinical trials to show that these practices reduce injury or death, but it is rational to follow them anyway because of high prior probability, because they are based on well-established principles and sound reasoning.

The opposite is also true. Here are a few examples of untested and unproven practices that we would avoid despite the absence of controlled scientific evidence because the prior probability of their efficacy is very low:

1. Using one’s Qi or spiritual energy to fly when jumping out of an airplane without a parachute
2. Using The Force to detect oncoming cars when crossing the street without looking
3. Calling a cab to the hospital and waiting by the curb for it to come after having been shot in the chest

The quantity and reliability of the evidence is, of course, often more complex and in greater dispute in medicine than in examples such as these, but my point is that sometimes the absence of evidence should be taken as a reason to avoid a therapy when it has low prior probability of being safe and effective. This is a rational, common practice that we all follow in many other situations, and it makes sense in medicine as well.

Finally, it is important to remember that absolute, 100% proof is never a product of science. Even the strongest evidence can be undone by the discovery of new facts, by unidentified weaknesses in the research, or by rare events. People do survive falling out of airplanes without parachutes.

Proponents of implausible or unproven therapies often trumpet this concept as a way of defending their practices. Since nothing is absolutely certain in science, and since a few crazy ideas have proven to be true, and a few well-demonstrated claims have turned out to be false, it is tempting to conclude that one can believe anything one likes since no one knows for certain. This is pretty obviously a silly and dangerous conclusion. While people may rarely survive jumping out of airplanes without parachutes, that doesn’t make choosing not to use a parachute a sensible decision.

The corollary of the fact that science never provides absolute, eternal truth is that at some point we have to accept the evidence as “good enough.” The precise point will be fuzzy and subject to debate, but in the real world we have to be able to make decisions based on some reasonable level of probability informed by science. If a therapy has been studied extensively over a significant period of time and no evidence of a benefit has emerged, it is not rational to say forever that “no conclusion can be drawn” and “more evidence is needed.” At some point, enough is enough.

The failure to produce good evidence of a meaningful benefit despite reasonable effort is itself evidence that there is no such benefit to find. When a therapy has a low prior probability and fails to be validated after a fair effort, this absence of evidence is evidence of absence of any real benefit. With limited time and resources, we cannot afford to forever keep trying to find evidence for implausible claims that have failed multiple attempts at validation, and we harm our patients by wasting our efforts and resources in such endless and almost certainly futile efforts. The fact that we can never say with 100% certainty that a claim is false does not justify never making the pragmatic decision to ignore that claim and move on to more promising hypotheses.

It is often true that the absence of evidence is not evidence of absence. It is also sometimes true that the absence of evidence is sometimes suggestive that a claim is false, particularly when highly motivated individuals have tried to find or produce positive evidence and have repeatedly failed. And it is always true that we must make decisions in the context of some uncertainty. EBM helps us quantify this uncertainty and integrate it into our decision-making, but it doesn’t obviate making decisions. And sometimes, the most appropriate decision for our patients is that “unproven” means “risky” and, in some cases, “unlikely to be true.”

The core principle of evidence-based medicine is that not all evidence is created equal. Controlled scientific studies provide more reliable evidence than anecdotes because they contain controls for universal sources of error in human perception and judgment. This is a major feature that distinguished science from pseudoscience, real science-based medicine from faith-based practices like homeopathy and energy medicine, in which belief and “seeing with your own eyes” always trumps controlled research evidence.

The fact that controlled research is better than haphazard observation does not, of course, mean that observation is always wrong. Nor does it mean that the results every scientific study are true. In fact, when proponents of alternative therapies try to claim that their practices actually are scientifically validated, they often parade lots of scientific studies that seem to support these claims. When the flaws in these studies that undermine their conclusions are pointed out by skeptics, this is often seen as cheating, as closed-minded rejection of evidence in favor of CAM. However, critical appraisal, the evaluation of the quality and limitations of scientific studies, is another core principle of evidence-based medicine, and it applied to all research, regardless of the kind of hypothesis under study. Scientists, unlike proponents of CAM, are generally their own toughest critics, because rigorous criticism is necessary to weed out error and eventually uncover the truth about nature.

One of the leading scientists working to identify the weakness in published scientific reports, and to develop strategies for correcting them, is John Ioannidis at Stanford University.

Ioannidis, J. Why most published research findings are false. PLoS Med 2(8): e124. doi:10.1371/journal.pmed.0020124.

Despite the inflammatory title of the article, it is actually a tightly reasoned and mathematically rigorous look at specific sources of error in the published scientific literature. Knowing the features that suggest the results of a particular article may not be reliable helps one determine the degree of confidence it is appropriate to have in a particular conclusion. Here are the features Ioannidis identifies as most significant:

1. The smaller the studies conducted in a scientific field, the less likely the research findings are to be true.

2. The smaller the effect sizes…

3. The greater the number and the lesser the selection of tested relationships…

4. The greater the flexibility in designs, definitions, outcomes, and analytical modes…

5. The greater the financial and other interests and prejudices…

6. The hotter a scientific field (with more scientific teams involved)…

Many of these variables are well-known to be associated with false positive results. And while they reduce our confidence in the literature of many fields, generally the CAM literature is weaker in terms of most of these factors than the literature of mainstream scientific medicine. In particular, CAM studies are seldom replicated, usually involve small numbers of patients, often show very small, marginally significant effects, and involve considerable preconceptions or bias on the part of investigators which the studies do little to control.

Many of these factors all hinge on the degree of individual judgment or flexibility allowed in a study. The more choices the investigators have to make, in the design, conduct, and analysis of a research study, the more likely their own biases are to influence the results.  Another article from a few years ago specifically addresses this issue:

Simmons JP, Nelson LD, Simonsohn U. False-positive psychology: undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychol Sci. 2011 Nov;22(11):1359-66. doi: 10.1177/0956797611417632. Epub 2011 Oct 17.

The conclusions of this paper reinforce that of Dr. Ioannidis and also emphasize a point I make here frequently; that when bias and error to creep into scientific research, they are far more likely to create the false impression of a positive result than a negative one. In other words, scientists, like all human beings, see what they want and expect to see, and if a scientific study allows these desires and expectations to influence the results, these results will tend to confirm the investigators’ beliefs even when they are false.

…flexibility in data collection, analysis, and reporting dramatically increases actual false-positive rates. In many cases, a researcher is more likely to falsely find evidence that an effect exists than to correctly find evidence that it does not.

This is, as always, true in all areas, but it is a particular problem in CAM research where investigators almost always begin with strong, nearly unshakeable faith in their beliefs and where research often has poor controls for bias generally.

The point, of course, is not to suggest that scientific studies are useless and we should all go back to believing whatever we like based on our own experiences or anecdotes told to us by others. The point is that understanding the nature and severity of the weaknesses in scientific research gives us power; power to avoid excessive confidence in our conclusions and power to correct the weaknesses that reduce this confidence.

Both Dr. Ionnidis and the authors of the article in Psychological Science offer concrete measures for improving the reliability of published research. Simmons and colleagues offer these suggestions:

These measures mostly involve more transparency in the reporting of how studies are conducted. This should both help us identify weaknesses that might reduce confidence in the results and also encourage authors to address these in the design and conduct of the trial, since they know they will have to disclose them later.

In a recent paper, Dr. Ioannidis also offers some advice for improving the quality of published scientific research:

Ioannidis JPA (2014) How to Make More Published Research True. PLoS Med 11(10): e1001747. doi:10.1371/journal.pmed.1001747

His suggestions are wide-ranging, from improved statistical practices to changing the financial and career incentives for scientists. These address both the way in which personal bias influences results and some of the sources of that bias.

Box 1. Some Research Practices that May Help Increase the Proportion of True Research Findings

• Large-scale collaborative research

• Adoption of replication culture

• Registration (of studies, protocols, analysis codes, datasets, raw data, and results)

• Sharing (of data, protocols, materials, software, and other tools)

• Reproducibility practices

• Containment of conflicted sponsors and authors

• More appropriate statistical methods

• Standardization of definitions and analyses

• More stringent thresholds for claiming discoveries or ‘‘successes’’

• Improvement of study design standards

• Improvements in peer review, reporting, and dissemination of research

• Better training of scientific workforce in methods and statistical literacy

Anything human beings do is imperfect, and this applies to science as much as anything. Often, when I suggest that science is a more reliable guide to how nature works, and what is or is not safe and effective medicine, people object that science conflicts with their personal beliefs or experiences, and they trust their gut or their eyes more than controlled data. But these sources of information contain very little in the way of controls for human error. The evidence of history, and the clear improvement in our health and well-being since we began to apply scientific methods, demonstrate that science is dramatically superior as a means for gathering knowledge than such methods.

It is true, however, that even the best methods we have for obtaining knowledge are still imperfect and still involve some freedom for bias to enter into our conclusions. Relying on science does not mean blinding trusting the results of every single study. Evidence-based medicine requires that we carefully and critically evaluate individual studies and the process of scientific research as a whole, always seeking to identify and reduce error. Merely criticizing or dismissing science as imperfect is not, in itself, useful. Such criticism must be sufficiently specific and focused to allow for strategies of improvement, and must contain at least an implicit recognition that science is still the best tool we have for understanding how nature works.

Human beings are diverse. We are diverse, as individuals and as communities, in our physical characteristics and in our experiences. This leads to a diversity of beliefs and opinions. It is inevitable that we will disagree with other people about many things, large and small. And it is also inevitable that these disagreements will sometimes engender strong, often primitive and unpleasant emotions. However, it is not inevitable, and it should never be accepted or acceptable, that such disagreements lead to violence.

In his book, The Better Angels of Our Nature, Stephen Pinker makes a strong and compelling case that despite the awful things people do to one another daily around the world, and our even greater and more immediate exposure to these acts through the media, we are actually living in the safest and most peaceful time yet in human history. This is not because our biology or our fundamental nature as human beings has changed. It is because we have exercised our inherent abilities to learn, to mature, and to set limits to our own behavior. The result is often called “civilization,” and it represents a realization of at least some of the human potential to manage our feelings and our disagreements in a way that limits the damage they do and allows a measure of peace and freedom to as many people as possible.

Yesterday in Paris, there was a small, horrible, and very clear attack against civilization. Whatever the details of your beliefs, about religion or any other subject, if you have ever had a belief or opinion that you wanted to share, if you have ever disagreed with someone else’s beliefs and felt the need to say so, if you believe there is or can be such a thing as civilization, then you should feel personally attacked and personally threatened by the murder of journalists, cartoonists, police officers, and bystanders at the offices of Charlie Hebdo.

Much of the content of this blog involves criticism and critique of the claims and beliefs of others. I try to make my criticism substantive and respectful and to focus on challenging ideas and claims about the world, not attacking individuals. I do this because I believe it is necessary to give animal owners the facts and information they need to make good healthcare choices for their pets and because I feel an obligation to make a contribution to my profession and the society I live in.  While I do not wish to hurt or offend people, I understand that such critiques will inevitably be taken personally and cause offense. This, again, is in the nature of human disagreements, and it is unavoidable if we are to strive for progress and to separate good ideas from bad ideas through debate and discussion.

As a result of my writing, I have certainly received plenty of hate mail and personal verbal attacks. I have been threatened with legal action on more than one occasion in order to suppress my criticism of others. I have not yet been threatened with physical violence, though some comments have come close: “You are a lying scumbag…… eat dog shit and die !!!! you evil money monger !!!!” But in some small way, I have been involved in the attempt to sustain and contribute to civilization through civil discourse and debate, and I have been touched in a very small way by the impulses to destroy this aspect of civilization that led to the atrocity in Paris. I’m not, in any sense, as brave or important to this process as the people who lost their lives at Charlie Hebdo, but I feel a kinship and a deep respect for their commitment to civilization, which has cost them so much.

I do not read French and have not been a follower of Charlie Hebdo. From what I’ve seen in the last day, I suspect I wouldn’t like or agree with much of the satire for which the people at this magazine were attacked. But that does not lesson my belief that they had the right to produce and publish that satire, and that the attack on them is an attack on anyone who values individual freedom, freedom of belief and expression, or anything that deserves to be called civilization.

As has been said more eloquently by others before, people deserve dignity and respect, ideas do not. Even ideas people feel strongly about, such as religion, must be subject to critical evaluation, even satire, or the kind of growth and progress of civilization that has made life better for all human beings is impossible. People of all faiths and none have appropriately condemned the physical attack in Paris and the symbolic attack against freedom of thought and expression it represents.

I can think of no more appropriate way to reject this kind of barbarism and intimidation than to reproduce some of the cartoons that these attackers felt justified this kind of violence, as well as some of the responses that have been published since. If you are offended, I am sorry. But I would ask you to remember that your beliefs and opinions are just as offensive to someone else somewhere. For any one of us to have true freedom, we must defend freedom for all of us. Je suis Charlie. Nous somme tous Charlie.

In veterinary medicine, we have greater incentives to make our clients happy than most physicians have to satisfy their patients. Pet owners choose their vets, often on the basis of location or personal recommendation, and they can choose to find another if they aren’t happy with the care they and their pets receive. Our clients don’t experience the benefits and discomforts of our treatments directly, and they often don’t have the background or information needed to judge our competence. The veterinarian-client relationship is more about effective communication and personal rapport than it is about pet owners objectively evaluating veterinarians knowledge and skill.

On the whole, I think this is a good thing. I believe veterinarians are often better at communicating with our clients and, somewhat ironically, at the “human” side of healthcare; talking to people about their goals and fears and giving them the information they need to make important decisions. We have to be! It is still unfortunately true that veterinarians often don’t have extensive training in the communication skills that are so critical to our being able to do our jobs effectively, but as a profession I think we have the desire and the incentive to serve our clients as well as our patients as well as possible.

There is, however, a down side to the importance of subjective impressions and communication skills in establishing and maintaining relationships with pet owners. Most owners judge our manner, and how well what we say fits with their own beliefs and values, but most cannot objectively judge our competence or the accuracy of what we tell them. Therefore, clients can easily misjudge the quality of the medical care their pets are receiving.

I once worked with a doctor whose clinical practices would have been considered grossly outdated and unacceptable by the standards of almost any other veterinarian. Yet his personal charm and comforting manner immediately engendered trust and loyalty from clients. It didn’t seem to matter how his patients fared, his clients adored him even when their pets received inadequate or ineffective care and did not improve.

In contrast, I have also worked with a vet who was intelligent, compassionate, and skilled and to whose care I would have trusted my own pets without hesitation. Yet clients consistently disliked her and distrusted her recommendations due entirely to her reserved manner. It is a reality that vets are judged on how they communicate more than on the truth of what they say or the objective quality of their medicine. We must accept this and make vigorous and good faith efforts to communicate more effectively with clients so that we have the opportunity to care for our patients.

I feel, however, that it is worth warning pet owners that the reliance on somewhat superficial personal attributes in judging whether or not you have found a good vet can be misleading. A recent and stark example of this is a glowing testimonial recently posted on the web site of the American Holistic Veterinary Medical Foundation (AHVMF).

This pet owner feels she has received the best possible care for her pet, and because her pet has done well, she believes the judgment and recommendations of her vet to be correct. An evaluation of some of these recommendations informed by science, however, shows that many of these recommendations are complete nonsense, and it is very likely this patient is thriving despite rather than because of the veterinary care discussed in this testimonial.

The case was a new puppy with, according to the anecdote, skin and ear infections and intestinal parasites. The first warning sign is the blame given to vaccination for the skin and ear problems.

Our doctor suspected that the rash was most likely an allergic reaction to the vaccinations he had received from the breeder so…we began working to bolster Ceelo’s immune system.

While acute hypersensitivity reactions do occur in response to vaccination, the limited information here does not suggest that was what this pet was experiencing. Vaccine reactions are quite rare and quite distinctive, and the mistrust of vaccines induced by incorrectly blaming them for unrelated medical problems does real harm.(some facts about veterinary vaccines) And if this were a case of a vaccine reaction, it would represent an inappropriate and excessive immune system response, so the notion of “boosting” the immune system is not only nonsense, it is exactly the opposite of what ought to be done.

Things go from bad to worse as the testimonial describes the use of “various supplements,” which in most cases in veterinary medicine are almost never supported by good evidence and have significant problems with quality control and potential risks. The fact that this use of supplements was guided by “muscle testing,” a term often used to describe the quack practice of applied kinesiology, is a further red flag. Inevitably, unproven but passionate claims about the value of raw diets appear as well:

I switched him off of kibble and on to raw venison mixed with simple fruits and vegetables. This had a huge impact on his skin.

However, the ultimate leap from dubious practices to quackery comes with the mention of Traditional Chinese Veterinary Medicine (TCVM).

…his skin was still red and inflamed, especially at night. Our doctor mentioned that in Chinese medicine, that was the Liver’s way of releasing toxins from the system so we then worked on purifying his liver and eliminating anything that might be an allergen…our integrative vet who took one look at Ceelo’s tongue and knew immediately that something was not right with his Spleen. He then did muscle testing to confirm his initial diagnosis. He prescribed a Chinese herb along with whole food and glandular supplements.

The principles of TCVM, including tongue diagnosis, are pre-scientific myths and metaphors that have never been reliably linked to health and disease through scientific research. While some of the herbal remedies used likely have beneficial compounds, the lack of a rational system for employing them and the lack of appropriate testing to determine their real risks and benefits make the system as a whole as likely to do nothing or even hurt patients as to benefit them. The addition of the nonsense of detoxification, applied kinesiology, and glandulars make this a classic description of pseudoscience which should serve as a warning to anyone thinking of bringing their pet to this veterinarian.

Unfortunately, the interpretation given by the client, and the message of the testimonial, is exactly the opposite. Despite the advocacy of multiple kinds of unproven and outright quack therapies, this vet has a happy, loyal client. Why?

Well, I don’t know anything about this doctor personally, but I suspect like most vets he or she is a genuinely smart, caring person. He/she probably demonstrates true concern for patients and clients alike and likely offers advice and recommendations in a confident, comforting manner. All of these characteristics are necessary to an effective veterinarian/client relationship. Unfortunately,  none of them have anything to do with whether or not the medicine being employed is effective or nonsense.

I also have clients who like me and trust me to care for their pets. Yet my approach to medicine is very, very different form that described here. The fact that we both have clients who are happy with the care we provide for their pets doesn’t say much about which style of medicine is superior because clients don’t judge us on the basis of the objective truth of our knowledge or effectiveness of our interventions, since these are not accessible to them.

“But,” this client might object, “the pet got better so that must be evidence the medical care was effective, right?” The core concept of the entire scientific method, and certainly of this blog, is that such anecdotes cannot be trusted and often do not mean what we think they mean. I have written about this often, but the bottom line is that it is deeply misleading to say that when an individual patient does well or badly that this validates or invalidates the treatment given. If this were true, science would be unnecessary, and yet the evidence of history is quite clear that science works far better than anecdotal methods or evaluating medical treatments. Here are a few reminders of why:

Medical Miracles: Should We Believe?
Testimonials Lie
Alternative medicine and placebo effects in pets
Placebo effects in epileptic dogs

Apart from the effect of a nice manner, and the ultimate positive outcome for the patient, the other reason why I suspect this client views her story as confirmation that she has a great vet is that she and the doctor share some key aspects of their world views. If this client were a scientist or a skeptic with a strong commitment to objective evidence and science-based medicine, I doubt she would have stuck with unproven recommendations based on pseudoscience for very long. We are naturally inclined to seek confirmation of our beliefs and to reject challenges to them, and this influences who we seek information from and how we interpret that information. Clients may like or dislike their vets personally, and they may believe the care their vets provide to be excellent or terrible based on the outcome, but they also tend to view more sympathetically advice consistent with their existing beliefs.

Bottom Line
It is natural to trust people we like, and this plays a large role in how pet owners judge their veterinarians. It is also natural to believe that something has been done right when things go well and that something has been done wrong when things go badly, though this is often not a reliable way to judge the quality of the care one’s pets receive. And finally, we are all predisposed to seek confirmation of our beliefs and to avoid challenges to them, which leads us to prefer vets who seem to think the way we do about health and disease, science and nature, and all the values-laden subjects that touch on veterinary medicine.

All of these inclinations are unavoidable, and for better or worse they influence the veterinarian-client relationship. Vets should recognize this and make their best effort to understand and respect their clients’ values and perspectives. We need not agree on everything with our clients, but we cannot help them or their pets if we cannot communicate effectively. I have plenty of clients with whom I disagree about the merits of specific therapies but with whom I get along great and who trust that I have their pets’ best interests at heart and the knowledge and skills to give them the best care.

On the other hand, clients should be aware of their own tendencies to judge their vets on the basis of factors that may not really have much to do with the quality of the medical care they provide. Without a thorough education in science and medicine, clients are stuck trusting their vets to some extent. Just as I cannot expect to effectively judge the competence or skill of my airplane pilot or tax accountant without having expertise in those domains, so pet owners have to recognize that while they have the right to control the care of their pets, they often don’t have the knowledge or skills to do so alone. If they did, they wouldn’t need a vet at all! And this means they must take a certain leap of faith when choosing a vet.

The key, then, is how one evaluates whether this faith is justified. The ability to empathise and communicate is essential, but it is not sufficient to mark a veterinarian as a skilled doctor. Ideally, an open and explicit adherence to established and accepted scientific standards of care would be the best way to know if your vet is doing the best they can for your pet. By this standard, an anecdote like the one on the AHVMF web site is a warning, not an endorsement!

Of course, if you doubt the value or reliability of science and science-based medicine, and if you already have a preference for the alternative philosophical perspective underling alternative medicine, well adherence to scientific principles and methods won’t seem a very reliable guide to quality to you. But at the least pet owners should be aware of the limitations of these natural human tendencies to trust those we like personally and who seem to agree with us. And we should recognize that individual anecdotes, good or bad, aren’t really a reliable way of evaluating the competence of those experts we hire to guide us in areas where our own expertise is insufficient, whether in medicine or other fields. Testimonials are far more persuasive than they are trustworthy.