The End of Chiropractic? Of Course Not.

At last, the definitive nail in the coffin of chiropractic? Hardly. A recent study in the journal Chiropractic and Osteopathy uses an epidemiological approach to examine the question of whether the founding “lesion” of the chiropractic philosophy, the vertebral subluxation, can be shown to be associated with any disease. Their conclusion reads like a Science-Based Medicine or Quackwatch summary:

“Subluxation was not found to be linked to any one disease complex…There were no studies that offered a biological plausibility that would isolate subluxation as a causal factor in disease. There were no studies linking the subluxation as a coherent construct and supported by generally known facts about the natural history and biology of any disease. There were no studies found that suggested the subluxation as a causal agent similar to other factually demonstrated causal agents…There is significant lack of evidence to fulfill basic criteria of causation. This lack of crucial supportive epidemiologic evidence prohibits the accurate promulgation of the chiropractic subluxation.”

Short and sweet, the basic abnormality chiropractors claim to fix cannot be shown to exist or to cause any disease. One might think this would be a fatal blow to the enterprise of chiropractic, but sadly that is not the case. For one thing, it has been shown before in other studies that the subluxation cannot be reliably identified by chiropractors and that when pushed to demonstrate it in some objective way, most chiropractors and their lobbying organizations engage in some impressive yoga to cover the fact that they cannot. But as chiropractic is fundamentally faith-based medicine, being unable to demonstrate that the disease they are treating exists doesn’t worry chiropractors. Even those who disavow the subluxation theory happily continue the sort of manipulative practices Palmer originally invented to fix the supposed subluxation, and they simply justify it with the time-honored refuge of all woo, “Well, I don’t know how or why it works, but it works!”

Of course, this conclusion cannot be reliably demonstrated by objective evidence either. Though chiropractic does seem to have some benefit on subjective perceptions idiopathic lower back pain, the fact is it has been shown to be at best roughly equivalent to conventional therapy (rest, NSAIDs, physical therapy, and patient education materials) or to a good massage. If it were truly risk free and presented truthfully as offering mild benefits for back pain, I would have no objections. And a few chiropractors have adopted this approach in an attempt to take what benefit their interventions might provide out of the realm of faith and into that of responsible medicine, but these are a minority and often reviled within their own profession. But while I believe evidence is critical and should be the cornerstone of medical practice, I also understand that it is not as compelling as personal experience for many people, and the first step on the road to recovery from addiction to unscientific therapies is to understand we have a problem and need something better than our own experience and intuition to validate or invalidate the safety and efficacy of medicine. So while this should be a critical study undermining the claims of mainstream chiropractic, it has garnered little attention and will probably have little impact on the popularity of the method.

Posted in Chiropractic | 2 Comments

Benefits & Risks of Neutering–What does the science say?

Many pet owners have questions about neutering their pet dogs and cats, and while there is much useful information available on the Internet, there is also much misinformation. I have composed a literature review looking at the benefits and risks of spaying and neutering. It is too large to post directly here, so I have uploaded it as a .pdf file on SkeptVet.com. I hope also to soon have a simplified, tabular version for those disinclined to wade through the detailed consideration of all the pros and cons.

http://www.skeptvet.com/web_documents/NeuterProsCons.pdf

Posted in Science-Based Veterinary Medicine | 3 Comments

Homeopathy Works for Arthritis–Or Maybe Not

I recently ran across an article in the journal Evidence-Based Complementary and Alternative Medicine (eCAM) titled Evaluating Complementary Therapies for Canine Osteoarthritis–Part II: A homeopathic Combination Preparation (Zeel) (Hielm-Bjorkman, A et al 2009:6(4)465-471).

According to the authors, “a homeopathic combination product (HCP) for canine osteoarthritic pain was evaluated in a randomized, double-controlled and double-blinded clinical trial…[and] that the HCP (Zeel) was beneficial in alleviating chronic orthopedic pain in dogs, although it was not as effective as Carprofen.”

There are many levels on which any clinical research article should be critically evaluated. The potential biases of the authors and the journal, the quality of the methodology, the statistical analysis of the data, and the degree to which the conclusions follow from the data are all common criteria by which such publications can be judged. R. Barker Bausell in his book Snake Oil Science does an outstanding job illustrating some reasons why not everything that makes its way into a scientific journal is reliable science and why such critical evaluation is necessary.

In this case, the journal makes some effort to follow the principles of evidence-based medicine, but it is guilty of some serious Tooth Fairy Science, in which rigorous methodology is applied to some fundamentally irrational premises. Skimming through some archival issues also indicates a pretty strong preference for publishing positive findings for CAM interventions. None of this automatically invalidates anything published, but it is one factor to consider since the effect of personal as well as financial biases on research outcomes is well established.

As for the authors, I am not able to establish much about their biases. Two of the authors are professors at the University of Helsinki School of Veterinary Medicine, and the lead author’s research summary  suggests a strong attachment to CAM. Her doctoral dissertation was a study on gold implantation, green-lipped mussel extract, and Zeel for use in canine osteoarthritis, and it appears she is publishing this thesis research as a series of articles in eCAM.

The methodology is generally sound, with a couple of exceptions. First, while the product studied is identified as homeopathic and an inject able version of it is listed in the US Homeopathic Pharmacopoeia, even the authors insert the caveat that “this is not a classical homeopathic treatment.” The preparation contains 14 listed ingredients, many of which are present after having been diluted 1:10 only 2-8 times, for “molar concentrations of 10-5 to 10-12mol/L.” Such concentrations are low, but still higher than the usual case for homeopathic preparations, which cannot conceivable contain any of the original ingredient. It is possible, then, that this substance could contain some pharmacologically active substances. The ingredients listed (same as the injectable product) are:

 

  1. Arnica montana, radix (mountain arnica)
    Dulcamara (bittersweet)
    Rhus toxicodendron (poison oak)
    Sanguinaria canadensis (blood root)
    Symphytum officinale (comfrey)
  2. Mineral ingredients:
    Sulphur (sulphur)
    (alpha)-Lipoicum acid (thioctic acid)
    Coenzyme A (coenzyme A)
    Nadidum (nicotinamide adenine dinucleotide)
    Natrum oxalaceticum (sodium oxalacetate)   
  3. Animal-derived ingredients
    Cartilago suis (porcine cartilage)
    Embryo totalis suis (porcine embryo)
    Funiculus umbilicalis suis (porcine umbilical cord)
    Placenta suis (porcine placenta)

The subjects were appropriately randomized into treatment, placebo, and positive control groups, with Carprofen as the positive control. The subjects in each group all appear to be comparable at baseline. The placebo group did have higher baseline scores on 5/7 measures of pain, but the authors state that no statistically significant differences were found between groups at this point.

The placebo control was not ideal. The treatment product was visibly different from the Carprofen and the placebo (which were identical to each other). The owners were given extra Carprofen in its original packaging for rescue, so clearly they would be able to identify the treatment product as different. In addition, all subjects also received an inert capsule as part of a separate study, so while the Zeel group received “an ampoule of clear liquid” once daily and “a slightly green (lactose) capsule,” the Carprofen and placebo groups received the green capsule and “a white pill” twice daily. It is not clear what if any affect such a discrepancy might have had on the subjective assessments of owners, or potentially blinded investigators who might have detected group assignment from comments made by owners.

Most of the assessment measures were subjective, such as owner rating scales or visual analog pain scores and investigator clinical exam assessment. Some force plate analysis was done, though this proved problematic. Two subjects had to have their force plate measurements discarded because they were too lame to allow accurate measurement. These subjects, however, were all in the placebo arm and so this would be expected to have the effect of decreasing perceived efficacy of the treatment.

The force plate measurements were “repeated until sufficient valid results were obtained for both left and right limbs.” It was not stated whether the number of trials needed to achieve this differed between groups, which could have affected the results is some subjects had to run back and forth significant more than others to get a valid reading, which might itself affect the reading.

The biggest methodological problem I see in the study is in order to calculate the percent of subjects improved or not improved in each group at the 8-week assessment period, “the results of each variable were converted into dichotomous responses of ‘improved’ or ‘not improved.'” Converting scale variables into dichotomous variables can exaggerate differences between groups. If the measurement was unchanged, the subject would be classified as “not improved,” but if there was even a miniscule change from baseline then the subject would fall into either “improved” or not “improved category.” Thus, subjects with dramatic improvements in scale measurements would be weighted the same as subjects with marginal, and likely clinically insignificant changes in the variable. Without the raw data, of course, it is impossible to tell what if any effect this procedure might have had on the final conclusions. However, the tabulation of the data presented in the article appears to show much greater improvement in terms of the percentage of subjects improved than in terms of the actual median improvement of the variables themselves, suggesting that in fact such an exaggeration occurred.

The authors also stated that “for dogs that had used extra Carprofen more than three times per week at W8 [4 dogs in the placebo arm] we changed all their variable values at evaluation W8 into the most negative value measured at that time.…to counteract the effect of the NSAID…” This seems a clear fudging of the data which made the placebo group appear to have worse outcome measures than it actually did. Certainly, it is possible that these dogs needed more Carprofen than the treatment group because the treatment was having a beneficial effect. But it is just as possible that the placebo group took more Carprofen because of differences in disease severity, in owner attitudes or behavior, or some other factor. And what makes the arbitrary designation of three times a week an appropriate justification for altering the data in this way is unclear. In any case, the effect of this decision is to make the outcomes appear worse for the placebo group, which clearly makes the treatment group outcomes appear relatively better.

Patients given Carprofen clearly showed improvement over baseline at a rate significantly higher than placebo. 67-86% of subjects were categorized as “improved” for the various outcome measures, and the actual values for each measure were improved from 2-5 times more than for the Zeel group. In the treatment group significantly more subjects were classed as “improved” compared to placebo in 3 out of 6 measures. Again, this is likely inflated by the conversion of scale data to dichotomous data. The Zeel group also showed significantly greater improvement than the placebo group in 4 out of 6 specific measures, though for one the P value was 0.049, quite close to the cutoff of 0.05.  

The authors also state that use of supplemental or rescue Carprofen occurred in 14% of the Carprofen group, 28% of the Zeel group, and 8% of the placebo group. Though they claim that the only significant difference was between the Carprofen and placebo groups, this is puzzling, both because the Zeel group had a percent of rescue use dramatically higher than the other groups, and because of the earlier statement about manipulating the data for the placebo group to “counteract” the effect of Carprofen use for that group.

No differences in bloodwork values or clinical side effects were seen between groups.

The authors also make the unsupported statement that “it is generally accepted that seasonal differences influence OA, with patients being worse in cold, damp and unstable weather.” A number of studies have found this traditionally assumed relationship to be difficult to verify and likely a minor and insignificant factor in arthritis pain for most patients (1, 2, 3), so it does not qualify as “generally accepted.” Nevertheless, the authors go on to claim that a trend observed of worsening symptoms for the placebo group during the treatment phase of the study and subsequent improvement during the post-treatment follow-up was due to the weather, and that the opposite trend seen in the Zeel and Carprofen groups was due to the effects of the treatment agents. It seems more likely that the placebo group simply different in significant ways from the other groups, which casts further doubt on the conclusion that the test product was of meaningful benefit.

The authors conclude by putting the usual best possible spin on the weak results, suggesting that combined with in vitro results reported elsewhere they justify further research and pointing out that NSAIDS, which even they acknowledge are clearly superior for treatment of pain, have side effects, despite the fact that none were seen in this study. As I’ve said before, the resource limitations on research in veterinary medicine requires the most efficient use of those resources to maximize benefit, and such studies of implausible interventions are not going to benefit our patients. The authors clearly wish to find something positive in their results, but the study does not justify the commitment of more time and resources and talent to this methodology when better therapies are already available, and when decades of research on homeopathic preparations has failed to validate them. Such papers provide the aura of scientific legitimacy to such methods, but they are tooth fairy science, not evidence-based medicine, and they are a dead end we would do well to stop travelling down.

Posted in Homeopathy | 9 Comments

Internet Information — The Good, Bad, and Ugly

As we all know, the Internet is a mixed blessing. It makes enormous amounts of information easily and cheaply available to billions of people. Unfortunately, not all information is created equal, and sometimes bad information is worse than none. Contrary to what many of my clients assume, I actually appreciate it when they come to me with questions or suggestions based on their own research. However, when they have been misled by unreliable sources, a great deal of their enthusiasm and effort ends up being wasted.

I am hoping to contribute eventually to a more comprehensive set of evidence-based veterinary medicine resources under the auspices of the EBVMA, but in the meantime I have assembled some general guidelines and specific  sources to trust, as well as some to avoid. I hope this is helpful.

General Principles for Evaluating Internet Information

1. Consider the Source: We need EBM precisely because even the most intelligent and honest of us cannot avoid the influence of bias on our conclusions. The universal cognitive biases and perceptual biases we all share, as well as the more obvious errors associated with strong feelings and preconceptions make a truly objective evaluation of any complex phenomenon impossible. The scientific method compensates for such sources of bias imperfectly, but better than anything else yet discovered.

When evaluating information from the Internet, it is important to consider the possible sources of bias lurking behind it. These biases do not automatically invalidate the information by any means, but being aware of them helps inject a salutary element of skepticism about their validity.

Obviously, anyone trying to sell you something is likely to believe quite deeply in what they are selling. Apart from deliberate scam artists, most people don’t enthusiastically promote or make a living selling things they aren’t strongly committed to, so they are unlikely to be truly dispassionate or objective. Confirmation bias is particularly likely to lead people with a financial interest in something to ignore potential problems with it. CAM proponents never tire of making this charge against mainstream medicine, especially the pharmaceutical industry, but they conveniently ignore their own financial interest in the products and services they provide.

Strong ideological positions also call into question the level of fairness in the presentation of an argument. It is so difficult to overcome the influence of one’s deepest held values and philosophies that it is often most useful to simply declare them up front and let those who are trying to puzzle out the truth take them into account. When looking at information provided on the Internet, it is often useful to find sites committed to both sides of an issue and compare the information and arguments.

Sometimes such ideological biases may be hidden, usually in an attempt to prevent people from recognizing them and the influence they may have over the information being provided. However, one advantage to the Internet is that it is often fairly simple to find out a lot about the affiliations and associations of people offering information to the public. My recent post on the association between conservative Catholics and anti-vaccine activists is an example of how hidden agendas behind information presented as objective and scientific can be uncovered.

Finally, while blind reliance on “experts” is a form of faulty reasoning known as the argument from authority, it is still true that specialized training and experience in an area does give somewhat more weight to one’s opinions on the subject. The idea that anyone can be an expert on any subject simply doesn’t hold water in today’s complex and technologically sophisticated world. Personal experience is especially unreliable as a source of deep insights that an entire profession of specialists have missed. A few hours on the Internet can’t make me into a nuclear physicist, and it can’t make a physicist into a veterinarian. So while no one is automatically right just because they are a specialist or professional in a subject, it is unlikely that dilettantes and self-made experts are going to have insights or wisdom that such professionals lack.

2. Check References: In CAVM it is very common to cite scientific sources in support of mistaken arguments. It is also often the case that when one reads the original source, it doesn’t say what the person citing it claims it says. Taking information or quotations out of context, overlooking obvious flaws in methods or argument, and simply cherry-picking sources that say what you want them to while ignoring those that don’t are all practices that make such research citations unreliable. So whenever possible, check the original source to see if it really does support the argument you’re evaluating.

3. Be Reasonable: If it’s too good to be true, it probably isn’t. Extraordinary claims require extraordinary evidence. Pick your cliché, but ultimately reality is complicated and messy and often not how we’d like it to be, so anyone who tells you it is simple and you can have your cake and eat it too is probably deluded or lying.

 

Trustworthy Internet Resources

These are resources that I believe provide information that is reliable. Being run by human beings, they may not always be right, but they are not regularly, egregiously wrong or out to push a product. Some may have obvious biases in favor of science and methodological naturalism, but if one rejects these positions then one has already decided rational and objective inquiry is impossible or undesirable anyway, so trying to evaluate the rationality and objectivity of Internet information isn’t likely to be a concern. Postmodernists and those who see faith and revelation as the keys to truth aren’t likely to find any of this useful anyway.

Many of these resources address human medicine primarily, and many include subjects outside of medicine all together. There is an unfortunate paucity of good resources for science-based veterinary medical information on the Internet, though obviously I am trying to change that! however, despite the dangers of extrapolation across species, the information gathered in human medicine can be used to assist our judgments regarding veterinary medical questions.

The SkeptVet- Though this blog is the more active part of my Internet project, I have assembled a collection of my more detailed and researched articles and my downloadable client information pamphlets on specific veterinary topics on my main web site. No commercial conflicts of interest, and my ideological biases should be quite obvious.

Evidence-Based Veterinary Medical Association (EBVMA)– This is an organization of veterinarians and other professions, both in academia and private practice, dedicated to promoting evidence-based veterinary medicine. Any veterinarian interested in supporting high quality, science-based veterinary medicine should join and get involved. And a new web site, with practical information and EBM tools for vets is coming soon!

VeterinaryWatch– This site contains a lot of good, science-based information and references, though unfortunately not in a very user-friendly form. I am working with several others to try and improve the design and content of the site, but it is a slow project.

Quackwatch– By far the most extensive collection of resources regarding CAM. Reviews of many products and therapies, warnings about corporate and individual providers or CAM services and information, and links to many other reliable resources.

Science-Based Medicine– Far and away the best blog on science and evidence-based medicine and a great resource on CAM. Intelligent, thoughtful, and well researched essays on a large variety of topical as well as perennial issues.

The Cochrane Collaboration– The premier site for EBM in the human medical field. Though some resources are available only on a subscription basis, the most helpful resource are summaries of the independent and rigorous systematic literature reviews on a large number of specific topics.

PubMed– An enormous and easy to use database of the medical literature. Abstracts are available for most articles, and links to sources for the full text. Unfortunately, full text access for many articles is available only to subscribers of the publishing journal, but the world of medical publishing is changing, albeit slowly, and more journals become accessible all the time.

Free Medical Journals– A convenient way to identify journals that offer free online access to their content. While many of the highest impact journals in human medicine are open access, far fewer veterinary journals are available–yet.

 

Unreliable Internet Resources

While the number of commercial organizations and individuals pushing misleading information or outright nonsense on the internet is vast beyond imaging, this is a list of some of the more popular, and hence more dangerous. And while the distinction between bad science, pseudoscience, and plain quackery can sometimes be a tough call, a place to start for the more egregious cases is The Quackometer. This automated tool evaluates websites for language patterns typical of pseudoscience and medical quackery. Of course, I would never suggest letting a machine make your decisions for you, but I’m surprised by how often I agree with the little black duck!

Shirley’s Wellness Cafe– This is perhaps the most egregious woo site I’ve found to date. Vicious and paranoid condemnation of all scientific medicine as paternalistic, venal, and harmful. Wild claims about safe, natural, free cures for almost everything. No logic or data, only the presumption that if it is in any way scientific it must be bad for you. And despite the railing against the greedy medical establishment, there sure seem to be a lot of things for sale!

Healthy Pet Journal– A sad example of veterinarians giving up on science in favor of blind faith and intuition. The site doesn’t have the hysterical shrieking tone of Shirley’s, but that makes it even more dangerous since the information is presented as if it were reasonable and scientific, when it almost universally is neither.

Naturallycomplementary.com– A beautiful, well-organized, easy to use resource for avoiding science-based medicine and finding all things CAVM. A huge variety of unrelated and mutually contradictory approaches are promoted, with the only apparent unifying feature being claims of “natural,” “alternative,” “holistic” and so on. Words which have lost whatever meaning they may once have had and have become merely shibboleths for anti- and pseudoscience. I tried to register as a member of the forum to offer a different perspective, but apparently the management is not interested in allowing members to hear other points of view, and they denied my application.

Academy of Veterinary Homeopathy– A professional organization of veterinarians devoted to this tooth fairy science.* (sigh)

American Holistic Veterinary Medical Association– Ditto. This one is good for some laughs if you check out the offerings at their annual CE meeting.

Sadly, I could go on endlessly. While thankfully science-based medicine is far more widely available and accepted than CAVM, promoting CAVM on the Internet is a viable commercial activity, while promoting EBVM and skepticism is a strictly pro bono volunteer task. Consequently, the volume and shiny façade of the CAVM offerings will always outshout and outshine the more rational alternatives. But once you examine a few examples of unreliable sources, the patterns of language and argument become obvious, and your internal quackometer should steer you straight.

 

*”This study falls into the category of what I call Tooth Fairy science. You could measure how much money the Tooth Fairy leaves under the pillow, whether she leaves more cash for the first or last tooth, whether the payoff is greater if you leave the tooth in a plastic baggie versus wrapped in Kleenex. You can get all kinds of good data that is reproducible and statistically significant. Yes, you have learned something. But you haven’t learned what you think you’ve learned, because you haven’t bothered to establish whether the Tooth Fairy really exists.” Harriet Hall, MD

Posted in General | 3 Comments

Skeptical Media

Yes, I’ll take the credit for seeing this one first. 🙂 The New York City Skeptics blog Gotham Skeptic has a post about recent pro-science articles appearing in the mainstream media. The post suggests we might finally be emerging from what a friend of mine refers to as the Golden Age of Woo and into a time when skepticism and sound science are again acceptable public positions to support.

They first mention Amy Wallace’s incisive piece in Wired magazine “An Epidemic of Fear: How Panicked Parents Skipping Shots Endangers Us All”, which has been discussed several times over at Science-Based Medicine. They then mention several other examples, including a piece by the Associated Press reporter Marilynn Marchione, whom I previously lauded back in June for her excellent reporting on CAM and pseudoscience. Hmm, I wonder if this might be evidence that I have psychic powers….. 😉

Posted in General | 1 Comment

CAM Tries to Cash in on Swine Flu (H1N1) Fears

As I’ve pointed out before, the image of CAM providers as selfless promoters of health without the venal concerns for profit of mainstream medicine and Big Pharma is advertising spin, not reality. A nice example of that has been the recent attempts of many snake oil peddlers to cash in on fears about swine flu (H1N1 Influenza). The Food and Drug Administration (FDA) keeps a list of Fraudulent H1N1 Products. Of course, not all the products are CAM-related, since greed knows no loyalties. However, the majority of the products making unsubstantiated claims about preventing or treating the swine flu make similar claims about disease and health in general, and they include a number of CAM standbys.

Dr. Andrew Weil, a paragon of woo, has received a cease and desist order from the FDA regarding claims, subsequently removed, about his dietary supplement line. The FDA list contains a large number of herbs, vitamins, supplements, and even some teas. Despite the frequent claim that Big Pharma has no interest in these kinds of therapies because they cannot be patented and so there is no money to be made from them, there seem to be a lot of folks making money selling them for many uses, whether they work or not.

Some products are obvious attempts to cash in on flu fears, such as the Flu Away inhaler containing eucalyptus and tea tree oil. Others play more subtly on the fears of H1N1, marketing bogus “information” about the flu along with a broad collection of products and services designed to “promote health.” It is encouraging that the FDA is doing what it can, with limited resources and less public and even government support than it should have, to prevent unscrupulous individuals and companies from making a profit selling useless products and a false sense of security to people with legitimate concerns about the H1N1 pandemic.

Posted in General | 2 Comments

Skeptical Dinosaur Enters the Modern Age

Ok, after much prodding I have set up Facebook and Twitter accounts, so anyone interested in skepticism and veterinary medicine, or for that matter Celtic Folk music, science fiction, or an eclectic array or other inteelectual and aesthetic pursuits should look me up there! 🙂

Posted in General | Leave a comment

123rd Skeptics’ Circle is Live!

Check out the 123rd Skeptics’ Circle. This is a regular blog carnival with posts from all over the world covering topics of interest to skeptics and critical thinkers. This week, you might even see a couple of posts you recognize! 🙂

Posted in General | Leave a comment

Approaches to Uncertainty in Medical Decision Making

When I was training to be a veterinarian, an older vet once described the nature of his work this way, “I make decisions for a living.” After more than 8 years in practice, I know exactly what he meant. The process of taking the information I get from a pet owner and from my physical examination and then using it to set up a diagnostic and treatment plan involves making many, many decisions. Which pieces of information are important and which irrelevant? How reliable are the owner’s observations? What information from the pet’s previous history might shed light on the current problem? What additional information do I need to make a diagnosis? What diagnostic tests are likely to be safe, useful, affordable, and logistically feasible? What is the true diagnosis and what treatment should I recommend? How do I assess the response to treatment, both beneficial and adverse effects? What do I do if the owner declines some or all of my recommendations? What ultimately is in the best interests of the pet and will best promote their health and well-being? And on, and on, and on for multiple patients all day every day.

This is often an intellectually exciting and stimulating process. It is also often frustrating, especially when information I need is unavailable, or when I make the wrong decision, as I inevitably will sometimes being human. One of the greatest challenges, in both a positive and negative sense, is dealing with uncertainty. Lay people often view clinical medicine as a rather mathematical process. The see the doctor as taking historical and physical exam information, and maybe some diagnostic test results, plugging them into an algorithm established by experience, tradition, and scientific research, and coming out with The Answer printed in black and white at the bottom of the paper like an old-fashioned adding machine. When the doctor expresses uncertainty or inserts caveats into his description of the diagnosis or treatment, the client is likely to assume this represents a gap in the clinicians knowledge or competence. The Answer is presumably out there waiting for the right doctor to ask the right question or run the right tests and find it.

While there are particles of truth in this image, it ignores many sources of uncertainty. I’ve already mentioned one which bedevils the veterinarian, which is the often surprising lack of knowledge owners may have about their own pets. Historical information is notoriously unreliable, as are the theories, often supported with internet citations, that owners bring as to the nature of their companion’s problem. Now I realize the owner is most often the single best source of information about the pet, but even the most astute owner may not notice things that only a trained health care provider would notice, and many owners unfortunately can’t even tell me whether their animal is eating or eliminating normally.

Obviously, the information that can be obtained directly from the patient is often limited. Animals can show signs of pain, pruritis, and other physical sensations that might indicate what the problem is, but they cannot give the sort of precise descriptions of their symptoms we might hope for. And despite the claims of innumerable self-professed dog/cat/horse whisperers out there, there are serious dangers to accepting as reliable one’s own intuitive impressions of what an animal is feeling, as I discussed in my post on Animal Hospice care.

Other sources of uncertainty include the fundamental limits of our understanding. Though the progress of the last century in understanding the complex relationships that constitute physiology and the various causes and risk factors associated with disease is absolutely astounding, and continuing at a breakneck pace, the reality is there is much we still do not know. And the volume of what is known is so vast that it is impossible for any one individual to have an adequate command of all the relevant and constantly changing information needed to properly diagnose and treat any condition. Atul Gawande illustrates this limitation in human medicine beautifully in his book Complications, and it is even more of a factor for veterinarians who rarely specialize to the extent MDs do and who treat multiple species.

And while I take a risk venturing into the territory of epistemology, I think it is likely that much of the true, underlying nature of complex phenomena, like organisms and their health or diseases, is probabilistic rather than strictly deterministic. Certainly, in medicine we can often say with great accuracy how a group of patients with a particular condition will do over time based on research, but we are rarely able to say with any certainty how an individual patient with the condition will fare. While this may be due, to a large extent, to the lack of adequate knowledge and measurement capacity, it may also be due to a fundamental unpredictability to complex systems. Chaos theory, quantum mechanics, and the philosophical approaches grouped under the label of  indeterminism all address this possibility, and while it is true that these theories may not always be applicable to the problems of  everyday life (especially in the case of quantum mechanics, which applies, as far as we can determine, only at the subatomic level), they do suggest that not all natural phenomena are going to be amenable to precise, accurate prediction regardless of how accurate and detailed our information about them is.

Given, then that we must make important decisions with significant consequences on the basis of imperfect information and with the understanding that we will never be as accurate in our assessments and predictions as we would wish, what is the best strategy for making the necessary decisions anyway? First, there is a psychological hurdle we must overcome. I have seen some new veterinarians absolutely paralyzed with indecision when they first encounter the imperfect world of clinical medicine outside of the university. And some eventually give up practicing because it is too difficult to accept the reality that they will make imperfect choices and mistakes despite their best efforts, and that some of these will result in harm to their patients. For most of us, though, we must find the confidence necessary to deal with this reality and to accept that we can only do the best we can and that however imperfect it is better than if we did not even try.

I have seen far more veterinarians, unfortunately, cope with the uncertainties of medical decision making through an excess of confidence in themselves and their choices. Given that we do not have the detailed accounting of outcomes that MDs often have in hospitals, there are rarely formal mechanisms in place for a vet to determine how well they are doing their job, either in an absolute sense or in comparison to their peers. Confirmation bias, cognitive dissonance, and many other normal human cognitive predispositions make it very difficult for us to accurately assess our own performance. Like most people, we will tend to see what we want to see, which is that we are competent, caring, and effectively helpful. And while this is true for most veterinarians, the reality is that we likely overestimate our own abilities and underrate our flaws just like everybody else.

Other vets will deal with uncertainty by slavishly following formulae. Memorizing simple guidelines along the lines of “If THIS, then THAT” and applying them to every case is comforting in that it essentially eliminates decision making all together. Of course, it also eliminates assiduous observation and critical thought, which are essential for the practice of good, sophisticated medicine. One of the common objections to evidence-based medicine is that it is simply a way to enforce this sort of robotic algorithmic approach on everyone. As I’ll discuss, this isn’t really true, but the objection is at least founded on an appropriate disdain for such an approach.

It will come as no shock to those who are familiar with this blog that my answer to dealing with the uncertainties of medical decision making is to apply the paradigm of science-based medicine (SBM — strictly speaking, this is different in subtle but important ways from evidence-based medicine [EBM], but for present purposes they are essentially the same). I view taking on the SBM approach as something a bit like embarking on a 12-step recovery program. Below, I outline and discuss some of the steps involved. (There is, of course, a certain whimsical tone in such a presentation, but the content is an honest representation of the approach I believe veterinary medicine needs).

1. We must begin by admitting we have a problem. This is probably the most important, and certainly the most difficult of the steps. We must acknowledge that our decisions are limited by inevitable uncertainties and that while we can never achieve perfection we can do better with help than we do with a traditional reliance on our own experience, intuition, knowledge, and skills. The greatest resistance I get from other veterinarians to SBM stems from the idea that they don’t need it because they can reliably assess themselves and their diagnostic and treatment approaches just fine. A lot of what this blog is about is pointing out how mistaken this excessive and misplaced confidence is. I acknowledge that it is a natural response to the distress caused by uncertainty, but it is not the right strategy to cope with this distress.

2. Having accepted our limitations, we must be willing to overcome them. This involves educating ourselves about the specific weakness and limitations that diminish the quality of our decision making, It also involves taking concrete steps to address them even though this means letting go of familiar patterns of behavior and strategies that we are comfortable with and trying out new and unfamiliar approaches. We must have an ongoing, never-ending willingness to identify and acknowledge weakness in our knowledge and our practices and the courage to make changes even in long and dearly-held beliefs and behaviors.

3. Accepting that we need to change, we must acknowledge that there is a higher source of knowledge and evaluation than experience or tradition. This is, of course, the scientific method. We will rarely have as high a quality of evidence as we might like given the financial and logistical barriers to clinical and basic veterinary medical research. But we must be committed to accepting the conclusions of the best available evidence as more reliable than those of lower-quality sources of evidence such as our personal experiences. If well-designed and conducted research contradicts my intuition or personal clinical experience, I must be willing to defer to the more reliable source of information.

I do understand how very difficult this is for people in general, and doctors in particular I suspect. But it has been shown time and time again that the beliefs and intuitions of human beings, even multiplied by many people over extended periods of time, are inferior to the conclusions of reliable scientific research. The progress in medicine and the increase in the length and quality of our lives since we began moving away from faith-based and tradition-based medicine and towards science-based medicine is dramatic and incontrovertible.

4. We must then proceed to educate ourselves, our colleagues, and our clients. We must set up systems for generating reliable scientific information and making it easy to access for general practitioners. The Evidence-Based Veterinary Medical Association (EBVMA) is a group dedicated to doing precisely this, and I encourage all veterinarians and pet owners to support their work. Setting up resources such as online journal access, evidence-based reviews along the lines of the Cochrane Reviews, and other such tools would make it easier for practitioners to employ a science-based strategy.

However, they will only turn to this model if we who understand its benefits can make the case to our colleagues that they should, for the sake of their patients and clients. By education and by example, we have to show other veterinarians how the inevitable uncertainties in medical decision making can be minimized by an SBM strategy.

And we must educate our clients so that they can be more effective participants in their pets’ healthcare. Clients who understand the nature of medical decision making will be better able to make sound, informed choices among the options offered to them by their veterinarians. Client values and resources are a key factor determining what we can do for our patients, so we must include them in the decision-making process for practical as well as ethical and philosophical reasons. And the frustration and miscommunication that all too often taints the veterinarian/client relationship will be lessened if our clients understand that medicine is neither a straightforward mathematical process with clear right and wrong answers nor a mystical and vague “art” that depends solely on the talent and wisdom of a particular clinician.

As I mentioned earlier, the objection often made to SBM and EBM approaches is that they seek to impose a cookie-cutter uniformity and robotic system of rigid decision trees on clinicians. This might be true in a deterministic world with the availability of perfect information, but in the real world there will always be uncertainties and gray areas that require a thinking and caring doctor. Contrary to the impression I sometimes give, I believe that clinical experience and intuition are forms off evidence. They are very low-level evidence, and far less reliable that higher-level forms, but they are often all we have to go on, and they can serve us well if carefully and judiciously applied, with great humility. And even when the evidence is strong and we can define the medical parameters of a situation well, medical decisions often depend on non-medical decisions about values and interests. We may be able to say very accurately what the likely treatment outcomes are for a particular disease given the alternative choices, but that doesn’t necessarily tell us what the right thing to do is for a given patient and their family.

And in order for the veterinarian and client to work effectively as a team for the best interests of the pet, the vet must be able to understand and communicate the nature of the medical situation, the options available, and the inherent uncertainties in the decision-making process to the pet owner. All of these are reasons why even in the best case where SBM guides our practice, there is a need for the experience and skills of the veterinarian.

Many of my biggest concerns about CAM and the philosophy that underlies much of it are related to how uncertainty is handled. Many of the inappropriate responses to uncertainty I see mainstream veterinarians take sometimes are proudly adopted as foundational principles in CAM. Often uncertainty is denied entirely, with CAM practitioners claiming all diseases can be definitively traced to a simple causal schema (unbalanced ch’i or humours, innate intelligence blocked by subluxations, toxins, and so on). Denying differences between individuals, species, or diseases and blaming all illness on universal vitalist forces gone awry is the ultimate abdication of responsibility for making careful, informed medical judgments.

Uncertainty in treatment outcomes are often denied as well. Worsening of the patient’s condition is frequently taken by homeopaths to be a sign of improvement, a so-called “healing crisis.” Of course, improvement in symptoms is also taken as a sign of improvement, so there’s no way to lose! As I’ve related before, CAM believers are often unwilling or unable to admit any room for improvement in the wisdom received from millennia or centuries or decades of tradition. While I believe the best response to uncertainty is open acknowledgement of it and an ongoing effort to improve and change, tradition-based practices are founded on the principle of preserving and not questioning received wisdom. And the reliance on anecdote and testimonial commonly seen in CAM venerates the individual experience and intuition. However, I believe less rather than more respect for this form of evidence is the key to more effectively managing the uncertainty in decision making.

Finally, in case I haven’t emphasized it sufficiently, I believe that a degree of uncertainty and unpredictability is intrinsic to medicine and we will never be free of it. Accepting this ourselves as veterinarians or our clients as owners to make perfect decisions is vital to providing good care. Recognizing our limitations not only helps us accept a better methodology, but it frees us from the illusion that we can be perfect or that we can control all outcomes for all of our patients. This makes us not only better doctors but, I suspect, happier people.

Posted in General | 2 Comments

The Doctor Is In

As a beginning blogger, one of the most exciting, and challenging things about running this site is finding new topics of interest to readers. Some of my favorite posts have come from questions or suggestions from readers. So I just wanted to officially announce that I welcome questions or suggestions for topics. I may or may not always have time to investigate and write about complex topics in detail, of course. But my goal for this site is to provide an informational resource for CAM skeptics, especially though not exclusively those interested in veterinary CAM, so letting me know what interests you, what sort of veterinary CAM you run into, and what you have questions or concerns you have is very helpful.

To facilitate this, I have added a Contact Form to The SkeptVet Blog. So tell me what you think. Thanks!

Posted in General | Leave a comment