Chiropractic: The More We Look, the Less We Find

Posted on March 20, 2011 by skeptvet

Edzard Ernst recently commented on two Cochrane reviews involving chiropractic, which drew my attention to these reviews. I have to admit finding the results a little surprising.

In general, the total body of evidence fails to support most of the claims chiropractors make about the nature of illness and the beneficial effects of chiropractic. However, there has been some decent research evidence to support chiropractic as a treatment for uncomplicated lower back pain, no more nor less effective than conventional treatment. Despite the theoretical problems with chiropractic, and all the associated nonsense (colon cleansing, cold laser therapy, supplements, and so on), I’ve always accepted that at least there is some legitimate role for chiropractic treatment in the management of back pain.

However, the Cochrane reviews that Dr. Ernst reports cast some doubt on even this claim. The updated review of spinal manipulative therapy finds good quality evidence only for ” a small, statistically significant but not clinically relevant, short-term effect on pain relief.” The reviewers could not determine whether this effect is any greater than placebo because only poor quality studies with a high risk of bias investigated this question.

Chiropractors, or at least “mixers,” often claim that they do more than just manipulate the spine. A new review looked at research for this combined chiropractic interventions for low back pain. The conclusions:

while combined chiropractic interventions slightly improved pain and disability in the short term and pain in the medium term for acute and subacute low-back pain, there is currently no evidence to support or refute that combined chiropractic interventions provide a clinically meaningful advantage over other treatments for pain or disability in people with low-back pain. Any demonstrated differences were small and were only seen in studies with a high risk of bias.

So the best we can say is that spinal manipulation alone has a small, short-term impact on back pain, probably not enough to matter to patients and certainly no better than conventional therapy. Combined or “holistic” chiropractic care does no better. So the research evidence, after more than a hundred years of study since chiropractic was invented, clearly shows no benefit for any problem other than uncomplicated lower back pain. And even for this indication, the benefit is small, possibly no greater than placebo and certainly no greater than conventional therapy. So is this benefit worth being exposed to the rampant pseudoscience promulgated by so many chiropractors, the aggressive marketing of “preventative” chiropractic treatment, and all the other nonsense that so often accompanies most chiropractic treatment?

Posted in Chiropractic | 30 Comments

SkeptVet Gets Hate Mail

Posted on March 14, 2011 by skeptvet

This blog grew out of my own efforts to educate myself and my clients. I began looking into alternative medicine in veterinary school (about 14 years ago now, gasp!). Having heard about James Reston’s supposed acupuncture anesthesia in the 1970s, I assumed acupuncture was a generally effective therapy, and I wanted to learn how to use it. However, the more I learned, the less I believed about the claims made for acupuncture, and the more aware I became of the role of our beliefs and desires in interpreting the evidence of our senses. I gradually learned to have a healthy awareness of our inherent limitations as human beings that lead us to mistaken beliefs about health and disease, and of the great value of science, however imperfect, in compensating for these limitations.  

Once in practice, I was called upon almost daily to offer my professional opinion on many common questions from clients about how to best restore or maintain their pet’s health. In response to these questions, I began actively and critically investigating and evaluating the evidence for a wide range of medical interventions, both conventional and alternative. I began writing a series of handouts on these questions, covering common CAM therapies such as homeopathy, acupuncture, chiropractic and so on, as well as questions about conventional interventions such as neutering and vaccines. This series, and my growing personal and professional involvement with evidence-based medicine, led to this blog.

The primary purpose of this blog is to make both the specific information I discover and the general principles of science-based medicine available to anyone who is interested. While SkeptVet is a nearly microscopic fish in the Internet ocean, readership has grown quite a bit over the nearly two years of its existence. As near as I can tell from the comments and emails I receive, my audience is a mix of veterinarians and other veterinary professionals and pet owners and breeders. I sometimes get great questions which lead me to investigating and learning about subjects I would have otherwise been unaware of. And I get some very rewarding feedback, by comment or email and, occasionally in person, from people who have found my posts informative, interesting, or helpful in some way. That keeps me slogging away since that is, after all, the point.

But of course many of the responses I get are highly critical. Some of the criticism I receive is thoughtful, informed, and civil and provides a beneficial and necessary impetus for me to challenge and question my own beliefs and assumptions. That I always welcome. Unfortunately, most of the critical responses I get are not thoughtful, informed, or even civil. There are a few “stock” ideas in many of these responses, and quite a variety in terms of styles and degree of eloquence.

My general policy is that I approve any comment to the site regardless of the content so long as it is not simply gratuitous personal abuse or completely unintelligible (and, of course, spam which is blocked automatically for the most part). I have only had to block about half a dozen comments since the blog began, so I think my policy is quite liberal. However, it amazes me how irate some people get at the mere fact that I moderate comments at all, as if I were somehow obligate to let them post anything they like, no matter how vitriolic or incoherent, on a site I created and maintain as an informational resource.

I thought it my by interesting, and even entertaining, to take a look at some of the criticism I receive and to see if any meaningful patterns can be. So far, I’ve come up with three basic categories (and excluding, of course, the most vicious and profane, which I simply delete unless I think it may one day be needed to obtain a restraining order).

I. Who the hell do you think you are? I’m as smart as you and I’ve seen XX work, so who cares what you think? Science doesn’t know everything and neither do you!

I’ve tried to address the issue of anonymity and my background, as well as the issue of how unreliable the “I’ve seen it work” argument is, in the FAQ, as well as in many posts and comments I make. Still, this is by far the most common critical comment I get. Here are some examples.

My dog has been diagnosed with Cushing’s disease 2 years ago. She was very sick. I did what the vet said but it all just got too expensive. I tried Superglan and she doesn’t even seem like she has the diesease. I think everyone has their own opinion and if you want to try it, then try it. Who cares what everyone else thinks!

Traditional Chinese Medicine has been around for thousands of years, and is a proven methodology of health care. Just because it doesn’t follow the Western medical culture doesn’t mean it doesn’t exist!… It is a shame that no Western medical studies have been done on this product- because that is the only way that close-minded people like you will ever even give it a try.

I started off taking you seriously, but after reading a few of these exchanges, you lost all credibility with me. Someone who was truly unbiased would at least consider the fact that it MIGHT work. Even if it has not bee scientifically proven, a true skeptic does’t come to ANY conclusions without evidence. You have clearly come to a conclusion without evidence, and your bias became clearer and clearer the further I read. Bias is nearly always based on something personal. So did you have a negative experience with this product, or a similar one, or are you being paid?… basically this was just a rant attacking a lack of controlled study, and showing your disdain for intuition and the powers of observation of pet owners. If you want to discard human powers of discernment and behave like a robot, or a computer, that’s fine…Clearly you expect to only encounter only non thinking persons. Your whole act of “Let me, the expert, the person who is more informed than you, enlighten you about critical thinking and scientific study so you do not harm yourself in your ignorance” is just as bad as any snake oil salesman. You can only peddle that crap to someone who doesn’t know enough to spot bullshit and call it…If this is just a hobby, playing expert on a product based just on looking at a website… well… Maybe you should try going out? Or knitting? Something productive?

My arguments are based on practice and outcome. I fix dogs that have suffered from commercial dog food related diseases with plenty of cases in hand…It’s not about science, it’s about results that speak for themselves.

You lack humility and exude arrogance not because you follow your conscience but because you believe that those who don’t agree with your premise must be wrong. You say that, “Being critical of bad ideas and false claims is also not arrogance, it is conscience.” How is it that you have the wisdom to decide which ideas are false and which claims are bad?… When confronted with evidence from pet owners and other vets you dismiss it with the back of your hand as “unscientific”. That is the hallmark of arrogance from a pseudo intellectual. You essentially are saying that your opinion is better than their observations. That is what I meant when I suggested that you need to develop a little humility….You ought to consider using that superior intellect of yours to actually do some research and solve some medical mysteries. To paraphrase Woody Allen, those that can’t do teach and those that can’t teach author critical blogs about those that do.

Had I found your website first, I likely would have not ever tried Supraglans. Im glad things worked out the way they did. As far as testimonials, or more specifically, personal experience with this product being worthless and not scientific enough to be valid, I thought clinical trials were pretty much a give it to em and see if it works deal.

It seems like I exposed your thin skin and tweaked one of your ivy league nerves. Maybe it’s the old doctor-as-god complex rearing its ugly head again. How dare a mere civilian untrained in the ways of the medical profession question my superior authority and intellect when I have the scientific method to validate my truth. Once again, even with the scientific methods that you are so proud of, the “truth” evades you…Your post, more than anything I could ever write, shows the incredible arrogance of your profession. Do you not see it? Are you so full of yourself that you can’t open your mind to the possibility that you don’t possess all of the knowledge worth knowing in the healthcare universe?… You, even with your so-called scientific methods and Ivy League education are no better than me or countless other caring pet owners who don’t care where you went to school or how many footnoted treatises you read to formulate your scientific opinion. You are wrong as much as you are right that’s why they call it “practicing medicine”. Keep practicing, good doctor; maybe someday you will get it right.

If you have had extensive experience in an alternative field… but months, if not years in an alternative field, and you still found that it did not work…. I would commend you for having an open mind and actually trying it. But you seem to be putting down every modality without experiencing it for yourself. Have you tried every modality to validate everything you write about? You can’t write about acupuncture and say it doesn’t work, if you have not followed a case from begining to end, with an experienced vet doing it… and found that it did not work….I think your problem is not that there is no evidence saying that alternative medicine works, but the fact that you have your fingers in your ears like a little kid saying lalala I can’t hear you. When you live your life with blinders on, you miss out on alot.

With nearly half of all dogs passing from cancer, the cancer rate in cats nearly doubling since the distemper vaccine was made a yearly thing, I think we need to look beyond the blinders that were placed on our faces. We don’t have to be one extreme or the other. We need to all come together for the health of our animals, and our own health. ..This type of aggression, stress and hate is extremely unhealthy for us. High blood pressure and heart attacks to name a few.

i don’t care what research has been done the evidence is in the results… But when you use it on a daily basis and see results you will never be able to tell me it doesn’t work because frankly it’s just not true.

All I have to add to this is that acupuncture does work. I’ve seen it work. To me that is all that matters.

II. You are only saying this because you’re afraid you’ll lose business. You must be a tool of Big Pharma. You vets make more money if our pets are sick, so you don’t want us to have safe, cheap, effective natural remedies.

Obviously this website is biased against complementary integrative veterinary medicine….keeping comments and ideas one sided and supported pharmaceutical and commercial pet food monopolies which have been raking in the money for many decades…Threatened financially and ideologically, they must resort to political tactics of attack, shock and awe using headlines inspired by the National Inquirer or some other ladies gossip rag…If you were a legitimate blog looking for the Truth and not a shill for the pharmaceutical companies, you would have researched both sides of any issue.

The old 70 year old urologist told me I was crazy for believing that acupuncture helped my stone. I think he is crazy for thinking I would have let him make another 15 grand off of a surgery that caused me more pain and suffering than I had ever experienced. I wish someone would have told me that I could have just bought a 12 dollar bag of herbs. They don’t taste so great, but hey, it beats the heck out of feeling like the mob got a hold of your kidney with a bat. Or should I say the rich Mob Doctor MD.

I believe that traditional veterinarians are today motivated by GREED and the medicine they practice does as much harm as much as it helps…And how about the local vet’s push for more and more, now found to be harmful, vaccines they are always telling us pet owner are needed- just so they can make more money, not to mention the those oh so toxic flea meds,

it looks like the only faith you have is in your holier than though self. You remind me of our consulting vet. you can show him and show him, but when it comes down to the bottom line what the drug companies will do for him, he will jump on their bandwagon even when it doesn’t work

After reading your “rant” (so accurately described by another reader), I am left feeling like you must have a financial interest in big pharma, for that is the only logical reason I can see that you would put such effort in condemning a product that has successfully treated and prolonged the lives of so many animals… Maybe you should conduct a clinical research study with neo and see the results for yourself? Oh wait, you wouldn’t make the money using neo as you do with chemo and radiation, my bad. It sure does seem like your motives are financial,

III. Just plain angry

Thanks to scientists. we don’t believe in religion any more. Huge department stores have become our Mecca. Now we are told CAM is bunk and not to believe it.

What I object to is this dictatorial attitude of medics that we should obey them or else, and their efforts to shut down anyone who opposes them. They have become the new Popes of the scientific religion!

Its a shame there are Veterinarians out there that think like you and thankfully they are becoming fewer by the day…It would be a sad day for all animals if they had only you to treat them for cancer.

All this talk about science vs. marketing and that only a vet (or doctor) that practices with prescriptions and surgery are worth anything. This is the same mentality that has almost all of Americans frail and sick! My god, when is every person going to realize that we are living breathing cells who need nutrition and balance, not knives and drugs

Posted in General | 21 Comments

Irreverent Comic Guide to Warning Signs of Quackery

Posted on March 10, 2011 by skeptvet

Anti-Science Red Flags of Quackery

From comic blogger Sci-ence (sorry, can’t get the schwa character in there).

Posted in Humor | 13 Comments

Is Recommending Glucosamine for Arthritis Evidence-Based Medicine, or Wishful Thinking?

Posted on March 9, 2011 by skeptvet

In January, I discussed growing skepticism about the benefits of oral glucosamine and chondroitin as a treatment for arthritis in dogs and cats. In that post, I made reference to a short feature I wrote which appeared in the Journal of the American Veterinary Medical Association (JAVMA) examining the clinical trial evidence for the use of glucosamine and chondroitin. In this article, I stated:

Oral administration of glucosamine and chondroitin is often used for prevention and treatment of osteoarthritis in dogs, and there is widespread belief in the safety and efficacy of this practice. However, it is important to base recommendations to clients on the best possible research evidence and not solely on the popularity of a practice or anecdotal reports of positive outcomes…

[T]here was insufficient evidence to support a recommendation of glucosamine and chondroitin as an alternative to NSAID medication for treatment of clinical signs attributed to osteoarthritis in dogs…Glucosamine and chondroitin are perhaps the most widely used nutraceuticals for treatment of osteoarthritis in human and veterinary patients. It is worth considering, however, that there is only very weak clinical trial evidence to support this practice and that it is appropriate for veterinarians to temper their recommendations to their clients accordingly.

A recent commentary in Veterinary Practice News by Dr. Narda Robinson, a professor of “integrative” veterinary medicine at the Colorado State University, and occasional participant in discussions here, takes issue with the conclusions of my review. I have a lot of respect for Dr. Robinson, and I think we agree on many issues associated with evidence-based veterinary medicine, but there are some fundamental disagreements I think are uncovered by her approach to evaluating the evidence concerning the use of oral glucosamine and chondroitin for arthritis.

She begins by referring to recent analyses  and clinical trials in humans, which add to the already existing data, that seem to indicate glucosamine and chondroitin are no better than placebos in treating arthritis pain. However, even in referring to these results, Dr. Robinson begins to lay the foundations of her ultimate conclusions, citing authors of glucosamine studies to illustrate that no one other than “skeptics” such as myself seems to care what the evidence says. This seems distinctly at odds with her claims to an evidence-based approach.

After finding no clinically relevant effects on perceived joint pain or joint space narrowing, the authors wrote, “We are confident that neither of the preparations [glucosamine or chondroitin] is dangerous.  Therefore, we see no harm in having patients continue these preparations as long as they perceive a benefit and cover the costs of treatment themselves.”

From another glucosamine review, “[I]t is likely that most consumers find the presence or absence of clinical evidence demonstrating efficacy to be irrelevant.”

According to Dr. Robinson, while “unsure of glucosamine’s benefits, many veterinarians nevertheless err on the side of hope…” and prescribe one of the plethora of possible glucosamine and chondroitin containing products on the market. She then refutes a couple of possible objections to the claims that oral glucosamine and chondroitin are beneficial for arthritis treatment. In my opinion, these refutations vary in quality from clearly correct, to correct but unbalanced, to outright misleading. I will address each briefly.

Specific Factual Points

1. Oral absorption of Glucosamine:

Dr. Robinson- Measurable amounts of glucosamine can be absorbed and reach the blood and the joints after oral administration at clinically reasonable doses.

SkeptVet- I agree. While the study cited was conducted in horses, which would not necessarily be relevant in dogs and cats, similar studies suggest low but measurable bioavailability for these species. This establishes the plausibility of oral glucosamine, though of course it doesn’t directly demonstrate any meaningful clinical effects.

The levels found in plasma and joint fluid after oral administration in these studies are often significantly lower than in many of the in vitro studies that show effects of glucosamine and chondroitin of the synthesis, degradation, or metabolic activity of cartilage cells (for an excellent review, see the introduction to this study). So the clinical relevance of the bioavailability of these substances remains to be demonstrated.

2. Glucosamine Doesn’t Build Cartilage

Dr. Robinson- “Glucosamine serves as a precursor for glycosoaminoglycans, the main component of joint cartilage.”

SkeptVet- I agree, sort of. Though glycosoaminoglycans are made naturally starting with glucose, glucosamine is an intermediate in the pathway to synthesizing them, and it appears that providing cartilage cells in vitro with extra glucosamine can affect this synthesis. It is far less clear, however, whether this actually happens in the joints of living animals given oral glucosamine. This is really what matters, of course, so referring to glucosamine as a precursor in cartilage production can be a bit misleading because it implies that taking glucosamine stimulates cartilage production in patients with arthritis, which may not be true.

The rat model study she refers to did find some observable effect on cartilage damage and metabolism in rats given glucosamine, and a similar rabbit study also found some effects. Both studies, however, also found no significant change in some measures of cartilage loss, and the subjects had extensive cartilage damage even when given glucosamine, so whether the supplement would have a meaningful clinical benefit even in these models, much less the very different environment of a cat or dog joint with naturally occurring arthritis, is hard to say. Human trials are also mixed, with some showing a beneficial effect on cartilage and others not. So a real benefit is possible, but certainly not proven. 

4. Glucosamine Doesn’t Reduce Inflammation

Dr. Robinson- “[glucosamine] reduces release of inflammatory mediators. Glucosamine appears to penetrate inflamed joints more readily than healthy ones…”

SkeptVet- I agree, sort of. Certainly, glucosamine given orally and absorbed at low levels into the bloodstream is likely to penetrate inflamed joints more readily than healthy ones. This is just a function of the increased permeability of capillaries in areas of inflammation, and it is true for most substances in the blood stream. It doesn’t tell us if the glucosamine is doing anything useful in these joints. And, of course, it actually argues against the notion sometimes advanced that glucosamine may have a protective benefit for joints not yet affected by arthritis, since apparently even less of the absorbed glucosamine is likely to penetrate into such joints.

As far as the anti-inflammatory effects of glucosamine and chondroitin, they are certainly established in vitro. There is some debate about whether or not they are clinically relevant effects at the concentrations actually achieved in joints when they are given orally. Much of the in vitro research involves either amounts of these agents that are much greater than achieved in actual patients or study of individual inflammatory markers or chemicals, which may or may not tell us much about the effects in the “real-world” environment of an arthritic joint. 

6. Glucosamine and Diabetes:

Dr. Robinson- “oral glucosamine/chondroitin does not affect glycemic control or lead to diabetes in the short term. But this research does not answer the question of whether glucosamine is safe for diabetic dogs.

SkeptVet- I agree, and I would probably state the case even more strongly. Given the extensive evidence in human diabetic, and the much more limited research in cats and dogs, I think it very unlikely that glucosamine would cause clinically meaningful problems for diabetic pets. Just as early studies on the benefits of glucosamine were not supported by later, better research, so many of the early studies on the risks of the supplement have not been sustained. The research in pets is not definitive, but the balance of the evidence available is pretty clear. 

7. Clinical Benefits of Glucosamine

This is one area in which Dr. Robinson and I pretty clearly disagree. I think she is correct that the results of clinical trials are conflicting, but I think it is a mistake to believe that this means no firm conclusion on efficacy can be drawn. In his book Snake Oil Science, published in 2007, R. Barker Bausell reviewed the clinical research on glucosamine in humans exhaustively, and the balance of the evidence was clearly against any meaningful clinical effects. The larger and better designed or controlled the study, the less likely it is that a benefit will be seen. Studies funded by companies marketing glucosamine products are much more likely to be positive than studies funded by more neutral parties. And since this book was published the studies and reviews I provided links to above have continued to produce mostly negative results.

The evidence concerning the clinical effect of glucosamine in humans is not absolutely uniform, of course. But it never is. There is a well-known process, sometimes called “the decline effect,” by which early studies, often small and poorly designed and often funded and or run by companies or individuals with vested interests in a given hypothesis, and later studies by other researchers with better samples and designs fail to confirm the initial findings. This is not a flaw in science but rather an example of why methodological quality and independent replication of results are so crucial in separating the wheat of truth from the chaff of all the great idea that ultimately turn out to be wrong. Glucosamine research is almost a paradigm of this process, so it is misleading to point to the inconsistency in clinical research results as if it indicated deep uncertainty, when in fact it indicates a gradual refining of the data leading to the conclusion that a meaningful clinical benefit in humans is quite unlikely.

Dr. Robinson raises the issue of heterogeneity to explain the inconsistency in clinical trial results. Heterogeneity simply means that any population of subjects in a clinical trial will, of course, not consist of identical individuals. It is possible that the intervention being tested might work for a subset of individuals with particular characteristics, but if we don’t know what these characteristics are, randomization will scatter these folks evenly between the placebo and treatment groups, and it will look like the treatment doesn’t work. Heterogeneity is a known limitation of clinical trials and population statistics, which can to some extent be controlled for but which will always present the risk of study results which are valid for populations but not for some individuals.

This is a legitimate concern, but unfortunately it is also an easy way to cast doubt on all clinical research and lay the foundations for the argument that clinical trials ultimately can’t tell us anything useful about how to treat individual patients. It is important to be careful that our concerns about the issue of heterogeneity does not become a kind of clinical trial nihilism that leads us to give up entirely on the immense proven value of clinical trials in guiding medical interventions. Put another way, nothing is perfect, including clinical medical research, but we must be careful not to let the perfect become the enemy of the good to the point where we cannot make meaningful conclusions based on the evidence that exists. This is a caution commonly directed at skeptics who challenge the quality and conclusions of CAM research, but it is also applicable to proponents of CAM who argue that the imperfections of science invalidate the conclusions it reaches on their preferred methods. I believe Dr. Robinson herself appreciates the value of clinical research, but the heterogeneity argument she uses is one that can easily be taken to unfortunate extremes, and has been by others.

The clinical research on glucosamine/chondroitin in dogs and cats is far less in quantity and quality than the human clinical trial data. In my JAVMA paper (which, incidentally, was not a complete review of the subject but only an illustration of a practical and efficient use of evidence-based medical reasoning to answer a focused clinical question), I mentioned two clinical studies. One found no benefit for glucosamine and the other showed little benefit. Both showed far greater and more predictable benefit to NSAID therapy, which is a consistent feature of clinical research on glucosamine and chondroitin.

In her commentary, Dr. Robinson mentions another study which I did not address in my JAVMA piece, since this study involves cats and my article only addressed glucosamine for dogs with arthritis. It is not really a direct investigation of glucosamine/chondroitin supplementation for arthritis but rather of dietary therapy with a diet that includes glucosamine and chondroitin along with other ingredients thought to be beneficial for arthritis, but it is an interesting study which deserves a closer look.

The study was a well-designed randomized, blinded trial with a 70-day monitoring period and a mixture of objective and subjective measurements. 40 cats completed the study, evenly divided between the test diet and a control diet identical except for the absence of fish oils, glucosamine, chondroitin, and green-lipped mussel extract. Obviously, one issue with this study is the presence of multiple ingredients aimed at treating the subjects’ arthritis, which makes it impossible to say anything definitive about whether the glucosamine and chondroitin were ore were not beneficial.

Another concern is that there was one significant difference between the control and test cats, namely that the control cats started the study weighing more and, while both groups lost weight during the study (though not very much), the control cats weighed significantly more at the end of the study. Weight is a very important factor in the clinical symptoms of arthritis. Heavier cats would be expected to have worse symptoms and more rapid progression, and weight loss would be expected to improve symptoms all by itself, so this could confound the findings of this particular study.

As usual, the results were mixed, but as I read the paper they strike me as not very impressive. Subjective pain scoring by owners (who may or may not have been effectively blinded to the treatment and placebo allocation, it’s impossible to tell from the report) improved for both groups, but the difference between test and control groups was not significant. This may represent a “placebo” effect of being enrolled in a clinical trial, or  less likely and effect of weight loss.

The objective measure, using an automated activity monitor, showed no overall significant difference between the groups. Interestingly, the control group showed significantly less activity overall and at 2 of the four times of day analyzed when 14-day periods at the beginning and end of the study were compared. A complex regression analysis looking at diet, weight at the start of the study, and weight loss during the study seemed to show an increase in the evening activity level of cats on the test diet, and no change in the activity of these cats at any other time. This analysis also showed a decrease in the activity of the cats on the control diet at all times except overnight.

I am not qualified to evaluate the validity of the statistics involved, but it seems odd that if the test diet had a beneficial effect on arthritis it would show up as a decrease in the activity of the cats on the control diet rather than an increase in the activity of the cats on the test diet. The authors suggest that this may mean that the cats on the test diet had less progression of their arthritis, but I am doubtful that such significant changes represent the progression of arthritis symptoms over only 70 days.

Other subjective measures, such as overall owner-assessed quality of life, improved for both groups with no significant differences between them. A few measures did change significantly in each group from the beginning to the end of the study:

Decreased aggression and eating in control group
Increased sleeping in the control group
Increased jumping and eating in the test group
Decreased sleeping in the test group

There were also some subjective owner assessments of behavior change during the study that differed significantly between the groups. Playing and interaction with other pets increase for both groups, but more for those on the test diet than those on the control diet. Both groups slept less and hid less over the course of the study, but again the change was greater for the test diet group than the control group.

Subjective measures of pain on physical examination by veterinarians in the study showed improvements in both groups but no significant difference between them.

So overall the study shows a lack of significant difference between the groups in most measures but a few differences that do reach statistical significance. The overall pattern is not consistent or compelling, but it may be that the cats on the control diet became less active over the 70 days of the study, which could conceivably suggest some influence of diet on activity. However, most measures improved for both groups, which illustrates the non-specific treatment effects (aka “placebo” effects) that are such a problem in figuring out what are real treatment effects in clinical research studies and what are artifacts of the research process.

In any case, even if the test diet did have real effects on the behavior of the cats in the study, that doesn’t tell us if the effects were clinically meaningful, if they had to do with treatment of arthritis symptoms, or if they were due to glucosamine. Yet the study is cited in Dr. Robinson’s article with the clear implication that it is positive evidence for the benefits of oral glucosamine in the treatment of arthritis. This is a seriously misleading oversimplification which is unfortunately all too common in debates about the merits of medical interventions, particularly those for which the research data is not absolutely unequivocal, which it rarely is.

The Bottom Line

So clearly Dr. Robinson and I agree on many things. We agree that glucosamine can be absorbed in low but measurable amounts when given orally, that some of it reaches joints, and that in vitro at least it can have anti-inflammatory effects. These facts establish the plausibility of glucosamine as a treatment for arthritis, but not that it actually works.

We also agree that it is most likely very safe. There is some risk associated with the poor regulation and quality control of nutraceuticals generally, which often contain dangerous contaminates and other ingredients not on the label. And I have had clients who eschewed proven therapies, such as NSAIDs, because they felt glucosamine alone was sufficient arthritis therapy, which leaves animals with inadequately treated pain. But Dr. Robinson carefully points out that she would not recommend this kind of single-agent use, and overall I think we agree that there is little risk associated with using glucosamine.

Where we disagree, as far as the facts, is in the interpretation of the clinical trial evidence. I think it is solidly against there being any meaningful benefit for oral glucosamine for all the reasons I have indicated. Dr. Robinson appears to believe that benefit is uncertain but possible. While I don’t believe her assessment is correct, it is certainly not irrational or unreasonable. Intelligent, informed, well-intentioned people can certainly disagree over the facts.

However, I think our greater disagreement is not on the question of the interpretation of the evidence but more fundamentally what constitutes evidence-based medicine. She concludes her article this way:

Although the evidence on glucosamine remains contradictory, there does appear to be some value and little risk. Not ready to abandon a product that could very well help and likely not hurt, this evidence-based practitioner will continue to mention glucosamine as one of many options for multimodal analgesia for OA patients and potential disease modifying OA benefits as well.

It seems to me that this amounts to saying, “It’s most likely harmless, and it could help, so why not try it?” I think she most accurately described her position herself when she said, “unsure of glucosamine’s benefits, many veterinarians nevertheless err on the side of hope.” The recommendation of glucosamine as a treatment for arthritis is not a decision based on evidence, but based on hope.

So, as Dr. Robinson asks, is there anything wrong with this? In the case of glucosamine, no not really. But I do worry a lot about the impact of practicing “hope-based medicine” generally, and especially of identifying it as evidence-based medicine. Ten years ago, when the data was less clear, I was hopeful glucosamine might be a useful therapy for my patients. I regularly recommended it, I gave it to my own pets, and I even took it myself. Then, as the data grew clearer, I abandoned the therapy because that ultimately is the approach to medicine that I think is most likely to lead us to the best, most effective practices.

Lots of people hope that homeopathy will help their pets, and many believe it does. And it is certainly harmless in itself, as it is only water. So is it right to use it alone, or even to add it to conventional therapy, “just in case?” I don’t think so. Granted, the evidence against glucosamine is strong but not indisputable, and the evidence against homeopathy is overwhelming. But I’m sure Dr. Robinson, no fan of homeopathy herself, has had exactly the kind of disagreements with proponents of homeopathy that she and I are having here. It is possible to find some positive studies, of poor quality and with high risk of bias, to support the use of homeopathy. Or prayer, or energy healing, or just about any intervention that people hope will benefit themselves or their patients patient. And so it is always possible for intelligent, rational people to be driven by this hope to put the most positive possible spin on the evidence, or at the last resort to say, as I think Dr. Robinson has in her article, “It’s most likely harmless, and it could help, so why not try it?” But that isn’t evidence-based medicine, and even if in individual cases, like that of glucosamine, such an approach may not lead to great harm, overall it undermines the quality and rigor of the reasoning that we use to decide what is truly helpful to our patients and what isn’t.

I regularly tell my clients that the evidence, while incomplete and not entirely consistent, is pretty strongly against any benefit from glucosamine and that the evidence of potential harm is very low, and then I let them decide if they want to use it or not. If I interpreted the existing evidence more positively, as Dr. Robinson does, I might tell them that glucosamine is harmless and might possibly have some benefits though the data is unclear, and again let them decide whether that is sufficient to warrant using it, which is what it sounds like Dr. Robinson does.

So it doesn’t sound like in practice Dr. Robinson and I approach the particular product all that differently with our clients. But the subtle difference in emphasis does seem of some importance to me. Her article gives the distinct impression that her reluctance to abandon the product is based not primarily on the balance of the evidence being positive but simply on the fact that the evidence is not uniformly negative, and if there is any reason to hope it might work it is worth trying (in the absence of clear evidence of harm). As I understand it, truly evidence-based medicine should give the greatest weight to the highest level available evidence and should not give any special weight or importance to our hopes. I understand Dr. Robinson’s perspective, and it makes a kind of sense. But it is all too familiar from many debates I have had with far less reasonable people about far more unlikely, or even ridiculous interventions. Erring on the side of hope is understandable, but it is not evidence-based medicine, and in the long run I don’t think it’s in the best interests of our pursuit of the truth about medical therapies or of our patients.

Posted in Herbs and Supplements | 19 Comments

Herb and Supplement Industry: Loves Marketing but not so Crazy about Research

Posted on February 28, 2011 by skeptvet

One of the standard clichés alternative medicine proponents like to employ is the David and Goliath Myth: pharmaceutical companies and the mainstream medicine are profit-driven behemoths and alternative medicine is the plucky little guy, small practitioners and innovators fighting against the machine to bring inexpensive natural remedies to those in need. This is quite clearly belied by that inconvenient thing called reality.

I have written before about the large and growing for-profit and corporate element of the alternative medicine movement, the underhanded and deceptive marketing practices of herb and supplement manufacturers, and the financial ties between the supplement industry and alternative veterinary practitioners and politicians. Now, Canadian Erik Davis at Skeptic North has provided a detailed and insightful article comparing the finances of several leading herbal remedy and supplement manufacturers and the largest pharmaceutical companies.

Bankers, Buyouts & Billionaires: Why Big Herba’s Research Deficit Isn’t About The Money

In this article, Davis shows quite clearly that while herbal remedy and supplement companies are truly much smaller in financial terms than the leading pharmaceutical companies, they spend proportionally at least as much of their money of marketing and far less on research.

I am certainly not one to deny the myriad sins of the pharmaceutical industry, and I am a strong supporter of vigorous and strict regulation of this industry. Profit is seldom an optimal motive to most effectively meet the medical needs of ordinary people, or veterinary patients. However, the herb, supplement, and homeopathy industries are far less closely regulated, and the numbers show quite clearly that they engage in exactly the same kind of venal behavior Big Pharma is so often accused of. And without the pressure of regulation, these companies make far less of an effort to investigate or demonstrate the safety and effectiveness of their products.

Here’s just one of many examples Davis provides

 

French giant Boiron (EPA:BOI) is by far the largest distributor of natural health products in Canada – they’re responsible for nearly 4000 (15%) of the 26,000 products approved by Health Canada’s Natural Health Products Directorate. They’re also one of largest natural health products companies globally, with 2010 revenues of €520M ($700M CAD)…

“Since 2005, we have devoted a growing level of resources to develop research,” they proclaim in the opening pages of their latest annual report, citing 70 in-progress research projects. Yet the numbers tell a different story – €4.2M in R&D expenditures in 2009, just 0.8% of revenues.

To put that in perspective, consider that in the same year, GlaxoSmithKline spent 14% of its revenues on R&D, Pfizer spent 15%, and Merck spent a whopping 21%….

But if Boiron’s not spending like pharma on research, there’s one line item where they do go toe to toe: Marketing. The company spent €114M – a full 21% of revenues on marketing in 2009. By contrast, GSK, Pfizer and Merck reported 33%, 29%, and 30% of revenues respectively on their “Selling, General, and Administrative” (SG&A) line – which includes not just sales & marketing expenses, but also executive salaries, support staff, legal, rent, utilities, and other overhead costs. Once those are subtracted out, it’s likely that Boiron spends at least as much of its revenues on marketing as Big Pharma.

I can think of nothing to add to Davis’ conclusion, which is an eloquent argument against the faux altruistic PR of Big CAM and for real, effective regulation of the supplement and herbal remedy industry.

Big Herba is clearly big business, and on a purely financial level, it’s hard not to be impressed by what they’ve achieved.  But that success — $2.5B in revenues concentrated in the seven companies above — makes it equally difficult to give them a pass on their research deficit.  Simply put, the leading natural health products companies have the coin for research, they just choose to spend it on marketing products and buying their competitors instead.  The result: while pharma typically spends upwards of 15-20% of revenues on research, Big Herba contributes less than a tenth of that.

To the question of why, I’d like to propose simply that they don’t need to.  The products are clearly selling well already, and without the regulatory approvals pharmaceuticals require, spending money on research presents more risk than reward.  After all, if you don’t conduct research, you can’t find out that your product doesn’t work.

In other words, Big Herba is behaving exactly as Big Pharma might if it had no government oversight.  And if that doesn’t give you reason for pause before you pop that next Ginko tablet, I don’t know what will.

Posted in Herbs and Supplements | 4 Comments

Over-Diagnosed by Gilbert Welch: A must-read for doctors and potential patients

Posted on February 27, 2011 by skeptvet

Dr. Harriet Hall at Science-Based Medicine recently reviewed the book Over-diagnosed: Making People Sick in the Pursuit of Health by Dr. Gilbert Welch. I was immediately interested enough to buy the book and read it, largely because it promised to address a subject I’ve discussed here before, the dangers of unnecessary or inappropriate screening tests and the irrational and counterproductive reaction that often occurs when scientists try to dissuade people from employing or taking such tests.

The book was well worth reading. Dr. Hall’s review covers the book’s contents in detail, so I won’t repeat all of that. In brief, Dr. Welch makes a strong, evidence-based case for several fundamental concepts:

1. The more sensitive our criteria for using a lab test or imaging method to diagnose a disease in someone without actual symptoms, the more likely we are to identify abnormalities that will never cause actual clinical problems.

2. This then leads to people being identified as ill who aren’t, and these people are then subjected to testing and treatment which will not help them, and may actively harm them.

3. There are a number of factors which drive and perpetuate this over-diagnosis:

a. Fear

b. Poor understanding of the meaning, use, and misuse of statistics

c. Uncritical, superficial, and excessively dramatic media coverage

d. Money (industry profits, political contributions, research grant allocation practices, fundraising of non-profit lobbying organizations etc)

e. Lawyers and litigation, which punishes missing a diagnosis but not over-diagnosis and unnecessary or harmful testing and treatment

f. Uncertainty about the risks and benefits of specific tests and interventions

e. All the usual cognitive biases we have, especially confirmation bias and the post hoc      ergo propter hoc fallacy

4. Only objective, systematic data from randomized clinical trials and good quality epidemiological studies can truly tell us whether screening tests for a specific condition help or hurt more people. As usual, anecdotes about individual patients are emotionally compelling and totally unreliable in making individual or public health decisions about disease screening.

The examples Dr. Welch provides are based on enviably comprehensive data and analysis. A surprising number of people without disease will have detectable abnormalities on blood tests, x-rays, and other screening diagnostics. And for almost every condition studied in detail, including many cancers, the majority of these people will never become ill from these abnormalities. The more sensitive our screening tests become for cancer, for example, the clearer it becomes that we all get cancer repeatedly and never know it. Just like we all take in potentially disease-causing organisms all the time. Most of these cancers never develop just like most of the bacteria and viruses we are exposed to never make us sick. So finding these cancers early and treating them does us no good at all. And apart from the anxiety of knowing we have cancer, procedures such as biopsy and surgery and cancer treatment can do very real harm.

Of course, the difficulty is that we do not, as yet, have a trustworthy way to know when we find a small cancer whether it will ever develop into a real problem  if we would benefit from treating it. This makes it difficult to resist looking and treating such abnormalities, even when the evidence is overwhelming that most people don’t benefit and that more are harmed than helped. Psychologically, we are far more afraid of the error of failing to respond to a true threat than we are of over-reacting to a harmless abnormality. Yet if we look at the issue rationally, we are far better off not employing screening tests that are clearly shown by objective data to lead to more over-diagnosis and harm than benefit.

The issue arises in veterinary medicine just as it does in human healthcare, though as always the quantity and quality of the evidence is far less and the degree of unavoidable uncertainty far greater than in human medicine. And, the limitations on resources and lesser significance of litigation alter the equation somewhat. But the general principles are still broadly applicable to the veterinary setting.

Just the other day I had a client with an older cat in for an annual exam and a followup on a known heart condition. The client asked me to perform whole-body x-rays of the cat as part of my workup. When I asked why, he showed me a book on “natural health” for dogs and cats by a prominent “holistic” veterinarian that recommends annual screening x-rays of older pets.

Apart from the obvious question what is “natural” about x-raying your cat every year, the recommendation is irrational and likely to lead to more harm than good. According to Dr. Welch, 86% of people with no signs of illness will have a lesion on a whole body CT scan. Chest x-rays will often find lung masses, and CT and MRI will find even more. And the vast majority of people without symptoms who have such lesions on random imaging screening will never experience any illness from them. But a clearly larger number of people will be harmed by additional testing and treatment, experience painful and unnecessary anxiety, and use up limited healthcare resources pursuing such findings. And though the radiation risk from a single x-ray is minimal, annual radiographs can expose the patient to small but unnecessary risk from radiation.

Similar data is not available in companion animal species, but there is no reason to believe the fundamental rules of biology are different for our pets than for us. The little research that has been done, such as studies of pre-anesthetic screening blood tests in dogs, does not generally support the value of such tests, but the data are still terribly limited. Nevertheless, the general principles and issues Dr. Welch illustrates so well certainly need to be considered in making decisions about screening tests in veterinary species.

I explained all of this to my client, who opted not to have the x-rays done. A surprising amount of my communication with clients involves convincing them not to do things, in contrast to the image CAM proponents often paint of conventional doctors aggressively selling unneeded drugs and treatments. And it is surprisingly difficult to reassure people with facts and statistics and convince them not to pursue unneeded diagnostics and treatment. And, unfortunately, reassuring them by performing meaningless rituals or providing placebo therapies works much better, which simply perpetuates ineffective therapies.

I highly recommend this book, both to potential patients (which is all of us) and to doctors. The conventional wisdom, that it is always better to look for trouble “just in case” is being shown more and more clearly to be wrong. Diagnostic testing, when there are symptoms or a sound rational reason to suspect disease, are absolutely vital. But screening tests for the apparently well are often far less likely to benefit us than is generally supposed, and they do real harm. The burden of proving such tests are a good thing has clearly shifted to those advocating them.

It seems unlikely that there will be a significant shift in the attitude of the public or doctors towards screening tests in the near future. The U.S. Preventative Services Task Force (USPSTF) is a government agency devoted to examining the scientific evidence concerning preventative healthcare testing and intervention, and they often provide sound, rational guidelines. These guidelines, are also often ignored by politicians, the public, and sometimes even healthcare providers. Our fears are still generally more persuasive than our reason. Still, reason and facts can glacially wear away at our fears and misconceptions over time, which is why I still write this blog and why I am happy to find and recommend excellent books such as Dr. Welch’s.

Posted in Book Reviews | 6 Comments

Veterinary Stem Cell Therapies Discussed at Fully Vetted Blog

Posted on February 23, 2011 by skeptvet

A prominent veterinary blogger, Dr. Patty Khuly of Fully Vetted, has written about the issue of veterinary stem cell therapies. The inspiration for her post appears to have been the recent article in JAVMA News, the news service of the Journal of the American Veterinary Medical Association, which discussed stem cell therapies in veterinary medicine. Dr. Khuly’s post was a pretty fair summary of some issues associated with stem cell therapies, which I have discussed numerous times in the past (for example). She seemed to imply pretty clearly in her conclusion that, though the evidence is not there to prove safety or efficacy, owners who are desperate and whose pets don’t have any other treatment options should have this therapy available to them. I don’t entirely disagree, but I think there may be some unjustified assumptions and a bit of a false dichotomy in her  conclusion:

Given the reality that the pets most affected by this are those with conditions that are untreatable by any conventional means, and that time-consuming clinical trials would require onerous delays in treatment, is it wrong to keep this therapy out of owners’ hands? Or is it worse to fail to live up to the oath that exhorts us to act as scientists first?

This question is a bit like “Have you stopped beating your wife?” in that there it contains assumption that may not be true and makes any answer imply something other than what is really meant. I certainly don’t disagree that for pets with severe arthritis and no other treatment options that stem cell therapy is worth trying, but I doubt that constitutes the bulk of the market for these treatments. And I think the objections I and others have raised to the aggressive marketing of stem cell treatments have a bit more to them than an intellectual desire to “act as scientists first.”

Here is the reply I posted to Dr. Khuly’s blog. Anyone interested should take a look at her post and consider participating in the discussion. Dr. Khuly likely reaches a much larger audience than I do, so it would be useful to have a variety of perspectives in the discussion there.

Dr. Khuly,

I’m happy to offer my take on the stem cell question, especially since this seems to be a subject on which I am all too easily misinterpreted. I certainly wouldn’t call myself a “naysayer,” for example. As I’ve said repeatedly, I see great promise in stem cell therapies, and I am cautiously optimistic that they will eventually be a useful tool.

My concerns have more to do with the widespread and aggressive marketing of a therapy in advance of adequate evidence to demonstrate safety and effectiveness. We are all familiar with drugs that had great promise, and that passed far more extensive pre-market testing than stem cells have undergone, and which later turned out to have less benefit than hoped or unexpected hazards (e.g. Vioxx, thalidomide). There have also been many procedural treatments like stem cell therapy which have come on the scene, enjoyed great popularity, and then turned out to be less useful or more dangerous than hoped (e.g. internal mammary artery ligation). The lesson is that the way things seem isn’t always the way things are, and good quality research isn’t just a nice extra, it is critical to making the best choices for our patients.

Given the reality that the pets most affected by this are those with conditions that are untreatable by any conventional means, and that time-consuming clinical trials would require nerous delays in treatment, is it wrong to keep this therapy out of owners’ hands?

I think you may be unintentionally setting up a bit of a false dichotomy here. You seem to be assuming that all the pet owners clamoring for this new treatment are those whose pets have serious arthritis symptoms that do not respond to currently available treatments. (The vast majority of stem cell treatments in small animals, which is my field, are marketed and used for arthritis, so that is what I will address here. As you point out, other applications are primarily still only available in the context of academic research). I am not aware of any objective statistics, but I suspect this isn’t true and that the market for stem cell therapies would be very small if it were limited to such cases. I frequently field inquiries about this therapy from people who have not taken full advantage of existing available treatments for arthritis, the most popular application for autologous stem cell treatments.

There are many therapies for arthritis which are well-established to be effective. Weight loss, physical therapy, non-steroidal anti-inflammatory medications (NSAIDs) are the most common examples. The majority of patients with arthritis respond well to these therapies. However, there is no free lunch in medicine, so anything with a benefit is going to have a downside.

Meaningful weight loss requires restricting a pet’s calorie intake significantly. The pet is going to be noticeably hungry and likely to beg or scrounge for food, and this is understandably hard for many owners. I spend a lot of time counseling owners on nutrition trying to prevent and treat obesity, but the reality is that many owners are not able to stick to a serious weight loss regime for their pets, particularly for older and relatively inactive pets.

Physical therapy is a great intervention, but it has its costs as well. Some are financial, and there is a significant investment in time and energy as well as money for owners seeking this kind of care. We are fortunate to have dedicated physical therapy facilities nearby, but not all veterinarians can easily offer or refer for this treatment, and not all owners readily take advantage of it.

And then, of course, there are NSAIDs. I would never consider denying that they have potentially serious side effects, but the evidence is very clear that, when properly used and monitored, they are very effective and very safe for the overwhelming majority of patients with severe arthritis (e.g. http://skeptvet.com/Blog/2010/12/safety-and-efficacy-of-nsaids-for-canine-arthritis/). Unfortunately, appropriate concern about potential side effects easily becomes irrational fear, and some pet owners avoid NSAIDs unnecessarily.

There is definitely a balance to be achieved between the need to act and the uncertainty about whether our actions will help or harm. For those patients who cannot maintain an acceptable quality of life using existing treatments with well-established benefits and risks, I have no objection at all to offering stem cell treatment as an experimental approach. Ideally, this would include some kind of objective, structured assessment of outcomes, both beneficial and unintended. Casual assessment by owners and individual veterinarians, or uncontrolled research by companies selling the therapy, will always show a benefit and miss all but the most dramatic side-effects for any therapy, so this can’t be our only form of assessment.

I think it is instructive to look at how differently stem cell therapies are being handled in human medicine. The major organization representing stem cell researchers, the International Society for Stem Cell Research, has said:

We have all heard about the extraordinary promise that stem cell research holds for the treatment of a wide range of diseases and conditions. However, there is a lot of work still needed to take this research and turn it into safe and effective treatments.

The International Society for Stem Cell Research (ISSCR) is very concerned that stem cell therapies are being sold around the world before they have been proven safe and effective.

Stem cell therapies are nearly all new and experimental. In these early stages, they may not work, and there may be downsides. Make sure you understand what to look out for before considering a stem cell therapy.

Remember, most medical discoveries are based on years of research performed at universities and companies. There is a long process that shows first in laboratory studies and then in clinical research that something is safe and will work. Like a new drug, stem cell therapies must be assessed and meet certain standards before receiving approval from national regulatory bodies to be used to treat people.

(http://skeptvet.com/Blog/2010/06/stem-cell-therapy-still-an-uncontrolled-experiment-on-our-pets/).

These people have devoted their careers to studying stem cell therapies, so they can’t simply dismissed as “naysayers” as so often happens when skeptics such as myself raise concerns. Stem cell therapies are moving forward much faster in veterinary medicine than in human medicine not because we have better data or a more serious problem to be addressed, but because we have more lax regulation and fewer concerns about lawsuits if something goes wrong. Hopefully, in the end this will turn out to be a good thing because these therapies will be proven safe and effective. However, we cannot totally ignore the possibility that proceeding without rigorous pre-market testing and post-market surveillance will lead us to doing harm unintentionally, or wasting time and resources on an ultimately ineffective therapy.

Just to be completely clear:

1. Stem cell therapy is indeed a promising, though as yet unproven, new therapy. However, only clinical trials can truly show it to be safe and effective, or not, so our current headlong rush into this field based on preclinical findings and individual patient anecdotes is not without risks.

2. Given the uncertainties about the safety and efficacy of this expensive and invasive procedure, all other treatments with established risks and benefits should be utilized fully before we turn to stem cell therapy. This includes NSAIDs which, though not without risks, are very effective and generally much safer than most of my clients seem to think.

3. For those patients who cannot achieve a good quality of life with established treatments, it is perfectly appropriate to offer stem cell therapy as an option. In human medicine, such treatments are generally available through clinical trials or special “compassionate use” provisions. This may not always be practical in veterinary medicine, but we should make our best effort to objectively and systematically assess the outcomes of patients treated with stem cells.

Think about how long it took for us to make the connection between certain vaccines and cancers that developed rarely and many years later. If a similar unanticipated risk exists with stem cell treatment, we’re going to miss it if we are looking carefully, and that would be irresponsible. I would like to see less aggressive, more fact-based marketing and more effort given to evaluating objectively the benefits and unintended effects or complications of these procedures.

You ask if we have the right to keep a potentially beneficial therapy with uncertain risks and benefits from owners who want it. I would phrase the question differently. Do we have a right to sell owners a potentially beneficial therapy with uncertain risks and benefits, often through misleading and unjustifiably positive marketing? Don’t we have a responsibility to ensure as best we realistically can the safety and effectiveness of any therapy we offer our patients?

Posted in Science-Based Veterinary Medicine | 10 Comments

Spiraling Empiricism: Antibiotic Use as a Model for Pitfalls in Medical Decision Making

Posted on February 14, 2011 by skeptvet

About two years ago, Mark Crislip over at Science-Based Medicine wrote about an article that had a profound impact on his practices as an M.D. specializing in infectious disease, Observations on Spiraling Empiricism. He recently mentioned this article again on his own infectious disease blog, which drew my attention to his earlier post and the original article. I strongly encourage anyone who has an interest and access to the medical literature to read the original article. Though it focuses particularly on empirical antibiotic therapy (that is, using antibiotics when an infection of some kind is suspected but no specific diagnosis is made), it provides an excellent general conceptual approach to therapy and an eloquent and cogent description of the dangers of initiating therapy in the absence of a definitive diagnosis or even a single prime suspect.

This problem is even more acute in veterinary medicine, where we have fewer resources and less detailed knowledge about our patients’ diseases. Even the case examples in the article which illustrates mistakes in medical decision making often have far more meticulous and thorough analysis done than we veterinarians can do at our best. The risks of acting on incomplete information are likely even greater for us since we are more often forced to do so and our information is often exponentially less comprehensive than that our MD colleagues have to work with.

The article begins by acknowledging that antibiotics have been a phenomenally successful medical therapy that have done by far more good than harm, and that advances in the drugs available since the first penicillins and sulfa drugs of the 30s have made them ever safer and more effective. It is important, especially when dealing with the pharmacophobia of much of the alternative medicine community, not to lose sight of the historical context when examining some of the problems with these drugs and how they are used.

The authors then state clearly, and with an unusual literary flair, the nature and complexity of the problem with inappropriate antibiotic use.

The imprecision of clinical practice establishes the context; the litigious nature of society unnerves; the absence of toxicity permits; and the sum of these encourages the incontinent, extemporaneous use of antimicrobial agents…

The term spiraling empiricism describes the inappropriate treatment, or the unjustifiable escalation of treatment, of suspected but undocumented infectious disease. Empiricism and empirical therapy, defined as the carefully considered, presumptive treatment of disease prior to establishment of a diagnosis, often are necessary in the proper practice of medicine. On the other hand, ill-considered or inappropriate use of antibiotics, incurring unnecessary risk and expense, should be indicted and condemned. The difficulty lies in distinguishing reasonable or appropriate from unreasonable or inappropriate therapy.

I would suggest that in veterinary medicine the role of “imprecision” is relatively greater and the role of litigation relatively smaller in driving inappropriate use of antibiotics, or indeed any therapy, but the general problem is broadly similar to that in human healthcare.

The article then presents a simplified conceptual framework for the options available to a clinician faced with the possibility of infection in a given patient. The examples (which I have omitted) are specific to human medicine, but the general approach is easily applicable to veterinary medicine, and provides a lot of insights into where medical decision making can go wrong.

Table I. Features and Objectives of Various Therapeutic Options

The first option, Observation, is often the hardest for both vets and clients. So many diseases get better without intervention (which is why so many ineffective therapies appear to work), but without being able to predict which will and which won’t, it is very hard to simply wait and see. There are, of course, clear reasons not to simply observe in many cases: the severity of the symptoms, the likelihood of specific diagnoses which are less treatable when treatment is delayed, and many other factors. But just as we don’t (or shouldn’t) go to our doctor and demand antibiotics for every little viral respiratory infection, since the drugs have risks and costs and won’t help anyway and since we will almost certainly get better on our own, so we as doctors should resist the temptation to treat illnesses that are mild and likely to get better on their own (feline upper respiratory viruses, mild acute diarrhea, low-grade fevers in otherwise normal patients, etc).

And not only should we resist the temptation to treat these patients with antibiotics or anti-inflammatories without a sound rationale for doing so, we should resist the temptation to pretend to treat them with “safe and natural” remedies that are likely to do nothing other than create the false impression we’ve fixed something when the patients spontaneously get better (homeopathy, energy medicine, many vitamins and supplements, etc).

Preventative use of antibiotics is a bit clearer an option, though it is still important to base our prophylactic use of these medications on sound evidence that they are needed. The prophylactic use of antibiotics in dogs with heart murmurs undergoing dentistry, for example is a commonly accepted practice, yet there is some evidence that it may be unnecessary. There are also other studies which cast doubt on the effectiveness of prophylactic antibiotic use before some kinds of surgery, so we must be careful not to indiscriminately use such drugs “just in case” or to compensate for poor surgical technique unless there is sound evidence to support the practice. Nothing with a benefit is without risks and costs, and we must balance this equation as carefully as possible given the available information.

Empiric Therapy, as defined above, is a common practice. Given some indication of an infection (fever, increased white blood cell count, etc), but before the necessary steps have been taken to establish a true diagnosis (or, unfortunately, in some cases instead of taking these steps), antibiotics are given to treat a range of possible causes of the clinical symptoms. This can be a mistake in a simple and direct way if the patient doesn’t have an infection but develops an adverse reaction to the antibiotics. However, a less obvious pitfall with this kind of approach is the possibility of delaying or missing the true diagnosis, especially when the symptoms improve after the drugs are given and we assume, sometimes mistakenly, that the drugs are responsible and that our suspicions of infection were correct. This post hoc ergo propter hoc bedevils us in medicine all the time, and this is one of the most common circumstances in which it does so. Just because we do something and the patient’s condition changes, it is not automatically true that we deserve the credit (or blame) for the change.

Empirical Therapy is distinct from a Therapeutic Trial, in which a specific infection is suspected and a specific antibiotic treatment is begun with the patient’s response used to support or contradict the presumed diagnosis. The difference is that the odds of a this-then-that relationship in time reflecting a true causal relationship is greater if there is a reasonable prior probability, based on factors other than the treatment, of there being such a relationship.

In a simpler example, if a dog has a cough and a chest x-ray shows a mass in the lung, the chance that the mass is truly relevant is fairly high. But if a dog with diarrhea has a chest x-ray that shows a mass, there is little reason to think the x-ray finding is relevant to the symptoms. A relationship between a potential cause and an effect is more likely to exist if there are multiple different lines of evidence supporting such a relationship. So if all the clinical symptoms are classic for a particular bacterial infection, and the symptoms go away on an antibiotic sited to that organism, then a cause/effect relationship between treatment and response is fairly likely. However, if the symptoms (such as a fever) could be due to any of a hundred diseases, and the symptoms go away after treatment for one of them, we cannot confidently use this fact to conclude the patient had that one disease.

Of course, even a therapeutic trial can lead to an erroneous conclusion. I once treated a young dog with a fever, neck pain, and a high white blood cell count with antibiotics, under the reasonable suspicion that it had an infectious meningitis (the owner declined the invasive and expensive tests to confirm my suspicion). The patient got better and I basked in the glow of my own self satisfaction. Until a few months later when it came back with the same signs and didn’t get better on antibiotics. Eventually, it became clear that the pet had symptoms every time it came into heat, and  that it was a non-infectious immune-mediated meningitis which responded to steroids. Once the dog was spayed it never recurred.

Finally, Specific Therapy is treatment of a particular disease we have actually diagnosed. Obviously, this is far more likely to be effective and to have a positive benefit to risk ratio than relatively blind empirical therapy.

The authors then go on to list a number of misconceptions doctors have about empirical antibiotic therapy, which again could apply just as well to many other kinds of medical intervention.

Table II. Fallacies in Antibiotic Therapy

In his discussion of the article, Dr. Crislip expands on each of these fallacies, and adds a couple. Some of his comments are worth reproducing here.

When in Doubt, Change Drugs or Add Another

Medicine is filled with uncertainty, and often it is the case that if an infection is thought of, it is treated, no matter how unlikely it is that it may be causing the disease. I have a fantasy where oncology is practiced like infectious disease. “It might be lymphoma, so lets start with CHOP, but we don’t want to miss adenocarcinoma, so lets add bleomycin, and in case its breast cancer we need to include tamoxifen and if could be prostrate so lets add etc , etc.”

When in doubt, increase your diagnostic certainty.

Response implies diagnosis

This is the most difficult fallacy for people to abandon. Patient has a fever, no diagnosis is evident, but the fever went away at some point after the institution of antibiotics.

Most fevers go away. Many diseases that are not infectious will have a fever. This is the medical equivalent of the skeptical motto ‘association is not causation’. The worst cognitive error physicians and patients make is the assumption, the Rigorously absence of a good diagnosis, that the improvement when a therapy is given is due to the therapy.

Rigorously and consciously avoiding this error is key to being a good health care provider.[emphasis added]

Some antibiotics are a big gun, are strong, or powerful

There are few things in medicine with 100% sensitivity and specificity. However, if your health care provider uses the adjectives big gun, strong, or powerful in reference to an antibiotic, they are either 1) talking out their backside or 2) ignorant about antibiotic use. 100% sensitive and specific

If I had neurosyphilis…, the ‘strong’ or ‘powerful’ ciprofloxicin would do nothing to treat my infection, put ancient, weak old penicillin remains the treatment of choice. What you want is to give are appropriate antibiotics: something that will reliably kill the organisms in whatever space is infected. These adjectives are advertising ploys used on fool gullible rubes, er, I mean health care providers, to think they are doing what is best for their patient. They provide a false security that you are giving the patient the best therapy.

  

The primary reason a particular antibiotic is given is that “I like it.

I like to say that the three most dangerous words in medicine, especially when it comes to therapeutic interventions, are “In my experience.” In my experience you can’t trust anyone who uses that phrase. Remembering hits and forgetting misses drives antibiotic use more than I would like to admit. Infectious disease docs are sometimes the opposite. We put too much emphasis on the antibiotics that have failed us in the past. It is o e extreme or the other.

Patient is admitted with a complicated infection and started on X. Why X? I ask. It worked for me in the past is the reply. What are you trying to kill? I might then ask. Often, they say the boogie man of the ICU, Pseudomonas. What I will continue, is the chance drug X will be effective against Pseudomonas? They don’t know. So why again are you using X? It is what we do. They have used it successfully in the past, so it should work again.

Sigh.

Fortunately, most drugs work most of the time in most patients, but that rule is slowly being lost as the organisms become increasingly resistant to our current armamentarium of antibiotics and there are few, if any, replacements in the pipeline. At my institution there is 2% resistance for Pseudomonas to Ceftazadime and 4% to pipercillin/tazobactam. We get a few cases a year at best of a blood stream infection and sepsis from Pseudomonas. Most physicians are not going to see enough infections by a given organism to get a sense of what does and does not work. It is why you cannot trust your experience.

And finally, what could easily be a motto for the whole project of evidence-based medicine:

But often getting the right diagnosis and therapy is less about what you know and more about being rigorous about understanding how you know. Only when you are conscious of your ability to think poorly, can you compensate.

References

1. Kim JH, Gallis HA. Observations on spiraling empiricism: its causes, allure, and perils, with particular reference to antibiotic therapy. Am J Med. 1989 Aug;87(2):201-6.

Posted in General, Science-Based Veterinary Medicine | 2 Comments

Integrative Medicine or Bait-and Switch?

Posted on February 13, 2011 by skeptvet

For a while now, the Huffington Post has been providing a platform for a prominent voice in the alternative veterinary medicine community, Dr. Richard Palmquist. Dr. Palmquist is involved in the American Holistic Veterinary Medical Association (AHVMA), and has had some rather unkind things to say about CAM skeptics in general, and about my criticisms of the AHVMA in particular. It is unfortunate that someone whose ideas are so far outside the mainstream has been given such a prominent pulpit for disseminating misinformation, and that pet owners are unlikely to realize that Dr. Palmquist’s theories do not accurately represent the science of veterinary medicine or the opinions of most veterinarians.

He must be given credit, however, for representing his position skillfully. One element to putting a positive PR spin on unproven or disproven remedies is the technique known as bait-and-switch (though it is more commonly referred to as “integrative medicine”). This involves recommending or using widely accepted conventional therapies based on scientific reasoning and evidence, and then tagging on unproven therapies or pure nonsense treatments as if they were in the same category. In his most recent post, Dr. Palmquist discusses allergies, which in dogs and cats usually manifest as itching and infections of the skin in response to allergens from fleas, food, or environmental sources (such as dust mites, pollens, etc). There is a good deal of sound scientific research on allergies in pets, and the veterinary dermatology community regularly provides summaries of the research in statements on Evidence-Based Allergy Treatment.

Much of Dr. Palmquist’s post is perfectly consistent with a scientific understanding of the cause and treatment of allergies. He discusses antihistamines, corticosteroids, immunotherapy (aka allergy shots), limited antigen diets, and fish oil supplements, all of which are therapies supported by reasonable plausibility and research evidence. In this discussion, of course, he continuously minimizes the benefits and expounds at length on the potential risks, a clear effort to bias the reader against these therapies while ostensibly acknowledging that they have proven efficacy.

In particular, he harps on the notion that antihistamines and steroids “turn off” parts of the immune system. This sounds like a terrible thing, because everyone knows the immune system is necessary to protect us from infectious disease. So are we compromising the defenses of our patients with these drugs? If properly and judiciously used, no. The implication ignores the complexity of immune function and the effects of anti-inflammatory medications, probably deliberately in order to increase the negative associations people may have in their minds with these medicines.

In response to stimuli, such as infectious agents or allergens, cells in the immune system release chemicals, called cytokines, which lead to the classic inflammatory response: redness, swelling, itching or pain, and sometimes systemic effects such as fever or lethargy. Inflammation is not inherently a good thing. In fact, it can be quite harmful. While a mild fever may help fight off an infection, a high fever that causes brain damage is clearly doing more harm than good. And while activation of the immune system in an effort to fight off disease-causing organisms is a good thing, allergies are by definition an inappropriate activation of the system against proteins that are not inherently harmful. They are uncomfortable chronic diseases caused by excessive, unnecessary activity of the immune system. Even worse are auto-immune diseases, like lupus, hemolytic anemia, and so on, are serious, even deadly consequences of inappropriate immune-system activity. So the implication that turning off part of the immune system is a dangerous or unnecessary part of treating diseases that manifest as excessive immune system activity is nonsense. Suppressing excessive inflammation is exactly what is needed to treat the discomfort or even serious harm these diseases can cause.

It is possible, of course, to increase a patient’s susceptibility to disease with excessive or prolonged use of anti-inflammatories, and undoubtedly some doctors do not use these drugs judiciously. But anything in medicine that has a benefit has a potential risk, and it is disingenuous of Dr. Palmquist to focus primarily on the risks of conventional therapies and the unproven but assumed benefits of alternative methods regardless of the state of the evidence.

Of course, the agenda behind this damning with faint praise of conventional treatments is to build a case for the alternative treatments with which he concludes his article. He first includes limited antigen diets in his list of alternative treatments, though it is a widely accepted conventional therapy, probably because of the mistaken notion that any treatment that is not a drug must be a form of alternative medicine. He also includes fish oils, which are another treatment widely used in conventional medicine, though with only limited supportive evidence.

His descriptions of herbs and supplements, homeopathy and homotoxicology, and other such alternative therapies fairly glow with praise: “can be amazing,” “allergic responses can vanish entirely,” etc. And with the sole exception of high-dose Vitamin C supplementation, no potential risks or unwanted effects are mentioned for any of these treatments.

Dr. Palmquist does state that, “The scientific evidence varies for these methods,” and he supports this statement by reference to the summary of evidence-based allergy treatment I referred to above. However, with the exception of diets and fish oils (neither truly alternative) and phytopica (which Dr. Palmquist doesn’t mention), this review does not provide any evidence in favor of alternative therapies, and in fact it mentions almost none of those he recommends. There is no legitimate scientific evidence to support the utility of acupuncture, Traditional Chinese Medicine theory, homeopathy, homotoxicology, glandular extracts, or digestive enzyme supplementation in the treatment of pet allergies. And most of the herbs and supplements he recommends have little to no evidence to support veterinary allergy use. And yet he discusses these therapies as if they were as effective, and certainly safer, than the proven conventional therapies he so lukewarmly discussed at the beginning of the article.

No question this is a slick bit of propaganda designed to cast doubt on the safety of conventional therapies and to guide readers towards unproven or outright useless alternatives that are presented as if they were legitimate, accepted therapies of proven safety and efficacy. There is an impressive list of supporting references, though many of them are either books written by other proponents of alternative therapies, or articles which, like the dermatology task force review, don’t actually support the safety and efficacy of the alternative therapies being promoted. Some are legitimate studies investigating the safety or efficacy of conventional treatments, but the purpose is clearly not to review the available treatments fairly but to build a public relations case for alternative and against conventional science-based allergy treatments. Though I’m sure Dr. Palmquist feels this is for the greater good, since he is a true believer in the therapies he promotes, in reality such misinformation presented as if it were established fact and biased towards the unproven is not truly in the best interests of our patients or clients.

Posted in General | 10 Comments

From Science-Based Medicine: Complementary and Alternative Medicine in Hospice Care

Posted on February 3, 2011 by skeptvet

From the Science-Based Medicine Blog: Complementary and Alternative Medicine in Hospice Care

A number of news outlets (e.g. Bloomberg Business Week, MSN.Com, US News, etc) have recently reported that use of complementary and alternative therapies (CAT) is widespread in hospice care facilities. This is based on a report from the Centers for Disease Control, Complementary and Alternative Therapies in Hospice: The National Home and Hospice Care Survey, Untied States, 2007. According to most news reports, about 42% of hospice care providers offer some kind of CAT.

I was initially inclined to find this a little worrisome. In my own field of veterinary medicine, advocates of alternative therapies are prominent among the organizers of the nascent hospice care movement. And while I am strongly supportive of better and more available veterinary hospice care, the involvement of CAM advocates raises the concern that animals at the end of their life might receive ineffective palliative care, or be denied the benefits of conventional treatments by some CAM providers, who often characterize “allopathic” treatments as “unnatural” and harmful.

In practice, I have seen this happen to patients with terminal diseases. I will never forget a Rottweiler dog I diagnosed with osteosarcoma, a very painful bone cancer, whose owner was convinced that homeopathy was adequate to control his pain and refused to use NSAIDs because of her conviction they were “toxic.” I have also seen my patients denied euthanasia even in the face of great suffering because so-called “animal communicators” claimed the pet was “not ready to leave” and had expressed a desire to remain with their owner as long as possible.

Perhaps these experiences have made me overly sensitive on this subject, but I saw these recent news reports and pictured people at the end of their lives being similarly denied effective palliative care or subjected to pointless therapies like homeopathy and “energy medicine,” or even more worrisome treatments like chiropractic or herbal remedies with real risks. However, a little digging into the details suggests that the headlines are a bit misleading, and these fears are probably unfounded.

As always, when trying to assess how popular alternative medical therapies are, the tricky issue arises of defining “alternative.” In this study, the authors referenced the MedlinePlus definition:

Complementary and alternative medicine (CAM) is the term for medical products and practices that are not part of standard care. Standard care is what medical doctors, doctors of osteopathy and allied health professionals, such as registered nurses and physical therapists, practice. Alternative medicine means treatments that you use instead of standard ones. Complementary medicine means nonstandard treatments that you use along with standard ones. Examples of CAM therapies are acupuncture, chiropractic and herbal medicines.

Personally, I prefer Dr. Novella’s definition:

CAM is a political/ideological entity, not a scientific one. It is an artificial category created for the purpose of promoting a diverse set of dubious, untested, or fraudulent health practices. It is an excellent example of the (successful) use of language as a propaganda tool.

In any case, in order to measure the popularity of something, one has to define it in some way, and in the past assessments of how popular or widespread CAM use is have created misleading impressions due to dodgy definitions. For example, the 2007 National Health Interview Survey (discussed in detail here) reported 30% of Americans to be regular CAM users. A closer look at the details of the survey, however, showed that very little of this self-reported usage involved the application of the usual dubious CAM approaches (e.g. acupuncture, chiropractic, homeopathy, various herbal traditions, etc) to treat specific medical problems. Much of this supposed CAM usage involved the non-medical application of massage, yoga, tai chi, prayer, and so on to provide psychological comfort or facilitate relaxation.

Of course, if one argues that massage, yoga, or even prayer are effective in reducing the objective signs or disease, or even bringing about a cure, then one could argue these are forms of alternative medicine. But such methods are mostly employed to provide comfort and help patients cope with their illness, and as such they can be valuable and legitimate interventions. This does not make them medical therapies, however, alternative or otherwise.

The hospice care survey suffers from the same kind of problematic definition for “complementary and alternative.” According to the report’s technical notes, providers of hospice care were asked first to choose all the services they offered from a list, and “Complementary and Alternative Medicine (CAM)” was one of the choices. Those that indicated they offered CAM were then asked to indicate “Which of these complementary and alternative medicine therapies does this agency use?”

Here is the list:

  1. Acupuncture
  2. Aromatherapy
  3. Art therapy
  4. Guided imagery or relaxation
  5. Massage
  6. Music therapy
  7. Pet therapy
  8. Supportive group therapy
  9. Therapeutic touch (a westernized version of reiki)
  10. TENS (Transcutaneous Electrical Nerve Stimulation)
  11. Other

Personally, I see little on this list that I would classify as CAM. Acupuncture, certainly, along with therapeutic touch (like reiki) and aromatherapy. But most of the rest, unless specifically marketed as treatments for disease, seem more like benign, pleasurable activities designed to provide comfort, relaxation, and enjoyable stimulation. As a veterinarian, I work with a lot of pet therapy dogs, and I have yet to run across a handler of one who thought they were practicing alternative medicine! (Though I suppose there might be some such folks out there). And TENS is a perfectly conventional intervention, often somewhat disingenuously confused with acupuncture.

The most popular of the “true” CAM therapies offered was therapeutic touch, available at 48.3% of facilities. Aromatherapy was offered by 39.7% of hospice providers. I cannot even find a number for acupuncture in the report. And by far the most popular “alternative” therapies offered were massage (71.7%), group therapy (69%), music therapy (62.2%), and pet therapy (58.6%).

The report also indicates that only 8.6% of patients discharged from a hospice facility that offered CATs actually received one of these therapies. So even under such a loose definition of alternative, there is no evidence that large numbers of hospice patients are receiving alternative medical treatments.

It wouldn’t surprise me if we begin to see advocates of alternative medicine proclaiming that this report shows CAM is widely available, popular, and even indispensible in hospice care. The 2007 National Health Interview Survey results were frequently used this way to create the impression that CAM is becoming mainstream and that resistance to it is the province of extremists and ultimately futile. The details of both surveys, however, indicate that even with aggressive expansion of the definition of CAM to include conventional therapies such as TENS and non-medical interventions like pet therapy, CAM is not truly as popular ubiquitous as its proponents claim.

There is little objectionable from a science-based medicine perspective in most of the therapies hospice care providers are offering, according to this study. I enjoy a good massage, relaxing music, and the company of a friendly dog as much as anyone. And those elements that are truly nonsense, such as therapeutic touch and aromatherapy, are unlikely to do harm or replace appropriate conventional therapies, and they seem in any case not to be especially popular with patients even when they are available. So regardless of what PR use is made of this study, it does not suggest that human hospice care is becoming predominantly the domain of CAM providers, as I might have feared. I only hope the same will be true of veterinary hospice care as that becomes, hopefully, more commonplace.

Posted in General | Leave a comment