A Review of Reviews of Acupuncture for Pain: Might Work, Might not; Could Kill You, Probably Won’t

Posted on March 27, 2011 by skeptvet

A new review of systematic reviews of acupuncture treatment for pain was recently published in the journal Pain. Basically, a systematic review is a study which looks at primary clinical research studies selected in a comprehensive and pre-determined manner and with explicit standards for evaluation of the quality of the studies included. It is a way of taking a collection of research studies on a single subject and evaluating them objectively to determine what the balance of the evidence overall says about the subject. A systematic review of systematic reviews, then, uses the same objective and explicitly stated methodology to examine multiple systematic reviews on a single subject. This approach sounds pedantic, but it serves to minimize the influence of the reviewer’s bias on the conclusions.

An informal or narrative review, in which an author picks whichever studies they want and evaluates them informally, tends to simply conclude whatever the author originally expected the balance of the evidence to say. A systematic review makes it harder to intentionally or unintentionally select some studies and ignore others in a way that biases the conclusions, or to give greater credence to some studies over others based on one’s preconceptions rather than on objective criteria for quality. Thus, systematic reviews and systematic reviews of reviews, if properly conducted, are the most reliable level of evidence.

This review identified 57 systematic reviews that met the criteria for inclusion in the analysis. The authors were primarily interested in the effectiveness of acupuncture as a treatment for pain and the side effects of acupuncture treatment. The results were mixed.  25 reviews reported a tentatively or clearly positive result, meaning acupuncture appeared to be beneficial for the conditions studied. 32 reviews did not report a positive result, meaning acupuncture appeared no better than placebo for pain. Of the 4 reviews deemed excellent in quality, 2 found a significant benefit and 2 did not. When there were multiple reviews for the same condition, they frequently contradicted each other.

The exact significance of these results is not clear. It could be that acupuncture helps with pain for some conditions but not others, though neither the traditional mystical theories nor more modern neurophysiological theories of how acupuncture is supposed to work would predict or explain that. However, as the authors of the review point out, several recent high-quality research studies hold out some hope of an answer. In these studies, fake acupuncture (applying treatment at random locations, using needles that don’t penetrate the skin, or even using toothpicks instead of needles) works as well or even better than real acupuncture. And one study found that acupuncture had a significantly greater benefits if the acupuncturists deliberately encouraged high expectations of a benefit in patients than if they were neutral about the likely effects. These findings suggest that psychological variables, such as the beliefs and expectations of patients and the behavior of those performing the acupuncture, are responsible for much of the perceived improvement in pain. These variables are often not specifically or effectively controlled in clinical trials, which would lead to variations in the apparent benefit of acupuncture unrelated to whether the treatment itself is actually doing anything to the body of the patient.

Of course, as I have argued in the past, such improvement in perceived pain is certainly real from the point of view of the patient, regardless of how it is generated. In many studies, the benefit is minimal and may or may not be clinically significant. It is almost certainly less than that achieved by properly tested pain medications which, when they are tested, rarely show the inconsistency in effects seen with acupuncture. Still , if a subjective experience like pain seems better to a patient, it cannot be said it isn’t better, only that it may not be due to any objectively measurable change in the patient’s body.

This, of course, also raises serious questions about whether the same benefits, if they exist at all, would be seen in veterinary patients receiving acupuncture. The only parties in the therapeutic interaction that have explicit beliefs and expectations are the owners and those performing the treatment, so it may be that they are the only ones to perceive a benefit. Without objective measures of response, such as force-plate measurements, activity monitors, and so on, it is dangerous to assume our pets truly feel less pain just because it looks to us like acupuncture is working on them.

The review also looked at side effects from acupuncture, something proponents of the therapy often argue are negligible or non-existent. They did find a number of serious adverse events, sometimes fatal or permanently disabling. The most common were pneumothorax (air getting into the chest through a hole made by an acupuncture needle and collapsing the lungs) and infections. These adverse events were not common, and they generally resulted from improper technique, so it is reasonable to argue that proper training and oversight of acupuncturists would prevent them. But of course if the therapy itself is nothing more than a placebo, it is difficult to argue that any life-threatening side effects are justifiable.

So the best we can say, after extensive research over decades, is that acupuncture might relieve pain for some patients with some conditions. Side effects are uncommon if it is properly performed but can be serious, even deadly, if not. The authors conclusions were these:

[N]umerous systematic reviews have generated little truly convincing evidence that acupuncture is effective in reducing pain. Serious adverse events continue to be reported.

Ernst, E. Soo Lee, Myeong. Choi, Tae-Young. Acupuncture: Does it alleviate pain and are there serious risks? A review of reviews. Pain. 2011;152:7555-764.

Posted in Acupuncture | 3 Comments

From SBM: How Popular Is Acupuncture?

Posted on March 25, 2011 by skeptvet

From Science-Based Medicine:

How popular is acupuncture?

One argument that often comes up when skeptics and proponents of so-called complementary and alternative medicine (CAM) debate is the question of the popularity of various CAM practices. Advocates of CAM often claim these practices are widely used and growing rapidly in popularity. Obviously, CAM proponents have an interest in characterizing their practices as widely accepted and utilized. Even though the popularity of an idea is not a reliable indication of whether or not it is true, most people are inclined to accept that if a lot of people believe in something there must be at least some truth to it. The evidence against this idea is overwhelming, but it is a deeply intuitive, intransigent notion that can only rarely be dislodged.

It might therefore be useful to get some idea of whether or not the claims of great popularity for CAM treatments are true. If they are not, fruitless debates about the probative value of such popularity could potentially be avoided, and it might be possible to diminish the allure associated with the belief that “everybody’s doing it.” 

Surveys of CAM Use

It is difficult to find good quality objective data on the popularity of particular CAM interventions, and many of the surveys that have been done are potentially misleading. For example,. the 2007 CDC National Health Interview Survey (NHIS) is widely cited as showing that about 30% of Americans use CAM therapies. A careful look at the details of this survey, however, shows that many of the supposed CAM therapies are really relaxation or exercise practices, such as massage and yoga, not medical therapies. Chiropractic is the only medical therapy generally classified as alternative that was used by more than 10% of people in the survey. And that was primarily for idiopathic lower back pain, an indication for which it is generally accepted, even by skeptics such as myself, as having some demonstrated benefit, about equal to standard medical interventions. These usage numbers haven’t changed in decades, which belies the notion that CAM is growing in popularity.

Similarly, much was made by the media of a recent CDC survey that supposedly showed widespread use of CAM therapies in hospice care facilities. A close analysis of this survey, also shows that most of the therapies listed are not truly alternative medical interventions and that fewer than half the facilities surveyed offered true CAM therapies, and fewer than 10% of patients in those facilities actually employed the CAM practices offered.

Studies of Acupuncture Use

I thought it might be useful to look at some of the data concerning the popularity acupuncture, since it is probably the most widely used and accepted CAM therapy after chiropractic, and there have been a few interesting studies in this area. There are certainly no comprehensive, high-quality data concerning how many people use acupuncture, for what indications, and with what sort of beliefs in its underlying theory or effectiveness. My purpose is not to make a definitive statement about how popular acupuncture is but simply to take a small step beyond vague impressions and unsupported claims about the popularity of this intervention and look at what research there is and what insight, if any, these numbers might give us. 

The 2007 NHIS data indicated 6.5% of Americans had reported ever using acupuncture. Of these, 22% had seen an acupuncturist in the last 12 months. 25% of those who had tried acupuncture had done so once, and 70% had seen an acupuncturist fewer than 5 times. The vast majority of those who had seen an acupuncturist had done so for some kind of pain, primarily arthritis and other orthopedic pain, headaches, or fibromyalgia. About 40% of the people who reported using acupuncture for a specific condition specifically reported not using conventional therapies for that condition, while 20-40% reported using some kind of conventional medical therapy for the same condition. 

It is often argued that lack of interest in the utilizing acupuncture is driven more by cultural prejudice or belief systems than by concerns about the evidence for its efficacy. There is likely some truth to the fact that people from different cultures prefer familiar styles of medical treatment, though of course this says nothing about what is actually safe or effective. And part of the appeal of acupuncture in the West is likely its exotic, “foreign” associations.

Some surveys of acupuncture use have looked at whether different ethnic groups in the West have differences in their utilization of acupuncture. Interestingly, one study from Canada found that while White and Chinese Canadians differed in their use of some CAM modalities, their overall use of CAM was the same, and their use of acupuncture specifically was about the same: roughly 8%. (Therapies included in definition of CAM in Quan et al 2008: Herbal remedies, massage therapy, chiropractic, acupuncture, amino acids, naturopathy, homeopathy, reiki, ayurvedic medicine, biofeedback, hypnosis.)

In contrast, a survey of Chinese Americans in a mental health services program found about 25% used acupuncture, and that this use was more prevalent among “less acculturated” individuals. This, of course, is a group not at all representative of the general population, so the relevance of this to overall acupuncture use among Chinese Americans and Americans of other ethnicities is not clear. Other studies have shown significant but complex relationships between ethnicity, education, and other variables and the likelihood of acupuncture use.

It seems reasonable that cultural traditions play some role in the acceptance or rejection of acupuncture as a medical therapy, but the current data do not support that cultural affiliation alone is the most important variable, and the reasons people use acupuncture seem quite consistent regardless of ethnicity or nationality. In any case, studies of populations in North America do not show anything approaching a majority of the population regularly using acupuncture as a medical therapy. Numbers vary from less than 10% to as high as 50% in some populations, but most tend to be in the lower end of that range.

Since acupuncture as it is currently understood and practiced in Europe and North America originated in China and has been employed there and in other Asian countries for a lot longer than it has been used in the West (though not nearly as long as is usually claimed), it makes sense that it would be far more widely used in that part of the world if it is truly as popular a therapy as its proponents claim.

One 2007 study in Taiwan found about 11% of beneficiaries of national health insurance had used acupuncture in a given year. Interestingly, while the survey found that overall use of Traditional Chinese Medicine (TCM) therapies was much higher than this (primarily due to use of herbal remedies), the use of TCM was still far behind the use of so-called “Western” medicine. TCM clinic visits accounted for only 9% of outpatient visits reimbursed under the national health insurance. This is similar to another study which found Chinese medicine (of all covered types) accounted for only 5% of the reimbursed care under the national health insurance system. The same study indicated that “Western” medicine was utilized more than Chinese medicine, especially among children, the elderly, and those with severe disease (consistent with the pattern of CAM use in the U.S., which is generally for self-limiting or chronic disease).

Yet another study in Taiwan specifically investigated acupuncture use and found about 6.2% of people covered by national health insurance utilized acupuncture in a given year, and over the seven years surveyed about 25% of covered individuals had received acupuncture treatment. As in the U.S., the vast majority of the acupuncture treatment sought was for musculoskeletal conditions or injuries (88%).

And a recently published series of surveys conducted in Japan found that about 5-7% of respondents used acupuncture in a given year, and that over a lifetime between 20-27% of respondents had at some time tried acupuncture. More than 80% of the use of acupuncture was for musculoskeletal complaints. About half of those who had used acupuncture indicated they would use it again, and about 37% indicated they would not.

The Bottom Line

So what does all of this mean? Well, probably not very much. Of course, differences in healthcare systems, insurance systems, study methods, and many other factors that are difficult to identify and asses, make direct comparisons between the use of specific CAM interventions in different countries very unreliable. I don’t believe the quality of the data generally allow very confident statements about the popularity of acupuncture or other specific CAM methods. However, proponents of acupuncture, and CAM generally often make such statements, trying to convey the impression that their approaches are growing rapidly in popularity and only perverse, closed-minded curmudgeons still resist them. The little evidence we have certainly does not support such claims. 

In the case of acupuncture, for example, the data show relatively low levels of utilization even in those countries generally regarded as having long historical traditions of using acupuncture. Informal investigations (e.g. 1, 2) have suggested that acupuncture and other CAM practices associated with China may not be as popular even in their native land as proponents in North America claim, and the formal studies I have discussed here seem to support that impression.

A large majority of people who seek acupuncture therapy, regardless of ethnicity or nationality, do so for treatment of musculoskeletal conditions and pain. There is good evidence that the therapeutic ritual of acupuncture has some symptomatic benefit for such indications. This is almost certainly a non-specific treatment effect (aka “placebo”). It does not seem to matter where needles are inserted or if they are inserted at all, and acupuncture therapy does not appear to measurably affect the course of any actual disease. (The Skeptic’s Dictionary has a clear and concise review).

The research data on acupuncture utilization suggests that from about 5-25% of people, regardless of nationality or ethnicity, will at some time try acupuncture for, mostly for some kind of musculoskeletal pain. Conventional therapies are often used along with acupuncture, and they are far more popular overall, especially for serious or acute conditions. So the little research there is suggests that acupuncture occupies a niche common to many CAM therapies. It is used at a low level by a small to moderate proportion of the population for conditions that are either mild, self-limiting, or without a definitive conventional treatment, and it is rarely used in lieu of conventional medical care. This is hardly a mounting wave of enthusiasm for acupuncture itself, much less the mystical theories and postmodern cognitive relativism often associated with it. 

So when proponents of acupuncture say it must work because it has been widely used for thousands of years in Asia and is growing rapidly in popularity in the West, rebutting the argumentum ad populum and argumentum ad antiquitatem fallacies may not be the skeptic’s only option. It may be worthwhile, and simpler, just to point out that acupuncture is neither as old nor as popular as is commonly supposed.

References

Chang LC. Huang N. Chou YJ. Lee CH. Kao FY. Huang YT. Utilization patterns of Chinese medicine and Western medicine under the National Health Insurance Program in Taiwan, a population-based study from 1997 to 2003. BMC Health Serv Res. 2008 Aug 9;8:170.

Chen FP. Chen TJ. Kung YY. Chen YC. Chou LF. Chen FJ. Hwang SJ. Use frequency of traditional Chinese medicine in Taiwan. BMC Health Serv Res. 2007 Feb 23;7:26.

Chen FP, Kung YY, Chen TJ, Hwang SJ. Demographics and patterns of acupuncture use in the Chinese population: the Taiwan experience. J Altern Complement Med. 2006 May;12(4):379-87.

Eisenberg DM. Kessler RC. Foster C. Norlock FE. Calkins DR. Delbanco TL. Unconventional medicine in the United States. Prevalence, costs, and patterns of use. N Engl J Med. 1993 Jan 28;328(4):246-52.

Ishizaki N. Yano T. Kawakita K. Public status and prevalence of acupuncture in Japan. eCAM 2010;7(4):493-500.

Quan H. Lai D. Johnson D. Verhoef M. Musto R. Complementary and alternative medicine use among Chinese and White Canadians. Can Fam Physician. 2008 Nov;54(11):1563-9.

Upchurch DM. Burke A. Dye C. Chyu L. Kusunoki Y. Greendale GA.A Sociobehavioral Model of Acupuncture Use, Patterns, and Satisfaction Among Women in the US, 2002Womens Health Issues. 2008; 18(1): 62–71. 

Posted in Acupuncture | 7 Comments

Veterinary Arthritis Treatments

Posted on March 24, 2011 by skeptvet

Over the couple of years of producing this blog, I have written about many different subjects. Some have come up repeatedly, and because they represent common and important issues, I thought it might be useful to collect related posts I’ve done into quick reference lists for some of these subjects. Maintaining an exhaustive and up-to-date list of resources on any one subject isn’t really feasible for a “spare time” project like this blog, but I will try to create and update some of these topic-based references as far as is practically possible.

SkeptVet Arthritis Treatment Reference List

Arthritis is a painful and potentially debilitating disease that commonly affects our animal companions. It is most often seen in older animals as a result of the normal wear and tear of living. However, certain congenital or developmental orthopedic abnormalities, such as hip dysplasia, as well as trauma, obesity, and other health problems, can create arthritis in younger pets as well. There are a number of well-supported conventional therapies for arthritis, including weight loss and non-steroidal anti-inflammatory drug therapy. There are also plausible and potentially useful treatments that have not yet been adequately studied, such as physical therapy and some supplements. And finally, there are implausible, unproven, and even demonstrably useless therapies, such as homeopathy and glucosamine. I have written about many of these therapies, and below are links to relevant articles along with brief summaries. I will try to update this reference list as I continue investigate and write about these and other arthritis therapies.

1. Acupuncture

The historical theories behind the use of acupuncture are unscientific and almost certainly false. And most of the marketing of acupuncture involves misleading and untrue claims about its historic origins and use. There is, however, very limited data concerning its use for arthritis in dogs and cats, and not much more for other conditions. What veterinary research there is is of poor quality and does not strongly suggest a benefit.

The research is much more extensive for humans and does suggest a subjective improvement in comfort when acupuncture is used to treat pain. However, the details of the research suggest this is almost certainly accomplished through psychological mechanisms and possible through very non-specific mechanisms. So while the benefit is small but real for some patients, it probably does not involve any actual change in the underlying disease. Whether such a “placebo” benefit would meaningfully help pets with arthritis is uncertain, but it does not seem likely.

Veterinary Acupuncture

Another acupuncture study shows it’s a placebo

The history of veterinary acupuncture: It’s not what you think

Electroacupuncture for intervertebral disk disease

2. Chiropractic

The theoretical foundations of chiropractic are vitalist pseudoscience and almost certainly false. There is no evidence that the “vertebral subluxation” chiropractors often claim to treat exists at all, and even many chiropractic professional organizations are beginning to distance themselves from this historical concept. There is virtually no controlled scientific research on the subject of chiropractic for arthritis in pets, though it is commonly recommended by chiropractors and some veterinarians. The research in humans shows some likely benefit for uncomplicated lower back pain, essentially equivalent to standard therapies such as physical therapy, stretching and exercise, massage, and non-steroidal anti-inflammatory medications. The evidence does not support the use of chiropractic for any other conditions.

Veterinary Chiropractic

The end of chiropractic? Of course not!

Chiropractic: the more we look the less we find

3. Homeopathy

The theoretical foundations of homeopathy are completely incompatible with well-established scientific understanding of physics, chemistry, and biology. The extensive research in humans shows no convincing evidence of any benefit for any condition beyond placebo effects. The research in animals is sparse and of poor quality and does not support the use of homeopathic remedies in the treatment of arthritis.

Veterinary Homeopathy

Homeopathy works for arthritis: Or maybe not

4. Glucosamine and Chondroitin

Glucosamine and chondroitin are chemicals found naturally in joints and cartilage. There was once good reason based on sound scientific reasoning and in vitro studies to think that oral supplementation of these chemicals would be useful in the prevention or treatment of arthritis. However, the research in companion animals, which is quite limited and of variable quality, has not shown convincing evidence that this is actually true in the real world. And the extensive research in thousands of humans over decades pretty clearly shows that these products have no benefit beyond placebo. They are almost certainly safe, and very likely useless for most pets.

Veterinary Glucosamine and Chondroitin

Growing skepticism about glucosamine for arthritis in dogs and cats

Is recommending glucosamine for arthritis evidence based medicine or wishful thinking?

Nope, glucosamine and chondroitin still don’t work in humans

Cognitive dissonance in action: Glucosamine no matter what!

LEGS Glucosamine Study-Little evidence of meaningful benefit

5. Fish Oil

Essential fatty acids from fish, including EPA and DHA, have many potential benefits based on sound theoretical and in vitro work showing their potential to reduce inflammation. There have been very few studies in companion animals on their usefulness for arthritis, and these do not seem to show much if any benefit, though the work is preliminary. The research in humans is also variable in quality and in results. Fish oils are very likely safe for most pets, and they may or may not have any beneficial effects on arthritis, but the data so far is not encouraging.

Two studies of fish oil for canine arthritis

Another study of fish oil for canine arthritis

5. Electromagnetic Therapy (PEMF)

Various devices that expose arthritis joints to electromagnetic fields as a treatment for arthritis are available and in use for humans and animals. There is some in vitro evidence that electrical fields certainly have effects on living cells, but this says nothing about exactly what these effects are and if they are helpful, harmful, or insignificant in living animals. Much of the marketing of these devices uses misleading pseudoscientific language to talk about mystical vitalist notions of “energy” as if they were established scientific principles. There is so far no reliable scientific research on the use of these devices for treatment of arthritis in dogs and cats. The limited research in humans is variable in quality and results and is so far inconclusive.

PEMF devices for pets

6. Cold Laser Therapy

Non-cutting laser therapy is widely used by chiropractors and some other CAM practitioners to treat pain and many other conditions. While laser light does have measurable effects on cells in vitro, this says nothing about potential clinical uses of laser light. Much of the marketing of these devices uses misleading pseudoscientific language to talk about mystical vitalist notions of “energy” as if they were established scientific principles. There is so far no reliable scientific research on the use of these devices for treatment of arthritis in dogs and cats. The limited research in humans is variable in quality and results and is overall very weak.

Cold laser therapy

7. Stem Cell Therapy

One of the hottest, most fashionable new therapies for a wide range of ailments is autologous stem cell therapy, in which fat is taken out of an animal, stem cells are extracted from it, and then these cells are injected back into the patient. There is extensive theoretical and laboratory work in animals and humans to indicate a variety of effects of these cells, and there is good reason to believe that clinical benefits may be possible. Unfortunately, there is no agreement about what these cells do in living animals and how they do it, and there is very little clinical research evidence to support any one of the many different commercial stem cell therapies marketed for arthritis in dogs and cats. Leading researching in human and veterinary stem cell therapies caution that our knowledge about these cells and what they do is too preliminary to justify claims that they are safe or effective in real patients. I am hopeful that safe and effective stem cell therapies will one day be available, but so far none have proven themselves and using them is still a gamble.

Veterinary stem cell research: Is this the best we can do?

Vet Stem’s stem cell therapy and Chemaphor’s Oximunol join forces

Selling veterinary stem cell therapies: Medivet’s dodgy advertising

Stem cell therapy: Still an uncontrolled experiment on our pets

Veterinary stem cell therapies discussed at Fully Vetted blog

8. Non-steroidal anti-inflammatory drugs (NSAIDs)

There is a huge group of NSAIDs license for treatment of arthritis in dogs, and the evidence from the theoretical and in vitro level through multiple clinical studies is unequivocal that they have a significant positive effect on arthritis. There is no doubt that they have potential side effects, as all effective medicines due. But these are mostly well-understood, and with proper use of the medicines and proper monitoring of the patient the risks are far less than the benefits, or than the propaganda of alternative medicine proponents often suggests.

Safety and efficacy of NSAIDs for canine arthritis

A new tool for evaluating the effects of arthritis treatments in dogs

9. Prolotherapy

Prolotherapy is a purported treatment for connective tissue and joint pain and disability. It involves injecting substances which induce inflammation and other chemical and cellular reactions into affected tissues. These reactions are theorized to relieve pain and improve function. The logic of this theory is questionable, and no clear mechanism for beneficial effects from prolotherapy has been described, but it is possible that the theory could be valid.

The clinical research on prolotherapy in humans is generally of low quality and results have been mixed. There is great variation in the techniques used by different investigators, so it is difficult to compare or generalize between studies.

There is virtually no controlled research investigating prolotherapy in companion animals, and all claims made for safety and efficacy in these species are based solely on anecdotal evidence.

The use of proltherapy in pets should be viewed as experimental with unknown risks and benefits. Such treatments should be reserved for patients that have significant symptoms that have failed to respond or cannot be treated by conventional means.

Prolotherapy for Dogs and Cats

10. Autologous Platelet Therapy

Encouraging Study of Platelet Therapy for Arthritis in Dogs

11. Other Supplements

Duralactin for Arthritis

12. Other Reviews

American Academy of Orthopaedic Surgeons Review of Arthritis Treatments

Posted in Topic-Based Summaries | 56 Comments

Chiropractic: The More We Look, the Less We Find

Posted on March 20, 2011 by skeptvet

Edzard Ernst recently commented on two Cochrane reviews involving chiropractic, which drew my attention to these reviews. I have to admit finding the results a little surprising.

In general, the total body of evidence fails to support most of the claims chiropractors make about the nature of illness and the beneficial effects of chiropractic. However, there has been some decent research evidence to support chiropractic as a treatment for uncomplicated lower back pain, no more nor less effective than conventional treatment. Despite the theoretical problems with chiropractic, and all the associated nonsense (colon cleansing, cold laser therapy, supplements, and so on), I’ve always accepted that at least there is some legitimate role for chiropractic treatment in the management of back pain.

However, the Cochrane reviews that Dr. Ernst reports cast some doubt on even this claim. The updated review of spinal manipulative therapy finds good quality evidence only for ” a small, statistically significant but not clinically relevant, short-term effect on pain relief.” The reviewers could not determine whether this effect is any greater than placebo because only poor quality studies with a high risk of bias investigated this question.

Chiropractors, or at least “mixers,” often claim that they do more than just manipulate the spine. A new review looked at research for this combined chiropractic interventions for low back pain. The conclusions:

while combined chiropractic interventions slightly improved pain and disability in the short term and pain in the medium term for acute and subacute low-back pain, there is currently no evidence to support or refute that combined chiropractic interventions provide a clinically meaningful advantage over other treatments for pain or disability in people with low-back pain. Any demonstrated differences were small and were only seen in studies with a high risk of bias.

So the best we can say is that spinal manipulation alone has a small, short-term impact on back pain, probably not enough to matter to patients and certainly no better than conventional therapy. Combined or “holistic” chiropractic care does no better. So the research evidence, after more than a hundred years of study since chiropractic was invented, clearly shows no benefit for any problem other than uncomplicated lower back pain. And even for this indication, the benefit is small, possibly no greater than placebo and certainly no greater than conventional therapy. So is this benefit worth being exposed to the rampant pseudoscience promulgated by so many chiropractors, the aggressive marketing of “preventative” chiropractic treatment, and all the other nonsense that so often accompanies most chiropractic treatment?

Posted in Chiropractic | 30 Comments

SkeptVet Gets Hate Mail

Posted on March 14, 2011 by skeptvet

This blog grew out of my own efforts to educate myself and my clients. I began looking into alternative medicine in veterinary school (about 14 years ago now, gasp!). Having heard about James Reston’s supposed acupuncture anesthesia in the 1970s, I assumed acupuncture was a generally effective therapy, and I wanted to learn how to use it. However, the more I learned, the less I believed about the claims made for acupuncture, and the more aware I became of the role of our beliefs and desires in interpreting the evidence of our senses. I gradually learned to have a healthy awareness of our inherent limitations as human beings that lead us to mistaken beliefs about health and disease, and of the great value of science, however imperfect, in compensating for these limitations.  

Once in practice, I was called upon almost daily to offer my professional opinion on many common questions from clients about how to best restore or maintain their pet’s health. In response to these questions, I began actively and critically investigating and evaluating the evidence for a wide range of medical interventions, both conventional and alternative. I began writing a series of handouts on these questions, covering common CAM therapies such as homeopathy, acupuncture, chiropractic and so on, as well as questions about conventional interventions such as neutering and vaccines. This series, and my growing personal and professional involvement with evidence-based medicine, led to this blog.

The primary purpose of this blog is to make both the specific information I discover and the general principles of science-based medicine available to anyone who is interested. While SkeptVet is a nearly microscopic fish in the Internet ocean, readership has grown quite a bit over the nearly two years of its existence. As near as I can tell from the comments and emails I receive, my audience is a mix of veterinarians and other veterinary professionals and pet owners and breeders. I sometimes get great questions which lead me to investigating and learning about subjects I would have otherwise been unaware of. And I get some very rewarding feedback, by comment or email and, occasionally in person, from people who have found my posts informative, interesting, or helpful in some way. That keeps me slogging away since that is, after all, the point.

But of course many of the responses I get are highly critical. Some of the criticism I receive is thoughtful, informed, and civil and provides a beneficial and necessary impetus for me to challenge and question my own beliefs and assumptions. That I always welcome. Unfortunately, most of the critical responses I get are not thoughtful, informed, or even civil. There are a few “stock” ideas in many of these responses, and quite a variety in terms of styles and degree of eloquence.

My general policy is that I approve any comment to the site regardless of the content so long as it is not simply gratuitous personal abuse or completely unintelligible (and, of course, spam which is blocked automatically for the most part). I have only had to block about half a dozen comments since the blog began, so I think my policy is quite liberal. However, it amazes me how irate some people get at the mere fact that I moderate comments at all, as if I were somehow obligate to let them post anything they like, no matter how vitriolic or incoherent, on a site I created and maintain as an informational resource.

I thought it my by interesting, and even entertaining, to take a look at some of the criticism I receive and to see if any meaningful patterns can be. So far, I’ve come up with three basic categories (and excluding, of course, the most vicious and profane, which I simply delete unless I think it may one day be needed to obtain a restraining order).

I. Who the hell do you think you are? I’m as smart as you and I’ve seen XX work, so who cares what you think? Science doesn’t know everything and neither do you!

I’ve tried to address the issue of anonymity and my background, as well as the issue of how unreliable the “I’ve seen it work” argument is, in the FAQ, as well as in many posts and comments I make. Still, this is by far the most common critical comment I get. Here are some examples.

My dog has been diagnosed with Cushing’s disease 2 years ago. She was very sick. I did what the vet said but it all just got too expensive. I tried Superglan and she doesn’t even seem like she has the diesease. I think everyone has their own opinion and if you want to try it, then try it. Who cares what everyone else thinks!

Traditional Chinese Medicine has been around for thousands of years, and is a proven methodology of health care. Just because it doesn’t follow the Western medical culture doesn’t mean it doesn’t exist!… It is a shame that no Western medical studies have been done on this product- because that is the only way that close-minded people like you will ever even give it a try.

I started off taking you seriously, but after reading a few of these exchanges, you lost all credibility with me. Someone who was truly unbiased would at least consider the fact that it MIGHT work. Even if it has not bee scientifically proven, a true skeptic does’t come to ANY conclusions without evidence. You have clearly come to a conclusion without evidence, and your bias became clearer and clearer the further I read. Bias is nearly always based on something personal. So did you have a negative experience with this product, or a similar one, or are you being paid?… basically this was just a rant attacking a lack of controlled study, and showing your disdain for intuition and the powers of observation of pet owners. If you want to discard human powers of discernment and behave like a robot, or a computer, that’s fine…Clearly you expect to only encounter only non thinking persons. Your whole act of “Let me, the expert, the person who is more informed than you, enlighten you about critical thinking and scientific study so you do not harm yourself in your ignorance” is just as bad as any snake oil salesman. You can only peddle that crap to someone who doesn’t know enough to spot bullshit and call it…If this is just a hobby, playing expert on a product based just on looking at a website… well… Maybe you should try going out? Or knitting? Something productive?

My arguments are based on practice and outcome. I fix dogs that have suffered from commercial dog food related diseases with plenty of cases in hand…It’s not about science, it’s about results that speak for themselves.

You lack humility and exude arrogance not because you follow your conscience but because you believe that those who don’t agree with your premise must be wrong. You say that, “Being critical of bad ideas and false claims is also not arrogance, it is conscience.” How is it that you have the wisdom to decide which ideas are false and which claims are bad?… When confronted with evidence from pet owners and other vets you dismiss it with the back of your hand as “unscientific”. That is the hallmark of arrogance from a pseudo intellectual. You essentially are saying that your opinion is better than their observations. That is what I meant when I suggested that you need to develop a little humility….You ought to consider using that superior intellect of yours to actually do some research and solve some medical mysteries. To paraphrase Woody Allen, those that can’t do teach and those that can’t teach author critical blogs about those that do.

Had I found your website first, I likely would have not ever tried Supraglans. Im glad things worked out the way they did. As far as testimonials, or more specifically, personal experience with this product being worthless and not scientific enough to be valid, I thought clinical trials were pretty much a give it to em and see if it works deal.

It seems like I exposed your thin skin and tweaked one of your ivy league nerves. Maybe it’s the old doctor-as-god complex rearing its ugly head again. How dare a mere civilian untrained in the ways of the medical profession question my superior authority and intellect when I have the scientific method to validate my truth. Once again, even with the scientific methods that you are so proud of, the “truth” evades you…Your post, more than anything I could ever write, shows the incredible arrogance of your profession. Do you not see it? Are you so full of yourself that you can’t open your mind to the possibility that you don’t possess all of the knowledge worth knowing in the healthcare universe?… You, even with your so-called scientific methods and Ivy League education are no better than me or countless other caring pet owners who don’t care where you went to school or how many footnoted treatises you read to formulate your scientific opinion. You are wrong as much as you are right that’s why they call it “practicing medicine”. Keep practicing, good doctor; maybe someday you will get it right.

If you have had extensive experience in an alternative field… but months, if not years in an alternative field, and you still found that it did not work…. I would commend you for having an open mind and actually trying it. But you seem to be putting down every modality without experiencing it for yourself. Have you tried every modality to validate everything you write about? You can’t write about acupuncture and say it doesn’t work, if you have not followed a case from begining to end, with an experienced vet doing it… and found that it did not work….I think your problem is not that there is no evidence saying that alternative medicine works, but the fact that you have your fingers in your ears like a little kid saying lalala I can’t hear you. When you live your life with blinders on, you miss out on alot.

With nearly half of all dogs passing from cancer, the cancer rate in cats nearly doubling since the distemper vaccine was made a yearly thing, I think we need to look beyond the blinders that were placed on our faces. We don’t have to be one extreme or the other. We need to all come together for the health of our animals, and our own health. ..This type of aggression, stress and hate is extremely unhealthy for us. High blood pressure and heart attacks to name a few.

i don’t care what research has been done the evidence is in the results… But when you use it on a daily basis and see results you will never be able to tell me it doesn’t work because frankly it’s just not true.

All I have to add to this is that acupuncture does work. I’ve seen it work. To me that is all that matters.

II. You are only saying this because you’re afraid you’ll lose business. You must be a tool of Big Pharma. You vets make more money if our pets are sick, so you don’t want us to have safe, cheap, effective natural remedies.

Obviously this website is biased against complementary integrative veterinary medicine….keeping comments and ideas one sided and supported pharmaceutical and commercial pet food monopolies which have been raking in the money for many decades…Threatened financially and ideologically, they must resort to political tactics of attack, shock and awe using headlines inspired by the National Inquirer or some other ladies gossip rag…If you were a legitimate blog looking for the Truth and not a shill for the pharmaceutical companies, you would have researched both sides of any issue.

The old 70 year old urologist told me I was crazy for believing that acupuncture helped my stone. I think he is crazy for thinking I would have let him make another 15 grand off of a surgery that caused me more pain and suffering than I had ever experienced. I wish someone would have told me that I could have just bought a 12 dollar bag of herbs. They don’t taste so great, but hey, it beats the heck out of feeling like the mob got a hold of your kidney with a bat. Or should I say the rich Mob Doctor MD.

I believe that traditional veterinarians are today motivated by GREED and the medicine they practice does as much harm as much as it helps…And how about the local vet’s push for more and more, now found to be harmful, vaccines they are always telling us pet owner are needed- just so they can make more money, not to mention the those oh so toxic flea meds,

it looks like the only faith you have is in your holier than though self. You remind me of our consulting vet. you can show him and show him, but when it comes down to the bottom line what the drug companies will do for him, he will jump on their bandwagon even when it doesn’t work

After reading your “rant” (so accurately described by another reader), I am left feeling like you must have a financial interest in big pharma, for that is the only logical reason I can see that you would put such effort in condemning a product that has successfully treated and prolonged the lives of so many animals… Maybe you should conduct a clinical research study with neo and see the results for yourself? Oh wait, you wouldn’t make the money using neo as you do with chemo and radiation, my bad. It sure does seem like your motives are financial,

III. Just plain angry

Thanks to scientists. we don’t believe in religion any more. Huge department stores have become our Mecca. Now we are told CAM is bunk and not to believe it.

What I object to is this dictatorial attitude of medics that we should obey them or else, and their efforts to shut down anyone who opposes them. They have become the new Popes of the scientific religion!

Its a shame there are Veterinarians out there that think like you and thankfully they are becoming fewer by the day…It would be a sad day for all animals if they had only you to treat them for cancer.

All this talk about science vs. marketing and that only a vet (or doctor) that practices with prescriptions and surgery are worth anything. This is the same mentality that has almost all of Americans frail and sick! My god, when is every person going to realize that we are living breathing cells who need nutrition and balance, not knives and drugs

Posted in General | 21 Comments

Irreverent Comic Guide to Warning Signs of Quackery

Posted on March 10, 2011 by skeptvet

Anti-Science Red Flags of Quackery

From comic blogger Sci-ence (sorry, can’t get the schwa character in there).

Posted in Humor | 13 Comments

Is Recommending Glucosamine for Arthritis Evidence-Based Medicine, or Wishful Thinking?

Posted on March 9, 2011 by skeptvet

In January, I discussed growing skepticism about the benefits of oral glucosamine and chondroitin as a treatment for arthritis in dogs and cats. In that post, I made reference to a short feature I wrote which appeared in the Journal of the American Veterinary Medical Association (JAVMA) examining the clinical trial evidence for the use of glucosamine and chondroitin. In this article, I stated:

Oral administration of glucosamine and chondroitin is often used for prevention and treatment of osteoarthritis in dogs, and there is widespread belief in the safety and efficacy of this practice. However, it is important to base recommendations to clients on the best possible research evidence and not solely on the popularity of a practice or anecdotal reports of positive outcomes…

[T]here was insufficient evidence to support a recommendation of glucosamine and chondroitin as an alternative to NSAID medication for treatment of clinical signs attributed to osteoarthritis in dogs…Glucosamine and chondroitin are perhaps the most widely used nutraceuticals for treatment of osteoarthritis in human and veterinary patients. It is worth considering, however, that there is only very weak clinical trial evidence to support this practice and that it is appropriate for veterinarians to temper their recommendations to their clients accordingly.

A recent commentary in Veterinary Practice News by Dr. Narda Robinson, a professor of “integrative” veterinary medicine at the Colorado State University, and occasional participant in discussions here, takes issue with the conclusions of my review. I have a lot of respect for Dr. Robinson, and I think we agree on many issues associated with evidence-based veterinary medicine, but there are some fundamental disagreements I think are uncovered by her approach to evaluating the evidence concerning the use of oral glucosamine and chondroitin for arthritis.

She begins by referring to recent analyses  and clinical trials in humans, which add to the already existing data, that seem to indicate glucosamine and chondroitin are no better than placebos in treating arthritis pain. However, even in referring to these results, Dr. Robinson begins to lay the foundations of her ultimate conclusions, citing authors of glucosamine studies to illustrate that no one other than “skeptics” such as myself seems to care what the evidence says. This seems distinctly at odds with her claims to an evidence-based approach.

After finding no clinically relevant effects on perceived joint pain or joint space narrowing, the authors wrote, “We are confident that neither of the preparations [glucosamine or chondroitin] is dangerous.  Therefore, we see no harm in having patients continue these preparations as long as they perceive a benefit and cover the costs of treatment themselves.”

From another glucosamine review, “[I]t is likely that most consumers find the presence or absence of clinical evidence demonstrating efficacy to be irrelevant.”

According to Dr. Robinson, while “unsure of glucosamine’s benefits, many veterinarians nevertheless err on the side of hope…” and prescribe one of the plethora of possible glucosamine and chondroitin containing products on the market. She then refutes a couple of possible objections to the claims that oral glucosamine and chondroitin are beneficial for arthritis treatment. In my opinion, these refutations vary in quality from clearly correct, to correct but unbalanced, to outright misleading. I will address each briefly.

Specific Factual Points

1. Oral absorption of Glucosamine:

Dr. Robinson- Measurable amounts of glucosamine can be absorbed and reach the blood and the joints after oral administration at clinically reasonable doses.

SkeptVet- I agree. While the study cited was conducted in horses, which would not necessarily be relevant in dogs and cats, similar studies suggest low but measurable bioavailability for these species. This establishes the plausibility of oral glucosamine, though of course it doesn’t directly demonstrate any meaningful clinical effects.

The levels found in plasma and joint fluid after oral administration in these studies are often significantly lower than in many of the in vitro studies that show effects of glucosamine and chondroitin of the synthesis, degradation, or metabolic activity of cartilage cells (for an excellent review, see the introduction to this study). So the clinical relevance of the bioavailability of these substances remains to be demonstrated.

2. Glucosamine Doesn’t Build Cartilage

Dr. Robinson- “Glucosamine serves as a precursor for glycosoaminoglycans, the main component of joint cartilage.”

SkeptVet- I agree, sort of. Though glycosoaminoglycans are made naturally starting with glucose, glucosamine is an intermediate in the pathway to synthesizing them, and it appears that providing cartilage cells in vitro with extra glucosamine can affect this synthesis. It is far less clear, however, whether this actually happens in the joints of living animals given oral glucosamine. This is really what matters, of course, so referring to glucosamine as a precursor in cartilage production can be a bit misleading because it implies that taking glucosamine stimulates cartilage production in patients with arthritis, which may not be true.

The rat model study she refers to did find some observable effect on cartilage damage and metabolism in rats given glucosamine, and a similar rabbit study also found some effects. Both studies, however, also found no significant change in some measures of cartilage loss, and the subjects had extensive cartilage damage even when given glucosamine, so whether the supplement would have a meaningful clinical benefit even in these models, much less the very different environment of a cat or dog joint with naturally occurring arthritis, is hard to say. Human trials are also mixed, with some showing a beneficial effect on cartilage and others not. So a real benefit is possible, but certainly not proven. 

4. Glucosamine Doesn’t Reduce Inflammation

Dr. Robinson- “[glucosamine] reduces release of inflammatory mediators. Glucosamine appears to penetrate inflamed joints more readily than healthy ones…”

SkeptVet- I agree, sort of. Certainly, glucosamine given orally and absorbed at low levels into the bloodstream is likely to penetrate inflamed joints more readily than healthy ones. This is just a function of the increased permeability of capillaries in areas of inflammation, and it is true for most substances in the blood stream. It doesn’t tell us if the glucosamine is doing anything useful in these joints. And, of course, it actually argues against the notion sometimes advanced that glucosamine may have a protective benefit for joints not yet affected by arthritis, since apparently even less of the absorbed glucosamine is likely to penetrate into such joints.

As far as the anti-inflammatory effects of glucosamine and chondroitin, they are certainly established in vitro. There is some debate about whether or not they are clinically relevant effects at the concentrations actually achieved in joints when they are given orally. Much of the in vitro research involves either amounts of these agents that are much greater than achieved in actual patients or study of individual inflammatory markers or chemicals, which may or may not tell us much about the effects in the “real-world” environment of an arthritic joint. 

6. Glucosamine and Diabetes:

Dr. Robinson- “oral glucosamine/chondroitin does not affect glycemic control or lead to diabetes in the short term. But this research does not answer the question of whether glucosamine is safe for diabetic dogs.

SkeptVet- I agree, and I would probably state the case even more strongly. Given the extensive evidence in human diabetic, and the much more limited research in cats and dogs, I think it very unlikely that glucosamine would cause clinically meaningful problems for diabetic pets. Just as early studies on the benefits of glucosamine were not supported by later, better research, so many of the early studies on the risks of the supplement have not been sustained. The research in pets is not definitive, but the balance of the evidence available is pretty clear. 

7. Clinical Benefits of Glucosamine

This is one area in which Dr. Robinson and I pretty clearly disagree. I think she is correct that the results of clinical trials are conflicting, but I think it is a mistake to believe that this means no firm conclusion on efficacy can be drawn. In his book Snake Oil Science, published in 2007, R. Barker Bausell reviewed the clinical research on glucosamine in humans exhaustively, and the balance of the evidence was clearly against any meaningful clinical effects. The larger and better designed or controlled the study, the less likely it is that a benefit will be seen. Studies funded by companies marketing glucosamine products are much more likely to be positive than studies funded by more neutral parties. And since this book was published the studies and reviews I provided links to above have continued to produce mostly negative results.

The evidence concerning the clinical effect of glucosamine in humans is not absolutely uniform, of course. But it never is. There is a well-known process, sometimes called “the decline effect,” by which early studies, often small and poorly designed and often funded and or run by companies or individuals with vested interests in a given hypothesis, and later studies by other researchers with better samples and designs fail to confirm the initial findings. This is not a flaw in science but rather an example of why methodological quality and independent replication of results are so crucial in separating the wheat of truth from the chaff of all the great idea that ultimately turn out to be wrong. Glucosamine research is almost a paradigm of this process, so it is misleading to point to the inconsistency in clinical research results as if it indicated deep uncertainty, when in fact it indicates a gradual refining of the data leading to the conclusion that a meaningful clinical benefit in humans is quite unlikely.

Dr. Robinson raises the issue of heterogeneity to explain the inconsistency in clinical trial results. Heterogeneity simply means that any population of subjects in a clinical trial will, of course, not consist of identical individuals. It is possible that the intervention being tested might work for a subset of individuals with particular characteristics, but if we don’t know what these characteristics are, randomization will scatter these folks evenly between the placebo and treatment groups, and it will look like the treatment doesn’t work. Heterogeneity is a known limitation of clinical trials and population statistics, which can to some extent be controlled for but which will always present the risk of study results which are valid for populations but not for some individuals.

This is a legitimate concern, but unfortunately it is also an easy way to cast doubt on all clinical research and lay the foundations for the argument that clinical trials ultimately can’t tell us anything useful about how to treat individual patients. It is important to be careful that our concerns about the issue of heterogeneity does not become a kind of clinical trial nihilism that leads us to give up entirely on the immense proven value of clinical trials in guiding medical interventions. Put another way, nothing is perfect, including clinical medical research, but we must be careful not to let the perfect become the enemy of the good to the point where we cannot make meaningful conclusions based on the evidence that exists. This is a caution commonly directed at skeptics who challenge the quality and conclusions of CAM research, but it is also applicable to proponents of CAM who argue that the imperfections of science invalidate the conclusions it reaches on their preferred methods. I believe Dr. Robinson herself appreciates the value of clinical research, but the heterogeneity argument she uses is one that can easily be taken to unfortunate extremes, and has been by others.

The clinical research on glucosamine/chondroitin in dogs and cats is far less in quantity and quality than the human clinical trial data. In my JAVMA paper (which, incidentally, was not a complete review of the subject but only an illustration of a practical and efficient use of evidence-based medical reasoning to answer a focused clinical question), I mentioned two clinical studies. One found no benefit for glucosamine and the other showed little benefit. Both showed far greater and more predictable benefit to NSAID therapy, which is a consistent feature of clinical research on glucosamine and chondroitin.

In her commentary, Dr. Robinson mentions another study which I did not address in my JAVMA piece, since this study involves cats and my article only addressed glucosamine for dogs with arthritis. It is not really a direct investigation of glucosamine/chondroitin supplementation for arthritis but rather of dietary therapy with a diet that includes glucosamine and chondroitin along with other ingredients thought to be beneficial for arthritis, but it is an interesting study which deserves a closer look.

The study was a well-designed randomized, blinded trial with a 70-day monitoring period and a mixture of objective and subjective measurements. 40 cats completed the study, evenly divided between the test diet and a control diet identical except for the absence of fish oils, glucosamine, chondroitin, and green-lipped mussel extract. Obviously, one issue with this study is the presence of multiple ingredients aimed at treating the subjects’ arthritis, which makes it impossible to say anything definitive about whether the glucosamine and chondroitin were ore were not beneficial.

Another concern is that there was one significant difference between the control and test cats, namely that the control cats started the study weighing more and, while both groups lost weight during the study (though not very much), the control cats weighed significantly more at the end of the study. Weight is a very important factor in the clinical symptoms of arthritis. Heavier cats would be expected to have worse symptoms and more rapid progression, and weight loss would be expected to improve symptoms all by itself, so this could confound the findings of this particular study.

As usual, the results were mixed, but as I read the paper they strike me as not very impressive. Subjective pain scoring by owners (who may or may not have been effectively blinded to the treatment and placebo allocation, it’s impossible to tell from the report) improved for both groups, but the difference between test and control groups was not significant. This may represent a “placebo” effect of being enrolled in a clinical trial, or  less likely and effect of weight loss.

The objective measure, using an automated activity monitor, showed no overall significant difference between the groups. Interestingly, the control group showed significantly less activity overall and at 2 of the four times of day analyzed when 14-day periods at the beginning and end of the study were compared. A complex regression analysis looking at diet, weight at the start of the study, and weight loss during the study seemed to show an increase in the evening activity level of cats on the test diet, and no change in the activity of these cats at any other time. This analysis also showed a decrease in the activity of the cats on the control diet at all times except overnight.

I am not qualified to evaluate the validity of the statistics involved, but it seems odd that if the test diet had a beneficial effect on arthritis it would show up as a decrease in the activity of the cats on the control diet rather than an increase in the activity of the cats on the test diet. The authors suggest that this may mean that the cats on the test diet had less progression of their arthritis, but I am doubtful that such significant changes represent the progression of arthritis symptoms over only 70 days.

Other subjective measures, such as overall owner-assessed quality of life, improved for both groups with no significant differences between them. A few measures did change significantly in each group from the beginning to the end of the study:

Decreased aggression and eating in control group
Increased sleeping in the control group
Increased jumping and eating in the test group
Decreased sleeping in the test group

There were also some subjective owner assessments of behavior change during the study that differed significantly between the groups. Playing and interaction with other pets increase for both groups, but more for those on the test diet than those on the control diet. Both groups slept less and hid less over the course of the study, but again the change was greater for the test diet group than the control group.

Subjective measures of pain on physical examination by veterinarians in the study showed improvements in both groups but no significant difference between them.

So overall the study shows a lack of significant difference between the groups in most measures but a few differences that do reach statistical significance. The overall pattern is not consistent or compelling, but it may be that the cats on the control diet became less active over the 70 days of the study, which could conceivably suggest some influence of diet on activity. However, most measures improved for both groups, which illustrates the non-specific treatment effects (aka “placebo” effects) that are such a problem in figuring out what are real treatment effects in clinical research studies and what are artifacts of the research process.

In any case, even if the test diet did have real effects on the behavior of the cats in the study, that doesn’t tell us if the effects were clinically meaningful, if they had to do with treatment of arthritis symptoms, or if they were due to glucosamine. Yet the study is cited in Dr. Robinson’s article with the clear implication that it is positive evidence for the benefits of oral glucosamine in the treatment of arthritis. This is a seriously misleading oversimplification which is unfortunately all too common in debates about the merits of medical interventions, particularly those for which the research data is not absolutely unequivocal, which it rarely is.

The Bottom Line

So clearly Dr. Robinson and I agree on many things. We agree that glucosamine can be absorbed in low but measurable amounts when given orally, that some of it reaches joints, and that in vitro at least it can have anti-inflammatory effects. These facts establish the plausibility of glucosamine as a treatment for arthritis, but not that it actually works.

We also agree that it is most likely very safe. There is some risk associated with the poor regulation and quality control of nutraceuticals generally, which often contain dangerous contaminates and other ingredients not on the label. And I have had clients who eschewed proven therapies, such as NSAIDs, because they felt glucosamine alone was sufficient arthritis therapy, which leaves animals with inadequately treated pain. But Dr. Robinson carefully points out that she would not recommend this kind of single-agent use, and overall I think we agree that there is little risk associated with using glucosamine.

Where we disagree, as far as the facts, is in the interpretation of the clinical trial evidence. I think it is solidly against there being any meaningful benefit for oral glucosamine for all the reasons I have indicated. Dr. Robinson appears to believe that benefit is uncertain but possible. While I don’t believe her assessment is correct, it is certainly not irrational or unreasonable. Intelligent, informed, well-intentioned people can certainly disagree over the facts.

However, I think our greater disagreement is not on the question of the interpretation of the evidence but more fundamentally what constitutes evidence-based medicine. She concludes her article this way:

Although the evidence on glucosamine remains contradictory, there does appear to be some value and little risk. Not ready to abandon a product that could very well help and likely not hurt, this evidence-based practitioner will continue to mention glucosamine as one of many options for multimodal analgesia for OA patients and potential disease modifying OA benefits as well.

It seems to me that this amounts to saying, “It’s most likely harmless, and it could help, so why not try it?” I think she most accurately described her position herself when she said, “unsure of glucosamine’s benefits, many veterinarians nevertheless err on the side of hope.” The recommendation of glucosamine as a treatment for arthritis is not a decision based on evidence, but based on hope.

So, as Dr. Robinson asks, is there anything wrong with this? In the case of glucosamine, no not really. But I do worry a lot about the impact of practicing “hope-based medicine” generally, and especially of identifying it as evidence-based medicine. Ten years ago, when the data was less clear, I was hopeful glucosamine might be a useful therapy for my patients. I regularly recommended it, I gave it to my own pets, and I even took it myself. Then, as the data grew clearer, I abandoned the therapy because that ultimately is the approach to medicine that I think is most likely to lead us to the best, most effective practices.

Lots of people hope that homeopathy will help their pets, and many believe it does. And it is certainly harmless in itself, as it is only water. So is it right to use it alone, or even to add it to conventional therapy, “just in case?” I don’t think so. Granted, the evidence against glucosamine is strong but not indisputable, and the evidence against homeopathy is overwhelming. But I’m sure Dr. Robinson, no fan of homeopathy herself, has had exactly the kind of disagreements with proponents of homeopathy that she and I are having here. It is possible to find some positive studies, of poor quality and with high risk of bias, to support the use of homeopathy. Or prayer, or energy healing, or just about any intervention that people hope will benefit themselves or their patients patient. And so it is always possible for intelligent, rational people to be driven by this hope to put the most positive possible spin on the evidence, or at the last resort to say, as I think Dr. Robinson has in her article, “It’s most likely harmless, and it could help, so why not try it?” But that isn’t evidence-based medicine, and even if in individual cases, like that of glucosamine, such an approach may not lead to great harm, overall it undermines the quality and rigor of the reasoning that we use to decide what is truly helpful to our patients and what isn’t.

I regularly tell my clients that the evidence, while incomplete and not entirely consistent, is pretty strongly against any benefit from glucosamine and that the evidence of potential harm is very low, and then I let them decide if they want to use it or not. If I interpreted the existing evidence more positively, as Dr. Robinson does, I might tell them that glucosamine is harmless and might possibly have some benefits though the data is unclear, and again let them decide whether that is sufficient to warrant using it, which is what it sounds like Dr. Robinson does.

So it doesn’t sound like in practice Dr. Robinson and I approach the particular product all that differently with our clients. But the subtle difference in emphasis does seem of some importance to me. Her article gives the distinct impression that her reluctance to abandon the product is based not primarily on the balance of the evidence being positive but simply on the fact that the evidence is not uniformly negative, and if there is any reason to hope it might work it is worth trying (in the absence of clear evidence of harm). As I understand it, truly evidence-based medicine should give the greatest weight to the highest level available evidence and should not give any special weight or importance to our hopes. I understand Dr. Robinson’s perspective, and it makes a kind of sense. But it is all too familiar from many debates I have had with far less reasonable people about far more unlikely, or even ridiculous interventions. Erring on the side of hope is understandable, but it is not evidence-based medicine, and in the long run I don’t think it’s in the best interests of our pursuit of the truth about medical therapies or of our patients.

Posted in Herbs and Supplements | 19 Comments

Herb and Supplement Industry: Loves Marketing but not so Crazy about Research

Posted on February 28, 2011 by skeptvet

One of the standard clichés alternative medicine proponents like to employ is the David and Goliath Myth: pharmaceutical companies and the mainstream medicine are profit-driven behemoths and alternative medicine is the plucky little guy, small practitioners and innovators fighting against the machine to bring inexpensive natural remedies to those in need. This is quite clearly belied by that inconvenient thing called reality.

I have written before about the large and growing for-profit and corporate element of the alternative medicine movement, the underhanded and deceptive marketing practices of herb and supplement manufacturers, and the financial ties between the supplement industry and alternative veterinary practitioners and politicians. Now, Canadian Erik Davis at Skeptic North has provided a detailed and insightful article comparing the finances of several leading herbal remedy and supplement manufacturers and the largest pharmaceutical companies.

Bankers, Buyouts & Billionaires: Why Big Herba’s Research Deficit Isn’t About The Money

In this article, Davis shows quite clearly that while herbal remedy and supplement companies are truly much smaller in financial terms than the leading pharmaceutical companies, they spend proportionally at least as much of their money of marketing and far less on research.

I am certainly not one to deny the myriad sins of the pharmaceutical industry, and I am a strong supporter of vigorous and strict regulation of this industry. Profit is seldom an optimal motive to most effectively meet the medical needs of ordinary people, or veterinary patients. However, the herb, supplement, and homeopathy industries are far less closely regulated, and the numbers show quite clearly that they engage in exactly the same kind of venal behavior Big Pharma is so often accused of. And without the pressure of regulation, these companies make far less of an effort to investigate or demonstrate the safety and effectiveness of their products.

Here’s just one of many examples Davis provides

 

French giant Boiron (EPA:BOI) is by far the largest distributor of natural health products in Canada – they’re responsible for nearly 4000 (15%) of the 26,000 products approved by Health Canada’s Natural Health Products Directorate. They’re also one of largest natural health products companies globally, with 2010 revenues of €520M ($700M CAD)…

“Since 2005, we have devoted a growing level of resources to develop research,” they proclaim in the opening pages of their latest annual report, citing 70 in-progress research projects. Yet the numbers tell a different story – €4.2M in R&D expenditures in 2009, just 0.8% of revenues.

To put that in perspective, consider that in the same year, GlaxoSmithKline spent 14% of its revenues on R&D, Pfizer spent 15%, and Merck spent a whopping 21%….

But if Boiron’s not spending like pharma on research, there’s one line item where they do go toe to toe: Marketing. The company spent €114M – a full 21% of revenues on marketing in 2009. By contrast, GSK, Pfizer and Merck reported 33%, 29%, and 30% of revenues respectively on their “Selling, General, and Administrative” (SG&A) line – which includes not just sales & marketing expenses, but also executive salaries, support staff, legal, rent, utilities, and other overhead costs. Once those are subtracted out, it’s likely that Boiron spends at least as much of its revenues on marketing as Big Pharma.

I can think of nothing to add to Davis’ conclusion, which is an eloquent argument against the faux altruistic PR of Big CAM and for real, effective regulation of the supplement and herbal remedy industry.

Big Herba is clearly big business, and on a purely financial level, it’s hard not to be impressed by what they’ve achieved.  But that success — $2.5B in revenues concentrated in the seven companies above — makes it equally difficult to give them a pass on their research deficit.  Simply put, the leading natural health products companies have the coin for research, they just choose to spend it on marketing products and buying their competitors instead.  The result: while pharma typically spends upwards of 15-20% of revenues on research, Big Herba contributes less than a tenth of that.

To the question of why, I’d like to propose simply that they don’t need to.  The products are clearly selling well already, and without the regulatory approvals pharmaceuticals require, spending money on research presents more risk than reward.  After all, if you don’t conduct research, you can’t find out that your product doesn’t work.

In other words, Big Herba is behaving exactly as Big Pharma might if it had no government oversight.  And if that doesn’t give you reason for pause before you pop that next Ginko tablet, I don’t know what will.

Posted in Herbs and Supplements | 4 Comments

Over-Diagnosed by Gilbert Welch: A must-read for doctors and potential patients

Posted on February 27, 2011 by skeptvet

Dr. Harriet Hall at Science-Based Medicine recently reviewed the book Over-diagnosed: Making People Sick in the Pursuit of Health by Dr. Gilbert Welch. I was immediately interested enough to buy the book and read it, largely because it promised to address a subject I’ve discussed here before, the dangers of unnecessary or inappropriate screening tests and the irrational and counterproductive reaction that often occurs when scientists try to dissuade people from employing or taking such tests.

The book was well worth reading. Dr. Hall’s review covers the book’s contents in detail, so I won’t repeat all of that. In brief, Dr. Welch makes a strong, evidence-based case for several fundamental concepts:

1. The more sensitive our criteria for using a lab test or imaging method to diagnose a disease in someone without actual symptoms, the more likely we are to identify abnormalities that will never cause actual clinical problems.

2. This then leads to people being identified as ill who aren’t, and these people are then subjected to testing and treatment which will not help them, and may actively harm them.

3. There are a number of factors which drive and perpetuate this over-diagnosis:

a. Fear

b. Poor understanding of the meaning, use, and misuse of statistics

c. Uncritical, superficial, and excessively dramatic media coverage

d. Money (industry profits, political contributions, research grant allocation practices, fundraising of non-profit lobbying organizations etc)

e. Lawyers and litigation, which punishes missing a diagnosis but not over-diagnosis and unnecessary or harmful testing and treatment

f. Uncertainty about the risks and benefits of specific tests and interventions

e. All the usual cognitive biases we have, especially confirmation bias and the post hoc      ergo propter hoc fallacy

4. Only objective, systematic data from randomized clinical trials and good quality epidemiological studies can truly tell us whether screening tests for a specific condition help or hurt more people. As usual, anecdotes about individual patients are emotionally compelling and totally unreliable in making individual or public health decisions about disease screening.

The examples Dr. Welch provides are based on enviably comprehensive data and analysis. A surprising number of people without disease will have detectable abnormalities on blood tests, x-rays, and other screening diagnostics. And for almost every condition studied in detail, including many cancers, the majority of these people will never become ill from these abnormalities. The more sensitive our screening tests become for cancer, for example, the clearer it becomes that we all get cancer repeatedly and never know it. Just like we all take in potentially disease-causing organisms all the time. Most of these cancers never develop just like most of the bacteria and viruses we are exposed to never make us sick. So finding these cancers early and treating them does us no good at all. And apart from the anxiety of knowing we have cancer, procedures such as biopsy and surgery and cancer treatment can do very real harm.

Of course, the difficulty is that we do not, as yet, have a trustworthy way to know when we find a small cancer whether it will ever develop into a real problem  if we would benefit from treating it. This makes it difficult to resist looking and treating such abnormalities, even when the evidence is overwhelming that most people don’t benefit and that more are harmed than helped. Psychologically, we are far more afraid of the error of failing to respond to a true threat than we are of over-reacting to a harmless abnormality. Yet if we look at the issue rationally, we are far better off not employing screening tests that are clearly shown by objective data to lead to more over-diagnosis and harm than benefit.

The issue arises in veterinary medicine just as it does in human healthcare, though as always the quantity and quality of the evidence is far less and the degree of unavoidable uncertainty far greater than in human medicine. And, the limitations on resources and lesser significance of litigation alter the equation somewhat. But the general principles are still broadly applicable to the veterinary setting.

Just the other day I had a client with an older cat in for an annual exam and a followup on a known heart condition. The client asked me to perform whole-body x-rays of the cat as part of my workup. When I asked why, he showed me a book on “natural health” for dogs and cats by a prominent “holistic” veterinarian that recommends annual screening x-rays of older pets.

Apart from the obvious question what is “natural” about x-raying your cat every year, the recommendation is irrational and likely to lead to more harm than good. According to Dr. Welch, 86% of people with no signs of illness will have a lesion on a whole body CT scan. Chest x-rays will often find lung masses, and CT and MRI will find even more. And the vast majority of people without symptoms who have such lesions on random imaging screening will never experience any illness from them. But a clearly larger number of people will be harmed by additional testing and treatment, experience painful and unnecessary anxiety, and use up limited healthcare resources pursuing such findings. And though the radiation risk from a single x-ray is minimal, annual radiographs can expose the patient to small but unnecessary risk from radiation.

Similar data is not available in companion animal species, but there is no reason to believe the fundamental rules of biology are different for our pets than for us. The little research that has been done, such as studies of pre-anesthetic screening blood tests in dogs, does not generally support the value of such tests, but the data are still terribly limited. Nevertheless, the general principles and issues Dr. Welch illustrates so well certainly need to be considered in making decisions about screening tests in veterinary species.

I explained all of this to my client, who opted not to have the x-rays done. A surprising amount of my communication with clients involves convincing them not to do things, in contrast to the image CAM proponents often paint of conventional doctors aggressively selling unneeded drugs and treatments. And it is surprisingly difficult to reassure people with facts and statistics and convince them not to pursue unneeded diagnostics and treatment. And, unfortunately, reassuring them by performing meaningless rituals or providing placebo therapies works much better, which simply perpetuates ineffective therapies.

I highly recommend this book, both to potential patients (which is all of us) and to doctors. The conventional wisdom, that it is always better to look for trouble “just in case” is being shown more and more clearly to be wrong. Diagnostic testing, when there are symptoms or a sound rational reason to suspect disease, are absolutely vital. But screening tests for the apparently well are often far less likely to benefit us than is generally supposed, and they do real harm. The burden of proving such tests are a good thing has clearly shifted to those advocating them.

It seems unlikely that there will be a significant shift in the attitude of the public or doctors towards screening tests in the near future. The U.S. Preventative Services Task Force (USPSTF) is a government agency devoted to examining the scientific evidence concerning preventative healthcare testing and intervention, and they often provide sound, rational guidelines. These guidelines, are also often ignored by politicians, the public, and sometimes even healthcare providers. Our fears are still generally more persuasive than our reason. Still, reason and facts can glacially wear away at our fears and misconceptions over time, which is why I still write this blog and why I am happy to find and recommend excellent books such as Dr. Welch’s.

Posted in Book Reviews | 6 Comments

Veterinary Stem Cell Therapies Discussed at Fully Vetted Blog

Posted on February 23, 2011 by skeptvet

A prominent veterinary blogger, Dr. Patty Khuly of Fully Vetted, has written about the issue of veterinary stem cell therapies. The inspiration for her post appears to have been the recent article in JAVMA News, the news service of the Journal of the American Veterinary Medical Association, which discussed stem cell therapies in veterinary medicine. Dr. Khuly’s post was a pretty fair summary of some issues associated with stem cell therapies, which I have discussed numerous times in the past (for example). She seemed to imply pretty clearly in her conclusion that, though the evidence is not there to prove safety or efficacy, owners who are desperate and whose pets don’t have any other treatment options should have this therapy available to them. I don’t entirely disagree, but I think there may be some unjustified assumptions and a bit of a false dichotomy in her  conclusion:

Given the reality that the pets most affected by this are those with conditions that are untreatable by any conventional means, and that time-consuming clinical trials would require onerous delays in treatment, is it wrong to keep this therapy out of owners’ hands? Or is it worse to fail to live up to the oath that exhorts us to act as scientists first?

This question is a bit like “Have you stopped beating your wife?” in that there it contains assumption that may not be true and makes any answer imply something other than what is really meant. I certainly don’t disagree that for pets with severe arthritis and no other treatment options that stem cell therapy is worth trying, but I doubt that constitutes the bulk of the market for these treatments. And I think the objections I and others have raised to the aggressive marketing of stem cell treatments have a bit more to them than an intellectual desire to “act as scientists first.”

Here is the reply I posted to Dr. Khuly’s blog. Anyone interested should take a look at her post and consider participating in the discussion. Dr. Khuly likely reaches a much larger audience than I do, so it would be useful to have a variety of perspectives in the discussion there.

Dr. Khuly,

I’m happy to offer my take on the stem cell question, especially since this seems to be a subject on which I am all too easily misinterpreted. I certainly wouldn’t call myself a “naysayer,” for example. As I’ve said repeatedly, I see great promise in stem cell therapies, and I am cautiously optimistic that they will eventually be a useful tool.

My concerns have more to do with the widespread and aggressive marketing of a therapy in advance of adequate evidence to demonstrate safety and effectiveness. We are all familiar with drugs that had great promise, and that passed far more extensive pre-market testing than stem cells have undergone, and which later turned out to have less benefit than hoped or unexpected hazards (e.g. Vioxx, thalidomide). There have also been many procedural treatments like stem cell therapy which have come on the scene, enjoyed great popularity, and then turned out to be less useful or more dangerous than hoped (e.g. internal mammary artery ligation). The lesson is that the way things seem isn’t always the way things are, and good quality research isn’t just a nice extra, it is critical to making the best choices for our patients.

Given the reality that the pets most affected by this are those with conditions that are untreatable by any conventional means, and that time-consuming clinical trials would require nerous delays in treatment, is it wrong to keep this therapy out of owners’ hands?

I think you may be unintentionally setting up a bit of a false dichotomy here. You seem to be assuming that all the pet owners clamoring for this new treatment are those whose pets have serious arthritis symptoms that do not respond to currently available treatments. (The vast majority of stem cell treatments in small animals, which is my field, are marketed and used for arthritis, so that is what I will address here. As you point out, other applications are primarily still only available in the context of academic research). I am not aware of any objective statistics, but I suspect this isn’t true and that the market for stem cell therapies would be very small if it were limited to such cases. I frequently field inquiries about this therapy from people who have not taken full advantage of existing available treatments for arthritis, the most popular application for autologous stem cell treatments.

There are many therapies for arthritis which are well-established to be effective. Weight loss, physical therapy, non-steroidal anti-inflammatory medications (NSAIDs) are the most common examples. The majority of patients with arthritis respond well to these therapies. However, there is no free lunch in medicine, so anything with a benefit is going to have a downside.

Meaningful weight loss requires restricting a pet’s calorie intake significantly. The pet is going to be noticeably hungry and likely to beg or scrounge for food, and this is understandably hard for many owners. I spend a lot of time counseling owners on nutrition trying to prevent and treat obesity, but the reality is that many owners are not able to stick to a serious weight loss regime for their pets, particularly for older and relatively inactive pets.

Physical therapy is a great intervention, but it has its costs as well. Some are financial, and there is a significant investment in time and energy as well as money for owners seeking this kind of care. We are fortunate to have dedicated physical therapy facilities nearby, but not all veterinarians can easily offer or refer for this treatment, and not all owners readily take advantage of it.

And then, of course, there are NSAIDs. I would never consider denying that they have potentially serious side effects, but the evidence is very clear that, when properly used and monitored, they are very effective and very safe for the overwhelming majority of patients with severe arthritis (e.g. http://skeptvet.com/Blog/2010/12/safety-and-efficacy-of-nsaids-for-canine-arthritis/). Unfortunately, appropriate concern about potential side effects easily becomes irrational fear, and some pet owners avoid NSAIDs unnecessarily.

There is definitely a balance to be achieved between the need to act and the uncertainty about whether our actions will help or harm. For those patients who cannot maintain an acceptable quality of life using existing treatments with well-established benefits and risks, I have no objection at all to offering stem cell treatment as an experimental approach. Ideally, this would include some kind of objective, structured assessment of outcomes, both beneficial and unintended. Casual assessment by owners and individual veterinarians, or uncontrolled research by companies selling the therapy, will always show a benefit and miss all but the most dramatic side-effects for any therapy, so this can’t be our only form of assessment.

I think it is instructive to look at how differently stem cell therapies are being handled in human medicine. The major organization representing stem cell researchers, the International Society for Stem Cell Research, has said:

We have all heard about the extraordinary promise that stem cell research holds for the treatment of a wide range of diseases and conditions. However, there is a lot of work still needed to take this research and turn it into safe and effective treatments.

The International Society for Stem Cell Research (ISSCR) is very concerned that stem cell therapies are being sold around the world before they have been proven safe and effective.

Stem cell therapies are nearly all new and experimental. In these early stages, they may not work, and there may be downsides. Make sure you understand what to look out for before considering a stem cell therapy.

Remember, most medical discoveries are based on years of research performed at universities and companies. There is a long process that shows first in laboratory studies and then in clinical research that something is safe and will work. Like a new drug, stem cell therapies must be assessed and meet certain standards before receiving approval from national regulatory bodies to be used to treat people.

(http://skeptvet.com/Blog/2010/06/stem-cell-therapy-still-an-uncontrolled-experiment-on-our-pets/).

These people have devoted their careers to studying stem cell therapies, so they can’t simply dismissed as “naysayers” as so often happens when skeptics such as myself raise concerns. Stem cell therapies are moving forward much faster in veterinary medicine than in human medicine not because we have better data or a more serious problem to be addressed, but because we have more lax regulation and fewer concerns about lawsuits if something goes wrong. Hopefully, in the end this will turn out to be a good thing because these therapies will be proven safe and effective. However, we cannot totally ignore the possibility that proceeding without rigorous pre-market testing and post-market surveillance will lead us to doing harm unintentionally, or wasting time and resources on an ultimately ineffective therapy.

Just to be completely clear:

1. Stem cell therapy is indeed a promising, though as yet unproven, new therapy. However, only clinical trials can truly show it to be safe and effective, or not, so our current headlong rush into this field based on preclinical findings and individual patient anecdotes is not without risks.

2. Given the uncertainties about the safety and efficacy of this expensive and invasive procedure, all other treatments with established risks and benefits should be utilized fully before we turn to stem cell therapy. This includes NSAIDs which, though not without risks, are very effective and generally much safer than most of my clients seem to think.

3. For those patients who cannot achieve a good quality of life with established treatments, it is perfectly appropriate to offer stem cell therapy as an option. In human medicine, such treatments are generally available through clinical trials or special “compassionate use” provisions. This may not always be practical in veterinary medicine, but we should make our best effort to objectively and systematically assess the outcomes of patients treated with stem cells.

Think about how long it took for us to make the connection between certain vaccines and cancers that developed rarely and many years later. If a similar unanticipated risk exists with stem cell treatment, we’re going to miss it if we are looking carefully, and that would be irresponsible. I would like to see less aggressive, more fact-based marketing and more effort given to evaluating objectively the benefits and unintended effects or complications of these procedures.

You ask if we have the right to keep a potentially beneficial therapy with uncertain risks and benefits from owners who want it. I would phrase the question differently. Do we have a right to sell owners a potentially beneficial therapy with uncertain risks and benefits, often through misleading and unjustifiably positive marketing? Don’t we have a responsibility to ensure as best we realistically can the safety and effectiveness of any therapy we offer our patients?

Posted in Science-Based Veterinary Medicine | 10 Comments