Another study of acupuncture in dogs has been published, and it’s worth a look. It’s a classic illustration of clearly negative results being misrepresented as positive. The authors set up a nice study of acupuncture compared with a common non-steroidal anti-inflammatory or an oral placebo for treatment of pain and lameness in dogs with hip dysplasia. The results pretty clearly showed no effect, and the authors duly wrote a conclusion suggesting acupuncture was a good choice for treating these dogs. Ah, the magic power of confirmation bias!
Teixeira LR, Luna SPL, Matsubara LM, et al. Owner assessment of chronic pain intensity and results of gait analysis of dogs with hip dysplasia treated with acupuncture. Journal of the American Veterinary Medical Association. November 1, 2016, Vol. 249, No. 9, Pages 1031-1039.
The authors of this study concluded that “acupuncture alleviated HD-related pain [and] …decreased lameness” and that “acupuncture performed with the protocol and acupoints used in this study appears to be a viable option for improving quality of life in dogs with HD.” Unfortunately, the results of the study do not support this conclusion.
For one thing, the authors stated that “the acupoints were selected on the basis of their functional effects in traditional Eastern medicine.” That is the only reference they make to the fact that the entire theoretical basis for the treatment rests on the mystical folk system of so-called Traditional Chinese Medicine (TCVM). This is undoubtedly intentional since it would be difficult to present acupuncture as a scientifically legitimate treatment if vets understood that needles were being placed in non-existent channels intended to control the flow of spiritual energy, balance heat and wind, and in keeping with other such pre-scientific notions. TCVM is incompatible with the scientific understanding of health and disease, and whether or not needling has any clinical benefits, it is irrational to use TCVM to select needling locations.
A more concrete problem with the study, however, is the data. The graphs below show the changes in various measures of pain and function over the course of the study, from 2 weeks before treatment to 6 weeks after. Treatment consisted of weekly acupuncture from Week 0 to Week 4 or an oral medication (carprofen) or placebo (lactose) daily from Week 0 to Week 4. Before even looking at the specific outcome measures, it is clear that all of the groups improved almost exactly the same and that overall the placebo group did as well or better than the acupuncture group. Not much more should be needed to conclude this study does not support acupuncture for dogs with hip dysplasia.
Figure 1—Mean scores derived by use of the HCPI questionnaire completed by owners of HD-affected dogs 2 weeks before (week –2), immediately before (week 0), and at 2, 4, and 6 weeks after initiation of treatment with acupuncture (diamonds), carprofen (squares), or placebo (triangles) and by owners of HD-free dogs (cross) at week 0 only. Fifty-four HD-affected dogs were initially allocated to receive once-weekly sessions of acupuncture (5 sessions within a 30-day period), 4.4 mg of carprofen/kg orally once daily (2.0 mg of carprofen/lb; positive control group), or placebo capsules containing lactose (1 mg/kg [0.45 mg/lb]) orally once daily (negative control group). Acupuncture-treated dogs also received capsules containing lactose orally once daily and carprofen- and placebo-treated dogs were brought to the acupuncture clinic once each week for a total of 5 occasions within a period of 30 days. All of the oral treatments were administered for 30 days. Data from only 15 acupuncture-treated dogs, 16 carprofen-treated dogs, and 16 placebo-treated dogs were available for analysis. The HD-free dogs (n = 16), were selected from kennels or belonged to clients interested in HD-free certification for their dog. At each time point, each owner evaluation involved 3 assessment instruments: the HCPI; the CBPI, including total scores and those corresponding to pain severity and pain interference; and VASs regarding pain and locomotion. *At this time point, the value for each group of HD-affected dogs differed significantly (P < 0.001) from the value for the HD-free dogs.
Figure 2—Median scores for pain severity (A), pain interference (B), and total pain scores (C) derived by use of the CBPI questionnaire completed by owners of the HD-affected and HD-free dogs in Figure 1. For the HD-affected dogs, assessments were made 2 weeks before (week –2), immediately before (week 0), and at 2, 4, and 6 weeks after initiation of treatment with acupuncture (diamonds), carprofen (squares), or placebo (triangles). The HD-free dogs (cross) were assessed only once (week 0). †At this time point, the value for the acupuncture-treated dogs differed significantly from the week –2 or week 0 value or both values. See Figure 1 for remainder of key.
Figure 3—Median scores derived by use of a VAS for pain completed by owners of the HD-affected and HD-free dogs in Figure 1. For the HD-affected dogs, assessments were made 2 weeks before (week –2), immediately before (week 0), and at 2, 4, and 6 weeks after initiation of treatment with acupuncture (diamonds), carprofen (squares), or placebo (triangles). The HD-free dogs (cross) were assessed only once (week 0). See Figure 1 and 2 for remainder of key.
The study employed three subjective measures (pain scales designated HCPI, CBPI, and VAS scales) and one objective measure (force plate analysis). There were also several subsidiary categories within most of these measures. These were compared within and across treatments at multiple points in time before, during, and after treatment, yielding a large number of potential outcomes and comparisons. The more opportunities researchers have to select some comparisons and ignore others, the more opportunity for bias to determine the conclusions. The strongest studies are those in which the outcomes are consistent and all show the same result. That was certainly not the case for this study.
According to the authors, “Outcomes of the 3 treatments did not differ significantly [and]…kinetic evaluation did not differ among the groups.” All groups showed variation and overall improvement, but those treated with acupuncture did no better than the other groups, including the placebo. A few comparisons showed a difference between acupuncture and placebo or carprofen treatment in selected comparisons between a few points in time, but these were the exceptions, and the majority of potential outcome measures failed to show any difference. Basically, the authors cherry picked a few comparisons they could make statistically significant and largely ignore the vast majority that were not. This is cheating, though it is sadly very, very common, and not just in alternative medicine research.
The authors used carprofen as a “positive control” and oral lactose capsules as a placebo in this study. Though it might be helpful to include a sham acupuncture treatment as well, these are reasonable choices for control groups, and blinding appeared appropriate. There is robust evidence carprofen and other NSAIDs are effective analgesics in dogs with osteoarthritis.1-2 It is surprising, then, that carprofen had no apparent effect beyond placebo in this study. This calls into question the accuracy of the outcome measures employed. If a proven arthritis therapy didn’t appear to work better than placebo, it is unclear why we should trust the other results of the study.
Given the apparent lack of efficacy of the positive control and the failure to identify differences between treatment groups in nearly all outcome measures, it seems the most appropriate conclusion is that this study does not provide evidence to support the use of acupuncture for dogs with hip dysplasia. The data are far more compatible with the hypothesis of no difference between treatment and placebo, and the improvement seen in all groups is likely the result of nonspecific clinical trial effects (e.g. the Hawthorne effect3) and natural variation over time.
Negative findings are just as important and useful as data that do show efficacy. This study makes a valuable contribution to our understanding of the potential role of acupuncture in treatment of hip dysplasia. But it is misleading to characterize the results as supportive of utilizing this acupuncture protocol to treat pain of lameness associated with this disorder. Undoubtedly, people will cite this as a positive study when advocating acupuncture for dogs, but it takes very little investigation to realize it suggests exactly the opposite.
Update 12/27/2016:
I wrote a letter to the editor of the Journal of the American Veterinary Medical Association (JAVMA) which presented a shorter version of my critique of this article. The authors’ response was disappointing, to say the least, and a classic example of cognitive dissonance, the inability to objectively consider evidence that challenges a strongly held belief.
Dr. McKenzie is correct that in our study1 of dogs with hip dysplasia treated with acupuncture, carprofen, or a placebo, results for our subjective (Helsinki Chronic Pain Index, Canine Brief Pain Inventory, and visual analogue scales for pain and degree of lameness) and objective (kinetic variables obtained with a pressure platform) outcome measures did not differ among groups at any time point. In addition, results of kinetic evaluations did not differ over time in any of the groups.
Nevertheless, we did detect some differences among groups. For instance, the percentage of dogs with at least a 30% improvement in CBPI score, compared with baseline score, was significantly (P = 0.034) higher for dogs treated with acupuncture (13/15) than for dogs treated with a placebo (7/16), although not significantly (P = 0.446) different from the percentage for dogs treated with carprofen (11/16). Only 2 dogs required rescue analgesia, and both were in the carprofen group. In addition, only dogs in the acupuncture treatment group were found to have significant improvements in CBPI scores (total score and pain severity and pain interference subscores) and the VAS score for pain at various times, compared with baseline scores, and only dogs in the acupuncture and carprofen groups had improvements in the VAS score for locomotion. Therefore, we do not agree that the statistical testing created “an impression of effect,” as suggested by Dr. McKenzie.
We are aware that carprofen and other NSAIDs have been reported to be effective analgesics in dogs with osteoarthritis, and we do not have an explanation for why dogs in the carprofen group in our study did not perform better than dogs in the placebo group. However, in one of the reviews supporting the efficacy of NSAIDs in the treatment of osteoarthritis mentioned by Dr McKenzie, Innes et al2 reported that none of the selected studies met all 3 of their criteria for inclusion of essential elements for evidence-based medicine. Therefore, the efficacy of NSAIDs should perhaps be questioned and we should possibly seek new strategies to treat osteoarthritis.
The authors have considerable experience in working with acupuncture and conventional medical approaches to treat dogs with chronic neurologic and musculoskeletal diseases, and our clinical impressions support the benefit of acupuncture. Therefore, we feel confident in our conclusion that acupuncture may be a viable option for improving the quality of life in dogs with hip dysplasia. However, we understand that conventional medicine, including NSAIDs, is very important as a multimodal approach to alleviate chronic pain and argue that we should use all possible therapeutic tools to treat our patients. [emphasis added]
The idea that the failure of the positive control (carprofen, an NSAID) might indicate that the most widely used and thoroughly studied class of drugs for arthritis pain is actually ineffective rather than that there is a serious flaw in this study is far-fetched, to say the least. And the final paragraph of this response illustrates the deep problem at the core of so much veterinary research, and in my opinion especially CAVM research; the use of science not to discover the truth but to buttress and provide marketing support for one’s existing beliefs. The fact that academic veterinarians and scientific researchers would openly admit that they believe a treatment is effective based on personal experience no matter what their own research evidence shows is a very bad sign for the health and usefulness of veterinary clinical research.
Reference
- Innes JF, Clayton J, Lascelles BD. Review of the safety and efficacy of long-term NSAID use in the treatment of canine osteoarthritis. Vet Rec. 2010 Feb 20;166(8):226-30.
- Aragon CL, Hofmeister EH, Budsberg SC. Systematic review of clinical trials of treatments for osteoarthritis in dogs. J Am Vet Med Assoc. 2007 Feb 15;230(4):514-21.
- McCambridge J, Witton J, Elbourne DR. Systematic review of the Hawthorne effect: New concepts are needed to study research participation effects. J Clin Epidemiol 2014 Mar; 67(3): 267–277.
If they had set this up as a non inferiority trial and left out the placebo group the results would look great! Odd that nsaid didn’t out perform placebo.
I went to our local vet society CE meeting last Thursday on emergency brain and spinal cord trauma. The last part of the lecture was where the helpful acupunture points were pointed out for treatment. It was a lecture given by two vets. One was boarded in internal medicine and taught neurology to students at the Florida vet school. The other vet was boarded in emergency medicine. The room was spinning. I’m required by law to listen to this CE bs. Your tax payer dollar is used to teach it to students.
Odd that nsaid didn’t out perform placebo.>>>>> There was a good BMJ article years ago that determined that Tylenol only worked short term for human arthritis. I will try to post the study.
This reminds me way too much of the currect (9/’18) plan in Oregon to ban prescription pain meds for all human chronic pain patients & send them to acupuncturists instead. The “task force” behind this plan consists entirely of “alternative therapy” types –including 3 of the 7 being acupuncturists (amazing coincidence!).
The following quote from your post applies perfectly to the travesty that’s going on in the human –& inhumane– treatment plan as well as veterinary medicine:
“Basically, the authors cherry picked a few comparisons they could make statistically significant and largely ignore the vast majority that were not. This is cheating, though it is sadly very, very common, and not just in alternative medicine research.” The primary difference is, they completely, not just “largely,” ignored the vast majority of the evidence for the humans.