A new review of systematic reviews of acupuncture treatment for pain was recently published in the journal Pain. Basically, a systematic review is a study which looks at primary clinical research studies selected in a comprehensive and pre-determined manner and with explicit standards for evaluation of the quality of the studies included. It is a way of taking a collection of research studies on a single subject and evaluating them objectively to determine what the balance of the evidence overall says about the subject. A systematic review of systematic reviews, then, uses the same objective and explicitly stated methodology to examine multiple systematic reviews on a single subject. This approach sounds pedantic, but it serves to minimize the influence of the reviewer’s bias on the conclusions.
An informal or narrative review, in which an author picks whichever studies they want and evaluates them informally, tends to simply conclude whatever the author originally expected the balance of the evidence to say. A systematic review makes it harder to intentionally or unintentionally select some studies and ignore others in a way that biases the conclusions, or to give greater credence to some studies over others based on one’s preconceptions rather than on objective criteria for quality. Thus, systematic reviews and systematic reviews of reviews, if properly conducted, are the most reliable level of evidence.
This review identified 57 systematic reviews that met the criteria for inclusion in the analysis. The authors were primarily interested in the effectiveness of acupuncture as a treatment for pain and the side effects of acupuncture treatment. The results were mixed. 25 reviews reported a tentatively or clearly positive result, meaning acupuncture appeared to be beneficial for the conditions studied. 32 reviews did not report a positive result, meaning acupuncture appeared no better than placebo for pain. Of the 4 reviews deemed excellent in quality, 2 found a significant benefit and 2 did not. When there were multiple reviews for the same condition, they frequently contradicted each other.
The exact significance of these results is not clear. It could be that acupuncture helps with pain for some conditions but not others, though neither the traditional mystical theories nor more modern neurophysiological theories of how acupuncture is supposed to work would predict or explain that. However, as the authors of the review point out, several recent high-quality research studies hold out some hope of an answer. In these studies, fake acupuncture (applying treatment at random locations, using needles that don’t penetrate the skin, or even using toothpicks instead of needles) works as well or even better than real acupuncture. And one study found that acupuncture had a significantly greater benefits if the acupuncturists deliberately encouraged high expectations of a benefit in patients than if they were neutral about the likely effects. These findings suggest that psychological variables, such as the beliefs and expectations of patients and the behavior of those performing the acupuncture, are responsible for much of the perceived improvement in pain. These variables are often not specifically or effectively controlled in clinical trials, which would lead to variations in the apparent benefit of acupuncture unrelated to whether the treatment itself is actually doing anything to the body of the patient.
Of course, as I have argued in the past, such improvement in perceived pain is certainly real from the point of view of the patient, regardless of how it is generated. In many studies, the benefit is minimal and may or may not be clinically significant. It is almost certainly less than that achieved by properly tested pain medications which, when they are tested, rarely show the inconsistency in effects seen with acupuncture. Still , if a subjective experience like pain seems better to a patient, it cannot be said it isn’t better, only that it may not be due to any objectively measurable change in the patient’s body.
This, of course, also raises serious questions about whether the same benefits, if they exist at all, would be seen in veterinary patients receiving acupuncture. The only parties in the therapeutic interaction that have explicit beliefs and expectations are the owners and those performing the treatment, so it may be that they are the only ones to perceive a benefit. Without objective measures of response, such as force-plate measurements, activity monitors, and so on, it is dangerous to assume our pets truly feel less pain just because it looks to us like acupuncture is working on them.
The review also looked at side effects from acupuncture, something proponents of the therapy often argue are negligible or non-existent. They did find a number of serious adverse events, sometimes fatal or permanently disabling. The most common were pneumothorax (air getting into the chest through a hole made by an acupuncture needle and collapsing the lungs) and infections. These adverse events were not common, and they generally resulted from improper technique, so it is reasonable to argue that proper training and oversight of acupuncturists would prevent them. But of course if the therapy itself is nothing more than a placebo, it is difficult to argue that any life-threatening side effects are justifiable.
So the best we can say, after extensive research over decades, is that acupuncture might relieve pain for some patients with some conditions. Side effects are uncommon if it is properly performed but can be serious, even deadly, if not. The authors conclusions were these:
[N]umerous systematic reviews have generated little truly convincing evidence that acupuncture is effective in reducing pain. Serious adverse events continue to be reported.
Ernst, E. Soo Lee, Myeong. Choi, Tae-Young. Acupuncture: Does it alleviate pain and are there serious risks? A review of reviews. Pain. 2011;152:7555-764.
Part of my dissertation looked at Ernst’s previous claims about the risks associated with acupuncture and especially pneumothorax and infection rates. He bases most of his claims on old data, mostly published outside the UK, from areas with poorer standards of training and hygiene. Since the advent of single use needles and improved training, the rates of reported adverse events in the UK is virtually nil, reflected in low insurance costs for acupuncture practitioners. He keeps telling the same old story, but nobody takes him seriously anymore.
Well, I would disagree with the broad characterization of “nobody takes him seriously anymore.” But his article did explicitly say that most of the serious side effects seemed to be associated with poor training and regulation, so I agree that the risks are generally quite low. That doesn’t change the fact that the efficacy for most conditions is still zero, and that those for which it seems to help are probably mostly psychological/placeb-based effects
Pingback: The Harm Complementary and Alternative Medicine Can Do | The SkeptVet Blog