I recently ran across an article in the journal Evidence-Based Complementary and Alternative Medicine (eCAM) titled Evaluating Complementary Therapies for Canine Osteoarthritis–Part II: A homeopathic Combination Preparation (Zeel) (Hielm-Bjorkman, A et al 2009:6(4)465-471).
According to the authors, “a homeopathic combination product (HCP) for canine osteoarthritic pain was evaluated in a randomized, double-controlled and double-blinded clinical trial…[and] that the HCP (Zeel) was beneficial in alleviating chronic orthopedic pain in dogs, although it was not as effective as Carprofen.”
There are many levels on which any clinical research article should be critically evaluated. The potential biases of the authors and the journal, the quality of the methodology, the statistical analysis of the data, and the degree to which the conclusions follow from the data are all common criteria by which such publications can be judged. R. Barker Bausell in his book Snake Oil Science does an outstanding job illustrating some reasons why not everything that makes its way into a scientific journal is reliable science and why such critical evaluation is necessary.
In this case, the journal makes some effort to follow the principles of evidence-based medicine, but it is guilty of some serious Tooth Fairy Science, in which rigorous methodology is applied to some fundamentally irrational premises. Skimming through some archival issues also indicates a pretty strong preference for publishing positive findings for CAM interventions. None of this automatically invalidates anything published, but it is one factor to consider since the effect of personal as well as financial biases on research outcomes is well established.
As for the authors, I am not able to establish much about their biases. Two of the authors are professors at the University of Helsinki School of Veterinary Medicine, and the lead author’s research summary suggests a strong attachment to CAM. Her doctoral dissertation was a study on gold implantation, green-lipped mussel extract, and Zeel for use in canine osteoarthritis, and it appears she is publishing this thesis research as a series of articles in eCAM.
The methodology is generally sound, with a couple of exceptions. First, while the product studied is identified as homeopathic and an inject able version of it is listed in the US Homeopathic Pharmacopoeia, even the authors insert the caveat that “this is not a classical homeopathic treatment.” The preparation contains 14 listed ingredients, many of which are present after having been diluted 1:10 only 2-8 times, for “molar concentrations of 10-5 to 10-12mol/L.” Such concentrations are low, but still higher than the usual case for homeopathic preparations, which cannot conceivable contain any of the original ingredient. It is possible, then, that this substance could contain some pharmacologically active substances. The ingredients listed (same as the injectable product) are:
- Arnica montana, radix (mountain arnica)
Rhus toxicodendron (poison oak)
Sanguinaria canadensis (blood root)
Symphytum officinale (comfrey)
- Mineral ingredients:
(alpha)-Lipoicum acid (thioctic acid)
Coenzyme A (coenzyme A)
Nadidum (nicotinamide adenine dinucleotide)
Natrum oxalaceticum (sodium oxalacetate)
- Animal-derived ingredients
Cartilago suis (porcine cartilage)
Embryo totalis suis (porcine embryo)
Funiculus umbilicalis suis (porcine umbilical cord)
Placenta suis (porcine placenta)
The subjects were appropriately randomized into treatment, placebo, and positive control groups, with Carprofen as the positive control. The subjects in each group all appear to be comparable at baseline. The placebo group did have higher baseline scores on 5/7 measures of pain, but the authors state that no statistically significant differences were found between groups at this point.
The placebo control was not ideal. The treatment product was visibly different from the Carprofen and the placebo (which were identical to each other). The owners were given extra Carprofen in its original packaging for rescue, so clearly they would be able to identify the treatment product as different. In addition, all subjects also received an inert capsule as part of a separate study, so while the Zeel group received “an ampoule of clear liquid” once daily and “a slightly green (lactose) capsule,” the Carprofen and placebo groups received the green capsule and “a white pill” twice daily. It is not clear what if any affect such a discrepancy might have had on the subjective assessments of owners, or potentially blinded investigators who might have detected group assignment from comments made by owners.
Most of the assessment measures were subjective, such as owner rating scales or visual analog pain scores and investigator clinical exam assessment. Some force plate analysis was done, though this proved problematic. Two subjects had to have their force plate measurements discarded because they were too lame to allow accurate measurement. These subjects, however, were all in the placebo arm and so this would be expected to have the effect of decreasing perceived efficacy of the treatment.
The force plate measurements were “repeated until sufficient valid results were obtained for both left and right limbs.” It was not stated whether the number of trials needed to achieve this differed between groups, which could have affected the results is some subjects had to run back and forth significant more than others to get a valid reading, which might itself affect the reading.
The biggest methodological problem I see in the study is in order to calculate the percent of subjects improved or not improved in each group at the 8-week assessment period, “the results of each variable were converted into dichotomous responses of ‘improved’ or ‘not improved.'” Converting scale variables into dichotomous variables can exaggerate differences between groups. If the measurement was unchanged, the subject would be classified as “not improved,” but if there was even a miniscule change from baseline then the subject would fall into either “improved” or not “improved category.” Thus, subjects with dramatic improvements in scale measurements would be weighted the same as subjects with marginal, and likely clinically insignificant changes in the variable. Without the raw data, of course, it is impossible to tell what if any effect this procedure might have had on the final conclusions. However, the tabulation of the data presented in the article appears to show much greater improvement in terms of the percentage of subjects improved than in terms of the actual median improvement of the variables themselves, suggesting that in fact such an exaggeration occurred.
The authors also stated that “for dogs that had used extra Carprofen more than three times per week at W8 [4 dogs in the placebo arm] we changed all their variable values at evaluation W8 into the most negative value measured at that time.…to counteract the effect of the NSAID…” This seems a clear fudging of the data which made the placebo group appear to have worse outcome measures than it actually did. Certainly, it is possible that these dogs needed more Carprofen than the treatment group because the treatment was having a beneficial effect. But it is just as possible that the placebo group took more Carprofen because of differences in disease severity, in owner attitudes or behavior, or some other factor. And what makes the arbitrary designation of three times a week an appropriate justification for altering the data in this way is unclear. In any case, the effect of this decision is to make the outcomes appear worse for the placebo group, which clearly makes the treatment group outcomes appear relatively better.
Patients given Carprofen clearly showed improvement over baseline at a rate significantly higher than placebo. 67-86% of subjects were categorized as “improved” for the various outcome measures, and the actual values for each measure were improved from 2-5 times more than for the Zeel group. In the treatment group significantly more subjects were classed as “improved” compared to placebo in 3 out of 6 measures. Again, this is likely inflated by the conversion of scale data to dichotomous data. The Zeel group also showed significantly greater improvement than the placebo group in 4 out of 6 specific measures, though for one the P value was 0.049, quite close to the cutoff of 0.05.
The authors also state that use of supplemental or rescue Carprofen occurred in 14% of the Carprofen group, 28% of the Zeel group, and 8% of the placebo group. Though they claim that the only significant difference was between the Carprofen and placebo groups, this is puzzling, both because the Zeel group had a percent of rescue use dramatically higher than the other groups, and because of the earlier statement about manipulating the data for the placebo group to “counteract” the effect of Carprofen use for that group.
No differences in bloodwork values or clinical side effects were seen between groups.
The authors also make the unsupported statement that “it is generally accepted that seasonal differences influence OA, with patients being worse in cold, damp and unstable weather.” A number of studies have found this traditionally assumed relationship to be difficult to verify and likely a minor and insignificant factor in arthritis pain for most patients (1, 2, 3), so it does not qualify as “generally accepted.” Nevertheless, the authors go on to claim that a trend observed of worsening symptoms for the placebo group during the treatment phase of the study and subsequent improvement during the post-treatment follow-up was due to the weather, and that the opposite trend seen in the Zeel and Carprofen groups was due to the effects of the treatment agents. It seems more likely that the placebo group simply different in significant ways from the other groups, which casts further doubt on the conclusion that the test product was of meaningful benefit.
The authors conclude by putting the usual best possible spin on the weak results, suggesting that combined with in vitro results reported elsewhere they justify further research and pointing out that NSAIDS, which even they acknowledge are clearly superior for treatment of pain, have side effects, despite the fact that none were seen in this study. As I’ve said before, the resource limitations on research in veterinary medicine requires the most efficient use of those resources to maximize benefit, and such studies of implausible interventions are not going to benefit our patients. The authors clearly wish to find something positive in their results, but the study does not justify the commitment of more time and resources and talent to this methodology when better therapies are already available, and when decades of research on homeopathic preparations has failed to validate them. Such papers provide the aura of scientific legitimacy to such methods, but they are tooth fairy science, not evidence-based medicine, and they are a dead end we would do well to stop travelling down.
Thanks for this dissection – I’m sure this product will soon find itself being recommended on the animal forums, and apart from the weak findings pointing to people saving their money on Zeel, the list of animal-derived ingredients should give the good folks some food for thought. So many people think homeopathy is exclusively plant based and cuddly, it’ll do them good to see what’s really going on. I wonder of they’ve tried to cure pigs with dog ingredients? Very strange this prioritising of species……….
Yes, it’s always fun to see what sort of things homeopaths use as “remedies.” Obviously, the gross factor doesn’t necessarily mean anything, but since they like to use it to indict the ingredients in commercial pet foods, it’s nice to demonstrate that their own nostrums are no better.
And the sympathetic magic aspect of homeopathy shows clearly in the choice of ingredients. Rhino-horn-for-virility sort of logic.
It is definitely interesting to see what the ingredients in homeopathic preparations are, even when diluted out of existence. I suspect many people would recoil if they used common names for the ingredients instead of the Latin the homeopaths are so fond of. The same could probably be said of many Chinese “herbal” remedies that sometimes contain minerals, heavy metals, and animal products. Those names sure sound exotic and mysterious, but are not quite as exciting and may even be off-putting when translated. Perhaps providing translations of ingredient names would be a useful resource for those looking into these types of remedies. even when diluted to 30C, I bet a lot of people would object to the use of some ingredients during production.
I do not know if Zeel will work on canines, but I DO KNOW that it works on humans. I am out of pain for the first time in six months due to Zeel. Since thousands of women have taken estrogen and progesterone for years (made from horse urine) I do not balk at ingredients obtained from pigs. It is well known that pig hearts are very similar to human hearts and research is being done to use pig hearts in humans. I have never used a product that gave me such relief in such a short amount of time.
Thanks for your comment. Unfortunately, for a lot of reasons, while your personal experience is deeply compelling to you, and while I’m always glad when someone gets relief from discomfort, that experience is not a reliable guide to whether the therapy actually works. The reason we need scientific evidence, and the reason human and animal health and life expectancy has improved so dramatically since we began to use it is that we are all too easily fooled into believing things help which do nothing or even harm us. Below are some suggestions for reaing on the subject:
Why bogus therapies often seem to work
Don’t Believe Everything You Think: The 6 Basic Mistakes We Make in Thinking by Thomas Kida
Pingback: Homeopathy, Unethical Quackery « Skeptical Vegan
We just started carrying this stuff at our clinic. There is a Disclaimer on the Website (after multiple claims how well it works and how vets reccomend it and stuff):
These statements have not been evaluated by the FDA. These products are not intended to diagnose, treat, cure, or prevent any disease.
The packaging doesnt have this disclaimer (though i didnt open it and look at the insert). does the FDA make all homeopathic stuff use this disclaimer, i didnt think the FDA regulated claims on websites? anyone know
Th details of how homeopathic remedies and supplements are regulated are somewhat complicated and bizarre. Here is an article I wrote with the details.
In short, homeopathy was grandfathered into the 1938 law establishing the FDA, since it was promoted by a senator who was also a homeopath. Basically, homeopathic remedies can be claimed to treat disease without having to prove safety and efficacy in advance as pharmaceuticals are required to do. However, the FDA does not endorse their use, and it is technically not legal to use them in animals since the original law only refered to human use. A conventional FDA approval process, complete with clinical trials, would technically be needed for legal veterinary use. However, the FDA has informally agreed not to enforce this rule since most veterinary use of conventional medicines are off label anyway.
This product is not actually a classical homeopathic fremedy, so it is not covered by the FDA rules that cover homeopathy. Instead, it is considerred a supplement and is regulated under the Dietary Supplement and Health education Act (DSHEA) of 1994. This allows marketing of dietary supplements without pre-market clinical trials showing safety and efficacy so long as they don’t claim to treat or cure any disease. Supplements can only be claimed to “support the structure and function” of organ systems. This is, of course, a meaningless claim since air, water, and everything else necesary to life “supports” the structure and function of the body, and many medicines which actually do treat or cure diseases do not. In addition, supplements are required to carry the Quack Miranda Warning that you quoted above, stating that the FDA has in no way examined any claims that are made to see if they are true.
The FDA does regulate medical claims, and there is also some regulation from the FTC, which is in charge of ensuring advertising is truthful. However, websites are so easy to set up and to move, and the resources for enforcement are so scarse, rarely are such claims challenged unless someone gets hurt or complaints are made by a large number of people or someone with influence.
Skeptvet said: This product is not actually a classical homeopathic fremedy, so it is not covered by the FDA rules that cover homeopathy.
I believe you’ve just coined a perfectly useful term, skeptvet. “Fremedy”. I like it. Fraudulent remedy: fremedy.
Sorry, couldn’t resist a friendly jab at your spelling.