Evidence Update: Still No Good Evidence to Support Yunnan Baiyao

I originally wrote about the Chinese herbal product Yunnan Baiyao in 2010, concluding at that time that the evidence supporting its use to reduce bleeding was poor and largely anecdotal:

So we have a treatment with uncertain and unregulated ingredients, no demonstrated plausible mechanism of action, a few in vitro and lab animal studies in journals of questionable reliability, a few small human clinical trials in similarly questionable sources, and two very small veterinary trials in equids which found no effect except possibly on a poorly reliable and subjective test of blood clotting.

I reviewed the literature again in 2016, finding a few new studies which did nothing to strengthen the case for this product:

Despite some suggestive in vitro and low-quality studies, the best evidence available so far does not support that Yunnan Baiyao has any benefit for dogs. The fact that it is unregulated, that there is demonstrated inconsistency in the mineral and metal contents of Yunnan Baiyao from different sources, and that the ingredients are still kept secret by the manufacturers, should also give clinicians pause in considering this for their patients.

Though it hasn’t been very long since my recent review, I ran across another study looking for possible effects of Yunnan Baiyao on one aspect of blood clotting, platelet function and the Buccal Mucosal Bleeding Time (BMBT), which involves measuring the time for bleeding to stop after a standardized cut is made in the gums. Like most of the studies in veterinary patients so far, this one found no reason to believe Yunnan Baiyao can stop hemorrhage.

Frederick J, Boysen S, Wagg C, Chalhoub S. The effects of oral administration of Yunnan Baiyao on blood coagulation in beagle dogs as measured by kaolin-activated thromboelastography and buccal mucosal bleeding times. Can J Vet Res. 2017 Jan;81(1):41-45.

In a randomized controlled crossover trial 8 beagle dogs were given either placebo or 1000 mg of YB orally every 12 h for 5 consecutive treatments. Blood was drawn 24 h before treatment and 2 and 24 h after the last treatment, and the BMBT was measured in each sample in duplicate…There were no adverse effects of treatment and no significant differences between the control and treatment BMBTs or [platelet function] parameters at any time point….Thus, at the dose and frequency of administration in this study YB did not appear to have any clinically significant effects on the measured coagulation parameters.

As I always take care to emphasize, no single study is sufficient evidence to definitively answer most medical questions. But despite the anecdotes that seem so persuasive to many owners and veterinarians, the gradual accumulation of evidence is not encouraging. And though no obvious harm has yet been found, the fact remains that the ingredients in this remedy are not standardized or tested for quality of contamination, and since this would certainly be considered unacceptable and illegal for any conventional medicine, it is hard to justify using it when the chances of significant benefits appear to be quite low.

Posted in Herbs and Supplements | 1 Comment

Evidence Update: Finally a Clinical Trial of Cranberry Supplements for Urinary Tract Infections in Dogs

One of the recurring subjects I have covered on this blog is the contention that cranberry supplements can help prevent or treat urinary tract infections. Each time I address the subject, the evidence points to pretty much the same general conclusion:

March, 2012

There is weak theoretical justification for using cranberry products for UTIs, though none of the supporting preclinical evidence involves dogs or cats. There is conflicting clinical trial evidence in humans, and no clinical studies in dogs and cats.

October, 2012

This [Cochrane] review indicates pretty clearly that overall, cranberry juice is not effective in preventing UTIs despite theoretical reasons why it might be. This illustrates, yet again, why we cannot rely on extrapolation from pre-clinical or in vitro studies to tell us what will work in actual patients.

April, 2016

The in vitro portion of this study is consistent with existing research that suggests cranberry extracts may reduce the ability of some bacteria to stick to the lining of the urinary tract. This could theoretically help prevent some urinary tract infections, though clinical research in human patients suggests this doesn’t really work to a significant extent in living people.

The portion of the study looking at prevention of UTIs in actual dogs, unfortunately, doesn’t help establish what benefit, if any, this product might have.

So basically, while there is a plausible argument that cranberry supplements might be helpful based on lab research, clinical studies don’t seem to show they actually do much in real human patients, and there isn’t any real research in veterinary patients. Fortunately, the last part of that conclusion is now less applicable, thanks to a newly published clinical trial in dogs. Unfortunately, the results of the trial tend to support the existing human literature which has failed to find any real-world benefits.

N.J. Olby, S.L. Vaden, K. Williams, et al. Effect of Cranberry Extract on the Frequency of Bacteriuria in Dogs with Acute Thoracolumbar Disk Herniation: A Randomized Controlled Clinical Trial. J Vet Intern Med 2017;31:60–68.

This study specifically evaluated cranberry extract as a method for preventing bacterial urinary tract infections in dogs with newly acquired neurologic dysfunction due to herniated intervertebral disks damaging their spinal cords. Dogs with this condition are predisposed to such infections, and it was hoped the supplement would reduce this risk. The supplement used, Crananadin, was the same I reviewed in my original post on this subject.

The study employed excellent methodology, with appropriate randomization, blinding, placebo control, and pre-determined objective outcome measures. The results were clear and consistent. The cranberry group actually had a higher rate of infections than the placebo group, though the difference was not statistically significant. And though pre-clinical evidence suggests cranberry supplements might be most likely to be useful with E. coli infections, there was no statistical difference between the groups in the rate of this type of infection, with most occurring in the cranberry group. The study was ultimately smaller than the investigators initially intended because an interim analysis clearly showed no trend towards any benefit and very little chance that a benefit would be seen even if more animals were enrolled.

While no single study should be the final word on any complex medical question, this trial is about as strong a clear a negative finding as one can get. And given the similar failure to find significant benefits in many human trials, it adds to a growing body of evidence suggesting that cranberry supplements are not useful in preventing or treating urinary tract infections. While additional studies may help clarify the issue, especially those involving different types of supplement and different patient populations, at some point consistent failure to find a benefit has to be seen as a reason to question the wisdom of expending scarce research resources on studying the question. We aren’t, in my view, quite at that point yet, but at least clinicians should be clear with pet owners that the evidence does not look good for cranberry supplements, and these should not be relied on in lieu of effective monitoring and medical treatment.

Bottom Line
Despite some promising laboratory studies suggesting cranberry supplements might help prevent or treat urinary tract infections, the evidence of studies in clinical patients has been disappointing. Conflicting studies in humans suggest, on balance, that there is probably no significant benefit. And now a high-quality clinical trial in dogs has failed to find any effect, even in the the of infections the pre-clinical research most strongly suggested there should be one.

While the risks of cranberry supplements are probably negligible, pet owners should understand, and veterinarians should make in clear to their clients, that there is no good reason to believe they have any real value in preventing or treating urinary tract infections.

Posted in Herbs and Supplements | 2 Comments

Music Therapy for Dogs: Does Your Dog Really Prefer Reggae to Pop?

Playing music has become a popular method for reducing stress in animals who are hospitalized or kenneled. A couple of years ago, I reported on a review of the literature investigating this practice. The conclusion of that review was this:

The studies described above support the idea that physiological and neurochemical changes can result from music exposure. Music’s ability to improve learning and memory and diminish stress suggests that music can result in positive physiologic effects. Other studies, however, suggest that music has no observable effect or can have a detrimental effect on animals. There is not yet sufficient evidence to suggest that animals react physically to music in a way that can be manipulated predictably and consistently.

Since then, a few new studies have been published, and I want to briefly review two of them since once has been making headlines recently.

Well, not exactly….

The first study was published in 2015.

Bowman, F.J. Dowell, N.P. Evans. ‘Four Seasons’ in an animal rescue centre; classical music reduces environmental stress in kennelled dogs. Physiology & Behavior, Volume 143, 1 May 2015, Pages 70-82

In this experiment, 50 dogs at two rescue shelters were assigned to be monitored during a period with no auditory stimulus and a period in which classical music was played. Heart rate monitors were used to measure heart rate variability (HRV), salivary cortisol levels were measured, and behaviors, such as time spent standing or lying down and barking, were observed. The hypothesis was that these proposed measures of stress would improve during the time the classical music was being played.

The results were mixed. The cortisol levels were too variable to show any association with the music. Cortisol has long been used as a measure of stress, but it is highly problematic and often cannot be convincingly shown to truly represent the level of stress an animal is experiencing at a given time. HRV showed some changes that might be consistent with a reduction in stress, though the exact significance of this measure in the well-being on dogs hasn’t been definitively shown either. And finally, the behavioral measures suggested the music reduced possible signs of stress. During the music condition, dogs appeared to lie down more and bark less.

There were a number of limitations to this study. While two groups of dogs experienced the music and silent conditions in different orders, it is not clear how the animals were selected for the study in the first place. And the authors acknowledge that not all animals were available for all observation times or finished the study, but no details on this are reported. And it is not stated whether observers were blinded to the treatment, which is key when using behavioral observations as a measure of effect. Finally, the study appeared to show that the effects of the music treatment that were observed only lasted as little as one day, and that continuing to play the same music beyond this time had no effect.

Overall, this study adds a bit of evidence to suggest that classical music might have some effects on dog behavior. However, while some findings are consistent with other studies, some are not. And the significance of some of the measures used and the changes seen are not clear.

The second study was done by the same group in the same facilities on a smaller group of dogs (38).

Bowman, F.J. Dowell, N.P. Evans. The effect of different genres of music on the stress levels of kennelled dogs. Physiology & Behavior, Volume 171, 15 March 2017, Pages 207-215

The methods used were very similar. However, the major difference was that a period of observation when no music was played both preceded and followed the period of music exposure. In addition, the genre of music varied each day during the music exposure period. Also, cortisol levels were measured in urine rather than saliva, which it was hoped would reduce the individual variability a bit. Otherwise, the circumstances and the limitations were similar to the previous study.

The results again showed changes in HRV that were suggestive of a reduction in stress during the music condition. Some behavioral changes were also observed that matched those seen in the previous study (an increase in time spent lying down, for example), but other behavior changes seen previously were not seen in this study (for example, there was no reduction in barking during the music condition). Finally, the cortisol levels showed a changed that was the reverse of what would be expected, showing an increase during the music condition as would be expected with an increase in stress. There were some small differences in HRV and behavior between different genres of music, but there was no clear, large, consistent evidence that one genre is better than another.

This figure illustrates how really small the differences between silence and music, and between different genres of music, were in this study. While some of these differences were statistically significant, that does not automatically mean they were clinically meaningful.

These mixed results illustrate the general level of uncertainty regarding the effects of music on stress in dogs. While they can be interpreted to suggest a benefit (as the authors do), the size of the effect is small and the pattern of changes varies enough from study to study that it is hard to be certain there is really a consistent effect. And since several of the measures used in these studies may or may not actually reflect stress experienced by the dog (such as HRV and cortisol levels), changes in those measures may not reflect real improvement.

This raises the bigger question of how do dogs really benefit from the kinds of effects seen in these studies? Behavioral and physiologic measures suggest they might be showing less of a stress response, and presumably that implies they feel less anxious or fearful, but of course it is really difficult to confidently judge how an animal feels from such proxy measures. And is a small reduction in stress for a few days a meaningful improvement in welfare? Would it be more worthwhile to focus on interventions that have measurable impact on health, adoptability and the behavior problems that keep pets from finding lifelong homes, etc?

Finally, there is the question of whether music does any harm to these dogs. Certainly, we’ve all been trapped listening to someone else’s music we couldn’t stand, and it can be a miserable experience. Before we force dogs similarly trapped in kennels or hospitals to listen to music, we want to be certain we aren’t making their experience worse. Unfortunately, these studies don’t appear to look specifically for negative effects. The fact that cortisol levels in the second study changed in a way you would predict with an increase in stress was dismissed as insignificant, but we should at least consider the possibility it represented the reaction of the dogs to the music more accurately than the more subjective behavioral measures. I tend to agree with the authors that cortisol levels are notoriously unreliable markers of stress, but if we choose to use them in studies like this, we can’t simply choose to interpret the numbers in the way most congenial to our pre-existing hypotheses.

Bottom Line
These new studies at a little bit of evidence to that covered in the review I wrote about previously. In general, they do suggest that some kinds of music, such as classical and “soft rock,” might help reduce arousal, stress, and anxiety in dogs in a rescue shelter. Overall, however, the evidence for this hypothesis is still weak.

There is little evidence of any harm from most music interventions, though unfortunately this is not always something investigators specifically look for.

On balance, then, I think it is possible that music might have some benefits for dogs in stressful circumstances, such as boarding kennels, hospitals, and shelters, but this is by no means clearly proven. The risk is also likely quite low, so there is probably little harm in using quiet music for this purpose so long as it is not substituted for other, more comprehensive approaches to reducing stress and anxiety.







Posted in General | 4 Comments

Evidence Update- New Review of Medical Marijuana Research

I have addressed the question of medical marijuana and the use of other cannabis-derived medicinal products twice before. My conclusion in 2013 was this:

The current research evidence supports a couple of uses in humans, including treatment of nausea and poor appetite and possibly pain. Most other uses are poorly supported by clinical research. And there are unquestionably side effects that make marijuana often less useful than isolated cannabinoids or other unrelated treatments.

There is virtually no useful research evidence in companion animals, so any use of cannabis products is based entirely on theory and extrapolation from the limited research results in humans. Canna-Pet as a specific product, is being marketed with very dramatic and aggressive claims about safety and efficacy that do not appear to be supported by specific research on the product but, again, are based entirely on theory and anecdote, both notoriously unreliable sources of evidence.

There are recognized behavioral and medical risks associated with marijuana use in humans. While the behavioral risks do not apply to use in companion animals, and the medical issues associated with THC do not apply to products with negligible amounts of this compound, the risks of cannabis-derived compounds in dogs and cats are largely unknown. Any use of such products, then, should be undertaken with a clear understanding of the high levels of uncertainty about the results, and claims should not be made for these products that go beyond the available evidence.

When I looked again in 2016, I found no further research in veterinary patients, and the basic bottom line had not changed. We often must extrapolate from research in humans to clinical practice in veterinary medicine because we often don’t have the research we need in our own patients to guide us. This is necessary, but it is also risky. Humans are not dogs or cats, and while there are many similarities in basic physiology and in disease and response to treatment, there are also critical differences. The ibuprofen or sugarless gum that is safe for us will easily kill our pets. And the problems with blood lipids and cardiovascular disease that are a major source of human suffering and death are virtually non-existent in our canine and feline companions. So we must also be aware of the risks of leaping from evidence in humans to evidence in veterinary patients even when we have no choice but to make such a leap.

In the case of marijuana and other cannabis-derived products, there is still effectively no research on the risks and benefits in companion animals. Unfortunately, this has not stopped an explosion of marketing of cannabis products to pet owners, with no testing or regulation to ensure safety and efficacy. This is a dangerous situation. The evidence of risks and benefits in humans can help us to some extent to guess at the effects in companion animals, though unfortunately we cannot know how accurate these guesses will be without doing the careful work of rigorously studying these products in the actual species in which we are thinking about using them.

As for the human evidence, on that front there is some good news. The stigma that has hampered research for so long is waning, which is beginning to open the doors to researchers investigating the real effects of the many compounds found in cannabis. And a new review has just been produced which clearly and comprehensively summarizes the existing evidence, making it a lot easy to see the potential risks and benefits that we should focus on investigating in dogs and cats.

The National Academies of Sciences, Engineering, and Medicine. The Health Effects of Cannabis and Cannabinoids: The Current State of Evidence and Recommendations for Research (2017)

This review does the valuable work of comprehensively searching the research literature and then grading the strength of evidence for specific uses. Here is the grading scale the report uses:

Conclusive Evidence
For this level of evidence, there are many supportive findings from good-quality studies with no credible opposing findings. A firm conclusion can be made, and the limitations to the evidence, including chance, bias, and confounding factors, can be ruled out with reasonable confidence.

Substantial Evidence
For this level of evidence, there are several supportive findings from good-quality studies with very few or no credible opposing findings. A firm conclusion can be made, but minor limitations, including chance, bias, and confounding factors cannot be ruled out with reasonable confidence.

Moderate Evidence
For this level of evidence, there are several supportive findings from good- to fair-quality
studies with very few or no credible opposing findings. A general conclusion can be made, but limitations, including chance, bias, and confounding factors cannot be ruled out with reasonable confidence.

Limited Evidence
For this level of evidence, there are supportive findings from good- to fair-quality studies with most favoring one conclusion. A conclusion can be made, but there is significant uncertainty due to chance, bias, and confounding factors.

No or Insufficient Evidence
For this level of evidence, there are mixed findings, a single poor study or health endpoint has not been studied at all. No conclusion can be made because of substantial uncertainty due to chance, bias, and confounding factors.

This is a reasonable and fairly user-friendly scheme for categorizing the strength of the research evidence for or against specific proposed risks and benefits. The summary of the study lists the specific risks or benefits that were reviewed and classified according to this scheme. The benefits and risks are divided into several categories. There are too many to list here, but I will highlight those for which the evidence is at moderate or better and which might be relevant to veterinary patients (meaning, I will skip uses of cannabis for diseases not seen in dogs and cats or risks, such as auto accidents or risks from smoking marijuana, which obviously don’t apply to these species.)

Therapeutic Benefits:
Treatment of chronic pain in adults (cannabis)
Treatment of chemotherapy-induced nausea and vomiting (oral cannabinoids)

Lower birth weight with maternal use
Impairment of cognition, including learning, memory and attention with acute use
Variable increase in risk of development or in symptomatic worsening of various psychiatric disorders (relevance to behavior problems in veterinary patients?)

That’s pretty much it. There are plenty of other risks and benefits with at least moderate evidence that appear only to be relevant to humans. And there are many risks and benefits which could be relevant to veterinary patients but for which the evidence in humans is weak or insufficient to draw any conclusions. And, as I keep pointing out, there is virtually no evidence for any of the many different products out there that directly evaluates risks and benefits of those products in dogs and cats.

So right now, we are at a place where the hype and the marketing far exceed the real evidence that cannabis-based products are safe and useful for our pets. Using them could be worthwhile, but currently it is essentially rolling the dice, an uncontrolled individual experiment that could also make the patient’s life worse. Hopefully, further research will elucidate the real harms and benefits for our pets so we can make sound decisions about the appropriate role of cannabis-based products in veterinary care.




Posted in Herbs and Supplements | 9 Comments

Evidence Update: Does Dry Food Increase the Risk of Diabetes in Cats?

A perennial question in veterinary nutrition is the relative pros and cons of dry commercial diets. In particular, pet owners and vets alike have a lot of questions about the nutritional value and potential risks of dry food for cats. The argument is often made by proponents of alternative diets that, as obligate carnivores, cat cannot utilize the carbohydrates in commercial diets and, in fact, these contribute to disease. There is no question that cats are carnivores, and it is reasonable to suppose plant foods may not be nutritionally ideal for them. However, the naturalistic fallacy, that says whatever happens in nature is somehow perfect and ideal, is also at play in this issue. And there is evidence that properly prepared plant foods can contribute to a healthy diet for cats, though it seems that strictly vegetarian diets are unlikely to meet all of a cat’s nutritional needs.

As for the role of dry commercial foods in disease, there is very little evidence to go on, despite the widespread, often inflexible opinions many people have on the subject. About five years ago, I reported on an expert consensus document put out by the American College of Veterinary internal Medicine (ACVIM), which addressed this question. Here were the conclusions of that group of experts:

  1. Are dietary carbohydrates an essential or required nutrient for cats?
    Answer- No. Based on a good quality and quantity of evidence, most cats do not require dietary carbohydrates. There are some simple sugars in feline milk, so it is possible that nursing kittens may require these but no clear deficiency has been demonstrated.
  2. Can cats effectively utilize dietary carbohydrates for energy and nutrition?
    Answer- Yes. Based on a good quality and quantity of evidence, cats can effectively digest, absorb, and utilize dietary carbohydrates.
  3. Do dietary carbohydrates in the diet cause obesity?
    Answer- No. The cause of obesity in almost all cats is excessive calorie intake irrespective of whether the calories come from protein, fat, or carbohydrate. In fact, low carbohydrate foods may be more likely to lead to obesity if they are higher in fat than regular diets.
  4. Do dietary carbohydrates contribute to the development of diabetes?
    Answer- The consensus was that they do not, however the research evidence is very limited and not always consistent. The consensus was that even if carbohydrates do play a role as a risk factor for diabetes, this is dwarfed by the much more important factor of obesity.
  5. Are low-carbohydrate diets useful in the management of feline diabetes?
    Answer-Maybe. The evidence is limited and conflicting, and the committee did not achieve a consensus.

I have generally encouraged clients to feed their cats canned rather than dry food when possible because there is at least some reasonable theoretical argument for doing so. However, in the absence of strong evidence, I do not make a crusade out of this as some vets do.

Recently, a new study was published addressing one aspect of this issue, the potential association of diet with the occurrence of diabetes in cats. The theory is sometimes advanced that the carbohydrates in dry diets encourage obesity and create stress on the pancreas, the organ which produces insulin, and that this increases the risk of diabetes for cats eating such diets. Again, there has not so far been much actual evidence that this theory is correct, but the new study does contribute some. As always, the study has strengths and weaknesses which influence how much weight we should give the results and conclusions.

M. € Ohlund, A. Egenvall, T. Fall, H. Hansson-Hamlin, H. R€ocklinsberg, and B.S. Holst. Environmental Risk Factors for Diabetes Mellitus in Cats. Journal of Veterinary Internal Medicine. online early view January, 2017

The study was a case-control design. This means that a database of cats (in this case, those covered by a Swedish pet insurance program) was analyzed to identify cats with diabetes (cases) and those that were as similar as possible in other respects but did not develop diabetes (controls), and any available lifestyle information was compared to see what occurred more often in the cases than the controls, and vice versa. This can identify factors associated with the occurrence of diabetes.

It is important to recognize up front that such studies cannot establish causal relationships, and they are subject to a variety of sources of error, some of which I will discuss relevant to this particular study. They are a useful tool, however, for identifying potential risk factors for disease.

The first issue is the population of cats involved. Cats in Sweden who are covered by health insurance could differ in a variety of relevant ways from other populations of cats. Any associations found for such cats may or may not apply to, say, indoor cats in San Francisco or barn cats in Texas.

The other major concern is that the data consisted entirely of owners’ answers to a questionnaire, not medical records. That means that the study relied on owners correctly remembering medications their cats had been given, characterizing their diet history and eating habits, and reporting other lifestyle details that were then analyzed for associations with the occurrence of diabetes. Studies of human patients and their caregivers and family members show that our memories of such things are often highly unreliable and influenced by the very factors such studies are trying to evaluate. Owners whose cats have developed diabetes are likely to remember their cats’ medical and dietary histories differently than those whose cats don’t have this disease, and this can influence the associations seen in the data.

The authors are, of course, aware of and openly address these limitations:

Limitations of our study are mainly related to the study design, especially the problems with dietary recall bias and the difficulties for owners to accurately assess their cat’s body condition. Owners of diabetic cats can be more prone to remember events preceding the cat’s diagnosis of DM [diabetes] because of a recall bias, causing a type I error. They can also have more knowledge about DM and be more aware of the risk factors for disease, which might influence their responses. For example, owners of diabetic cats are more likely to be aware of their cat’s body condition, and also of the dangers of obesity. Moreover, owners of diabetic cats are asked about circumstances the year preceding their pet’s diagnosis of DM, which can be several years back in time. This differs from the control group, whose replies refer to the last year in the cat’s life, in general more recently.

Such limitations do not, of course, invalidate the research. All research has some sources of error that cannot be perfectly controlled. This is why consistency in findings across studies by different investigators and using different designs is crucial to arriving at an accurate understanding. However, we always need to incorporate an awareness of these sources of error into our interpretation of research studies and try not to blindly rely on the results of any one.

And what were the results of this study? A number of factors were associated with either a greater or lesser risk of diabetes. Breed and sex and previous use of corticosteroid medications were factors that have been associated with diabetes in other studies and were in this research as well. With regard to medication, however, no details about type of steroid, dose, or frequency were available, and the association relied entirely on owner memory, so this is not a particularly useful finding in terms of guiding the use of such medications.

There were some associations with diet and diabetes, but these were complex and not entirely clear. Partly, this is because obesity is a well-established risk factor for diabetes, and the association between type of diet and occurrence of diabetes seems to vary with body weight (again, with both diet and body weight determines solely by owner memory and judgment). Here is one example from the paper.

The only association that is statistically significant (the confidence interval does not include 1) is the comparison of dry food and wet food only in cats of normal weight. A couple of others are close, but it is impossible to tell if these would achieve statistical significance under different circumstances. The more important question, however, is do the pattern of results make sense? If dry food increases the risk of diabetes, shouldn’t this be the case for cats of all body weights? Of is the role of obesity so great that it swamps any effect of diet, so diet only shows up as a risk factor in cats of normal weight? The authors are very reasonable and circumspect in how they address these findings, and they are clearly aware that the implications of these results need to be sorted out in the context of the existing literature and, ideally, through further research.

The association of DM risk with dry food diet in normal weight cats is to our knowledge previously not reported, and it has earlier been proposed that the proportion of dry food in the diet might not be a risk factor for DM. However, because cats are obligate carnivores, whose natural diet consists mainly of protein- rich animal prey, it has been hypothesized that a high carbohydrate diet such as commercial dry food might put an increased demand on the cat’s insulin secretion, thereby predisposing them to the development of DM….

In our study, an effect of type of diet was found only in cats with a normal body condition, suggesting that for overweight cats, the risk of the obesity per se is more important than the type of diet….

Our results should be interpreted with caution, as the macronutrient content in food given to cats in our study is unknown, although a typical commercial dry diet generally contains more carbohydrates than a typical wet diet. It is also possible that the difference between dry and wet food detected in our study might relate to a protein effect rather than a carbohydrate effect, as it is not possible to alter one macronutrient without another.

The study also found some associations suggesting being indoors might increase the risk of diabetes. However, this involves a number of potential confounding factors, and only one of these was directly evaluated. Obviously, if obesity is a known risk factor and indoors cats are more likely to be obese, due perhaps to constant access to food or lower levels of activity, then being indoors would be associated with diabetes without being directly causal. Indoor cats kept lean would not be at higher risk just because they are indoors in obesity is the intervening condition.

In this study, the results were presented only in terms of the interaction between indoor status and activity level (again,  reported by owners, who may well not characterize their cats’ levels of activity accurately; who sits home and monitors their cat’s activity all day, and who sees what their cat is doing when it is outside all the time?). Cats reported to have moderate or low levels of activity seemed less likely to have diabetes if allowed outdoors, whereas outdoor access did not seem associated with diabetes in cats reported to be active.

This is an interesting finding, but incomplete and in need of a lot more direct study to find the direct and modifiable risk factors involved. (Not to mention the question of whether the overall health and longevity of cats is better when kept indoors or allowed outdoors. Even if indoor-only cats are at greater risk for diabetes, they may be healthier overall and live longer than outdoor cats because they are protected from infectious disease, parasites, trauma, and predation.)

Undoubtedly, some anti-vaccine activists will make much of the association seen between vaccination and diabetes in this study. However, there are many reasons to be skeptical of this association. Apart from all the study limitations already noted, there is no plausible mechanism by which vaccines could lead to diabetes (unlike, say, obesity or dry food), the vast majority fo the cats were vaccinated (93%), and this seems more likely to be a spurious associated confounded by the fact that cat owners who take their pet to the doctor more often are both more likely to have them vaccinated and more likely to have diabetes diagnosed than owners who don’t see the vet much and tend not to pay for vaccines or diagnostic tests. Here’s what the authors say about this issue:

An association with an increased risk of DM was seen in fully vaccinated cats compared with unvaccinated cats, but the proportion of unvaccinated cats in the study population was low (7%). There are no supporting evidence that vaccines cause DM in cats, and the association detected in our study should not be interpreted to support a decision not to routinely vaccinate cats. The finding can be explained by diabetic cats having had more visits to a veterinarian than control cats, and can also refer to a recall bias because owners of diabetic cats could be more aware of their cat’s health and vaccination status than owners of healthy cats.

Bottom Line
Overall, this study is a preliminary step in looking for lifestyle factors that might increase the risk of diabetes in cats. It is plausible that dry foods, for example, could be a risk factor, and this study makes more focused investigation of that hypothesis clearly worthwhile. However, there are a lot of limitations to the study and unanswered questions, so this study clearly cannot be taken to prove any such causal relationship. It also appears to show an association between vaccination and diabetes, and this is almost certainly a spurious relationship caused by uncontrolled confounding and other study limitations, which illustrates why these results have to be taken with a grain of salt, or perhaps even a few grains.

Posted in Nutrition, Science-Based Veterinary Medicine | Leave a comment

New Review Reminds us Doctors are Lousy at Predicting Benefits and Harms of their Tests and Treatments

One of the major focuses of my criticism of both science-based and alternative medicine is the failure of doctors to recognize their own limitations. There are innumerable cognitive biases and other sources of error that interfere with accurate and effective clinical decision-making. And there are many reasons why uncontrolled observations, whether by doctors or anyone else, are unreliable and inferior to controlled scientific research when trying to understand the causes of disease and the effects of healthcare treatments. The problems caused by opinion-based and faith-based medicine, including the issue of overdiagnosis and overtreatment in mainstream medicine and the persistence of ineffective treatments, especially in alternative medicine, are directly related to our tendency as individuals to trust our own judgment and beliefs far beyond their real reliability.

A new review of studies involving medical doctors illustrates this problem, and reminds us why we have to rely more on science and less on our own opinions and beliefs if we want to provide the best care for our pets and patients.

Hoffmann TC, Del Mar C. Clinicians’ Expectations of the Benefits and Harms of Treatments, Screening, and Tests: A Systematic ReviewJAMA Intern Med. Published online January 09, 2017. doi:10.1001/jamainternmed.2016.8254

This review looked at studies evaluating how well MDs did at predicting the likely benefit or harm to patients of tests and treatments compared with controlled research. The findings starkly illustrated that, “clinicians rarely had accurate expectations of benefits or harms of the interventions, with inaccuracies in both directions, although they more often overestimated rather than underestimated benefits and underestimated rather than overestimated harms.”

The magnitude of this effect was impressive.

Among the studies comparing benefit expectations…most participants provided correct estimation for only 3 outcomes (11%). Of the studies comparing expectations of harm …a majority of participants correctly estimated harm for 9 outcomes (13%). Where overestimation or underestimation data were provided, most participants overestimated benefit for 7 (32%) and underestimated benefit for 2 (9%) of the 22 outcomes, and underestimated harm for 20 (34%) and overestimated harm for 3 (5%) of the 58 outcomes.

Guessing the correct benefit or risk less than 15% of the time and overestimating benefits by 32% and underestimating harm by 34% is a recipe for ineffective, even dangerous care. Though this is a study of MDs, there is no reason to think vets would do any better, and in fact it is likely that vets would perform worse than MDs and alternative medicine practitioners would do worse than those practicing science-based medicine. Vets generally have less pressure to know and conform to evidence-based standards than MDs due to less regulation and litigation. And alternative practitioners are, at best, often uninterested in scientific evidence and sometimes actively hostile to it.

The direction of the effect was also interesting, and consistent with what we know about how the human mind works. As doctors, we overestimate the benefits of our actions and underestimate the risks because we feel pressure to act and to fix things and because we need to believe we are helping our patients effectively. We are quite worried about causing harm, but we also worry more about the risks of not acting than of taking action.

The finding of more instances of clinicians underestimating harms and overestimating benefits than the opposite provides some support for the existence of therapeutic illusion (“an unjustified enthusiasm for treatment on the part of both doctors and patients,” which is a proposed contributor to the inappropriate use of interventions. Other potential contributors include the often-misleading portrayal of intervention benefits and absence of harms data in journal articles and information from commercial sources, such as pharmaceutical advertisements in medical journals.

We found much more focus on assessing expectations about harm than benefit (67% of studies measured harm expectations only) in contrast to our review5 of patient expectations where most studies (63%) focused on benefit expectations. Clinicians may be more sensitive to harming patients rather than just not providing benefit, which may stem from a fundamental concern of primum non nocere: the primary duty of doing no harm. Medicolegal concerns may also influence clinicians to place greater emphasis on the risks of not doing something rather than the risk of harm from intervening.

The response to this kind of information is not, of course, to give up on medicine. Medical care is tremendously effective at reducing suffering and death. The takeaway message is that individual observation and judgment should always be supported and informed by scientific research, which does a better job at evaluating the causes of disease and the effects of treatments than our ad hoc observations.

Posted in General | Leave a comment

New York and FTC Tired of Prevagen False Claims (which are the same as for the veterinary version, Neutricks)

I have written several times about Neutricks, a supplements marketed for cognitive dysfunction in dogs:

Neutricks: Another Nutraceutical for Canine Cognitive Dysfunction

Evidence Update:  Old Tricks Used to Massage Neutricks Study Data

Evidence Update: Neutricks Still up to Same Old Tricks

This supplement has become the paragon of snake oil supplements for its dramatic claims supported by anecdote and shoddy science. The company and its founders have been warned by regulators about their blithe disregard for unsupported claims about the product, and now the Attorney General of New York and the Federal Trade Commission (FTC) have filed a lawsuit to stop the fraudulent marketing of the human version of this product, Prevagen.

Such vigorous action is rare in the current anti-regulatory climate, and it is encouraging to see an effort like this to protect consumers from false advertising, even if it currently is only aimed at consumers of the human version of the product. The New York AG was quite blunt in describing the suit:

“The marketing for Prevagen is a clear-cut fraud, from the label on the bottle to the ads airing across the country,” the New York Attorney General Eric Schneiderman said in a statement. “It’s particularly unacceptable that this company has targeted vulnerable citizens like seniors in its advertising for a product that costs more than a week’s groceries, but provides none of the health benefits that it claims.”

The FTC was equally direct:

According to the FTC, the makers of Prevagen relied on a single study to back up their false claims. And the study didn’t even show that Prevagen improved memory better than a placebo. To make matters worse, the FTC and New York Attorney General allege that the company behind Prevagen was actively targeting seniors who were struggling with deteriorating memory.

“The marketers of Prevagen preyed on the fears of older consumers experiencing age-related memory loss,” Jessica Rich, director of the FTC’s Bureau of Consumer Protection, said in a statement. “But one critical thing these marketers forgot is that their claims need to be backed up by real scientific evidence.”

Even in this supposed “post-fact” era, it is good to see that at least some in government recognize the need for real scientific evidence for healthcare products to protect consumers from snake oils like this. Hopefully, if this suit is successful, the same principles of consumer protection and scientific will be applied to the veterinary market and pet owners, though sadly standards are often lower for pet products.

Posted in Law, Regulation, and Politics | Leave a comment

Pet Remedy- A “Natural” Herbal Product to Reduce Stress and Anxiety

There are an enormous number of unregulated over-the-counter remedies marketed to pet owners, often for serious health problems, and most of them have never been scientifically tested to see if they are safe or effective. It is difficult to evaluate these products objectively when there is no real evidence. There are always, of course, plenty of anecdotes and testimonials, but while people frequently find these persuasive, I have explained many times in detail why they do not actually tell us much at all about the safety or effectiveness of such products.

Often, the best pet owners and veterinarians can do is consider whether the claims made for such products are reasonable or fanciful, whether the theoretical justifications for how they might work make sense or not, and whether there is any obvious danger. This is not a very reliable or satisfactory way to evaluate healthcare products, but in the absence of regulatory requirements to prove any claims manufacturers make or any ethical commitment on the part of these manufacturers to back up their own claims, this is sometimes the best we can do.

Even when there is some research on a product, however, often the evidence is of limited reliability. And of course, companies never accept any negative reports on their products, and they tend to exaggerate or even mislead consumers regarding the strength of the evidence for their products. I recently ran across a nice study on an herbal product called Pet Remedy, which led me to look at the company marketing materials. Not surprisingly, the marketing does not reflect the evidence for this product.

What Is It?
Pet Remedy is a mixture of several plant extracts. According to the manufacturer, these include extracts from valerian root, vetiver, sweet basil, and Spanish sage. The exact amount of these ingredients, unfortunately, is not disclosed, though the company indicates the valerian content is “very low,” suggesting enough to have the purported benefits of relaxation without the potential. The product comes in a variety of forms, including sprays and plug-in diffusers. The advertising material provides this information about the development of Pet Remedy:

Developed as a result of one of our directors having a cat with behavioural issues! Pet Remedy sorted the problem so effectively that Pet Remedy was then created as a marketable product with a range of plug diffuser, calming sprays, and battery operated atomiser. Pet Remedy was developed with the help of a team comprising several vets, nurses, a psychiatrist and using our own experience in the aromatherapy and essential oil markets.

The company makes broad and sweeping claims for it including:

Clinically proven to calm pets without sedating

Pet Remedy is 100% safe for all mammals and birds, including dogs, cats, rabbits, guinea pigs, hamsters, parrots and horses.

Regular readers will be familiar with my views on claims of significant benefits with absolutely no risk, which I have codified into McKenzie’s Law:

The company quite explicitly claims scientific support for their product. They propose a mechanism of action based on previous studies of valerian extracts:

…the active ingredients in the special Pet Remedy blend will sit on the cell receptor as if a GABA molecule was sitting on it, and that activates (excites) the receptor into giving the cell the message to calm.

The other ingredients are presented in terms of their traditional use, with a lot of vacuous language and extravagant claims that sound like they were copied out of herbal medicine books:

This plant has been appreciated for its calming properties since records began.

Basil oil is a good tonic for the treatment of nervous disorders and stress related headaches, migraines and allergies. It is used to clear the mind and relieve intellectual fatigue, while giving clarity and mental strength.

The therapeutic properties of sage oil are anti-inflammatory, antibacterial, antiseptic, antispasmodic, astringent, digestive, diuretic, emmenagogue, febrifuge, and hypertensive,.There also seems to be a more general relaxant effect, so that the plant is suitable in the treatment of nervousness and excitability. It helps to fortify a generally debilitated nervous system.

The company is also quite specific about the strength of the purported scientific evidence behind their product:

We have already conducted three major clinical Trials…The results of all trials are consistent with pet remedy being an effective natural aid to calm stressed pets.

They also list a number of other studies in progress and offer to send copies of the study reports upon request. Needless to say, I sent such a request since I was otherwise only able to find the one study I had initially run across.

Does It Work?
Valerian has been claimed to be useful as a sleep aid and anxiety reducing herb. The evidence for these claims is limited and mixed in humans (e.g. 1, 2), suggesting there may be some benefits but without definitive proof of these. As usual, there is virtually no good scientific research on the use of this herb in veterinary patients. It is plausible that this herb might have some calming effects in the species the company markets it for, but this does not obviate the need for actual clinical studies of the product itself.

The evidence for medicinal uses of vetiver, basil, and Spanish sage in humans is weaker than for valerian, and again there are essentially no controlled clinical studies to support veterinary use.

When I requested copies of the studies the company uses to promote Pet Remedy, I received two unpublished reports from a statistical analysis firm hired to analyze data collected in two studies and also a draft report of the published study I had already seen. Interestingly, the company did not direct me to the published report of this last study, even though that is available freely through the journal. The company representative also sent me the unpublished results of a survey of groomers suggesting that, when asked, many reported they would be willing to recommend the product. This, of course, amounts to no better evidence than the many testimonials and anecdotes available on the company web sites.

The first unpublished study involved 60 dogs receiving behavior modification therapy who were alternately exposed to Pet Remedy or a placebo. The report doesn’t indicate that important controls for bias were included (randomization, blinding, etc.), and it reports vague and subjective outcome measures: “excitement” on a 5-point scale from “not excited” to “very excited” and “behavior” on a similar scale from “poor/unacceptable” to “better/good.” These are certainly not the hallmarks of good-quality behavioral research, which likely explains why the study has not been published.

The company statistician appears to recognize that the design disallows any meaningful conclusions about the efficacy of the product. Despite reporting some differences in the outcome measures for the dogs when exposed to Pet Remedy or placebo, several cautionary statements are included, such as:

It was noted that as no data were available on the baseline (pre-study intervention) behaviour scores and excitement levels for the dogs, it would not be appropriate to compare the August outcomes between the dogs receiving the Pet Remedy and the placebo in the first month. This would fail to take into account the baseline excitement level and behaviour score of the dogs when they entered the study.

It is important to acknowledge that, due to the design of the study, it is not possible to directly attribute the changes observed to the study treatment received in the current month in each case. As each dog was switched between placebo and Pet Remedy from month-to-month, it is not possible to rule out the possibility of so-called carryover or lagged effects of the treatments received in previous months.

It should be reiterated that, due to the design of the study, it is not possible to directly attribute the changes observed in the trial to the study treatment received in the current follow-up month in each case. This is because of the possibility of so-called carryover or lagged effects of the treatments received in previous months.

We would recommend running another study to replicate the effect with a study design that would allow us to determine whether this was indeed due to the study treatment. Select would be more than happy to provide advice on how best to design the study from a statistical point of view.

I was sent another report from the same statistical company that was named as reporting a different study. The date on the report was different, but the content was exactly the same, so I suspect this may have been an error, but I was not able to evaluate the report from this trial. Information from the company web site indicates both trials used the same design and outcome measures, so it seems likely the caveats of the one I was able to review apply to the one I was not, especially given neither have been published.

I was also sent a couple of Excel spreadsheets with data and notes about the subjects in the two studies.  These were surprisingly revealing. In fact, they contain individual identifying information for study subjects and owners, which is typically not supposed to be shared outside of the study team. The study was conducted at a private training facility, not an academic institution, which may explain the failure to follow standard study design or ethics guidelines.

These spreadsheets do seem to confirm the lack of blinding and randomization, and support the subjective nature of assessments and potential for bias. A note from one of the investigators, for example, indicates that dogs in one of the studies who were exposed to Pet Remedy in the first month were switched to placebo in the second month, as per the study design, but that those owners who thought the remedy was helpful the first month and asked to switch back to it were allowed to do so. This would be a pretty serious violation of basic study design procedures.

One of the spreadsheets also contains notes about the prognosis for the dogs. These show that the evaluators make judgements about whether or not the dogs could be helped as part of their initial assessments. Some were described positively (e.g. “this one will be easy to resolve” and “can’t wait to work with this little man.”) and others quite negatively (e.g. “Not all together put together this dog has brain issues… This dog needs to be PTS ASAP he has mental health issues” and “This dog is a menace.”).

A rather large number of these comments, in fact, suggest the dogs are very dangerous or should be euthanized. This both indicates this is not a typical population of dogs whose owners are likely to want an over-the-counter herbal remedy and also that, in the absence of blinding and randomization, there is a significant risk of individual bias in how the dogs and the treatments were evaluated. In any case, neither of these studies has been published or appears to meet the basic criteria for a reliable controlled scientific study.

The final study was the one I originally found published in the journal Animals.

Taylor,  S. Madden, J. The Effect of Pet Remedy on the Behaviour of the Domestic Dog (Canis familiaris) Animals 2016, 6(11), 64; doi:10.3390/ani6110064

The published study had much better methodology than the other two, including appropriate randomization and blinding. It involved exposing 28 dogs (a pretty small number) to either Pet Remedy or a placebo in an unfamiliar and evaluating for behavioral signs of anxiety. Dogs were chosen whose owners had indicated that they showed signs of anxiety or aggression in unfamiliar places. The study concluded that the product had no effect:

In the current study, no statistically significant differences in behaviour were found in either the Pet Remedy or placebo condition. A lack of a discernible effect under either condition suggests that Pet Remedy did not affect behaviour of dogs’ placed in a potentially stressful non-familiar environment.

This is a pretty clearly negative result in a study that did a much better job controlling for bias than the unpublished studies the company publicizes. In fact, I initially thought it pleasantly surprising that the company would direct consumers to scientific evidence that does not support their claims. However, it turns out the company is doing everything it can to present this study as a success despite the lack of any actual effect.

For one thing, the company lists and describes the study on its web page but leaves out the fact that it found no benefit. The site even says say, “The results of all trials are consistent with pet remedy being an effective natural aid to calm stressed pets.” This is clearly not true. The reality is that the two unpublished studies are unreliable and the published study showed no effect.

The misleading attempt to spin weak and even negative evidence even extended to sending me an unpublished preliminary version of the study report that differed from the published version, even though the published paper is freely available. This was done because the preliminary version presents a much more positive interpretation of the data. In the initial version, two of the many outcome measures evaluated were reported as showing a statistical difference between Pet Remedy, and then this was presented as suggestive of a benefit despite the lack of any difference on all the other measures.

Pet Remedy also significantly decreased overall yawning behaviour; indicative of reduced anxiety although overall heart rate did not significantly differ between treatments. Pet Remedy also significantly affected rates of change in locomotory behaviour but not change in heart rate, with activity decreasing more slowly in comparison to a placebo. The results suggest that Pet Remedy may be a useful tool for reducing canine stress and anxiety in dogs that display suppressed behaviour by increasing activity whilst reducing anxious behaviour. Reduced anxiety may also lower the number of dogs being relinquished to shelters as a result of anxiety related behavioural problems. In turn, Pet Remedy may have potential value in enhancing the welfare of these individuals.

The final paper in published form was apparently edited as a result of the peer review, resulting in the removal of these likely spurious findings and a conclusion more clearly reflecting the lack of any real effect in the data.

Is It Safe?
As pointed out already, the claim of 100% safety can only be true if there is no effect at all, since any treatment which has beneficial effects will almost certainly have potential undesirable effects as well.  The company acknowledges that some such effects have been reported with valerian, but of course they claim that their product is perfectly designed to reap the benefits without these side effects:

Many Valerian preparations are too potent and can sedate rather than calm. Pet Remedy is a low concentration valerian blend and the diffuser delivers a constant slow release, which is very kind and gentle in its effect on the metabolism.

No evidence is provided to substantiate this claim.

Given the limited veterinary research available on this product and its ingredients, the question of safety remains unanswered. It seems unlikely that small amounts of these substances volatilized through a diffuser would have serious risk, just as it seems unlikely they would have dramatic benefits, but in the absence of good evidence, we can only speculate.

Bottom Line
Pet Remedy is an example of the tremendous number of herbal products marketed to pet owners with extravagant claims but little real evidence to support these. This company explicitly claims strong scientific support for their dramatic claims of safety and efficacy. However, the evidence consists only of anecdote, poor-quality unpublished studies funded by the company, and one reasonably good published study that shows no effects, despite the company’s attempts to imply otherwise.

Like the purported benefits, the claimed absolute safety of the product is also not supported by good scientific evidence. There is little reason to think it is especially dangerous, just as there is little reason to think it has significant benefits, but the marketing claims for both safety and efficacy are not, as the company claims, supported by good science.

The company claims a number of additional studies are in progress. If these turn out to have appropriate design and execution, they may clarify whether there are any benefits to this product. If, however, the company persists in exaggerating the significance of poor-quality unpublished studies and ignoring negative results from more reliable research, then these additional studies will do nothing but create the false appearance of scientific legitimacy where it does not exist.

Posted in Herbs and Supplements | 3 Comments

Does Challenging or Ridiculing Crazy Ideas Change Anyone’s Mind?

It has long been a sad and frustrating but generally accepted notion among skeptics that facts don’t matter to people who believe in pseudoscience, conspiracy theories, and other dubious ideas or claims. The so-called “backfire effect” appears to mean that people will only fortify their beliefs if challenged with contradictory facts, so arguing with people only reinforces their beliefs. And it is also widely believed that, as the old saw has it, “You catch more flies with honey than vinegar.” Finding common ground, building trusting relationships, and showing empathy and respect are generally believed to be more conducive to changing someone’s mind than ridicule or hostility.

A recent study appears to challenge these concepts, suggesting that beliefs about conspiracy theorie can be wekaned by both facts and ridicule directed at those beliefs.

Gábor Orosz, Péter Krekó, Benedek Paskuj, et al. Changing Conspiracy Beliefs through Rationality and Ridiculing. Frontiers in Psychology. 2016, Vol 7.

Orac has done a thorough job of summarizing this study, so I will just hit a few highlights. This study exposed subject to unfamiliar conspiracy theories and then to several different attempts to undermine these theories, divided into factual rebuttal, ridicule, and attempting to generate empathy for the “villains” of the conspiracy theory. They then measured several indices of the subjects’ views. The authors concluded that,

Rational and ridiculing arguments were effective in reducing CT [conspiracy theories], whereas empathizing with the targets of CTs had no effect. Individual differences played no role in CT reduction, but the perceived intelligence and competence of the individual who conveyed the CT belief-reduction information contributed to the success of the CT belief reduction. Rational arguments targeting the link between the object of belief and its characteristics appear to be an effective tool in fighting conspiracy theory beliefs…

They also recognized that this conclusion is contrary to established ideas and existing evidence, and so it must itself be treated with a bit of skepticism.

Our findings on the efficiency of rational argumentation go against the mainstream of the communication literature and “common wisdom,” as well as the current affective wave of social psychology emphasizing that emotions constitute the most important factor behind shaping beliefs and attitudes. Considering the modest effect sizes, we assume that rationality has a bigger impact on shaping (sometimes irrational) beliefs than previously expected, given that in the current communication environment, people are overloaded with emotional messages coming from ads, political and social campaigns. Future studies should also investigate the role of rationality and the “rationality heuristic” in belief change.

Orac points out what may be a key aspect of this study. The subjects were not established believers in the conspiracy theories they were exposed to. They were, to relate the study to the issues of this blog, less like homeopaths and more like conventional vets or pet owners with little knowledge and no firm opinions about homeopathy. I have long argued that while battling with true believers accomplishes nothing in terms of changing their minds, playing out such arguments in public may have an influence on the uncommitted majority, who should be the real audience for factual rebuttal and ridicule of unproven and false medical claims. This study provides at least a tiny bit of hope that such an approach may have value.

Posted in General | 2 Comments

Another Systematic Review of Homeopathy and–Wait for It– Still No Good Evidence It Works!

Yet another systematic review of veterinary homeopathy has just been published, bringing the total to six. I have discussed several of these before (e.g. 1, 2). They have consistently failed to find high-quality research evidence to support homeopathy in animals, despite the fact that the previous reviews have been conducted by advocates for homeopathy. Most of the studies identified fail even basic methodological criteria for reliability, and there is a lack of consistency and reproducibility to any positive findings. So while there are individual studies that appear to show homeopathy having some effect, a comprehensive review of the evidence at all levels fails to support any specific use. And while this failure might not be damning for a new therapeutic approach, after more than 150 years, it is past time that a truly effective therapy should have been able to prove its worth.

The same conclusion has been reached by many others, from the systematic reviews of homeopathic trials and systematic reviews of these systematic reviews, to the formal investigations of governmental agencies such as the British House of Commons Science and Technology Committee and the Australian National Health and Medical research Council. While homeopaths have had more than a century to try, they have not been able to produce a body of scientific evidence to convince anyone but themselves that homeopathy works.

C. Doehring, A. Sundrum. Efficacy of homeopathy in livestock according to peer-reviewed publications from 1981 to 2014. Veterinary Record doi:10.1136/vr.103779

The latest review is the first in veterinary medicine not written by the group of homeopathy advocates that has produced the previous systematic reviews. Not surprisingly, it too concludes that despite some trials appearing to show benefits, the scientific literature does not provide good evidence sufficient to validate or recommend homeopathy. As well as reinforcing this consistent finding, it also provides support for a number of the objections that I and others have raised regarding the apparently positive literature often cited by homeopaths to justify their claims, including:

  1. Frequently there is poor reporting of study details, so it is not possible to accurately assess the risk of bias (this, sadly, is a general problem in the veterinary literature, not just in the domain of alternative medicine).
  2. Studies with better control for bias are less likely to show positive results.
  3. Studies published in dedicated alternative medicine journals are more likely to show positive results than those in the mainstream scientific literature, suggesting publication bias.
  4. Positive studies are not replicated. This is key since no single study can prove any hypothesis, much less one as inherently dubious as those behind homeopathy. Consistent findings by different researchers in different settings is required to have confidence in positive effects of a treatment.

The authors made perhaps the most comprehensive search of the livestock literature yet for homeopathy studies. Their goal was to identify any study of homeopathy used in farm animals in a way that might reduce or replace antibiotics, either in the prevention or treatment of infectious diseases or as a growth promoter. Since this is a common use claimed for homeopathy, this is a relevant subject within the larger literature.

From 1981-2014, they found 48 published studies that fit their search criteria. Only 11 of these were double-blinded randomized controlled trials, the best kind of study to evaluate medical treatments. The trials included had a variety of shortcomings. 39 of the studies were considered to have a significant risk of bias based on how they were conducted. Many also included only very small numbers of patients. Some selected a conventional treatment for comparison that did not meet current guidelines or standards for selecting antibiotic treatment, which would have been a misleading comparison.

Slightly over half of the studies showed a positive effect, whereas the rest did not. A number of factors were associated with the chances of a positive finding. For example, the better the study design and the controls used for bias, the less likely a study was to find a positive effect:

Homeopathic trials performed as a single-blind or non-blind RCT, parallel groups or an observational trial (Fig 4) tended to be more frequently efficacious than a double-blind RCT, indicating that positive outcomes may partly be due to a bias caused by a conscious or unconscious preference for a certain treatment.

Altogether, studies with a high possible risk of bias were more prone to report efficacy of homeopathy than studies with a low risk of bias (OR 1.71, 95 per cent CI 0.48 to 6.11, P=0.41 vs OR 0.58, 95 per cent CI 0.16 to 2.09).

Also, studies published in alternative medicine journals were far more likely to show an effect than those published in mainstream journals:

In journals focusing on alternative treatments, 15 of 18 trials reported that the homeopathic remedy tested was effective (odds ratio [OR] 3.75, 95 per cent confidence interval [CI] 0.63 to 22.04, P=0.14), while in journals with a broader focus on veterinary medicine, 12 out of 18 trials found the homeopathic treatment was ineffective (OR 0.27, 95 per cent CI 0.05 to 1.57).

Finally, not a single positive study has been reproduced, which is an essential criterion for establishing an effective treatment:

When taking the total number of studies into account, not even one study was repeated under comparable conditions. Consequently, the existing conditions, which enable a systematic review to be carried out completely are not given. The current evidence of studies providing evidence in favour of homeopathy lacks reproducibility and therefore cannot claim to have sufficient prognostic validity.

This review reinforces the conclusions of previous reviews. Despite 150 years of effort, no reliable, high-quality evidence has been produced showing homeopathy to be consistently effective for any veterinary use. Added to the inherent inconsistency of homeopathic theories with established scientific principles, and the much more rapid and dramatic validation many other treatments have been able to achieve, it is past time to accept that this practice has no value beyond placebo and is inherently a waste of resources and misleading to animal owners.

Posted in Homeopathy | Leave a comment