Raw Diets and Allergies-SkeptVet and DogRisk Exchange Letters in the Journal of Veterinary Internal Medicine

In March of this year, I wrote a post discussing the limitations and misleading conclusions of a research study in the Journal of Veterinary Internal Medicine (JVIM) which claimed that feeding puppies a raw diet could reduce the risk of allergies later in life. This article was produced by the DogRisk research group, which has consistently supported raw diets in scientific publications and through other platforms, including the bogus documentary The Truth about Pet Cancer. My main concerns with the article were:

• The potential for ideological and funding bias
• The reliance on a convenience sample of online owner surveys as the main source of data
• The inconsistency of many of the findings with the underlying hypotheses and conclusions

Because such an article in such a prestigious journal is likely to create the false impression of strong evidence in favor of feeding a raw diet, I felt it was worthwhile to submit a letter to the editor of JVIM detailing my concerns about the report. I was joined in this by Dr. Jennifer Larson, a board-certified veterinary nutrition specialist. 

The process of getting this letter printed in the journal was surprisingly prolonged and difficult. The letter was initially submitted in January, and has finally appeared today, seven months later, which is an unusually long time. The main problem appears to have been concerns on the part of the JVIM legal team that the journal might be sued over my characterization of one of the funders, Dr. Joseph Mercola. Regular readers will know that Dr. Mercola, and the leader of the veterinary side of his website, Dr. Karen Becker, are stalwart promoters of pseudoscience and alternative medicine. Dr. Mercola has been warned and sanctioned several times by the FDA and the FTC for illegal medical claims, but it was quite a struggle to get JVIM to accept this factual information as part of the letter. 

Despite this, and the compulsory insertion of much excessive qualifying language, the letter has finally appeared in the journal. Though it repeats much of what I wrote in my blog post, I will reproduce the full content here:

The report Puppyhood diet as a factor in the development of owner-reported allergy/atopy skin signs in adult dogs in Finland, which appeared in the September/October issue of the Journal of Veterinary Internal Medicine (JVIM), suggests that “puppyhood exposure to raw animal-based foods,” “human meal leftovers” and other “real foods” might have a protective influence against canine atopic dermatitis (CAD) while “heat-processed foods” might increase later occurrence of CAD. We are concerned that the results of this study likely represent residual bias in data collection and analysis rather than the actual influence of specific dietary components on the risk of CAD.

Common recognized sources of bias in research studies include a priori beliefs on the part of researchers, recall and selection bias in survey respondents,¹ funding bias,² and a large number of researcher degrees of freedom in the design, conduct, and analysis of the study.³All of these are present in this study.

Dr. Hielm-Björkman and the Dog Risk research group have expressed strong beliefs about the health benefits of raw diets and the dangers of conventional commercial pet foods in previous research reports and popular media.^4, 5

The main source of data in this study is an online survey available only in Finnish. Previous publications regarding validation of this data source have identified low response rates to validation questions and important proportions of duplicate, automated, and discordant responses.^6, 7 Asking owners to remember in detail what they fed their puppy between 2 and 8 months of age and then trying to associate that with health outcomes years later is, in our view, a questionable strategy. Self-reporting of diet and health information are unreliable in humans, and we believe that it is unlikely to be more reliable among dog owners.¹ Dog owners consistently misperceive even straightforward measures such as body condition score despite formal training in this assessment, so their assessment of signs of CAD is, in our view, likely to be equally unreliable.⁸ The survey responses are expressions of the perceptions and beliefs of the owners who participated, not necessarily the true nutritional and environmental exposures nor health outcomes experienced by the dogs.

Dr. Hielm-Björkman has previously cited funding bias as a problem in the study of raw diets, stating that raw foods are “not really researched in universities. Most universities get sponsored by these big billion-dollar companies, and you don’t really want to step on their toes, I guess.”⁵ In light of the recognized problem of funding bias it could be relevant that disclosed funding sources for this research include companies selling raw pet foods.

The authors also report accepting funding from Dr. Joseph Mercola. Dr. Mercola’s organization advocates for raw pet diets and argues against conventional pet foods on its web site.^9–11 We believe that he is also a consistent promoter of unapproved medical practices and medical product claims, as evidenced by multiple warning letters from the Food and Drug Administration and a lawsuit by the Federal Trade Commission related to therapeutic claims that violate the U.S. Food, Drug, and Cosmetic Act.^12–16 Such an affiliation is at least as relevant to assessing potential bias as accepting funding from an organization with a commercial interest in raw or conventional pet foods. Bias in scientific research is as likely to arise from ideological as financial factors.

We believe that the results of the analysis and how they are reported also suggest the influence of bias. Many comparisons are listed only in supplemental materials, and those reported in the main article tend to be those which support the authors’ hypotheses. This creates the appearance of consistency when, in fact, the associations identified are often inconsistent and do not logically support the claims in the conclusions. For example, why would raw tripe and organ meats be protective against CAD but raw red meat, eggs, and poultry not be? If cooking is the key risk factor, why would cook vegetables be protective and raw vegetables would not or why would both cooked and raw eggs be protective while neither cooked nor raw poultry is associated with the likelihood of CAD? If exposure to bacteria is the main variable, why is eating dirt, sticks or carcasses protective but eating clay and grass is not and drinking from puddles is actually associated with increased risk? If excessive processing is the issue, why was there no association with eating processed meats or canned foods and only a marginal association with dry food when it was the only food offered?

Dogs with CAD were reportedly more likely to be eating no raw food at all than controls, and dogs without CAD were more likely to be fed 20% or 90% raw, but there were no statistically significant differences at any other ratio of the 2 foods. Similarly, allergic dogs were more likely to be fed 80% dry than controls, but there was no significant difference in CAD risk if they were fed more than 80% dry. Control dogs were more likely to be fed 50% or less than 10% dry, but there was no difference at intermediate ratios. It is easier to see cherry picking and researcher degrees of freedom than a consistent dose-response in these results.

While the authors do acknowledge the potential for recall bias and misclassification, and they do state that their findings cannot prove causal relationships, the overall message of the paper is that uncooked foods and human meal leftovers likely have health benefits, and this is inconsistent with the methodological limitations and potential for residual bias of this study.

Given the importance of CAD, in terms of both prevalence and the negative impacts on pet and owner quality of life, we agree that identification of protective factors is warranted. However, considering the significant risks of illnesses and death that have been consistently associated with the feeding of raw animal products, rigorous and well-designed research methodology, objectivity, and full reporting of results are needed to explore if any benefits exist for this practice, and certainly are required before this practice can be confidently recommended.

CONFLICT OF INTEREST DECLARATION
Jennifer Larsen declares the following conflicts:

Investigator in clinical trials and other research partly or fully sponsored by Royal Canin, Nature’s Variety Instinct, and Nestle Purina PetCare.

Develops educational materials for Mark Morris Institute and HealthyPet magazine.

Serves as advisory group consultant for Elanco Animal Health.

Participates in continuing education events, as a speaker & as an attendee, sponsored/organized by Royal Canin, Nestle Purina PetCare, Nature’s Variety Instinct, and Hill’s Pet Nutrition.

A resident of the Nutrition Service received funds through the Hill’s Pet Nutrition Resident Clinical Study Grants program.

The Veterinary Medical Teaching Hospital at the University of California, Davis receives funds from Nestlé Purina PetCare to partially support a nutrition technician.

Brennen McKenzie declares no conflict of interest.

As is usual and appropriate, the authors had an opportunity to provide a response to the letter. The response didn’t really contain any meaningful rebuttal of the concerns we expressed, merely a series of assertions that they weren’t valid, relevant, or important. Here is the full content of the authors’ rebuttal:

We read Drs McKenzie and Larsen letter with interest, and we thank the Editor for the opportunity to discuss the scientific questions through this forum.

Dr McKenzie (skeptvet¹) and Larsen in their letter to the editor indicate that the DogRisk research group has a priori notions about the benefits of raw feeding. We want to point out that we only have three veterinarians and only three dog owners in our research group of eight, only two feeding raw, and that we take pride in discarding our hypotheses every time we prove them wrong. Regarding raw food we have time after time, through published articles and 10 university student theses in Finnish, seen that there are health benefits when owners use a raw diet for their dogs, compared to a dry. We acknowledge the variety of raw and dry dog diets available and therefore we also know that the reasons behind the health benefits are hard to estimate. We have analyzed gene-expression² and metabolomics³ in a raw-dry diet-intervention study. We look at processing, the absence of heat, macronutrient profile (proteins, fats, and carbohydrates, content and origin), bacterial load, among other factors. We now work with 10 hypotheses of why the raw food diet repeatedly comes up as healthier in our studies. In a world where our dogs suffer increasingly from noncommunicable diseases that are similar to those of humans (atopy, allergies, IBD, diabetes, certain cancers and similar) we can also make veterinary medicine relevant for human medicine by reporting what we see in dogs, that share most of the other environmental factors with us, their human owners. To this end, diet is an excellent variable to investigate, as dog owners tend to keep their dogs on the same diets for extended periods of time, sometimes a whole lifetime. Also, because of diets being so homogenous, owners do not tend to forget or misrepresent it, as they might do about their own diet. These are our motivators, not proving a preconceived hypothesis.

The second issue that Dr McKenzie and Larsen were concerned about was survey problems. All relevant concerns are mentioned in the limitations part of the article, as they themselves point out. In contrast to what was said in the letter, we want to point out that our research is, and has been in all our papers, completely transparent. We always mention the reasons for omitting cases: for example, robot answers, dogs not reported to eat enough to stay alive, too young controls as they still could develop the disease etc. As expected from an academic research group, our survey is validated,⁴ including the dependent variable of this paper (allergy/atopy). We are of the same opinion with the authors of the letter that owners misperceive especially body condition score; our data show that only 13% of owners have reported that their dogs are overweight (12%) or obese (1%). But we also know that they are good at perceiving the clinical signs their dogs have: itching, ear, eye or skin infections, anal gland problems, teeth and gum problems etc., as those are the reasons that they contact their veterinarians for appointments. The owners in our survey also had the possibility to tick other skin related disorders (eg, hot spots, demodicosis, seborrhea etc.) than allergy/atopy. And as they did not, there are not so many other diagnoses than allergy/atopy they could have had, as flea allergic dermatitis is not a common entity in Finland.

Further, the authors of the letter to the editor addressed an issue we again take pride in, analysis of the data. We totally disagree with the notion of “cherry picking” data and instead again want to highlight that by putting all results in either the main article or supplemental material for other researchers to see, we can best forward this area of research. Regarding selection bias, when using backward stepwise regression, the computer starts with all variables and excludes the ones with the lowest coefficient of determination. The result is a computer-generated final model, including the significant variables. Only the machine chooses, not humans so no bias is even possible. It is common practice to discuss the significant results and trends while the pre-analyses are put into the supplemental data (S1-2). Figure 2A,B in our article show that there are more non-atopic dogs than atopic dogs when the dogs have been eating more than 10% raw food and on the contrary, more atopic dogs, when they have been eating more than 80% of their diets as dry. Not all 10% intervals will show significance between the two diets as there will not be enough cases at all intervals and it is also normal that some of 20 intervals (here one; eating 60% dry) will be out of line, for the same reason. That it is so consistent, is remarkable.

At the start of our response, we mentioned some of our hypotheses for why raw diets come up as healthy, but that we lack an answer to the “why?”. The authors to the letter also had many “If-why?” questions. At this time, we do not have the answers, but we hope that we will be able to answer them within a couple of years.

Another issue raised was funding bias. Also here, we have been completely transparent as can be seen in our long list of funders. We have been able to attract funding, for example, from state funding bodies, foundations, private people and companies by crowdfunding, raw food companies etc. and we are equally thankful to all. However, the big traditional dry feed companies have not been interested in funding our research, despite enquires. On our website, we disclose and thank all our sponsors (Moomin trolls and all) while letter author McKenzie instead has paid advertisements from Hills’ and Mars, for example, featuring the same Royal Canin advertisement on atopy diets both as “sponsored content” and in his “education center,” on his Veterinary Practice News homepage.⁵ Dr JA Larsen discloses her close co-operation with Mars, Hills’ and Nestle’ Purina in the Conflict of Interest section in all of her published articles.

Finally, we would like to point out that we have no “raw food agenda” but if we find that the “raw” is of value, we feel that we have an ethical obligation to the community to report it. We will therefore now recommend that people who feed dry food supplement the diet with at least 20% raw. If the health benefits come from beneficial bacteria, we will recommend that they should be added to the dry food, etc. Also, contrary to the letter authors’ comments, we have and will continue to report on raw feeding not being dangerous for neither dogs nor family. In our risk-analysis study, we found only three verified cases of food pathogen transmission from raw dog food to humans in 16?475 households feeding raw and in 98?353 pet years at risk.⁶ This study has been replicated with similar results.⁷ As a university-based independent research group, we are driven by a concern for the growing disease load in our dogs. We hope our research will benefit both canine and human health and therefore we report everything we find, as we find it.

Here is how I view these responses:

• “We only have three veterinarians and only three dog owners in our research group of eight, only two feeding raw”
  
  If I interpret this correctly that all eight members of the group own dogs and only two of them feed raw, I would say that is interesting, but not really relevant to the issue of whether the group in general has a bias in favor of raw diets that influences their work. Our personal practices don’t always perfectly mirror our beliefs.

• The next point they make is that their studies have consistently found benefits to feeding raw and that they have numerous hypotheses for why, though they cannot yet claim a specific reason for these benefits. Again, the fact that their work consistently finds what they believe and expect to find doesn’t say much about whether those beliefs, of the evidence they have generated, are correct. It would be much more convincing if other research groups with other perspectives and agendas found the same results. 
  
  It is always the case in science that when much of the evidence for a claim comes from a single source, that is a reason to be skeptical of that evidence, and there is no reason not to apply that principle to the subject of raw diets or the DogRisk group.

• The group next acknowledges the limitations of an owner survey as a data source while simultaneously claiming that the particular evidence and conclusions they have drawn from such a survey are correct. This is just an assertion of belief, not evidence that the limitations Dr. Larson and I raised are not influencing the findings.
  

• They then respond to our pointing out the inconsistency of the findings and the emphasis only on those that support their claims while those that do not are relegated to the supplemental material. This response is, again, just to assert that there is nothing about these findings or reporting that is problematic or should undermine confidence in their conclusions. That is for readers of the article to decide, but simply responding to the identification of a potential problem by saying “that’s not a problem” isn’t a very compelling argument.
  
  In particular, they acknowledged that they cannot explain some of the inconsistencies we pointed out in the findings, but they don’t see these inconsistencies as a reason to question their underlying conclusions about the benefits of raw diets. That response certainly has all the hallmarks of cognitive dissonance.
  

• With regard to the issue of funding bias, the authors respond by arguing that it isn’t a problem since they disclose their funding sources and then move on to the tu quoque fallacy, which is the strategy of arguing that if someone else has a potential financial conflict of interest, that negates the potential significance of their own. 
  
  Apart from being irrelevant, this claim fails with regard to my own supposed financial interests. Since I receive no funding of any kind from the pet food industry, the only way they could make a claim of funding bias was by pointing out that the Veterinary Practice News puts ads for pet food on some of the pages where they print my column on evidence-based medicine. 
  
  I not only have no influence on advertisers the magazine works with and receive no income from ad in the magazine, I am actually unaware of what ads appear in the magazine since I rarely even read my own columns once they have been published. How this could possibly influence my positions on raw diets is unclear, so this is a bit of a desperate tactic.
  

• They close their rebuttal by simply reasserting their claims that raw diets have benefits and making the truly astounding claim that they have virtually no risks because they haven’t seen any in their survey-based research. Given the robust literature showing the risks of infectious disease for humans and animals associated with raw pet foods, this is a completely incredible claim, and nothing could possibly illustrate the biases and blindness to contrary evidence of this group more clearly.

References from Original Letter

1.         Ravelli MN, Schoeller DA. Traditional Self-Reported Dietary Instruments Are Prone to Inaccuracies and New Approaches Are Needed. Front Nutr. 2020;7:90. doi:10.3389/fnut.2020.00090

2.         Resnik DB, Elliott KC. Taking financial relationships into account when assessing research. Account Res. 2013;20(3):184-205. doi:10.1080/08989621.2013.788383

3.         Simmons JP, Nelson LD, Simonsohn U. False-positive psychology: undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychol Sci. 2011;22(11):1359-1366. doi:10.1177/0956797611417632

4.         Fredriksson-Ahomaa M, Heikkilä T, Pernu N, Kovanen S, Hielm-Björkman A, Kivistö R. Raw Meat-Based Diets in Dogs and Cats. Vet Sci. 2017;4(4):33. doi:10.3390/vetsci4030033

5.         Habib R, Becker K. The Dog Cancer Series: Rethinking the Cancer Epidemic Vol. 1- Chapter 4 (Transcript).; 2018:73.

6.         Hemida M, Vuori KA, Salin S, Moore R, Anturaniemi J, Hielm-Björkman A. Identification of modifiable pre- and postnatal dietary and environmental exposures associated with owner-reported canine atopic dermatitis in Finland using a web-based questionnaire. PLoS One. 2020;15(5):e0225675. doi:10.1371/journal.pone.0225675

7.         Roine J, Uusitalo L, Hielm-Björkman A. Validating and reliability testing the descriptive data and three different disease diagnoses of the internet-based DOGRISK questionnaire. BMC Vet Res. 2016;12(1):30. doi:10.1186/s12917-016-0658-z

8.         Eastland-Jones RC, German AJ, Holden SL, Biourge V, Pickavance LC. Owner misperception of canine body condition persists despite use of a body condition score chart. J Nutr Sci. 2014;3:e45. doi:10.1017/jns.2014.25