I have written about the issue of homemade pet diets here several times (1, 2, 3, 4). They are appealing to some owners because they appear more “natural” than commercial dry or canned diets, which is supposed to imply they are better for pets. And, of course, many proponents of alternative medicine make hysterical and unsupported accusations about the dangers of conventional pet diets.
People also equate conventional commercial pet food with what is typically called “processed food,” though they are entirely different things. Human snack foods and other processed foods are laden with excessive sugar, salt and fat and generally nutritionally poor. Commercial pet foods, if properly formulated and manufactured, are nutritionally balanced to a greater degree than our haphazard diet of whatever looks appealing in the moment, even when we the packaged junk foods are avoided.
Homemade diets can be perfectly healthy, and there are circumstances in which a diet formulated for the specific needs of a particular pet is better than any commercially available diet. And fresh food is certainly attractive to many pets. But the dramatic claims of health benefits made for them are entirely unproven, and the existing research suggests most recipes for homemade diets, even those promoted by veterinarians, are not appropriately balanced nutritionally and not ideal for long-term health. I encourage anyone interested in preparing food at home for their pets to consult with a board-certified veterinary nutritionist for guidance.
I recently ran across a press release from a company which appears to be trying to cash in on fears of commercial pet food and the appeal of homemade diets in order to sell—you guessed it, their commercial pet food. Reminiscent of the “just like homemade” marketing approach often used to sell packaged foods for people, JustFoodForDogs makes heavy use of terms like “scientific” and “evidence” in their marketing to suggest that dry commercial diets and the ingredients they contain are unsafe and that their packaged frozen cooked diets are better. While these diets appear to meet all the same standards for balanced nutrition of other commercial diets, including AAFCO feeding trial tests, the evidence offered for their superiority is so far scant.
One unpublished study funded by the company and run by one of their veterinarians is referred to in their press release as “groundbreaking” and “game changing.” Science by press release is always a bad sign (anybody remember “cold fusion?”), but the presentation of the study is clearly designed to maximize its marketing value without providing any of the information that would be needed to determine if the methods were really appropriate.
Twenty-one dogs of unspecified breeds were fed some of the company’s diets (the details are not reported) and basic bloodwork and exams were conducted at the beginning of feeding the diet and again at 6 months and twelve months. No control group, no blinding, no pre-specified outcomes or hypotheses, no reported accounting for repeated measures or multiple comparisons in the statistical analysis, no discussion of any other aspects of the dogs health or environment, and overall none of the hallmarks of an actual controlled clinical study. All of this would be fine of the purpose were merely to explore the effects of the diet and generate hypotheses. But the company clearly intends to present these results as earthshattering, paradigm-shifting research that (coincidentally?) favors their product.
And after setting up everything with no apparent effort to control for the obvious risk of bias, what were the reported results? One kind of blood protein, globulins, went up (by how much isn’t disclosed). Some kinds of white blood cell numbers increased (again, by how much isn’t disclosed, but the numbers were apparently still within the range of normal). And one measure of red blood cells increased, though another did not.
Given the comparisons of many different values with no explicit reason and no reported use of statistical methods to control for making them, it is almost guaranteed some values would change to a degree judged “statistically significant.” This is not the same thing as medically significant, and there is no evidence these changes had any clinical relevance, especially with no control group for comparison. But the company promotes the results as showing their foods “could benefit immune health” and that if the purported trends in the blood values continue for the animals’ lifetimes “we may see a decrease in chronic diseases such as cancer, renal failure, kidney disease, inflammatory bowel disease, dental disease, etc.”These results certainly don’t support anything even approaching such claims.
The hypothesis that fresh foods could have health advantages over extruded kibble or commercial diets is not an unreasonable one, and I am open to the possibility this might be true. But this is not something we can simply assume without evidence, and that evidence does not yet exist. Furthermore, the claims made about the dangers of conventional commercial diets are rarely supported by evidence either, whereas there is abundant scientific research and real-world experience showing that pets can live long, health lives on these foods.
If the folks behind this company genuinely believe their claims about health and nutrition, and I have no reason to think they don’t, then they should make an effort to design and conduct properly controlled scientific research to evaluate their hypotheses. But they do a disservice to pets and pet owners when they perform “studies” clearly designed with marketing rather than science in mind, hype the results to an extreme degree, and then use this as a marketing strategy to promote their own products.
Skeptvet, will you have access of the upcoming ACVIM conference presentations when it concludes? JustFoodForDogs claim they’re presenting it this week.
I noticed immediately the fear-mongering on their site, it’s everywhere. I’m so turned off by that tactic.
Reply
I believe I will eventually have access to the proceedings, so hopefully some of the methodological info missing from the .pdf on the company web site will be reported there, though it’s being presented as an abstract, not a peer-reviewed paper, so likely a lot of details will still be missing.
Reply
I agree that science by press release is never a good sign. My Yorkies eat a commercial kibble, the same diet that served my previous dog very well. I would like to see good research done on homemade pet diets, ideally covering much more than one year. If I ever make the leap into homecooking for my dogs, I would rather buy the whole foods and prepare them myself.
Reply
Skeptvet,
The staff at JustFoodForDogs thanks you for the opportunity to address these comments and your concerns; this venue is a perfect place to have this discussion given the namesake of this blog:
skep·ti·cal [skep-ti-kuh l] adjective
1. inclined to skepticism; having doubt.
2. showing doubt: a skeptical smile.
3.denying or questioning the tenets of a religion: a skeptical approach to the nature of miracles.
4. (initial capital letter) of or pertaining to Skeptics or Skepticism.
It would seem you boast an evidenced based approach that doubts (is skeptical of) any claims that contradict evidence. Please clarify if this is not an accurate assessment, as we admit that we are not well acquainted with your blog. We write this response in this context.
We have a few points below:
1) Typo, third line, fourth word “hich” should be “which”
2) You make the comment: “hysterical and unsupported accusations about the dangers of conventional pet diets”. This is a very surprising statement from this blog, considering the evidence that is available for anyone that is looking. Have you ever considered questioning (being skeptical of) of the tenets of extruded diets as the best form of nutrition in pets? A simple review of peer review literature should help. For links to the studies, please visit our website (your spamming program will not allow us to post them here):
a. In 2008, The Journal of the American Veterinary Medical Association confirmed that 70 pets died of renal disease due to necrosis (dying) of the kidneys linked to melamine-tainted food during the recall.
b. Also in 2008, a study published in the Journal of Veterinary Pathology analyzed kidney samples from a dead Parson Russell Terrier, a dead Burmese Mountain Dog, and a dead mixed breed dog suspected to have died from contaminated food. They confirmed the deaths were linked to illnesses caused by melamine from commercial pet
c. The Journal of Toxicological Sciences also published a paper in 2008 citing a conclusive link between melamine contaminated feed ingredients and the outbreak of renal toxicity during the recall.
d. In 2009, The Journal of Interdisciplinary Toxicology described two examples where melamine was found in two feed (animal grade) products, and took it one step further to warn of possible human food contamination.
e. In 2010, The Journal of Medical Toxicology looked at 278 pets that died during the outbreak, confirmed the connection, and found that those that were weak or debilitated (sick or aging pets) were the most vulnerable.
f. Another 2010 study in the Journal of Veterinary Medical Science analyzed the death of two young dogs in Treviso, Italy. The study concluded these dogs died from a similar cause as those who died during the 2007 recall – melamine toxicity from pet food; proving the issue is of testing for melamine contamination is not yet resolved.
g. In February 2013, JAVMA published concerns about copper toxicity from years of over supplementation as recommended by AAFCO.
In addition to chemical toxins, there are bacteria that sometimes get into pet foods. In fact, the FDA released a video focusing on Salmonella bacterial contamination in dry pet foods (not just raw). The study that this video is based on is:
Imanishi M et al. (2014). Outbreak of Salmonella enterica serotype Infantis infection in humans linked to dry dog food in the United States and Canada, 2012. Journal of the American Veterinary Medical Association. 244(5):545-53. doi: 10.2460/javma.244.5.545.
i. As early as 1993, concerns of toxins in feed (animal grade) ingredients were referenced in the literature. At that time, it was reported that the FDA would continue to monitor the issue.
j. In 1997, the journal of Food Additives and Contaminants reported a specific concern regarding mycotoxins in pet food, when it was established that low levels of them could be found in feed grade ingredients.
k. A study published in the Journal of Food Protection in 2001 cited concerns regarding the identification of various fungi (the source of mycotoxins) in commercial pet foods imported from Argentina and warned about the “risk for animal health”.
l. A 2006 study published in the Journal of Agricultural and Food Chemistry looked at mycotoxins in pet feed around the world and concluded that contamination can lead to chronic effects on the health of pets.
m. In 2007, the International Journal of Food Microbiology published a study that stated “mycotoxin contamination in pet food poses a serious health threat to pets”, and listed them: aflatoxins, ochratoxins, trichothecenes, zearalenone, fumonisins and fusaric acid; which have been found in feed ingredients for pets and are linked to acute toxicity and chronic health problems.
n. A 2008 study published in the Journal of Animal Physiology and Animal Nutrition found high levels of mycotoxins in the raw ingredients used for pet food in Brazil.
o. A 2010 study in the journal of Mycotoxin Research tested 26 commercial dog foods for a variety of mycotoxins and found sub-lethal levels that were concerning; it was determined that long-term exposure to them could pose chronic health risks.
p. A 2012 study published in the journal of Toxins found concerning levels of multiple mycotoxins in European manufactured pet food despite some regulatory oversight
q. In February 2014, the FDA commented their concerns about feed ingredients as defined by AAFCO, saying most are not Generally Recognized As Safe (GRAS).
3) You make a comment about “processed food” vs unbalanced processed food, but just to clarify that is not what JFFD is. JFFD is balanced unprocessed whole food. Thus it is complete and balanced just as the extruded diets, but instead of using processed feed ingredients and relying on supplementation, it uses real food and maximizes nutrients from whole foods. The USDA 2010 dietary guidelines for American promote this approach in our own (human) nutrition. I’m not sure how this point applies in this case.
4) You go on to cite existing research about homemade diets, and you previously covered the recent study by Stockman et al. with thorough coverage of the study.
You comment a disapproval of science by press release, yet isn’t that what happened when UC Davis issued a press on this study release on July 16, 2013?
You (and many others) were so impressed by the Stockman et. al study that reinforced their ‘tenets’ that no one thought to look at it objectively. If you do, you will see it is inaccurate and flawed. The authors use the wrong citation for Vitamin D in their tables. They write that the NRC RA requirement is 339 IU/1000kcals when it is – in fact – 136 IU. 1 year after publication, no one has thought to be skeptical of their work. JFFD has alerted the editor of JAVMA and a correction is being printed – look for it in July or August.
5) You criticize JFFD for using “scientific” and “evidence” based approach, but isn’t that what everyone is asking for? Critics set the goal posts, and then when a small company actually strives to meet the goal post, they move them. By the way – take a look at the guides for popular prescription diets. Check out the standard of “evidence” used. One very famous high fiber diabetic diet that starts with a W and ends with a D, cites: Data on file,..
Their heavily marketed new “brain diet” that was released in 2010 cites these as the evidence behind the formulation; two unrelated past studies from 2002 and 2003. Where is their “original”, “evidence” based, “scientific” research? Have you been skeptical of their work?
• Milgram NW, Zicker SC, Head E, et al, Dietary enrichment counteracts age associated cognitive dysfunction in canines, Neurobiology of Aging, 2002; 23:737-745.?
• Dodd CE, Zicker SC, Jewell DE, et al., Can a fortified food affect the behavioral manifestations of age-related cognitive decline in dogs, Vet. Med., 2003; May: 396-408.
6) The process of academia is a poster presentation, then a manuscript, then publication and that process is being followed in this case. Dr. John Tegzes VMD DABVT presented the poster at the AAVN conference and the researchers are submitting a manuscript. It is true that the small company is going to want to spread the word of the favorable research but it’s not their research, it’s independent research from the universities. It’s ground breaking in that no one else is looking at whole food diets like this. It is hard to imagine a small, growing company that was featured in independent research favorable to them NOT making a big deal about it if its robust data and following the proper academic process. We were cleared to announce the poster presentation since it was accepted by a panel of experts. I’m sure we’ll hear many more similar research milestones from this company. The press release is clear that this is only the beginning for us.
More importantly, if you had questions, why didn’t you call us in your skepticism in order to avoid writing a blog that was missing info? The company is very transparent and we would be happy to answer all your questions. Dr. Chavez’s direct extension is 777, and he’s usually available for all inquiries.
Some details you ask about are available if you just call. The others (mainly the discussion) must wait for peer review. The academic process must be respected. This is independent of the company’s wish to promote the findings that have already been released by the researchers.
7) None of the vets/researchers involved had formal affiliations with JFFD during the research. One vet (Dr. Chavez) left the university and joined JFFD full time months after the research ended because by that point he was among the leading researchers in whole food nutrition. This transition from academia to industry is common in innovation.
8) “the claims made about the dangers of conventional commercial diets are rarely supported by evidence either”
a. We hope you find that concern has been suitably addressed with the above citations.
9) There are plans for ongoing research, and history will be the judge of which diets are better and who defended what modality. Some people will be caught smoking cigarettes in white coats, while others will be innovators. We’ll see what happens.
Reply
I appreciate your response, and will attempt to address the points you raise in detail, though it may take some time to work through your extensive comment.
To begin with, “skepticism” is a term with a complex provenance, with much debate and discussion in the domain of philosophy in particular as to what it means. The variety I embrace is Scientific Skepticism, which emphasizes the provisional nature of knowledge claims about the physical world and the primacy of empirical evidence. Here are a few stabs at a definition which I think give a more accurate sense of my use of the term than the less detailed definitions you have cited:
“A skeptic is one who prefers beliefs and conclusions that are reliable and valid to ones that are comforting or convenient, and therefore rigorously and openly applies the methods of science and reason to all empirical claims, especially their own. A skeptic provisionally proportions acceptance of any claim to valid logic and a fair and thorough assessment of available evidence, and studies the pitfalls of human reason and the mechanisms of deception so as to avoid being deceived by others or themselves. Skepticism values method over any particular conclusion.”
“Skepticism is a provisional approach to claims. It is the application of reason to any and all ideas — no sacred cows allowed. In other words, skepticism is a method, not a position.”
“Skepticism is a method of examining claims about the world. The skeptical “toolbox” includes a reliance upon reason, critical thinking, and a desire for verifiable, testable evidence about particular claims (especially extraordinary ones). Usually, the “skeptical way of thinking” is embodied in the scientific method.”
You are correct that I advocate the evidence-based medicine (EBM) approach to evaluating claims about veterinary medicine, and this entails the EBM process of evaluating and critically appraising all evidence, ranking evidence based on both level and quality, and constantly re-assessing claims as the evidence evolves. I am particularly interested as well in how evidence is employed, since there are uses of scientific evidence that are fundamentally at odds with the basic principles of science and EBM. My appraisal of the abstract your company has posted on your web site included not only evaluation of the evidence itself, which necessarily is of pretty limited value at this time in the absence of any reporting of the methodological details that would allow an accurate assessment of risk of bias, but also a discussion of how that evidence is put to use in a marketing strategy. Your press release characterizes the significance of this evidence (“groundbreaking” and “game-changing”) in a way that appears entirely unjustified based on even a fairly charitable interpretation of the findings and the assumption that there are no significant methodological limitations not reported in the abstract. “Interesting,” or “preliminary” would be far more appropriate adjectives, though obviously less useful for a marketing perspective.
Reply
To clarify, I have never asserted that extruded diets are “the best form of nutrition in pets.” so this is a straw man. I assert only that there is strong evidence that they are nutritionally adequate if properly formulated and produced and that millions of pets, including my own, have lived long and healthy lives consuming these diets. I have always been very clear that I do not believe we know what the optimal diet is for individual pets, or even for specific species or breeds, and that further research may very well find alternatives to conventional commercial diets have health benefits. Even raw diets, which you are as critical of on your site as I am, may very well actually be healthier than cooked diets, though I doubt it based on the weakness of the theoretical rationales put forward to support the idea. At this point, I am not aware of good quality empirical evidence to support that hypothesis, so I maintain a skeptical stance, which is to say I have a provisional view held at a very low level of confidence and I consider strong claims for or against the hypothesis to be unjustified.
I hold a similar view for the claims you make concerning “fresh” or “human grade” ingredients being safer and healthier for pets that extruded diets. This is a reasonable hypothesis but there is little empirical evidence to support it, and I consider the claims you make to go well beyond what is justified by the evidence I am aware of. Though I am not a veterinary nutritionist (I am not board-certified in that specialty), I have some familiarity with the literature, and those nutritionists that I have spoken to on the subject are also not aware of evidence to justify such claims. I consider this, then, an unproven hypothesis which is worth investigating but which does not justify the kind of wholesale rejection of current feeding practices you seem to support.
Reply
Studies a-g.
It is indisputable that melamine at the levels found in pet food during the outbreak you refer to was toxic and caused harm. I know of no one who disagrees with this or recommend putting this substance in pet foods. The contamination event represented a serious problem with the ingredient supply chain and regulatory oversight, and these issues certainly need to be addressed.
However, this is not about a fundamental characteristic of extruded diets compared to “fresh” commercial diets or homemade diets. Kibble is not inherently unhealthy because at one point in time many batches of kibble became contaminated with a toxin through malfeasance on the part of a supplier substituting the melamine for an ingredient which, in itself, is perfectly health and appropriate. To say kibble should be abandoned because of this incident is the same as to say all use of pharmaceuticals should be abandoned because a batch of injectable steroids became contaminated with a fungal pathogen or that no one should ever eat beef because people have died due to bacterial contamination of beef sold for human consumption.
The occurrence of such incidents raises issues that need to be solved to improve safety, but it does not justify a global assessment that conventional pet foods are inherently unhealthy or unsafe. If you wish to assert that your commercial food is safer than other kinds of commercial pet diets, you will need to demonstrate this empirically rather than simply suggesting that such incidents justify trusting your product above others.
Reply
Mycotoxins-
Again, there is no doubt mycotoxins in foods for both humans and animals can be a health hazard, and the FDA and relevant agencies in other countries actively monitor and regulate this problem. Acute toxicoses do occur in humans and in animals, though they are rare. The concern about chronic effects of doses within the regulatory limits is a valid one, though I am not aware of the extent of the evidence showing this is a significant cause of morbidity and mortality in pets, or whether the risk differs with the type of diet fed (commercial vs homemade, extruded vs raw or fresh diets, grain free vs grain-containing diets, etc).
Once again, the issue you raise is a real one, but it is a step too far to say that it supports the claim that existing diets are causing significant health problems which would be prevented by switching to your diet or others like it. This is yet another hypothesis that needs to be demonstrated not simply assumed true.
Reply
3) My point as simply that in ordinary discussion people use the phrase “processed foods” to refer to convenience foods for humans, such as Twinkies and potato chips, not in any technical sense. The term is then applied to conventional pet foods in order to imply that these are nutritionally equivalent to Twinkies and potato chips, which is not accurate.
I am not aware of a legal or regulatory definition for “processed,” but one could reasonably call any food not eaten directly off the plant, out of the ground, or raw as “processed.” My point is that the term itself tells us little about the nutritional value or healthfulness of a food, but it is often used to convey an impression of unhealthful “junk” food, and that is a bit of misdirection that does not promote an accurate, nuanced debate.
Reply
4) I don’t consider that paper fundamental to the point I’m making, though it is part of a body of evidence which seems to support the claim that homemade diets are often nutritionally inadequate when not carefully formulated by someone with the appropriate training. In any case, if there are inaccuracies then pointing them out and seeing that the correction is made in print is exactly the right thing to do. If there are significant, meaningful inaccuracies, than of course a re-assessment of the value of this piece of evidence needs to be made, as is the routine process of EBM.
You are mistaken, however, in suggesting that no one but yourself thought to look at the study “objectively” because of confirmation bias. You have chosen to critique it because it contradicts your “tenets,” and that is precisely how science should work, as a critical community endeavor. In the long run, of course, the truth will become progressively clearer as both you and the paper’s authors and other interested parties contest the facts. Personally, I have no investment in the issue of whether this particular paper ends up being largely, partly, or not at all accurate once that process is complete. For the moment, as I said, it seems to fit a larger body of evidence on the subject, but it’s not at all a closed question.
Reply
5) I criticized your company’s use of the terms “scientific” because they conflicted with your marketing use of the abstract referred to, which very much distorted science for the purposes of advertising. I applaud a genuine commitment to an evidence-based approach, and in other ways your company may well be taking such an approach. In this case, however, I think you blew it.
And I don’t play favorites in that I am equally critical of any inappropriate manipulation of science and the aura of science to sell products with claims that go beyond what is truly supported by legitimate research evidence. Industry is guilty of this at all levels, large and small companies, pet food and pharmaceutical, conventional and alternative products. You aren’t getting any special treatment positive or negative, you were simply included in this article as an example of a larger problem. I cannot physical write an article on every product or every company or every instance of the misuse of science for marketing, so there are undoubtedly many examples I will not remark upon. That does not mean, however, that I am picking on anyone in particular or giving a free pass to anyone else. I understand you will not appreciate being noted, but I believe my reporting is accurate and fair.
Reply
6) I was very clear in my article that information relating to the internal and external validity of the study was missing, and that a full critical appraisal will have to wait for a peer-reviewed publication. That is the gold standard of science, and not final judgment of the value of this evidence can be made until that information is released in that format.
However, it was your choice to present what you did, gaps notwithstanding, and to use it in the way you did, so there is nothing unfair about highlighting the problems with such misuse of scientific research evidence. It is, as I have said, widespread problem, and both veterinarians and pet owners deserve to be made aware of the problems with using science as a marketing tool in this way. I will be happy to present an update when the full study is published, and if it appears to justify the interpretation and the dramatic language used to describe it in the press release, I will certainly acknowledge that. As I said earlier, I have not at all rejected the hypothesis that “fresh” foods (if that can be rigorously, consistently defined) may be healthier than extruded kibble, and in fact I have a slight personal bias towards the idea that they will be. However, my own inclination aside, such a hypothesis needs to be proved through proper research, and at this point I still believe you are making claims not supported by published evidence. I don’t make or sell any food and I don’t have any brand preference, so I don’t have a horse in this race personally, I simply object to inappropriate use of science as a marketing device prior to establishing the facts through the academic process you refer to.
Reply
7) Thank you for this clarification. It was not clear that Dr. Chavez had participated in this research prior to any formal association with JFFD.
Could you also disclose whether there were any other potentially relevant conflicts of interest? Was the food tested a company product? Was it provided by the company? Were there any other forms of financial or logistical support for the study?
Please understand, I am not one who believes that industry funding automatically invalidates research. It is an inescapable fact that veterinary research requires resources that cannot come entirely from non-profit foundations, government, and academia. However, it is also clearly demonstrated in human medicine (and a subject of my own research) that funding bias is one of many potential sources of bias that must be considered in the critical appraisal of research. I am a believer in transparency and disclosure so that this factor can be duly considered as one among many.
Reply
9) This is a rather embarrassingly hyperbolic a way to finish what was otherwise a very useful and interesting comment. Are you really suggesting that by refraining from a final judgment on the health effects of kibble vs “fresh” foods for pets, or by criticizing your use of incomplete research results to announce an earthshaking paradigm shift in pet nutrition that I am the equivalent of an apologist for cigarette smoking?
I quite agree that history, or more accurately the scientific process, will be the judge of this hypothesis, and I am content to wait for that judgment. Science does not make progress without innovators, but it also proves most innovators to be wrong in the end. Many of the greatest minds in science were wrong about most things they believed. Enthusiasm for a hypothesis must be tempered by humility and skepticism, and since it is difficult to have all of these things in one person, science is a community process.
I appreciate your enthusiasm, but that doesn’t equate to accepting your claims before you’ve provided sufficient evidence for them. Far more people are hurt by reckless adoption of inadequately tested practices than by the inevitable delay attendant upon careful, rigorous scientific scrutiny. History may judge me harshly as a late adopter of a beneficial feeding practice, or it may judge JFFD as it has the proponents of thousands of good ideas that proved wrong when closely examined. Statistically, the latter seems the more likely outcome, but either way the important thing is that the truth is reached, not that one or the other of us get the ego satisfaction of having been right.
Reply
Skeptvet,
Thank you, for your more detailed response. Please note we have requested publishing permission of the poster, which is owned by the universities – not us. Also thank you for clarifying the context of you skepticism, it makes much more sense once explained, and in fact – that definition of skepticism is healthy. We sincerely hope you can apply it to this discussion, as if one reflects – extruded diets remain unproven compared to a whole food standard. They have made claims of being healthy and the best available nutrition for many years that are simply untested. For example, there is no study that has tested a whole food formulation against it’s extruded counterpart and measured long term results – yet in general, extruded diets are marketed as “proven nutrition” – compared to what?
We agree “complete and balanced” is proven, but that is not in dispute. We only advocate complete and balanced, whole food, lightly cooked diets that have undergone feeding trials.
We really are a small but highly reputable company trying to do what is right, and where other companies have bypassed vets to market to consumers we have not, and we have made a point to become involved in the work that vets want to see. We have sponsored tens of thousands of dollars in CE seminars for vets that have nothing to do with nutrition. In short, we love our vets and they love us.
Does our marketing department get excited about results as they come in? Sure – we all do, but as you will see on our site everything we claim can be verified and comes from integrity. If you see something that is not accurate, contact us and if verified we will change it gladly.
Reply
I agree, absolutely, and I certainly would like to see this sort of research. You know, better than I certainly, the challenges of conducting such studies, but they would be very valuable.
Reply
Sorry to clarify, the Melamine cases in the literature extend to 2010, it’s not meant to be an isolated event – it is ongoing as was confirmed in April 2014 by this:
http://www.scmp.com/news/hong-kong/article/1482845/carcinogen-found-hong-kong-pet-food-consumer-council
Reply
With respect to the USDA certified, inspected, and FDA/USDA approved process (we avoid “Human Grade” – as it is not legally defined), it is the highest quality standard of food production in the world – that’s all we say. From there, we assert that our diets, being formulated from ingredients that are produced using the best standards in the world – are higher in overall quality and produce a higher quality diet. This is an assertion that is supported by the research.
Conversely, if someone asked you to opine whether complete and balanced extruded kibble made with feed grade ingredients was good enough quality to export to feed starving humans (once the nutrients were adjusted for humans) wouldn’t you want to see some evidence before the first shipment goes out? Maybe research comparing the complete and balanced kibble diet to a complete and balanced USDA approved/certified standard? We seemed to have skipped this step in veterinary medicine. I suppose I would ask what is the evidence going the other way?
Instead, it is our assessment that for the last 80 years, our pets have been that test. Early signs of problems didn’t fix the modality, instead they were labeled “taurine deficiency in cats” or “ca/phos issues in growing large breed dogs”. We now know cats would get plenty of taurine from a meaty diet (except poultry) but that extrusion denatured taurine, thus it needs to be supplemented back in. Our pet population has been a huge field test in extruded nutrition for decades – and we’ve learned a lot, but for some, it is no longer acceptable. However, for many that level of evidence (or lack thereof) is acceptable. Before you accuse a small company like ours of “marketing” a false conclusion, can you look at the entire industry’s standard of proof to say what they have said about extruded diets as a whole, including veterinary prescription brands?
I suggest you contact Dr. Tegzes, a board certified toxicologist and professor at Western University vet school. He has researched our diets (and this topic) and has shared with us, on many occasions, that our food is safer when it comes to testing, allowable toxins, supplementation, and standards than most diets made with feed grade ingredients; just from the source of the ingredients alone.
Reply
You questioned in another post whether Acupuncture always testing favorable in studies represents a bias rather than evidence. I don’t know in either case, but it’s a very interesting question to ponder if this happens in nutrition, and very appropriate for a skeptic. I would say from your comment:
“I don’t consider that paper fundamental to the point I’m making, though it is part of a body of evidence which seems to support the claim that homemade diets are often nutritionally inadequate”
Is it? Or if fundamentally flawed in a simple reference citation that is very easy to access and verify (NRC RA standard for Vitamin D), and allowed to stand unchallenged for a year, then is there more to it?
You will remember that there was another similar issue with a study looking at raw diets by Freeman in 2001. Pubmed lists the publication of an erratum in the May issue on page 1582, a brief mention is there and there and then Freeman cites the detailed correction will be published on page 1716 of the June 1st issue. The interesting part is this page of June 1 issue is missing online, but you can get it from the journal by asking – it is the correction in detail – once again Vitamin D is the issue.
Reply
My area of expertise, apart from general practice, is epidemiology and evidence-based medicine, and I can tell you that the majority of scientific papers published, in human as well as veterinary journals, contain a stunning number of errors and inadequacies. So while they certainly need to be corrected, and I applaud you for taking the time to address this one, I don’t think we can draw any larger conclusion from this mistake without tossing out the bulk of the scientific literature.
You might be interested in these illustrations of the extent of problems with the scientific literature. Quite sobering, though not, I hasten to add, a cause for despair! 🙂
RetractionWatch
Why Most Published Research Studies are False
Reply
With all due respect, you can’t use the melamine incidents to justify why your food is better, or to denigrate commercial pet food. This is a common ploy for manufacturers such as yourself to denigrate other manufacturers. Fear-mongering is a common ploy used in marketing and does not lend to your credibility.
Likewise, the Hong Kong article. Their (and other countries) standards, regulations, quality control and manufacturing processes are different than the US. You can’t use those examples to say one or more bad actors (at least in the US) mirrors the entire industry.
Reply
Skeptvet,
Your article aligned our efforts to begin to answer important questions with “cold-fusion”. You claim “I understand you will not appreciate being noted, but I believe my reporting is accurate and fair.”
We appreciate the discussion, and we hope it has become clear that this wasn’t a fair post – at least not to us. You could have called to find out more, we also provide a video interviewing one of the independent authors and his own interpretation; a board certified veterinary toxicologist. Its not the ‘leap in interpretation’ that you are suggesting .
The researchers and marketers felt the release is appropriate and robust pending further progression through the process, given the data and where we are at this time. We would have loved to share what we could on the phone if you had questions and were genuinely skeptical.
We hope an additional question you will consider following this discussion is: “If they really didn’t care about the scientific process or the claims they are making – and were only in it for the reasons suggested in this blog – would we even be having this extensive discussion?”
In addition, we will make an effort to get reprint permission of the poster and post it for your review. We will keep you updated and thank you so much for your interest in JustFoodForDogs whole food canine diets.
I hope you will contact us; we may be able to send you some samples for review.
Reply
I am happy to review any additional information you would like to provide, with the proviso that I have your permission to do so publically. This is not, of course, a substitute for peer-reviewed publication, which I understand you are pursuing. However, I repeat that there is nothing unfair about critically evaluating the material you chose to make public in an effort to influence consumers. Presumably, not everyone who reads your press release or reprints it as a “news” piece is to be expected to call you personally to obtain additional information. You released the information in a certain configuration to generate a certain impression, and I maintain that is fundamentally misleading.
As I said previously, I will be happy to update this article once the full study report is published, and I will appraise the report as carefully and fairly as I would any other. But until that information is published, and until a body of consistent evidence is available to support your claims, I still believe you press release was an example of the misuse of partially reported, unpublished research to create a favorable impression that might or might not turn out to be consistent with the actual results ultimately published.
Reply
v.t. – As you point out the Melamine issue is ongoing, it is not being used solely for the 2007 concerns. We still have problems today in April 2014. Also did you read the article on the Hong Kong study? you seem unclear….
The brands testing positive for concerning contaminants include very popular USA imported brands to Hong Kong (two of them are brands recommended by vets) – not food made there – it is our exported food to them. One would argue, they may have increased standards if they are testing their imports and we (USA) are testing at levels that are concerning them.
Again, sometimes even with evidence, right in front of us to read, tenets are hard to challenge.
Reply
Skeptvet,
To clarify #9, we apologize if felt that our tone at the start and end of our response had to match the general tone in your post. One way to genuinely engage someone you do not know well in a healthy discussion is “mirror and matching”, thus I hope you will understand it is not our usual rhetoric. Nevertheless the points you make in your response to #9 are very good – and I agree. However to clarify we are not making the comment we are making regarding an unwillingness to adopt another standard, but instead the unwillingness to look at the evidence objectively, that has been mounting up that this entire structure needs to be visited. Aflatoxin B1 is a known carcinogen at sublethal levels, yet is was found only recently at alarming levels in brands of diets that many health care professionals would recommend today as a first line diet. That – is where the analogy is coming from.
Thanks again for the discussion.
Reply
I suppose that would be an issue for them to be working with the manufacturer. I’m not saying there isn’t cause for alarm, I’m saying that due to the differences in regulations etc, there are likely to be problems. And I’m saying that using examples like that to denigrate other manufacturers is not cool. Using it to bring attention to a serious matter, yes. Using it to justify your own claims about your special pet food, no.
The article has a brief table that isn’t exactly specific, one would have to have in had the complete findings to determine the extent of the problem and the average layperson usually has no direct access to that.
The melamine and cyanuric acid: I thought that Purina, among other major manufacturers, addressed their suppliers long ago – if they didn’t, then yes, it’s a problem. It’s not clear in the article what “vets” and which spokespeople for “Purina” are making those statements about short-term vs long-term consumption and acceptable amounts in the products. I would think there is NO acceptable amount of melamine or cyanuric acid in any food but I don’t know how exports work in that regard vs EU regulation limits.
Reply
Someday I hope skeptvet will allow me to edit my horrible typos.
….one would have to have in hand…
Reply
Skeptvet,
Thank you for the opportunity to discuss a very exciting topic and your willingness to revisit this subject as more information is available. That is very encouraging. We are not exactly sure what happens next in the process, as that is all in the researchers’ hands. We are told, however, that the data is robust and sound and that things should move along properly. We will keep you updated as we are made aware of any updates since you have expressed a genuine interest in our diets.
As for “claims” related to our diets, we are not sure what you and v.t. are referring to. The press release is factual and reports what we know at this point and the expert opinions of some professionals involved in the research. “Game changing”, “exciting”, “we may see a reduction in diseases” are genuine quotes from them. They are not “our” claims, but we are thrilled to quote them. On the website, we offer a comprehensive report of peer reviewed literature as a justification of why we choose to use only the quality of ingredients we choose to use, so as to avoid being perceived as ‘gimmicky’. There is real scientific reasoning behind our choice to incorporate only USDA certified meats and ingredients approved for human consumption into our food at the complete exclusion of feed grade ingredients. Concidently, Congress had a hearing on the matter this week and is also very concerned: http://www.cecc.gov/events/hearings/pet-treats-and-processed-chicken-from-china-concerns-for-american-consumers-and-pets
Is all the evidence in? Are we done investigating? Are the benefits definitive? Should research on the matter stop? No – but did we say that? Or even imply it? This is the beginning and it’s exciting.
We cite all our references, and we don’t “claim” anything that is cannot be verified; and you have agreed to many of those criticisms of extruded diets that we’ve listed. We ask questions that some others refuse to ask and we use evidence to ask those questions – an approach a skeptic should appreciate. There are copious testimonials from boarded specialists and clients on our site, but those aren’t our claims, they are their reports. None of the people that provided those testimonials or videos were paid to do so.
We have presented evidence to justify our methods, and our diets have been involved in some research that is promising to our modality. A press release has communicated this to the public factually, and some opinions were also included. Language such as “shown to”, “exciting”, “may suggest”, infers that we know exactly where we are at this junction in the scientific process and so do the researchers. The very title of the press release says “may be healthier”, which you admit may be true. You present your arguments as if we’ve tried to portray or “mislead” what has happened, and that is simply not the case anywhere.
There are many companies out there that do make false claims, have no research, no feeding trials, no vets, and would not even pay attention to your blog. We are not one of those companies. Please do not confuse our genuine efforts to promote discovery in veterinary medicine and nutrition with the false claims that other companies make.
Instead we are serious and we will continue to support research in nutrition as long as we are able to do so. We appreciate this opportunity to discuss our company because – like you – we do feel this modality is likely to prove to be a healthier option when properly formulated. Like you, we eagerly await the future. At the same time we must respond when someone has misunderstood or misrepresented our efforts.
Thank you.
Reply
I guess I’m not persuaded by how your dog food is any more special than that of other pet food companies making nearly the same claims for home-cooked, frozen or raw pet foods – other than your repeated use of the term “USDA-human-grade” meats and nutrients and the term “lightly cooked”.
You sell supplements (so do other pet food companies) but yours are special because they are manufactured in an FDA-approved facility? Are the herbs in your “Calm” formula any more special than any other herbal supplement company making the same claims?
You sell “recipe kits”. The owner buys packages of nutrients from you but is expected to buy the bulk of the ingredients. How is this different from other companies selling home-cooked, frozen or raw food packages with instructions for supplementation?
You also dedicate space on your website the same rhetoric and fear-mongering as other pet food companies, as your justification that yours is better. Quoting Ann Martin, Martin Goldstein and the like and propagating the “euthanized pets in pet food” hysteria, naming the major pet food manufacturers who “are invested in the dog food business”, implying said companies are conspiring to suppress nasty secrets from the average pet owner. Are you also not “invested in the dog food business?”
Isn’t human-grade also that of beef tongue, pig’s feet, head cheese, frog legs, fish eggs, rocky mountain oysters, insects and a plethora of other disgusting things that humans eat every day?
Frankly, with all the rage about “human-grade” dog food, particularly the cooked fresh, shipped frozen and ready in an instant, seems to me just a new way to market Doggy TV Dinners. Granted, I’m sure there is a better production standard involved, but we’ve seen time and again that no one is immune to trial and error in production, (contamination, etc) and neither will you be.
Reply
On behalf of Nigel Warner (blocked by spam filter):
I’m interested in the raised white cell counts, if the dogs had normal baselines but counts were raised whilst on the diets then one must ask why? One expects raised counts when fighting infection, allergy etc so why do they consider raised counts beneficial? If this was a Neutrophilia, Esonophila the question must be what were the stressors infection, allergy? Then there’s the raised Creatinine again what one would expect in dehydration, kidney injury etc! In truth as all remained within normal values they are of limited relevance so likely no harm done but how do these results become positive??
Reply
Nigel, I wondered that about the WBC’s as well, would love to hear their explanation for that one.
Reply
Skeptvet,
I’m please to inform you that the researchers have released the poster as presented by Dr. John Tegzes VMD DABVT to JFFD:
http://justfoodfordogs.com/wp-content/uploads/2014/06/JFFD-Research-Whole-Food-For-Dogs-Poster-AAVN.pdf
We also posed Nigel Warner’s questions and some other concerns. We were told that the full discussion will be available after peer review, so we will have to wait for details. At this time, we can address some of those questions in that we understand there was no pathology common to the group that could explain the results. Furthermore, a perceived immunosuppression (outside reference intervals) was observed at day zero. While it’s true that changes in values within the normal range in healthy dogs is not necessarily exciting, some values were outside normal ranges initially, then seemed to normalize (p<0.05). CREA is a product of creatine metabolism and can be found in higher concentrations in meat diets vs non (real) meat diets. It is a nonessential nutrient that is absorbed in the GI track just like other nutrients (human athletes take it as a supplement). It is known that more real meat in a diet could raise mean serum CREA levels, if by nothing else, by simple absorption in the GI track. CREA means increased but no values were in pathological range.
V.T. as far as fear mongering – we can appreciate your interpretation. A variety of opinions is healthy is skeptical scientific review. We feel what we do is simply justifying our modality. If the peer reviewed facts behind that justification is scary – then we agree it shouldn't be ignored, and we communicate that as well. Arguably, some commercial pet food companies, including studies on homemade diets, use a similar approach. Many of their point of view is that if you feed anything but kibble you might be harming your pet. In fact there was a time "people food" was absolutely forbidden by vets. We no know that's not really necessary, and only some foods are.
It's always important to look at the source of information, and then decide.
As already stated, time and history will continue to provide facts, and we may know more certain info then – until, of course, it all changes again.
Thank you for the discussion. We appreciate being part of the conversation on whole food nutrition for pets.
Reply
I appreciate the access to the poster. It does not include much of the additional information needed to really evaluate the trial, but it does provoke some interesting questions.
1. The legend for Fig. 1 does not explain the details of the figure, however I presume the triangles represent the mean (at the tip) and the lines represent the range? Clearly, there was quite a range, and though the means increased at each time point, the differences were small and of questionable clinical significance, and there were certainly individuals with lower values at the end of the study than the mean values at earlier points. I think it is a stretch to call this a clear trend even in these values.
2. What would be your theoretical hypothesis for why these values changed and what their clinical or health significance would be? Is it necessarily a good sign that these values increased? It would strengthen your case that these differences are meaningful if you had a priori predictions for which values would change significantly and in which direction.
3. Without a control group, of course, it is not possible to say that the diet was responsible for any changes seen. If individual animals had values below the normal range at the beginning of the study, perhaps they simply corrected over time with the resolution of the original cause regardless of diet. Certainly, it is appropriate to use uncontrolled trials to generate hypotheses, but not to support or validate them.
4. There was no mention of a couple of key statistical issues. Were the analyses done in a way that accounted for repeated measurements or the lack of independence of the data points? Was there any adjustment or correction for multiple comparisons, such as a Bonferroni correction of the critical P-value? A p-value of 0.05 means there is a 5% chance of reaching statistical significance without a real difference in the actual population means, so with 26 pairwise comparisons a couple of significant differences would be expected by chance alone. This is another reason why a plausible theory to establish a priori probability is important in interpreting these results.
5. It is true that eating cooked meat can transiently increase creatinine levels, though this effect only lasts few hours and should not affect fasted creatinine levels which reflect underlying glomerular function. Were the dogs fasted before blood sampling in this study? If so, for how long?
6. The dropout (from 21 to 9 dogs) at 365 days was substantial, more than half the subjects? Any information as to why these individuals dropped out or whether they differed in any way from the individuals who stayed in the study? This would obviously have a significant effect on the results at 365 days, which represent about half of the statistically significant differences reported.
7. What do you mean by “real meat?” Are you suggesting that the meat used in the test diet has some property that would contribute more creatine or raise Creatinine levels more than the meat used in other diets, such as extruded kibble? What would be the salient characteristic? Is there specific evidence for this? And, of course, is this necessarily a good thing from a health perspective?
Reply
I’d say Jack (t=180), Duke and Daisy (t=365) were statistically significant in their own regard.
I too would like to know why so many dogs were dropped from the study at 6 months. Were any owners wanting to drop out even sooner? (since the study “required” commitment to 6 months at least)
Reply
Skeptvet,
The full discussion and data has not gone through peer review yet, and the way we understand it the posters in these academic settings are purposely vague so as to stimulate discussion in preparation of the manuscript. We are told more data and discussion is likely to come. We will keep you posted as you have shown genuine intrigue. Thank you!
With the caveat that these are only offered for healthy discussion, and by no means the opinions of the authors – we can try to address some of your queries, as this has developed into a very healthy skeptical analysis; which is highly appreciated.
1) Yes, the triangles are the means, and the lines the ranges. If not specified in the figures, it is probably discussed when presented. We could be wrong, but the ranges and spread of the data is probably not of value except in order to show overall trends of mean values when WF diet is the only intervention. You can try to make more sense of the spread and distribution, etc – but not sure what else it shows other than arguably a below range hypoglobulinemia for a significant potion of the data at t=0, relative compensatory out of range high ALB, and some values of WBCs below the minimum as well. Those numbers outside the RI seemed to correct with intervention at t=180 and t=365.
2) For this question all we can do is direct you to the interview with John Tegzes from the press release, as that is the official word as to ‘impact’ for now. It was part of the press release as well: https://www.youtube.com/watch?v=1HIMklYNIic
Dr. Tegzes explains possible impacts of the trends. Our understanding is that this is the start of the investigation process. A foundation for hypotheses as you state. Lifetime studies tracking disease and death rates between a WF diet and a EX control could be very interesting and would be definitive. Science has to start somewhere.
3) Not sure I fully understand this question – please excuse if I misunderstand. Paired t-tests for analyzing trends in blood parameters are arguably the only way to make sense of changes in the same group of animals because it uses the same population of dogs. The study was a clinical field study – dogs were a colony of home owned dogs that all had different environments and owners, thus the only commonality (other than living in Southern California) was diet intervention and 3 vet visits to collect blood, spread over time. These were not lab dogs. You would have to believe they were all affected by the “something in the water” that caused the profile at time zero, that corrected over time. It is true that the cohort effect you postulate would be conclusively ruled out with a concurrent control – and that is stated by the authors. However, the data stands alone given the details of the methodology. Data showing trends this specific, across different but corresponding blood parameters painting a clear picture, is robust.
Nevertheless, if the data only proves to postulate additional hypotheses – like you say it does – that alone would be groundbreaking in nutrition. As far as we know no one has done this – started the investigation on the impact of nutrition beyond EX diets. This is the definition of groundbreaking.
4) Again, please excuse me if I don’t fully understand this question. The authors may have analyzed the data in many different ways, we don’t know. What we do know is that they settled on paired t tests using SAS – so that each dog can serve as it’s own control and experimental unit. I believe this is standard when looking for changes in blood parameters in the same group of animals and seeing if there’s any significance to them. My understanding is that a paired t test looks at the group as a whole, including all of its variation for each individual parameter, and compares the same group, and all of it’s variation for the same parameter, at another point in time. Thus at t=0 21 dogs had a certain mean and spread (inclusive of a given variation between them) for GLOB, for example. The same 21 dogs had a different mean and spread at t= six months. The paired t test looks for the probability that the new mean and spread is a chance event, taking into account the population’s own variability and statistical characteristics at t=0. It is a very robust measure for exactly this type of data comparison for the same group at two different points.
The P<0.05 isn't exactly as you describe for the t-test (unless I'm not understanding you). Instead, for a paired t test one sets up a null hypothesis. In this case one could be "Any variation or changes in the data is a chance event; feeding WF diets is not the cause of the changes in blood parameters." The t test gives you the probability that this null hypothesis is correct. A resulting P<0.05 means it is there is a less than 5% chance that a random landing of data points could give you this specific spread of data from the same population for that parameter, given all of it's statistical variation, six months later. Thus at P<0.05, the null hypothesis is rejected and the premise that "something else", another intervention common to the group, must have caused these changes. In this case, it is diet.
It was interesting for us to learn that while P is set at <0.05, many of the trends measured at a P value well below 0.01. IE: there's less than a 1% chance that the data points landed randomly (given no other common intervention to this population) to provide this specific spread of data for those parameters. Also, if you look at the physiological trends as a whole, a picture is painted that is difficult to "unsee" once appreciated.
I think a concurrent control would be great, maybe with an ANOVA, to rule out cohort effects, but a paired t test will still likely be the most robust measure to evaluate the significance of the actual changes within the experimental group.
5) Fasting/non fasting was not specified as far as we know. But as this was never required (before or after) it is probably a random effect – if any – to the data. Similar to how you would expect in a vet hospital. Sorry, we don't have more information on that.
It is well known that extrusion denatures some nutrients, and thus they are added back in after processing. AAFCO requires this consideration specifically for thiamin, lysine, taurine (cats), and other vit/min that are known to be effected. They are monitored and accounted for because they are essential. Since creatine is a non-essential nutrient found in muscle, if destroyed during extrusion, it will not be replenished. Thus real muscle meat ie: flesh, raw steak, etc. Is likely to maintain its creatine (and CREA) intact when lightly cooked versus extruded.
6) The data was collected from feeding trials and JFFD only had a certain amount of recipes to 'test'. Five to be exact. Thus, the first set included 3 diets (more food, more dogs), but there were only 2 diets left for the second set (less food, less dogs for round two). This, and the fact that participants were only asked for a six month commitment, made for the smaller sample size for the 365 group. Some participants were no longer needed, or couldn't commit to, or didn't want to take on, another rigorous six months of reporting and monitoring data (it was alot to ask, given holidays, vacations, lifestyle etc). As far as we know, no one 'dropped' at the six month mark from food related issues, and in fact many of the participants who were let go became customers and still are today.
I suppose the point is there was nothing 'fishy' about the reduction in numbers, and n = 21 or n=9, even at 6 months are pretty impressive numbers in veterinary longitudinal studies. Again – can be considered groundbreaking.
7) Already answered. Real meat refers to actual real food. Our chicken and rice formula uses USDA Chicken Breast, USDA Chicken Thighs, USDA Chicken Gizzards, etc. The CREA trapped in these meats is likely intact and simply adding (through GI absorption) to the circulating baseline CREA from metabolism that is normal for that dog. Unless the kidneys aren't working, this shouldn't be of consequence, and there is definitely no data to show that this is an adverse effect on the kidneys. The likelihood here (although we will have to wait and see the official interpretation), is that we will learn that nutrition impacts the normal distribution of reference ranges for healthy dogs, with no other consequence. This has been shown in comparing different populations before, including retired greyhounds, for many different parameters. This is also why in vet med they always use a lab's reference range for a geographical region – not some standard reference.
Thank you!
Reply
Thanks for this additional information.
1) The point of trying to clarify the degree of variation within the study population is that it helps us decide whether any changes seen are consistent effects across subjects or the result of individual variation. If a few animals showed changes which shifted the mean but the spread included essentially the same range of values, that would support random variation more than a predictable treatment effect.
2) This video is an example of exactly what led me to write the original article and what I find disturbing about the promotion and interpretation of this preliminary, uncontrolled study. Dr. Tegzes attaches implications and interpretations to the data that are not in any way justified by the data itself, which is the definition of “spin.” As he acknowledges, there were no a priori hypotheses about the specific values measured, no underlying plausible theoretical rationale for why specific values would change in particular directions. Trials that begin with the idea of measuring everything regardless of probable relevance inevitably find statistically significant but clinically meaningless differences, as this study appears to have done. Yet such differences are inevitably interpreted as supporting the beliefs of the investigators, and this kind of over-interpretation of scientific data without respect to its limitations leads to confusion, not clarity.
Dr. Tegzes also suggests that the diets the dogs were eating before the trial taxed their immune systems, and he suggests that the JFFD diet relieved that. There is absolutely nothing in these data to support such an interpretation. If anything, an immune system reacting to chronic stimulation should have elevated globulins and white blood cell counts, yet Dr. Tegzes implies that low values suggest chronic immune stimulation and that the increase seen during the study suggests a beneficial effect of the food. It would be more consistent with the usual response to challenge to suggest that the food contained some pathogens to which the dogs responded with an increase in globulins and white blood cells, and there is no evidence to distinguish these two explanations, so the interpretation given is purely a reflection of pre-existing belief, not a fair interpretation of the data.
And again, the fact that the values were almost entirely all within normal references ranges suggests the changes were not clinically relevant. To say the values moved to the “healthier side of normal” just doesn’t make any sense. Dr. Tegzes goes even further by suggesting that the apparent increase in the incidence of chronic disease with increased lifespan is somehow preventable by diet (which may or may not be true), and that these data suggest a trend towards greater health and less chronic disease with a switch from kibble to the JFFD diet (which is absolutely not a reasonable interpretation of the data at all).
As I said earlier, I am all for exploratory research to generate hypotheses for more rigorous testing, and I am open-minded about the idea that “fresh” food might have health benefits over extruded kibble. But this kind of over-interpretation of very limited data is not responsible or appropriate, and it is part of a deep problem in medicine in which research is misused to buttress beliefs or claims that it does not actually support. Mixing marketing and science in this way degrades the legitimacy and the value of the science in generating better understanding, and it is unfortunate.
3) My point was that the statistical methods for analyzing repeated measures from the same individual subjects differ from those used to compare independent measurements from different subjects, and I was just asking for clarification that a repeated measures analysis was done here. A paired analysis is appropriate for comparison of two values, but since there were three values for many subjects, a repeated measures analysis would have been more appropriate. Also, an aanalysis which takes into account other potential confounders (breed, size, age, sex) would strengthen any conclusions.
That said, I do not at all agree that these data show specific trends across parameters all of which support the same hypothesized treatment effect. As discussed elsewhere, I think the data are far more compatible with the hypothesis of random variation, and it will take a controlled study and replication to suggest that these specific changes have the meaning you are choosing to impute.
4) My question was related to multiple pairwise post-hoc comparisons and the need for a family-wise error rate. The t-test is calculated in the same way, but the assessment of significance is different when one is making a limited number of comparisons specified in advance and when one is “fishing” as done here, where one has no a priori prediction but simply measures and compares a large number of variables looking for those that differ. If we do not correct the significance levels for such multiple post hoc comparisons, the interpretation of the result is subject to bias because we have made deliberate choices about which comparisons to look at post hoc.
Generally, corrections of this sort result in a much more conservative critical p-value, which reduces the number of values reaching statistical significance. One of the most common, and most conservative methods would be the Bonferroni correction which simply divides the critical p-value by the number of comparisons. In this case, that would be 0.05/26=0.002, so no result would be considered statistically significant unless P<0.002. This would lead to the rejection of all your comparisons. There are, however, less conservative methods, so I was just wondering if any correction for multiple post hoc comparisons had been made.
5) The question about fasting was just a response to the suggestion you made that creatinine levels increasing might be a result of some difference in the meat content of the JFFD diets compared to previous diets. Since ingestion of meat only effects creatinine for a short time, and fasted levels should reflect the underlying level rather than a post-prandial effect, I just wondered if the samples collected were fasted. If not, it might be that the differences in this parameter simply reflected sampling at a certain time point after a meal rather than any real change in baseline creatinine due to diet.
6) I wasn't suggesting anything fishy, but loss to followup in any longitudinal study is a source of bias that has to be accounted for. When the results at T=365 are based on fewer than half the individuals tested at t=0 or t=180, these are different populations, and some of the differences might have affected the results of the study. Without some evaluation of this issue, the impact of selection bias cannot be accurately assessed.
7) Sorry, but terms like "real meat" and "actual real food" reflect value judgments, not scientific distinctions here. The meat in extruded kibble isn't "fake meat." The issue is whether the differences is source, cut, and processing affect the health of the individuals eating the different forms, and that is by no means established. If, as you suggest, the only effect on creatinine levels is to shift the mean slightly within the normal range (which, again, it is not at all clear from these data is a consistent, predictable effect of these diets), then why do you and Dr. Tegzes continue to suggest the changes represent or suggest dramatic health benefits might be gained from switching to this type of diet? Once again, while I have no quarrel with the study or the data, I think your interpretation is clearly driven by pre-existing beliefs rather than the data themselves, and while this may serve a marketing purpose, it doesn't advance our genuine understanding of the health effects of different kinds of nutrition. I understand that you believe what you believe, and given that it is difficult not to generate and interpret data in a way consistent with those beliefs. But the whole point of scientific research is to separate our beliefs from the actual reality of nature, and it only works if we are careful and strict about our methods and if we are circumspect in how we interpret the results of such studies.
Again, I look forward to more details emerging after peer review, and to further research to clarify whether there is actually a real effect here to be understood.
Reply
v.t – it’s interesting you mention Daisy and Duke.
They are still customers today – never went back to kibble. As far as we know, neither did Roxy, Biscuit, and probably some others – but I don’t have a full list.
Sorry – but there’s just nothing juicy there.
As an aside, we are told the researchers considered whether to give actual names (not usually done). We are thrilled they did as it shows these were real dogs with real owners, living happy lives eating our food.
Thank you.
Reply
I was thinking of the significance in weight of those 3 dogs, and in particular, Daisy and Duke.
I watched the video you linked to. I’m sorry, but I fail to see how that video answers skeptvet’s question #2. All I heard in the video was a lot of focus on the “immune system” but no explanation to support the remarks:
“benefit the immune system” (but doesn’t explain how).
“unexpected findings in a good way” (but doesn’t explain what).
“had an effect on the animal’s immune system in a positive way” (but does not explain how).
“not one animal deteriorated or worsened, all animals improved, ended up thriving on the diet, improved weight“. (Are you saying Daisy and Duke’s significant weight gain is a healthy weight gain? What was their starting and ending BCS?)
“overtaxing of the immune system” implying this was the result of kibble fed diets prior to study and “the overtaxing of the immune system went away” after being fed your diet. It begs the question if all 21 dogs had health issues, chronic issues, or other issues that you can contribute so much “improvement” from feeding your diet.
The repetition of “immune system” remarks is telling and commonly used as marketing gimmicks. To me, the video seems more like an infomercial.
Reply
Skeptvet and v.t.
Thank you for the discussion, unfortunately the rest of the details we will have to wait for, so we will close our discussion here. In fact, the resolution of this topic will likely require time as well, but it should be resolved. We are thrilled to be part of the process, and respectfully disagree with some of your subjective assessments as to the ‘intentions’ of our communication; but appreciate your opinion.
As such, we do believe here at JFFD that extruded diets are not made using real meat as you and I eat. In fact feed grade (rejected from human consumption) meat is arguably not the same. Again, perhaps semantics, and not necessarily a ‘scientific’ distinction of meat (yet) – but nutritionally they are absolutely not the same. A premium cut of steak is not equivalent to beef meal or extruded beef product in digestibility, bioavailability of its nutrients, and composition (maintenance) of nutrients after extrusion. You can interpret our language as ‘marketing tactics’ if you like, but I will pose to you they are as much “marketing” as the pictures of steaks, fruits, and vegetables that you see on the packages of extruded diets. Scientific skepticism does include a realization of ones own bias – correct?
Lastly, to address your concern on the Bonferroni correction, the serum protein changes (Alb, Glob, A/G ratio – and perhaps others) were significant to P<0.001 (we believe by a few more multitudes of 10 even in some cases, but we don't have the actual numbers). We were told this at the time the results were shared with us and that for simplicity the poster would default at P<0.05 (or P<0.01). Your comments on this correction in order to provide even a higher level of significance is a good one! I will pass it on the the researchers in case they have not already considered it. It may be that they already have this in mind, and were mentioning this when going over the results with us, but if so we missed the detail on their explanation.
On a final note, it is interesting the level of skepticism by which you approach this data (albeit not yet published), and the lack of questioning you displayed regarding the published study on homemade diets that you wrote about in agreement. Had you analyzed it with the same level of scrutiny (which we appreciate and even found useful) you would have caught the Vitamin D citation error, which is rather a large portion of their discussion. It has recently been revealed here:
http://truthaboutpetfood.com/oops-big-error-found-in-university-pet-food-study
Nonetheless, the error persisted for one year despite your skepticism and peer review. I suppose the lesson is – we should never stop being skeptical, even after peer review and publication – and always be questioning. Our presentation of the data, press release, etc is actually consistent with your skepticism as we ask the question: "Is this (EX diets) the best we can do in veterinary nutrition?"
Thank you all!
Reply
Thanks to you as well for engaging on the subject. Always the best way to work through differences in perspective.
And there is no question I have my own biases. We all do, and I have written extensively about the fact that this is why rigorous adherence to the methods of science is so critical; it is our best protection against bias. In this context, I suspect my own biases are not what you might imagine. I am not conscious, at least, of any bias against the notion that so-called fresh foods might have health benefits; in fact I am slightly inclined to think this will prove true. I am, however, particularly sensitive to the misuse of science to support beliefs and claims not truly justified by the evidence. It seems this weakens both our knowledge and our confidence in our methods for obtaining this knowledge.
As far as the paper discussing potential inadequacies in homemade diets, I have no opinion on the issue of whether the Vitamin D requirements were misstated as you suggest. I have seen this question raised elsewhere, and I have not yet seen a response, so I will provisionally take your word for the mistake. As I said previously, however, this seems a small though important error, and if you have found no others despite an obvious effort to do so, it hardly seems like it invalidates the entire paper.
In any case, the conclusions of those authors seem a good bit more measured and circumspect than the promotional materials regarding the JFFD study that initiated this discussion. My particular concern is the misinterpretation, and frankly the misuse of scientific research to promote a point of view or a product, and while both studies merit equal scientific skepticism, I started this discussion because of my concern with how the results of the study of JFFD foods were being presented and used, and that issue does not seem to apply to the paper you mention.
The Conclusions of the paper you refer to:
The language of your press release:
Reply
It remains that far and away the primary, and frequently only, nutritional disease the typical companion animal veterinarian is called to treat in his/her career is obesity. That alone should lead to caution in attempting to quantify benefit from dietary manipulation.
It is also so easy to make nutritional claims of better health, longer lifespan or disease prevention when the consequences of marginal changes in diet may take years or a lifetime to discern. Our household pets live long lives today, no doubt due to a variety of better care choices than those made in past decades. With so many factors in play the debate about what role nutrition has played in this lifespan increment will likely never be conclusively settled.
Reply
JFFD, I too appreciate your willingness to come forward and discuss ‘skeptical’ concerns, and as you say, will just have to wait to see the publication and future research.
Reply
All good points. Just to address the last comparison, Skeptvet, it is not entirely accurate to compare a published peer reviewed paper from an independent university with a private company’s press release.
Instead, if you are going to compare our efforts to this study, you should look at the UC Davis press release as a more accurate comparison of language: http://news.ucdavis.edu/search/news_detail.lasso?id=10666
Thank you.
Reply
Agreed. The press release from UCD seems to take most of the quotes directly from the paper, so it still seems a bit more reasonable and measured than the PR from JFFD. In any event, we’ll all be waiting for the published version as well as additional research on the subject.
Reply
I am not conscious, at least, of any bias against the notion that so-called fresh foods might have health benefits; in fact I am slightly inclined to think this will prove true.>>>
Also locally grown food ?? I think their is science for that,at least for better taste.
My spin sales pitch on all this is “single source maintance diets are a risk factor for disease.” I think I got that quote from Paul Pion who found and published that Hills c\d caused cardiomyopathy in cats. I had a clinic cat fed only Hills c/d die of cardiomyopathy right before Pion published. It made sense to me I should have at least alternated brands of urinary adjusted food. So in my mind, no matter what diet a boarded veterinary nutritionist, pet food company, or the latest new diet “expert” has formulated (over formulated ???) for your pet, you run a health risk feeding that same formula day in and day out.
Reply
Seems a reasonable hypothesis to me, and I tend to agree. I’m not aware of specific research, though, showing a single source commercial diet is more likely to lead to morbidity than a rotation feeding practice.
I also prefer locally sourced food for environmental reasons and for the taste, though again I know of no evidence it is nutritionally superior. There is a lot of evidence to show that organic food is no better nutritionally than conventionally produced, and I would once have assumed it was, so as always we have to be wary of accepting reasonable ideas only because they are reasonable.
Reply
A quote my notes say is by Dr. Paul Pion and a reference.
“The conclusion that cats fed a single commercial food exclusively were at greater risk for developing taurine deficiency and DCM than cats fed a variety of foods is not unexpected. This and other examples of diet-induced disease should serve as a warning to veterinarians who prescribe or endorse the feeding of 1 food exclusively to any animal, especially for maintenance.”
Reference:
Journal of the American Veterinary Medical Association, July 15, 1992: 267-274
guest
Reply
When and where was it published that C/D caused cardiomyopathy in cats? C/D was and still is regularly prescribed for long-term and even for the life of many cats with FLUTD. I have to assume this was long ago and that the Rx diets are properly supplemented with taurine, as fed as “single source only”, right?
Reply
We also have to bear in mind that we must balance known and unknown risks. There may very well be a risk to feeding a single diet only, but the evidence is limited as to how much of a risk this is and for which populations. On the other hand, recurrence of oxalate bladder stones in a cat who has had them previously is a known, and quite high risk, so it might be justified to recommend a single diet that reduces this risk even if it may come with other, unclear risks. Nothing in physiology is free, and any therapy that works is going to have risks, so we have to do the best we can to balance these with the information available at the time.
Reply