Evidence Update: Old Tricks Used to Massage Neutricks Study Data

A few years ago, I reviewed a nutritional supplement called Neutricks which is marketed to “support brain health.” The company is careful not to make any specific claims about prevention or treatment of disease, which would be illegal, but the marketing materials strongly imply the supplement can help prevent or treat Canine Cognitive Dysfunction (CCD), a type of dementia seen in older dogs. In my previous review I concluded:

Could it work? Sure.

Is there clear evidence it doesn’t work? No.

Is there any significant evidence of risk? No.

And finally, is there any meaningful evidence of beneficial effects? Nope.

So while I certainly would love to see additional, and more relevant testing of apoaequorin and the underlying hypothesis behind its  use, at this time it is just another example of selling wishful thinking to people without a lot of better options.

A new study of the chief ingredient, apoaequorin , has been published which claims to support the use of Neutricks.

Milgram, NW. Landsberg, G. Merrick, D. Underwood, MY. A novel mechanism for cognitive enhancement in aged dogs with the use of a calcium-buffering protein. Journal of Veterinary Behavior. 10 (2015) 217e222.

The authors of this study conclude:

The current results demonstrate selective cognitive benefits from use of apoaequorin in an aged dog model… Further work will be needed to better define therapeutic benefits from apoaequorin in the aging canine… Further controlled studies in clinically diagnosed patients are needed to validate which signs might be improved and whether apoaequorin may have an effect early in cognitive decline.

This is a fairly modest conclusion, but I believe it still goes farther than what is supported by the data presented in the paper. The study consisted of two experiments in older laboratory beagles, and it was quite similar to a previous study conduct by the same organization and at least some of the same researchers, which I discussed in my previous post. In brief, this study compared two doses of apoaequorin (2.5mg and 5mg) to a placebo by giving the dogs in each of these three groups two cognitive behavioral tests (a delayed non-matching to position task (DNMP) test, a discrimination task, and an attention task). The investigators also ran a second experiment comparing 5mg and 10mg of apoaequorin to slegilene, a medication approved and marketed for CCD, using the same discrimination and attention tests. The results reported were as follows:

Experiment 1-

1. There was no difference in performance on the DNMP test between any of the three groups.
2. On the discrimination task, there was a statistically significant difference between the low-dose group and placebo but not between the high-dose and placebo or the two apoaequorin dose levels.
3. On the attention task, overall there was no significant difference between the groups. When the data were manipulated to compare the groups under different testing conditions (the number of distractors used during the tests), then a statistical difference could be seen between the placebo and the high-dose group under some conditions, but no difference could be found between the placebo and the low-dose group.

Experiment 2-

1.  In the discrimination task, the most errors were seen in the selegilene group, followed by the low-dose apoaequorin, the control group, and the high-dose apoaequorin. Only the difference between the selegilene and high-dose apoaequorin groups was statistically significant.
2. On the attention task, there was no overall difference  between the groups. Again, when additional variables were factored in (age and cognitive level at baseline of the dogs) a statistical difference was found between the high-dose and selegilene groups, but not between any of the other groups.

So what do these results mean? Essentially, they do not show much of an effect of the apoaequorin or provide much reason to use Neutricks in actual patients. Here’s why:

A. Most of the comparisons between apoaequorin and placebo of selegeline showed no difference. Those that did were produced by manipulating the data to add other variables or compare different subgroups. This can be a legitimate way to identify differences if planned in advance, but it does not seem like this is what happened.

B. If the investigators got negative results and then conducted a bunch of additional or unplanned analyses and reported those that showed a difference while ignoring those that didn’t, this is a bit of statistical sleight-of-hand. There is no way to know if this was what happened since veterinary studies and their protocols are not published in advance. However, such data mining is common practice, and it can create the impression of findings that do not reflect reality, so such post-hoc analyses are always viewed with skepticism.

C. Those findings that were statistically significant don’t fit a consistent pattern suggestive of a real biological effect. In the previous study, which was very similar to Experiment 1, there were no differences on the DNMP task or the attention task, and the low-dose group appeared to have fewer errors on the discrimination task than the control or high-dose group. In this study, there were again no differences on the DNMP task or the attention task (except for some questionable post-hoc analyses), and the low-dose group did better on the discrimination task than the high-dose or placebo.

In Experiment 2, however, the high-dose group appeared to perform better than the selegilene group, while the low-dose and placebo groups were the same for the discrimination task, and no differences were seen overall on the attention task.So basically, a handful of inconsistent statistical differences were found which did not fit a consistent pattern showing a dose-response or superiority to placebo. This might be enough to warrant further research, but if this is the best that multiple studies have come up with, it is not very persuasive.

D. The real-world significance of even the limited findings in this study are unclear. The relevance of these cognitive tasks to normal life function in dogs with CCD hasn’t been demonstrated.  And while there is limited experimental evidence of an effect for selegilene, it is widely regarded as not very effective in real patients, so showing equivalence or superiority to it doesn’t add much support to the rationale for trying apoaequorin. CLinicla trials in actual patients would be a better kind of evidence to suggest real-world benefits.

E. There is a significant risk of bias to be considered in this study. The investigators are either employees of the company marketing Neutricks or of the research firm hired to do the study, CanCog, which markets itself specifically in terms of working to provide supporting evidence to help companies market veterinary products. While I am sure the individuals involved all have appropriate ethical standards, their positions and employment virtually require that they have at least unconscious bias in favor of the product they are testing.

The manufacturer of Neutricks not only funded the study but was apparently actively involved in its conduct:

This study was funded under contract from Quincy Animal Health and conducted by CanCog Technologies. The sponsor helped plan and approved the study design and was also consulted in the decision to submit and publish the article and provided input in the final content of the article…

This provides a lot of motive and opportunity to subtly influence the outcome. Some controls against such bias are present in the study, such as the use of a placebo, but others are not specifically described (randomization of subjects and blinding of investigators), and as previously mentioned a number of post-hoc manipulations of the data were employed which could easily allow unconscious bias to affect the results.

Bottom Line
This new study does little to support claims of real-world clinical benefit for dogs with CCD taking Neutricks. There is some theoretical and preclinical evidence to suggest apoaequorin might be beneficial for dogs with this disease, but the studies so far presented by the manufacturer of this supplement have weak and inconsistent results and some concerning potential for uncontrolled  bias. The experimental models used in these studies might or might not be relevant to naturally occurring disease even if the study outcomes were strong and consistent, but with relatively weak results, it is difficult to know whether this product is likely to have any meaningful benefits.

Undoubtedly, people will come forward with anecdotes and testimonials for Neutricks that supposedly show real-world benefits. Unfortunately, while these stories are very psychologically persuasive, they are even less reliable a guide to the truth than the lab animal studies I’ve discussed. Below are some links that illustrate why in more detail

To be clear, I am not saying this product or compound are not beneficial, since there is not sufficient evidence to make that assertion. However, as of now there is also insufficient reason to claim that it has any benefits, and the question is unlikely to be settled without independent, well-controlled clinical trials in real CCD patients.

Why Anecdotes and Testimonials are Unreliable

Don’t Believe your Eyes (or Your Brain)

Medical Miracles: Should We Believe?

Testimonials Lie

Alternative medicine and placebo effects in pets

Placebo effects in epileptic dogs

Medical Practices Once Widely Accepted that Proved Ineffective or Harmful when Studied Scientifically

Why We’re Often Wrong

Share on Facebook

Tweet

Follow us