Why Anecdotes & Testimonials Can’t Be Trusted

By far the most frequent response I get to any article critical of claims for an alternative therapy or focused  on the lack of good evidence for such claims is an appeal to anecdotal evidence.  Some variation on “I tried it and it worked for me” or “Don’t knock it ’til you’ve tried it” is a common and, for many people,  persuasive rebuttal to even the most carefully researched and objective, science-based critique. Since I end up responding to such anecdote-based rebuttals almost every day, I have written frequently about why anecdotes and testimonials aren’t reliable evidence and why personal experience isn’t a necessary, or even very good, way to evaluate a medical treatment. I am collecting these articles here for convenience, both as a reference for interested and open-minded readers and so I only have to post one link when I answer such arguments.

Why We’re Often Wrong
Testimonials Lie
The Role of Anecdotes in Science-Based Medicine
Don’t Believe your Eyes (or Your Brain)
Caregiver Placebo Effects

Though a bit more technical and aimed at vets, this article also touches on cognitive biases which reduce the reliability of uncontrolled personal observations.

McKenzie, B. Veterinary clinical decision-making: cognitive biases, external constraints, and strategies for improvement. Journal of the American Veterinary Medical Association. 2014;244(3):271-276.


This entry was posted in Topic-Based Summaries. Bookmark the permalink.

7 Responses to Why Anecdotes & Testimonials Can’t Be Trusted

  1. Alice says:

    I get this error on the first link.

    Fatal error: Incompatible file format: The encoded file has format major ID 1, whereas the Loader expects 7 in /home/skeptve1/public_html/index.php on line 0

  2. L says:

    If you type “Why we’re often wrong” in the search engine on the right above recent comments, I think it will take you to the blog.

  3. skeptvet says:

    Sorry about the link. I’ve fixed it and checked all the others. Thanks!

  4. erika christina says:

    my ten year old Rottweiler has bone cancer. I give her the prescribed meds and essiac because it was recommended to me and “can’t hurt” right? She did a downward spiral so abnormally fast with all around joint pain I thought I was loosing her 3 weeks after diagnosis. On a whim I stopped the essiac. She has improved every day since. It has been 5 days no essiac she is back to a slight limp and running around coincidence? I can’t find any info on this.

  5. skeptvet says:

    Unfortunately, such anecdotes aren’t any better at telling us what doesn’t work than they are at telling us what does. We could line up a thousand such stories, for or against the treatment, and we still wouldn’t have a reliable answer. That’s why science is needed.

  6. Linda Geer says:

    My dog was put on a Chinese herb because of a cancerous tumor on his leg. It was removed and got all of it but was suggested to prevent it returning. It has been 11/2 yrs now and he just had a 4 day hoist if seusres and now on Levetiracetam and seizures stopped but I now want to end the herbs and try cbd oil to try to get him off of antiseizure meds along with adding cooked good food w calcium, salmon oil with reputable dog food may change health a good direction for adding a few more years to his life. I cannot afford a mri for a pinpoint on why the seizures. The vet is just clinically giving him what will immediately help. I wonder if the herb he was living on just had a reaction? I am giving him xue fu Zhu Tang 1/2 tsp 1x day. Along with 25 mg benadryl and 10 mg Famotadine and Ligaplex 11 for joints? Too much?

  7. skeptvet says:

    There is zero clinical research evidence to support the safety or effectiveness of Chinese herbal remedies, and there have been many reports of contamination with dangerous toxins or unreported drugs, so I do not recommend using them. While CBD has been shown to help human children with a specific seizure disorder, it is not yet clear if ithelpsdogsacd, unfortunately, the cause does matter in terms of choosing the remedy.Unlike the TCM herbs, there is less reason to be concerned about serious side-effects from the CBD. Good luck.

Leave a Reply

Your email address will not be published. Required fields are marked *