Why Anecdotes & Testimonials Can’t Be Trusted

By far the most frequent response I get to any article critical of claims for an alternative therapy or focused  on the lack of good evidence for such claims is an appeal to anecdotal evidence.  Some variation on “I tried it and it worked for me” or “Don’t knock it ’til you’ve tried it” is a common and, for many people,  persuasive rebuttal to even the most carefully researched and objective, science-based critique. Since I end up responding to such anecdote-based rebuttals almost every day, I have written frequently about why anecdotes and testimonials aren’t reliable evidence and why personal experience isn’t a necessary, or even very good, way to evaluate a medical treatment. I am collecting these articles here for convenience, both as a reference for interested and open-minded readers and so I only have to post one link when I answer such arguments.

Why We’re Often Wrong
Testimonials Lie
The Role of Anecdotes in Science-Based Medicine
Don’t Believe your Eyes (or Your Brain)
Caregiver Placebo Effects

Though a bit more technical and aimed at vets, this article also touches on cognitive biases which reduce the reliability of uncontrolled personal observations.

McKenzie, B. Veterinary clinical decision-making: cognitive biases, external constraints, and strategies for improvement. Journal of the American Veterinary Medical Association. 2014;244(3):271-276.

 

This entry was posted in Topic-Based Summaries. Bookmark the permalink.

5 Responses to Why Anecdotes & Testimonials Can’t Be Trusted

  1. Alice says:

    I get this error on the first link.

    Fatal error: Incompatible file format: The encoded file has format major ID 1, whereas the Loader expects 7 in /home/skeptve1/public_html/index.php on line 0

  2. L says:

    If you type “Why we’re often wrong” in the search engine on the right above recent comments, I think it will take you to the blog.

  3. skeptvet says:

    Sorry about the link. I’ve fixed it and checked all the others. Thanks!

  4. erika christina says:

    my ten year old Rottweiler has bone cancer. I give her the prescribed meds and essiac because it was recommended to me and “can’t hurt” right? She did a downward spiral so abnormally fast with all around joint pain I thought I was loosing her 3 weeks after diagnosis. On a whim I stopped the essiac. She has improved every day since. It has been 5 days no essiac she is back to a slight limp and running around coincidence? I can’t find any info on this.

  5. skeptvet says:

    Unfortunately, such anecdotes aren’t any better at telling us what doesn’t work than they are at telling us what does. We could line up a thousand such stories, for or against the treatment, and we still wouldn’t have a reliable answer. That’s why science is needed.

Leave a Reply

Your email address will not be published. Required fields are marked *

This blog is kept spam free by WP-SpamFree.