By far the most frequent response I get to any article critical of claims for an alternative therapy or focused on the lack of good evidence for such claims is an appeal to anecdotal evidence. Some variation on “I tried it and it worked for me” or “Don’t knock it ’til you’ve tried it” is a common and, for many people, persuasive rebuttal to even the most carefully researched and objective, science-based critique. Since I end up responding to such anecdote-based rebuttals almost every day, I have written frequently about why anecdotes and testimonials aren’t reliable evidence and why personal experience isn’t a necessary, or even very good, way to evaluate a medical treatment. I am collecting these articles here for convenience, both as a reference for interested and open-minded readers and so I only have to post one link when I answer such arguments.
Why We’re Often Wrong
Testimonials Lie
The Role of Anecdotes in Science-Based Medicine
Don’t Believe your Eyes (or Your Brain)
Caregiver Placebo Effects
Though a bit more technical and aimed at vets, this article also touches on cognitive biases which reduce the reliability of uncontrolled personal observations.
McKenzie, B. Veterinary clinical decision-making: cognitive biases, external constraints, and strategies for improvement. Journal of the American Veterinary Medical Association. 2014;244(3):271-276.
I get this error on the first link.
Fatal error: Incompatible file format: The encoded file has format major ID 1, whereas the Loader expects 7 in /home/skeptve1/public_html/index.php on line 0
If you type “Why we’re often wrong” in the search engine on the right above recent comments, I think it will take you to the blog.
Sorry about the link. I’ve fixed it and checked all the others. Thanks!
my ten year old Rottweiler has bone cancer. I give her the prescribed meds and essiac because it was recommended to me and “can’t hurt” right? She did a downward spiral so abnormally fast with all around joint pain I thought I was loosing her 3 weeks after diagnosis. On a whim I stopped the essiac. She has improved every day since. It has been 5 days no essiac she is back to a slight limp and running around coincidence? I can’t find any info on this.
Unfortunately, such anecdotes aren’t any better at telling us what doesn’t work than they are at telling us what does. We could line up a thousand such stories, for or against the treatment, and we still wouldn’t have a reliable answer. That’s why science is needed.
My dog was put on a Chinese herb because of a cancerous tumor on his leg. It was removed and got all of it but was suggested to prevent it returning. It has been 11/2 yrs now and he just had a 4 day hoist if seusres and now on Levetiracetam and seizures stopped but I now want to end the herbs and try cbd oil to try to get him off of antiseizure meds along with adding cooked good food w calcium, salmon oil with reputable dog food may change health a good direction for adding a few more years to his life. I cannot afford a mri for a pinpoint on why the seizures. The vet is just clinically giving him what will immediately help. I wonder if the herb he was living on just had a reaction? I am giving him xue fu Zhu Tang 1/2 tsp 1x day. Along with 25 mg benadryl and 10 mg Famotadine and Ligaplex 11 for joints? Too much?
There is zero clinical research evidence to support the safety or effectiveness of Chinese herbal remedies, and there have been many reports of contamination with dangerous toxins or unreported drugs, so I do not recommend using them. While CBD has been shown to help human children with a specific seizure disorder, it is not yet clear if ithelpsdogsacd, unfortunately, the cause does matter in terms of choosing the remedy.Unlike the TCM herbs, there is less reason to be concerned about serious side-effects from the CBD. Good luck.
Der Skeptvet, I would like to share my thoughts with you.
I am not opposed to science in any way, but the problem that I see is that for everything we think we know, there is so much more we don’t. I know that you acknowlege this. But from my personal experience both with human doctors and veterinarians, this is not always the case.
What I have found in the traditional medical field is that curiosity is often lacking. Of course I can only speak from personal experience, and experience of those close to me, and I can’t tell you a “percentage” of medical practitioners this applies to.
But, I truly believe one of the main problems in the medical field, and also something that drives people away from traditional science-based medicine, is the lack of curiousity ie willingness to further knowledge beyond what is minimally neccesary.
It is fully possible to treat a patient based on science that was valid 15 years ago, completely disregarding, or not even knowing about the scientific advances that have been made the latest 15 years. If this approach is used, it is not truly science-based.
I have a dog with Cushing’s disease and have made several ACTH-stimulation-tests and full blood tests.
According to best practice, the blood tests should be made on an empty stomach.
According to best practice, the ACTH-stimulation-test should be done after the dog has been fed and medicated as usual.
My veterinarians choice was to perform the blood tests and the ACTH-test at the same time, on a fasted stomach, although the medicine could be given the same morning.
The manufacturer of the medicine (Vetoryl) claims that it is very important that the dog is fed the same morning before the test is done, my veterinarian went against this.
I called my veterinarian and asked about this, he told me that it is fine and that I should trust him because he goes to conferences etc and knows what he is doing.
In many aspects, he is a competent veterinarian, he is the one who made me aware that the new, quick-test for measuring cortisol is not as reliable and it all made sense
He is also the one who adviced me to give my dog the medicine with a little bit of fat for increased absorption which makes sense because it is a fat soluble.
However, in this particular instance, I do not trust his judgement. I told him that the manufaturer stresses the importance that nothing changes in the routine on the day of the test, that it is given with food as usual etc. but he would not have it.
He insisted that I need to bring my dog fasted becase if I don’t, the blood values will not be accurate (liver, cholesterol etc).
So I suggested that I do the ACTH-stimulation-test and the blood test on two separate occations.
He did not want to do this because he thought it would add unneccesary stress for my dog.
I can not accept this reason.
The stress is a problem to begin with since it elevates cortisol, so simply going to the veterinarian may already give him a false result.
The correct result is extremely important since the medication dosage is dependend on it. Giving the medication without food may affect how the medicine is absorbed.
And not overdosing is extremely important because too much can send the dog into an adrenal crisis.
In my humble, uneducated opinion, going against the manufacturers advice, and changing the routine on the day of the test is not a safe thing to do, no matter how many seminars you have gone to, or how many years of experience you have.
The veterinarian could not provide me an appropriate explanation as to why doing it his way was safe, but instead asked me to trust him.
And this veterinarian was still curteous to engage in conversation with me, acknowledged that I had done a lot of research and did not dismiss me entirely.
Still, this did hurt my trust in him as a medical provider for my dog.
Many veterinarians and doctors unfortunately are very confident in their own knowledge to the point that they can come off as lacking humility.
And we are all only human, we are all wrong all the time so humility is very important.
And unfortunately, this is the kind of experience that many people have with medical professionals that increases scepticism, decreases compliance, and in too many cases, sends the person straight into the claws of an “alternative doctor” that now confirms what they have now come to believe about the medical field, and offers them a plausible cure-it-all.
I understand that it is frustrating to deal with false accusations and people believing in miracle cures, I understand that it gets old, but oftentimes, these people are people who feel let down by the medical professionals, either because of actual malpractice in some cases, or because of lack of communication.
I would like to say that again, the most important factor or lack of communication.
You need to understand that even though a regular person who lacks a medical education can only get fragments of the knowledge you have aquired from years of studying and practicing, it is still important to acknowledge that the every day person now has access to more information than ever before, and it is very important, if you are to dismiss a persons claims, to explain why they are wrong, but also to be open to the idea that perhaps they have gotten access to information that you have not yet.
With all that is going on in all things science, it is impossible for one single individual to be flawlessly up to date with each and every aspect of medicine.
Sorry for the long text, I hope it is not too incoherent.
In general, I agree with you. Communication is critical, and not all healthcare workers are good at it, just as not all people in other professions are. We are, after, just people like anyone else, with strengths and weaknesses. I also agree that sometimes individual doctors, just like pet owners, trust their personal experiences more than scientific evidence and so make choices that contradict the current best evidence. That is precisely the sort of problem evidence-based medicine attempts to solve, by encouraging us to trust the evidence more than our own beliefs, opinions, and experiences.
I will disagree that science and scientists lack curiosity. This is, in fact, the primary intellectual driver of scientific research and discovery. Individuals, and certainly clinicians who are not researchers may get stuck in ruts and not look for new ways of doing things, but that is a problem with people, not with the method of science, which is constantly exploring new ideas. Unfortunately, many new ideas turn out to be wrong, and science frustrates people when it rejects new ideas even if that rejection is based on good evidence. People often mistake an unwillingness to believe claims with no better evidence than anecdote for closed-mindedness, but this is not the case. Being curious and open-minded doesn’t mean believing anything and everything.
Overall, I think we are in agreement, but I think individual human failings are inevitable and they are the reason science and skepticism is necessary, not an indictment of the scientific approach.
More people need to see this article. After my cat developed CKD I was scouring “Dr Google” for answers (hundreds and hundreds of tabs, I was so overwhelmed that I got a panic attack) as well as support groups online. SO. MANY. CONFLICTING. THINGS. The support groups were great overall (they helped me find a bunch of low-phos foods) but there were a few stragglers who suggested things with no scientific proof. There was even a woman who wanted to give her dying cat expired antibiotics … ??!? I found out the internet can be a useful tool, but always find a good veterinarian, work closely with them, and ask them before doing anything you read online, such as rubbing slippery Elm bark and colodial silver all over their inflamed gums -_- I believe pets are naturally very capable and durable and people fool THEMSELVES with placebo effects thinking whatever “natural” remedy they’ve given their pet made things better, when in fact, it could be masking things or making things worse. Take the “kidney support gold” shills for example, if you read the ACTUAL one star reviews, there are countless people who’s animals died horribly by taking it… All they wanted to do was help their pet and those $$$ hungry charlatans only made things worse. Less is more, in my opinion. If your vet prescribes something, use it and monitor for side effects, but don’t use any herbals without asking your vet first (again get a GOOD VET) and looking up the side effects and cautions (which there ALWAYS ARE) and unbiased negative experiences. I may have caused my cat’s CKD by working with a “holistic” vet and I’ll always hate myself for it. I may not have, too, though, but there’s no way of truly knowing. Now I’d rather my pet be affected by something mildly than die a few days after being given snake oil. If those alternative treatments really were so great, EVERYBODY would be using it, or cashing in on it… Think about it. Just my two cents.