I recently gave a lecture at the Western Veterinary Conference called “What You Know that Ain’t Necessarily So.” The purpose of this was to take some common or controversial beliefs and practices in veterinary medicine and discuss the scientific evidence pertaining to these. This was not intended as a definitive, “final word” on these subjects, but as an illustration of how weak and problematic the evidence often is even behind widely held beliefs. In some cases, these practices or ideas may actually be valid, but without good quality scientific evidence, we should always be cautious and skeptical about them.

Eventually, I will post recordings of the presentations themselves, but for now I am posting a summary of each topic.

Each starts with a focused clinical question using the PICO format.

P– Patient, Problem Define clearly the patient in terms of signalment, health status, and other factors relevant to the treatment, diagnostic test, or other intervention you are considering. Also clearly and narrowly define the problem and any relevant comorbidities. This is a routine part of good clinical practice and so does not represent “extra work” when employed as part of the EBVM process.

I– Intervention Be specific about what you are considering doing, what test, drug, procedure, or other intervention you need information about.

C– Comparator What might you do instead of the intervention you are considering? Nothing is done in isolation, and the value of most of our interventions can only be measured relative to the alternatives. Always remember that educating the client, rather than selling a product or procedure, should often be considered as an alternative to any intervention you are contemplating.

O– Outcome What is the goal of doing something? What, in particular, does the client wish to accomplish. Being clear and explicit, with yourself and the client, about what you are trying to achieve (cure, extended life, improved performance, decreased discomfort, etc.) is essentially in evidence-based practice.

This is then followed by a summary of the evidence available at each of the levels in the following pyramid (which is a pragmatic reinterpretation of the classical pyramid of evidence that is a bit more useful for general practice veterinarians).

Finally, I list the Bottom Line, which is my interpretation of the evidence.

Immune-mediated Blood Disease & Vaccination

1. Clinical question

P- healthy dogs & cats

I- routine vaccinations

C- no vaccination (fewer)

O- incidence of Immune-mediated hemolytic anemia (IMHA) and immune-mediated thrombocytopenia (ITP)

1. Synthetic Veterinary Literature
   a. No systematic reviews:
   b. No CATs
2. Primary Veterinary Literature

• Case/control study
• Cases more likely to be vaccinated in previous month (26%) than controls (7%)
• Did not r/o pre-existing disease

Duval, D. (1996)

• 10% of cases vaccinated within 1 month
• No difference between cases and controls in time from vaccination to presentation

Carr, A.P. (2002)

• Proportion of dogs vaccinated within 2 months of onset not different between cases and controls
  • Cases- 16.1%
  • Controls- 44.4%

Davidow, E.B. (2004)

• Proportion of dogs vaccinated within 42 days of onset not different between cases and controls-
  • Cases- 8%
  • Controls- 14%

Huang, A.A. (2012)

• 4% vaccinated within 2 weeks
• Not primary purpose of study

Reimer, M.E. (1999)

• 2.4% vaccinated within 2 weeks
• Not primary purpose of study

Klag, A.R. (1993)

3. Human Literature
a. Systematic Reviews

• Database of 4.2 million children
• 55 cases of IMHA reported 1991-2001
• No association with vaccination

Naleway, A.L. (2009)

• ITP only associated with MMR
• 1-3 children per 100,000 doses
• This is lower than the rate of ITP caused by the diseases MMR prevents! (1:3000 to 1:6000 cases)

Cecinati, V. (2013)

Bottom Line

• Little evidence vaccination causes IMHA/ITP
• No consistent temporal association
• Data are weak
• Overwhelming majority of vaccinated animals do not develop these diseases
• Infection can be a greater risk for IMHA/ITP than vaccination
• Don’t vaccinate more than necessary
• Don’t vaccinate less than necessary
• Don’t avoid vaccination out of fear of IMHA/ITP

Reference

Carr AP, et al. Prognostic factors for mortality and thromboembolism in canine immune-mediated hemolytic anemia: A retrospective study of 72 dogs. J Vet Internal Med.2002;16:504-509.

Cecinati V. Hum Vaccin Immunother. Vaccine administration and the development of immune thrombocytopenic purpura in children. 2013 May;9(5):1158-62.

Davidow EB, et al. Risk factors for development of IMHA-A prospective case-control study. Abstract. VECCS 2004.

Duval D et al. Vaccine-associated immune-mediated hemolytic anemia in the dog. J Vet Internal Med. 1996;10:290-295.

Huang AA, et al. Idiopathic immune-mediated thrombocytopenia and recent vaccinations in dogs. JVIM 2012; 26: 142-148.

Klag AR, et al. Idiopathic immune-mediated hemolytic anemia in dogs: 42 cases (1986-1990). J Am Vet Med Assoc. 1993;202:783-788.

Naleway AL. et al. Risk of immune hemolytic anemia in children following immunization. Vaccine. 2009 Dec 9;27(52):7394-7.

Reimer ME, Troy GC, Warnick LD. Immune-mediated hemolytic anemia: 70 cases (1988-1996). J Am Anim Hosp Assoc. 1999;35:384-391

I recently gave a lecture at the Western Veterinary Conference called “What You Know that Ain’t Necessarily So.” The purpose of this was to take some common or controversial beliefs and practices in veterinary medicine and discuss the scientific evidence pertaining to these. This was not intended as a definitive, “final word” on these subjects, but as an illustration of how weak and problematic the evidence often is even behind widely held beliefs. In some cases, these practices or ideas may actually be valid, but without good quality scientific evidence, we should always be cautious and skeptical about them.

Eventually, I will post recordings of the presentations themselves, but for now I am posting a summary of each topic.

Each starts with a focused clinical question using the PICO format.

P– Patient, Problem Define clearly the patient in terms of signalment, health status, and other factors relevant to the treatment, diagnostic test, or other intervention you are considering. Also clearly and narrowly define the problem and any relevant comorbidities. This is a routine part of good clinical practice and so does not represent “extra work” when employed as part of the EBVM process.

I– Intervention Be specific about what you are considering doing, what test, drug, procedure, or other intervention you need information about.

C– Comparator What might you do instead of the intervention you are considering? Nothing is done in isolation, and the value of most of our interventions can only be measured relative to the alternatives. Always remember that educating the client, rather than selling a product or procedure, should often be considered as an alternative to any intervention you are contemplating.

O– Outcome What is the goal of doing something? What, in particular, does the client wish to accomplish. Being clear and explicit, with yourself and the client, about what you are trying to achieve (cure, extended life, improved performance, decreased discomfort, etc.) is essentially in evidence-based practice.

This is then followed by a summary of the evidence available at each of the levels in the following pyramid (which is a pragmatic reinterpretation of the classical pyramid of evidence that is a bit more useful for general practice veterinarians).

Finally, I list the Bottom Line, which is my interpretation of the evidence.

Pre-anesthetic Bloodwork in Healthy Animals

1. Clinical question

P– healthy dogs & cats

I– routine cbc/chem before anesthesia

C– no bloodwork

O– mortality, complications, change plan

2. Synthetic Veterinary Literature

a. Systematic Reviews- none
b. CATs- none
c. Guidelines- none

3. Primary Veterinary Literature-

• cbc/biochemistry profiles for 1537 dogs
• university surgery population
• variety of ASA stages
• No indication in PE/Hx for labwork in 84%
• Recategorized in ASA level- 8%
• Procedure postponed- 0.8%
• Additional therapy- 1.5%
• Change in protocol- 0.2%
• Complications in 1.9% of patients
  • Lab values normal or unrelated in 84% of these
  • 3.8% incidence with lab abnormalities
  • 1.8% incidence with normal labs (no statistical difference)

The changes revealed by pre-operative screening were usually of little clinical relevance and did not prompt major changes to the anaesthetic technique…In dogs, pre-anaesthetic laboratory examination is unlikely to yield additional important information if no potential problems are identified in the history and on physical examination.

(Alef, 2008)

• 101 dogs
• Private practice
• > 7 years of age (avg=11)
• Routine and emergent cases
• 87% had no pre-existing conditions
• New problem found- 29.7%
• Anesthesia cancelled- 12.9%
• Further tests- 5.9%
• Euthanized- 4%
• Procedure postponed- 1%
• Additional therapy- 1%
• Age did not predict abnormalities
• Abnormalities not associated with complications

This study concluded that screening of geriatric patients important and that sub-clinical disease could be present in nearly 30 % of these patients. The value of screening before anaesthesia is perhaps more questionable in terms of anaesthetic practice but it is an appropriate time to perform such an evaluation.

(Joubert, 2007)

• 100 cats > 6 years old
• Not pre-anesthetic screening, just general screening
• No know abnormalities on Hx
• Many not normal on PE
• Lots of abnormalities
  • 13% increased wbc
  • 29% increased creatinine
  • 15% increased BUN
  • 25% increased glucose
  • 4% increased T4
  • 6% increased ALT

Some new diagnoses

• 14% FIV positive
• 2% CKD
• 1% hyperthyroidism
• 1% UTI
• Relevance to pre-anesthetic screening?
• Some abnormalities were related to choice of reference interval
• Many abnormalities were clinically irrelevant
• Not truly screening since some had PE abnormalities

(Paepe, 2013)

4. Human Literature

a. Systematic Reviews

CBC

• Abnormal <1% to 5%
• Change protocol- 0.1% to 2.7%

Hemostasis

• Abnormal 3.8-15.6%
• Change protocol- rarely

Biochemistry

• Abnormal <1% to 5%
• Change protocol- rarely

Urinalysis

• Abnormal 1-35.1%
• Change protocol- 1% to 2.8%

The tests reviewed produce a wide range of abnormal results, even in apparently healthy individuals.

The tests lead to changes in clinical management in only a very small proportion of patients, and for some tests virtually never.

The clinical value of changes in management which do occur in response to an abnormal test result may also be uncertain in some instances.

The power of preoperative tests to predict adverse postoperative outcomes in asymptomatic patients is either weak or non-existent.

For all the tests reviewed, a policy of routine testing in apparently healthy individuals is likely to lead to little, if any, benefit.

The clinical importance of many of these abnormal results is uncertain.

(Munro, 1997)

b. Guidelines

• Don’t obtain baseline laboratory studies in patients without significant systemic disease (ASA I or II) undergoing low-risk surgery
• Performing routine laboratory tests in patients who are otherwise healthy is of little value in detecting disease.
• Evidence suggests that a targeted history and physical exam should determine whether pre-procedure laboratory studies should be obtained.
• American Society of Anesthesiologists

Bottom Line

• If you test, you will find abnormalities
• The clinical significance of these abnormalities is unclear
• You will find more abnormalities if pre-test probability is high
• Indication for test in Hx
• Abnormality on PE
• There is no evidence testing healthy patients reduces morbidity or mortality

What’s the harm?

• Expense
• Risk of testing
• Overdiagnosis
• Expense
• Stress
• Direct harm

References

Alef, M.; Praun, F. von; Oechtering, G. Is routine pre-anaesthetic haematological and biochemical screening justified in dogs? Veterinary Anaesthesia and Analgesia 2008 Vol. 35 No. 2 pp. 132-140

Munro J, et al. Routine preoperative testing: a systematic review of the evidence. Health Technol Assess. 1997;1(12):i-iv; 1-62.

Paepe D, et al. Routine health screening: findings in apparently healthy middle-aged and old cats. J Feline Med Surg. 2013 Jan;15(1):8-19.

Joubert K.E., Pre-anaesthetic screening of geriatric dogs. J S Afr Vet Assoc. March 2007;78(1):31-5.

I recently gave a lecture at the Western Veterinary Conference called “What You Know that Ain’t Necessarily So.” The purpose of this was to take some common or controversial beliefs and practices in veterinary medicine and discuss the scientific evidence pertaining to these. This was not intended as a definitive, “final word” on these subjects, but as an illustration of how weak and problematic the evidence often is even behind widely held beliefs. In some cases, these practices or ideas may actually be valid, but without good quality scientific evidence, we should always be cautious and skeptical about them.

Eventually, I will post recordings of the presentations themselves, but for now I am posting a summary of each topic.

Each starts with a focused clinical question using the PICO format.

P– Patient, Problem Define clearly the patient in terms of signalment, health status, and other factors relevant to the treatment, diagnostic test, or other intervention you are considering. Also clearly and narrowly define the problem and any relevant comorbidities. This is a routine part of good clinical practice and so does not represent “extra work” when employed as part of the EBVM process.

I– Intervention Be specific about what you are considering doing, what test, drug, procedure, or other intervention you need information about.

C– Comparator What might you do instead of the intervention you are considering? Nothing is done in isolation, and the value of most of our interventions can only be measured relative to the alternatives. Always remember that educating the client, rather than selling a product or procedure, should often be considered as an alternative to any intervention you are contemplating.

O– Outcome What is the goal of doing something? What, in particular, does the client wish to accomplish. Being clear and explicit, with yourself and the client, about what you are trying to achieve (cure, extended life, improved performance, decreased discomfort, etc.) is essentially in evidence-based practice.

This is then followed by a summary of the evidence available at each of the levels in the following pyramid (which is a pragmatic reinterpretation of the classical pyramid of evidence that is a bit more useful for general practice veterinarians).

Finally, I list the Bottom Line, which is my interpretation of the evidence.

1. Clinical question

P– dogs with naturally occurring mitral valve disease (MVD)

I– angiotensin-converting enzyme inhibitor (ACE-I) before onset of congestive heart failure (CHF)

C– no ACE-I before onset CHF

O– time to CHF, survival

2. Synthetic Veterinary Literature

a. Systematic Reviews- none

b. Critically Appraised Topics (CAT)-

Best Bets for Vets
Benazepril in dogs with asymptomatic mitral valve disease
(1 study, significant weaknesses)

There is insufficient evidence to suggest that dogs with asymptomatic mitral valve disease treated with benazepril will live longer.

3. Primary Veterinary Literature

Long-term treatment with enalapril in asymptomatic dogs with MVD and MR did not delay the onset of heart failure regardless of whether or not cardiomegaly was present at initiation of the study.

(Kvart, 2002) SVEP: Scandinavian Veterinary Enalapril Prevention Trial

Chronic enalapril treatment of dogs with naturally occurring, moderate to severe MR significantly delayed onset of CHF…Improvement in the primary endpoint, CHF-free survival, was not significant. Results suggest that enalapril modestly delays the onset of CHF in dogs with moderate to severe MR.

• No difference in primary endpoint (time to CHF)
• Questionable secondary endpoint (all-cause mortality)
• Selective exclusion of some subjects from analysis
• Questionable selection of time points for comparisonBNZ had beneficial effects in asymptomatic dogs other than CKC and KC affected by MVD with moderate-to-severe MR.

(Atkins, 2007) VETPROOF

• No difference overall in development of CHF or “cardiac events”
• Questionable endpoints (all-cause mortality)
• Differences between groups in disease severity

(Pouchelon, 2008)

4. Human Literature-

a. Systematic Reviews

CE inhibitors…reduced the RF, RV, and left ventricular size by a modest degree in chronic primary MR.. and may offer a pharmacologic treatment option to delay the deleterious hemodynamic effects of left ventricular volume overload.

(Strauss, 2012)

b. Guidelines

• In the asymptomatic patient with chronic MR, there is no generally accepted medical therapy.
• There has not been a consistent improvement in LV volumes and severity of MR in the small studies that have examined the effect of ACE inhibitors.
• in the absence of systemic hypertension, there is no known indication for the use of…ACE inhibitors in asymptomatic patients with MR and preserved LV function.
• If LV systolic dysfunction is present, primary treatment of the LV systolic dysfunction with drugs such as ACE…[has] been shown to reduce the severity of functional MR
• (American College of Cardiology/American Heart Association, 2006)

Bottom Line–

• Research is conflicting
• Cardiologists disagree
• Critical review of the evidence does not suggest ACE-I
  • Delay onset of CHF
  • Prolong survival when begun before CHF

References

Atkins CE, et al. Results of the veterinary enalapril trial to prove reduction in onset of heart failure in dogs chronically treated with enalapril alone for compensated, naturally occurring mitral valve insufficiency. J Am Vet Med Assoc. 2007 Oct 1;231(7):1061-9.

Kvart C, et al. Efficacy of enalapril for prevention of congestive heart failure in dogs with myxomatous valve disease and asymptomatic mitral regurgitation. J Vet Intern Med. 2002 Jan-Feb;16(1):80-8.

Pouchelon JL, et al. Effect of benazepril on survival and cardiac events in dogs with asymptomatic mitral valve disease: a retrospective study of 141 cases. J Vet Intern Med. 2008 Jul-Aug;22(4):905-14.

Strauss CE, et al. Pharmacotherapy in the treatment of mitral regurgitation: a systematic review. J Heart Valve Dis. 2012 May;21(3):275-85.

I recently gave a lecture at the Western Veterinary Conference called “What You Know that Ain’t Necessarily So.” The purpose of this was to take some common or controversial beliefs and practices in veterinary medicine and discuss the scientific evidence pertaining to these. This was not intended as a definitive, “final word” on these subjects, but as an illustration of how weak and problematic the evidence often is even behind widely held beliefs. In some cases, these practices or ideas may actually be valid, but without good quality scientific evidence, we should always be cautious and skeptical about them.

Eventually, I will post recordings of the presentations themselves, but for now I am posting a summary of each topic.

Each starts with a focused clinical question using the PICO format.

P– Patient, Problem Define clearly the patient in terms of signalment, health status, and other factors relevant to the treatment, diagnostic test, or other intervention you are considering. Also clearly and narrowly define the problem and any relevant comorbidities. This is a routine part of good clinical practice and so does not represent “extra work” when employed as part of the EBVM process.

I– Intervention Be specific about what you are considering doing, what test, drug, procedure, or other intervention you need information about.

C– Comparator What might you do instead of the intervention you are considering? Nothing is done in isolation, and the value of most of our interventions can only be measured relative to the alternatives. Always remember that educating the client, rather than selling a product or procedure, should often be considered as an alternative to any intervention you are contemplating.

O– Outcome What is the goal of doing something? What, in particular, does the client wish to accomplish. Being clear and explicit, with yourself and the client, about what you are trying to achieve (cure, extended life, improved performance, decreased discomfort, etc.) is essentially in evidence-based practice.

This is then followed by a summary of the evidence available at each of the levels in the following pyramid (which is a pragmatic reinterpretation of the classical pyramid of evidence that is a bit more useful for general practice veterinarians).

Finally, I list the Bottom Line, which is my interpretation of the evidence.

Neutering and Mammary Neoplasia in Dogs

1. Clinical question

P- female dogs

I- neutering (timing)

C- remaining intact (timing)

O- incidence/mortality mammary neoplasia

What is Risk in Intact Females?

• 53.3% (varies by breed and age: 4-35/1,000/yr)
  (Moe, 2001)

• 1% 6yrs, 6% 8yrs, 13% 10yrs
• 111/10,000 DYAR (5-319/10,000 DYAR)
  (Egenvall, 2002)

• 205 tumors/100,000 dogs/year SI
  (Dobson, 2002)

2. Synthetic Veterinary Literature

a. One systematic review:

13 peer-reviewed journal articles in English

     9 high risk of bias

     4 moderate risk of bias

          1 found protective association

         2 found no association

         1 found“some protective effect” but no numbers

(Beauvais, 2012)

b. One critically appraised topic

Best Bets for Vets Age at neutering and mammary tumours in bitches

Spaying bitches before the first or second season, or before the age of 2.5 years, may be associated with a reduced risk of developing malignant mammary tumours later in life…However, the evidence is relatively weak, and this should be taken into account alongside other considerations when recommending whether and when to neuter.

3.  Primary Veterinary Literature

• Strong protective effect
• No confidence interval or p-value
• Matched cases/controls, matched analysis?
• Cases/controls from different time periods
• Only for cases with histopathology
• Previous hormone use?

(Schneider, 1969)

• “inconsistent although some protective effect”
• No quantitative assessment
• No control for age, previous hormone use
• Research Beagles
• Neutered at 10-12 years of age
• Primary purpose to evaluate Rad Tx

(Bruenger, 1994)

• No association (neutering & any type of mammary mass)
• Age at neutering not reported
• No control for age, breed, previous hormone use
• Only for cases with histopatholog
• Primary purpose diet and mammary tumors

(Pérez Alenza 1998)

• No association (neutering & proportion of submitted tumor samples neoplastic)
• Age and age at neutering not reported
• No control for age, previous hormone use
• Only for cases with histopathology
• No confidence intervals

(Richards, 2001)

• No association between age at neutering and neoplasia
• All neutered 6 weeks-12 months
• Mammary neoplasia not specifically addressed
• Shelter dogs
• Underpowered for uncommon outcomes

(Spain, 2004)

4. Human Literature- Primary

> 66,000 women studied, observational

The risk for breast cancer was reduced by 27% among women who had hysterectomy and BSO before 45 years of age, and by 20% among those who had simple hysterectomy before age 45 years.

(Gaudet, 2014)

~4,500 women studied, case/control

Bilateral ovariectomy was associated with reduced breast cancer risk overall (odds ratio (OR) = 0.59, 95% confidence interval (CI): 0.50, 0.69) and among women <45 years of age (ORs ranged from 0.31 to 0.52), but not among those who were older at surgery.

(Press, 2011)

Bottom Line–

• Available evidence suggests neutering reduces mammary cancer risk
• Earlier neutering may be more protective
• Great variation with breed, age, other factors
• Existing evidence is very weak
• Strong or numeric claims not justified

Reference

Beauvais W, Cardwell JM, Brodbelt DC. The effect of neutering on the risk of mammary tumours in dogs–a systematic review. J Small Anim Pract. 2012 Jun;53(6):314-22.

Bruenger, FW, et al. Occurrence of mammary-tumors in beagles given RA-226. Radiation Research 1994;138:423-434.

Dobson JM, Samuel S, Milstein H, Rogers K, Wood JL. Canine neoplasia in the UK: estimates of incidence from a population of insured dogs. Journal of Small Animal Practice 2002;43(6);240-6.

Egenvall A, Bonnett BN, Ohagen P, Olson P, Hedhammar A, von Euler H. Incidence of and survival after mammary tumors in a population of over 80,000 insured female dogs in Sweden from 1995 to 2002. Preventative Veterinary Medicine 2002;69:109-27.

Gaudet MM, et al. Obstet Gynecol. Oophorectomy and Hysterectomy and Cancer Incidence in the Cancer Prevention Study-II Nutrition Cohort. 2014:123;1247-1255.

Moe L. Population-based incidence of mammary tumors in some dog breeds. Journal of Reproduction and Fertility 2001;57:439-43.

Pérez Alenza D. et al. Relation between habitual diet and canine mammary tumors in a case-control study. Journal of Veterinary Internal Medicine. 1998;12;132-139.

Press DJ, et al. Breast cancer risk and ovariectomy, hysterectomy, and tubal sterilization in the Women’s Contraceptive and Reproductive Experiences Study. Am J Epidemiol. 2011;173(1):38-47.

Richards HG, et al. An epidemiological analysis of a canine-biopsies database compiled by a diagnostic histopathology service. Preventive Veterinary Medicine. 2001;51:125-136.

Schneider R, Dorn CR, Taylor DO. Factors influencing canine mammary cancer development and postsurgical survival. Journal of the National Cancer Institute 1969;43:1249-61.

Spain CV, Scarlett JM, Houpt KA. Long-term risks and benefits of early-age gonadectomy in dogs. Journal of the American Veterinary Medical Association 2004;224(3):380-7.

Without question, the most frequent comment I get in response to my articles is that people feel some of the therapies I write about must be effective regardless of the scientific evidence because they have had personal experiences that suggest this. The experience of having used a therapy and seen an improvement is very powerful psychologically, and it makes us very confident that the therapy we used caused the improvement we saw. Unfortunately, the evidence that this kind of conclusion is not reliable is overwhelming. This kind of thinking is so common, and so untrustworthy, it constitutes a unique ogical fallacy, the post hoc ergo propter hoc, or “false cause” fallacy.

Reality is complex, and our minds seek simple, direct causal explanations and satisfying narratives to explain things. Sometimes, of course, these explanations are true. But they are false much more often than we realize. The dismissal of anecdotal evidence by scientists isn’t casual, and it isn’t based on the idea that people who have anecdotal experiences must be stupid. It is based on centuries of study of the human mind, including decades of controlled research in cognitive psychology that shows things simply aren’t always as they seem.

John Stuart Mill described eloquently the problem that while we all realize we are imperfect and acknowledge in general terms that we can be wrong, we are very, very reluctant to ever admit we are wrong about any specific belief:

Unfortunately for the good sense of mankind, the fact of their fallibility is far from carrying the weight in their practical judgment, which is always allowed to it in theory; for while everyone knows himself to be fallible, few think it necessary to take any precautions against their own fallibility, or admit the supposition that any opinion, of which they feel very certain, may be one of the examples of the error to which they acknowledge themselves to be liable.

This helps to explain why I so often have to repeat, to pet owners and other veterinarians alike, that just because they have seen something appear to work with their own eyes, that isn’t really a good reason to believe it actually does work without supporting controlled scientific evidence.

A recent article on the subject both explains the problem, and discusses some of the possible solutions.

Your Brain is Primed to Reach False Conclusions by Christie Aschwanden

Here’s her conclusion:

With a lot of evidence that erroneous beliefs aren’t easily overturned, and when they’re tinged with emotion, forget about it. Explaining the science and helping people understand it are only the first steps. If you want someone to accept information that contradicts what they already know, you have to find a story they can buy into. That requires bridging the narrative they’ve already constructed to a new one that is both true and allows them to remain the kind of person they believe themselves to be.

The good news is that the fallibility of uncontrolled personal observation is well-documented, and this has been known for hundreds of years. This means it should be possible to inoculate people against excessive confidence in their own experiences early, teaching critical and skeptical thinking early in school, even before formal teaching of scientific facts. The bad news, however, is that once people reach a conclusion based on anecdotal experience, facts are not very effective at challenging that conclusion. Being given evidence that we are wrong tends to strengthen our false beliefs and impel us to build more and stronger arguments to support them.

Sadly, simple presentation of facts isn’t enough. People need to be led to reconsider their opinions through arguments that speak to their emotions and their core values, not simply their intellect. This is certainly more challenging than simply presenting the facts, especially for scientists who tend to think in terms of objective evidence and are suspicious or arguments that appeal primarily to emotions of beliefs rather than facts. And, of course, the biggest challenge in changing minds about scientific topics is that the stronger one’s belief, the less willing one is to seriously consider alternative explanations. The very people who most need to read about why anecdotes can’t be trusted won’t.

Despite all that, since I have to make the same argument over and over again, I have collected a few resources on this subject to which I refer anyone interested and open-minded enough to consider them:

Medical Miracles: Should We Believe?

Testimonials Lie

Alternative medicine and placebo effects in pets

Placebo effects in epileptic dogs

Medical Practices Once Widely Accepted that Proved Ineffective or Harmful when Studied Scientifically