Evidence-based Veterinary Medicine: What Is It & Why Does It Matter?

The following is a summary and the slides for a presentation I gave recently on evidence-based veterinary medicine:

WHAT IS EVIDENCE-BASED MEDICINE?
Evidence-based medicine (EBM) has been defined as the “conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients.”¹ More generally, EBM is the formal application of the philosophy and methods of science to generating knowledge and making decisions in veterinary medicine. What distinguishes evidence-based veterinary medicine from other approaches is the explicit and formal integration of scientific research evidence into the clinical decision-making process. Evidence-based veterinary medicine (EBVM) is the adaption of EBM principles and techniques to the environment and circumstances of veterinary medicine.

As clinicians we need information to evaluate our patient’s health problems and to provide effective preventative and therapeutic interventions. EBVM provides tools and guidance to those who generate this information (through clinical research), those who disseminate it (through publication, continuing education, clinical practice guidelines, etc.), and those who utilize it (in clinical practice as well as public health and policy making). With better information, and more efficient information management, we are able to make better decisions, provide the best patient care possible, and more reliably achieve our intended outcomes.

WHY DO WE NEED EBM?
In the absence of EBM practices, clinicians typically base their decisions on a number of sources of evidence other than formal research data. Studies of veterinary decision making have found that veterinarians rely largely on the opinions of colleagues and perceived experts.^2-3 To the extent that clinicians refer to research findings to guide their practice, they appear to utilize an informal, haphazard consultation of textbooks, journal articles, consensus statements and clinical guidelines. Above all, veterinarians, come to rely on their own clinical experience, judgment, and intuition in making diagnostic and therapeutic decisions. This collection of strategies is often referred to as opinion-based medicine.

There are a number of limitations to these approaches. Personal experience and opinion, even that of intelligent, educated, and experienced individuals, is subject to a wide range of cognitive biases and other sources of error that make it less reliable that is generally believed. Limitations in human perception, cognition, and memory and the influence of our beliefs and expectations lead us to erroneous conclusions which undermine the safety and efficacy of our interventions.⁴

There are many examples of how such error-prone assessment has supported ineffective or dangerous medical practices. Historically, interventions such as bloodletting, have been able to become ubiquitous and to persist for centuries with the best and brightest minds in medicine convinced they were effective, only to disappear rapidly when controlled scientific research revealed no benefits and significant risks.

In modern times, informal assessment not based on objective research has led to the similar widespread adoption of ineffective practices in many areas of healthcare. Based largely on the opinion of one individual, the practice of putting infants to sleep on their bellies to prevent Sudden Infant Death Syndrome (SIDS) was once nearly universal. This behavior persisted for two decades past the discovery of adequate scientific evidence to show it actually increased the risk of SIDS. Only once this external research evidence was integrated into public health recommendations and parent education did the practice, and the rate of SIDS, rapidly decrease.⁵

Surgical procedures such as internal mammary artery ligation and arthroscopic knee surgery have been widely employed until shown by controlled trials to be no more effective than sham surgery.^6-7 In the veterinary field, practices such as prescribing antibiotics for young cats with hematuria and the nearly universal use of oral glucosamine as a therapy for osteoarthritis illustrate the potential for common and persistent use of ineffective or inappropriate therapies in the absence of a rigorous evidence-based approach to evaluating our interventions.^8-9

It has been said the three most dangerous words in medicine are “In my experience.” And one definition for clinical experience is “Making the same mistakes with increasing confidence over an impressive number of years.” The reality is that our judgments are far less reliable than we feel them to be, and we are easily fooled by circumstances, by the complexity of the living organisms we deal with and their diseases, and by our own perceptual and cognitive biases. EBVM offers strategies and tools to help compensate for the limitations of uncontrolled observation and judgment.

THE STEPS OF EBVM
While EBM is concerned with the production and reporting of scientific research as well as its use in guiding clinical decision making, from the point of the clinician the most important elements are the steps involved in integrating research evidence with clinical experience and the circumstances of a particular case in order to inform patient care. The basic steps of an EBVM clinical process are these:

1. Ask useful questions
2. Find relevant evidence
3. Assess the value and reliability of the evidence
4. Draw a conclusion
5. Assign a level of confidence to your conclusion

This is an iterative process that will be repeated regularly to build a body of knowledge with a known degree of uncertainty that can guide our clinical practice.

Asking Useful Questions
Vague or overly broad questions impede effective use of research evidence in informing clinical practices. “Does drug X work?” or “What should I do about disease Y?” are not questions that are likely to lead to the recovery of useful information from published research. There are a number of schemes for constructing questions the scientific literature can help answer. One of the easiest is the PICO scheme.

P- Patient, Problem Define clearly the patient in terms of signalment, health status, and other factors relevant to the treatment, diagnostic test, or other intervention you are considering. Also clearly and narrowly define the problem and any relevant comorbidities. This is a routine part of good clinical practice and so does not represent “extra work” when employed as part of the EBVM process.

I- Intervention Be specific about what you are considering doing, what test, drug, procedure, or other intervention you need information about.

C- Comparator What might you do instead of the intervention you are considering? Nothing is done in isolation, and the value of most of our interventions can only be measured relative to the alternatives. Always remember that educating the client, rather than selling a product or procedure, should often be considered as an alternative to any intervention you are contemplating.

O- Outcome What is the goal of doing something? What, in particular, does the client wish to accomplish. Being clear and explicit, with yourself and the client, about what you are trying to achieve (cure, extended life, improved performance, decreased discomfort, etc.) is essentially in evidence-based practice.

FIND RELEVANT EVIDENCE
Experienced clinicians typically have opinions on the value of most interventions they routinely consider. Unfortunately, we rarely know where those opinions originally came from or how consistent they are with the current best scientific evidence. And given the constraints of time and resources, practitioners will rarely have the ability to find and critically evaluate all the primary research studies relevant to a particular question. Fortunately, there are sources of evidence that can provide reliable guidance in an efficient, practical manner.

The best EBVM resource for busy clinicians is the evidence-based clinical practice guideline. These are comprehensive evaluations of the research in a general subject area that explicitly and transparently identify the relevant evidence and the quality of that evidence and make recommendations with clear disclosure of the level of confidence one should place in those recommendations based on the evidence.

Sadly, many guidelines produced in veterinary medicine are not evidence based but opinion-based (so-called GOBSAT or “Good Old Boys Sat At a Table” guidelines). These are no more reliable than any other form of expert opinion. Excellent examples of truly evidence-based guidelines are those of the RECOVER Initiative for small animal CPR and the guidelines produced by the International Task Force for Canine Atopic Dermatitis.

After evidence-based guidelines, the next most useful resources are systematic reviews and critically-appraised topics (CATs). These are more focused but still explicit and transparent reviews of the available evidence on specific topics. Systematic reviews can be identified by searching the VetSRev database, a free online resource produced by the Centre for Evidence-based Veterinary Medicine (CEVM) at the University of Nottingham. Unfortunately, getting full-text copies of these reviews can be challenging for vets not at universities, but there are a number of options depending on where one practices.

Critically appraised topics are also produced by CEVM and freely available on the web as BestBetsforVets. There are a number of other free CAT resources, including the Banfield Applied Research and Knowledge web site.

Finally, primary research studies are a useful source of guidance for clinicians, though they take more effort and expertise to find and critically evaluate.

ASSESS THE VALUE OF THE EVIDENCE
All research has limitations, and these must be formally assessed through the process of critical appraisal (discussed below). Only when the limitations of a study are clearly understood can we decide how much confidence to have in the results and conclusions of the study and whether it should lead to changes in our clinical practices. And even the best studies may not be applicable to our patients if the population studied differs in important ways from our patient population, or if the tools and techniques described are unavailable, impractical, or unacceptable to our clients. It is not enough merely to read published research reports. We must critically evaluate them in terms of reliability and applicability to our needs.

DRAW A CONCLUSION
Ultimately, the job of a veterinarian is to guide the client in making decisions about care for their animals. When the clinician is aware of the existing evidence and its limitations and clearly appreciates the degree of uncertainty, then he or she can best help the client to understand their options. Making evidence-informed decisions and clearly communicating with clients about the needs and choices for their animal is the core of clinical veterinary medicine, and this is what the tools and methods of EBVM exist to support.

ASSIGN A LEVEL OF CONFIDENCE TO YOUR CONCLUSIONS
Often, the relevant research evidence is incomplete or flawed, and sometimes there is little or no such evidence applicable to a given patient’s needs. EBVM is still useful in this situation, because it allows us to clearly, systematically identify and communicate the uncertainty inherent in our work.

EBVM is, above all, an approach for helping clinicians reach conclusions that can guide their decisions about the diagnosis and treatment of individual patients. It is often believed that a determination in Step 3 that the evidence is weak or flawed precludes making clinical decisions and taking action, and that this limits the usefulness of EBM in the veterinary field, where research evidence is often severely limited in quantity and quality. However, this is incorrect. The purpose of assessing the reliability of the research evidence is to assign a degree of confidence to conclusions or decisions based upon it. If the evidence is weak, it is often still be necessary to make a decision and act on it, especially if there is an urgent clinical problem. EBVM does not prohibit or undermine such action, it simply facilitates a clear and accurate understanding of the degree of uncertainty involved. This helps the clinician and also allows fully informed consent for the client.

It is also important that we openly discuss with clients our use of evidence to inform our recommendations. Research has suggested that clients want to be told about the uncertainties involved in the treatment of their animals, and that discussing this does not reduce their confidence in their veterinarians.¹⁰ Clients also identify truthfulness as their highest priority in communication with their vet.¹¹ By explicitly discussing our process in identifying and evaluating relevant evidence, we enhance our clients’ understanding of the role we play, and we help them to appreciate the value of our expertise, not only the products and procedures we sell.

CRITICAL APPRAISAL
Critical appraisal is the term used to describe the formal assessment of the quality and limitations of published research evidence. Different study designs have strengths and weaknesses that bear on how the reliability and applicability of their results. And many individual aspects of how a research study is designed, conducted, and reported influence how much weight the study results should be given in developing an answer to a specific clinical question.

The well-known hierarchy of evidence, often used as a symbol for EBVM, is simply one of many tools employed in the critical appraisal process. There are also a number of key methodological factors that need to be evaluated when deciding how much confidence to place in the conclusions of a given research study:

1. Control Group- If a treatment is applied to a group of subjects and there is no control group receiving a placebo or alternative treatment, there is no way to be sure any changes observed in the treatment group are actually due to the treatment or are greater than would be seen if we did something else or nothing at all. Uncontrolled trials are very weak evidence.
2. Allocation- How the subjects are assigned to the different groups in a study is important. If there is not truly randomized allocation, in which every subject has an equal chance of being assigned to any group, then there is a risk that subjects will be assigned in a biased manner and that any differences seen between the groups will be due to inherent differences between subject rather than any treatment being tested.
3. Blinding- One of the greatest strengths of formal scientific research is that it can compensate for the cognitive biases that lead us to the wrong conclusions when making informal observations. If, however, investigators and animal caregivers are able to determine which group in a study particular subjects are in, all of these biases can operate freely, and any assessments of the subjects, especially those that are at all subjective, can be skewed by unconscious bias. Unblinded or ineffectively blinded studies almost always find what the investigators expect to find, and this is no accident. Research has shown that in veterinary clinical trials, owner perceive a response in patients on placebo treatment nearly 57% of the time, and veterinarians perceived a response in these subjects 40-45% of the time.¹¹
4. Statistics- While the details of evaluating the statistical analysis in a given paper are complex and beyond the expertise of most veterinarians, it is important to bear in mind that research has demonstrated statistical errors are extremely common in published veterinary research and that even properly applied analyses are often inappropriately used to draw conclusions.
5. Effect Size- Statistical significance is largely irrelevant to the question of whether an effect observed in a study is real or important. The effect size, or the absolute value of the effects seen and the differences between groups, is far more important. One can often show a statistically significant difference that would be clinically undetectable and irrelevant.
6. Replication- No single study is ever sufficient to confidently demonstrate any hypothesis to be true or false. Replication is essential to uncovering the truth in science, and any conclusions based on research that has not been replicated should be viewed as tentative at best.

The studies which have evaluated the reliability of published veterinary research is not encouraging. Significant flaws are present in the majority of published studies, and this limits the confidence that can be placed in the results or conclusions of these studies.^13-23

RESOURCES
Print Resources

Buczinski, S. Vandeweerd, J. (Eds.). Evidence-Based Veterinary Medicine for the Bovine Veterinarian.  Veterinary Clinics of North America: Small Animal Practice. 2012 March: 28(1).

Cockroft, P. Holmes, M. (2003). Handbook of Evidence-Based Veterinary Medicine. Oxford: Blackwell.

Ramey DW. (Ed.). Evidence-based veterinary medicine. Veterinary Clinics of North America: Equine Practice. 2007 Aug;23(2).

Schmidt, PL. (Ed.). Evidence-Based Veterinary Medicine.  Veterinary Clinics of North America: Small Animal Practice. 2007 May: 37(3).

Smith RD. Veterinary clinical epidemiology. 3rd ed. Boca Raton, FL: CRC/Taylor & Francis, 2006. 280 pgs. ISBN: 0849315662.

 

Electronic Resources
Evidence-Based Veterinary Medicine Association
http://ebvma.org (includes an extensive EBVM bibliography curated by Susan Whittaker)

Centre for Evidence-Based Veterinary Medicine
http://nottingham.ac.uk/cevm

VeSRev- A database of veterinary systematic reviews http://webapps.nottingham.ac.uk/refbase/

BestBetsfor Vets- Critically appraised topics
http://bestbetsforvets.org/

BARK_Banfield’s EBVM resources
http://www.banfield.com/veterinary-professionals/resources/research

BOTTOM LINE
EBVM is the formal, explicit integration of controlled scientific research into clinical decision making. It can reduce error and lead to better patient outcomes. However, published clinical research is not always reliable, and clinicians must carefully assess the limitations of specific studies and the applicability of their results to individual patients.

REFERENCES

1. Sackett DL, Rosenberg WMC, Muir Gray JA, et al. Evidence based medicine: what it is and what it isn’t. British Med J 1996;312:71.
2. Vandeweerd JMEF, Vadeweerd S, Gustin C, et al. Understanding veterinary practitioners’ decision-making process: Implications for veterinary medical education. J Vet Med Edu 2012;39(2):142-51.
3. Everitt S. (2011) Clinical decision making in veterinary practice. (Unpublished doctoral dissertation). University of Nottingham, U.K. Available at: http://etheses.nottingham.ac.uk/2051/
4. McKenzie, BA. Veterinary clinical decision-making: cognitive biases, external constraints, and strategies for improvement. J Amer Vet Med Assoc. 2014;244(3):271-276.
5. Gilbert R, Salanti G, Harden M, See S. Infant sleeping position and the sudden infant death syndrome: systematic review of observational studies and historical review of recommendations from 1940 to 2002. International Journal of Epidemiology. 2005;34:874–887.
6. Cobb LA, Thomas GI, Dillard DH, Merendino KA, Bruce RA. An evaluation of internal-mammary-artery ligation by a double-blind technic. New England Journal of Medicine 1959;260(22):1115–8.
7. Moseley JB, O’Malley K, Petersen NJ, Menke TJ, Brody BA, Kuykendall DH, Hollingsworth JC, Ashton CM, Wray NP . A controlled trial of arthroscopic surgery for osteoarthritis of the knee. New England Journal of Medicine. 2002;347(2):81–8.
8. Dru Forrester S, Roudebush P. Evidence-based management of feline lower urinary tract disease. Vet Clin North Am Small Anim Pract. 2007; 37(3):533-58.
9. McKenzie BA. What’s the evidence? There is only very weak clinical trial evidence to support the use of glucosamine and chondroitin supplements for osteoarthritis in dogs. J Am Vet Med Assoc. 2010 Dec 15;237(12):1382-3.
10. Mellanby, RJ. Crisp, J. DePalma, G. et al. Perceptions of veterinarians and clients to expressions of clinical uncertainty. J Small Anim Pract. 2007 Jan;48(1):26-31.
11. Stoewen, DL. Coe, JB. McMartin, C. et al. Qualitative study of the information expectations of clients accessing oncology care at a tertiary referral center for dogs with life-limiting cancer. J Am Vet Med Assoc. 2014 Oct 1;245(7):773-83.
12. Conzemius MG. Evans RB. Caregiver placebo effect for dogs with lameness from osteoarthritis. Journal of the American Veterinary Medical Association. 2012;241(10):1314-1319.
13. Sargeant JM, Elgie R, Valcour J, Saint-Onge J, Thompson a, Marcynuk P, et al. Methodological quality and completeness of reporting in clinical trials conducted in livestock species. Prev Vet Med. 2009 Oct 1;91(2-4):107–15.
14. Sargeant JM, Thompson A, Valcour J, Elgie R, Saint-Onge J, Marcynuk P, et al. Quality of reporting of clinical trials of dogs and cats and associations with treatment effects. J Vet Intern Med. 2010;24(1):44–50.
15. Elbers A, Schukken Y. Critical features of veterinary field trials. Vet Rec. BMJ Publishing Group Limited; 1995 Feb 25;136(8):187–92.
16. Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA. 1995 Feb 1;273(5):408–12.
17. Lund EM, James KM, Neaton JD. Veterinary randomized clinical trial reporting: a review of the small animal literature. J Vet Intern Med. 1998;12(2):57–60.
18. Brown DC. Control of selection bias in parallel-group controlled clinical trials in dogs and cats: 97 trials (2000-2005). J Am Vet Med Assoc. 2006 Sep 15;229(6):990–3.
19. Brown DC. Sources and handling of losses to follow-up in parallel-group randomized clinical trials in dogs and cats: 63 trials (2000-2005). Am J Vet Res. 2007 Jul;68(7):694–8.
20. Arlt S, Dicty V, Heuwieser W. Evidence-based medicine in canine reproduction: quality of current available literature. Reprod Domest Anim. 2010 Dec;45(6):1052–8.
21. Simoneit C, Heuwieser W, Arlt S. Evidence-based medicine in bovine, equine and canine reproduction: Quality of current literature. Theriogenology. Elsevier Inc.; 2011;76(6):1042–50.
22. Giuffrida MA, Agnello KA, Brown DC. Blinding terminology used in reports of randomized controlled trials involving dogs and cats. J Am Vet Med Assoc. 2012 Nov 1;241(9):1221–6.
23. Giuffrida MA. Type II error and statistical power in reports of small animal clinical trials. J Am Vet Med Assoc. 2014 May 1;244(9):1075–80