Here are the notes and slides for a recent presentation on strategies for effectively choosing diagnostic tests.
GOALS OF DIAGNOSTIC TESTING
Ultimately, the goal of any test we run should be obtaining information that allows us to more effectively treat or prevent health problems in our patients. This seems obvious, but it is all too easy to lose sight of this core purpose. We may feel obligated to run tests to confirm a diagnosis even when the level of confidence is already high and the outcome of the test won’t change what the client chooses to do. We may employ diagnostic tests as a preemptive defense against litigation or because of a perceived pressure from the client to do something even when our action likely won’t change the outcome for the patient. In some situations, we may be completely confused by a case and throw a bunch of tests at it hoping for some insight to emerge.
All of these are understandable, and all too common, reasons for using diagnostic tests, but unfortunately such approaches reduce the reliability and utility of the tests themselves. Effective testing requires not only an understanding of the strengths and weaknesses of the tests we use but also a clear understanding of how to employ them and how to integrate the results into our clinical decision making. We need a rational strategy for when and how to test, how to interpret results, and somewhat counterintuitively, when not to test at all.
BEYOND SENSITIVITY AND SPECIFICITY
The most common measures used to describe diagnostic test are sensitivity and specificity . These are characteristics of the tests themselves, and they indicate how likely, compared with some gold standard, a test is to correctly identify a disease which is present or to correctly identify that a patient does not have the disease. Unfortunately, the meaning of these numbers is often misunderstood. If a test has, for example, a 98% sensitivity, this is the proportion of patients with the disease who will correctly test positive. It is NOT an indication that any patient who tests positive has a 98% chance of having the disease. Under certain conditions, the majority of patients testing positive on such a test may actually not have the disease even with such a high sensitivity.
More clinically useful measures of a test’s reliability are the positive predictive value and the negative predictive value (Fig. 1). These are, respectively, the probability a patient with a positive test actually has the disease and the probability a patient with a negative test result does not have the disease. These numbers depend not only on the test but also how common the disease is in the population being tested.
As an example, if a population of feral cats has an FIV prevalence of 2%, 2/100 cats tested will test positive with a perfectly sensitive test (sensitivity=100%). If the test also has a specificity of 98%, then about 2/100 cats will test positive even though they do not have FIV. The positive predictive value, then, is 50%, meaning half of the cats who test positive do NOT have FIV. Even with a great test, this is a pretty big error rate, especially if we are planning on euthanizing cats diagnosed with FIV!
This example illustrates how important it is we have some idea how likely a disease is to be present before running a test for that disease if we want our test results to be reliable. Which brings us to a new and somewhat fashionable way to look at diagnostic testing….
BAYESIAN ANALYSIS FOR THE MATHEMATICALLY CHALLENGED
The work of 18th-century mathematician Thomas Bayes is enjoying something of a renaissance as an alternative, in some respects, to the frequentist statistical methods most of us were taught in vet school. The details of the math involved are complex, but the logic of the approach is simple and intuitive. Diagnostic tests should not be viewed as determining whether or not a disease is present. They should be viewed, instead, as one piece of evidence shifting the existing probability of a diagnosis higher or lower.
If, as in the example above, I know that the prevalence of FIV is 2% in this population of cats, I can say the probability of any given cat having FIV is very low. A positive test does not mean a cat has FIV, only that the probability it might have the disease has increased a bit. The test doesn’t make or break the diagnosis, it simply shifts out understanding of the likelihood of the diagnosis.
In a practical sense, then, a Bayesian approach means estimating the probability of a diagnosis based on all of the usual factors we consider (signalment, personal history, prevalence rates, physical exam findings, other test results, etc.). If this probability is high enough or low enough to make or rule out a diagnosis, no additional test is needed. If, however, the probability leaves significant uncertainty, then we should select a test that will meaningfully raise or lower that probability to help us make the diagnosis.
SCREENING
Screening is a special case in which we are testing asymptomatic individuals with the idea of detecting preclinical disease so we can more effectively intervene to reduce symptoms and mortality. Because the prior probability of disease is usually very low by definition in screening, since patients have no symptoms, the positive predictive value of even very good tests is low. It has been recognized in human medicine that screening can often lead to overdiagnosis and overtreatment, which can waste medical resources and ultimately do more harm than good for patients.1There are, therefore, requirements for screening programs, and these include not only accurate tests but proven interventions that actually improve outcomes for patients diagnosed with the disease and rational plans for confirming and following up both positive and negative test results.
In veterinary medicine, we often employ diagnostic tests in asymptomatic patients “just in case” we might find subclinical disease. Whether or not such testing improves outcomes for patients or leads to significant overdiagnosis is almost never evaluated, so the benefits and risks of screening are often assumed but not truly known. This means that significant caution is warranted in conducting screening and interpreting the results of diagnostic tests in clinically well individuals.
CARDINAL RULES OF DIAGNOSTIC TESTING
Based on this understanding of the limitations of diagnostic testing, there are a few cardinal rules we can apply to reduce the potential mistakes and harms resulting from our tests:
Cardinal Rule #1
If the results of the test isn’t going to change what you do, don’t run the test.
Cardinal Rule #2
If the prior probability of a diagnosis is very high or very low, don’t run the test.
Cardinal Rule #3
Don’t screen (test asymptomatic individuals) without a plan of action based on solid evidence that the benefits of testing and diagnosis outweigh the risks.
REFERENCES
- McKenzie, BA. Overdiagnosis. J Amer Vet Med Assoc. 2016;249(8):884-889.
Why do some vets recommend “Early Dz Det CBC/Profile” as part of the annual exam for asymptomatic dogs (especially for seniors)?
When I ask why and if it is important, I am told “We can’t tell everything by looking”. I go along with it as over the years I have had a dog or two come down with sudden disease and I will never know if preventative screening would have made a difference.
Also, I guess it depends on the vet, my friend’s senior dog recently had an annual checkup and her vet said lab work was not necessary at this time, other than heartworm/Lyme test.
Her annual vet visits are half the cost of mine. :-/
How do you feel about screening of liver values and diagnosis before onset of symptoms? Your previous posts weighed heavily on my mind when my dog had bloodwork done prior to being prescribed Rimadyl, and we saw his liver values were high and did not go down/did increase with subsequent screenings. We ended up opting for a biopsy though his enzyme levels were still not horrendous, and he ended up being diagnosed with copper storage disease. He now is on a chelation therapy which is messing with his appetite, and I tell myself its better for him in the long run than having waited until he was symptomatic and in liver failure but it does feel strange having him be sicker after starting treatment than before. I am pretty set in my decision to continue treatment under the advice of his internal medicine specialist and his liver values have already dropped for the first time after only 2 weeks of pennacillamine (he had multiple checks over several months before this, ALT was stable and ALP continued rising) so hoping we are on the right course. Just from my reading by the time liver disease causes symptoms it is typically pretty advanced, though its hard to say when my 10 year old boy would have hit that point.
Well, the issue is not a clearly settle done. Vets are understandably afraid of missing an important abnormality that could allow them to help the patient. The logic of trying to detect disease early to get a better outcome is rational, it just isn’t clear that a better outcome is the actual result. It might be, but it often turns out not to be for humans, which raises a question about the value of such testing in vet patients that we need to generate data to answer.
The bottom line is that we don’t know when such testing will help and when it won’t since we don’t have the data. It is perfectly reasonable to recommend testing asymptomatic individuals, particularly when screening for occult disease that might impact the safety of subsequent treatment. And there is no question that in some cases it will benefit some individuals. However, the data in humans shows us that in other cases it will harm some individuals, and the balance of risks and benefits for our patient population as a whole is what we don’t know. If I help some and harm more, then it makes sense not to do such testing. If I help more than I hurt, then I should do the tests. In either case, some will benefit and some will be hurt, and tat’s unavoidable. Unfortunately, we don’t have the kind of data we need to know what the balance is, so we are likely to err on the side of testing since we fear missing something more than we fear doing something unnecessary that leads to harm indirectly later. That’s natural, but we just need to keep in mind the possibility that it’s the wrong choice, and we need to make the effort to generate the data we need to find out.
Thanks for your response. Also, I think sometimes the vets may suspect something based on the dogs history, breed, condition and observations during the physical exam.
I don’t think they say much because they don’t want to alarm or upset the owner, I also think that vets may make a decision based on what they perceive to be the financial means of the pet owner.
For example, they hesitate to recommend extensive testing if they know the owner is on a fixed income, or like you implied, the treatment options may be limited no matter what is discovered.
“If you own a Cairn Terrier you may be interested to learn more about Ocular Melanosis”.
https://www.crctc.org/calendar/event/213-eye-clinic/
The dogs eyes are checked during the annual vet visit. Should I be asking for a specific test to be done? That site is recommending additional testing.
I know you have not examined my dog, just interested in any thoughts you or others that participate here may have.
Thanks
As a small animal vet, I find this issue interesting, and our practice is constantly evaluating and reevaluating our protocols. Although I have seen the studies that seem to show the harm in overscreening human patients, I am hesitant to draw too much of a parallel to veterinary medicine because of the lack of our ability to rely on firsthand accounts from our patients.
For example, how many clients can accurately assess how much a cat is drinking and urinating, especially if they have more than one in the house? Even as a veterinarian, I would be hard pressed to immediately notice this in one of my own cats, since they share litter boxes and water bowls. However, on our senior wellness screening, we often find cats and dogs with isosthenuria (significantly dilute urine), which could signal diabetes, early renal disease, etc. In a human patient, this screening would be done much sooner, often at the first onset of signs. A (human) friend of mine noticed he was much thirstier and urinating more over the course of a week or so, and he was diagnosed with diabetes very shortly thereafter.
Adding to this is the phenomenon (in cats especially) of masking clinical signs of disease for a long time, owing to their status as both a predator and prey species, and being programmed to not mark themselves as a target, so to speak. So, the cat in kidney failure I saw last week (who has a fairly normal appetite and has only dropped a little weight) has extremely abnormal renal and electrolyte values that should benefit from treatment. Yes, some of the clinical signs of advanced disease will be obvious to the trained eye during a physical exam, but there are many reasons why that may not be the case (the dog who is too obese to palpate the splenic tumor, the cat who is too aggressive to look in his oral cavity, the dog who is breathing too hard to appreciate a subtle heart murmur…). Because we can’t ask them how nauseous they feel, or if it hurts to urinate, or if they feel dizzy after they eat a meal, we miss a lot of information that would guide a human physician to a diagnosis much more quickly. Sometimes we find a problem that we can’t fix, but can let an owner know that their pet has a poor prognosis, and they can spend their time left with him/her accordingly.
Lastly, just because we find a lab abnormality does not mean we need to start treatments that may cause harm – it may have more subtle benefits, like guiding a dietary choice, or having the owner watch more closely for specific changes. Perhaps I see a dog who shows early signs of renal disease, such as protein in the urine, and choose to start him on a fatty acid supplement and a lower phosphorus diet. Or I notice an asymptomatic patient whose thyroid tests are slightly low, so I have the owner check her weight more often, so I can have them identify a point when he starts to gain inappropriately (and would then benefit from treatment). A prudent clinician should be able to balance the assertiveness of the treatment with the degree and stage of disease. In doing so, we can minimize the harm done from overly aggressive screening, making the occasions that we DO catch a big problem in time to correct it well worth it.
You are quite correct that there are differences between veterinary and human medicine that might be relevant to the balance of risks and benefits associated with screening tests. The lack of self-reporting of symptoms and the unreliability of owner histories would suggest such testing might be more likely to be beneficial in our patients than in humans. Of course, this still needs to be demonstrated with appropriate research to assess the risk of overdiagnosis and associated harms. My argument is never that we shouldn’t do such testing, only that we don’t yet have real evidence to support the benefits we believe it offers, only anecdote and logical but untested reasoning. Given the number of owners who decline diagnostics and therapy for any number of tests and conditions, it ought not to be difficult to collect data comparing individuals who have such testing and those who do not and see if the testing reduces morbidity and mortality. Such studies would be very useful in guiding our clinical practice.