As we all know, the Internet is a mixed blessing. It makes enormous amounts of information easily and cheaply available to billions of people. Unfortunately, not all information is created equal, and sometimes bad information is worse than none. Contrary to what many of my clients assume, I actually appreciate it when they come to me with questions or suggestions based on their own research. However, when they have been misled by unreliable sources, a great deal of their enthusiasm and effort ends up being wasted.
I am hoping to contribute eventually to a more comprehensive set of evidence-based veterinary medicine resources under the auspices of the EBVMA, but in the meantime I have assembled some general guidelines and specific sources to trust, as well as some to avoid. I hope this is helpful.
General Principles for Evaluating Internet Information
1. Consider the Source: We need EBM precisely because even the most intelligent and honest of us cannot avoid the influence of bias on our conclusions. The universal cognitive biases and perceptual biases we all share, as well as the more obvious errors associated with strong feelings and preconceptions make a truly objective evaluation of any complex phenomenon impossible. The scientific method compensates for such sources of bias imperfectly, but better than anything else yet discovered.
When evaluating information from the Internet, it is important to consider the possible sources of bias lurking behind it. These biases do not automatically invalidate the information by any means, but being aware of them helps inject a salutary element of skepticism about their validity.
Obviously, anyone trying to sell you something is likely to believe quite deeply in what they are selling. Apart from deliberate scam artists, most people don’t enthusiastically promote or make a living selling things they aren’t strongly committed to, so they are unlikely to be truly dispassionate or objective. Confirmation bias is particularly likely to lead people with a financial interest in something to ignore potential problems with it. CAM proponents never tire of making this charge against mainstream medicine, especially the pharmaceutical industry, but they conveniently ignore their own financial interest in the products and services they provide.
Strong ideological positions also call into question the level of fairness in the presentation of an argument. It is so difficult to overcome the influence of one’s deepest held values and philosophies that it is often most useful to simply declare them up front and let those who are trying to puzzle out the truth take them into account. When looking at information provided on the Internet, it is often useful to find sites committed to both sides of an issue and compare the information and arguments.
Sometimes such ideological biases may be hidden, usually in an attempt to prevent people from recognizing them and the influence they may have over the information being provided. However, one advantage to the Internet is that it is often fairly simple to find out a lot about the affiliations and associations of people offering information to the public. My recent post on the association between conservative Catholics and anti-vaccine activists is an example of how hidden agendas behind information presented as objective and scientific can be uncovered.
Finally, while blind reliance on “experts” is a form of faulty reasoning known as the argument from authority, it is still true that specialized training and experience in an area does give somewhat more weight to one’s opinions on the subject. The idea that anyone can be an expert on any subject simply doesn’t hold water in today’s complex and technologically sophisticated world. Personal experience is especially unreliable as a source of deep insights that an entire profession of specialists have missed. A few hours on the Internet can’t make me into a nuclear physicist, and it can’t make a physicist into a veterinarian. So while no one is automatically right just because they are a specialist or professional in a subject, it is unlikely that dilettantes and self-made experts are going to have insights or wisdom that such professionals lack.
2. Check References: In CAVM it is very common to cite scientific sources in support of mistaken arguments. It is also often the case that when one reads the original source, it doesn’t say what the person citing it claims it says. Taking information or quotations out of context, overlooking obvious flaws in methods or argument, and simply cherry-picking sources that say what you want them to while ignoring those that don’t are all practices that make such research citations unreliable. So whenever possible, check the original source to see if it really does support the argument you’re evaluating.
3. Be Reasonable: If it’s too good to be true, it probably isn’t. Extraordinary claims require extraordinary evidence. Pick your cliché, but ultimately reality is complicated and messy and often not how we’d like it to be, so anyone who tells you it is simple and you can have your cake and eat it too is probably deluded or lying.
Trustworthy Internet Resources
These are resources that I believe provide information that is reliable. Being run by human beings, they may not always be right, but they are not regularly, egregiously wrong or out to push a product. Some may have obvious biases in favor of science and methodological naturalism, but if one rejects these positions then one has already decided rational and objective inquiry is impossible or undesirable anyway, so trying to evaluate the rationality and objectivity of Internet information isn’t likely to be a concern. Postmodernists and those who see faith and revelation as the keys to truth aren’t likely to find any of this useful anyway.
Many of these resources address human medicine primarily, and many include subjects outside of medicine all together. There is an unfortunate paucity of good resources for science-based veterinary medical information on the Internet, though obviously I am trying to change that! however, despite the dangers of extrapolation across species, the information gathered in human medicine can be used to assist our judgments regarding veterinary medical questions.
The SkeptVet- Though this blog is the more active part of my Internet project, I have assembled a collection of my more detailed and researched articles and my downloadable client information pamphlets on specific veterinary topics on my main web site. No commercial conflicts of interest, and my ideological biases should be quite obvious.
Evidence-Based Veterinary Medical Association (EBVMA)– This is an organization of veterinarians and other professions, both in academia and private practice, dedicated to promoting evidence-based veterinary medicine. Any veterinarian interested in supporting high quality, science-based veterinary medicine should join and get involved. And a new web site, with practical information and EBM tools for vets is coming soon!
VeterinaryWatch– This site contains a lot of good, science-based information and references, though unfortunately not in a very user-friendly form. I am working with several others to try and improve the design and content of the site, but it is a slow project.
Quackwatch– By far the most extensive collection of resources regarding CAM. Reviews of many products and therapies, warnings about corporate and individual providers or CAM services and information, and links to many other reliable resources.
Science-Based Medicine– Far and away the best blog on science and evidence-based medicine and a great resource on CAM. Intelligent, thoughtful, and well researched essays on a large variety of topical as well as perennial issues.
The Cochrane Collaboration– The premier site for EBM in the human medical field. Though some resources are available only on a subscription basis, the most helpful resource are summaries of the independent and rigorous systematic literature reviews on a large number of specific topics.
PubMed– An enormous and easy to use database of the medical literature. Abstracts are available for most articles, and links to sources for the full text. Unfortunately, full text access for many articles is available only to subscribers of the publishing journal, but the world of medical publishing is changing, albeit slowly, and more journals become accessible all the time.
Free Medical Journals– A convenient way to identify journals that offer free online access to their content. While many of the highest impact journals in human medicine are open access, far fewer veterinary journals are available–yet.
Unreliable Internet Resources
While the number of commercial organizations and individuals pushing misleading information or outright nonsense on the internet is vast beyond imaging, this is a list of some of the more popular, and hence more dangerous. And while the distinction between bad science, pseudoscience, and plain quackery can sometimes be a tough call, a place to start for the more egregious cases is The Quackometer. This automated tool evaluates websites for language patterns typical of pseudoscience and medical quackery. Of course, I would never suggest letting a machine make your decisions for you, but I’m surprised by how often I agree with the little black duck!
Shirley’s Wellness Cafe– This is perhaps the most egregious woo site I’ve found to date. Vicious and paranoid condemnation of all scientific medicine as paternalistic, venal, and harmful. Wild claims about safe, natural, free cures for almost everything. No logic or data, only the presumption that if it is in any way scientific it must be bad for you. And despite the railing against the greedy medical establishment, there sure seem to be a lot of things for sale!
Healthy Pet Journal– A sad example of veterinarians giving up on science in favor of blind faith and intuition. The site doesn’t have the hysterical shrieking tone of Shirley’s, but that makes it even more dangerous since the information is presented as if it were reasonable and scientific, when it almost universally is neither.
Naturallycomplementary.com– A beautiful, well-organized, easy to use resource for avoiding science-based medicine and finding all things CAVM. A huge variety of unrelated and mutually contradictory approaches are promoted, with the only apparent unifying feature being claims of “natural,” “alternative,” “holistic” and so on. Words which have lost whatever meaning they may once have had and have become merely shibboleths for anti- and pseudoscience. I tried to register as a member of the forum to offer a different perspective, but apparently the management is not interested in allowing members to hear other points of view, and they denied my application.
Academy of Veterinary Homeopathy– A professional organization of veterinarians devoted to this tooth fairy science.* (sigh)
American Holistic Veterinary Medical Association– Ditto. This one is good for some laughs if you check out the offerings at their annual CE meeting.
Sadly, I could go on endlessly. While thankfully science-based medicine is far more widely available and accepted than CAVM, promoting CAVM on the Internet is a viable commercial activity, while promoting EBVM and skepticism is a strictly pro bono volunteer task. Consequently, the volume and shiny façade of the CAVM offerings will always outshout and outshine the more rational alternatives. But once you examine a few examples of unreliable sources, the patterns of language and argument become obvious, and your internal quackometer should steer you straight.
*”This study falls into the category of what I call Tooth Fairy science. You could measure how much money the Tooth Fairy leaves under the pillow, whether she leaves more cash for the first or last tooth, whether the payoff is greater if you leave the tooth in a plastic baggie versus wrapped in Kleenex. You can get all kinds of good data that is reproducible and statistically significant. Yes, you have learned something. But you haven’t learned what you think you’ve learned, because you haven’t bothered to establish whether the Tooth Fairy really exists.” Harriet Hall, MD
Shirley’s wellness cafe……….words fail me – urine therapy for animals…Eeeeuuuuuuuuw, as they say. What a load of – better not say.
Pingback: Evaluating Medical Advice and Credentials on the Internet « The SkeptVet Blog
You are absolutely correct about the lack of readily available, easily accessed, evidence based information for pet owners. You have to sift through several pages of google results to get to something that isn’t holistically, naturopathic, and / or conspiracy theory based. Sometimes outright dangerous. Thank you for your efforts to correct this.