As I discussed in a previous post, the call for a reliance on sound evidence in veterinary medicine resonates less strongly that we might hope because of the paucity of such evidence. Financial resources are the main limiting factor in obtaining good-quality basic and clinical research evidence, but apparently the attitude or culture of the profession may be a factor as well. According to a recent news story from the Veterinary Information Network (VIN), a widely used information and social resource for veterinarians, Dr. Richard Vulliet, a researcher and professor at the UC Davise School of Veterinary medicine has accepted a $100,000 grant from the AKC to study the use of bone marrow stem cells in degenerative myelopathy, a mysterious and all too common disease in which older dogs gradually lose the function in their hind legs.
Dr. Vulliet is clearly a trained, experienced researcher who should understand the importance of properly designed and conducted trials. Unfortunately, if the comments and attitude attributed to him in this article are correct, his frustration with the pace and limitations of proper research have led him to use the resources he has to “try out” some ideas without the structure or controls that make the results meaningful or useful to the rest of the profession.
According to the article, “Vulliet has treated four dogs with degenerative myelopathy with their own stem cells…Vulliet derives the mesenchymal stromal cells from bone marrow. He injects the cells systemically into the circulation because it appears that they home to an area of injury.”
The idea for the project apparently came from some laboratory rat research. “Vulliet says he got interested in treating these conditions because he was working with mesenchymal stem cells and their interaction with connective tissue, and it was boring. Then he came across two papers. In one of the papers, Japanese researchers described treating induced cardiomyopathy in experimental rats (Circulation 2005;112:1128-35)…Vulliet has not treated any dogs with dilated cardiomyopathy. But he has been in contact with Doberman groups to recruit possible subjects…In the other paper, researchers at Tulane University in New Orleans induced spine injuries in experimental rats and treated them with mesenchymal stem cells.”
Clearly, Dr. Vuillet recognizes some of the limitations of a research project based on limited information about plausibility and mechanism, “When I talk to possible clients, I generally get the impression they think I know what I am doing. But no, this is research.” Unfortunately, the limitations don’t seem to bother him as much as those of properly conducted scientific research, which he has chosen not to pursue because “the protocols he would be forced to adhere to at the university, working with pet dogs, would be too cumbersome.”
The article goes on to describe the haphazard followup of subjects, including one dog whose owner, apparently a neighbor of Dr. Vuillet, almost forgot to mention that she believed the dog had improved after treatment. The issue of potential hazards also arises, when the article mentions that 1 of the 4 dogs treated so far eventually developed a “malignant tumor on its spleen.” Clearly, splenic tumors are quite common in elderly dogs of the breeds Dr. Vuillet is studying, but any association, causal or chance, between the treatment and such disease is unlikely to be detected by casual, unsystematic followup.
What Dr. Vuillet seems to be doing is selecting a few owner with dogs who appear to have degenerative myelopathy (though how that is confirmed is unclear from the article) and who appear to have sufficiently low expectations for benefit from the procedure, and then giving them some stems cells and asking the owner someday how things are going. The value of data collected this way is so low that it is hard to justify spending $100,000 of the scarce funds available for veterinary research on it. Even more disturbing, from someone who clearly ought to know better, are statements suggesting this independant, idiosyncratic, approach to research is somehow superior to proper clinical studies; “ I think we will learn more from these dogs than from the thousands of Ph.D.s who are experimenting in the labs.”
If this is the attitude towards science that professors at our veterinary schools are teaching to their students, I fear that better studies and better data will not be more available in the future. And while I am only speculating, I will be very interested to see if Dr. Vuillet eventually introduces some version of stem cell therapy to the veterinary market as an entrepreneurial inventor in the coming years. Such a therapy would be more than welcome, and I would be eager to provide it to my clients, if the proper evidence for its safety and efficacy were available. I am doubtful, however, that such evidence will come from projects such as this one.
I am afraid that that is the attitude that quite a few students are getting from their college and veterinary school careers. It is very easy to get the impression that science is a noun rather than a verb, and that any clinical experience qualifies as scientific evidence. There was very little in the veterinary curriculum that explained how we know what we know, and how the scientific method has resulted in medicine as we know it today, versus all the pre-scientific systems that preceded it.
Colorado State had a summer program for first and second year veterinary students that I was able to participate in. We had to give brief presentations to each other at the end of the summer, and some of the students were never told, and never thought to ask, why they did what they did for the whole summer. They did not even know what question they were trying to answer. It was a bit shocking and embarrassing to see. Because of that experience I am not too surprised that many of our colleagues don’t really get the epistemology of medicine and science.
Nice to see vibrant discussion on a topic like this.
To be honest, I normally read all VIN News Service articles before posting, but last week I was so overwhelmed iwth other issues that this one went out without me reading it. Reading this post, I was concerned that poor reporting on an irresponsible research project had slipped by when I wasn’t looking
I just finished reading it and I am confused by the reaction.
As I read it, Rick (who I do know personally — and I suggested the story be done after hearing what he was working on and how ), had an idea based upon prior studies that he wanted to “try out” (I’m not even saying “test”)
Something I didn’t know about Rick until we briefly talked and from reading this story, was his frustration (that I share) with how current funding and research practices promote research that often “doesn’t matter.” We all too often fund and collate lots of data in rats which never proves useful in human or the major veterinary species. Billions of dollars are spent this way — following protocol — while a pittence of this amount would be wonderful to use in more practical veterinary studies.
Reading the article, what I hear is:
1. Rick had an idea
2. Rick wanted to pursue the idea
3. Rick thought he knew how to get the cells he was after in large enough numbers to use
4. I’m assuming Rick believed injecting those cells was safe (how he arrived at that I can ask him)
5. As a first step, Rick wanted to “just give it a try.”
6. Rick has been totally honest with the owners involved — basically saying, he’s just going to give it a try and see what happens. He has made no claims or warrantys and has approached only owners of dogs who know they have a dog with an incurable condition.
7. Rick has not taked anyone out of other treatment options or made any claims his would be better
8. Rick has funded much of this himself because he believes it is worth a try — and wants to find answers to important questions.
9. Rick has — by his own admonition — very early uncontrolled clinical outcomes that are noteworthy, but do not amount to proof of efficacy.
10. Nothing in the article overstates the results. Nothing in the article claims success.
The article sounds to me like throwing an idea — an educated idea — against a wall and seeing if it sticks.
Are we that afraid and stuffy, that we can’t see that this is what our profession likely needs more?
===before I get skewered for that last statement, let me clearly state what I would have been upset to read now or in the future.
I would be very upset and disappointed if I read CLAIMS OF EFFICACY before more careful and controlled studies were done.
But I read none of that. And I applaud Rick for taking the chance he is taking. It is courageous. It is putting one’s time and money where one’s mouth is.
And it is being done on the right types of diseases (ones we know near nothing about helping) with the right hopes for outcomes — helping — even a little — cause we can’t help at all at this time in most cases.
Believe me, when I started sprinkling white powder (Taurine) on cats with incurablly damaged hearts (DCM) in 1986, many laughed at me. Many tried to destroy my reputation. Some even tried to ruin my reputation for implicating commercial cat foods as the cause of many cases.
Also know that I made no claims of efficacy — even after some early “successes” — until I was sure. We made no claims to owners or anyone, until we could almost predict success. And we, like Rick, didn’t pursue controlled trials as our initial design. The results were so dramatic — and the well -documented prognosis for these cases was so dismal, that we were willing to pursue a “good idea” that came from an unrelated observation.
The only difference with how we approached our studies and Rick’s approach was that we were at the university and were willing to ask forgiveness for our methods rather than ask permission. Believe me, some tried to shut us down. But after our initial positive results I’d have none of it. I sensed that we were “on to something.”
Still, we made no claims publicly — we only said we had a new idea for a dismal disease and asked for cases to try it on.
And as the data piled up, it became clear that controllled trials on client-owned cats would be unethical. The results were that clear.
That doesn’t mean we questioned the value of controlled trials. By the time we had enough data to publish in the journal SCIENCE without controls, we had submitted several grants with classical designs in patients and “colony cats”
We aborted the controlled trials in patients because we believed them unethical and pursued the more controlled trials in colony cats to try to better understand the nuances of the treatment effect and underlying pathology/pathophysiology.
You can be sure that if Rick sees this pasta “stick to the wall.” He will similarly pursue more controlled research.
And even if it appears to stick, if the results are not as clear cut as ours were, I’m sure that Rick will pursue more controlled trials before making claims. He is that kind of guy.
Still, I applaud his approach. It is unlikely he is doing harm (but yes, that one tumor is reason to keep our eyes open).
But if this works, Rick can help patients much sooner. And if this doesn’t clearly work, then it is back to the drawing board.
Because to be honest, I’d not pursue trying to detect small effects. They just don’t matter for this type of research.
Small statistical differences are for building careers and approving marginally effective drugs, not making a real difference for patients.
No one believes more in the need for evidence based veterinary medicine than I.
At the same time, I think that common sense needs to rule and that the best controlled trial “proving” a small and clinically insignificant treatment effect is a waste of effort compared to an observation that clearly demonstrates a treatment effect in black and white — even if uncontrolled.
Do we need controlled clinical trials to tell us that survival is higher in those who jump out of airplanes WITH parachutes vs those who jump WITHOUT parachutes?
If I give furosemide to a dog in CHF, and the dog pees and the edema resolves, I am happy concluding the furosemide worked and was effective. I am satisfied making this conclusion despite the fact that there are NO published controlled trials testing this in canine patients.
I say we should all applaud Rick’s courage in undertaking this and sit back and see what comes of it.
To all who have concerns with this I say two things:
1. What are YOU doing (and putting on the line) to try to help patients with these or other incurable conditions?
2. If at any point Rick starts making claims of efficacy without quality data to back those claims — skewer him and castrate him. But until then, stand back, be proud that some amongst us have the courage to take chances — cause the way we have been approaching these and other diseases over the past few decause ain’t getting us “there”.
I don’t know what are the chances that Rick’s approach will lead to a cure or benefit. But I see no harm in letting him try. What is there to be afraid of?
I’ll end with a statement I’ve said many times about my experience with throwing pasta against the wall to see what sticks — finding the relationship between taurine deficiency and DCM in cats.
“it didn’t come from brilliance. It came from reaching the point where if someone told me arsenic would help I’d have tried it, a lot of luck, a lot of background knowledge, a lot of help from others, and the courage to persevere when many around me who “knew better” told me it couldn’t be.”
>>Paul<<
Paul D. Pion, DVM, DACVIM (Cardiology)
co-founder, VIN
Davis, CA
I agree a vibrant discussion on this subject is all to the good. It is an excellent way to explore underlying issues about epistemology within veterinary medicine.
One of the luxuries of our profession is we aren’t watched 24/7 by lawyers waiting to pounce on us for any little variation from standard protocols. And since there are no evidence-based standards for much of what we do, we often have great latitude to try out our best guess and see what happens. There’s nothing wrong with that, and despite the free use of words like “stuffy” and “afraid,” I don’t see anyone suggesting that we never make a move without data we’d feel happy to publish in SCIENCE.
On the other had, it doesn’t take much effort to identify all sorts of common medical practices that are followed because they make sense and appear to work when in fact we’re being fooled. How long did we give young cats with hematuria antibiotics and watch them get better (for which we took the credit) before somebody did some research and figured out they had sterile cystitis and the practice was unjustified? And at the risk of getting into even more contentious territory, I know plenty of vets who justify their use of homeopathy, pet communicators, and all sorts of nonsense because their experience tells them it works. And the same thing happens on the human side. Mammary artery ligation was a great surgical therapy for angina with >90% efficacy, until someone figured out sham surgery worked just as well.
The problem with the “rough and ready” individualist approach to medicine is that there seem to be many more bad ideas that get “confirmed” by sloppy research than good ones. We’re still trying to get rid of the ideas that Hahneman and Palmer and Bach cooked up and “validated” with their own idiosyncratic experiments. I submit that evidence-based medicine may be less glamorous and dramatic, and for some less emotionally satisfying that going with your gut, but that it is ultimately a better and more efficient use of scarce resources and leads to more effective medicine.
It’s not enough to just be honest that we’re guessing and have no idea what will happen. We need to acknowledge the natural limitations of personal experience in deciding what is or isn’t an effective therapy and use the methods of science which compensate for these limitations. This is not pedantry of timidity, it is sound judgement supported by the evidence of history. The improvements in human and veterinary health care since the advent of science-based medicine outshine the entire prior history human medical understanding and practice. The methods works better, and that’s why we should rely on it rather than our personal inspiration, hunches, or playing at lobbying pasta at the wall. What am I doing to help find real treatments for our patients? Well, to start with I’m trying to encourage my colleagues to stop relying on the outdated and inferior epistemology of “clinical experience” and rely on sound scince and evidence based medicine.
****Since Dr. Pion copied his response on the VIN thread to the comment section here, I’ve taken the liberty of continuing to reproduce the discussion verbatim from that site for anyone interested in following it but without access to VIN. I have left out posts that pertained only to VIN membership issues not the primary thread topic.****
****At Dr/ Pion’s request, I am removing his posts to the VIN discussion thread that I copied here****
Paul,
I’m not “afraid” of anything or I wouldn’t be arguing with someone as distinguished as yourself, would I? :-). Nor do I think there is anything “religious” about my opinions on this subject, which I hold for the reasons I have outlined, not out of any sort of blind faith. I do think the references to ineffective therapies are relevant, and I’ll try to explain why.
The underling issue is how we decide what is or isn’t an effective or safe medical therapy. Do we try it out and see what happens, or do we apply the principles of controlled scientific research to it? I didn’t critique (and I think that’s a fairer description than “skewer and castrate”!) the project in the original article because Dr. Vulliet was making unfounded claims but because he conveyed the impression, which you see to agree with, that uncontrolled, unblinded, haphazard trials by individuals with a hunch are superior to formal scientific research. You referred to your own work with taurine as an example of how successful this kind of approach can be, so I was merely pointing out that I think there are many more examples of the approach leading to error and wasted time and resources.
I don’t object to people investigating ideas that interest them. I do object to them basing conclusions of uncontrolled personal observations rather than real science. You’re right that so far Dr. Vulliet hasn’t offered any strong conclusions, so I’m not criticizing him for doing so. I am certainly NOT saying I have any evidence for or against his idea, nor do I have any a priori opinion on whether or not it will work. But presumably he’s pursing the project in the way he is because he thinks it will eventually lead to reliable conclusions, and do so better and faster than formal research. Why else would he be doing it? And what I am not convinced by is the idea that that is a justifiable expectation. I hope his idea is right, and I think it is sufficiently plausible to justify real research, not just a haphazard personal trial that won’t prove anything either way.
As I’ve already said, stem cell research is an interesting area that deserves lots of study. But there are already on the market expensive, invasive stem cell procedues sold by testimonial and anecdote to paying clients without what I believe is adequate evidence to justify them. And it is the idea that we don’t really need well-designed clinical trials when we can just try stuff out and make up our own minds based on that that leads to such practices, which I think are ethically and scientifically questionable. And the same attitude supports the continued reliance on clearly ineffective therapies like the ones I mentioned before.
The issue is not that stem cell therapy is the same as homeopathy. Clearly the latter is nonsense and the former is a promising idea based on sound scientific principles. The issue is that believing our personal experiences to be as or more reliable than real scientific research leads us to believe in things that aren’t really true, and I still maintain a solid reliance of EBM works better.
I would like to know why my prior post is not appearing here?
Are we censuring?
Paul,
I have no idea what you’re talking about. Everything you posted has appeared in the comments section. The only thing I have deleted was the one post from the VIN thread that you requested I delete. You are, of course, free to post any comments you like and if it is consistent with the usual rules of civility it will appear here.
Thanks — perhaps it was a technical glitch. Unfortunately I didn’t make copies of the now missing posts.
****I will continue to post my responses to discussion of the original article here, though I will respect Dr. Pion’s request not to quote anything said by others in the discussion without express permission****
Paul,
I will be happy to talk with you, though I’m not sure why that is necessary. As requested, I have also removed all comments copied from here to my blog. I’ve also approved without editing all comments you’ve posted there.
I really hope we can keep this discussion focused on substantive issues and not let it become some sort of personal conflict. I have no desire to challenge the rules of VIN, and if you feel my participation here is somehow inappropriate I’m happy to withdraw. I only intended to express a personal opinion on what I consider an important issue in veterinary medicine, and I would hope we could discuss the merits of the ideas involved without acrimony.
****In response to a “defense” of Dr. Vulliet by the reporter who wrote the original article. Clearly, my tone has led people to the mistaken impression that I intended to attack Dr. Vulliet personally. I tried to qualify myself repeatedly and emphasize that it was the attitude about research and EBM I was concerned with more than the original project or the person, but that seems to be consistently ignored no matter how many times I repeat it.****
I still feel that some are personalizing this discussion unecessarily. Perhaps this is partiallly because the most passionate rejections of my concerns have come from a personal friend of Dr. Vuliiet who suggested the writing of an article about him in the first place (Dr. Pion) and the author of the article. In any case, I am far more interested in the general resistence to EBM and the penchant of veterinarians as a whole to rely excessively on their clinical experience to judge treatment success or failure than I am in Dr. Vulliet or his project. Despite the fact that I keep tryng to redirect the conversation to that topic it keeps being cast as a personal vendetta of some sort, which simply isn’t accurate. To the extent that the tone of my own article was passionate as well, I accept some respoinsiblity for this, but I really wish we could focus on issues not persons here.
I set up no strawmen nor did I attack Dr. Vulliet personally. What I did was respond to the presentation in the article of an attitude that suggests impatience with the niceties of well-designed academic research and a suggestion that less controlled research is preferable. The following remarks illustrate the tone I think some of us perceived, not without reason I would argue:
“The owner did not immediately tell Vulliet about the dog’s improvement. It came out by chance during a conversation. “Oh, yeah, by the way, I forgot to tell you. But I think he is a lot better,” Vulliet recalls the neighbor saying.
That kind of lax reporting is one of the pitfalls of doing the research the way Vulliet is conducting his.”
Note, “lax” was the very word used in the article, and the followup certainly sounds haphazard even for a pilot study.
“Vulliet says he got interested in treating these conditions because he was working with mesenchymal stem cells and their interaction with connective tissue, and it was boring”
Fair enough, but connected with the final comment in the piece shows a disdain for prleiminary laboratory studies which I think is inappropriate, as these often provide necessary guidance in deciding efficiently where clinical trials are warranted.
“Vulliet isn’t even doing the research at UC Davis. He has rented a small, nondescript office in town, which he is paying for himself and with a $100,000 grant he received from the AKC Canine Health Foundation. The protocols he would be forced to adhere to at the university, working with pet dogs, would be too cumbersome, he says.”
This may very well be true, but those protocols exist for legitimate purposes: the protection of subjects and the production of reliably predictive data. Discarding them may make studies easier and faster, but there is likley to be some sacrifice in terms of quality.
““I think we will learn more from these dogs than from the thousands of Ph.D.s who are experimenting in the labs,” he says”
Again, I think this is inaccurate and represents a disdain for formal academic research, and I believe that attitude holds back progress in veterinary medicine. We have a very shallow pool of high-quality data to draw on when evaluating therapies, and part of the reason is because we sometimes don’t hold ourselves to a high enough standard. Of course resoource limitations are a far bigger problem, but that is precisely why we should use our resources as effectively as possible to try and get the best quality data out of them we can.
Despite the increasing temperature of the discussion hear, I have said repeatedly and will reiterate:
1. I intend no personal disapragement of Dr. Vulliet, whom I do not know at all and have no reason to think is anything other than a smart, experienced veterinarian doing what he can to find beneficial treatments for a serious disease.
2. I have no prior opinion on the liklihood that his experiements will or will not lead to a useful treatment. They certainly seem plausible in terms of basic principles.
I do have some concerns that the preliminary work he is doing as it is described in the article will not really tell us very much about the safety or efficacy of the treatment. Whether it “looks like” it is working or not on this scale really doesn’t seem to me the best way to decide whether it is worth pursuing more formally. Everyone seems to assume that the subjective impressions conveyed by this kind of study will be useful in guidng further research, but I am not convinced that’s necessarily true.
3. If I have misinterpreted Dr. Vulliets sentiments based on the selections from the article given above, I apologize. I don’t think I am wildly misreading them, but I’ve said before and will say again that I’m only responding to the impression they create, and this may not be in any way related to the relaity of the research or the philosophy behind it.
As for vet stem, I have alluded to that twice now myself. I agree that the published trials in canine djd, one uncontrolled and one controlled with minimal evidence of efficacy, are inadequate to justify marketing the therapy, though they are certainly promising enough to support more definitive research. The fact that such minimal data is seen as sufficient justification to market and use the therapy in pets is exactly the kind of attitude I am concerned about. It clearly exists in the profession, and I am not making it up. It seemed to me, for the reasons stated above, to be involved in Dr. Vulliets work as well, but regardless of whether this is actually true I think it’s an attitude that is widespread and needs to be challenged. Dr. Pion seems to agree, though he does not see it in Dr. Vulliet’s work but in other places. Fair enough. Can we then agree that the problem exists and talk about that, which was my point in the first place, and get away from this rhetoric of “atttack” and “defend?”
Interesting how many seem to be proving your point whether they mean to or not. I was not able to follow the discussion on VIN-I let my subscription lapse because I found the discussion there generally unsatisfying and shallow. The same can be said for the AVMA forums. The SGU forums and Science-Based medicine are much more satisfying, IMO. when I need veterinary information, I have found my library and/or a call to my local specialists to be more effective and efficient. Unfortunately critical thinking skills, an understanding of scientific medicine and the ability to discuss a controversial issue without feeling personally attacked are less common than they should be in the veterinary profession.
Can I ask why posters like Skeptivet and Bartimaeus choose to post without including their real name? I know Skeptivet is Brennan. I much prefer to know who I am talking to when discussing topics such as this.
If there is a way to find that on the site, please direct me to it.
Thanks
I realize Skeptivet is Brennan. I am curious why he and Bartimaeus choose to not post under their real names. If there is a way to determine names, affiliations and background from the site, please let me know. I much prefer to know whom I am addressing rather than type to an anonymous pseudo-identify.
Thanks
Dr. Pion-I choose to blog (and comment) under a pseudonym because it gives a little more freedom to post about certain cases and questions that clients ask while still protecting their identities-for example, when someone asks about a rabies scare or some type of CAM, I can write about it and still allow some privacy and protect their privacy. Another reason to use a pseudonym is to help to avoid the appearance of a cheap appeal to authority and allow people to evaluate your arguments in an unbiased manner. It is not really meant to disguise my identity for any other reason. Skeptivet may have other reasons. I post under my real name on the SGU forums and Science-Based Medicine, and I am sure that some people have figured it out, and I have told some people who might need or like to know and who I know are discreet enough not to spill the beans.
It is really a quite common practice among professionals with blogs and I could list several examples of excellent “anonymous” bloggers-look at Respectful Insolence, Jack of Kent, and The Quackometer for examples.
I know the identities of two of those bloggers, but would not post them frivolously.
Not sure I get it. How can one be certain that they are dealing with a DVM and not someone saying they are a DVM or MD or ????, who is not.
As they say, “on the internet, no one knows if you are a dog”
I am a believer that if you believe in something, you should be willing to put your name upon it.
To each their own.
Dr. Pion, you still have not really addressed the issues SkeptVet raised, which is the lack of understanding of the scientific method in a large portion of the veterinary community and the many ways in which our clinical experience can fool us if we are not careful. Stem Cells show promise for treating many different disease processes, but should be evaluated carefully and thoughtfully for each indication. Whether I am a DVM or not is really irrelevant to the discussion. A potential client has just as much right to ask about the evidence supporting a treatment as a veterinarian or MD does. There are plenty of people with lots of credentials (some in academic positions) who still promote therapies that lack evidence or even despite the evidence (Andrew Weil is a sterling example). As I said above, sometimes emphasizing credentials gives the appearance of a cheap appeal to authority.
Paul,
As far as the issue of non de plume I have little to add to what Bartimaeus said. As you indicated, it is not difficult to figure out my identitfy iof someone wants to, but it is standard practice to pick a username that reflects the focus of a particular blog. Many folks have mulltiple blogs on different topics and this helps reduce confusion. I am not “Skeptvet” on the Irish Traditional music forums I frequent, for example! It is true that some people also prefer true anonymity to avoid harrassments by zealots and outright wackos that frequent then internet, though I have not made sufficient efforts to actuallya chieve that level of anonymity.
In any case, the question itself is yet another irrelevant personalization of the discussion which doesn’t have anything to do with the substantive issues. You seem to be much more interested in the motives and personal quirks of others than the actual subject under discussion.
****further responses I have posted to the VIN discussion thread on this topic****
I think you may have hit on something improtant. As a day-to-day private general practice vet, I am confronted with strong adherence to outdated, unproven, or outright nonsense therapies which people justify through personal experience and anecdote. I think such unscientific thinking pervades our profession, and it seems that widespread use of EBM will require a change in culture and attitude. And as already discussed, Vetstem has marketed a stem-cell therapy which has potential but which I think remains inadequately supported by clinical trial evidence and yet is used by a number of vets in my area. For the reasons I’ve already given, I perceived a similar epistemological attitude in the original article and responded to that.
Dr. Pion clearly sees the use of uncontrolled observations in this context as just a preliminary step in deciding future research direction, and while I don’t entirely agree I can see why that is a different issue than the justification of clinical therapies, so to some extent we are probably talking past each other. Thanks for making such a useful observation.
****from VIN thread****
Paul,
I find it hard to think of anything to say to you that will convert this into a productive exchange. I have tried to be conciliatory and dispassionate and acknowledge the points others make and the weakness in my own argument, but you insist on ratcheting up the intensity and emotionalism of the discussion and casting it as a personal grudge against Dr. Vulliet. You seem unwilling to accept that we can have a genuine, good-faith disagreement on the merits of an idea and insist that any disagreement must be the product of irrational and misguided zeal on my part.
I can hardly imagine a more mild, circumspect, or collegial critique than “I think this is inaccurate,” yet you insist on interpreting it as an expression of collosal arrogance. And then you turn around and accuse ME of creating boogey-men to attack?!! Unbelievable. Apparently the only thing I need to do to make you “angry” is express an opinion with which you disagree, no matter how I express it.
You have repeatedly suggested that we should leave behind the issue of Dr. Vulliet’s research and discuss EBM and the attitude within the profession towards it in another thread, and I think you are right. This discussion certainly doesn’t seem to be going anywhere useful.
****from VIN thread****
Thannks for your comment. I absolutely do see how my original comments could be perceived as a personal attack on Dr. Vulliet, and I deeply regret that. Apart from the fact that it wasn’t my intent, it has proven an intransigent distraction from the real issues. I have tried to explain that I did not intend a personal attack and that I am sorry my tone gave that impression, but my disavowels of the impression I mistakenly gave at first haven’t apparently been heard or accepted. Likewise, I do feel charicatured a bit by Dr. Pion dspite my attempts to clarify what I meant, and I think that undermines the usefullness of such discussions.
I think my reply to Alex above addresses your question to me. I saw a link between the attitude towards controlled clinical researech and preliminary laboratory research expressed in the article and the common attitudes used to justify non-evidence based approaches in practice, and I responded to what I considered an unfortunate lack of respect for the value of these kinds of studies. As I said to Alex, I can see how uncontrolled observations could be useful as a preliminary part of looking into a new idea.
I still believe, however, that there are dangers to this approach. If all of the initial cases appear subjectively to improve by chance or by wishful thinking on the part of the owner, this may appear to justify time and resources spent on an idea which doesn’t in fact have merit. Likewise, if a few cases do poorly at first, it might lead to giving up on a good idea.
The history of clinical research is full of ideas that acquired passionate believers and advocates based on initial uncontrolled observations but which only larger, better trials could show to be useless or even harmful. R. Barker Bausell’s book Snake Oil Science is a cogent and thorough examination of how even the brightest and most conscientioous scientists are not immune to the factors that generate mistaken impressions, and that strict controls are necessary to guard against this. The article seemed to say that Dr. Vulliet was deliberately trying to avoid the sorts of controls a university environment would impose on his work, and I really don’t agree with the idea that has been put forward here that these are mere bureaucratic niceties that dont’ have any real value for research and we’re better off without them, at least in the early stages. I do understand the point that freedom from such constraints has advantages, expecially in the preliminary stages of investigating an idea, I just think it also has costs and risks which aren’t being acknowledged.
I don’t know much about the grant process, and of course AKC isn’t NIH so I imagine the details are quite different. NCCAM has funded over $2 billion in research on generally implausible therapies for huans, so I’m not sure that simply getting a grant suggests one has a solid idea or reserach plan. As I’ve said before, I have no opinion on whether Dr. Vulliet’s idea will actually turn out to be correct. And apart from the lab animal studies cited in the original article as a justification for looking into it, I have no idea what background evidence exists to support the idea. All I was trying, and am still trying, to say is that I think uncontrolled personal observations are unreliable and that I am concerned that whatever conclusions are drawn from this work will be based on uncontrolled observations so I’m not sure how useful or accurate they will be.
****from VIN****
Yes, the conundrum of EBVM is the paucity of high quality evidence. That requires several responses:
1. Don’t give up! We must continually aim to base our practices on the best available evidence and update them as the data justify. Accepting the reality that much of what we do is not well-founded in research evidence is unavoidable, and I’ll repeat that I’m not suggesting we quit practicing medicine until we reach EBM Nirvana. We just have to be careful not to let acceptance of reality become complacence or to ignore evidence when it does emerge if it contradicts our established practices. I’m not suggesting you or anyone here is doing that, but it is something I see in regular practice all the time. Practices which are clearly contraindicated by the evidence that is available are still widespread, which suggests we have to better balance our realism and our idealism.
2. Generate better evidence. If we acknowledge the principles of EBM and the need for them, we can hold the research we are able to do to the highest possible standard, and ultimately the data will be better.
Here is some information about The Shady Side of Embryonic Stem Cell Therapy
by Stephen Barrett, M.D.
http://74.125.113.132/search?q=cache:ks8-ZxGQZTEJ:quackwatch.com/06ResearchProjects/stemcell.html+stem+cell+quackwatch&cd=1&hl=en&ct=clnk&gl=us
Art Malernee dvm
As I owner of a beloved dog with DM…i applaud Dr. Vulliets work.
Perhaps the critics have lost site of the fact that there is no cure for DM and very few treatments either. Keep going Dr.Vulliet and ignore the noise!
So because there is no cure criticism is “noise” and should be ignored? Should we not have standards for how we focus our limited resources to maximize the chances of finding something that is actually of benefit? Why is it that calling for the profession to hold to a higher standard of science is so offensive? Is it just that we’re not quietly letting people believe what they want or hope regardless of the evidence?
Pingback: FDA Might Regulate Stem-Cell Therapies for Dogs & Cats: But Probably Not | The SkeptVet
I just found this thread, many years later. My dog, a German Shepherd, received the stem cells (donor cells) Dr. Vulliet prepared. I videotaped his neuro exam prior to the treatment. My dog experienced dramatic improvement with stem cell therapy. I documented before and after on video. The treatment bought him 1.5 years of good quality life. He was arond 10.5 years old at the time of treatment. I am a biomedical researcher myself, I was well-aware of the risks and the uncertainty of the outcome. Dr. Vulliet was very upfront about it. I am incredibly grateful to Dr. Vulliet for giving my dog this chance. We were discussing how to quantfy the progress and outcomes, it would have required building specialized tools and would have been very expensive, certainly not feasible within 100k. I am hoping some day proper funding will be made available and that animal studies will become more humane. Using dogs (sentient animals) as “animal models” is something that I cannot condone. Breaking dogs’ spines to scientifically assess the efficiency of the treatment would take a special kind of person and I am glad Dr. Vulliet did not do this.
I’m glad you had a good experience with this therapy. Unfortunately, such uncontrolled observations are unreliable and turn out to be misleading often in medicine, which is why controlled studies are needed. So far, the results of limited research into stem cell treatments has not been nearly this miraculous, in humans or in animals, and there have been serious injury to some patients, so we have a long way to go before we can safely and confidently treat our patients this way. In the meantime, here is some further discussion of why anecdotes aren’t as useful as most people think they are (and a humorous illustration of the problem as well).
Why Anecdotes Can’t Be Trusted