Hill’s Criteria of Causation–What Separates Science from Faith

Mark Crislip at Science-Based Medicine recently discussed Hill’s Criteria of Causation, but after looking at Dr. Hill’s original paper I felt obliged to examine the subject here as well because it is central to science-based medicine, and key in differentiating real medicine from quackery.

As I often discuss, a crucial issue in medicine is the question of epistemology; how we know what we know. Our therapies ought to be based on a real understanding of health and illness, and sound evaluation of the safety and efficacy of the interventions we undertake. The key difference between scientific and alternative medicine is in the criteria  for such evaluation. CAM proponents are often satisfied with theories of disease than are either made up out of whole cloth by individuals (e.g. homeopathy, chiropractic) or that are an agglomeration of folk beliefs (e.g. acupuncture, reiki and other energy therapies, TCM). Whether these theories are consistent with our general understanding of the universe, or with each other, is not a concern.

Even more damning, CAM practitioners are frequently satisfied that their personal impressions and experiences are adequate to validate the truth of the causal associations they see underlying disease or in the response to their treatments. Despite the manifold errors the human mind is known to exhibit in making causal associations, these people don’t see any need for a type of validation that compensates for these errors. They may lay claim to any scientific evidence that can be found for their approaches, for the aura of legitimacy it confers, but they will never reject the conclusions of their experience or intuition in light of any other kind of evidence.

In 1965, Dr. Sir Austin Bradford Hill gave a talk to the Royal society of medicine entitled The Environment and Disease: Association or Causation? In this address, he laid out nine criteria for concluding that an observed association was in fact representative of a causal relationship. His lucid and cogent presentation provides us with an excellent framework for making decisions about the causes of disease or the effects of treatments, all the more because he was so careful to emphasize that his criteria were guidelines which should support, not replace critical thought and judgment.

Here are Dr. Hill’s criteria, with a brief explication of each:

1. Strength– The strength of an association can be supportive of a true underlying causal relationship. If a proposed cause for a disease is associated with the disease itself only sporadically or unpredictably, this is weaker evidence for its causal role than if it is reliably present in conjunction with the disease. As always, Dr. Hill cautioned that “we must not be too ready to dismiss a cause and effect hypothesis merely on the grounds that the observed association appears to be slight,” but the strength of an association is a relevant factor in evaluating its significance.

2. Consistency– An association between cause and effect should be robust enough to be demonstrable in multiple studies by different investigators. This is key to understanding the fallacy of presenting a single study as definitive evidence for or against a disease etiology or treatment effect. The preponderance of the evidence as it accumulates is more meaningful than the results of particular studies, though the quality as well as quantity of the evidence must also be considered.

3. Specificity– If a putative cause is associated with a very specific set of symptoms, or a treatment with very specific effects, this supports a causal relationship. If the cause is present with a wide variety of different clinical presentations or the results following a treatment are highly variable, this argues against a causal relationship. As always, Dr. Hill correctly cautions against excessive rigidity in the application of this criterion, pointing out that many diseases are multifactorial and may not exhibit great specificity of association with a single causal factor. As he puts it, “if specificity exists we may be able to draw conclusions without hesitation; if it is not apparent, we are not thereby necessarily left sitting irresolutely on the fence.”

4. Temporality– Causes by definition precede their effects, so if a potential causal agent is observed after the condition it is speculated to be causing, this argues strongly against a true etiologic relationship. The obverse of this is, of course, the post hoc ergo propter hoc fallacy, perhaps the most intransigent and troublesome reasoning error in medicine. The observation that one thing precedes another is not in any way evidence for a causal connection because of the myriad of alternative explanations that often turn out to dictate the order of precedence. Unfortunately, precedence is very compelling as proof of causality despite being unreliable.

5. Biological Gradient– Also known as a dose-response curve, this relationship is common in biological systems and is strongly supportive of a causal/effect relationship. If the amount of a drug given or the intensity of an exposure to a potential cause of disease is correlated with the likelihood or severity of the resultant effect or the disease, this supports a conclusion of causation. If, however, a little bit is as good (or bad) as a lot, the hypothesis of causation should be questioned.

6. Plausibility– Dr. Hill was less convinced of the reliability of this criterion because, as he correctly observed, “What is biologically plausible depends upon the biological knowledge of the day.” However, as he explains in discussing his subsequent criterion, a proposed relationship which contradicts well-established understandings in biology or other areas of knowledge should be viewed as less likely to be correct, and less worthy of implementation or investigation, than a relationship consistent with such understandings.

7. Coherence– In contrast to his caution regarding the criterion of plausibility, Dr. Hill states, “On the other hand, the cause-and-effect interpretation of our data should not seriously conflict with the generally known facts of the natural history and biology of the disease.” A proposed mechanism, such as that underlying homeopathy, which requires overturning well-established principles in biology, chemistry, and physics must face a far greater burden of proof to be accepted as valid than a hypothesis consistent with established knowledge.

8. Experiment– Clearly, the results of controlled laboratory, animal model, or clinical trial studies are critical in drawing firm conclusions about the causation of disease or the effects of medical interventions. Contrary to the stereotype sometimes promulgated by CAM advocates, scientific medicine does not rely solely on this criterion, but it can be a very powerful tool for confirming or invalidating proposed relationships.

9. Analogy– This is perhaps the weakest of Hill’s criteria, though it is popular because of the simplicity and apparent clarity of arguments by analogy. While toxins may cause illness, for example, not all illness should be attributed to toxins. Likewise, while infection may cause a fever, not all fevers are due to infection. Analogies are most useful for suggesting possible relationships, which should then be confirmed or disproven by application of more rigorous criteria.

After discussing his criteria, Dr. Hill goes on to eloquently addresses two critical issues in the epistemology of medicine. First, he is very clear that conclusions about cause and effect relationships must be based on assessment of multiple criteria and application of sound reasoning, not on slavish adherence to one criterion or to any algorithmic model, including his:

“What I do not believe – and this has been suggested – that we can usefully lay down some hard-and-fast rules of evidence that must be obeyed before we can accept cause and effect. None of my nine viewpoints can bring indisputable evidence for or against the cause-and-effect hypothesis and none can be required as a sine qua non. What they can do, with greater or less strength, is to help us to make up our minds on the fundamental question – is there any other way of explaining the set of facts before us, is there any other answer equally, or more, likely than cause and effect?”

This shows a sophistication of reasoning inconsistent with the stereotype of science-based medicine advocates as obsessed with randomized clinical trials and blind to all other evidence for or against proposed causal relationships.

Dr. Hill also presciently addresses the proper role of statistical analysis in medical decision making. In dealing with questions of cause and effect, he argues that “no formal test of significance can answer those questions. Such tests can, and should, remind us of the effects that the play of chance can create, and they will instruct us in the likely magnitude of those effects. Beyond that they contribute nothing to the “proof” of our hypothesis.” He goes on to discuss how statistical analysis entered medical research in the 1920s and 1930s as a needed anodyne to the practice of basing firm conclusions and therapeutic practices on small, not representative case examples. However, he speaks strongly of the excessive role such methods were then coming to play in drawing conclusions from medical research, and the problems he describes have only worsened in the intervening time:

“I am told, some editors of journals will return an article because tests of significance have not been applied. Yet there are innumerable situations in which they are totally unnecessary – because the difference is grotesquely obvious, because it is negligible, or because, whether it be formally significant or not, it is too small to be of any practical importance. What is worse the glitter of the t table diverts attention from the inadequacies of the fare….Of course I exaggerate. Yet too often I suspect we waste a deal of time, we grasp the shadow and lose the substance, we weaken our capacity to interpret the data and to take reasonable decisions whatever the value of P. And far too often we deduce ‘no difference’ from ‘no significant difference.’ Like fire, the chi-squared test is an excellent servant and a bad master.”

Lastly, Dr. Hill answers definitively and in no uncertain terms those critics who falsely accuse practitioners of science-based medicine of passivity or helplessness in the absence of definitive clinical trial evidence:

“Finally, in passing from association to causation I believe in ‘real life’….In asking for very strong evidence I would, however, repeat emphatically that this does not imply crossing every ‘t’, and swords with every critic, before we act.

All scientific work is incomplete – whether it be observational or experimental. All scientific work is liable to be upset or modified by advancing knowledge. That does not confer upon us a freedom to ignore the knowledge we already have, or to postpone the action that it appears to demand at a given time.

Who knows, asked Robert Browning, but the world may end tonight? True, but on available evidence most of us make ready to commute on 8:30 the next day.”

Share on Facebook

Tweet

Follow us