Thursday, November 11, 2010

The problem with evidence based medicine

I don't have a lot of patience for woo at the best of times, and I have almost none for woo when it comes to treating people with real illnesses. The only patience I have for it is that placebo can be a Good Thing, and if woo doesn't cost too much, does no harm and induces a nice, effective placebo, I'm down with that. However, it often costs lots, is potentially harmful, and placebo may not be enough to cure any given ill. It's at it's absolute worst when it stops people also seeking evidence based medicine.

However, one of the reasons woo gets a look-in is that people have a gut feeling that evidence based medicine isn't always right. Most people know that we have very individual responses to various medicines, they know that very often their own symptoms don't match the text book, or are not even close. So the woo that offers individually tailored "treatments" look attractive.

There's been more than one occasion lately where I've found myself arguing with prevailing medical wisdom, based entirely on anecdotal evidence. Anecdata does not a theory make. However, basic trial and error about what triggers which, and then which relieves what is also empirical evidence. It can't be entirely ignored because it isn't conducted by a person in a white coat. It certainly says nothing about the way the same things might operate in the next person, but if it's in direct conflict with what conventional, evidenced based medicine is telling us, something's got to give.

For example, both gout and diabetes have come up a few times in my meanderings recently. I have neither, so I'm merely collecting anecdata here, and comparing it with the received wisdom of evidence based medicine. In both examples, people who have monitored their own health for a length of time have found wildly different things exacerbate or improve their condition from each other, and from what the doctors have told them.

There is nothing surprising in this. Evidence based medicine is based on statistical inferences from samples. The results are averaged across participants. Evidence based medicine is an excellent approximation of what happens to the Average Person, and Average Person doesn't exist. The trouble is, nowhere near enough doctors and other health professionals really understand this. They have a list of foods that cause gout in Average Person, but the people I know that have it have entirely different triggers from each other and from that list. The list isn't wrong, it's just that it's the most common triggers, averaged across all the people the studies have been conducted on. That's fine, as long as the doctor understands that, and presents it as such - as a starting point for your own trial and error to find out what your problem foods are. Same story with diabetes and blood sugar levels.

I'd like to see medicine start to engage more fully in the study of difference. To follow up the studies describing the average symptoms/responses/profiles with careful analysis of the differences between subgroups and even individuals. I'd also like to see it engage patients more fully in the understanding of their own bodies. It's not that no-one does it now (or I wouldn't have my anecdata), it's just that it's not mainstream for patients, and it's barely even tolerated by the medical profession, nevermind encouraged.  

I will back existing evidence based medicine over woo any day, but it's important to remember how it's done, what it's really telling us, and that at the end of the day, as Brian said, "You're all individuals!".

(As a footnote, this is a description of evidence based medicine as it should be. I'm not even beginning to address deliberate misrepresentations of data for the purposes of profit and/or power - that's a rant in and of itself.)


  1. I tried to respond to the blog, but failed, so here is my next attempt.

    There is suppose to be more to EBM than just statistics and averaging.

    "Evidence-based clinical practice is an approach to decision-making in which the clinician uses the best evidence available, in consultation with the patient, to decide upon the option which suits that patient best [2].

    Evidence-based medicine is the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients. The practice of evidence-based medicine means integrating individual clinical expertise with the best available external clinical evidence from systematic research [3]."

    I like the point made in this article:

    "Good doctors use both individual clinical expertise and the best available external evidence, and neither alone is enough"

    The important word is "Good" (We seem to be most lacking in this variety of doctor around here)

    BTW, This is probably the best evidence for me not writing a blog...boring!


  2. That came out much drier than intended, I drafted it at work when I didn't have very much time

  3. It's not dry, Toni, it's a valid point. Some of the gap between the statistical approach and the individual should definitely be filled by good doctoring. It's a shame such doctoring is so bloody hard to find.

    I'd still like to see some systematic study of those differences - both for basic science reasons, and also to highlight to your more run of the mill doctor just how varied humans are. The devil's in the detail, and I can't help thinking there's some interesting science lurking in there too.

    I think this (constructive) criticism can be leveled at other largely stats based areas - psychology as a perfect example. Some bits of psych embrace the whole range of experience, and others are very normative, and are poorer for it. I have a strong suspicion I'm about to find that education is much the same.