One thing this article touches, but does not give enough attention to (maybe because it is so counter-intuitive and hard to explain) is the false positive paradox.
To put in perspective, you should only do a test if you have good a priori chance of having a positive. Even a test with 1% false-positive rate you give you back much more false positives than real positives if the a priori chance is 0.1%.
Probabilistics and statistics are hard, very important an counter-intuitive. Med and pre-med, and even high schools really should teach much more math than they do right now.
Counterpoint here, the false positive paradox won't surprise many physicians. It stems from concepts that are well-known among my colleagues and have been tested on each of my licensure exams. We apply them at least 10s of times each day.
I will add my anecdote to support this comment. Being an EE with probability-heavy background, I tried to give a
"you guys don't use prior probabilities" sermon to a physician and his physician-in-training daughter. I was made to eat my words by their reply, which was essentially that it is something they are all taught and use all the time in assessing risks, interpreting results, etc.
The one time I had to pay $500 for some tests not covered by insurance, but egregiously wrong in the context of my ailment, the referral was from a nurse practitioner.
Two things:
1. Perhaps doctors don't uniformly apply their training in interpreting test results and analyzing risks? (It is after all, quite common in all professions to forget some parts of one's training)
2. In addition to doctors, there are also technicians and nurse practitioners, who some times may not have the rigorous training that one associates with med school.
> To put in perspective, you should only do a test if you have good a priori chance of having a positive.
There are several reasons to run a test that yields a small posteriori probability even after a positive result (i.e., a test such that most positives are false positive). The most common reason is that this test is much cheaper (or has fewer risks, etc.) than a more accurate test. The protocol, then, is to run the cheap test and then run the expensive test if you get an initial positive result. The only time you end up believing it's a true positive is when the expensive test returns positive.
Or, if the false positives are probabilistically independent, just run the same test again. If your test gives back 1% false positives, running it twice gets you down to .01%, three times gives you one in a million.
Unfortunately, this is essentially never the case. The reason is that if the outcomes were probabilistically independent, then you would just declare a standard protocol of always running the test N times, and call that the test. It's safe to assume that such trivial optimizations have already been done for medical tests.
The common syphilis test looks for antibodies that are produced in response to the bacteria. However, the body may produce those antibodies (on a continuing basis) for other reasons. If you find these antibodies the first time in a given person, you'll keep finding them when you re-test.
I was best able to understand it by running the numbers. Take a condition with a 0.1% occurrence rate in the population: then 1,000 out of every million people have this condition.
Then, say you develop a test for the condition that's "99% accurate". Sounds great! Out of the thousand people with the condition, 99%, or 990, will get a positive test result! Unfortunately, of the people without the condition, 1%, or 9,990, will get a positive test result.
So if you're one of the 10,980 people who get a positive test result from this "99% accurate" test, it turns out you still only have a 9% chance of actually having the condition.
We tend to remember the one wasteful case and forget the next 25 which were all reasonable - the old man with the worsening shortness of breath, the middle aged guy with gross hematuria, the drunk who crashed into a parked car riding his moped, the kid with the early onset puberty for a bone age study.
Dr. Gawande is a smart person, a contemporary of Ezra Klein and Ezekiel Emanuel, and a lifelong politically involved leftist. His essays on waste in medicine over the past 10 years have shaped progressive thinking on the subject, with citations from everyone from President Obama (during the sale of Obamacare) and many others. His political alignment is why Dr. Gawande cannot speak the obvious heresy that the United States' insane malpractice climate has anything to do with overtesting (although he has considered that in a prior book). He has made his career promoting the idea of waste and accordingly sees it behind every bush. I don't mention this to say that there is no merit to some of his observations; just to provide context of his background, alignment, and world view.
To put in perspective, you should only do a test if you have good a priori chance of having a positive. Even a test with 1% false-positive rate you give you back much more false positives than real positives if the a priori chance is 0.1%.
This is counter intuitive, I'm not good at explaining stuff, so I'll give Doctorow the word, through this excerpt in Wikipedia: http://en.wikipedia.org/wiki/False_positive_paradox
Probabilistics and statistics are hard, very important an counter-intuitive. Med and pre-med, and even high schools really should teach much more math than they do right now.