While I won't go into the article in any detail, suffice it to say that the overall accuracy of the test for diagnosing depression was between 91% to 94%, depending on the group studied. Based on this, the authors report that the test "has excellent performance in confirming a diagnosis of MDD (major depressive disorder)."
The article is a classic example of the pitfalls of focusing on glitzy-sounding statistics while downplaying the actual clinical usefulness, which in this case is close to nil, as both of the Journal's commentators agreed.
I recently discussed the same problem in an article I wrote for CCPR about the NEBA EEG test for ADHD. Like the MDDScore, the NEBA test promises to aid in the diagnosis of a psychiatric illness. The NEBA's accuracy is high, with a positive predictive value for ADHD of 96% for kids, and 81% for adolescents. But no matter how accurate it is, the crucial question is whether it adds value above and beyond the standard psychiatric interview. Neither the MDDScore nor the NEBA do.
In my article, I used a hypothetical analogy of a new test to diagnose apples:
"Let’s imagine that there’s a new apple-recognizing device on the market called the “Apple Rec,” which uses various technologies to measure the wavelength of light reflected by an object, its mathematical curvature, etc. The manufacturer provides impressive data showing that the Apple Rec has 100% sensitivity and 100% specificity for diagnosing (recognizing) an object as being an apple. Given these dazzling statistics, would you buy the Apple Rec? No, because even though it’s exquisitely accurate, it provides you with no useful diagnostic information beyond what you can obtain by looking at the apple yourself. However, if the Apple Rec provided you with added value, you might consider it a good investment. For example, if, in addition to correctly recognizing it as an apple, it also calculated its sweetness and crispness, the Apple Rec suddenly becomes a useful tool, because these are qualities that you would otherwise struggle to ascertain."
The apple principal applies to diagnostic tests in
psychiatry. Before you refer your patients to an expensive test that diagnoses
ADHD, depression, or anything else, you need to make sure that it does something that you can’t easily do
yourself.
Subscribe to our RSS Feed!
8 comments:
Daniel:
Your “yes / no apple identifier” analogy is misplaced. First, NEBA isn't a binary ADHD / not ADHD test as you've characterized in your blog and in your article in the Carlat Report. NEBA aids in ADHD assessment by either confirming ADHD or by providing support for testing for other conditions. Second, ADHD is more often than not comorbid with other disorders that may better account for observed symptoms. It is less likely for a patient to present as pure ADHD. If we are using your apple analogy, then e.g. patients often present with comorbid depression (let's say "pear"), and anxiety ("plum"). So you as a clinician have a patient who might be 1) an apple; 2) a "PearApple"; 3)a "PlumApple"; or maybe just 4) a “pear" or just a “plum". Now, squaring this circle, NEBA is more akin to a scanner that helps clinicians determine appleness, pearness or plumness. This is much more like the real clinical world most practitioners face. NEBA would help confirm “apple” or encourage further testing for "pearnesss" or "plumness". You can find all this out in publically available sources (such as FDA) and in our paper - available on our website under /research. Finally, you suggest that a useful test would sample a biological trait, e.g. sweetness. NEBA does exactly this – we are sampling a biological trait – brainwaves – akin to sweetness, to help determine if we have an apple or whether we should perform further testing to see if we have some hybrid variety. In sum, we are not testing to see if we have an apple or “not an apple”.
There's definitely something intriguing about the NEBA test. There seems to be a close relationship between the Theta/beta ratio and the presence of some sort of attentional issue. But the crucial unanswered question is whether the test provides information that clinicians can't gather easily via a standard interview and talking to informants. If not, then it becomes extremely hard to justify spending the money for the test.
My admittedly inexact apple analogy was used to highlight the fact that if a sophisticated test tells you something that you already know, it's not worth buying.
Thank you for your response. We validated NEBA in with a mixed population of kids presenting with attention and behavior problems. We posit that when a child concomitantly has ADHD symptoms and symptoms of some other disorder, then NEBA has clinical utility - i.e. offers new information - in helping a clinician determine if another disorder better accounts for the observed symptoms. And subsequently, what to treat as primary.
It could help clarify the etiology perhaps, regardless, and improve the DSM so psychiatrists aren't constantly saying ignore the code on the insurance.
Its good to know little has changed since this blog suspended and now has returned. People still desperate to turn junk science and the unscientific into something that resembles science. Why not a test for the complaining parent or guardian or maybe the teacher for the poor kid with "ADD". It is so obvious why biologic tests in psychiatry are not only a pipe dream but not even wrong conceptually. I am glad the blog is back as its great for a laugh but not in a ha ha way.
http://www.thelancet.com/pdfs/journals/lancet/PIIS0140-6736%2815%2960696-1.pdf
But how can major depression possibly be accurately diagnosed when it doesn't exist... i.e. when there is no validity and reliability behind this fictitious disorder?
Yes, of course severe distress and all the symptoms associated with the DSM description of depression exist in innumerable variations and are often extremely painful. But that in no way means that there is a valid reliable illness connecting these unique human experiences which can be identified through some physical test. What a bunch of BS...
What if you placed an orange behind a picture of an apple?
JimG, is that a Zen koan? Reminds me of "What is your original face before you were born?".
So far, I have no epiphanies regarding how to use NEBA!
Post a Comment