This is top-notch stuff! And damn, I think you are right. I’ve suspected so for a while but didn’t quite know how to articulate it. I’ve been thinking about this problem in recent years from the angle of “non-specific psychopathology” as discussed in a 2023 paper by Peter Zachar: https://onlinelibrary.wiley.com/doi/10.1002/wps.21043
Your post adds new consideration to this. I’ll try to articulate my own thoughts on how we can navigate this mess. I love your idea of assigning confidence/probability ratings to diagnoses. This is something I’ve doing informally when I’m giving an official diagnosis for various reason but I am highly uncertain or have low confidence in it’s applicability.
Thank you, I appreciate that. The non-specific psychopathology concept (neuroticism?) is something I have thought about for a long time. Rereading the Ioannidis paper supplied a framework that I think is valid here.
As I read this, I was thinking of restacking (? Is that the word) a paragraph, but then I read more and I wanted to restack that, and that feeling kept going throughout this piece. Definitely one of the best I have read here.
It has been something I have been sensing more and more as I have more years under my belt, but you articulated it much better than I ever could have - my sense that, even as a psychiatrist, most psychiatric diagnoses are false.
You hinted at many of the biases leading to that result, from individual (clinician inexperience, or “pet diagnoses”), to systemic (needing a diagnosis for billing reasons). Ultimately the whole system is set up to diagnose, which compounds the problem.
This is an outstanding piece that's rigorous, clear-eyed, and soberingly honest about the limits of our tools. Thanks for laying it all out with such precision. I especially appreciated the framing of diagnostic interviews as underpowered tests run in biased conditions, often with unacknowledged thresholds and poor predictive value.
Reading this, I found myself thinking less about the technical implications (which are important, of course, but already familiar to me) and more about the *existential* ones. Particularly, the way the psy-disciplines (e.g., psychiatry, psychology, neuropsychology, psychotherapy) increasing seem to be coping with the terror of diagnostic uncertainty by doubling down on diagnostic essentialism.
That is, as our understanding of what a diagnosis *means* complexifies, we more fervently assert that having a diagnostic label is THE key to understanding a person. We've trained our clients to twist their complex selves and multifaceted pain into the constricted question of “Why do I have symptoms X and Y?”, which we then answer even more narrowly with “Because you have Symptom X and Y Syndrome.” The tautology is rarely acknowledged. Nor is the fact that the entire process reflects not just symptoms, but the patient’s worldview, the clinician’s worldview, and the social meanings attached to different diagnoses.
What most worries me is that the diagnostic “if” (if the person has a disorder) becomes a stand-in for more perceptive and generative questions we could be asking. When “if” becomes the central question, it subtly conveys that diagnostic legitimacy is the only path to being taken seriously --especially if the answer is “yes” to a popular diagnosis (internalizing, neurodevelopmental, or trauma-related), and “no” to an unpopular one (personality disorder, psychosis, low IQ, externalizing). This maps disturbingly well onto cultural scripts about whose suffering is understandable, whose is dangerous, and whose is inconvenient.
In my field of psychological and neuropsychological assessment, this narrow frame is often even more entrenched. We specialize in lengthy and complex evaluations, so we have the time, the tools, the training. And yet, we chase the “if” with near-religious intensity, rationalizing our preferred answer even when the data wavers. We sidestep more fruitful questions, such as:
-- Why is this (usually longstanding) problem surfacing so acutely now, and in this form?
-- When is the problem better or worse, and what helps?
-- Where in this person’s life are the conditions more favorable for growth or ease?
-- Who is this person, and how does that self-understanding shift the landscape of intervention?
When we chase diagnosis as destiny, we forget to ask how the person can engage with long-standing human needs: for meaning, self-compassion, challenge, responsibility, rest, and contribution. We don't ask, how is the balance for this person? Are they experiencing too much adversity (perhaps too much challenge, too little meaning, and too little rest), or are they instead adrift, avoiding responsibility and meaning (perhaps mistaking accountability for blame and desirable difficulty for suffering)? What small actions could shift them toward a life where they feel more alive, authentic, accomplished, or adored?
None of this is easy. The pressures (though they shift with setting) are real: time constraints, reimbursement demands, sociocultural dynamics around self-diagnosis, and the sheer inertia of systems. But I do wonder, have we lost the art (and nerve) of case conceptualization because we feel it doesn’t matter anymore? Do we feel that the treatments are the same regardless (statistically, the person is probably already on antidepressants and stimulant medication and in therapy anyway), and we're worried that without something more to say, some new diagnosis to add to the client's chart, we risk becoming irrelevant? Do we feel like our own value as clinicians is unearthing the most 'invisible' diagnoses, since we often feel powerless to shift the structures of a person's life? Is this because we can barely touch the cultural malaise making so many of us feel less attentive, less awake, less able, and less attached to meaning and community? Or are we afraid to recommend anything that involves real effort, loss, responsibility, or hope?
It’s a strange time to be in this field. Many of us sense that something isn’t working. But instead of returning to the harder questions (why, what now, how else), I fear we're increasingly clinging to the shallow comfort of “if.” For a while, this may keep us afloat, supporting our ability to feel useful in the face of complexity we can’t resolve. But what happens when AI can answer the 'if' questions as "accurately" as we can, with much less time and expense? Won't it be essential for us to have more to say?
Thank you again for this thoughtful piece. I will definitely be sharing it with my colleagues.
Thank you! I really feel that psychiatry/clinical psychology has found itself at something of a dead end with the current nosology, but we are in too deep to change it. Imagine the public reaction if we came up with some entirely new system and people's favored diagnoses went away.
This is a good article. Far better than most on this topic. A few critiques.
Critique 1:
Firstly, you don’t distinguish sharply enough between two claims:
(1) Most patients who receive a psychiatric diagnosis are diagnosed with at least one condition they don’t have
(2) Most of the time when someone is diagnosed with a specific psychiatric condition, they do not have this condition
Claim (1) is what your models actually show. They do not show claim (2). In particular, the ‘hitting a bullseye…’ section would not apply if you were trying to show (2). And the ‘typical evaluations are underpowered’ section would have to be modified as well.
Several commentors have clearly mixed up claims (1) and (2) and you haven't corrected them. Suppose I'm reading someone's chart and I see 'schizophrenia' . It's one thing to say 'this person probably has at least one incorrect diagnosis'. It's quite another to say 'this person probably does not have schizophrenia'. (1) does not imply (2).
Critique 2:
By focusing on (1), what you mean by an ‘incorrect diagnosis’ is that at least one of the conditions the patient has been diagnosed with is a condition that they don’t have. That’s not necessary completely unreasonable. But I think you should clarify why you’re focusing on this.
Personally, I wonder if it would make more sense to say that a patient has been incorrectly diagnosed if the most severe condition they have been diagnosed with is one they don’t have. Arguably, it’s this issue that would raise concerns about undermining trust in the psychiatry profession.
Critique 3 (a minor point):
Technically, the multiple comparisons issue that you point to can also increase the probability of a *correct diagnosis.*. You don’t make this explicit but it looks like what you’re modelling as a correct diagnosis is:
the patient receives at least one psychiatric diagnosis, and all of the diagnoses they receive are for conditions they have.
Now consider a patient who actually has conditions A, B, and C. And suppose multiple testing means that the patient gets tested for B and C instead of just A. This will increase the probability of a correct diagnosis. I don’t think this would seriously undermine your argument for (1) but it should be factored into your model.
But finally, note: you could derive (2) from a much simpler model, at least for certain conditions. Suppose almost everyone that shows up looking to get diagnosed with ADHD gets their diagnosis. And suppose almost no one who doesn't look for the diagnosis gets it. In this case, the probability an ADHD diagnosis is true will be roughly equal to the base rate of ADHD amongst people who show up looking for that diagnosis
Thank you for the thoughtful critiques. I truly appreciate it. I think (1) is certainly true and (2) is often true but depends on the condition. I think I state (3) but do not emphasize it.
As I think more, the biggest issue seems to be my point re small effect sizes, both in terms of distinguishing normal from abnormal, and distinguishing one similar condition from others (depression, anxiety, ADHD, PTSD, social anxiety, etc).
So if somebody presents with what could be mild depression, the standard approach is to diagnose them with MDD and add a mild specifier. But the fact of mild symptoms (small difference from normal population) also implies greater uncertainty about the diagnosis and raises the rate of false positives when the test is underpowered. This can lead to a single erroneous diagnosis or multiple erroneous diagnoses (maybe they get a depression diagnosis and an anxiety diagnosis), made at a single visit or multiple visits.
Re your point (2), I do think that many chart diagnoses are effectively worthless. I have long instructed trainees that seeing a diagnosis of bipolar disorder in somebody's chart should have no effect on their suspicion that the person has bipolar disorder. The same has become true of PTSD, autism, ADHD etc in more recent years. Others are more reliable - OCD, MDD with psychosis, probably schizophrenia, dementia, delirium
Bigger picture, I would describe the situation I see as something like this. Take 100 people who each have at least one psychiatric diagnosis. Of these 100 people, 70 of them have at least one correct diagnosis. But the total number of diagnoses in the group might be 250. Some people have one diagnosis (correct or not), some people have 2 or 3 or 4 (less likely to be all correct). So most of the diagnoses are false, even if a majority of the individuals do have one correct diagnosis.
You seem more mathy than me, so if you think of a good way to model this, please let me know.
Great job explaining sensitivity/specificity and PPV/NPV, especially considering you did a nice job explaining prevalence for the latter. Regardless of specialty, many clinicians will describe sensitivity/specificity fine but then go into PPV/NPV without contextualizing the measures within prevalence beyond a passing blow. I always try to give the clinician-authors the benefit of the doubt (I’m an ID epidemiology trainee) with epi methods, but even JAMA was publishing stuff up til the relatively recent JAMA statistics and methods editorial shift that any well-trained, competent epidemiologist could smell a mile away. The ADHD Dx and Tx literature gets this stuff right maybe 50% of the time. (With the disclaimer that I'm not a psychiatric epidemiologist; this stuff is more of a hobby research interest of mine 🥲)
As a layman who listens to many discussions on ideation, including that of psychiatric diagnoses, it strikes me that much of the time what is described as a disorder is an adaptation of thought to circumstance. In physical terms, a man without fingers (or with two thumbs, for that matter) must adapt to the circumstance he finds himself in, as best he can (and often poorly).
Psychiatric explanations seem to be made up out of thin air for just about everything over the past century. Prior to that, how was behavior explained? The actions of the planets (melancholy, due to the effect of Saturn, for example, resulting in the common adjective, saturnine) or witchcraft or the tenets of a religious belief, even numerology. Psychiatry is, of course, godless, but only to the extent that the self has taken the place of gods (or God) in the minds of many.
You might enjoy Dr. Randy Nesse's work, Good Reasons for Bad Feelings. I think you are right that some psychiatric disorders are best seen as adaptations, or failure to adapt. You could imagine a parallel between what we call chronic depression due to life circumstances and physical over-use injuries that many people have, for example.
Completely agree! Thankyou for your comment, it balances the weight of the 'intellectual' to that of being human. I always wonder where the spiritual comes in within psychiatry. And where this is on all of these 'scales' of madness.
Every so often, that story cycles around about how doctors were unable to apply Bayes' theorem to reason about false positive rates.
That's not really a problem of statistical literacy, it's just basic probability. I'm doubtful that adding adding more *statistics* to the medical curriculum would be very useful at this point in history.
But I think it probably would be good if MDs all knew basic probability in their bones the way a trader or poker player does. Not sure how to make this happen. Having trainees gamble on patient outcomes in the hospital would be very effective but unseemly.
I've been thinking about this. It would be easy enough to put a statistics calculator in an EMR so you could plug in the numbers. I think also changing how we write notes. If you imagine the one-liner/chief complaint as an introduction to the note, part of the expectation could be including at least your initial differential diagnosis, if not numeric probabilities. Some specialties, like Emergency Department, know these numbers pretty well already: 60 yo with this medical probability presenting with chest pain has this likelihood of STEMI, etc.
Having taught FM residents EBM decision making, the issue is actually the attendings. If I am the only attending showing them how to use likelihood ratio to train intuition, and they only work with me once a month they won’t learn it. I am not a big fan of just generating numbers from calculators since people tend to get false certainty from numbers. I used more of green, yellow, red tier system with green being diagnosis certain and treatment clear, yellow diagnosis is somewhat likely but other factors in play so gather information frequently while being cautious with treatment, red diagnosis not clear and need more information before treating or get a consult. Something with a good likelyhood ratio could take from red to green. Intermediate likely hood ratios would need multiple pieces of data pointing the same way. My big problem with the average therapist is lack of critical thinking about the certainty of diagnosis and then with shear amount of time they spend with the patient pigeon hole them into what is going on.
Good point about numbers and false certainty. That seems to be a general pitfall with Bayesian thinking. There also seems to be an issue with EMRs, the "publication bias" I alluded to, where a diagnosis that was intended to be provisional soon becomes "official" when the next provider looks at the problem list.
This all sounds nice, but I find it doubtful your suggestions will find its way in most (non-academic) inpatient settings. There is simply no economic incentive to implement what you said.
Sir, do you remember that 90s movie, K-Pax, with Kevin Spacey? In the opening scenes, for an entire 15 minutes, we watch dozens of psychiatrists, neurologists, and so forth conducting all these tests, to gaslight the public into believing this is how inpatient settings work: EVERYONE gets tens of thousands of dollars worth of fMRIs, EEGs, EKGs, neuropsych exams, 500+ item MMPIs, SCIDs, a 20-hour+ multi-week interview, a Rorschach and so forth. And from there, hollywood tries to tell us "This is how comprehensive the average inpatient setting really is". It's all propaganda.
When I worked in them, not once did anyone administer any scale. Where are all the clinical psychologists? The psychometrists? We didnt have those. It was a handful of NPs, PA-Cs, remote-only MDs (who are not trained in psychometrics) and LPCs and LCSWs (who were not trained in signal detection theory, but they sure were trained in critical theory). So i kept asking myself "But why did i get all this training in the MMPI? Why did i get this training in scale development from start to finish, and how to CFA them, and how to discriminate validity scales on different categories, if no one uses it in real life? Aren't we supposed to be administering, say, the MCMI if were to throw around big words like BPD? Not even the SCL-90??? And why aren't they all walking around with clipboards and white coats? You mean to tell me they don't even PRETEND to be clinicians, either?
Instead, dozens of "clients" per week (inevitably, future anti-psychiatry persons) were labeled with vague cluster B's following single 25 minute teams meetings conducted by the "Chief Clinician": an LPC. Not an MD, not a DO, not a Phd: a lowly, MA-level, only-recently-licensed LPC. Because this is the minimum standard in that state. And again, the MDs refuse to show up in person. They do all their diagnosing behind a computer screen from home. Neither did our LPC actually speak to clients much. Rather, her evidence base was reading prior charts, some years old, and searching for "key words": anything pathologizable. From there she role-played out confrontational dynamics to "test the hypothesis". Ah, yes, Bayes theorem! But this merely caused (relatively normal?) people to become upset, who refused to "play along", and as a consequence, were nevertheless labeled with PD for the crime of negatively reacting toward being treated as a-priori personality disordered. So because they did not "act disordered", she then used this, itself, as a diagnostic criterion. "They're faking-good!", she asserted, "These are not good people, Scott. If they weren't here, they'd be in jail instead".
Half were merely homeless. An extremely consistent pattern emerged from the actual, first-person, "raw" data: In the hours leading upto scheduled discharge, our clients would "develop" delusions/conspiracies/pseudo-psychosis/somatic problems, "I'm in PAIN!" but they were very bad at actually being believable by those of us who DO have histories of involuntary institutionalization. Still, our resident PA-C, "by regulation", would then have to initiate a new hold, where they'd be escalated to another hospital to repeat the cycle. Sometimes, however, if a client learned their stay were to be extended by a week, all the sudden they behaved as if they were in a luxury hotel: they shifted to being enthusiastic, highly assertive, and then would make demands that we order them pizza, upgrade the coffee machines and use more caffeine, and facilitate DoorDash runs to the 7/11 out of pocket.
All the while, the LPC would claim during our "clinical development" meetings that "NPD is an extremely common diagnosis in our particular area. It's interesting but a little over half of my direct clients will receive an NPD diagnosis. I cant believe how many of my colleagues miss these. I host talks all the time at our satellite sites". Without any self-awareness, she was veridical that the base rate for NPD is ~60%.
(Isnt NPD like <1% in gen pop and still <5% inpatient? I bet you its much higher in med schools tho!)
I went to HR about the whole fiasco, to report there's medical fraud, in the form of non-existent and false Dx's going on, that's also being billed to tax payers (its a 501c contracted by the state, and some of these are court ordered, so, like, there's real consequences to this. Or are there??). They told me my actions of reporting these constituted "A threat to our operations and our professional community" and that I must resign.
It was February 2020, and they were about to hit the jackpot from the whole PPI/CARES thing.
By claiming that there are false diagnoses you suggest the possibility of true diagnoses in psychiatry. A look at the current state of psychopathology and its taxonomy raises lots of doubts.
So, I imagine that most clinicians couldn't tell you the Bayesian likelihoods of MDD (what I'm familiar with, I'm certain other diseases are also predictive for other psych disorders) on cancer versus "Metabolic Syndrome" (heart disease). If a doctor can't even estimate "what's the probability" based on the known diseases, of course you're a lot of false diagnoses.
(By the same token: do you diagnose someone with MDD if they have a "contributory illness"? If they have IBS, which is significantly more of a cause than a contributing factor? If you don't diagnose, are you still allowed to treat?)
Two comments: 1. Curious in these clinical gold standard interviews, when there is lower reliability, how far off the mark so to speak the diagnosis is. It’s one thing if someone who has schizophrenia is being diagnosed as just ptsd or GAD, but it’s another if they are “off” by diagnosing schizoaffective, schizophreniform, schizotypal, substance induced psychosis etc. If people are diagnosing their diagnostic neighbors, then I don’t think it is necessarily as bad of a problem. I for example have almost no interest in appropriately diagnosing MDD versus PDD versus GAD, versus primary panic instead of a panic subtype - I think of it as general internalizing and neuroticism, and don’t want to falsely reify these diagnoses to patients. Typically the treatment each of these follows in the same way.
2. I worry about prototype matching as an explicit approach (since we do this anyways as clinicians) - this would work if our diagnoses were not so heterogeneous and frequently have poor construct validity, and as you mention largely the absence of diagnostic hierarchy. I’ve seen, for example, residents say a patient doesn’t have schizophrenia because the patient wasn’t hallucinating or disorganized; of course you only need 2 out of 5 core symptoms to meet criteria, and so you can have totally non-overlapping pictures for someone meeting criteria for schizophrenia but if you anchor to a certain prototype you will miss this. You would neglect lots of types of schizophrenia. Not to mention prototype matching biases towards an inertia in practice, and whatever you saw/conceptualized early isn’t necessarily right. Given that we have no blood test or image that can validate your early suppositions were right, you can easily veer off in psychiatry to continue to see what you want to see.
I don't know all the details re inter-rater reliability. It seems like, from the DSM-5 field trials paper ( https://psychiatryonline.org/doi/10.1176/appi.ajp.2012.12070999 ) that the raters were provided a menu of diagnoses at each site, so for instance one site had PTSD, MDD, AUD, mild TBI, and BPD, and other diagnosis. Others had schizophrenia, Bipolar 1, etc. So the diagnoses are in the same realm, but also the options were limited. Overall this seems like it would inflate the kappa values.
I don't think prototype matching is a "solution," but I do think it's an improvement. A benefit of the prototype system is that it's easy to define several subtypes of a disorder, so I would think that helps the heterogeneity problem. Also, it would be useful to have a way to distinguish "strong" matches vs "weak" matches. I don't think severity quite does this.
I agree that many of these diagnoses are very similar/overlapping and that traits capture a lot of it. Something like an unspecified neurosis would go a long way...
It would be interesting if patients knew that their depression diagnosis was likely wrong... Especially if they got it from a GP. I admit I come from a place of superiority - NGRI is a pretty good place to start from to assert the credibility of my bipolar diagnosis, I'm not sure what else 4 nights of no sleep, literally didn't lie down for 4 nights, bizarre behaviour repeatedly brought to ED by police, in someone who's never abused substances, could be called... Let alone severe psychosis into the bargain... And even I had a senior psychiatrist review me, floridly psychotic, and write in my notes that I didn't have a psychiatric illness (3 forensic psychiatrists disagreed with him)
I agree! I intentionally stayed away from discussing treatment.. I think it's a problem that many patients have clearly incorrect diagnoses on their charts that stick around for years, and that people seem to place great value on obtaining certain diagnoses even when their accuracy is low.
This is my primary interest as well! The missing piece of Psychiatry is a physiological model similar to what other medical specialties have. I think about this a lot. It seems that a big barrier to progress is that we "can't say no" to people. If someone has new shortness of breath, we can do an echocardiogram and say "good news, you don't have heart failure." But even if we had adequate brain imaging, it would seem wrong to tell a sad/down person, "good news, your dopamine circuits are functioning normally. You don't have depression!" For example.
There's a difference between "you need help" and "you have this issue." The first is a statement of "care" -- and most doctors are able to say "You have unspecified pain."
"You need treatment" is fine and dandy to say -- but the insurance company may ask for a diagnosis. And, particularly for mental health, they don't want to hear "Situation Unclear, ask me again in 6 more sessions."
People come to doctors because they want someone else's perspective. Unfortunately, 50% of doctors or so will prescribe placebos, rather than saying "you're fine!"
Yes. In other areas of medicine, it is understood that some symptoms correlate to "objective" findings or test results, and some do not, and there is usually a distinction between these. You can imagine something similar in psychiatry, where we ascribe causes to symptoms/syndromes in some cases, but in other cases are not able to do so. This could lead to progress in understanding pathophysiology. Maybe!
I want to suggest a different paradigm. It's weather forecasting, of all things. You generally have a "face" prediction (50% chance of rain today), and then a "technical discussion" where the NWS forecaster talks about all the models, where they differ, and when he's going with his gut.
You kind of want that "technical discussion" to be passed from clinician to clinician.
Ideally, of course, you want predictive "here's what I think will happen" so that you can actually be differentiating between diagnoses. "If he's got anxiety as primary, he will do XYZ in response to going to the anime convention." "If he's got MDD as primary, he will do ABC in response to going to the anime convention."
The way people cope with physical illnesses is more psychology than psychiatry... I'd say there's no evidence for any psychiatric aspects of OSA, it'll all be non evidence based and so small effects they're statistical noise... beyond the obvious consequences of sleepiness...
This is top-notch stuff! And damn, I think you are right. I’ve suspected so for a while but didn’t quite know how to articulate it. I’ve been thinking about this problem in recent years from the angle of “non-specific psychopathology” as discussed in a 2023 paper by Peter Zachar: https://onlinelibrary.wiley.com/doi/10.1002/wps.21043
Your post adds new consideration to this. I’ll try to articulate my own thoughts on how we can navigate this mess. I love your idea of assigning confidence/probability ratings to diagnoses. This is something I’ve doing informally when I’m giving an official diagnosis for various reason but I am highly uncertain or have low confidence in it’s applicability.
Thank you, I appreciate that. The non-specific psychopathology concept (neuroticism?) is something I have thought about for a long time. Rereading the Ioannidis paper supplied a framework that I think is valid here.
As I read this, I was thinking of restacking (? Is that the word) a paragraph, but then I read more and I wanted to restack that, and that feeling kept going throughout this piece. Definitely one of the best I have read here.
It has been something I have been sensing more and more as I have more years under my belt, but you articulated it much better than I ever could have - my sense that, even as a psychiatrist, most psychiatric diagnoses are false.
You hinted at many of the biases leading to that result, from individual (clinician inexperience, or “pet diagnoses”), to systemic (needing a diagnosis for billing reasons). Ultimately the whole system is set up to diagnose, which compounds the problem.
This is an outstanding piece that's rigorous, clear-eyed, and soberingly honest about the limits of our tools. Thanks for laying it all out with such precision. I especially appreciated the framing of diagnostic interviews as underpowered tests run in biased conditions, often with unacknowledged thresholds and poor predictive value.
Reading this, I found myself thinking less about the technical implications (which are important, of course, but already familiar to me) and more about the *existential* ones. Particularly, the way the psy-disciplines (e.g., psychiatry, psychology, neuropsychology, psychotherapy) increasing seem to be coping with the terror of diagnostic uncertainty by doubling down on diagnostic essentialism.
That is, as our understanding of what a diagnosis *means* complexifies, we more fervently assert that having a diagnostic label is THE key to understanding a person. We've trained our clients to twist their complex selves and multifaceted pain into the constricted question of “Why do I have symptoms X and Y?”, which we then answer even more narrowly with “Because you have Symptom X and Y Syndrome.” The tautology is rarely acknowledged. Nor is the fact that the entire process reflects not just symptoms, but the patient’s worldview, the clinician’s worldview, and the social meanings attached to different diagnoses.
What most worries me is that the diagnostic “if” (if the person has a disorder) becomes a stand-in for more perceptive and generative questions we could be asking. When “if” becomes the central question, it subtly conveys that diagnostic legitimacy is the only path to being taken seriously --especially if the answer is “yes” to a popular diagnosis (internalizing, neurodevelopmental, or trauma-related), and “no” to an unpopular one (personality disorder, psychosis, low IQ, externalizing). This maps disturbingly well onto cultural scripts about whose suffering is understandable, whose is dangerous, and whose is inconvenient.
In my field of psychological and neuropsychological assessment, this narrow frame is often even more entrenched. We specialize in lengthy and complex evaluations, so we have the time, the tools, the training. And yet, we chase the “if” with near-religious intensity, rationalizing our preferred answer even when the data wavers. We sidestep more fruitful questions, such as:
-- Why is this (usually longstanding) problem surfacing so acutely now, and in this form?
-- When is the problem better or worse, and what helps?
-- Where in this person’s life are the conditions more favorable for growth or ease?
-- Who is this person, and how does that self-understanding shift the landscape of intervention?
When we chase diagnosis as destiny, we forget to ask how the person can engage with long-standing human needs: for meaning, self-compassion, challenge, responsibility, rest, and contribution. We don't ask, how is the balance for this person? Are they experiencing too much adversity (perhaps too much challenge, too little meaning, and too little rest), or are they instead adrift, avoiding responsibility and meaning (perhaps mistaking accountability for blame and desirable difficulty for suffering)? What small actions could shift them toward a life where they feel more alive, authentic, accomplished, or adored?
None of this is easy. The pressures (though they shift with setting) are real: time constraints, reimbursement demands, sociocultural dynamics around self-diagnosis, and the sheer inertia of systems. But I do wonder, have we lost the art (and nerve) of case conceptualization because we feel it doesn’t matter anymore? Do we feel that the treatments are the same regardless (statistically, the person is probably already on antidepressants and stimulant medication and in therapy anyway), and we're worried that without something more to say, some new diagnosis to add to the client's chart, we risk becoming irrelevant? Do we feel like our own value as clinicians is unearthing the most 'invisible' diagnoses, since we often feel powerless to shift the structures of a person's life? Is this because we can barely touch the cultural malaise making so many of us feel less attentive, less awake, less able, and less attached to meaning and community? Or are we afraid to recommend anything that involves real effort, loss, responsibility, or hope?
It’s a strange time to be in this field. Many of us sense that something isn’t working. But instead of returning to the harder questions (why, what now, how else), I fear we're increasingly clinging to the shallow comfort of “if.” For a while, this may keep us afloat, supporting our ability to feel useful in the face of complexity we can’t resolve. But what happens when AI can answer the 'if' questions as "accurately" as we can, with much less time and expense? Won't it be essential for us to have more to say?
Thank you again for this thoughtful piece. I will definitely be sharing it with my colleagues.
Thank you! I really feel that psychiatry/clinical psychology has found itself at something of a dead end with the current nosology, but we are in too deep to change it. Imagine the public reaction if we came up with some entirely new system and people's favored diagnoses went away.
Phenomenal. Exactly what I needed to find today. Thank you!!! 👏👏👏💯🏆
This is a good article. Far better than most on this topic. A few critiques.
Critique 1:
Firstly, you don’t distinguish sharply enough between two claims:
(1) Most patients who receive a psychiatric diagnosis are diagnosed with at least one condition they don’t have
(2) Most of the time when someone is diagnosed with a specific psychiatric condition, they do not have this condition
Claim (1) is what your models actually show. They do not show claim (2). In particular, the ‘hitting a bullseye…’ section would not apply if you were trying to show (2). And the ‘typical evaluations are underpowered’ section would have to be modified as well.
Several commentors have clearly mixed up claims (1) and (2) and you haven't corrected them. Suppose I'm reading someone's chart and I see 'schizophrenia' . It's one thing to say 'this person probably has at least one incorrect diagnosis'. It's quite another to say 'this person probably does not have schizophrenia'. (1) does not imply (2).
Critique 2:
By focusing on (1), what you mean by an ‘incorrect diagnosis’ is that at least one of the conditions the patient has been diagnosed with is a condition that they don’t have. That’s not necessary completely unreasonable. But I think you should clarify why you’re focusing on this.
Personally, I wonder if it would make more sense to say that a patient has been incorrectly diagnosed if the most severe condition they have been diagnosed with is one they don’t have. Arguably, it’s this issue that would raise concerns about undermining trust in the psychiatry profession.
Critique 3 (a minor point):
Technically, the multiple comparisons issue that you point to can also increase the probability of a *correct diagnosis.*. You don’t make this explicit but it looks like what you’re modelling as a correct diagnosis is:
the patient receives at least one psychiatric diagnosis, and all of the diagnoses they receive are for conditions they have.
Now consider a patient who actually has conditions A, B, and C. And suppose multiple testing means that the patient gets tested for B and C instead of just A. This will increase the probability of a correct diagnosis. I don’t think this would seriously undermine your argument for (1) but it should be factored into your model.
But finally, note: you could derive (2) from a much simpler model, at least for certain conditions. Suppose almost everyone that shows up looking to get diagnosed with ADHD gets their diagnosis. And suppose almost no one who doesn't look for the diagnosis gets it. In this case, the probability an ADHD diagnosis is true will be roughly equal to the base rate of ADHD amongst people who show up looking for that diagnosis
Thank you for the thoughtful critiques. I truly appreciate it. I think (1) is certainly true and (2) is often true but depends on the condition. I think I state (3) but do not emphasize it.
As I think more, the biggest issue seems to be my point re small effect sizes, both in terms of distinguishing normal from abnormal, and distinguishing one similar condition from others (depression, anxiety, ADHD, PTSD, social anxiety, etc).
So if somebody presents with what could be mild depression, the standard approach is to diagnose them with MDD and add a mild specifier. But the fact of mild symptoms (small difference from normal population) also implies greater uncertainty about the diagnosis and raises the rate of false positives when the test is underpowered. This can lead to a single erroneous diagnosis or multiple erroneous diagnoses (maybe they get a depression diagnosis and an anxiety diagnosis), made at a single visit or multiple visits.
Re your point (2), I do think that many chart diagnoses are effectively worthless. I have long instructed trainees that seeing a diagnosis of bipolar disorder in somebody's chart should have no effect on their suspicion that the person has bipolar disorder. The same has become true of PTSD, autism, ADHD etc in more recent years. Others are more reliable - OCD, MDD with psychosis, probably schizophrenia, dementia, delirium
Bigger picture, I would describe the situation I see as something like this. Take 100 people who each have at least one psychiatric diagnosis. Of these 100 people, 70 of them have at least one correct diagnosis. But the total number of diagnoses in the group might be 250. Some people have one diagnosis (correct or not), some people have 2 or 3 or 4 (less likely to be all correct). So most of the diagnoses are false, even if a majority of the individuals do have one correct diagnosis.
You seem more mathy than me, so if you think of a good way to model this, please let me know.
Great job explaining sensitivity/specificity and PPV/NPV, especially considering you did a nice job explaining prevalence for the latter. Regardless of specialty, many clinicians will describe sensitivity/specificity fine but then go into PPV/NPV without contextualizing the measures within prevalence beyond a passing blow. I always try to give the clinician-authors the benefit of the doubt (I’m an ID epidemiology trainee) with epi methods, but even JAMA was publishing stuff up til the relatively recent JAMA statistics and methods editorial shift that any well-trained, competent epidemiologist could smell a mile away. The ADHD Dx and Tx literature gets this stuff right maybe 50% of the time. (With the disclaimer that I'm not a psychiatric epidemiologist; this stuff is more of a hobby research interest of mine 🥲)
As a layman who listens to many discussions on ideation, including that of psychiatric diagnoses, it strikes me that much of the time what is described as a disorder is an adaptation of thought to circumstance. In physical terms, a man without fingers (or with two thumbs, for that matter) must adapt to the circumstance he finds himself in, as best he can (and often poorly).
Psychiatric explanations seem to be made up out of thin air for just about everything over the past century. Prior to that, how was behavior explained? The actions of the planets (melancholy, due to the effect of Saturn, for example, resulting in the common adjective, saturnine) or witchcraft or the tenets of a religious belief, even numerology. Psychiatry is, of course, godless, but only to the extent that the self has taken the place of gods (or God) in the minds of many.
You might enjoy Dr. Randy Nesse's work, Good Reasons for Bad Feelings. I think you are right that some psychiatric disorders are best seen as adaptations, or failure to adapt. You could imagine a parallel between what we call chronic depression due to life circumstances and physical over-use injuries that many people have, for example.
Completely agree! Thankyou for your comment, it balances the weight of the 'intellectual' to that of being human. I always wonder where the spiritual comes in within psychiatry. And where this is on all of these 'scales' of madness.
I know the way it is evaluated but i believe that this is just another form of oppression unfortunately.
Every so often, that story cycles around about how doctors were unable to apply Bayes' theorem to reason about false positive rates.
That's not really a problem of statistical literacy, it's just basic probability. I'm doubtful that adding adding more *statistics* to the medical curriculum would be very useful at this point in history.
But I think it probably would be good if MDs all knew basic probability in their bones the way a trader or poker player does. Not sure how to make this happen. Having trainees gamble on patient outcomes in the hospital would be very effective but unseemly.
I've been thinking about this. It would be easy enough to put a statistics calculator in an EMR so you could plug in the numbers. I think also changing how we write notes. If you imagine the one-liner/chief complaint as an introduction to the note, part of the expectation could be including at least your initial differential diagnosis, if not numeric probabilities. Some specialties, like Emergency Department, know these numbers pretty well already: 60 yo with this medical probability presenting with chest pain has this likelihood of STEMI, etc.
Having taught FM residents EBM decision making, the issue is actually the attendings. If I am the only attending showing them how to use likelihood ratio to train intuition, and they only work with me once a month they won’t learn it. I am not a big fan of just generating numbers from calculators since people tend to get false certainty from numbers. I used more of green, yellow, red tier system with green being diagnosis certain and treatment clear, yellow diagnosis is somewhat likely but other factors in play so gather information frequently while being cautious with treatment, red diagnosis not clear and need more information before treating or get a consult. Something with a good likelyhood ratio could take from red to green. Intermediate likely hood ratios would need multiple pieces of data pointing the same way. My big problem with the average therapist is lack of critical thinking about the certainty of diagnosis and then with shear amount of time they spend with the patient pigeon hole them into what is going on.
Good point about numbers and false certainty. That seems to be a general pitfall with Bayesian thinking. There also seems to be an issue with EMRs, the "publication bias" I alluded to, where a diagnosis that was intended to be provisional soon becomes "official" when the next provider looks at the problem list.
This all sounds nice, but I find it doubtful your suggestions will find its way in most (non-academic) inpatient settings. There is simply no economic incentive to implement what you said.
Sir, do you remember that 90s movie, K-Pax, with Kevin Spacey? In the opening scenes, for an entire 15 minutes, we watch dozens of psychiatrists, neurologists, and so forth conducting all these tests, to gaslight the public into believing this is how inpatient settings work: EVERYONE gets tens of thousands of dollars worth of fMRIs, EEGs, EKGs, neuropsych exams, 500+ item MMPIs, SCIDs, a 20-hour+ multi-week interview, a Rorschach and so forth. And from there, hollywood tries to tell us "This is how comprehensive the average inpatient setting really is". It's all propaganda.
When I worked in them, not once did anyone administer any scale. Where are all the clinical psychologists? The psychometrists? We didnt have those. It was a handful of NPs, PA-Cs, remote-only MDs (who are not trained in psychometrics) and LPCs and LCSWs (who were not trained in signal detection theory, but they sure were trained in critical theory). So i kept asking myself "But why did i get all this training in the MMPI? Why did i get this training in scale development from start to finish, and how to CFA them, and how to discriminate validity scales on different categories, if no one uses it in real life? Aren't we supposed to be administering, say, the MCMI if were to throw around big words like BPD? Not even the SCL-90??? And why aren't they all walking around with clipboards and white coats? You mean to tell me they don't even PRETEND to be clinicians, either?
Instead, dozens of "clients" per week (inevitably, future anti-psychiatry persons) were labeled with vague cluster B's following single 25 minute teams meetings conducted by the "Chief Clinician": an LPC. Not an MD, not a DO, not a Phd: a lowly, MA-level, only-recently-licensed LPC. Because this is the minimum standard in that state. And again, the MDs refuse to show up in person. They do all their diagnosing behind a computer screen from home. Neither did our LPC actually speak to clients much. Rather, her evidence base was reading prior charts, some years old, and searching for "key words": anything pathologizable. From there she role-played out confrontational dynamics to "test the hypothesis". Ah, yes, Bayes theorem! But this merely caused (relatively normal?) people to become upset, who refused to "play along", and as a consequence, were nevertheless labeled with PD for the crime of negatively reacting toward being treated as a-priori personality disordered. So because they did not "act disordered", she then used this, itself, as a diagnostic criterion. "They're faking-good!", she asserted, "These are not good people, Scott. If they weren't here, they'd be in jail instead".
Half were merely homeless. An extremely consistent pattern emerged from the actual, first-person, "raw" data: In the hours leading upto scheduled discharge, our clients would "develop" delusions/conspiracies/pseudo-psychosis/somatic problems, "I'm in PAIN!" but they were very bad at actually being believable by those of us who DO have histories of involuntary institutionalization. Still, our resident PA-C, "by regulation", would then have to initiate a new hold, where they'd be escalated to another hospital to repeat the cycle. Sometimes, however, if a client learned their stay were to be extended by a week, all the sudden they behaved as if they were in a luxury hotel: they shifted to being enthusiastic, highly assertive, and then would make demands that we order them pizza, upgrade the coffee machines and use more caffeine, and facilitate DoorDash runs to the 7/11 out of pocket.
All the while, the LPC would claim during our "clinical development" meetings that "NPD is an extremely common diagnosis in our particular area. It's interesting but a little over half of my direct clients will receive an NPD diagnosis. I cant believe how many of my colleagues miss these. I host talks all the time at our satellite sites". Without any self-awareness, she was veridical that the base rate for NPD is ~60%.
(Isnt NPD like <1% in gen pop and still <5% inpatient? I bet you its much higher in med schools tho!)
I went to HR about the whole fiasco, to report there's medical fraud, in the form of non-existent and false Dx's going on, that's also being billed to tax payers (its a 501c contracted by the state, and some of these are court ordered, so, like, there's real consequences to this. Or are there??). They told me my actions of reporting these constituted "A threat to our operations and our professional community" and that I must resign.
It was February 2020, and they were about to hit the jackpot from the whole PPI/CARES thing.
By claiming that there are false diagnoses you suggest the possibility of true diagnoses in psychiatry. A look at the current state of psychopathology and its taxonomy raises lots of doubts.
https://www.tandfonline.com/doi/full/10.1080/09638237.2021.2022638
So, I imagine that most clinicians couldn't tell you the Bayesian likelihoods of MDD (what I'm familiar with, I'm certain other diseases are also predictive for other psych disorders) on cancer versus "Metabolic Syndrome" (heart disease). If a doctor can't even estimate "what's the probability" based on the known diseases, of course you're a lot of false diagnoses.
(By the same token: do you diagnose someone with MDD if they have a "contributory illness"? If they have IBS, which is significantly more of a cause than a contributing factor? If you don't diagnose, are you still allowed to treat?)
This is quite good.
Two comments: 1. Curious in these clinical gold standard interviews, when there is lower reliability, how far off the mark so to speak the diagnosis is. It’s one thing if someone who has schizophrenia is being diagnosed as just ptsd or GAD, but it’s another if they are “off” by diagnosing schizoaffective, schizophreniform, schizotypal, substance induced psychosis etc. If people are diagnosing their diagnostic neighbors, then I don’t think it is necessarily as bad of a problem. I for example have almost no interest in appropriately diagnosing MDD versus PDD versus GAD, versus primary panic instead of a panic subtype - I think of it as general internalizing and neuroticism, and don’t want to falsely reify these diagnoses to patients. Typically the treatment each of these follows in the same way.
2. I worry about prototype matching as an explicit approach (since we do this anyways as clinicians) - this would work if our diagnoses were not so heterogeneous and frequently have poor construct validity, and as you mention largely the absence of diagnostic hierarchy. I’ve seen, for example, residents say a patient doesn’t have schizophrenia because the patient wasn’t hallucinating or disorganized; of course you only need 2 out of 5 core symptoms to meet criteria, and so you can have totally non-overlapping pictures for someone meeting criteria for schizophrenia but if you anchor to a certain prototype you will miss this. You would neglect lots of types of schizophrenia. Not to mention prototype matching biases towards an inertia in practice, and whatever you saw/conceptualized early isn’t necessarily right. Given that we have no blood test or image that can validate your early suppositions were right, you can easily veer off in psychiatry to continue to see what you want to see.
Thanks.
I don't know all the details re inter-rater reliability. It seems like, from the DSM-5 field trials paper ( https://psychiatryonline.org/doi/10.1176/appi.ajp.2012.12070999 ) that the raters were provided a menu of diagnoses at each site, so for instance one site had PTSD, MDD, AUD, mild TBI, and BPD, and other diagnosis. Others had schizophrenia, Bipolar 1, etc. So the diagnoses are in the same realm, but also the options were limited. Overall this seems like it would inflate the kappa values.
I don't think prototype matching is a "solution," but I do think it's an improvement. A benefit of the prototype system is that it's easy to define several subtypes of a disorder, so I would think that helps the heterogeneity problem. Also, it would be useful to have a way to distinguish "strong" matches vs "weak" matches. I don't think severity quite does this.
I agree that many of these diagnoses are very similar/overlapping and that traits capture a lot of it. Something like an unspecified neurosis would go a long way...
It would be interesting if patients knew that their depression diagnosis was likely wrong... Especially if they got it from a GP. I admit I come from a place of superiority - NGRI is a pretty good place to start from to assert the credibility of my bipolar diagnosis, I'm not sure what else 4 nights of no sleep, literally didn't lie down for 4 nights, bizarre behaviour repeatedly brought to ED by police, in someone who's never abused substances, could be called... Let alone severe psychosis into the bargain... And even I had a senior psychiatrist review me, floridly psychotic, and write in my notes that I didn't have a psychiatric illness (3 forensic psychiatrists disagreed with him)
Outstanding summary. Thank you for writing it.
I agree! I intentionally stayed away from discussing treatment.. I think it's a problem that many patients have clearly incorrect diagnoses on their charts that stick around for years, and that people seem to place great value on obtaining certain diagnoses even when their accuracy is low.
This is my primary interest as well! The missing piece of Psychiatry is a physiological model similar to what other medical specialties have. I think about this a lot. It seems that a big barrier to progress is that we "can't say no" to people. If someone has new shortness of breath, we can do an echocardiogram and say "good news, you don't have heart failure." But even if we had adequate brain imaging, it would seem wrong to tell a sad/down person, "good news, your dopamine circuits are functioning normally. You don't have depression!" For example.
There's a difference between "you need help" and "you have this issue." The first is a statement of "care" -- and most doctors are able to say "You have unspecified pain."
"You need treatment" is fine and dandy to say -- but the insurance company may ask for a diagnosis. And, particularly for mental health, they don't want to hear "Situation Unclear, ask me again in 6 more sessions."
People come to doctors because they want someone else's perspective. Unfortunately, 50% of doctors or so will prescribe placebos, rather than saying "you're fine!"
Yes. In other areas of medicine, it is understood that some symptoms correlate to "objective" findings or test results, and some do not, and there is usually a distinction between these. You can imagine something similar in psychiatry, where we ascribe causes to symptoms/syndromes in some cases, but in other cases are not able to do so. This could lead to progress in understanding pathophysiology. Maybe!
I want to suggest a different paradigm. It's weather forecasting, of all things. You generally have a "face" prediction (50% chance of rain today), and then a "technical discussion" where the NWS forecaster talks about all the models, where they differ, and when he's going with his gut.
You kind of want that "technical discussion" to be passed from clinician to clinician.
Ideally, of course, you want predictive "here's what I think will happen" so that you can actually be differentiating between diagnoses. "If he's got anxiety as primary, he will do XYZ in response to going to the anime convention." "If he's got MDD as primary, he will do ABC in response to going to the anime convention."
The way people cope with physical illnesses is more psychology than psychiatry... I'd say there's no evidence for any psychiatric aspects of OSA, it'll all be non evidence based and so small effects they're statistical noise... beyond the obvious consequences of sleepiness...