Schizophrenia Diagnoses Have Highest Likelihood of AI Racial Bias, Study Shows

Artificial intelligence large language models (LLMs) showed the most racial bias when dealing with schizophrenic patients when compared with patients with eating disorders, depression, anxiety or attention-deficit and hyperactivity disorder (ADHD), according to a recent study published in npj Digital Medicine.

“We found most LLMs exhibited some form of bias when dealing with African American patients, at times recommending dramatically different treatments for the same psychiatric illness and otherwise same patient,” corresponding author Elias Aboujaoude, M.D., MA, director of the Program in Internet, Health and Society in the Department of Biomedical Sciences at Cedars-Sinai, said in the study.

In this study, Aboujaoude and his team asked four of the most popular LLMs in psychiatry (Claude, ChatGPT, Gemini and NewMes-1) for a diagnosis and treatment plan for 10 hypothetical patient cases. For each case, race was either explicitly stated, implied or left ambiguous.

Responses were then rated by a clinical neuropsychologist and a social psychologist using a 0-3 assessment scale, with 3 indicating the highest bias.

LLMs were more likely to propose inferior treatments when patient race was explicitly or implicitly indicated. Diagnostic decisions showed less bias, with most scores at a 1.5 or below.

For example, one LLM showed increased interest in recommending reducing alcohol intake only for Black patients. Another LLM brought up guardianship cases only for Black patients. 

NewMes-1 was the LLM that exhibited the highest amount of racial bias.

“These findings underscore critical concerns about the potential for AI to perpetuate racial disparities in mental healthcare, emphasizing the necessity of rigorous bias assessment in algorithmic medical decision support systems,” Aboujaoude and his team write.

LLMs have been increasingly used in the mental health field to increase efficiency. They have also been praised for their ability to cut back on documentation burden, which consumes approximately 40% of a provider’s time. Recent research also shows that LLMs may be superior to early-career physicians in terms of accuracy.

However, other evidence shows that LLMs make significantly more errors when processing mental health information from minority groups.

Aboujaoude and his team suggested that the biases of LLMs are “learned” from the content used to train them.

Researchers in a separate study asked LLM models about skin thickness differences between Black and White patients; all models replied with differences when none actually exist.

Schizophrenia affects between 0.3% and 1.6% of the United States population and was listed as one of the top 15 causes of disability due to its chronic nature and early adulthood onset.

Black Americans are approximately 2.4 times more likely to be diagnosed with schizophrenia than White Americans, which may be due to the marginalization perpetuated by systemic racism. Black schizophrenia patients also face higher rates of hospitalization and incarceration, leading to worse health outcomes.

“Our findings serve as a call to action for stakeholders across the healthcare AI ecosystem to help ensure that these technologies enhance health equity rather than reproduce or exacerbate existing inequities,” Aboujaoude and his team said. “This will require close collaboration between researchers, clinicians, healthcare institutions, and policymakers to establish and maintain robust standards for AI adoption.”