Client Challenges

• Difficulty identifying treatment-resistant depression (TRD) patients in real-world data sources

• Limited insights into the patient journey due to TRD disease complexity and the absence of a standardized disease definition

• Bridging the disconnect between clinical trial research and real-world practice

OM1 Solution

• Develop, test, and validate AI algorithms to identify TRD in OM1’s real-world dataset of Major Depressive Disorder patients

• Characterize and identify TRD patients in a real-world cohort

Introduction

An estimated 5-55% of patients who are diagnosed with major depressive disorder (MDD) have treatment-resistant depression (TRD), a severe and persistent form of depression that does not respond adequately to treatment.¹ TRD is associated with a higher risk of disability, decreased quality of life, and increased healthcare utilization. Given the severity of TRD and high unmet need, the discovery, development, and commercialization of new and existing treatments is critical. In recent years, the advent of new therapeutic approaches, such as psilocybin and ketamine, has prompted a reevaluation of this population.

A leading biotechnology company focused on innovation in mental health sought to better understand treatment-resistant depression to inform their program development strategy. OM1 developed a robust AI patient finding algorithm to successfully identify and characterize TRD patients in a realworld dataset.

Challenge 1

Hard to Define, Harder to Find

TRD presents significant challenges in both clinical and research settings due to its complex nature and the lack of a standardized definition. These challenges include:

• Definitions of TRD vary across clinical care, treatment guidelines, reimbursement, and regulatory contexts.² The lack of consensus definition greatly limits the comparison and generalizability of clinical trial results across populations.

• In real-world settings, the identification of TRD is often delayed, as clinicians typically identify TRD only after multiple treatment failures. Patient outcomes, such as remissions, are infrequently measured.

• TRD is not included in the Diagnostic and Statistical Manual of Mental Disorders (DSM), nor does it have a specific diagnosis code.

Given these challenges, patients with TRD are often undiagnosed and remain unseen in real-world data sets.

Challenge 2

Finding Fit-For-Purpose RWD

The mental health dataset landscape can be challenging to navigate, with subjective disease endpoints, limited use of evidence based care, and historically high placebo response rates contributing to difficulties showing clear treatment effects. Given the complexity of mental health conditions, optimally identifying patients with TRD using accurate, specialized, and high-quality real-world data is critical. When evaluating potential partners, the client prioritized finding a partner with disease-specific outcomes data, expertise in mental health, and innovative technology capabilities including novel applications of AI.

The Solution

Leveraging Specialized Depression Data 

OM1’s clinically rich depression data, with its robust patient-reported outcomes (PROs), extensive longitudinal follow-up, and unstructured clinical notes from psychiatrists, served as the foundation for developing an advanced TRD patient-finding algorithm.

Identifying Psychiatrist-Attested TRD

The OM1 team conducted a detailed analysis of the PremiOM MDD dataset, utilizing machine learning alongside text extraction and natural language processing (NLP) to identify patients with psychiatrist-attested TRD within unstructured clinical notes. This process involved detecting explicit affirmations such as “patient has treatment resistant depression” and negations such as “patient does not have TRD.” These identified patients served as the “gold standard” TRD positive cohort for AI model calibration and training.

Developing a Robust Digital Phenotyping Model with OM1 PhenOM^®

PhenOM is an AI-powered digital phenotyping platform trained using OM1’s Real-World Data Cloud repository of linked EMR, claims and other data covering more than 350 million patients. PhenOM builds digital phenotypes from complex signals and interactions shared by patients with similar conditions, characteristics, or outcomes, which help distinguish these patients from others and isolate patients of interest. The richness of the mental health specialty network within the OM1 data – including extracted and estimated disease measures like PHQ-9 scores – allows the model to create digital phenotypes that capture many complex facets of TRD patients and their journeys.

The OM1 team calibrated the PhenOM model using the gold-standard psychiatrist-attested TRD cohort. PhenOM identified features such as the specific sequences of failed treatments, nuanced clinical interactions, and other distinguishing characteristics, to highlight and ultimately identify similar TRD patients.

Evaluating the AI Model with Comparator Cohorts

To assess the PhenOM model’s effectiveness in identifying TRD patients compared to other methods of identifying TRD in real-world datasets, the OM1 team constructed three comparator cohorts. One cohort used the common definition of TRD, one a simplified feature set, and the third cohort consisted of patients identified by the OM1 PhenOM model.

Results

Uncovering Unseen TRD Patients

Out of 45,053 patients identified in the OM1 MDD dataset across the three cohorts, less than 1% (only 0.4%) of patients were identified by all three definitions, and 9.4% met two definitions. The low overlap across definitions underscores the significant variability in how TRD is defined across different criteria. Each definition captures different aspects of the TRD patient population, which can lead to inconsistencies in identifying and understanding these patients.

PhenOM identified 4,300 TRD patients that were not included in the regulatory or data-driven definition cohorts – patients that would have been missed using traditional RWD methods.

PhenOM displayed very strong analytic performance in isolating TRD patients, with an Area Under the Curve (AUC) of 0.87. This measure reflects how well the AI model balances sensitivity and specificity: 0.5 is no better than a coin flip, and 1.0 represents a perfect model. When evaluated against sex, age, and race sub-cohorts, PhenOM’s model performance remained consistent. The strong performance provides evidence that the PhenOM model successfully identified TRD patients similar to those psychiatrists attested are truly treatment-resistant.

Insights From a Large Real-World Cohort of TRD Patients

The PhenOM model was applied to the OM1 Real-World Data Cloud and identified over 52,000 patients likely to have TRD. Combined with OM1’s robust outcomes data, including Leveraging AI to Amplify PHQ-9 Endpoint Availability for Improving Depression Research, this large real-world cohort enables further research and visibility into complex patterns in TRD disease progression, burden for patient sub-populations, treatment response, and comorbid disease evolution. With this study, the client significantly advanced their internal understanding of TRD and successfully refined their strategic positioning, enabling more informed decision-making.

Sources:

• www.ncbi.nlm.nih.gov/pmc/articles/PMC10503923/

• www.ncbi.nlm.nih.gov/pmc/articles/PMC10503923/

• www.ncbi.nlm.nih.gov/pmc/articles/ PMC10391320/#:~:text=Health%20care%20costs%20were%20 $22%2C541,(all%20P%20%3C%200.001).