Introduction
Worldwide, it is estimated that up to 425 million adults aged 30 to 69 years have moderate to severe obstructive sleep apnea (OSA), for which treatment is recommended [1]. Continuous positive airway pressure (CPAP) is the typical first-line treatment for OSA. Excessive daytime sleepiness (EDS) is a frequent symptom of OSA and is associated with adverse consequences such as an increased risk of driving accidents [2].
Sleepiness is routinely estimated in the clinic using the Epworth Sleepiness Scale (ESS), which ranges from 0 to 24 points using 8 questions on a 0 to 3 Likert scale [3]. While meta-analyses report that CPAP improves ESS by an average of 2.4 to 2.9 points more than placebo [4, 5], up to 28% of all treated patients and 19% of optimally treated patients (≥7 hours/night CPAP use) have residual sleepiness (ESS >10) [6, 7]. As such, predicting the change in ESS from baseline data could support clinicians in prescribing CPAP and assist patients in making informed decisions about CPAP uptake.
Several previous studies have investigated the statistical association between baseline characteristics and the change in sleepiness following CPAP treatment. In particular, higher baseline ESS was correlated with a greater change in ESS (R2 = 0.29 to 0.68) [8], while Bhat et al. identified significant correlations between the change in ESS and both baseline apnea–hypopnea index (AHI) and baseline time below 90% oxygen saturation (T90) in severe OSA [9]. Conversely, Kingshott et al. reported no significant correlations between the change in ESS and baseline AHI, arousal index, oxygen desaturation index (ODI), or minimum oxygen saturation in severe OSA [10].
The Pittsburgh Sleep Quality Index (PSQI) questionnaire assesses sleep quality and has a weak correlation with the ESS (r = 0.16) [11]. One study revealed that baseline AHI was not associated with the change in ESS, and that lower baseline AHI was paradoxically linked to greater PSQI improvement [12]. Balakrishnan et al. demonstrated that baseline OSA severity category (mild, moderate, and severe) and a composite metric, the Sleep Apnea Severity Index (SASI), were associated with CPAP-related change in PSQI [13].
Recently, several algorithms have been developed to quantify different domains of OSA pathophysiology from diagnostic polysomnography (Figure 1), including event-related [14] and breath-related [15] ventilatory burden, flow limitation severity and frequency [16, 17], and hypoxemia severity [18, 19]. These measures have stronger associations with outcomes such as sleepiness, vigilance, cardiovascular disease, and all-cause mortality than conventional clinical metrics such as the AHI [20–22]. Despite this evidence, no studies to date have attempted to rigorously predict the CPAP-related change in ESS using conventional or advanced polysomnography metrics measured before treatment initiation.
The goal of this study was to conduct a multi-cohort analysis to determine whether baseline polysomnographic, clinical, and demographic characteristics can predict the change in ESS following CPAP treatment. We hypothesized that: (1) statistical inference would reveal that physiology-based measures of OSA severity (such as ventilatory burden, flow limitation, and hypoxemia severity) are associated with both baseline ESS and the change in ESS, independent of potential confounders and mediators; (2) change in ESS can be predicted from baseline characteristics using machine learning regression models, and that physiology-based measures of OSA severity would have greater predictive ability than conventional measures of OSA severity such as the AHI; and (3) predicted change in ESS can be used to stratify responders (≥2 points ESS improvement [23, 24]) from non-responders using machine learning classification models. Finally, we aimed to develop a chart that allows clinicians to use baseline characteristics to predict the change in ESS following CPAP treatment.
Materials and methods
Study design, datasets, and participants
This study analyzed polysomnographic, demographic, and ESS data from five cohorts available from the National Sleep Research Resource (NSRR) [25]: Home Positive Airway Pressure (HomePAP; n = 133), Best Apnea Interventions for Research (BestAIR; n = 62), Apnea, Bariatric Surgery, and CPAP (ABC; n = 18), Stanford Technology Analytics and Genomics in Sleep (STAGES; n = 324), and the Multi-Ethnic Study of Atherosclerosis (MESA; n = 1795). To manage the inherent inhomogeneity of combining multiple datasets, cohort selection involved optimizing the trade-off among three competing criteria: maximizing sample size for robust machine learning model training and evaluation, maximizing the number of quantifiable OSA severity measures, and ensuring participant comparability. Participants with OSA in the HomePAP, BestAIR, and ABC cohorts had a baseline ESS measurement and diagnostic polysomnography before CPAP treatment, and a follow-up ESS measurement after treatment.
Treatment duration varied between and within studies and the median duration was 3 [3–6] months (IQR: 3–6 months). The STAGES and MESA cohorts were selected because they are large cross-sectional studies where participants had a single ESS measurement and polysomnogram. These two cohorts were analyzed to determine which predictor variables are associated with baseline sleepiness and might therefore be predictive of the change in ESS after CPAP treatment.
See Figure 2 for a summary of the data analysis steps. Study details and exclusion criteria are presented in prior publications [21, 22, 26–28] and Figure 3. Analyses were approved by The University of Queensland Research Ethics and Integrity unit (2023/HE000064, 2023/HE001953, and 2021/HE002256).
The study was conducted in accordance with STROBE guidelines, and the prediction models were developed and reported according to the TRIPOD+AI statement.
Polysomnography scoring and respiratory and desaturation event harmonization
All polysomnograms from all cohorts were scored according to the American Academy of Sleep Medicine (AASM) guidelines; however, different criteria were used between cohorts. Inconsistent criteria to define apneas, hypopneas, and oxygen desaturations may lead to subsequent inconsistencies in defining and categorizing OSA severity [29]. Therefore, in this study, scored respiratory and desaturation events were algorithmically adjusted to make scoring consistent across cohorts [30].
Specifically, only respiratory events that began in an epoch of sleep were included. All apneas were included and only hypopneas with ≥3% desaturation or arousal were kept (AASM recommended criteria [31]). Only desaturations ≥3% from baseline to nadir were kept.
The HomePAP study
HomePAP participants had baseline AHI ≥15 events/hour and ESS ≥12 points and were randomized to home or in-lab diagnostic and management pathways. HomePAP polysomnograms were scored according to AASM criteria, requiring a 30% reduction in the nasal airflow signal along with at least a 4% desaturation [26]. Embletta polysomnography recording equipment was used for home studies.
Participants with a home study AHI <15 events/hour also underwent in-lab polysomnography to determine their eligibility for inclusion in the study. In the analyses conducted here, the AHI for patients who underwent both home and in-lab polysomnography was calculated based on their in-lab study. A subset of in-lab studies used a split-night CPAP titration protocol; all signals, events, and sleep stages were discarded during titration periods.
In HomePAP home polysomnograms, sleep staging was not conducted, and total sleep time (TST) could not be calculated because electroencephalography was not recorded. However, using a validated method, the total recording time was calculated from channels other than electroencephalography [32] and scaled to estimate TST. Of the initial 373 HomePAP participants, 63 underwent both home and in-lab polysomnography and had ≥2 hours of analyzable nasal airflow signal.
The median ratio of home polysomnography total recording time to in-lab polysomnography TST was 1.298 for these participants. The 5th and 95th percentiles of the ratio values were 1.015 and 2.010, respectively. Therefore, a scaling factor of 1.298 was used to adjust the total recording time of home polysomnography to align with the range of laboratory polysomnography TST.
This TST surrogate was applied in HomePAP home studies for calculating the AHI, desaturation severity, event-related ventilatory burden, hypoxic burden, T90, and ODI.
We used a minimum threshold of 2 hours for analyzable nasal airflow because it represents the standard minimum clinical window required to capture a valid polysomnogram [33]. Ventilation and flow limitation vary dramatically based on body position and sleep stage (particularly during REM sleep, which typically occurs about 90 minutes after sleep onset). A signal duration of less than 2 hours carries a high risk of sampling bias—potentially capturing data exclusively from a single body position or ignoring data from sleep stages—which would fail to accurately reflect the participant’s true pathophysiology during sleep [34].
The BestAIR study
The BestAIR study recruited participants with AHI ≥15 events/hour and prior cardiovascular disease who were not severely sleepy (ESS ≤14 points) to examine the effect of CPAP treatment on blood pressure. BestAIR sleep studies were conducted at home (n = 57) or in-lab (n = 5). Embletta polysomnography recording equipment was used in the home studies.
When arousal data was available, polysomnograms were scored according to AASM alternative hypopnea criteria; otherwise, they were scored using the AASM recommended criteria when electroencephalographic data was not recorded [27]. Wake and sleep were manually scored in the Type 3 home studies based on channels other than electroencephalography. Sleep stages and arousals were manually scored in the Type 1 in-lab studies using electroencephalography channels.
The ABC study
ABC participants had at least one OSA symptom, body mass index (BMI) between 35 and 45 kg/m2, and AHI ≥20 events/hour; and were randomized to CPAP or bariatric surgery groups. The participants who had surgery were excluded from this study. All analyzed ABC polysomnograms were Type 1 and scored according to AASM guidelines [28, 35].
The STAGES study
The STAGES cohort consists of participants with suspected OSA who underwent diagnostic polysomnography. Only polysomnograms recorded at the Stanford Sleep Medicine Center (Redwood City, United States) were used because they contained unfiltered nasal pressure airflow needed for flow limitation analysis. Polysomnograms were scored according to the 2007 AASM guidelines [36, 37].
Respiratory events were not algorithmically adjusted due to the absence of manually scored arousal data; applying such adjustments could have resulted in excluding hypopneas with associated arousals. Note that respiratory events were originally scored using arousals, however, scored arousals were unavailable in the sleep studies provided by the NSRR.
The MESA study
MESA is an ethnically diverse community sample of participants without known cardiovascular disease. Hypopneas were scored irrespective of desaturation using the AASM 30% recommended and 50% alternative criteria [38, 39]. In our study, algorithmic adjustment retained only hypopneas associated with a ≥3% desaturation or arousal.
ESS and other follow-up measurements
Change in ESS was calculated from untransformed follow-up ESS and untransformed baseline ESS (i.e. follow-up ESS—baseline ESS) and then transformed using the Box–Cox transformation (see the “Missing Data and Data Transformations” section below). In HomePAP, follow-up ESS, BMI, and average nightly CPAP use were recorded 1 month and 3 months after treatment initiation. Data from the 3-month follow-up was preferentially used.
Fifteen participants did not have this data, so their 1-month follow-up data was used instead. In BestAIR, follow-up ESS and BMI were recorded at 6 months and 12 months after treatment initiation. Data from the 6-month follow-up was preferentially used to be closer to the 3-month follow-up from HomePAP studies.
Two participants did not have this data, so their 12-month follow-up data was used instead. In ABC, follow-up ESS and BMI were recorded at 9 months and 18 months after treatment initiation. Data from the 9-month follow-up was preferentially used to be closer to the 3-month follow-up from HomePAP studies.
All participants had 9-month follow-up ESS data. CPAP adherence data (average nightly CPAP use) was only available in HomePAP and was unavailable in BestAIR and ABC studies.
Quantifications of OSA severity
Consistent algorithmic adjustment of respiratory and desaturation events was applied across cohorts prior to calculating OSA severity measures [30]. Where possible, multiple algorithms that quantify the same pathophysiological phenotype were used to compare algorithms and in case an algorithm could not be used due to insufficient or inadequate data. OSA severity was quantified using two measures of ventilatory burden [14, 15], two measures of flow limitation [16, 17], two measures of hypoxemia severity [18, 19], and the SASI [40].
Conventional clinical metrics were also calculated, i.e. the AHI, ODI (3% threshold), T90, and lowest oxygen saturation.
Calculating ventilatory burden
“Event-related ventilatory burden” is cumulatively lost ventilation per hour during sleep and was calculated using the uncalibrated nasal pressure (airflow) signal and scored respiratory events [14, 41, 42]. It is quantified by multiplying the frequency of a participant’s respiratory events by the average duration and depth (ventilation deficit) of those events. Algorithmically adjusted respiratory events according to AASM recommended criteria were used.
Breaths were automatically detected, and tidal volume was estimated by integrating the nasal airflow signal. Eupneic ventilation was considered the mean tidal volume per 7-minute sliding window (2-minute steps). This approach eliminates the requirement for a calibrated airflow signal.
Windows with low estimated signal-to-noise ratio (SNR) nasal airflow were automatically excluded [43]. Minute ventilation on a breath-by-breath basis was calculated based on eupneic ventilation. The depth of lost ventilation was calculated for each respiratory event.
Event-related ventilatory burden was calculated from average respiratory event depth × average respiratory event duration × respiratory event frequency. The units of event-related ventilatory burden are %eupnea*min/hour. Two minor modifications were made to the published event-related ventilatory burden algorithm [14].
First, the average respiratory event depth under 90% eupnea was used instead of the average depth under 90% of the ensemble-averaged respiratory event ventilation signal. This modification was made because the original algorithm could not produce a value for ventilatory burden if the ensemble-averaged respiratory event ventilation signal could not be calculated. Second, the average event duration was calculated by averaging the duration of ventilation under 90% eupnea for each event instead of averaging the durations of entire scored events.
This modification made the depth and duration calculations consistent, i.e. both are based on ventilation under 90% eupnea. The combined cohort had a strong correlation between the original and modified event-related ventilatory burden algorithms (Pearson’s r = 0.99).
“Breath-related ventilatory burden” is the percentage of breaths during sleep under 50% eupnea, based on Parekh et al. [15]. Where possible, all breaths during sleep (excluding arousal breaths) were used to calculate breath-related ventilatory burden. In STAGES, all breaths during sleep (including arousals) were used because arousal scoring was unavailable.
In HomePAP home studies, all breaths during the study were used because sleep stages and arousals could not be scored. In BestAIR studies, breaths during arousals were included because the home studies had sleep and wake scoring but no arousal scoring.
Both ventilatory burden measures were quantified in all cohorts—even those with high-pass filtered and low SNR nasal airflow—because ventilation measurements are tolerant of noisy nasal airflow signals [41].
Calculating flow limitation
Using the nasal airflow signal, we used two validated algorithms to objectively assess the degree of flow limitation on a breath-by-breath basis (flow limitation frequency and severity). These methods are reported elsewhere [20]. Flow limitation frequency and severity were not calculated in ABC and HomePAP in-lab studies, which used high-pass filtered nasal airflow data, or BestAIR polysomnograms, which had a low nasal airflow SNR.
To quantify flow limitation severity in MESA, a model trained on 10 Hz flow-shape features was used [21]. For flow limitation frequency, the original flow limitation certainty model was used because using 10 Hz flow-shape features was not inferior to using 25 Hz flow-shape features [21].
Where possible, only breaths during sleep (excluding arousals) were used to calculate flow limitation frequency and severity. Arousal scoring was unavailable in the STAGES cohort polysomnograms, so flow limitation severity during all sleep epochs, including arousal breaths, was quantified instead. In STAGES studies, a published correction factor based on the estimated SNR was used to adjust flow limitation severity [22, 41].
Flow limitation frequency was not quantified in STAGES because no correction factor has yet been developed for this algorithm. Sleep stage and arousal scoring were unavailable in HomePAP home studies because electroencephalography was not recorded. Therefore, flow limitation for all breaths during the polysomnograms was quantified.
In the MESA cohort, there was a strong correlation between flow limitation frequency excluding arousal breaths and flow limitation frequency for all breaths (Pearson’s r = 0.94). Similarly, the correlation between flow limitation severity excluding arousal breaths and flow limitation severity for all breaths was high (r = 0.81).
Calculating hypoxemia severity
The oxygen saturation signal and scored respiratory events (algorithmically adjusted according to AASM recommended criteria) were used to automatically detect resulting desaturations and compute hypoxic burden [18]. The original method was modified to improve accuracy, as described elsewhere [44].
The oxygen saturation signal and scored desaturations (algorithmically adjusted using the ≥3% criterion) were used to calculate the desaturation severity parameter [19] and ODI. Desaturation severity and ODI were not quantified in the STAGES cohort because desaturation scoring was unavailable in the sleep studies provided by the NSRR.
Calculating the SASI
The SASI was calculated as described elsewhere [40] using baseline ESS, AHI, BMI, lowest oxygen saturation, and redundant pharyngeal mucosa. The three SASI categories are mild (I), moderate (II), and severe (III). The modified SASI uses tonsil size instead of redundant pharyngeal mucosa [45].
Both measurements were unavailable in all the cohorts analyzed in this study, so redundant pharyngeal mucosa was assumed absent when calculating the SASI. The SASI was not included as a predictor variable in multivariable statistical inference analyses because the SASI itself is a multivariable model.
Electroencephalography-based metrics
Electroencephalography-based metrics were not calculated because electroencephalography was not recorded in HomePAP and BestAIR home (Type 3) polysomnograms.
Data processing and statistical analyses
Data processing and statistical analyses were mainly performed in MATLAB (version R2024a, MathWorks, Natick, MA, United States). R (version 4.2.2, R Foundation, Vienna, Austria) was used for Box–Cox transformations (see below).
Missing data and data transformations
One STAGES participant was missing ESS data; one HomePAP in-lab participant and three MESA participants were missing BMI data; and four participants’ hypoxic burden could not be quantified. One HomePAP participant was missing data on nightly CPAP usage. Depression data was missing in three HomePAP home participants, three HomePAP in-lab participants, and four BestAIR participants.
All continuous variables were transformed to be normally distributed using the two-parameter Box–Cox [46] transformation. Transformed values were used in all subsequent analyses. Back-transformation was used to show values in original units of measurement.
Box–Cox transformation parameters were derived from all participants from all studies. The two-parameter Box–Cox transformation is \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} ${\mathrm{Y}}^{\prime }={\left(\mathrm{Y}+{\mathrm{\lambda}}_2\right)}^{\lambda_1}-1)\div{\mathrm{\lambda}}_1$\end{document} or \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} ${\mathrm{Y}}^{\prime }={\log}_{\mathrm{e}}\left(\mathrm{Y}+{\mathrm{\lambda}}_2\right)$\end{document} if \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} ${\mathrm{\lambda}}_1=0$\end{document}. Optimal values of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} ${\mathrm{\lambda}}_1$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} ${\mathrm{\lambda}}_2$\end{document} were estimated using the “boxcoxfit” function from the geoR package (version 1.9-2) of R [47, 48].
After transformation, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} ${\mathrm{Y}}^{\prime }$\end{document} was standardized between 0 and 1 by subtracting the minimum value and dividing by the range. For parameters such as TST, n = 2332 participants were used to calculate transformation parameters. For parameters such as the change in ESS, n = 213 participants were used because this variable could only be calculated in HomePAP, BestAIR, and ABC studies.
Cross-sectional analysis: association between baseline OSA severity measures and baseline ESS (statistical inference)
Associations between predictor variables and ESS were evaluated in 2332 participants across MESA and STAGES cohorts and HomePAP, BestAIR, and ABC studies (using their baseline ESS and diagnostic polysomnography before CPAP treatment). Linear regression models were adjusted for age, sex, race, BMI, and TST. The significance threshold for all statistical analyses was p < .05.
Response to CPAP intervention: association between baseline OSA severity measures and the change in ESS (statistical inference)
Relationships between baseline OSA severity measures and the change in ESS were evaluated using multivariable linear regression in the HomePAP home cohort (n = 64), noting that the polysomnograms from these studies had unfiltered and high SNR nasal airflow, which is needed to confidently calculate flow limitation metrics. Linear regression models were adjusted for age, sex, race, and baseline BMI (model 0); adjusted for baseline ESS (model 1); and further adjusted for age, sex, race, baseline BMI, TST, and average nightly CPAP use (model 2).
Development of machine learning regression and classification models to predict the change in ESS
Predictor variable selection
Cross-validated Least Absolute Shrinkage and Selection Operator (LASSO) selection of variables was performed using HomePAP home studies (n = 64) to determine the best combination of predictors of the change in ESS. LASSO is a regression method incorporating variable selection and regularization (i.e. sensible simplification to avoid overfitting). The “lasso” MATLAB function was used with alpha set to 1.
The optimal lambda value was found by running the “lasso” function with leave-one-out cross-validation (by setting the number of cross-validation folds to the sample size, i.e. 64) and using the lambda value with minimum cross-validation error. Then, leave-one-out cross-validation was performed using the optimal lambda value for the “Lambda” parameter and “resubstitution” for the “CV” parameter.
Training and evaluation of machine learning regression models
Data from HomePAP, BestAIR, and ABC studies (n = 139) were used to train a linear regression model to predict the change in ESS using the variables selected by LASSO. This model was tested in 74 holdout participants from these cohorts (rather than a single holdout cohort) to ensure homogeneity of sleepy and non-sleepy participants at baseline in both training and holdout datasets. Participants were ordered by identification number and alternately assigned to the training and holdout datasets; for example, the first participant was assigned to the training dataset, the second to the holdout dataset, and so on.
In exploratory analyses, we compared the predictive performance of alternative linear regression models for 1000 iterations of randomly assigning participants to either the training or holdout dataset. Charts showing predictions of the change in ESS were generated using the final linear regression models trained on all 213 HomePAP, BestAIR, and ABC participants.
Training and evaluation of machine learning classification models
The linear regression prediction models trained on the training dataset were used as classifiers to predict responders/non-responders in both the training and holdout datasets. The change in ESS cut-point used to differentiate responders from non-responders was -2 based on the minimum clinically important difference of ESS in OSA [23, 24]. In sensitivity analyses, this cut-point was varied from -1 to -9.
A receiver operating characteristic (ROC) curve was generated and the optimal responder probability threshold was determined by the ROC convex hull method [49]. False positives and negatives were given an equal penalty weight of 1 and true positives and true negatives were given a penalty weight of 0. The model and optimal threshold were applied in the holdout dataset.
In sensitivity analyses, false positives were given a penalty weight of 0.5 and false negatives were given a penalty weight of 1 to reflect the current clinical practice of overprescribing rather than under-prescribing CPAP [50].
Results
Participant characteristics
Participant characteristics are summarized in Table 1. Notably, OSA severity measures and ESS scores (baseline, follow-up, and change) exhibited substantial variability both within and between cohorts. Table S1 provides further details about the race of participants.
Figure 4 presents histograms for all participants across all cohorts showing distributions of key variables covering demographics (age, sex, and BMI), polysomnography-derived quantifications of sleep and OSA severity (AHI, ventilatory burden, flow limitation, hypoxemia severity, and the SASI [40]), and sleepiness (baseline ESS, follow-up ESS, and change in ESS). Scatter plots showing relationships between these variables are also presented in Figure 4. Box–Cox transformation parameters (λ1, λ2, minimum value, range, and sample size) for each variable are reported in Table S2 and can be applied to external datasets whose values fall within the ranges displayed in Figure 4.
Cross-sectional analysis: association between baseline OSA severity measures and baseline ESS (statistical inference)
In the combined cohort composed of 2332 participants (Figure 5), increased ESS was associated with increased breath-related ventilatory burden (0.60 points per standard deviation [SD]; 95% confidence interval [CI] 0.39 to 0.82; p < .001), event-related ventilatory burden (0.56; 0.34 to 0.78; p < .001), hypoxic burden (0.51; 0.28 to 0.74; p < .001), AHI (0.45; 0.23 to 0.68; p < .001), T90 (0.36; 0.15 to 0.57; p < .001), and flow limitation severity (0.31; 0.10 to 0.53; p = .004).
Response to CPAP intervention: association between baseline OSA severity measures and the change in ESS (statistical inference)
Multivariable linear regression (Figure 6) revealed that the change in ESS was most strongly associated with baseline ESS (-2.75 points change per SD; 95% CI -3.83 to -1.69; p < .001) and baseline event-related ventilatory burden (-1.08; -2.13 to -0.06; p = .039). The association with AHI was weaker (-0.97; -2.01 to 0.05; p = .062) and not statistically significant. Event-related ventilatory burden was significantly associated with the change in ESS after adjusting for age, sex, race, and baseline BMI (-1.35; -2.69 to -0.04; p = .044), and not significant after further adjusting for TST, baseline ESS, and average nightly CPAP use (-0.86; -2.01 to 0.26; p = .128).
The AHI (-0.87; -1.97 to 0.22; p = .115) and hypoxic burden (-0.86; -2.01 to 0.26; p = .130) had similar effect sizes and were also not statistically significant. None of the covariates (age, sex, race, baseline BMI, TST, and average nightly CPAP use) were significantly associated with the change in ESS (Figure S1).
Machine learning regression and classification models to predict the change in ESS
Predictor variable selection
Variable selection was performed in a subset of the training dataset (HomePAP home studies, n = 64). Baseline event-related ventilatory burden and baseline ESS were the only two variables selected to predict the change in ESS using cross-validated LASSO (Figure S2). Cross-validated stepwise linear regression was also conducted as a secondary analysis, and results were consistent, i.e. baseline event-related ventilatory burden and baseline ESS were the most frequently selected variables (Table S3).
Machine learning regression models to predict the change in ESS
The linear regression model developed using baseline event-related ventilatory burden and baseline ESS provided an adjusted R2 (between predicted change in ESS and actual change in ESS) of 0.484 in the training dataset and 0.313 in the holdout dataset (Figure 7). Bland–Altman plots showed no heteroscedasticity or bias, and CPAP treatment duration had no obvious effect. In exploratory analyses involving 1000 iterations of randomized training/holdout assignment, the mean (SD) adjusted R2 was 0.450 (0.027) in the training dataset and 0.409 (0.080) in the holdout dataset (Table 2).
Using baseline hypoxic burden rather than baseline event-related ventilatory burden provided an adjusted R2 of 0.401 (0.081) in the holdout dataset; using baseline AHI rather than baseline event-related ventilatory burden provided a value of 0.397 (0.084); using baseline ESS alone provided a value of 0.370 (0.090); and using the SASI provided a value of 0.240 (0.066). Charts showing predicted change in ESS from baseline ESS and either baseline ventilatory burden or AHI were developed using linear regression models trained on all HomePAP, BestAIR, and ABC participants (Figure 8).
Machine learning classification models to predict the change in ESS
After defining an optimal threshold using a ROC curve (Figure S3), the linear regression model change in ESS = baseline ESS + baseline event-related ventilatory burden + intercept provided a dichotomous prediction of ≥2 points ESS improvement [23, 24]. In the holdout dataset, predicted responders exhibited a median (IQR) change in ESS of -5.0 (-10.0 to -2.0), which was significantly different from predicted non-responders (0.0 [-1.0 to 1.0], p < .001, Figure 9). The classifier had 91.5% sensitivity, 59.3% specificity, and 79.7% accuracy.
In sensitivity analyses, similar performance was observed when the change in ESS responder/non-responder threshold was varied from -1 to -9 (Table 3). In exploratory analyses involving 1000 iterations of randomly assigning participants to either the training or holdout dataset, the mean (SD) accuracy was 84.7% (1.7%) in the training dataset and 78.1% (3.5%) in the holdout dataset (Table 4). Using baseline AHI rather than baseline event-related ventilatory burden had an accuracy of 77.3% (3.4%) in the holdout dataset; using baseline hypoxic burden rather than baseline event-related ventilatory burden had an accuracy of 76.9% (3.2%); using baseline ESS alone had an accuracy of 76.7% (3.2%); using the SASI had an accuracy of 68.8% (5.5%); and allocating CPAP based on typical current clinical guidelines had an accuracy of 68.4% (3.7%).
Higher sensitivities, lower specificities, and lower accuracies were observed when false positives had half the penalty weight of false negatives (Figures S4 and S5 and Tables S4 and S5).
Discussion
This study aimed to predict the change in ESS with CPAP treatment using baseline characteristics. We hypothesized that statistical inference would reveal that physiology-based measures of OSA severity are associated with both baseline ESS and its change after treatment, independent of potential confounders and mediators. In support of this hypothesis, baseline ESS was statistically associated with two measures of ventilatory burden, hypoxic burden, AHI (consistent with Budhiraja et al. [53]), T90, and flow limitation severity.
Furthermore, the change in ESS after treatment was most strongly associated with baseline ESS and baseline event-related ventilatory burden (i.e. cumulatively lost ventilation per hour during sleep). Our second hypothesis was that the change in ESS can be predicted from baseline characteristics using machine learning models, and that models using physiology-based metrics would outperform models using conventional metrics such as the AHI. Supporting this, a machine learning regression prediction model using baseline ventilatory burden and ESS showed a modestly higher correlation with observed ESS changes compared to models using hypoxic burden and ESS, baseline AHI and ESS, baseline ESS only, and the SASI.
Finally, we hypothesized that predicted change in ESS can be used to stratify responders from non-responders using machine learning classification models. The model using baseline ventilatory burden and baseline ESS distinguished responders (≥2-point ESS improvement) from non-responders with high sensitivity, low specificity, and greater accuracy than current clinical guidelines. To facilitate clinical use, we developed a chart showing the predicted ESS change using baseline ventilatory burden and baseline ESS.
An alternative chart using the AHI instead of ventilatory burden has slightly reduced performance but improves immediate clinical applicability.
Mechanistic insights
We found that event-related ventilatory burden was associated with baseline ESS and CPAP treatment-related change in ESS. Ventilatory burden may cause EDS via two pathways. First, lost ventilation during respiratory events may cause hypercapnia and respiratory acidosis leading to daytime sleepiness [54, 55].
Indeed, patients with OSA treated with CPAP showed significant decreases in wake pCO2 and daytime sleepiness [56]. Second, lost ventilation can lead to intermittent hypoxemia and hypoxia, leading to oxidative injury, changes in neuronal connectedness, and increased inflammation in wake-promoting regions of the brain and brainstem [57]. We observed that hypoxic burden was associated with baseline ESS and the change in ESS, supporting this.
However, the effect size of ventilatory burden was slightly larger than that of hypoxic burden, possibly because ventilatory burden captures hypoxemia and hypercapnia aspects of OSA. Longitudinal studies are required to confirm whether this relationship holds true for long-term clinical sequelae and patient outcomes.
To our knowledge, this is the first study that has explicitly attempted to develop a machine learning model to predict the change in ESS using baseline characteristics (including with standard clinical metrics such as the AHI). However, several studies have examined the statistical association between baseline characteristics and the change in ESS following CPAP treatment. Bhat et al. reported significant correlations between the change in ESS and baseline AHI and T90 in severe OSA [9].
However, their study included only adherent patients (≥4 hours CPAP use/night for 60% of nights) and those who were adequately treated (residual AHI ≤5 events/hour). Our research results (including all individuals regardless of adherence or residual AHI) align with theirs, i.e. higher baseline OSA severity (measured using ventilatory burden) was associated with greater ESS improvement. In contrast, Kingshott et al. found no significant correlation between the change in ESS and baseline AHI, arousal index, ODI, and minimum oxygen saturation in severe OSA [10].
However, they discovered that CPAP use (hours/night) was associated with the change in ESS. Similarly, Otsuka et al. showed that baseline AHI was not a predictor of the change in ESS, whereas CPAP use was [12]. Cistulli et al. demonstrated a dose–response relationship between CPAP use and improvement in self-reported sleepiness [58].
Although not statistically significant, greater CPAP use was associated with larger improvements in ESS scores in our analysis of the HomePAP home cohort (Figure S1, n = 63). Similarly, an analysis of the entire HomePAP cohort (n = 119) found that CPAP use was associated with the change in ESS, but not after adjusting for baseline AHI, and that higher baseline AHI was associated with ESS normalization [50]. Therefore, it is possible that the HomePAP cohort is underpowered to detect a statistically significant association between CPAP use and the change in ESS.
In support of this, Bonsignore et al. showed in a larger cohort that CPAP use was weakly correlated with the change in ESS (R2 = 0.023, n = 2190) [59]. Drawing on previous studies and our findings, there is substantial evidence that baseline disease severity, baseline sleepiness, and treatment adherence are key factors influencing improvements in ESS among patients with OSA receiving CPAP therapy. However, since adherence is not known at baseline, it cannot be used to predict the change in ESS prior to CPAP initiation.
We also note that in secondary exploratory analyses using the the percentage change in ESS as the outcome measure instead of change in ESS, the strength in associations and prediction accuracy was similar or weaker to the results of the primary analysis (Results S2, Figure S6–S7, Table S6).
We observed that flow limitation was a significant predictor of ESS in the cross-sectional analysis, consistent with earlier studies [20, 21, 60]. However, flow limitation severity and frequency were not significant predictors of the change in ESS in the HomePAP home cohort. The explanation for this may lie with the severity of OSA.
Flow limitation appears to be most strongly associated with sleepiness in patients without severe OSA [21], and the HomePAP trial participants had relatively severe OSA. Flow limitation may therefore fail to adequately summarize the severity of breathing disturbance in more severe OSA. Nonetheless, in the absence of more severe OSA, flow limitation appears to contribute to OSA-related symptoms.
In particular, studies have demonstrated that sustained inspiratory flow limitation from suboptimal CPAP pressure was associated with decreased vigilance [61], and that the elimination of residual flow limitation with CPAP improves vigilance [62]. However, the relationship between flow limitation and sleepiness and vigilance is likely heavily influenced by patient comorbidities. Chronic conditions frequently co-occurring with severe OSA (such as cardiovascular disease, metabolic syndrome, insomnia, and depression) can confound these relationships [63, 64].
Thus, these comorbidities should be considered in future studies.
Clinical implications
The AASM International Classification of Sleep Disorders states that to diagnose adult OSA, a patient must have either AHI ≥15 events/hour or AHI ≥5 events/hour and one or more symptoms, e.g. sleepiness [51]. AASM guidelines for the treatment of adult OSA recommend that clinicians use CPAP to treat OSA in adults experiencing EDS [52]. The ESS is the most common measure of sleepiness in OSA in the clinic, with scores >10 indicating EDS [3].
As such, this guidance suggests clinicians should prescribe CPAP to patients with AHI ≥5 and ESS >10. However, we discovered that the change in ESS following CPAP treatment is modulated independently by baseline ventilatory burden and baseline ESS. From our chart (Figure 8), there may be patients with baseline ESS ≤10 points who could have a clinically meaningful improvement in sleepiness (≥2-point ESS improvement [23, 24]) with CPAP treatment, but would not be prescribed CPAP according to clinical guidelines.
For instance, a patient with baseline ESS = 8 points, ventilatory burden = 400%eupnea*min/hour, and AHI = 10 events/hour is expected to achieve a 2-point ESS improvement. Conversely, there may be patients with AHI ≥5 and ESS >10 who are prescribed CPAP but do not have a meaningful improvement in sleepiness due to low ventilatory burden. For example, a patient with baseline ESS = 11 points, ventilatory burden = 20%eupnea*min/hour, and AHI = 10 events/hour is expected to have a 1-point ESS improvement.
When used as a classifier to predict responders/non-responders, the model based on baseline ventilatory burden and baseline ESS had 78.1% (3.5%) average (SD) accuracy, whereas current clinical guidelines had 68.4% (3.7%). Therefore, our modeling approach shows potential for prescribing CPAP for sleepiness more judiciously than existing clinical guidelines.
Our study has addressed several key considerations recommended in the development of a clinically useful prediction model [65]. In particular, our model addresses a clear clinical decision point: whether to prescribe CPAP for sleepiness in OSA. The model outputs the expected change in ESS with confidence intervals, and our chart helps clinicians make this decision.
The model’s input parameters are baseline ESS and baseline event-related ventilatory burden, which is calculable from routine polysomnograms using publicly available code [14]. Calculating all OSA severity measures from a European Data Format file takes approximately 10 minutes per patient on a standard desktop computer. Once baseline ventilatory burden and ESS are known, it takes less than a second to predict the change in ESS or whether a patient is expected to be a responder.
While novel measures of OSA severity (such as ventilatory burden, flow limitation, and hypoxic burden) are not currently calculated in the majority of clinical data collection and reporting systems, it is technically and practically feasible for them to be integrated into these systems. Since these metrics are derived from standard polysomnography signals, these algorithms could be incorporated into existing software to calculate these metrics without requiring any other changes to standard clinical data acquisition. In the interim, the AHI can be used in place of ventilatory burden (with a modest loss of performance) when ventilatory burden is unfeasible to calculate, and this is immediately clinically translatable.
In addition to predicting continuous change in ESS, our model can identify likely responders/non-responders with high sensitivity (91.5%) and low specificity (59.3%), reflecting the current clinical practice of prescribing CPAP for sleepiness in OSA [50, 52]. We showed that 79.6% of predicted responders were actual responders (positive predictive value, Figure 9D); and if only predicted responders were treated with CPAP, there would have been a 44.2% reduction in the non-response rate compared to treating all participants. Clinical application of this predictive model would provide better information about the likelihood of the intervention improving sleepiness; and therefore assist clinicians and patients to make more judicious decisions about whether to adopt CPAP treatment.
The findings from our study suggest that event-related ventilatory burden is a promising alternative to the AHI as a measure of OSA severity. Ventilatory burden is the product of average respiratory event depth, average event duration, and event frequency (Figure 1). Thus, for interpreting the effects of OSA on OSA-related symptoms, respiratory event depth and duration likely provide additional insight beyond event frequency.
Furthermore, ventilatory burden may be a better disease severity measure of OSA than the AHI, evidenced by the multiple complementary analyses in this study showing that ventilatory burden has (1) a cross-sectional association with sleepiness, (2) an association with the treatment-related improvement in sleepiness, and (3) demonstrated ability to predict the improvement in sleepiness when used as an input to machine learning models.
Our results may have a relevant clinical implication for assessing EDS after CPAP therapy. It has been suggested that if an adherent patient still has EDS after CPAP therapy, several steps should be taken to diagnose the issue [6]. Specifically, polysomnography should be performed while on CPAP; checks for depression, medication, narcolepsy and sleep hygiene should be made; and wake promoting drugs should be considered.
Therefore, if a patient initially predicted to experience reduced sleepiness does not improve after CPAP treatment, these steps can guide further diagnosis and therapy.
Methodological considerations
This study has several limitations. First, the ESS is a subjective measure of sleepiness. Therefore, replication of our findings using objective measures of sleepiness such as the maintenance of wakefulness test (MWT) and multiple sleep latency test (MSLT) is warranted.
However, it should be noted that Sun et al. observed that functional brain changes following CPAP treatment could be identified by grouping participants based on ESS but not MSLT [66]. Second, baseline ventilatory burden and baseline ESS could not fully account for all the variance in the change in ESS with CPAP treatment. Although CPAP alleviates upper airway obstruction, patients may continue to experience EDS for reasons unrelated to OSA, including lifestyle factors (e.g. insufficient sleep duration), comorbidities (e.g. depression and other sleep or circadian disorders), medication effects, and genetic or epigenetic influences [57].
Furthermore, heterogeneity in OSA-related neuronal damage due to long-term exposure to intermittent hypoxia [67] may contribute to the challenge of fully predicting the change in ESS (i.e. some changes may be irreversible, or not able to be improved during the treatment period in the studies analyzed here). Third, the prediction of the change in ESS may have been more precise if treatment duration had been consistent between and within studies. While CPAP treatment duration did not have an obvious effect on predictions (Figure 7), larger studies with greater consistency in treatment duration would likely assist in developing more accurate prediction models.
Fourth, we did not quantify sleep fragmentation or arousal parameters because electroencephalography was not recorded in the HomePAP and BestAIR home polysomnograms, which were essential for this study. However, previous research in a cross-sectional cohort shows that arousal severity was less strongly associated with ESS compared to flow limitation, hypoxemia, and AHI [20]. Finally, no control group was included in this analysis.
Patient expectations regarding CPAP benefits can influence the change in ESS, although this placebo effect is estimated to be approximately -1.2 points [68], which is less than the minimum clinically important difference of ESS in OSA (-2 points). Therefore, it is unlikely that many participants would have been CPAP responders based on the placebo effect alone.
To address data limitations and the known effects of depression, insomnia, and arousals on EDS in OSA [53, 69, 70], we conducted secondary exploratory analyses on subsets of participants. We found that adding insomnia as a covariate did not alter statistical inferences regarding baseline ESS, and incorporating it into the clinical definition of OSA failed to improve the guideline’s ability to predict ESS changes (Results S2, Figure S8 and Table S7). Similarly, while depression was independently associated with the change in the ESS, its inclusion did not improve overall prediction model performance (Results S4, Figures S9–S13 and Tables S8–S10).
Finally, neither arousal index nor arousal burden [71] were associated with baseline ESS, and incorporating these arousal parameters into our regression and classification models yielded poor predictive performance (Results S5, Figure S14 and Tables S11–S12).
Conclusion
This is the first study to rigorously predict CPAP-related improvement in sleepiness in patients with OSA using baseline characteristics. Baseline event-related ventilatory burden and baseline ESS were independently associated with the change in ESS and could be used together to inform clinicians whether CPAP treatment will likely improve a patient’s sleepiness. Alternatively, baseline AHI and baseline ESS could be used with a modest loss of predictive performance.
Supplementary Material