In follow up to my post (#21 above) and all of the great comments that followed I thought I'd do a quick internet search (by no means intended to be exhaustive) on what research has been done on the question of the accuracy of APAP AHI scoring versus sleep study (PSG). I found two studies that seem to be on point.
Accuracy of Auto-Titrating CPAP to Estimate the Residual Apnea-Hypopnea Index in Patients
with Obstructive Sleep Apnea on Treatment with Auto-Titrating CPAP.
Himanshu Desai M.D.1, Anil Patel M.D.2, Pinal Patel M.B.B.S.1, Brydon J.B. Grant M.D.1 and M. Jeffery Mador M.D.1, 3.
"Bland and Altman plots demonstrate that the difference between auto-CPAP AHI and PSG AHI was not uniform with auto-CPAP overestimating the AHI at lower values of AHI and underestimating the AHI at higher values of AHI." (My quotes added)
The second study: (I have excerpted a few paragraphs. You should read the whole thing when you have time.)
Volume 2013 (2013), Article ID 314589, 6 pages
Accuracy of Positive Airway Pressure Device—Measured Apneas and Hypopneas: Role in Treatment Followup
Carl Stepnowsky,1,2 Tania Zamora,1 Robert Barker,3 Lin Liu,4 and Kathleen Sarmiento2,3
1Health Services Research & Development Unit, Veterans Affairs San Diego Healthcare System, San Diego, CA 92161, USA
2Department of Medicine, University of California, San Diego, CA 92037, USA
3Pulmonary Service, Veterans Affairs San Diego Healthcare System, San Diego, CA 92161, USA
4Department of Family and Preventive Medicine, University of California, San Diego, CA 92037, USA
Received 30 April 2013; Revised 16 July 2013; Accepted 19 July 2013
Academic Editor: Giora Pillar
Copyright © 2013 Carl Stepnowsky et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Improved data transmission technologies have facilitated data collected from positive airway pressure (PAP) devices in the home environment. Although clinicians’ treatment decisions increasingly rely on autoscoring of respiratory events by the PAP device, few studies have specifically examined the accuracy of autoscored respiratory events in the home environment in ongoing PAP use. “PAP efficacy” studies were conducted in which participants wore PAP simultaneously with an Embletta sleep system (Embla, Inc., Broomfield, CO), which was directly connected to the ResMed AutoSet S8 (ResMed, Inc., San Diego, CA) via a specialized cable. Mean PAP-scored Apnea-Hypopnea Index (AHI) was 14.2 ± 11.8 (median: 11.7; range: 3.9–46.3) and mean manual-scored AHI was 9.4 ± 10.2 (median: 7.7; range: 1.2–39.3). Ratios between the mean indices were calculated. PAP-scored HI was 2.0 times higher than the manual-scored HI. PAP-scored AHI was 1.5 times higher than the manual-scored AHI, and PAP-scored AI was 1.04 of manual-scored AI. In this sample, PAP-scored HI was on average double the manual-scored HI. Given the importance of PAP efficacy data in tracking treatment progress, it is important to recognize the possible bias of PAP algorithms in overreporting hypopneas. The most likely cause of this discrepancy is the use of desaturations in manual hypopnea scoring.
Given the improved PAP data transmission technologies and resultant increased use of these data, we sought to investigate the accuracy of the PAP-measured AHI. We had the opportunity to conduct “PAP efficacy” studies in which participants wore PAP devices simultaneously with Type III cardiopulmonary recording equipment. Therefore, the goal of the present study was to specifically examine the accuracy of the identification of apneas and hypopneas by the PAP device.
2.2. Equipment Used
The Embletta (Embla, Inc., Broomfield, CO) was directly connected to the ResMed AutoSet S8 (ResMed, Inc., San Diego, CA) via a specialized cable that allowed for the direct recording of S8 data. Signals recorded include oximetry, chest effort, and body position. Airflow from the PAP device was used for scoring. RemLogic software was used for manual respiratory scoring. Apneas and hypopneas were manually scored according to the 2007 American Academy of Sleep Medicine guidelines, which included defining a hypopnea as being associated with a ≥4% oxygen desaturation . AutoSet respiratory events were autoscored by the device, and summary statistics were obtained within RemLogic. Manual scoring was blind to the AutoSet-scored respiratory events.
In this study of home-based PAP efficacy, as measured by the S8 APAP device, the PAP-scored HI was on average more than double the manual-scored HI. Given the importance of PAP efficacy data in tracking treatment progress, it is important to recognize that this particular APAP device may overscore hypopneas. The most likely causes of this discrepancy are (a) the use of a proprietary algorithm and (b) the use of desaturations in manual hypopnea scoring. Because the number of apneas was underscored relative to manual scoring, the overall AHI does not appear to be different from manual scoring. This study and the evolving literature in this area suggest that it is important to understand how a specific PAP device identifies both apneas and hypopneas.
In summary, PAP devices have automated, proprietary algorithms for respiratory event detection. When event detection scoring is combined with PAP use duration in the denominator, a proxy AHI value is derived. Given the increased reliance on the PAP-scored events by both providers and patients, it is important to better understand the nuances of specific algorithms and how the PAP-scored AHI, HI, and AI values compare to those same values from manual scoring. Doing so is an important step toward making more informed treatment decisions.