Very Poor Quality of OSA Research and Almost Non-existent Specific CPAP Testing
In 2019 I submitted a thread regarding an inherent risk of the Resmed Airfit P10 entitled,
![[Image: paperclip.png]](https://www.apneaboard.com/forums/images/focus/paperclip.png)
EVIDENCE OF AIRFIT P10 VENT OBSTRUCTION CAUSING HYPERCAPNIA
In it I mentioned that there was really no real clinical premarketing testing of the Airfit P10 by the FDA.
Currently, I am doing some research on the potential association between OSA and the risk for Neurodegeneration/Dementia. This exploration has led me to a very extensive assessment from the Agency for Healthcare Research and Quality (AHRQ), commissioned by Medicare. This study casts doubt on the quality of the cummulative research behind the diagnosis, diagnostic procedures, CPAP compliance and therapeutic efficacy in treating OSA and comorbid medical conditions.
"Long-Term Health Outcomes in Obstructive Sleep Apnea: A Systematic Review of Comparative Studies Evaluating Positive Airway Pressure and the Validity of Breathing Measures as Surrogate Outcomes"
https://www.ahrq.gov/sites/default/files...report.pdf
A Scathing Criticism of Thirty Years of OSA Research
The following is the paper's conclusions:
Conclusions The studies are highly inconsistent in how they define breathing measures (such as AHI and ODI) and, by extension, how they define OSA, despite statements of following standard sleep study criteria. Reporting of specific diagnosis and sleep study criteria is generally poor.
RCTs have not found significant effects of CPAP on long-term health outcomes for adults with OSA. RCTs of long-term outcomes do not provide evidence that CPAP has an effect on all cause mortality, stroke, acute myocardial infarction, composite CV outcomes, accidents, incident diabetes (all with low SoE), and on clinically significant improvements in (unvalidated) measures of depression and anxiety symptoms, cognitive function, or quality of life (all with low SoE). The effect of CPAP on other long-term health outcomes from RCTs is unclear due to insufficient evidence related to sparse studies and/or highly imprecise estimates. Inclusion of information from adjusted NRCSs mostly does not alter these conclusions, except that the NRCSs support an association between CPAP use and lower risk of all-cause death with longer term follow up. Underpowered studies have contributed to some of this evidentiary weakness. RCTs have mostly been powered for composite CV events, however the definitions of this outcome is highly variable across studies. Clinical equipoise remains. Future high quality, long term studies that are adequately powered for clinical endpoints are needed.
A small number of studies comparing CPAP with mandibular advancement devices provided low SoE that (unvalidated) measures of depression and anxiety symptoms are unaffected by either OSA treatment. There is insufficient or no evidence regarding the relative effect of CPAP versus mandibular advancement devices on other outcomes. There is no evidence regarding comparisons of CPAP with other (nonsurgical, nonpharmacologic) interventions. There is low strength of evidence that functional status remains similar between patients prescribed either fixed CPAP or autoCPAP. Other long-term health outcomes have not been reported.
Across studies, there is insufficient evidence to evaluate which, if any, patients have greater benefits with any given treatment.
No studies have evaluated whether breathing measures or sleepiness symptom measures may be valid surrogate or mediator measures for long-term health outcomes. Across studies, there was insufficient evidence to support whether changes in intermediate or surrogate measures are correlated with health outcomes.
Future well-conducted, well-reported studies are needed to allow more definitive conclusions regarding the clinical effect of CPAP for adults with OSA, to determine who might most benefit from long-term CPAP treatment, and to evaluate the validity of intermediate measures as potential surrogate or mediator measures of long-term health outcomes.
(ODI=Oxygen Desaturation Index; RCT=Randomized Clinical Trials; NRCS=Nonrandomized Comparative Study; SoE=Strength of Evidence)
A rather frantic letter endorsed by 21 professional and patient societies and comments by other stakeholders during the public comment period criticized the draft version of the AHRQ report for failing to clearly acknowledge the established therapeutic benefits of the treatment of OSA for excessive sleepiness, sleep-related quality of life, motor vehicle accidents, and blood pressure.
https://aasm.org/wp-content/uploads/2021...letter.pdf
Multisociety response to the Medicare Report:
"Long-term health outcomes for patients with obstructive sleep apnea: placing the Agency for Healthcare Research and Quality report in context—a multisociety commentary"
https://jcsm.aasm.org/doi/10.5664/jcsm.10832
Almost Nonexistant CPAP Testing
The report also refered to the almost complete lack of testing of CPAPs by the industry. As I had mentioned in my earlier thread, the industry uses a gaping 510(k) loophole in the FDA premarketing approval process that says a device maker does not need to submit new workbench testing if they claim the new device is ostensively the same as a preexisting device. Of the 143 CPAP devices with available records for review, only four devices had actual testing data, all the others were cleared for sale based on a series of references to other surrogate devices that ultimately ended back to these four devices.
CPAP Device Premarket Notification Clearances
Numerous CPAP devices are available on the market. According to Section 510(k) of the Food, Drug and Cosmetic Act, device manufacturers must provide a Premarket Notification to the FDA of their intent to market a medical device at least 90 days in advance. This allows the FDA to determine whether the device is equivalent to an already-cleared device. Specifically, “medical device manufacturers are required to submit a premarket notification if they intend to introduce a device into commercial distribution for the first time or reintroduce a device that will be significantly changed or modified to the extent that its safety or effectiveness could be affected. Such change or modification could relate to the design, material, chemical composition, energy source, manufacturing process, or intended use.”81 Upon premarket notification by manufacturers, the FDA provides devices “clearance” upon review of the 510(k) submission. After rigorous review of the devices, the FDA determines whether to grant “approval” for the device to be legally marketed in the U.S. In addition, devices may be granted a “de novo” categorization for novel medical devices for which there is no legally marketed predicate (previously approved) device.
We examined the Premarket Notification [Section 510(k)] records on CPAP devices to determine whether new versions of CPAP devices were cleared (and/or approved or granted de novo categorization) on the basis of new clinical research or on the basis of substantial equivalence to an already-cleared device.
We searched the Devices@FDA database82 without date restrictions (last search on March 1, 2020) using the terms “sleep apnea,” “OSA,” “continuous positive airway pressure,” and “CPAP;” the names of major manufacturers (“Phillips-Respironics”, “ResMed”, “Simons”, “Fisher & Paykel”); and product codes for non-continuous ventilator devices (“BZD”); and additional codes listed on the returned records (“LRK”, “MNQ”). We identified 812 unique Premarket Notification records, of which 163 referred to CPAP devices used for treating sleep apnea in adults. Of these, 143 records were approved in 1996 or later and had downloadable documents; we analyze these. The remaining 20 records without downloadable documents were approved/cleared between 1976 and 1994.
For the 163 510(k) Premarket Notification records on CPAP devices, we extracted an identification number, the device name and manufacturer, and the approval date (see Appendix Table D-4). From the 143 that had downloadable documents we also extracted whether a clinical study was mentioned and whether they cited another approval document to claim equivalence with an approved device. Upon examination of the publicly available data in FDA database, it was apparent that we did not have access to the complete Premarket Notification records. Publicly available data via Devices@FDA database included only: basic device and manufacturer information provided by the submitter; predicate device(s); reason for submission; submitted information about substantial equivalence with previously cleared devices, intended use, and device description; and the clearance letter written by the FDA. The available records do not include explicit references to any clinical studies that may have supported the claims. There was also no explicit reporting of, or reference to, any new (unpublished) clinical data. However, such information might exist in other inaccessible parts of the Premarket Notification records.
The large majority of Premarket Notification records for CPAP devices cite other previously approved/cleared device records to support claims equivalence. We evaluated the more-recent 143 CPAP device records with available data. The 143 CPAP device records also cited 78 records of CPAP accessories, which are included in the analysis. 21 We found a total of 266 citation relationships. Figure 1 graphs the citation relationships reported in the 143 device records. The figure demonstrates 13 groups of records that are connected with citation relationships. The largest relationship group includes 169 device records. The other relationship groups include many fewer records: 3 groups have between 9 and 12 records, and nine groups have between 1 and 4 records.
Among the 143 CPAP device records, four (large red dots in Figure 1) did not cite other records but they were each cited by at least one other record. These include: Apex Medical XT1 CPAP model 9S-005 (document number K070609, approved in 2007), Apex Medical CPAP RT 21XX (document number K022650, approved in 2004), Orion Nasal CPAP System (Bird Products Corp., document number K020730, approved in 2002), and Sullivan Autoset Portable II Nasal CPAP System (ResMed, Ltd., document number K970771, approved in 1997).
Half of the device records (n = 76, empty circles) both cited and were cited by other records. Many device records (n = 61, blue circles) only cited other device records. Two device records neither cited nor were cited by other available device records (gray circles); they were both approved based on predicate devices marketed prior to 1976.
Figure 1. Citations between records to support
Shown are citation relationships (266 grey arrows) between 143 continuous positive airway pressure (CPAP) device records (large circles) and 78 CPAP accessory records (small circles). The arrows point from the record being cited to the citing record, to denote the “flow of information” in an equivalence claim. The 4 CPAP records shown as large red dots (“CPAP, parent”) are cited by other CPAP device records but do not cite other device records themselves. The 61 records shown as large blue dots (“CPAP, terminal”) are CPAP device records that cite other records of CPAP devices (large circles) or CPAP accessories (small circles). Two CPAP device records (large gray circles) are “standalone”, in that they are not cited by other devices nor citing other devices. The remaining 76 CPAP records are shown as large empty circles and are part of a chain of citation relationships (“CPAP, in chain”).
So, virtually no current CPAPs had been specifically assessed by the FDA before they were permitted to be marketed.
With the bar set so low, maybe not that surprising something like the Phillips Respironics debacle could happen.
EVIDENCE OF AIRFIT P10 VENT OBSTRUCTION CAUSING HYPERCAPNIA
In it I mentioned that there was really no real clinical premarketing testing of the Airfit P10 by the FDA.
Currently, I am doing some research on the potential association between OSA and the risk for Neurodegeneration/Dementia. This exploration has led me to a very extensive assessment from the Agency for Healthcare Research and Quality (AHRQ), commissioned by Medicare. This study casts doubt on the quality of the cummulative research behind the diagnosis, diagnostic procedures, CPAP compliance and therapeutic efficacy in treating OSA and comorbid medical conditions.
"Long-Term Health Outcomes in Obstructive Sleep Apnea: A Systematic Review of Comparative Studies Evaluating Positive Airway Pressure and the Validity of Breathing Measures as Surrogate Outcomes"
https://www.ahrq.gov/sites/default/files...report.pdf
A Scathing Criticism of Thirty Years of OSA Research
The following is the paper's conclusions:
Conclusions The studies are highly inconsistent in how they define breathing measures (such as AHI and ODI) and, by extension, how they define OSA, despite statements of following standard sleep study criteria. Reporting of specific diagnosis and sleep study criteria is generally poor.
RCTs have not found significant effects of CPAP on long-term health outcomes for adults with OSA. RCTs of long-term outcomes do not provide evidence that CPAP has an effect on all cause mortality, stroke, acute myocardial infarction, composite CV outcomes, accidents, incident diabetes (all with low SoE), and on clinically significant improvements in (unvalidated) measures of depression and anxiety symptoms, cognitive function, or quality of life (all with low SoE). The effect of CPAP on other long-term health outcomes from RCTs is unclear due to insufficient evidence related to sparse studies and/or highly imprecise estimates. Inclusion of information from adjusted NRCSs mostly does not alter these conclusions, except that the NRCSs support an association between CPAP use and lower risk of all-cause death with longer term follow up. Underpowered studies have contributed to some of this evidentiary weakness. RCTs have mostly been powered for composite CV events, however the definitions of this outcome is highly variable across studies. Clinical equipoise remains. Future high quality, long term studies that are adequately powered for clinical endpoints are needed.
A small number of studies comparing CPAP with mandibular advancement devices provided low SoE that (unvalidated) measures of depression and anxiety symptoms are unaffected by either OSA treatment. There is insufficient or no evidence regarding the relative effect of CPAP versus mandibular advancement devices on other outcomes. There is no evidence regarding comparisons of CPAP with other (nonsurgical, nonpharmacologic) interventions. There is low strength of evidence that functional status remains similar between patients prescribed either fixed CPAP or autoCPAP. Other long-term health outcomes have not been reported.
Across studies, there is insufficient evidence to evaluate which, if any, patients have greater benefits with any given treatment.
No studies have evaluated whether breathing measures or sleepiness symptom measures may be valid surrogate or mediator measures for long-term health outcomes. Across studies, there was insufficient evidence to support whether changes in intermediate or surrogate measures are correlated with health outcomes.
Future well-conducted, well-reported studies are needed to allow more definitive conclusions regarding the clinical effect of CPAP for adults with OSA, to determine who might most benefit from long-term CPAP treatment, and to evaluate the validity of intermediate measures as potential surrogate or mediator measures of long-term health outcomes.
(ODI=Oxygen Desaturation Index; RCT=Randomized Clinical Trials; NRCS=Nonrandomized Comparative Study; SoE=Strength of Evidence)
A rather frantic letter endorsed by 21 professional and patient societies and comments by other stakeholders during the public comment period criticized the draft version of the AHRQ report for failing to clearly acknowledge the established therapeutic benefits of the treatment of OSA for excessive sleepiness, sleep-related quality of life, motor vehicle accidents, and blood pressure.
https://aasm.org/wp-content/uploads/2021...letter.pdf
Multisociety response to the Medicare Report:
"Long-term health outcomes for patients with obstructive sleep apnea: placing the Agency for Healthcare Research and Quality report in context—a multisociety commentary"
https://jcsm.aasm.org/doi/10.5664/jcsm.10832
Almost Nonexistant CPAP Testing
The report also refered to the almost complete lack of testing of CPAPs by the industry. As I had mentioned in my earlier thread, the industry uses a gaping 510(k) loophole in the FDA premarketing approval process that says a device maker does not need to submit new workbench testing if they claim the new device is ostensively the same as a preexisting device. Of the 143 CPAP devices with available records for review, only four devices had actual testing data, all the others were cleared for sale based on a series of references to other surrogate devices that ultimately ended back to these four devices.
CPAP Device Premarket Notification Clearances
Numerous CPAP devices are available on the market. According to Section 510(k) of the Food, Drug and Cosmetic Act, device manufacturers must provide a Premarket Notification to the FDA of their intent to market a medical device at least 90 days in advance. This allows the FDA to determine whether the device is equivalent to an already-cleared device. Specifically, “medical device manufacturers are required to submit a premarket notification if they intend to introduce a device into commercial distribution for the first time or reintroduce a device that will be significantly changed or modified to the extent that its safety or effectiveness could be affected. Such change or modification could relate to the design, material, chemical composition, energy source, manufacturing process, or intended use.”81 Upon premarket notification by manufacturers, the FDA provides devices “clearance” upon review of the 510(k) submission. After rigorous review of the devices, the FDA determines whether to grant “approval” for the device to be legally marketed in the U.S. In addition, devices may be granted a “de novo” categorization for novel medical devices for which there is no legally marketed predicate (previously approved) device.
We examined the Premarket Notification [Section 510(k)] records on CPAP devices to determine whether new versions of CPAP devices were cleared (and/or approved or granted de novo categorization) on the basis of new clinical research or on the basis of substantial equivalence to an already-cleared device.
We searched the Devices@FDA database82 without date restrictions (last search on March 1, 2020) using the terms “sleep apnea,” “OSA,” “continuous positive airway pressure,” and “CPAP;” the names of major manufacturers (“Phillips-Respironics”, “ResMed”, “Simons”, “Fisher & Paykel”); and product codes for non-continuous ventilator devices (“BZD”); and additional codes listed on the returned records (“LRK”, “MNQ”). We identified 812 unique Premarket Notification records, of which 163 referred to CPAP devices used for treating sleep apnea in adults. Of these, 143 records were approved in 1996 or later and had downloadable documents; we analyze these. The remaining 20 records without downloadable documents were approved/cleared between 1976 and 1994.
For the 163 510(k) Premarket Notification records on CPAP devices, we extracted an identification number, the device name and manufacturer, and the approval date (see Appendix Table D-4). From the 143 that had downloadable documents we also extracted whether a clinical study was mentioned and whether they cited another approval document to claim equivalence with an approved device. Upon examination of the publicly available data in FDA database, it was apparent that we did not have access to the complete Premarket Notification records. Publicly available data via Devices@FDA database included only: basic device and manufacturer information provided by the submitter; predicate device(s); reason for submission; submitted information about substantial equivalence with previously cleared devices, intended use, and device description; and the clearance letter written by the FDA. The available records do not include explicit references to any clinical studies that may have supported the claims. There was also no explicit reporting of, or reference to, any new (unpublished) clinical data. However, such information might exist in other inaccessible parts of the Premarket Notification records.
The large majority of Premarket Notification records for CPAP devices cite other previously approved/cleared device records to support claims equivalence. We evaluated the more-recent 143 CPAP device records with available data. The 143 CPAP device records also cited 78 records of CPAP accessories, which are included in the analysis. 21 We found a total of 266 citation relationships. Figure 1 graphs the citation relationships reported in the 143 device records. The figure demonstrates 13 groups of records that are connected with citation relationships. The largest relationship group includes 169 device records. The other relationship groups include many fewer records: 3 groups have between 9 and 12 records, and nine groups have between 1 and 4 records.
Among the 143 CPAP device records, four (large red dots in Figure 1) did not cite other records but they were each cited by at least one other record. These include: Apex Medical XT1 CPAP model 9S-005 (document number K070609, approved in 2007), Apex Medical CPAP RT 21XX (document number K022650, approved in 2004), Orion Nasal CPAP System (Bird Products Corp., document number K020730, approved in 2002), and Sullivan Autoset Portable II Nasal CPAP System (ResMed, Ltd., document number K970771, approved in 1997).
Half of the device records (n = 76, empty circles) both cited and were cited by other records. Many device records (n = 61, blue circles) only cited other device records. Two device records neither cited nor were cited by other available device records (gray circles); they were both approved based on predicate devices marketed prior to 1976.
Figure 1. Citations between records to support
Shown are citation relationships (266 grey arrows) between 143 continuous positive airway pressure (CPAP) device records (large circles) and 78 CPAP accessory records (small circles). The arrows point from the record being cited to the citing record, to denote the “flow of information” in an equivalence claim. The 4 CPAP records shown as large red dots (“CPAP, parent”) are cited by other CPAP device records but do not cite other device records themselves. The 61 records shown as large blue dots (“CPAP, terminal”) are CPAP device records that cite other records of CPAP devices (large circles) or CPAP accessories (small circles). Two CPAP device records (large gray circles) are “standalone”, in that they are not cited by other devices nor citing other devices. The remaining 76 CPAP records are shown as large empty circles and are part of a chain of citation relationships (“CPAP, in chain”).
So, virtually no current CPAPs had been specifically assessed by the FDA before they were permitted to be marketed.
With the bar set so low, maybe not that surprising something like the Phillips Respironics debacle could happen.
