(329 days)
ARCHITECT Risk of Ovarian Malignancy Algorithm (ROMA) is a qualitative serum test that combines the results of ARCHITECT HE4. ARCHITECT CA 125 II and menopausal status into a numerical score.
ARCHITECT ROMA is intended to aid in assessing whether a premenopausal woman who presents with an ovarian adnexal mass is at high or low likelihood of finding malignancy on surgery. ARCHITECT ROMA is indicated for women who meet the following criteria: over age 18, ovarian adnexal mass present for which surgery is planned, and not yet referred to an oncologist. ARCHITECT ROMA must be interpreted in conjunction with an independent clinical and radiological assessment. The test is not intended as a screening or stand-alone diagnostic assay.
PRECAUTION: ARCHITECT ROMA should not be used without an independent clinical fradiological evaluation and is not intended to be a screening test or to determine whether a patient should proceed to surgery. Incorrect use of ARCHITECT ROMA carries the risk of unnecessary testing, surgery, and/or delayed diagnosis.
ARCHITECT ROMA is a qualitative serum test that combines the results of 2 analytes, HE4 (ARCHITECT HE4) and CA125 (ARCHITECT CA 125 II) and menopausal status into a numerical score between 0.0 and 10.0. The premenopausal status must be based on ovarian function determined with information available from clinical evaluation and medical history.
The test system consists of the ARCHITECT HE4 assay, the ARCHITECT CA 125 Il assay and the ARCHITECT i2000SR. The ARCHITECT i2000SR is capable of calculating the ROMA score. The immunoassays are performed according to the directions detailed in each product insert.
This document describes the ARCHITECT ROMA, a qualitative serum test that combines results of ARCHITECT HE4, ARCHITECT CA 125 II, and menopausal status to provide a numerical score. It aims to assess the likelihood of malignancy in women with an ovarian adnexal mass for whom surgery is planned.
Here's an analysis of the acceptance criteria and the study that proves the device meets them:
1. Table of Acceptance Criteria and Reported Device Performance
The acceptance criteria are not explicitly laid out as distinct "criteria" with corresponding "acceptance values" in the provided document. Instead, the document presents clinical performance metrics, specifically sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) for both standalone ROMA and its adjunctive use with Initial Cancer Risk Assessment (ICRA). The tables below summarize the reported device performance for these key metrics.
Standalone ARCHITECT ROMA Performance for Stratification into Likelihood of Finding Malignancy on Surgery
(Based on N=238 Premenopausal and N=184 Postmenopausal women, identifying Epithelial Ovarian Cancer, EOC)
| Metric | Premenopausal Performance (95% CI) | Postmenopausal Performance (95% CI) |
|---|---|---|
| Sensitivity | 100.0% (70.1, 99.2) | 92.3% (79.7, 97.2) |
| Specificity | 86.5% (81.4, 90.3) | 85.5% (78.9, 90.3) |
| PPV | 22.5% (12.3, 37.4) | 63.2% (50.2, 74.4) |
| NPV | 100.0% (98.1, 99.9) | 97.6% (93.3, 99.2) |
| Prevalence | 3.8% | 21.2% |
Adjunctive Use of ARCHITECT ROMA with ICRA Performance
(Based on N=85 Malignancies and N=374 No Malignancy, covering EOC, LMP, non-epithelial ovarian cancer, other gynecologic, and non-gynecologic cancers)
| Metric | ICRA Performance (95% CI) | ARCHITECT ROMA Performance (95% CI) | Adjunctive Performance (95% CI) |
|---|---|---|---|
| Sensitivity | 72.9% (62.7, 81.2) | 80.0% (70.3, 87.1) | 87.1% (78.3, 92.6) |
| Specificity | 84.2% (80.2, 87.6) | 86.1% (82.2, 89.2) | 75.7% (71.1, 79.7) |
| PPV | 51.2% (42.4, 60.0) | 56.7% (47.7, 65.2) | 44.8% (37.5, 52.5) |
| NPV | 93.2% (90.0, 95.4) | 95.0% (92.1, 96.8) | 96.3% (93.4, 97.9) |
| Prevalence | 18.5% | 18.5% | 18.5% |
2. Sample Size Used for the Test Set and Data Provenance
- Sample Size for Clinical Study (Test Set):
- Total: 459 women
- Premenopausal: 250
- Postmenopausal: 209
- For the performance tables (sensitivity, specificity, etc.):
- Premenopausal: 238
- Postmenopausal: 184
- Adjunctive use: 459 (85 malignancies, 374 no malignancy)
- Data Provenance: The study was a "prospective, multi-center, blinded clinical trial." The country of origin is not explicitly stated, but the submission is to the US FDA, implying data collected for a US market, potentially including US sites.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts
The document states that "The corresponding histopathology reports were collected after surgery" for each patient to determine the truth (benign or malignant). It does not specify the number of individual experts (e.g., pathologists) involved in establishing the histopathological ground truth or their specific qualifications (e.g., years of experience). However, histopathological classification is a standard and well-established method for confirming benign or malignant status in medical practice.
4. Adjudication Method for the Test Set
The document does not describe a formal adjudication method for discrepancies in ground truth or interpretation of results. The ground truth was established by "histopathology reports," which typically represent a definitive diagnosis.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done, and the Effect Size of How Much Human Readers Improve with AI vs. Without AI Assistance
No, a multi-reader, multi-case (MRMC) comparative effectiveness study focusing on human reader improvement with AI assistance was not explicitly detailed in this document. The study evaluated the adjunctive use of ARCHITECT ROMA with ICRA (Initial Cancer Risk Assessment, which is typically a clinical and radiological assessment performed by human experts).
The document states: "Adjunctive use ARCHITECT ROMA with ICRA produced a statistically significant improvement in the negative predictive value (NPV). The NPV for correctly classifying benign patients into the low likelihood group increased from 93.2 to 96.3%, making the adjunctive use of ARCHITECT ROMA with ICRA effective in ruling out malignancy."
While this shows an improved performance metric (NPV) when ARCHITECT ROMA is used in conjunction with ICRA, it does not quantify human reader performance with and without AI assistance in an MRMC setting. It rather compares the performance of ICRA alone versus ICRA plus ROMA.
6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) Was Done
Yes, a standalone performance evaluation of ARCHITECT ROMA was conducted. The table "Stratification of 422 women... into low likelihood and high likelihood groups for finding malignancy on surgery using ARCHITECT ROMA" directly reports the sensitivity, specificity, PPV, and NPV of the ARCHITECT ROMA algorithm alone, without explicit human-in-the-loop assessment beyond the initial patient selection criteria.
7. The Type of Ground Truth Used
The primary ground truth used for the clinical study was histopathology reports collected after surgery, classifying ovarian adnexal masses as benign or malignant.
8. The Sample Size for the Training Set
The document does not explicitly state the sample size for the training set used to develop the ARCHITECT ROMA algorithm itself. It mentions that ARCHITECT HE4 and ARCHITECT CA 125 II are "previously cleared devices" and their analytical performance was validated separately. The clinical study described in this document serves as a validation or test set for the combined ARCHITECT ROMA algorithm, rather than a training set for its development. The algorithm's equations are provided, indicating it was developed and fixed prior to this validation study.
9. How the Ground Truth for the Training Set Was Established
Since the clinical study described is a validation study and not a development study for the algorithm, the document does not provide details on how the ground truth for an initial training set (if any, for the specific ROMA algorithm) was established. It's plausible the algorithm was developed using a separate dataset, or by leveraging data from the original ROMA™ (HE4 EIA + ARCHITECT CA 125 II) predicate device (K103358), which also used "histopathological findings" as ground truth in its own studies.
§ 866.6050 Ovarian adnexal mass assessment score test system.
(a)
Identification. An ovarian/adnexal mass assessment test system is a device that measures one or more proteins in serum or plasma. It yields a single result for the likelihood that an adnexal pelvic mass in a woman, for whom surgery is planned, is malignant. The test is for adjunctive use, in the context of a negative primary clinical and radiological evaluation, to augment the identification of patients whose gynecologic surgery requires oncology expertise and resources.(b)
Classification. Class II (special controls). The special control for this device is FDA's guidance document entitled “Class II Special Controls Guidance Document: Ovarian Adnexal Mass Assessment Score Test System.” For the availability of this guidance document,see § 866.1(e).(c)
Black box warning. Under section 520(e) of the Federal Food, Drug, and Cosmetic Act these devices are subject to the following restriction: A warning statement must be placed in a black box and must appear in all advertising, labeling, and promotional material for these devices. That warning statement must read: