Search Results
Found 1 results
510(k) Data Aggregation
(887 days)
Digital cervical cytology slide
imaging system with artificial
intelligence algorithm | 21 CFR 864.3900
cervical cytology slide imaging system with artificial intelligence algorithm Class: II Regulation: 21 CFR 864.3900
The Genius TM Digital Diagnostics System with the Genius™ Cervical AI algorithm includes the Genius TM Digital Imager, Genius™ Image Management Server (IMS), the Genius™ Review Station, and the Genius™ Cervical AI algorithm. The Genius™ Digital Diagnostics System with the Genius™ Cervical AI algorithm is intended for the creation and viewing of digital images of scanned ThinPrep® Pap Test glass slides. Objects of interest selected by the Genius™ Cervical AI algorithm from the scanned digital image are presented in a gallery format next to the image of the whole cell spot on the Genius™ Review Station for review and interpretation. The Genius Digital Diagnostics System with the Genius™ Cervical AI algorithm is intended to aid in cervical cancer screening for the presence of atypical cells, cervical neoplasia, including its precursor lesions (Low Grade Squamous Intraepithelial Lesions, High Grade Squamous Intraepithelial Lesions) and carcinoma, as well as all other cytological categories as defined by The Bethesda System for Reporting Cervical Cytology1.
After digital review with the Genius Cervical AI algorithm, if there is uncertainty in the diagnosis, then direct examination of the glass slide by light microscopy should be performed. Digital images from the Genius Digital Diagnostics System with the Genius™ Cervical AI algorithm should be interpreted by qualified cytologists in conjunction with the patient's screening history, other risk factors, and professional guidelines which guide patient management.
The Genius Digital Diagnostics System includes, Genius Digital Image. Management Server, Genius Review Station(s), and Genius Cervical AI algorithm, as shown in Figure 1 below.
Here's a breakdown of the acceptance criteria and the study that proves the Genius™ Digital Diagnostics System with the Genius™ Cervical AI algorithm meets them, based on the provided text:
Acceptance Criteria and Device Performance for Genius™ Digital Diagnostics System with Genius™ Cervical AI algorithm
1. Table of Acceptance Criteria and Reported Device Performance
The document doesn't explicitly lay out bulleted acceptance criteria for the entire system in a single section. However, performance benchmarks derived from various studies are detailed. The primary clinical acceptance criteria revolve around the AI-assisted review being comparable or superior to the standard of care (manual review) and other automated systems (TIS review) in terms of sensitivity and specificity at various diagnostic thresholds for cervical cytology. Key technical performance criteria are also highlighted.
Here is a summary, inferred from the "Performance Characteristics" section:
| Acceptance Criteria Category (Inferred/Stated) | Specific Metric/Target | Reported Device Performance |
|---|---|---|
| Technical Performance | ||
| Color Reproducibility | Accuracy and precision of colors displayed: Acceptance criteria unspecified but stated to be met. | "The colors displayed on the Genius Review Stations for scans taken using the Genius Digital Imager meets the acceptance criteria specified." |
| Structural Similarity Index (SSIM) | SSIM > 0.9 for image capture reproducibility (between-imager, within-imager, imaging stability). | "Study results showed all SSIM were greater than the 0.9. Performance met the defined criteria." |
| Spatial Resolution | Optics of Digital Imager meet prespecified criteria. | "The spatial resolution of the optics in the digital cytology imager meets the prespecified acceptance criteria." |
| Focusing Quality | Meets acceptance criteria. | "The focus quality of whole slide images produced by the Digital Diagnostics system meets the acceptance criteria specified." |
| Whole Cell-Spot Coverage | Acceptable based on visual inspection. | "The Genius Digital Imager scanned the entire cell spot of the slide and the cell spot coverage was determined to be acceptable based on visual inspection." |
| Stitching Error | Meets prespecified acceptance criteria. | "The stitching of image swaths in the Genius Digital Imager met the pre-specified acceptance criteria." |
| Turnaround Time | Acceptable. | "The turnaround time used in the Genius Digital Imager was determined to be acceptable." |
| Analytical Performance | ||
| OOI (Object of Interest) Selection Accuracy | Proportion of abnormal OOIs and category+ OOIs presented by the AI algorithm should align with reference diagnosis; high agreement rates for various thresholds. | For all Abnormal (30 slides, 810 evaluations): 100% Proportion of Abnormal OOIs, 98% Proportion Category+ OOIs. Agreement Rates (e.g., ASCUS+: 100%, LSIL+: 98%, HSIL+: 99%, CANCER: 92%). |
| OOI Reproducibility | High agreement rates for between-instrument and within-instrument for Category+ OOIs. | Between-instrument: 96% agreement. Within-instrument: 99% agreement. |
| Cell Count Accuracy | Acceptable results from linear regression comparing AI-derived count to manual count (slope and intercept confidence intervals). Relative systematic difference at 5,000 cells. | Slope 1.06 (95% CI: 1.01; 1.11), intercept 213 (95% CI: 28; 398). Relative systematic difference at 5,000 cells was 10% (95% CI: 4%; 17%). "The results of the cell count study were acceptable." |
| Clinical Performance (Comparative) | ||
| Sensitivity (Genius AI vs. Manual Review) | For ASCUS+, Genius AI comparable to or better than Manual Review. For LSIL+, ASC-H+, HSIL+, Genius AI sensitivity statistically significantly higher than Manual Review. For Cancer, Genius AI sensitivity comparable to Manual Review. | ASCUS+: Genius AI: 91.7%, Manual: 90.1%. Difference: +1.6% (95% CI: -0.1, 3.2) - not statistically significant. |
| LSIL+: Genius AI: 89.1%, Manual: 84.7%. Difference: +4.4% (95% CI: 2.1, 6.7) - statistically significant. | ||
| ASC-H+: Genius AI: 87.8%, Manual: 79.6%. Difference: +8.2% (95% CI: 4.8, 11.6) - statistically significant. | ||
| HSIL+: Genius AI: 81.5%, Manual: 74.0%. Difference: +7.5% (95% CI: 4.0, 11.4) - statistically significant. | ||
| Cancer: Genius AI: 66.7%, Manual: 66.7%. Difference: 0.0% (95% CI: -9.8, 11.1) - not significant. | ||
| Specificity (Genius AI vs. Manual Review) | For all diagnostic thresholds, specificity of Genius AI comparable to Manual Review, or that any statistically significant decreases are acceptable given clinical benefits. | ASCUS+: Genius AI: 91.0%, Manual: 92.2%. Difference: -1.3% (95% CI: -2.3, -0.2) - statistically significant decrease. |
| LSIL+: Genius AI: 91.7%, Manual: 94.1%. Difference: -2.4% (95% CI: -3.5, -1.4) - statistically significant decrease. | ||
| ASC-H+: Genius AI: 94.2%, Manual: 97.0%. Difference: -2.9% (95% CI: -3.8, -1.9) - statistically significant decrease. | ||
| HSIL+: Genius AI: 94.8%, Manual: 97.2%. Difference: -2.4% (95% CI: -3.0, -1.7) - statistically significant decrease. | ||
| Cancer: Genius AI: 98.5%, Manual: 98.5%. Difference: -0.0% (95% CI: -0.4, 0.4) - not significant. | ||
| False Negative Rate (False NILM) | Reduction in false NILM results compared to manual. | Overall false NILM: Genius AI: 8.3%, Manual: 9.9%. Genius AI shows a 1.6% reduction in overall false NILM results. Specific reductions for LSIL, ASC-H. Increase for Cancer. |
| UNSAT Sensitivity | Genius AI should correctly identify UNSAT cases. | Genius AI: 80.4%, Manual: 61.8%. Genius AI correctly identified 18.6% more UNSAT cases (or ASCUS+) than Manual review. |
| Sensitivity (Genius AI vs. TIS Review) | Genius AI sensitivity comparable or better than TIS. | ASCUS+: Genius AI: 91.7%, TIS: 91.6%. Difference: +0.1% (not significant). |
| LSIL+: Genius AI: 89.1%, TIS: 87.7%. Difference: +1.4% (not significant). | ||
| ASC-H+: Genius AI: 87.8%, TIS: 84.3%. Difference: +3.6% (statistically significant). | ||
| HSIL+: Genius AI: 81.5%, TIS: 77.9%. Difference: +3.6% (statistically significant). | ||
| Specificity (Genius AI vs. TIS Review) | Genius AI specificity comparable to TIS. | ASCUS+: Genius AI: 91.0%, TIS: 92.6%. Difference: -1.6% (statistically significant decrease). |
| LSIL+: Genius AI: 91.7%, TIS: 93.3%. Difference: -1.6% (statistically significant decrease). | ||
| ASC-H+: Genius AI: 94.2%, TIS: 96.4%. Difference: -2.2% (statistically significant decrease). | ||
| HSIL+: Genius AI: 94.8%, TIS: 96.6%. Difference: -1.7% (statistically significant decrease). | ||
| Cytologist Workload | A defined CLIA slide equivalent for AI-assisted review. | Genius Cervical AI (GCAI) case reviews count as 0.5 CLIA slide equivalent. This allows for a higher volume of cases to be screened per day (200 cases = 100 CLIA slide equivalents). |
2. Sample Size Used for the Test Set and Data Provenance
- Sample Size for Clinical Test Set: 1994 slides.
- Data Provenance:
- Country of Origin: United States ("A multi-center Genius Cervical AI Algorithm Clinical Study was performed within the United States.").
- Retrospective/Prospective: The study used "residual material after the clinical sites signed out the case," implying these were historical samples (retrospective) collected from women screened for cervical cancer using the ThinPrep Pap test.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications
- Number of Experts: An adjudication panel composed of 3 adjudication CT/PT teams, each consisting of 1 Cytotechnologist (CT) and 1 Pathologist (PT). So, a minimum of 3 CTs and 3 PTs were involved in establishing the ground truth.
- Qualifications of Experts: They were "qualified cytologists" and "qualified CTs and PTs" (implied as experts in cervical cytology based on their role in the study and laboratory context). The participating laboratories had "extensive experience in the processing and evaluation of gynecologic ThinPrep Pap test slides" and the experts were "trained in the use of the Genius Digital Diagnostics System with Genius Cervical AI algorithm."
4. Adjudication Method for the Test Set
- Method: Consensus-based adjudication followed by multi-headed microscope review for discordant cases.
- Each of the three adjudication CT/PT teams independently reviewed the slides.
- A consensus result was obtained if there was majority agreement (at least two of the three adjudication CT/PT teams).
- If a consensus was not initially obtained, these cases underwent review by the three adjudication PTs simultaneously using a multi-headed microscope (multi-head review).
- The final "reference" or "ground truth" diagnosis was based on either the initial consensus or the multi-head review result.
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
-
Was it done? Yes, a multi-center, multi-reader, multi-case study was performed, comparing Genius AI review, Manual review, and TIS review. This is evident from the "Clinical Studies" section, where 4 clinical sites participated, and at each site, 3 independent CT/PT teams reviewed cases.
-
Effect Size of Human Readers Improvement with AI vs. without AI Assistance:
The study directly compares human performance with AI assistance (Genius AI review) to human performance without AI assistance (Manual review). The improvement is shown in sensitivity at various diagnostic thresholds:- LSIL+: Sensitivity increased by 4.4% with Genius AI assistance compared to Manual review (89.1% vs 84.7%). This was statistically significant.
- ASC-H+: Sensitivity increased by 8.2% with Genius AI assistance compared to Manual review (87.8% vs 79.6%). This was statistically significant.
- HSIL+: Sensitivity increased by 7.5% with Genius AI assistance compared to Manual review (81.5% vs 74.0%). This was statistically significant.
- ASCUS+: Sensitivity increased by 1.6% (91.7% vs 90.1%), but this was not statistically significant.
- UNSAT (for correct identification): Sensitivity increased by 18.6% with Genius AI assistance (80.4% vs 61.8%).
While specificity generally saw small, statistically significant decreases at some thresholds with AI assistance, the overall benefit-risk assessment concluded that the probable benefits outweigh the risks due to the increased sensitivity, particularly for higher-grade lesions. The document also highlights a reduction in false negative rates (false NILM) with the Genius AI: "The 7.5% increase in HSIL + sensitivity means a decrease in Manual false negative rate of 26% to 18.5% false negative rate by the Genius Digital Diagnostics System with the Genius Cervical AI algorithm resulted in 28.8% reduction in the number false negative reviews (28.8%=(26%-18.5%) / 26%)."
6. Standalone (Algorithm Only Without Human-in-the-Loop Performance) Study
- Was it done? Not explicitly for diagnostic interpretation. The "Objects of Interest (OOI) Reproducibility" study evaluated the AI algorithm's ability to select OOIs against the adjudicated reference diagnosis, but the primary clinical performance assessment was human-in-the-loop (AI-assisted review). The AI's function is to "selects OOIs to be displayed for review by a CT or PT," not to render a final diagnosis on its own.
7. Type of Ground Truth Used
- Type of Ground Truth: Expert Consensus. The "adjudicated diagnosis" served as the "gold standard" or "reference" (ground truth). This was established by a panel of 3 CT/PT teams reaching consensus, with multi-head microscope review for discordant cases.
8. Sample Size for the Training Set
- The document mentions the use of "Convolutional Neural Network (CNN) technology" for the AI algorithm but does not explicitly state the sample size for the training set. It only mentions that the "Genius Cervical AI algorithm v1.0.16.0 will remain locked for use with the authorized device and will not be continually trained and improved with each cohort analyzed in clinical practice, after marketing authorization," implying it was trained on a fixed dataset prior to these performance studies.
9. How Ground Truth for the Training Set Was Established
- The document does not explicitly describe how the ground truth for the training set was established. It focuses only on the ground truth for the test set used for performance validation.
Ask a specific question about this device
Page 1 of 1