(229 days)
The Duet™ System is an automated scanning microscope and image analysis system. It is intended for in vitro diagnostic use as an aiding tool to the pathologist in the detection, classification and counting of cells of interest based on color, intensity, size, pattern, and shape. The Duet™ System is intended to:
- Detect Hematopoietic cells stained by Giemsa stain, Immunohistochemistry or ISH (with bright field and fluorescent) prepared from cell suspension.
- Detect amniotic cells stained by FISH (using direct labeled DNA probes for chromosomes X, Y, 13, 18 and 21).
- Detect cells in urine specimens, stained by FISH (using the Vysis UroVysion™ Bladder Cancer Recurrence Kit for chromosomes 3, 7, 17 and 9p21 locus), from subjects with transitional cell carcinoma of the bladder.
- Detect and quantify chromosome 17 and the HER-2/neu gene via fluorescence in situ hybridization (FISH) in interphase nuclei from formalin-fixed, paraffin embedded human breast cancer tissue specimens, probed by the Vysis® PathVysion™ HER-2 DNA Probe Kit. The Duet™ is to be used as an adjunctive automated enumeration tool, in conjunction with manual visualization, to assist in determining HER-2/neu gene to chromosome 17 signal ratio.
The Duet™ System is a fully integrated imaging and scanning platform that automates time-consuming and difficult laboratory tasks of slide screening by making a significant reduction in time and labor currently required.
The Duet™ System workstation integrates a microscope, CCD camera, motorized stage, computer, keyboard, mouse, joystick, monitor and a dedicated software program.
The Duet™ System scans in high resolution and in full color cell samples at high speed both in bright illumination and in fluorescent illumination.
The Duet™ System suggests classification of the cells according to their morphological features, their staining (Giemsa, IHC) and fluorescent signals, and allows the user to quickly examine the results, correct them as needed and generate a report summarizing the sample's data. The unique feature of the Duet™ System allows the combined presentation of morphological and specific staining information of the same cell, for all the cells of the sample.
Here's a breakdown of the acceptance criteria and the study that proves the device meets them, based on the provided text:
Acceptance Criteria and Device Performance
The core acceptance criteria for the Duet™ System's expanded indication for HER-2/neu gene amplification detection in breast cancer tissue specimens via FISH appears to be demonstrating high agreement with manual scoring methods. This is quantified through metrics like Negative Predictive Accuracy (NPA), Positive Predictive Accuracy (PPA), Overall Percentage Agreement, and Kappa coefficient.
The following table summarizes the reported performance for the device against these implied acceptance criteria, based on the analysis of all methods comparison studies (Studies 1 and 4 combined):
| Metric | Acceptance Criteria (Implied) | Reported Device Performance (3 FOVs Analysis, Pooled-studies Results) |
|---|---|---|
| Negative Predictive Accuracy (NPA) | High agreement, ideally near 100% | 100% (95% Lower confidence limit: 93%, 95% Upper confidence limit: 100%) |
| Positive Predictive Accuracy (PPA) | High agreement, ideally near 100% | 96% (95% Lower confidence limit: 77%, 95% Upper confidence limit: 100%) |
| Overall Percentage Agreement | High agreement, ideally near 100% | 99% (95% Lower confidence limit: 92%, 95% Upper confidence limit: 100%) |
| Kappa Coefficient | High agreement, ideally near 100% | 97% (95% Lower confidence limit: 90%, 95% Upper confidence limit: 100%) |
Additional Implied Acceptance Criteria and Performance:
- Reproducibility and Repeatability: High repeatability coefficients of variation (CV) for within-run, day-to-day, and site-to-site analyses. The reported CV values were generally low (0.56% to 12.90%), with one exception of 31.92% for a highly amplified sample where accurate spot counting was inherently difficult.
- Optimal Number of Fields of View (FOVs): Demonstration that a minimal number of FOVs (e.g., 3) provides sufficient accuracy. The study concluded that 3 FOVs are sufficient, with a correlation of 0.996 and an R-squared of 0.993 between 3 and 6 FOV analyses.
Study Details
The study that proves the device meets the acceptance criteria is a performance evaluation study that comprises four sub-studies:
- Methods Comparison (Study 1): Compares the Duet™ System to manual scoring.
- Precision/Reproducibility Performance (Study 2): Evaluates repeatability and reproducibility.
- The Optimal Number of Fields of View (Study 3): Determines the optimal FOV count.
- Methods Comparison at the Borderline Range (Study 4): Focuses on performance in challenging cases.
Detailed information for each requested point is provided below:
2. Sample size used for the test set and the data provenance:
- Study 1 (Methods Comparison): A total of 56 specimen slides were used. These were prepared from human breast cancer tissue specimens in various stages of the disease, representing the entire intended use population. The data provenance is implied to be retrospective as the slides were "prepared from human breast cancer tissue specimens." The country of origin is not explicitly stated.
- Study 2 (Precision/Reproducibility): Five (5) slides were used, representing the range of intended use (two negative controls, two mid-low amplified controls, one highly amplified slide).
- Study 4 (Methods Comparison at the Borderline Range): A total of 21 human breast cancer tissue specimen slides were used. This included the seven (7) borderline samples from Study-1 and an additional fourteen (14) new borderline samples collected from an "additional site." This suggests retrospective data, and the country of origin is not explicitly stated.
3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts:
- The text frequently refers to the "manual scoring method" as the comparator, which served as the ground truth. However, it does not specify the number of experts who performed the manual scoring nor their specific qualifications (e.g., "radiologist with 10 years of experience"). It implicitly refers to "the trained lab technician operator" for clinical use, suggesting the manual scoring was done by qualified laboratory personnel.
4. Adjudication method for the test set:
- The document does not explicitly describe an adjudication method (like 2+1 or 3+1) for resolving discrepancies between manual readings or between manual and automated readings. The "manual scoring method" is treated as the reference standard against which the Duet™ System is compared.
5. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:
- No, an MRMC comparative effectiveness study was not explicitly described in terms of human readers improving with AI assistance. This study primarily focuses on the standalone performance of the Duet™ System compared to manual scoring. The device is intended to be an "adjunctive automated enumeration tool, in conjunction with manual visualization," implying it assists. However, the study design does not quantify the improvement of human readers when using the AI compared to when they don't. It assesses the agreement between the AI system and manual reading.
6. If a standalone (i.e. algorithm only without human-in-the loop performance) was done:
- Yes, a standalone performance study was done. The "Methods Comparison" studies (Study 1 and Study 4) directly compare the Duet™ System's results (algorithm only, as it performs automated scanning and image analysis) with the manual scoring method. The reported NPA, PPA, Overall Agreement, and Kappa values reflect the Duet™ System's accuracy as a standalone enumeration tool for the specified task. The text states: "...the Duet™ System method for detection of amplification... in comparison to the manual scoring method." This indicates a standalone evaluation of the device's output.
7. The type of ground truth used:
- The ground truth used was expert consensus / manual scoring. The text consistently refers to the "manual scoring method" as the reference standard against which the Duet™ System's performance was evaluated. 'Amplification' and 'No-Amplification' were defined based on ratios from the Vysis® PathVysion™ Kit's package insert, which guides manual interpretation.
8. The sample size for the training set:
- The document does not specify the sample size for the training set. The performance evaluation study focuses on the test set used to validate the already developed system after "minor software changes, including new algorithm for the automatic calculation of the overall amplification ratio of HER-2/neu gene, were made." It also states "All these changes were fully verified and validated through software verification and validation," implying a development and training phase occurred, but the details are not provided in this summary.
9. How the ground truth for the training set was established:
- As the document does not specify the training set size, it also does not detail how the ground truth for the training set was established. However, given the nature of the application (HER-2/neu FISH analysis) and the reliance on "manual scoring method" as ground truth for the validation studies, it is highly probable that the training data's ground truth was also established through expert manual scoring and adherence to the Vysis® PathVysion™ Kit instructions.
§ 864.5260 Automated cell-locating device.
(a)
Identification. An automated cell-locating device is a device used to locate blood cells on a peripheral blood smear, allowing the operator to identify and classify each cell according to type. (Peripheral blood is blood circulating in one of the body's extremities, such as the arm.)(b)
Classification. Class II (performance standards).