(486 days)
This assay is designed to provide a method for the detection and quantification of chromosome 8 in both interphase nuclei and metaphase spreads by fluorescence in situ hybridization (FISH).
The CEP 8 SpectrumOrange DNA Probe is a SpectrumOrange fluorescent labeled DNA probe specific for the centromeric region of chromosome 8.
Here's a breakdown of the acceptance criteria and the study that proves the device meets them, based on the provided text:
Acceptance Criteria and Device Performance for CEP 8 SpectrumOrange DNA Probe Kit
1. Table of Acceptance Criteria and Reported Device Performance
| Performance Metric | Acceptance Criteria (Implicit) | Reported Device Performance |
|---|---|---|
| Analytical Sensitivity & Specificity | ||
| Hybridization Failure Rate | < 2% cells with no signal | Pilot Study: 0.22% (s.d. 0.20%) on 35 BM specimensPivotal Study: 0.11% (s.d. 0.21%) on 60 BM specimens |
| Limit of Detection (Interphase) | Not explicitly stated but derived from 0% and 5% specimen overlap | 4.0% tri-signaled nuclei |
| Locus Specificity | No cross-hybridization to other chromosome loci | Hybridization limited to centromere region of chromosome 8 in 62 metaphase spreads |
| Reproducibility | ||
| Inter-site, Inter-day, Inter-observer (Pilot Study - Normal Specimen) | High classification accuracy for trisomy 8 | 96% correct classification (using 2.2% cutoff) |
| Inter-site, Inter-lot, Inter-day, Inter-observer (Pivotal Study - Low Level Trisomy 8 Specimen) | High classification accuracy for trisomy 8 | 100% correct classification (using 2.2% cutoff) |
| Clinical Performance (Interphase Analysis vs. Standard Cytogenetics) | ||
| Relative Sensitivity (using 2.2% cutoff) | Not explicitly stated but considered acceptable for diagnostic use | 96.03% (145/151) [95% C.I. 92.55-99.51%] |
| Relative Specificity (using 2.2% cutoff) | Not explicitly stated but considered acceptable for diagnostic use | 98.01% (197/201) [95% C.I. 96.08-99.94%] |
| Correlation Coefficient (Trisomy 8) | High correlation | 0.91 |
| Clinical Performance (Metaphase Analysis vs. Standard Cytogenetics) | ||
| Relative Sensitivity (using ≥ 2 tri-signaled metaphases cutoff) | Not explicitly stated but considered acceptable for diagnostic use | 89.19% (132/148) [95% C.I. 84.19-94.19%] |
| Relative Specificity (using ≥ 2 tri-signaled metaphases cutoff) | Not explicitly stated but considered acceptable for diagnostic use | 91.30% (168/184) [95% C.I. 87.22-95.37%] |
| Correlation Coefficient (Trisomy 8) | High correlation | 0.91 |
| Clinical Performance (Interphase Analysis vs. Metaphase Analysis) | ||
| Concordance | High concordance | 90.8% [(133+183)/348] |
| Correlation Coefficient (Trisomy 8) | High correlation | 0.95 |
2. Sample Size Used for the Test Set and Data Provenance
- Analytical Sensitivity & Specificity:
- Hybridization Efficiency: Pilot study had 35 bone marrow (BM) specimens. Pivotal study had 60 BM specimens.
- Analytical Specificity: 62 metaphase spreads.
- Data Provenance: Not explicitly stated (e.g., country of origin), assumed to be within the US given FDA submission. It's retrospective for analytical specificity (archived metaphase spreads).
- Reproducibility:
- Pilot Study: One normal bone marrow specimen. N=24 observations/measurements.
- Pivotal Study: One low-level (approximately 7%) trisomy 8 bone marrow specimen. N=24 observations/measurements.
- Data Provenance: Not explicitly stated, likely multi-site within the US. Retrospective (single specimens used for repeated testing).
- Clinical Performance (Methods Comparison):
- Test Set Size: 368 archived bone marrow specimens.
- Data Provenance: Multi-center study involving four laboratories providing specimens (Site 1: 101, Site 2: 57, Site 3: 130, Site 4: 80, with Site 4 specimens analyzed at Site 3). The source of the specimens is described by the diseases: Acute myeloid leukemia (AML), Myeloproliferative disorder (MPD), Myelodysplastic syndrome (MDS), Chronic myelogenous leukemia (CML), Hematological disorder, not otherwise specified (HIDNOS). The data is retrospective as it used "archived bone marrow specimens." The country of origin is not specified but the study was submitted to the FDA.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications
- The ground truth for the clinical studies (Methods Comparison) was established by "standard cytogenetic analysis" performed by the participating laboratories.
- The text does not specify the number of individual experts per lab or their specific qualifications (e.g., "radiologist with 10 years of experience"). However, "standard cytogenetic analysis" implies trained and qualified cytogeneticists or laboratory personnel are performing the traditional karyotyping and interpretation.
4. Adjudication Method for the Test Set
- The text does not explicitly describe an adjudication method for the ground truth (standard cytogenetics) in the methods comparison study. It implies that the "standard cytogenetic analysis" results from each participating laboratory were accepted as the ground truth.
- For the device's own FISH analysis, inter-observer and inter-site variability were assessed in the reproducibility studies, suggesting that multiple observers/sites independently evaluated the samples, but it doesn't detail a formal adjudication process for discordant results.
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
- No, a MRMC comparative effectiveness study was not explicitly conducted to assess human readers' improvement with AI vs. without AI assistance.
- This device is a DNA probe kit for Fluorescence In Situ Hybridization (FISH), which is a laboratory assay where trained personnel visually count signals or use automated enumeration systems. It is not an "AI" device as described in modern AI/ML contexts that would assist radiologists or other human readers. The study compares the performance of the FISH assay (both interphase and metaphase interpretation) against the standard cytogenetic analysis.
- The reproducibility studies assessed inter-observer variability for the FISH assay itself, reflecting the inherent subjectivity of visual enumeration, but not the impact of an AI assistance tool on human performance.
6. Standalone (Algorithm Only Without Human-in-the-Loop Performance) Study
- Yes, standalone performance was assessed. The CEP 8 SpectrumOrange DNA Probe Kit is the "algorithm" in this context (or more accurately, the assay and its interpretation guidelines).
- The entire clinical methods comparison study (comparing FISH interphase and metaphase analysis to standard cytogenetics) represents the standalone performance of the assay and its associated interpretation criteria (e.g., 2.2% cutoff for interphase, >=2 tri-signaled metaphases for metaphase). The sensitivity, specificity, and correlation coefficients reported are for the device's performance when applied according to its instructions.
7. Type of Ground Truth Used
- For the clinical methods comparison study, the ground truth was expert consensus / established diagnostic method: Standard Cytogenetic Analysis. This method is referred to as "the standard of care."
8. Sample Size for the Training Set
- The text does not explicitly describe a separate training set in the way modern machine learning models would have one.
- The "pilot study" and "pivotal study" results for hybridization efficiency and reproducibility involve smaller numbers of specimens (35 BM, 60 BM, single normal and single trisomy 8 specimens) that could be considered part of the development and refinement of the assay and its interpretation rules, but not a formal "training set" for an algorithm. The 2.2% cutoff for interphase analysis appears to have been 'validated by the same pivotal clinical study,' suggesting it was developed and then verified within the context of the overall validation.
9. How the Ground Truth for the Training Set Was Established
- Since there's no explicitly defined "training set" for an algorithm in the modern sense, the concept of establishing ground truth for it doesn't directly apply here.
- However, the underlying biological understanding and the methodology of interpreting FISH signals (e.g., what constitutes a "tri-signaled nuclei") would be based on established cytogenetic principles and likely refined through internal studies during the probe's development, informed by standard cytogenetic analysis, which is the gold standard for chromosomal abnormalities. The cutoff values (like 2.2%) were likely empirically determined and then validated against the clinical performance.
{0}------------------------------------------------
SAFETY AND EFFECTIVENESS INFORMATION SUMMARY: FOR CEP 8 SpectrumOrange DNA Probe Kit
The CEP 8 SpectrumOrange DNA Probe is a SpectrumOrange fluorescent labeled DNA probe specific for the centromeric region of chromosome 8. This assay is designed to provide a method for the detection and quantification of chromosome 8 in both interphase nuclei and metaphase spreads by fluorescence in situ hybridization (FISH).
Standard cytogenetic analysis detects cytogenetic abnormalities such as trisomy 8 by karyotyping metaphase spreads after staining the chromosomes with a dye in cultured tissue cells.
Safety and effectiveness issues relevant to FISH CEP 8 assay may include cross-reactivity, poor sensitivity, poor specificity, or poor reproducibility. The following summarize the specific performance characteristics of CEP 8 assay:
Analytical Sensitivity and Specificity
Hybridization Efficiency
In a pilot study, the average percentage of cells with no hybridization signal was 0.22% (s.d. 0.20%) on 35 bone marrow (BM) specimens. In a pivotal study, the average percentage of cells with no hybridization signal was 0.11% (s.d .- 0.21%) on 60 BM specimens. Thus, <2% cells with no signal is a realistic standard of acceptance, especially for FISH metaphase analysis.
Analytical Sensitivity
The analytical sensitivity of the CEP 8 probe was tested in the reproducibility study described below. In that study, the 0% specimen was estimated with a mean of 0.95% (s.d. 0.51%) tri-signaled nuclei and the 5% specimen, 5.37% (s.d. =0.98%). There was little overlap between the 0% and 5% specimens; the upper 95% confidence limit for the 0% specimen was 1.95% and the lower 95% confidence limit for the 5% specimen was 3.45%. Thus, the limit of detection for CEP 8 in interphase cells is estimated to be 4.0%.
Analytical Specificity
Locus specificity studies were performed with metaphase spreads according to standard Vysis OC protocols. A total of 62 metaphase spreads were examined sequentially by Gbanding to identify chromosome 8, followed by FISH. No cross-hybridization to other chromosome loci was observed in any of the 62 cells examined; hybridization was limited to the centromere region of chromosome 8.
{1}------------------------------------------------
Reproducibility
In a pilot study, the reproducibility of CEP 8 interphase analysis for the %tri-signaled cells cells was assessed for inter-site, inter-day, and inter-observer reproducibility, using one lot of the CEP 8 DNA probe. One normal bone marrow specimen was evaluated for the %tri-signaled cells according to the instructions for signal enumeration in the package insert. Statistically significant site-to-site and observer-to-observer variations were observed, reflecting the subjectivity of the visual enumeration process. The results of classification of slides as positive or negative for trisomy 8 (using a cutoff of 2.2%) was 96% correct (1 of 24 had 2.4% tri-signaled nuclei) for this normal specimen. For the overall variation, the N, mean, SD, and percent CV of the observed %tri-signaled nuclei was 24, 0.99%, 0.57%, and 58%, respectively.
In a pivotal study, the CEP 8 interphase analysis for the %tri-signaled cells was again assessed for inter-site, inter-lot, inter-day and inter-observer reproducibility. One low level (approximately 7%) trisomy 8 bone marrow specimen was evaluated for the %trisignaled cells according to the instructions for signal enumeration in the package insert. Statistically significant site-to-site and observer variations were observed, reflecting the subjectivity of the visual enumeration process. The results of classification of slides as positive or negative for trisomy 8 (using a cutoff of 2.2%) was 100% correct for this low level (7%) trisomy 8 bone marrow specimen. For the overall variation, the N, mean, SD, and percent CV of the observed percentage of trisignaled nuclei was 24, 7.70%, 1.45%, and 19%, respectively.
Methods Comparison; Clinical Specimens
A multi-center, blinded, controlled, comparative study was conducted to further define the performance characteristics of the CEP 8 probe. The objective of the study was to determine the sensitivity and specificity of the CEP 8 assay relative to standard cytogenetic analysis, the standard of care. Four laboratories provided 368 archived bone marrow specimens for assay at three investigation sites. Site 1 provided 101 specimens; Site 2 provided 57 specimens; Site 3 provided 130 specimens; Site 4 provided 80 specimens. Specimens from Site 4 were analyzed at Site 3. By standard cytogenetic analysis. 151 of these specimens were classified as positive for trisomy 8; 201 negative for trisomy 8; and 16 ambiguous for trisomy 8 (1 trisomy 8 per 30 metaphases analyzed). Fifteen of the 16 ambiguous cases were selected "purposefully" after study completion. These specimens were derived from patients with one of the following diagnoses.
- Acute myeloid leukemia (AML): 102 specimens 1.
- Myeloproliferative disorder (MPD), including polycythemia vera: 44 2. specimens
-
- Myelodysplastic syndrome (MDS): 80 specimens
- Chronic myelogenous leukemia (CML): 72 specimens 4.
- Hematological disorder, not otherwise specified (HIDNOS): 70 specimens 5. (including hyperproliferative states such as leukemoid reaction, lymphoproliferative disorders or chronic lymphocytic leukemia, without trisomy 8).
At one trial site, approximately 50% of the archived bone marrow specimens failed to produce informative FISH results. Further examination of a subset of these specimens revealed a lack of specimen integrity and it was determined that specimens at this site were stored at 4℃ rather than at the recommended temperature of -20℃. The conclusion was that some specimens and/or slide preparations were inadequate.
{2}------------------------------------------------
FISH Internhase Analysis versus Standard Cytogenetics
From the same multi-center comparative study described above, the analysis of interphase nuclei by FISH compared to standard cytogenetics was performed. Based on the cutoff point of 2.2% tri-signaled nuclei that was validated by the same pivotal clinical study, the relative sensitivity was 96.03% (145/151) [95% C.I. 92.55 -99.51%] and the relative specificity was 98.01% (197/201) [95% C.I. 96.08 -99.94%] for the CEP 8 interphase analysis.
Among the 10 discrepant cases, the standard cytogenetic analysis results ranged from 0/20 to 8/22 metaphase cells with trisomy 8: the CEP 8 interphase results of 9 trisignaled nuclei ranged from 0.6% to 6.0%. Among the 16 ambiguous cases, the range of % tri-signaled nuclei by CEP 8 interphase analysis was from 0.2% to 2.2%.
From the 368 clinical specimens, the correlation coefficient of trisomy 8 between standard cytogenetic metaphase analysis and CEP 8 interphase analysis was 0.91. The regression coefficient of CEP 8 assay on cytogenetic analysis was 0.71. The following equation describes the plot of CEP 8 assay results on cytogenetic analysis results:
$$\mathbf{y} = 1.2944 + 0.7112\mathbf{x}$$
where:
x = percent metaphase spreads with trisomy 8 by standard cytogenetics y = percent tri-signaled interphase nuclei by CEP 8 analysis
FISH Metanhase Analysis versus Standard Cytogenetics
A total of 348 cases were included in the comparison of CEP 8 metaphase analysis to standard cytogenetics. Twenty of the 368 cases included in the CEP 8 interphase analysis were excluded due to insufficient number of metaphase spreads for analysis with the CEP 8 assay (a specimen must have ≥ 20 metaphase spreads).
By the rules of standard cytogenetic analysis, a case is declared positive for trisomy 8 if two or more metaphase spreads are trisomic for chromosome 8. Using this cutoff of ≥ 2 tri-signaled metaphases, the relative sensitivity was 89.19% (132/148) [95% C.I. 84.19 - 94.19%] and the relative specificity was 91.30% (168/184) [95% C.I. 87.22 - 95.37%] for the CEP 8 metaphase analysis. Among the 16 ambiguous cases by standard cytogenetic analysis, 15 were negative and 1 was ambiguous for trisomy 8 by CEP 8 metaphase analysis.
From the 348 clinical specimens, the correlation coefficient of trisomy 8 between cytogenetic metaphase analysis and CEP 8 metaphase analysis was 0.91. The regression coefficient of CEP 8 assay on cytogenetic analysis was 0.83. The following equation describes the plot of CEP 8 assay results on cytogenetic analysis results:
$$\mathbf{y} = 1.6979 + 0.8325 \mathbf{x}$$
where: x = percent metaphase spreads with trisomy 8 by standard cytogenetics y = percent tri-signaled metaphase spreads by CEP 8 analysis
{3}------------------------------------------------
FISH Interphase Analysis versus FISH Metaphase Analysis
In addition to the methods comparison between FISH and standard cytogenetics described above, a comparison between FTSH interphase and FISH metaphase was made. There was a 90.8% [(133+183)/348] concordance between FISH interphase and FISH metaphase.
From the 348 clinical specimens, the correlation coefficient of trisomy 8 between CEP 8 interphase analysis and CEP 8 metaphase analyses was 0.95. The regression coefficient of CEP 8 metaphase analysis on interphase analysis was 0.81. The following equation describes the plot of CEP 8 assay metaphase results.on CEP 8 interphase analysis results:
y = 1.0942 + 0.8119x
x = percent tri-signaled interphase cells by CEP 8 analysis where: y = percent tri-signaled metaphase spreads by CEP 8 analysis
Conclusions
Consistent performance of CEP 8 is guaranteed by the Vysis Quality Control Procedures. When the CEP 8 SpectrumOrange DNA Probe is used as instructed in the package insert, the above statements describe its performance.
{4}------------------------------------------------
Image /page/4/Picture/1 description: The image is a black and white logo for the U.S. Department of Health & Human Services. The logo features a circular design with the text "DEPARTMENT OF HEALTH & HUMAN SERVICES • USA" arranged around the perimeter. Inside the circle is a stylized symbol that resembles a human figure or head in profile, with flowing lines suggesting movement or connection.
Food and Drug Administration 10903 New Hampshire Avenue Silver Spring, MD 20993
JAN 25 2012
Vysis c/o Dr. Russel K. Enns Vice President, Regulatory Affairs 3100 Woodcreek Dr. Downers Grove, IL 60515
Re: K953591
Trade/Device Name: CEP 8 SpectrumOrange DNA Probe Kit Regulation Number: 21 CFR §866.4700 Regulation Name: Automated fluorescence in situ hybridization (FISH) enumeration systems. Regulatory Class: Class II Product Code: OYU, KIR Dated: October 21, 1996 Received: October 22, 1996
Dear Dr. Enns:
This letter corrects our substantially equivalent letter of November 29, 1996.
We have reviewed your Section 510(k) premarket notification of intent to market the device referenced above and have determined the device is substantially equivalent (for the indications for use stated in the enclosure) to legally marketed predicate devices marketed in interstate commerce prior to May 28, 1976, the enactment date of the Medical Device Amendments, or to devices that have been reclassified in accordance with the provisions of the Federal Food, Drug, and Cosmetic Act (Act) that do not require approval of a premarket approval application (PMA). You may, therefore, market the device, subject to the general controls provisions of the Act. The general controls provisions of the Act include requirements for annual registration, listing of devices, good manufacturing practice, labeling, and prohibitions against misbranding and adulteration.
If your device is classified (see above) into class II (Special Controls), it may be subject to such additional controls. Existing major regulations affecting your device can be found in Title 21, Code of Federal Regulations (CFR), Parts 800 to 895. In addition, FDA may publish further announcements concerning your device in the Federal Register.
Please be advised that FDA's issuance of a substantial equivalence determination does not mean that FDA has made a determination that your device complies with other requirements of the Act or any Federal statutes and regulations administered by other Federal agencies. You must comply with all the Act's requirements, including, but not limited to: registration and listing (21 CFR Part 807); labeling (21 CFR Parts 801 and 809); medical device reporting of medical device-related
{5}------------------------------------------------
Page 2 – Dr. Russel K. Enns
adverse events) (21 CFR 803); and good manufacturing practice requirements as set forth in the quality systems (QS) regulation (21 CFR Part 820). This letter will allow you to begin marketing your device as described in your Section 510(k) premarket notification. The FDA finding of substantial equivalence of your device to a legally marketed predicate device results in a classification for your device and thus, permits your device to proceed to the market.
If you desire specific advice for your device on our labeling regulation (21 CFR Parts 801 and 809), please contact the Office of In Vitro Diagnostic Device Evaluation and Safety at (301) 796-5450. Also, please note the regulation entitled, "Misbranding by reference to premarket notification" (21 CFR Part 807.97). For questions regarding the reporting of adverse events under the MDR regulation (21 CFR Part 803), please go to
http://www.fda.gov/MedicalDevices/Safety/ReportaProblem/default.htm for the CDRH's Office of Surveillance and Biometrics/Division of Postmarket Surveillance.
You may obtain other general information on your responsibilities under the Act from the Division of Small Manufacturers, International and Consumer Assistance at its toll-free number (800) 638-2041 or (301) 796-7100 or at its Internet address http://www.fda.gov/cdrh/industry/support/index.html.
Sincerely yours,
m chan
Maria M. Chan, Ph.D. Director Division of Immunology and Hematology Devices Office of In Vitro Diagnostic Device Evaluation and Safety Center for Devices and Radiological Health
§ 866.4700 Automated fluorescence
in situ hybridization (FISH) enumeration systems.(a)
Identification. An automated FISH enumeration system is a device that consists of an automated scanning microscope, image analysis system, and customized software applications for FISH assays. This device is intended for in vitro diagnostic use with FISH assays as an aid in the detection, counting and classification of cells based on recognition of cellular color, size, and shape, and in the detection and enumeration of FISH signals in interphase nuclei of formalin-fixed, paraffin-embedded human tissue specimens.(b)
Classification. Class II (special controls). The special control is FDA's guidance document entitled “Class II Special Controls Guidance Document: Automated Fluorescencein situ Hybridization (FISH) Enumeration Systems.” See § 866.1(e) for the availability of this guidance document.