Search Results

The Vysis® AutoVysion™ System is an automated scanning microscope and image analysis system. It is intended for in vitro diagnostic use with the Vysis® PathVysion® HER-2 DNA Probe Kit to aid in the detection and enumeration of FISH signals in interphase nuclei. and to determine the LSI® HER-2 to CEP® 17 signal ratio of the HER-2/neu gene via FISH in formalinfixed, paraffin-embedded human breast cancer tissue specimens. The AutoVysion System is intended to reduce overall hands-on time by performing automated enumeration. For a small percentage of samples (less than 7%) manual enumeration may be required.

The Vysis® AutoVysion™ System is an adjunctive computer-assisted methodology to assist in the acquisition and measurement of images from microscope slides of formalin-fixed, paraffin-embedded breast cancer tissue sections for the presence of amplified HER-2/neu gene. The Vysis® AutoVysion™ System is intended for use as aid in determining HER-2/neu amplification status. in coniunction with optional manual visualization directly through the fluorescence microscope.

When used with the Vysis® PathVysion® HER-2 DNA Probe Kit, the Vysis® AutoVysion™ System is indicated for use as
a) an adjunct to existing clinical and pathologic information currently used as prognostic factors in stage II, node-positive breast cancer patients
b) an aid to predict disease-free and overall survival in patients with stage II. node positive breast cancer treated with adjuvant cyclophosphamide, doxorubicin and 5-fluorouracil (CAF) chemotherapy; and,
c) aid in the assessment of patients for whom HERCEPTIN® (Trastuzumab) treatment is being considered (see HERCEPTIN package insert).

The Vysis® AutoVysion™ System consists of an automated fluorescence microscope with motorized scanning stage, a large-format monochrome CCD camera, computer and scanning and assay specific analysis software. The microscope is equipped with a mercury arc lamp for fluorescence epi-illumination; three single-pass fluorescence filter sets for DAPI, SpectrumGreen™ (SG) and SpectrumOrange™ (SO) and a triple-pass fluorescence filter set for DAPI/SG/SQ, all mounted in a motorized filter turret; 10x and 40x objectives in a motorized objective turret; 10x eyepieces; a CCD camera; and a motorized scanning stage that holds up to 8 slides. Images of single fluorescence colors are captured by the CCD camera and transferred to the computer. All functions are controlled by the System software.

Here is a summary of the acceptance criteria and the study proving the device meets them, based on the provided text:

1. Table of Acceptance Criteria and Reported Device Performance

2. Sample Size Used for the Test Set and the Data Provenance

• Reproducibility Test Set: 36 specimen slides prepared from four human breast tissue specimens (one normal, one borderline, one moderate, one high amplification). Each site received three of each specimen randomized over three days.
• Method Comparison Test Set: 234 clinical slides from 39 tumors with varying levels of HER-2/neu copy number.
• Data Provenance: Not explicitly stated, but it conducted at "3 clinical sites" and refers to "clinical slides from 39 tumors," implying a source from a clinical setting. It is a retrospective analysis in the sense that the slides were prepared "prior to shipment to the study sites" and then retrospectively analyzed using both methods. However, the exact country of origin or broader demographic information is not provided.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and the Qualifications of Those Experts

• For the reproducibility and method comparison studies, the "ground truth" or reference method was conventional manual microscopy performed in accordance with Vysis® PathVysion® HER-2 DNA Probe Kit.
• The study states: "A qualified user visually inspects the tumor regions...identifies areas of tumor invasion...and records the coordinates." and "Final review and reporting of sample results is performed by a qualified user."
• It also states that the predicate method was "manual microscopy performed in accordance with Vysis® PathVysion® HER-2 DNA Probe Kit." This implies that the standard manual reading by trained personnel using the established kit constitutes the reference.
• The number of experts/manual readers for the comparison is not explicitly given, but it implies standard clinical practice where at least one "qualified user" would perform the manual enumeration. For the "two false positive results," the "manual" repetition implies multiple manual reads.
• Qualifications of experts: Described as "qualified user" or standard manual enumeration. Specific qualifications (e.g., "radiologist with 10 years of experience") are not provided.

4. Adjudication Method for the Test Set

• No formal adjudication method (e.g., 2+1, 3+1) is explicitly described for the initial ground truth establishment. The "ground truth" is defined by the results of "conventional manual microscopy performed in accordance with Vysis® PathVysion® HER-2 DNA Probe Kit." This suggests a single or standard manual read serves as the reference.
• For the two "false positive" discrepancies, a re-testing protocol was used where samples were "repeated six times manually and by the scanner," which could be considered a form of informal adjudication/re-evaluation for specific discrepant cases.

5. If a Multi Reader Multi Case (MRMC) Comparative Effectiveness Study was Done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance

• No, a MRMC comparative effectiveness study was not performed in the context of human readers improving with AI vs without AI assistance.
• This study is a standalone performance study that compares the automated system's performance directly against the manual enumeration (the gold standard at the time).
• The device is described as an "adjunctive computer-assisted methodology to assist in the acquisition and measurement of images...in conjunction with optional manual visualization directly through the fluorescence microscope." However, the study presented compares the automated system's result to manual enumeration rather than measuring how human performance changes when using the system as an aid.

6. If a Standalone (i.e. algorithm only without human-in-the loop performance) was done

• Yes, a standalone performance study was done. The "Method comparison" section directly compares the "Vysis® AutoVysion™ System produced HER-2 to CEP 17 signal ratio" to "manually enumerated... HER-2 to CEP 17 signal ratio."
• The system operates "fully automatically" once target areas are identified by a user, then "hybridization signals in each area are detected and enumerated automatically." Final review and reporting are performed by a qualified user, but the core enumeration is automated. The performance metrics reported (concordance, agreement, bias) are for this standalone automated enumeration process compared to manual.

7. The Type of Ground Truth Used (expert consensus, pathology, outcomes data, etc.)

• The ground truth used was expert manual enumeration (or "conventional manual microscopy") following the Vysis® PathVysion® HER-2 DNA Probe Kit protocol. This is considered the established clinical method for determining HER-2/neu amplification status at the time.

8. The Sample Size for the Training Set

• The document does not specify the sample size for the training set for the AutoVysion™ System. It only describes the test sets used for analytical performance evaluations.

9. How the Ground Truth for the Training Set Was Established

• The document does not describe how the ground truth for any potential training set was established. It focuses solely on the validation/performance evaluation of the already developed system.

Ask a specific question about this device

The UroVysion Bladder Cancer Recurrence Kit (UroVysion Kit) is designed to detect aneuploidy for chromosomes 3, 7, 17, and loss of the 9p21 locus via fluorescence in situ hybridization (FISH) in urine specimens from subjects with transitional cell carcinoma of the bladder. Results from the UroVysion Kit are intended for use as a noninvasive method for monitoring for tumor recurrence in conjunction with cystoscopy in patients previously diagnosed with bladder cancer.

The UroVysion Kit is based upon fluorescence in situ hybridization (FISH) DNA probe technology. The UroVysion probes are fluorescently labeled nucleic acid probes for use in in situ hybridization assays on urine specimens fixed on slides. The UroVysion Kit consists of a 4-color, four-probe mixture of DNA probe sequences homologous to specific regions on chromosomes 3, 7, 9, and 17. The UroVysion probe mixture consists of Chromosome Enumeration Probe (CEP®) 3 SpectrumRed™, CEP 7 SpectrumGreen™, CEP 17 SpectrumAqua™ and Locus Specific Identifier (LSI®) 9p21 SpectrumGold™.

Here's an analysis of the acceptance criteria and the study that proves the UroVysion Bladder Cancer Recurrence Kit meets those criteria, based on the provided text:

1. Table of Acceptance Criteria and Reported Device Performance

The document doesn't explicitly state quantitative acceptance criteria in a dedicated section. However, based on the "Substantial Equivalence vs. BTAstat Test" section, the key performance metric for substantial equivalence appears to be that the 95% Confidence Intervals (CIs) for UroVysion's agreement percentages (positive, negative, and overall) are greater than the BTAstat test's scores minus 15%.

For other aspects, the "acceptance criteria" are implied by the reported performance deemed satisfactory for marketing authorization.

2. Sample Size Used for the Test Set and Data Provenance

• Specificity Study (Non-cancer patients):
  • Sample Size: 315 patient visits initially, resulting in 309 usable office visits and ultimately 275 unique patients yielding informative results. The data points used for specificity calculation totaled 357 (some patients had multiple conditions).
  • Data Provenance: Multi-center, prospective study. Patients were "healthy volunteers and urology patients without prior history or clinical evidence of bladder cancer." (implicitly from the US, given FDA submission context).
• Performance vs. Standard of Care (Bladder cancer recurrence monitoring):
  • Sample Size: 309 patient visits initially, resulting in 251 usable office visits and 176 unique patients with informative FISH results.
  • Data Provenance: Multi-center, prospective, longitudinal study. Patients had a "history of bladder cancer." (implicitly from the US, given FDA submission context).
• Reproducibility: 4 specimens from bladder carcinoma cell lines.
• Interference/Preservative: Three voided urine pools from normal healthy volunteers, spiked with various substances.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts

The document does not explicitly state the number of experts used to establish the ground truth or their specific qualifications (e.g., "radiologist with 10 years of experience").

• For the Performance vs. Standard of Care study, the comparative reference was "cystoscopy followed by histology." Histology is typically interpreted by pathologists. The document mentions "central pathologist" in the longitudinal study details (Page 18), but doesn't specify how many or their experience level.

4. Adjudication Method for the Test Set

The document does not explicitly detail an adjudication method (e.g., 2+1, 3+1) for the ground truth establishment.

• For the "Performance vs. Standard of Care" study:
  • "The comparative reference used for all percent agreement calculations was cystoscopy with histology confirmation for positive or suspicious cystoscopies."
  • "If a patient had a positive cystoscopy but histology was absent (e.g., the lesion was fulgurated), then the specimen was considered positive for bladder cancer."
  • "If a test had a suspicious cystoscopy but histology was absent, then the case was omitted from analysis."
    This indicates a defined hierarchy and decision rule for establishing the ground truth (cystoscopy + histology).

5. Multi Reader Multi Case (MRMC) Comparative Effectiveness Study

A MRMC study was not explicitly done comparing human readers with and without AI assistance, because the device is a Fluorescence in situ hybridization (FISH) DNA probe technology, not an AI-assisted diagnostic tool.

However, the study did compare the UroVysion Kit's performance to two other human-interpreted methods:

• BTAstat test (lateral flow assay, antigen-specific antibodies, qualitative interpretation)
• Cytology (visual interpretation of cells)

The study demonstrated the superiority of the UroVysion Kit over BTAstat and cytology, particularly in agreement of positive results across various tumor stages and grades. For example:

• Overall Agreement of (+) Results:
  • UroVysion: 71.0%
  • BTAstat: 50.0%
  • Cytology: 26.2%
• For patients treated with BCG:
  • UroVysion: 92.3%
  • BTAstat: 69.2%
  • Cytology: 30.8%

The basis for substantial equivalence was to demonstrate that the UroVysion Kit's 95% CIs for agreement were greater than the BTAstat test's score minus 15%, which it met across positive, negative, and overall agreement categories. This showcases a significant effectiveness improvement compared to a legally marketed predicate directly by the device itself, rather than human readers improving with AI assistance.

6. Standalone (Algorithm Only Without Human-in-the-Loop Performance) Study

The UroVysion Kit is a FISH assay that involves human visual interpretation by an analyst to recognize fluorescent signals. Therefore, its performance is inherently human-in-the-loop, and a standalone (algorithm only) performance study, in the context of AI, is not applicable. The detection relies on DNA probes and fluorescence, with an analyst visually counting signals, rather than an automated algorithm making the final call.

7. Type of Ground Truth Used

• Specificity Study: Ground truth was "healthy volunteers and urology patients without prior history or clinical evidence of bladder cancer."
• Performance vs. Standard of Care Study: Ground truth for bladder cancer recurrence was established by cystoscopy followed by histology confirmation. If cystoscopy was positive but histology was absent (e.g., lesion fulgurated), it was considered positive. Suspicious cystoscopies without histology were omitted. This can be categorized as a form of clinical outcome/pathology-confirmed diagnosis.
• Reproducibility and Analytical Specificity: Used cultured human bladder carcinoma cell lines (positive target) and normal lymphoblast cell lines (negative target), and metaphase spreads for structural analysis.

8. Sample Size for the Training Set

The document describes premarket clinical studies and does not refer to a "training set" in the context of machine learning. The studies are for device validation.

9. How the Ground Truth for the Training Set Was Established

As there is no mention of a "training set" for an AI algorithm, this question is not applicable to the provided document. The studies describe validation of a diagnostic kit based on established biological principles (FISH technology).

Ask a specific question about this device

The UroVysion Bladder Cancer Recurrence Kit (UroVysion Kit) is designed to detect aneuploidy for chromosomes 3, 7, 17, and loss of the 9p21 locus via fluorescence in situ hybridization (FISH) in urine specimens from subjects with transitional cell carcinoma of the bladder. Results from the UroVysion Kit are intended for use as a noninvasive method for monitoring for turnor recurrence in conjunction with cystoscopy in patients previously diagnosed with bladder cancer.

The UroVysion Kit is based upon fluorescence in situ hybridization (FISH) DNA probe technology. The UroVysion probes are fluorescently labeled nucleic acid probes for use in in situ hybridization assays on urine specimens fixed on slides. . The UroVysion Kit consists of a 4-color, four-probe mixture of DNA probe sequences homologous to specific regions on chromosomes 3, 7, 9, and 17. The UroVysion probe mixture consists of Chromosome Enumeration Probe (CEP®) 3 SpectrumRed™, CEP 7 SpectrumGreen™, CEP 17 SpectrumAqua™, and Locus Specific Identifier (LSI®) 9p21 SpectrumGold TM .

The UroVysion Bladder Cancer Recurrence Kit is intended for use as a noninvasive method for monitoring for tumor recurrence in conjunction with cystoscopy in patients previously diagnosed with bladder cancer. The study demonstrated that the device is substantially equivalent to the predicate device, the Bard® (Bion) BTAstat™ Test, and meets the acceptance criteria for specificity and performance compared to the standard of care (cystoscopy/histology).

Acceptance Criteria and Reported Device Performance

Study Information

1. Sample size used for the test set and the data provenance:

   • Specificity Study: 309 usable office visits (resulting in 357 data points due to patients having multiple conditions) from healthy volunteers and urology patients without a prior history or clinical evidence of bladder cancer. This was a multi-center, prospective study. The country of origin is not explicitly stated but implies US clinical sites due to FDA submission.
   • Performance vs. Standard of Care Study: 251 usable office visits (representing 176 unique patients) from patients with a history of bladder cancer. This was a multi-center, prospective, longitudinal study conducted at 21 investigation sites. The country of origin is not explicitly stated but implies US clinical sites.
   • Reproducibility (Cell Lines): 80 specimens prepared from human bladder carcinoma cell lines.
   • Interference Study: Three voided urine pools from normal healthy volunteers, spiked with 29 different substances.
   • HYBrite/VP 2000 Validation: Three human urine pools from normal donors.

2. Number of experts used to establish the ground truth for the test set and the qualifications of those experts:

   • The document implies that the "standard of care" (cystoscopy with histology confirmation) was the ground truth. While no specific number of experts is given for the histological analysis, it is implicitly performed by qualified pathologists, which is standard practice for histology confirmation. No specific qualifications (e.g., 10 years of experience) are provided for these pathologists.

3. Adjudication method for the test set:

   • For the "Performance vs. Standard of Care" study, the comparative reference for all percent agreement calculations was cystoscopy with histology confirmation for positive or suspicious cystoscopies.
   • If a patient had a positive cystoscopy but histology was absent (e.g., lesion fulgurated), the specimen was considered positive for bladder cancer.
   • If a test had a suspicious cystoscopy but histology was absent, the case was omitted from analysis.
   • This indicates a hierarchical adjudication or ground truth definition rather than a consensus among multiple readers of the test device's results.

4. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:

   • No MRMC comparative effectiveness study involving human readers and AI assistance was conducted. The device is a diagnostic kit (FISH probes), and its interpretation relies on visual recognition by an analyst. The studies compare the UroVysion kit's performance against the standard of care (cystoscopy/histology) and the predicate device (BTAstat), and there is no mention of AI.

5. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done:

   • The UroVysion Kit requires visual analysis by an analyst to recognize fluorescent signals on chromosomes. Therefore, this is not a standalone algorithm-only device. The "analyst visually recognizes chromosomes" (page 1) indicating human-in-the-loop performance.

6. The type of ground truth used (expert consensus, pathology, outcomes data, etc.):

   • Pathology/Outcomes Data: The primary ground truth for the "Performance vs. Standard of Care" study was cystoscopy with histology confirmation. For cases where biopsy was not performed but cystoscopy was positive, it was still considered positive. The longitudinal study also used recurrence confirmed by cystoscopy/histology as an outcome.

7. The sample size for the training set:

   • The document does not explicitly describe a "training set" in the context of machine learning. The studies described are for validation/testing of the UroVysion Kit itself. The device is a FISH-based diagnostic kit, not an AI/ML algorithm that undergoes a distinct training phase with a labeled dataset. Its "training" would align more with its development and optimization, rather than a quantifiable dataset used for algorithm training.

8. How the ground truth for the training set was established:

   • As there is no explicit "training set" for an AI/ML algorithm described, this question is not directly applicable. The performance of the UroVysion Kit itself (hybridization efficiency, specificity, etc.) was established through laboratory tests and clinical studies, where outcomes like histology served as the reference standard.

Ask a specific question about this device

Ask a specific question about this device

Ask a specific question about this device

The AneuVysion™ (CEP 18, X, Y-alpha satellite, LSI 13 and 21) Multicolor Probe Panel is intended to use CEP 18/X/Y probe to detect alpha satellite sequences in the centromere regions of chromosomes 18, X, and Y, and LSI 13/21 probe to detect the 13q14 region and the 21q22.13 to 21q22.2 region. The AneuVysion™ kit is indicated for use as an adjunct to standard cytogenetic metaphase analysis for identifying and enumerating chromosomes 13, 18, 21, X, and Y via fluorescence in situ hybridization (FISH) in metaphase cells and interphase nuclei obtained from amniotic fluid in subjects with presumed high risk pregnancies. It is not intended to be used as a stand alone assay for test reporting. FISH results are intended to be reported and interpreted only in conjunction with results of standard cytogenetic analysis, performed concurrently, utilizing the same patient specimen. FISH results should not be reported prior to standard cytogenetic results except in instances where reporting of FISH results alone is medically indicated or standard cytogenetic results are not available, e.g., culture failure. Reporting and interpretation of FISH should be consistent with professional standards of practice! This device is intended for use only with amniocyte cells; it is not intended for and has not been validated for use with other test matrices. This FISH assay will not detect the presence of structural chromosome abnormalities frequently associated with birth defects. This FISH assay will be performed in cytogenetics laboratories.

The AneuVysion™ kit is a combination of two DNA probe mixtures; CEP 18/X/Y and LSI 13/21. The CEP 18/X/Y probe is a mixture of directly labeled fluorescent DNA probes specific for the D18Z1, DXZ1 and DYZ3 regions of chromosomes 18, X, and Y respectively. The LSI 13/21 probe contains a mixture of unique DNA sequences that hybridize in the 13q14 region of chromosome 13, and unique DNA sequences complementary to the D21S259, D21S341, and D21S342 loci contained within the 21q22.13 to 21q22.2 region on the long arm of chromosome 21. The LSI 13 probe was created from a set of overlapping clones which contain the entire RB-1 gene as well as regions extending beyond the gene on both sides. The probe extends beyond the 180 kb RB-1 gene for 110-170 kb in the 5' direction and approximately 120 kb in the 3' direction; the entire probe is 410-470 kb. CEP 18/X/Y is an aqua, green, and orange tri-color probe mixture and LSI 13/21 is a green and orange dual-color probe mixture.

Table of Acceptance Criteria and Reported Device Performance

Study Details Proving Device Meets Acceptance Criteria:

The information provided describes several studies that collectively demonstrate the performance of the AneuVysion™ kit.

1. Sample Size used for the test set and the data provenance:

   • Analytical Specificity: 705 metaphase spreads.
   • Reproducibility (Pivotal Study): Not explicitly stated how many individual specimens, but 24 measurements were taken for each of the four specimen types (100% XY, 10% X/90% XX, 17% X/47% XX/36% XXX, 50% XY+21/50% XY). The specimens were "cultured human amniocyte specimens."
   • Methods Comparison / Clinical Specimens (Pivotal Multi-center Study):
     • Total Patients: 1516 patients.
     • Total Amniocyte Specimens: 2,238 specimens.
     • Aneuploid Cases: 861 cases.
     • Mosaic Cases: 62 cases.
     • Pseudomosaic Cases: 76 cases.
     • Euploid Cases: 504 cases.
     • Pairwise Comparison (Cultured vs. Uncultured): 722 patients.
   • Data Provenance: The studies were conducted at "Thirty one investigation sites" in a "multi-center, blinded, controlled, comparative study." The specimens were "human amniotic fluid specimens obtained from a total of 1516 patients." The origin seems to be implicitly from the countries where these investigation sites are located, likely including the USA given the FDA submission. The study is prospective in nature as it involved collecting and analyzing patient samples for the purpose of the study.

2. Number of experts used to establish the ground truth for the test set and the qualifications of those experts:

   • The ground truth for the clinical specimens was established by "standard cytogenetic analysis." While the document doesn't explicitly state the number of experts, cytogenetic analysis is a specialized field, and it's implied that qualified cytogeneticists performed this analysis at each of the "Thirty one investigation sites." The qualifications would inherently be those required to perform and interpret standard cytogenetic metaphase analysis.

3. Adjudication method for the test set:

   • The document states that the "results of interphase FISH analysis were compared to standard cytogenetics." Standard cytogenetic analysis served as the gold standard. There is no mention of a specific adjudication method (like 2+1 or 3+1 consensus) for discrepancies between FISH and cytogenetics; rather, standard cytogenetics was the reference.

4. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:

   • This is not an AI-based device. It is a DNA probe kit (FISH assay) for manual interpretation by human cytogeneticists. Therefore, an MRMC study comparing human readers with and without AI assistance is not applicable and was not performed.

5. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done:

   • No. The AneuVysion™ kit "is not intended to be used as a stand alone assay for test reporting." FISH results are "intended to be reported and interpreted only in conjunction with results of standard cytogenetic analysis, performed concurrently, utilizing the same patient specimen." The device is a diagnostic reagent, requiring human interpretation of the hybridization signals on cells.

6. The type of ground truth used:

   • The primary ground truth used for the clinical validation was standard cytogenetic analysis (karyotyping of metaphase spreads). This is widely considered the gold standard for detecting chromosomal abnormalities.

7. The sample size for the training set:

   • The document does not explicitly describe a separate "training set" for the AneuVysion™ kit. As this is a probe-based assay that relies on established biological principles of hybridization and is interpreted by human experts, there isn't a machine learning algorithm that requires a traditional training set. The various studies on hybridization efficiency, analytical sensitivity/specificity, and reproducibility likely served to establish and refine the assay's performance characteristics rather than "train" an algorithm.

8. How the ground truth for the training set was established:

   • See point 7. Since a traditional training set for an algorithm is not applicable, the concept of establishing ground truth for it is also not applicable in the context of this device. The performance characteristics were established via validation studies using reference methods (like G-banding for specificity and standard cytogenetics for clinical concordance).

Ask a specific question about this device

This assay is designed to provide a reliable method for the simultaneous detection and enumeration of X and Y chromosomes in interphase nuclei and metaphase spreads in bone marrow by fluorescence in situ hybridization (FISH).

The CEP X/Y probe is a combination of CEP X SpectrumOrange and CEP Y SpectrumGreen continuous DNA probes for the centromeric region of chromosome X and the satellite III DNA at the Yq12 region of chromosome Y.

Here's a breakdown of the acceptance criteria and study details for the CEP X SpectrumOrange/Y SpectrumGreen DNA Probe Kit, based on the provided text:

1. Table of Acceptance Criteria and Reported Device Performance

Note: The document implicitly defines acceptance criteria through stated performance goals (e.g., "<2% cells with only one signal is a realistic standard of acceptance") or by reporting outcomes that confirm no issues (e.g., no cross-hybridization).

2. Sample Sizes Used for the Test Set and Data Provenance

• Hybridization Efficiency: 143 bone marrow specimens.
• Analytical Specificity: 65 metaphase spreads.
• Reproducibility: Mixtures with various XY/XX percentages (0%/100%, 1%/99%, 5%/95%, 95%/5%, 99%/1%, 100%/0%). Specific n values for each mixture are in Table 1 (ranging from 10 to 24). Additionally, four bone marrow specimens with approximately 0%/100%, 1%/99%, 5%/95%, and 95%/5% were prepared and analyzed at one site.
• Methods Comparison (Clinical Specimens):
  • Opposite-sex BMT: 143 patients (72 males and 71 females were recipients of opposite-sex BMTs). 143 specimens collected, but only 141 had metaphase spreads for FISH analysis.
  • Like-sex BMT: 153 patients.
• Data Provenance: The document does not explicitly state the country of origin. It mentions a "multi-center" study and "three sites" for the clinical specimen comparison. The study is described as a "pivotal study," and the data appears to be retrospective as it involves "bone marrow specimens, which were previously evaluated by standard cytogenetic analysis."

3. Number of Experts Used to Establish Ground Truth for the Test Set and Their Qualifications

The document does not explicitly state the number of experts or their qualifications for establishing ground truth. However:

• For analytical specificity, the 65 metaphase spreads were "examined sequentially by G-banding to identify chromosomes X and Y, followed by FISH." This G-banding analysis would have been performed by a cytogeneticist.
• For the Methods Comparison (Clinical Specimens), standard cytogenetic analysis was used as the comparator for previously collected bone marrow specimens. This implies that the ground truth for these clinical specimens was established by cytogeneticists through traditional karyotyping.

4. Adjudication Method

The document does not mention an explicit adjudication method (e.g., 2+1, 3+1). It describes "inter-observer" assessments for reproducibility, implying multiple observers, but not a formal adjudication process to resolve discrepancies for ground truth. For the clinical studies, standard cytogenetic analysis served as the established ground truth for comparison.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

No, a multi-reader multi-case (MRMC) comparative effectiveness study comparing human readers with AI assistance versus human readers without AI assistance was not done. This study is evaluating a standalone diagnostic kit (a DNA probe kit), not an AI-assisted diagnostic tool.

6. Standalone Performance Study

Yes, the studies described are standalone (algorithm/device only) performance studies. The CEP X/Y probe kit is the device, and its performance characteristics (hybridization efficiency, sensitivity, specificity, reproducibility) and agreement with standard cytogenetics are being evaluated. It is not an AI algorithm with a human-in-the-loop.

7. Type of Ground Truth Used

• Hybridization Efficiency, Analytical Sensitivity, Reproducibility: These were assessed against known compositions (e.g., 0% XX, 1% XY specimen, various XY/XX mixtures). The "ground truth" here is the expected percentage or presence/absence of signals based on the prepared specimens.
• Analytical Specificity: Ground truth was established by sequential G-banding to identify chromosomes X and Y.
• Methods Comparison (Clinical Specimens): The ground truth was established by standard cytogenetic analysis on bone marrow specimens from BMT recipients. This is a well-established diagnostic method.

8. Sample Size for the Training Set

The document does not mention a training set in the context of machine learning or AI. This is a DNA probe kit, not an AI model.

9. How the Ground Truth for the Training Set Was Established

Not applicable, as there is no training set for an AI model.

Ask a specific question about this device

The CEP 12 SpectrumOrange Direct Labeled Chromosome Enumeration DNA Probe Kit is designed to be an adjunct to standard cytogenetic analysis for in vitro diagnostic use in identifying and enumerating chromosome 12 via fluorescence in situ hybridization (FISH) in interphase nuclei of cells obtained from peripheral blood lymphocytes in patients with chronic lymphocytic leukemia (CLL). Results from the CEP 12 SpectrumOrange Direct Labeled Chromosome Enumeration DNA Probe Kit are intended for use in conjunction with standard cytogenetics and for further assessing the trisomy 12 status in normal and abnormal tissue specimens characterized by standard cytogenetics, in patients with and/or without clinical symptoms of CLL. It is not intended to be a stand alone assay for test reporting.

The CEP 12 SpectumOrange DNA Probe is a SpectumOrange fluorescent labeled DNA probe specific for the centromeric region of chronosome 12. This assey is designed IDN provide a rapid and reliable method for the desertion and quantification of chromosome 12 In interphase nuclei by fluorescence in situ hybridization (FISH).

Here's a breakdown of the acceptance criteria and study information for the CEP 12 SpectrumOrange DNA Probe Kit, based on the provided text:

Acceptance Criteria and Reported Device Performance

Study Details

2. Sample Size Used for the Test Set and Data Provenance:

• Pivotal Study (Hybridization Efficiency): 402 peripheral blood specimens.
• Analytical Specificity: 56 metaphase spreads.
• Reproducibility Study:
  • 0% specimen: 22 replicates
  • 5% specimen: 23 replicates
  • 10% specimen: 23 replicates
  • 13% specimen: 24 replicates
  • Provenance: Hematologically derived human cells (mixtures with known percentages of trisomy 12).
• Clinical Specimens (Methods Comparison):
  • Total 402 peripheral blood specimens from patients with B-cell chronic lymphocytic leukemia (B-CLL).
  • Provenance: Multi-center study; specimens from three sites (Site 1: 97 specimens, Site 2: 205 specimens, Site 3: 100 specimens). One site (from the United Kingdom) provided 157 specimens.
  • Retrospective/Prospective: Not explicitly stated, but the collection of "peripheral blood specimens... from a total of 402 patients" suggests they were likely retrospective, selected for the study.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts:

• Analytical Specificity: Examination "sequentially by G-banding" followed by FISH. This implies cytogeneticists, but the number and specific qualifications are not detailed.
• Reproducibility Study: "Known percentages of trisomy 12" in the cell mixtures suggests a predefined truth, likely established by advanced cytogenetic techniques, but no "experts" are explicitly described for this part of ground truth establishment.
• Clinical Specimens (Methods Comparison): Standard cytogenetic analysis was used as the comparator (ground truth). Each site followed "its own in-house protocol for standard cytogenetic analysis." This implies trained cytogeneticists performed the standard cytogenetic analysis and interpretation due to the nature of G-banding and karyotyping. The number of experts per site is not specified, nor are their detailed qualifications (e.g., years of experience).

4. Adjudication Method for the Test Set:
The document does not explicitly describe a formal adjudication method (e.g., 2+1, 3+1) for the interpretation of either the FISH results or the standard cytogenetic results. For clinical specimens, it notes that "inter-site and observer-to-observer variations were observed" in the reproducibility study, suggesting independent assessments without a specific adjudication process mentioned for discrepancies in the broader clinical study.

5. Multi Reader Multi Case (MRMC) Comparative Effectiveness Study:
No, an MRMC comparative effectiveness study, in the sense of evaluating how much human readers improve with AI vs. without AI assistance, was not performed. This study compares a device (FISH probe) to a standard clinical method (cytogenetics), which are both human-interpreted methods, not an AI system.

6. Standalone (Algorithm Only Without Human-in-the-Loop Performance) Study:
Yes, the analytical sensitivity, specificity, and reproducibility studies for the CEP 12 SpectrumOrange DNA Probe Kit essentially represent standalone performance where the probe kit itself is being evaluated. The reading of the FISH signals is a human-in-the-loop step, but the performance metrics (hybridization efficiency, LOD, cross-hybridization) are inherent to the probe and its interaction with the biological sample.

7. Type of Ground Truth Used:

• Analytical Specificity: G-banding (a traditional cytogenetic technique for chromosome identification).
• Reproducibility: "Known percentages of trisomy 12" in cell mixtures, implying a highly characterized true state.
• Clinical Specimens (Methods Comparison): Standard cytogenetic analysis (G-banded karyotyping).

8. Sample Size for the Training Set:
The document does not mention a training set or machine learning/AI algorithms that would require one. This submission is for a DNA probe kit, not an AI-powered diagnostic device.

9. How the Ground Truth for the Training Set Was Established:
Not applicable, as there is no training set mentioned.

Ask a specific question about this device

This assay is designed to provide a method for the detection and quantification of chromosome 8 in both interphase nuclei and metaphase spreads by fluorescence in situ hybridization (FISH).

The CEP 8 SpectrumOrange DNA Probe is a SpectrumOrange fluorescent labeled DNA probe specific for the centromeric region of chromosome 8.

Here's a breakdown of the acceptance criteria and the study that proves the device meets them, based on the provided text:

Acceptance Criteria and Device Performance for CEP 8 SpectrumOrange DNA Probe Kit

1. Table of Acceptance Criteria and Reported Device Performance

2. Sample Size Used for the Test Set and Data Provenance

• Analytical Sensitivity & Specificity:
  • Hybridization Efficiency: Pilot study had 35 bone marrow (BM) specimens. Pivotal study had 60 BM specimens.
  • Analytical Specificity: 62 metaphase spreads.
  • Data Provenance: Not explicitly stated (e.g., country of origin), assumed to be within the US given FDA submission. It's retrospective for analytical specificity (archived metaphase spreads).
• Reproducibility:
  • Pilot Study: One normal bone marrow specimen. N=24 observations/measurements.
  • Pivotal Study: One low-level (approximately 7%) trisomy 8 bone marrow specimen. N=24 observations/measurements.
  • Data Provenance: Not explicitly stated, likely multi-site within the US. Retrospective (single specimens used for repeated testing).
• Clinical Performance (Methods Comparison):
  • Test Set Size: 368 archived bone marrow specimens.
  • Data Provenance: Multi-center study involving four laboratories providing specimens (Site 1: 101, Site 2: 57, Site 3: 130, Site 4: 80, with Site 4 specimens analyzed at Site 3). The source of the specimens is described by the diseases: Acute myeloid leukemia (AML), Myeloproliferative disorder (MPD), Myelodysplastic syndrome (MDS), Chronic myelogenous leukemia (CML), Hematological disorder, not otherwise specified (HIDNOS). The data is retrospective as it used "archived bone marrow specimens." The country of origin is not specified but the study was submitted to the FDA.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications

• The ground truth for the clinical studies (Methods Comparison) was established by "standard cytogenetic analysis" performed by the participating laboratories.
• The text does not specify the number of individual experts per lab or their specific qualifications (e.g., "radiologist with 10 years of experience"). However, "standard cytogenetic analysis" implies trained and qualified cytogeneticists or laboratory personnel are performing the traditional karyotyping and interpretation.

4. Adjudication Method for the Test Set

• The text does not explicitly describe an adjudication method for the ground truth (standard cytogenetics) in the methods comparison study. It implies that the "standard cytogenetic analysis" results from each participating laboratory were accepted as the ground truth.
• For the device's own FISH analysis, inter-observer and inter-site variability were assessed in the reproducibility studies, suggesting that multiple observers/sites independently evaluated the samples, but it doesn't detail a formal adjudication process for discordant results.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

• No, a MRMC comparative effectiveness study was not explicitly conducted to assess human readers' improvement with AI vs. without AI assistance.
• This device is a DNA probe kit for Fluorescence In Situ Hybridization (FISH), which is a laboratory assay where trained personnel visually count signals or use automated enumeration systems. It is not an "AI" device as described in modern AI/ML contexts that would assist radiologists or other human readers. The study compares the performance of the FISH assay (both interphase and metaphase interpretation) against the standard cytogenetic analysis.
• The reproducibility studies assessed inter-observer variability for the FISH assay itself, reflecting the inherent subjectivity of visual enumeration, but not the impact of an AI assistance tool on human performance.

6. Standalone (Algorithm Only Without Human-in-the-Loop Performance) Study

• Yes, standalone performance was assessed. The CEP 8 SpectrumOrange DNA Probe Kit is the "algorithm" in this context (or more accurately, the assay and its interpretation guidelines).
• The entire clinical methods comparison study (comparing FISH interphase and metaphase analysis to standard cytogenetics) represents the standalone performance of the assay and its associated interpretation criteria (e.g., 2.2% cutoff for interphase, >=2 tri-signaled metaphases for metaphase). The sensitivity, specificity, and correlation coefficients reported are for the device's performance when applied according to its instructions.

7. Type of Ground Truth Used

• For the clinical methods comparison study, the ground truth was expert consensus / established diagnostic method: Standard Cytogenetic Analysis. This method is referred to as "the standard of care."

8. Sample Size for the Training Set

• The text does not explicitly describe a separate training set in the way modern machine learning models would have one.
• The "pilot study" and "pivotal study" results for hybridization efficiency and reproducibility involve smaller numbers of specimens (35 BM, 60 BM, single normal and single trisomy 8 specimens) that could be considered part of the development and refinement of the assay and its interpretation rules, but not a formal "training set" for an algorithm. The 2.2% cutoff for interphase analysis appears to have been 'validated by the same pivotal clinical study,' suggesting it was developed and then verified within the context of the overall validation.

9. How the Ground Truth for the Training Set Was Established

• Since there's no explicitly defined "training set" for an algorithm in the modern sense, the concept of establishing ground truth for it doesn't directly apply here.
• However, the underlying biological understanding and the methodology of interpreting FISH signals (e.g., what constitutes a "tri-signaled nuclei") would be based on established cytogenetic principles and likely refined through internal studies during the probe's development, informed by standard cytogenetic analysis, which is the gold standard for chromosomal abnormalities. The cutoff values (like 2.2%) were likely empirically determined and then validated against the clinical performance.

Ask a specific question about this device