(452 days)
The BD MAX CT/GC/TV assay, as performed using the BD MAX System incorporates automated DNA extraction and real-time polymerase chain reaction (PCR) for the direct, qualitative detection of DNA from Chlamydia (CT), Neisseria gonorrhoeae (GC) and/or Trichomonas vaginalis (TV). The assay may be used for detection of CT and/or GC DNA in male urine specimens, and the detection of CT, GC and/or TV DNA in female urine specimens, cliniciancollected female endocervical swab speciment-collected vaginal swab specimens (in a clinical setting). The assay is indicated for use to aid in the diagnosis of chlamydial urogenital disease, gonococcal urogenital disease and/or trichomoniasis in asymptomatic and symptomatic individuals.
The BD MAX System and the BD MAX CT/GC/TV are comprised of an instrument with associated hardware and accessories, disposable microfluidic cartridges, master mixes, unitized reagent strips, and extraction reagents. The instrument automates sample preparation including target lysis, DNA extraction and concentration, reagent rehydration, and target nucleic acid amplification and detection using real-time PCR. The assay includes a Sample Processing Control (SPC) that is present in the Extraction Tube. The SPC monitors DNA extraction steps, thermal cycling steps, reagent integrity and the presence of inhibitory substances. The BD MAX System software automatically interprets test results. A test result may be called as POS, NEG or UNR for each of the assay's targets, based on the amplification status of the target and of the Sample Processing Control. IND (Indeterminate) or INC (Incomplete) results are due to BD MAX System failure.
The BD MAX CT/GC/TV assay, performed using the BD MAX System, is an in vitro diagnostic device for the direct, qualitative detection of DNA from Chlamydia trachomatis (CT), Neisseria gonorrhoeae (GC), and Trichomonas vaginalis (TV).
Here's an analysis of the acceptance criteria and the study proving the device meets them, based on the provided document:
1. Table of Acceptance Criteria and Reported Device Performance
The document does not explicitly present a table of acceptance criteria with corresponding performance metrics. However, the "Clinical Performance Studies" section implicitly establishes the performance expectations through its reported sensitivity and specificity values. The acceptance criteria can be inferred as the achieved sensitivity and specificity values for each pathogen across different specimen types and patient populations (asymptomatic and symptomatic).
For the purpose of this response, I will present the key performance metrics from the "Clinical Performance Studies" (Table 18) as the reported device performance, which are implicitly the performance targets the device met. The clinical performance is compared against a "Patient Infected Status (PIS)," which is a composite reference method.
Table: Acceptance Criteria (Inferred from Achieved Performance) and Reported Device Performance
| Target Organism | Specimen Type | Patient Symptom Status | Acceptance Criteria (Inferred from Study Results) | Reported Device Performance (95% CI) |
|---|---|---|---|---|
| Chlamydia trachomatis (CT) | Vaginal Swab | Asymptomatic | Sensitivity: ≥93.0% | |
| Specificity: ≥97.5% | Sensitivity: 100% (93.0-100) | |||
| Specificity: 98.7% (97.5-99.3) | ||||
| Symptomatic | Sensitivity: ≥94.0% | |||
| Specificity: ≥97.7% | Sensitivity: 98.9% (94.0-99.8) | |||
| Specificity: 98.6% (97.7-99.2) | ||||
| Endocervical Swab | Asymptomatic | Sensitivity: ≥84.1% | ||
| Specificity: ≥98.1% | Sensitivity: 94.1% (84.1-98.0) | |||
| Specificity: 99.1% (98.1-99.5) | ||||
| Symptomatic | Sensitivity: ≥90.3% | |||
| Specificity: ≥98.6% | Sensitivity: 96.6% (90.3-98.8) | |||
| Specificity: 99.4% (98.6-99.7) | ||||
| Female Urine | Asymptomatic | Sensitivity: ≥81.8% | ||
| Specificity: ≥99.0% | Sensitivity: 92.3% (81.8-97.0) | |||
| Specificity: 99.7% (99.0-99.9) | ||||
| Symptomatic | Sensitivity: ≥83.4% | |||
| Specificity: ≥98.6% | Sensitivity: 91.1% (83.4-95.4) | |||
| Specificity: 99.4% (98.6-99.7) | ||||
| Male Urine | Asymptomatic | Sensitivity: ≥92.3% | ||
| Specificity: ≥98.1% | Sensitivity: 98.6% (92.3-99.7) | |||
| Specificity: 99.5% (98.1-99.9) | ||||
| Symptomatic | Sensitivity: ≥88.7% | |||
| Specificity: ≥97.3% | Sensitivity: 94.6% (88.7-97.5) | |||
| Specificity: 99.3% (97.3-99.8) | ||||
| Neisseria gonorrhoeae (GC) | Vaginal Swab | Asymptomatic | Sensitivity: ≥73.0% | |
| Specificity: ≥99.3% | Sensitivity: 94.1% (73.0-99.0) | |||
| Specificity: 99.9% (99.3-100) | ||||
| Symptomatic | Sensitivity: ≥81.7% | |||
| Specificity: ≥99.3% | Sensitivity: 96.3% (81.7-99.3) | |||
| Specificity: 99.8% (99.3-99.9) | ||||
| Endocervical Swab | Asymptomatic | Sensitivity: ≥73.0% | ||
| Specificity: ≥99.5% | Sensitivity: 94.1% (73.0-99.0) | |||
| Specificity: 100% (99.5-100) | ||||
| Symptomatic | Sensitivity: ≥81.7% | |||
| Specificity: ≥99.4% | Sensitivity: 96.3% (81.7-99.3) | |||
| Specificity: 99.9% (99.4-100) | ||||
| Female Urine | Asymptomatic | Sensitivity: ≥67.2% | ||
| Specificity: ≥98.7% | Sensitivity: 88.9% (67.2-96.9) | |||
| Specificity: 99.5% (98.7-99.8) | ||||
| Symptomatic | Sensitivity: ≥87.9% | |||
| Specificity: ≥99.4% | Sensitivity: 100% (87.9-100) | |||
| Specificity: 99.9% (99.4-100) | ||||
| Male Urine | Asymptomatic | Sensitivity: ≥37.6% | ||
| Specificity: ≥99.1% | Sensitivity: 80.0% (37.6-96.4) | |||
| Specificity: 100% (99.1-100) | ||||
| Symptomatic | Sensitivity: ≥96.4% | |||
| Specificity: ≥98.7% | Sensitivity: 100% (96.4-100) | |||
| Specificity: 100% (98.7-100) | ||||
| Trichomonas vaginalis (TV) | Vaginal Swab | Asymptomatic | Sensitivity: ≥78.0% | |
| Specificity: ≥94.9% | Sensitivity: 93.1% (78.0-98.1) | |||
| Specificity: 97.5% (94.9-98.8) | ||||
| Symptomatic | Sensitivity: ≥91.9% | |||
| Specificity: ≥98.6% | Sensitivity: 96.7% (91.9-98.7) | |||
| Specificity: 99.5% (98.6-99.8) | ||||
| Endocervical Swab | Asymptomatic | Sensitivity: ≥82.8% | ||
| Specificity: ≥95.8% | Sensitivity: 96.6% (82.8-99.4) | |||
| Specificity: 98.2% (95.8-99.2) | ||||
| Symptomatic | Sensitivity: ≥86.7% | |||
| Specificity: ≥99.1% | Sensitivity: 92.7% (86.7-96.1) | |||
| Specificity: 99.8% (99.1-100) | ||||
| Female Urine | Asymptomatic | Sensitivity: ≥78.0% | ||
| Specificity: ≥95.8% | Sensitivity: 93.1% (78.0-98.1) | |||
| Specificity: 98.2% (95.8-99.2) | ||||
| Symptomatic | Sensitivity: ≥86.9% | |||
| Specificity: ≥99.1% | Sensitivity: 92.8% (86.9-96.2) | |||
| Specificity: 99.8% (99.1-100) |
Note: The "Acceptance Criteria" values are inferred as the lower bounds of the reported 95% Confidence Intervals for Sensitivity and Specificity, representing the minimum performance demonstrated to be acceptable.
2. Sample Size Used for the Test Set and Data Provenance
-
Test Set Sample Size:
- Female Subjects: 2,114 evaluable subjects for Chlamydia trachomatis (CT) and Neisseria gonorrhoeae (GC), and 1,291 of these for Trichomonas vaginalis (TV).
- Male Subjects: 892 evaluable subjects for CT and GC.
- Specimens for CT: 1,836 patient-collected vaginal swabs, 1,831 endocervical swabs, 1,849 female urine, and 830 male urine specimens.
- Specimens for GC: 1,836 patient-collected vaginal swabs, 1,824 endocervical swabs, 1,849 female urine, and 840 male urine specimens.
- Specimens for TV: 1,048 patient-collected vaginal swabs, 1,039 endocervical swabs, and 1,047 female urine specimens.
- Total specimens initially evaluated: 6,573 specimens.
-
Data Provenance: Retrospective and Prospective. The study mentions that it was a "multicenter study where clinical sites enrolled subjects and also performed testing." This suggests prospective collection of real-world samples for the clinical performance evaluation. The provenance is from "nine (9) geographically diverse clinical sites in North America."
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications
The document refers to a "Patient Infected Status (PIS)" as the reference method for establishing ground truth. The PIS is described as a "composite reference method algorithm." This suggests that the ground truth was not established by a panel of individual experts directly reviewing each case. Instead, it was based on results from multiple reference methods integrated by an algorithm. The specific number of experts or their qualifications involved in developing or validating this algorithm (if any) is not specified in this document.
4. Adjudication Method for the Test Set
The ground truth was established by a "composite reference method algorithm" (PIS). This implies an algorithmic adjudication rather than human expert adjudication. The study states, "This multicenter study evaluated results obtained with the BD MAX CT/GC/TV compared to reference methods defining the Patient Infected Status (PIS)." No specific human adjudication method (e.g., 2+1, 3+1) is mentioned, as the PIS serves as the "truth."
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done
No, an MRMC comparative effectiveness study was not explicitly mentioned or presented in the document. The study focuses on the standalone performance of the BD MAX CT/GC/TV assay against a defined Patient Infected Status (PIS). There is no comparison of human readers with versus without AI assistance.
6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) Was Done
Yes, the study primarily presents standalone performance of the BD MAX CT/GC/TV assay. The device description states, "The BD MAX System software automatically interprets test results. A test result may be called as POS, NEG or UNR for each of the assay's targets..." and "The BD MAX System performs results interpretation automatically." The clinical performance tables (Table 18) present the sensitivity and specificity of the device's automated results against the PIS, which represents a standalone evaluation of the algorithm's performance.
7. The Type of Ground Truth Used
The type of ground truth used is a "Patient Infected Status (PIS)" which is described as a "composite reference method algorithm." This indicates that the truth was derived from the results of multiple established laboratory reference methods, combined using a predefined algorithm, rather than directly from pathology, individual expert consensus, or outcomes data solely.
8. The Sample Size for the Training Set
The document does not explicitly state the sample size used for the training set for the BD MAX CT/GC/TV assay. This type of information (training set details) is often omitted in premarket notification summaries which focus on clinical validation data for regulatory approval. The "Analytical Performance" section (Precision, Reproducibility, Analytical Sensitivity, Analytical Specificity, Interfering Substances, Carryover/Cross-Contamination, Mixed Infection/Competitive Interference) describes laboratory-based studies used to characterize the assay's analytical capabilities, which might be considered part of the development and optimization process, but not a distinct "training set" in the context of machine learning model development. For in vitro diagnostic assays like this, the "training" aspect is more about optimizing reaction parameters and thresholds rather than training a machine learning model on patient data.
9. How the Ground Truth for the Training Set Was Established
Since a distinct "training set" for a machine learning model is not explicitly mentioned, and the device is an IVD assay based on PCR and automated interpretation, the concept of "ground truth for the training set" as it applies to AI/ML devices is not directly applicable here. For the analytical studies, the "ground truth" (e.g., in LoD or analytical specificity) would have been established by precisely creating samples with known concentrations of organisms or known non-target organisms, based on established laboratory methods and controls.
§ 866.3860
Trichomonas vaginalis nucleic acid assay.(a)
Identification. ATrichomonas vaginalis nucleic acid assay is a device that consists of primers, probes, enzymes, and controls for the amplification and detection of trichomonas nucleic acids in endocervical swabs, vaginal swabs, and female urine specimens, from women symptomatic for vaginitis, cervicitis, or urethritis and/or to aid in the diagnosis of trichomoniasis in asymptomatic women. The detection of trichomonas nucleic acids, in conjunction with other laboratory tests, aids in the clinical laboratory diagnosis of trichomoniasis caused byTrichomonas vaginalis .(b)
Classification. Class II (special controls). The special controls are set forth in FDA's guideline document entitled: “Class II Special Controls Guideline: Nucleic Acid Amplification Assays for the Detection ofTrichomonas vaginalis; Guideline for Industry and Food and Drug Administration Staff.” See § 866.1(e) for information on obtaining this document.