(88 days)
The BD ProbeTec™ ET Legionella pneumophila (LP) Amplified DNA Assay, for use with the BD ProbeTec ET System, employs Strand Displacement Amplification (SDA) technology for the direct qualitative detection of Legionella pneumophila DNA (serogroups 1-14) in sputum specimens from patients with a clinical suspicion of pneumonia. It is intended to aid in the presumptive diagnosis of Legionnaires' disease in conjunction with culture and other methods.
The BD ProbeTec™ ET LP Amplified DNA Assay is a new test designed for use on the BD ProbeTec™ ET System. The test is indicated for use with sputum specimens from patients with a clinical suspicion of pneumonia as an aid in the presumptive diagnosis of Legionnaires' disease in conjunction with culture and other methods. The BD ProbeTecTM ET LP Amplified DNA Assay is based on the simultaneous amplification and detection of target DNA sequences using nucleic acid primers and fluorescently-labeled detector probes in a process known as strand displacement amplification (SDA). The SDA reagents are dried in two separate disposable microwells. Processed sample containing DNA is added to a Priming Microwell, which contains the amplification primers, fluorescently-labeled detector probes, and other reagents necessary for amplification. Following incubation, the reaction mixture is transferred to an Amplification Microwell, which contains two enzymes (a DNA polymerase and a restriction endonuclease) necessary for SDA. The Amplification Microwells are sealed to prevent contamination and then incubated in a thermally controlled fluorescent reader, which monitors each reaction for the generation of amplified products. Each reaction coamplifies and detects an Internal Amplification Control (IAC), as well as the target DNA. The purpose of the IAC is to verify that proper conditions exist for amplification and to reduce the possibility of reporting a false negative result due to specimen inhibitors. The presence or absence of L. pneumophila DNA is determined by calculating PAT scores (Passes After Threshold) for the specimen based on predefined threshold values. The instrument automatically reports the results as positive, negative or indeterminate.
Here's an analysis of the acceptance criteria and the study that proves the device meets them, based on the provided text:
1. Table of Acceptance Criteria and Reported Device Performance
The document does not explicitly state pre-defined acceptance criteria (e.g., "The device must achieve a sensitivity of at least X%"). Instead, it presents the determined performance metrics from a retrospective study, which presumably were considered acceptable for market clearance given the "Substantial Equivalence" finding. For the purpose of this analysis, I will treat the reported performance as the de facto "met acceptance criteria" based on the FDA's clearance.
| Metric | Acceptance Criteria (Implicit) | Reported Device Performance (Retrospective Study) |
|---|---|---|
| Sensitivity (Agreement with Culture for Positive Samples) | High agreement expected | 91.3% (21/23) |
| Specificity (Agreement with Culture for Negative Samples) | High agreement expected | 86.7% (52/60) |
| Overall Agreement with Culture | High agreement expected | 88% (73/83) |
Note on Prospective Study: The prospective study found 100% (114/114) negative agreement with culture, but as there were no culture-positive specimens, sensitivity could not be estimated. This highlights the reliance on the retrospective study for key performance metrics.
2. Sample Sizes Used for the Test Set and Data Provenance
- Prospective Study Test Set:
- Sample Size: 114 sputum specimens from 114 patients.
- Data Provenance: Prospective, collected from seven clinical centers within the United States and Canada during the 2002-2003 respiratory season.
- Retrospective Study Test Set (to estimate sensitivity):
- Sample Size: 83 retrospective sputum specimens (23 positive for L. pneumophila by culture, 60 negative by culture).
- Data Provenance: Retrospective, acquired from clinical sites within the United States, Europe, and Canada.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts
The document does not specify the number of experts or their qualifications used to establish the ground truth for the test set.
4. Adjudication Method for the Test Set
The primary ground truth for both the prospective and retrospective studies was bacterial culture.
- For the retrospective study, in cases of discordance between the BD ProbeTec™ ET LP Amplified DNA Assay and culture, Polymerase Chain Reaction (PCR) testing was performed on some of the discordant samples for further investigation. This acted as a secondary, confirmatory method, particularly when the culture was positive but the assay negative, or vice-versa. However, PCR was not uniformly applied to all discordant samples (e.g., 6 of 8 culture-negative, assay-positive samples were tested with PCR, and 1 of 2 culture-positive, assay-negative samples was tested with PCR). This suggests a form of partial adjudication.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance
No, a Multi-Reader Multi-Case (MRMC) comparative effectiveness study was not done. This device is an in-vitro diagnostic (IVD) assay designed for automated detection of DNA, not an AI-assisted imaging tool for human readers. Therefore, the concept of "human readers improve with AI vs without AI assistance" does not apply here.
6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) was done
Yes, the performance reported is that of the standalone algorithm. The "BD ProbeTec™ ET LP Amplified DNA Assay" (which uses Strand Displacement Amplification - SDA) is an automated system that provides results ("positive, negative or indeterminate") based on predefined threshold values and calculation of "PAT scores." Human involvement is limited to sample processing and running the instrument, not interpreting raw data or making diagnostic decisions without the device's output.
7. The Type of Ground Truth Used
The primary ground truth used for both the prospective and retrospective clinical studies was bacterial culture. For discordant results in the retrospective study, PCR testing was used as a secondary, confirmatory method.
8. The Sample Size for the Training Set
The document does not explicitly state the sample size used for the training set. It mentions that "threshold values for both the L. pneumophila target DNA and the IAC were initially established using Receiver Operator Characteristic curve analyses of data obtained from positive and negative controls." These controls would constitute a form of training or calibration data, but the specific breakdown and size are not provided. The thresholds were then "verified in clinical studies and with retrospective L. pneumophila positive and negative lower respiratory specimens" and "validated in additional clinical studies."
9. How the Ground Truth for the Training Set Was Established
The ground truth for establishing the initial threshold values (which can be considered part of the "training" or calibration) was based on positive and negative controls. The nature of these controls (e.g., cultured bacteria, synthetic DNA, confirmed clinical samples) is not detailed, but they would represent samples with known L. pneumophila presence or absence. Further verification involved "retrospective L. pneumophila positive and negative lower respiratory specimens," implying that their status (positive/negative) was also determined by a reference method, most likely culture.
§ 866.3300
Haemophilus spp. serological reagents.(a)
Identification. Haemophilus spp. serological reagents are devices that consist of antigens and antisera, including antisera conjugated with a fluorescent dye, that are used in serological tests to identifyHaemophilus spp. directly from clinical specimens or tissue culture isolates derived from clinical specimens. The identification aids in the diagnosis of diseases caused by bacteria belonging to the genusHaemophilus and provides epidemiological information on diseases cause by these microorganisms. Diseases most often caused byHaemophilus spp. include pneumonia, pharyngitis, sinusitis, vaginitis, chancroid venereal disease, and a contagious form of conjunctivitis (inflammation of eyelid membranes).(b)
Classification. Class II (special controls). The device is exempt from the premarket notification procedures in subpart E of part 807 of this chapter subject to § 866.9.