(87 days)
The Aptima BV assay is an in vitro nucleic acid amplification test that utilizes real time transcription-mediated amplification (TMA) for detection and quantitation of ribosomal RNA from bacteria associated with bacterial vaginosis (BV), including Lactobacillus (L. gasseri, L. crispatus, and L. jensenii), Gardnerella vaginalis, and Atopobium vaginae. The assay reports a qualitative result for BV and does not report results for individual organisms. The assay is intended to aid in the diagnosis of BV on the automated Panther system using clinician-collected and patient-collected vaginal swab specimens from females with a clinical presentation consistent with vaginitis and/or vaginosis.
The Aptima BV assay is an in vitro nucleic acid amplification test for the detection and quantitation of rRNA from bacteria associated with bacterial vaginosis in women with a clinical presentation consistent with vaginitis/vaginosis. The Aptima BV assay utilizes the automated Panther system to provide qualitative results to aid in the diagnosis of bacterial vaginosis. The assay involves three main steps, all of which take place in a single tube on the Panther system: target capture, target amplification by TMA, and detection of the amplification products (amplicon) by fluorescent labeled probes (torches). The assay incorporates an internal control (IC) in every test to monitor nucleic acid capture, amplification and detection. The Aptima BV assay is provided as a 100-test kit containing 8 reagents, 1 calibrator, and 2 controls.
Here's a summary of the acceptance criteria and study details for the Aptima BV Assay based on the provided FDA 510(k) summary:
1. Table of Acceptance Criteria and Reported Device Performance
The acceptance criteria are generally inferred from the "Brief Description of Analytical (Non-Clinical) Studies" and "Brief Description of Clinical Studies" sections, focusing on the demonstrated acceptable performance. Specific numerical acceptance criteria are not explicitly stated as pass/fail thresholds in percentage terms, but rather performance results are reported which are presumably acceptable for clearance.
| Performance Metric | Inferred Acceptance Criteria (Type of Result) | Reported Device Performance |
|---|---|---|
| Analytical Studies: | ||
| Limit of Detection (LoD) | Ability to detect target organisms at low concentrations | - A. vaginae: 2901 CFU/mL (95% detection) |
| - G. vaginalis: The content of the document in this context in not in English. It says This village has primary school, panchayat house, anganwadi, and milk dairy facilities. The main occupation of the people of this village is agriculture, farm labor CFU/mL (95% detection) | ||
| - L. crispatus: The content of the document in this context in not in English. It says This village has primary school, panchayat house, anganwadi, and milk dairy facilities. The main occupation of the people of this village is agriculture, farm labor CFU/mL (95% detection) | ||
| - L. gasseri: 2,207 CFU/mL (95% detection) | ||
| - L. jensenii: 10 CFU/mL (95% detection) | ||
| Analytical Inclusivity | Detection of various strains of target organisms | - All 5 strains of G. vaginalis and A. vaginae detected at 3X C95. |
| - All 5 strains of L. crispatus and L. gasseri detected at 3X LoD. | ||
| - 3 of 5 strains of L. jensenii detected at 3X LoD, remaining 2 at 10X LoD. | ||
| Cross-Reactivity & Microbial Interference | No interference from common non-target organisms and substances | - No cross-reactivity or microbial interference observed for 62 organisms at specified concentrations (except Lactobacillus acidophilus at ≥1x10⁴ CFU/mL). |
| Interference | No interference from common endogenous and exogenous substances | - No interference observed from 20 tested substances at specified concentrations (except Mucus at ≥2% V/V, Tioconazole 6.5% Ointment at 5% W/V, Vaginal Moisturizing Gel at ≥0.5% W/V). |
| Within Laboratory Precision | Consistent performance across operators, instruments, days, lots, and runs | - BV percent positive results for panels ranged from 0% (negative) to 100% (positive) as expected. |
| - Signal variability (Total CV) for Lactobacillus, G. vaginalis, and A. vaginae panel members ranged from 3.30% to 5.74%. | ||
| Clinical Studies (Symptomatic Subjects): | ||
| Sensitivity (Patient-collected) | High sensitivity for BV detection | 97.3% (95% CI: 95.8-98.2) |
| Specificity (Patient-collected) | High specificity for BV detection | 85.8% (95% CI: 83.1-88.2) |
| PPV (Patient-collected) | High PPV for BV detection | 87.0% (95% CI: 84.8-88.9) |
| NPV (Patient-collected) | High NPV for BV detection | 97.0% (95% CI: 95.5-98.1) |
| Sensitivity (Clinician-collected) | High sensitivity for BV detection | 95.0% (95% CI: 93.1-96.4) |
| Specificity (Clinician-collected) | High specificity for BV detection | 89.6% (95% CI: 87.1-91.6) |
| PPV (Clinician-collected) | High PPV for BV detection | 89.8% (95% CI: 87.7-91.7) |
| NPV (Clinician-collected) | High NPV for BV detection | 94.8% (95% CI: 93.1-96.3) |
| Reproducibility: | ||
| Agreement with Expected Results | 100% agreement expected for controls and panels | 100% (95% CI: 96.6-100) for all 7 panel members (True Neg, BV Neg, Gvag Low Pos, Avag Low Pos, BV Low Pos, Gvag Mod Pos, Avag MosPos). |
| Signal Variability | Low variability across sites, operators, etc. | - Total %CV for analyte-positive panels: 4.21% to 4.76%. Total SD values: ≤1.12. |
| - Total %CV for controls and calibrators: 4.47% to 5.36%. Total SD values: ≤1.11. |
2. Sample Size Used for the Test Set and Data Provenance
-
Clinical Performance Study (Symptomatic Subjects):
- Evaluable Subjects: 1417 symptomatic women.
- Patient-collected Aptima vaginal swab samples: 1405
- Clinician-collected Aptima vaginal swab samples: 1413
- Data Provenance: Prospectively-collected patient- and clinician-collected vaginal swab samples from women ≥14 years in 21 geographically diverse US private and academic family practice, obstetric-gynecologic, family planning, public health, STI, medical group clinics, and clinical research centers.
-
Clinical Performance Study (Asymptomatic Subjects):
- Evaluable Subjects: 172 asymptomatic women.
- Clinician-collected Aptima vaginal swab samples: 172
- Data Provenance: Prospectively-collected clinician-collected vaginal swab samples from asymptomatic women in 21 geographically diverse US sites (same as symptomatic subjects).
-
Within Laboratory Precision:
- Replicates: A minimum of 20 replicates per panel member for LoD. For precision, each operator performed 2 runs per day, with 3 replicates of each of the 11 panel members per run, across 21 days. (Total N = 168 or 165 for some panels due to exclusions).
- Data Provenance: Laboratory study using synthetic panels (SVSM).
-
Reproducibility:
- Replicates: 3 replicates of each of the 7 panel members per run. Testing performed for at least six days at each of three sites. (Total N = 108 for each panel member).
- Data Provenance: Multi-site laboratory study using synthetic panels (SVSM).
3. Number of Experts Used to Establish the Ground Truth for the Test Set and the Qualifications of Those Experts
The document does not explicitly state the number or specific qualifications of experts (e.g., "radiologist with 10 years of experience"). However, it mentions that for the clinical performance study, the ground truth for BV infection status was established using Nugent score evaluation, and modified Amsel criteria if necessary. This implies trained laboratory personnel and clinicians performing these established diagnostic methods.
4. Adjudication Method for the Test Set
- Clinical Performance Study:
- "A Nugent interpretation established positive and negative BV reference status, except in cases of intermediate determinations. For intermediate Nugent interpretations, BV reference status was established using modified Amsel criteria."
- This indicates a hierarchical adjudication or ground truth establishment method where Nugent scoring was primary, and Amsel criteria served as a secondary method for intermediate Nugent results. The document does not describe a multi-reader adjudication process in the sense of multiple experts independently reviewing and then reaching a consensus for the ground truth.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done
No, the document does not describe a multi-reader multi-case (MRMC) comparative effectiveness study comparing human readers with AI assistance versus without AI assistance. The study evaluates the standalone performance of the Aptima BV Assay against established clinical criteria (Nugent score/Amsel criteria).
6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) Was Done
Yes, the studies presented are standalone performance evaluations of the Aptima BV Assay (an in vitro nucleic acid amplification test performed on the automated Panther system). The results (BV positive or negative status) are generated by the system's software based on signal emergence times and calibration information, without human interpretation of the assay's output for diagnostic purposes in the performance studies. Its performance is compared to a ground truth established by clinical methods.
7. The Type of Ground Truth Used
-
Clinical Performance Study:
- The primary ground truth for Bacterial Vaginosis (BV) infection status was established using Nugent score evaluation.
- For cases with intermediate Nugent interpretations, modified Amsel criteria were used to establish the BV reference status.
-
Analytical and Reproducibility Studies:
- Ground truth was based on the known composition of the synthetic panels (Simulated Vaginal Swab Matrix - SVSM) spiked with specific concentrations of target organism lysates.
8. The Sample Size for the Training Set
The document does not provide information on the sample size used for the training set of the Aptima BV Assay. This is because the Aptima BV Assay is a nucleic acid amplification test, a type of IVD, that relies on biochemical reactions and calibrated thresholds rather than machine learning algorithms that typically require explicit training data sets for model development. The "calibration information" mentioned in the description of the Panther system suggests a pre-defined set of parameters, likely established through analytical studies.
9. How the Ground Truth for the Training Set Was Established
As noted above, for this type of in-vitro diagnostic device, there isn't typically a "training set" in the machine learning sense with an associated ground truth established by experts.
Instead, the assay's operational parameters (e.g., thresholds for positivity based on signal emergence times) would be established through a rigorous process of analytical validation (like the LoD and precision studies described) using samples with known concentrations of targets. This ensures the assay's performance characteristics (sensitivity, specificity) meet predefined criteria. The "calibration information" is central to how the device makes its determination.
§ 866.3975 Device that detects nucleic acid sequences from microorganisms associated with vaginitis and bacterial vaginosis.
(a)
Identification. A device that detects nucleic acid sequences from microorganisms associated with vaginitis and bacterial vaginosis is a qualitative in vitro diagnostic device intended for the detection of microbial nucleic acid sequences in vaginal specimens collected from patients with signs and symptoms of vaginitis or bacterial vaginosis. This device is intended to aid in the diagnosis of vaginitis or bacterial vaginosis when used in conjunction with clinical signs and symptoms and other laboratory findings.(b)
Classification. Class II (special controls). The special controls for this device are:(1) Design verification and validation must include:
(i) Documentation with a detailed device description of device components; ancillary reagents required but not provided; and explanation of the methodology including primer/probe sequence, design, and rationale for sequence selection.
(ii) Documentation with information that demonstrates the performance characteristics of the device, including:
(A) Limit of Detection;
(B) Precision (reproductivity);
(C) Analytical specificity;
(D) Analytical reactivity (inclusivity);
(E) Specimen stability; and
(F) Effects of interfering substances.
(iii) Detailed documentation from a prospective clinical study. As appropriate to the intended use, the prospective clinical study must be performed on an appropriate study population, including women of various ages and ethnicities. The prospective clinical study must compare the device performance to results obtained from well-accepted comparator methods.
(iv) Detailed documentation for device software, including software applications and hardware-based devices that incorporate software.
(2) The labeling required under § 809.10(b) of this chapter must include:
(i) A detailed explanation of the interpretation of results and acceptance criteria;
(ii) For devices with an intended use that includes detection of nucleic acid sequences from bacteria associated with bacterial vaginosis, clinical performance stratified by patient demographics such as race, ethnicity, age, and pregnancy status.
(iii) For devices with an intended use that includes detection of nucleic acid sequences from bacteria associated with bacterial vaginosis, a summary of device results in an asymptomatic population with demographic characteristics appropriate to the intended use population.
(iv) For devices with an intended use that includes detection of either Candida species or bacteria associated with bacterial vaginosis, a limitation that
Candida species and bacterial compositions associated with bacterial vaginosis can be present as part of normal vaginal flora and results should be considered in conjunction with available clinical information.