(89 days)
Lyme B. burgdorferi (IgM) MarStripe Test is an immunoblot assay for the in vitro qualitative detection of human IgM antibody to individual proteins of Borrelia burgdorferi in human serum or plasma (K2-EDTA) in samples which have been found positive or equivocal using an EIA or IFA test procedure to provide supportive evidence of infection with B. burgdorferi.
The kit is an immunoblot method to detect IgM antibodies against B. burqdorferi antigens. The test kit contains:
. Nitrocellulose Test Strips with purified B. burgdorferi antigens (3) and quality control lines (3) present in specific positions
. Sample Diluent. Provided for specimen dilutions. Contains BSA and PBS
Positive Control derived from human serum positive for Lyme disease. Contains
The provided document describes the Lyme B. burgdorferi (IgM) MarStripe Test, an immunoblot assay for detecting IgM antibodies to Borrelia burgdorferi. The submission aims to establish substantial equivalence to a legally marketed predicate device, the MarDx B. burgdorferi IgM MarBlot Strip Test System (K951709).
Here's an analysis of the acceptance criteria and the study data:
1. Table of Acceptance Criteria and Reported Device Performance
The document does not explicitly state pre-defined acceptance criteria in terms of numerical thresholds for performance metrics. Instead, the study aims to demonstrate substantial equivalence to a predicate device. Therefore, the "acceptance criteria" can be inferred as achieving comparable performance to the predicate device and demonstrating acceptable analytical characteristics.
| Performance Metric | Implied Acceptance Criteria (based on predicate equivalence) | Reported Device Performance (Lyme B. burgdorferi (IgM) MarStripe Test) |
|---|---|---|
| Method Comparison (vs. Predicate IgM WB for EIA-positive/equivocal samples) | Substantial equivalence to predicate device's agreement percentages. | Positive % Agreement: 93.5% (95% CI: 90.1% - 95.8%) |
| Negative % Agreement: 93.5% (95% CI: 90.2% - 95.7%) | ||
| Analytical Specificity (Normal Individuals) | High specificity (e.g., >95%) | 99.5% (95% CI: 97.1% - 100%) |
| Cross-reactivity | All positive results should be confirmed by predicate device. Low overall false positivity rate. | All 7 positive specimens (out of 246 potentially cross-reactive) by MarStripe Test were confirmed positive by predicate IgM Western blot device. Overall positive specimens in cross-reactive cohort: 7 (2.8%) |
| Sensitivity Comparison (Well-characterized Lyme disease specimens) | Comparable sensitivity to predicate device. | 18.4% (16/87) (95% CI: 11.2% - 28.4%) |
| Sensitivity Comparison (CDC Panels) | Comparable sensitivity to predicate device. | Overall: 23.8% (10/42) |
| Precision (Intra-laboratory) | High qualitative agreement, especially for negative and strongly positive samples. | Low Negative and Moderate Positive samples: 100% agreement. |
| Low Positive: 99.0% positive agreement. | ||
| Cutoff: 81.3% positive agreement. | ||
| Reproducibility (Inter-laboratory) | High qualitative agreement across sites and operators. | Low Negative and one Moderate Positive samples: 100% final positive/negative agreement. |
| One High Negative: 99.7% negative agreement. | ||
| One Low Positive: 99.3% positive agreement. | ||
| Cutoff: 68.4% positive agreement. | ||
| Interference | 100% qualitative agreement with and without interfering agents. | 100% qualitative agreement for all tested specimens. |
| Serum vs. Plasma Matrix Comparison | 100% qualitative agreement between serum and plasma samples. | 100% qualitative agreement for all 20 pairs. |
2. Sample Sizes Used for the Test Set and Data Provenance
- Method Comparison: 676 specimens tested (those found positive or equivocal on an FDA-cleared first-step EIA). The study was a prospective study performed at three geographically distinct study sites. The country of origin for the data is not explicitly stated but implied to be the USA given the FDA context and reference to CDC.
- Analytical Specificity: 220 sera from normal individuals (blood bank donors) representing endemic geographic regions of the United States. Data provenance is from US blood bank donors, retrospective or prospective not specified but typical for such collections.
- Cross-reactivity: 246 potentially cross-reactive specimens from individuals with various other infectious conditions. Data provenance is not specified.
- Sensitivity: 87 well-characterized Lyme disease clinical specimens. Samples included early, early disseminated, and late phases. Data provenance is not specified but implied to be clinical.
- CDC Panels: 10 samples from the Lyme Disease Validation Panel and 32 samples from the Lyme Disease Basic Research Panel (Total 42 samples). These are reference panels from the Centers for Disease Control and Prevention (CDC).
- Precision: 8 specimens, each tested in 4 replicates, two runs per day over 12 days (total 96 tests per specimen).
- Reproducibility: 8 specimens, each tested in 4 replicates, two runs per day over 12 days, at each of three laboratory sites (total 192 tests per specimen per site, or 576 read-outs in total).
- Interference: 2 Lyme IgM negative and 3 IgM positive sera.
- Serum vs. Plasma Matrix Comparison: 20 pairs of sera/plasma (total 40 samples).
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications
The document does not explicitly state the number or qualifications of experts used to establish ground truth for the test sets.
- For the Method Comparison study, the ground truth was established by comparison to results from an "FDA cleared immunoblot results" (the predicate device) which followed "recommended criteria described by the Centers for Disease Control (CDC) and the Second National Conference on Serological Diagnosis of Lyme Disease". This implies standardized interpretation criteria rather than individual expert consensus.
- For the Sensitivity study, "87 well characterized Lyme disease clinical specimens" were used, implying their disease status was previously determined, likely through a combination of clinical diagnosis and laboratory testing, but specific expert involvement is not detailed.
- For the CDC Panels, the ground truth is the CDC's characterization of these reference panels.
4. Adjudication Method for the Test Set
The document does not describe an adjudication method for the test set interpretation by human readers. For the reproducibility study, results were read by "2 human operators at each site," but it does not specify if or how discrepancies between these operators were resolved (e.g., 2+1, 3+1). The "ground truth" for the overall studies appears to be based on established reference methods or clinical characterization rather than adjudicated interpretations of the MarStripe test itself.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was done
No, a Multi-Reader Multi-Case (MRMC) comparative effectiveness study was not explicitly described in the context of human readers improving with AI vs. without AI assistance. The studies performed were primarily focused on the device's analytical performance and its agreement with a predicate device and established reference panels.
6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) was done
The device itself (Lyme B. burgdorferi (IgM) MarStripe Test) is an immunoblot assay that requires visual reading. The results show "Positives", "Negatives", "Band Type" (Pos., Neg., Cut., Wpos.), implying visual interpretation by a human. For example, the reproducibility study explicitly mentions "Results were read by 2 human operators at each site". Therefore, this is not a standalone algorithm without human-in-the-loop performance; human visual interpretation is an integral part of the test.
7. The Type of Ground Truth Used
The ground truth varied depending on the study:
- Method Comparison: Ground truth was the results from an FDA-cleared predicate IgM Western blot device, interpreted according to CDC and Second National Conference on Serological Diagnosis of Lyme Disease criteria.
- Analytical Specificity: Implied ground truth is "normal individuals" from blood banks, presumed negative for Lyme disease.
- Cross-reactivity: Ground truth was the clinical diagnosis of other infectious conditions, with validation against the predicate IgM Western blot device for any MarStripe positive results.
- Sensitivity: "Well characterized Lyme disease clinical specimens," implying clinical diagnosis of Lyme disease stages.
- CDC Panels: Characterization provided by the CDC for their reference panels.
- Precision/Reproducibility/Interference/Serum vs. Plasma: The ground truth for these analytical studies was related to the known characteristics of the samples (e.g., known negative, known positive, spiked with interferents, paired serum/plasma) as determined by prior testing (e.g., FDA-cleared B. burgdorferi ELISA results).
8. The Sample Size for the Training Set
The document describes performance studies for the Lyme B. burgdorferi (IgM) MarStripe Test. This device is a diagnostic assay, not an AI/ML model that requires a "training set" in the computational sense. The studies presented are validation studies of the finished device. There is no mention of an algorithm development phase with a dedicated training set.
9. How the Ground Truth for the Training Set Was Established
Since the device is a diagnostic assay (not an AI/ML model), there is no "training set" as understood in machine learning. Therefore, this question is not applicable to the information provided.
§ 866.3830
Treponema pallidum treponemal test reagents.(a)
Identification. Treponema pallidum treponemal test reagents are devices that consist of the antigens, antisera and all control reagents (standardized reagents with which test results are compared) which are derived from treponemal sources and that are used in the fluorescent treponemal antibody absorption test (FTA-ABS), theTreponema pallidum immobilization test (T.P.I.), and other treponemal tests used to identify antibodies toTreponema pallidum directly from infecting treponemal organisms in serum. The identification aids in the diagnosis of syphilis caused by bacteria belonging to the genusTreponema and provides epidemiological information on syphilis.(b)
Classification. Class II (performance standards).