(80 days)
The Bio-Rad Syphilis IgG kit is a multiplex flow immunoassay intended for the qualitative detection of Treponema pallidum IgG antibodies in human serum. The test system, when used in conjunction with non-treponemal based assays, provides serological evidence of infection with T. pallidum. This test system is also indicated for use in confirming reactive test results from non-treponemal based screening assays.
The Syphilis IgG kit is intended for use with the Bio-Rad BioPlex 2200 System.
The BioPlex 2200 Syphilis IgG kit is not intended for use in screening blood or plasma donors.
Warning: A positive result is not useful for establishing a diagnosis of Syphilis. In most situations, such a result may reflect prior treated infection; a negative result can exclude a diagnosis of syphilis except for incubating or early primary disease.
The Syphilis IgG kit uses multiplex flow immunoassay, a methodology that greatly resembles traditional EIA, but permits simultaneous detection and identification of many antibodies in a single tube. Three (3) different populations of dyed beads are coated with recombinant proteins associated with T. pallidum (15kD, 17kD and 47kD). The BioPlex 2200 System combines an aliquot of patient sample, sample diluent, and bead reagent into a reaction vessel. The mixture is incubated at 37°C. After a wash cycle, anti-human IgG antibody, conjugated to phycoerythrin (PE), is added to the dyed beads, and this mixture is incubated at 37°C. The excess conjugate is removed in another wash cycle, and the beads are re-suspended in wash buffer. The bead mixture then passes through the detector. The identity of the dyed beads is determined by the fluorescence of the dyes, and the amount of antibody captured by the antigen is determined by the fluorescence of the attached PE. Raw data is calculated in relative fluorescence intensity (RFI).
Three additional dyed beads, Internal Standard Bead (ISB), Serum Verification Bead (SVB) and a Reagent Blank Bead (RBB) are present in each reaction mixture to verify detector response, the addition of serum or plasma to the reaction vessel and the absence of significant non-specific binding in serum or plasma. Refer to the BioPlex 2200 System Operation Manual for more information.
The system is calibrated using a set of four (4) distinct calibrators vials, supplied separately by Bio-Rad Laboratories. Four (4) vials representing two (2) or three (3) different antibody concentrations are used for calibration. Results are calculated for each of the three (3) antibodies and are compared against their own respective cut-off and are expressed as an antibody index (AI). A single result is reported after completing a composite analysis of all the antibodies (the highest Al value is reported).
Here's a breakdown of the acceptance criteria and study details for the BioPlex 2200 Syphilis IgG device, based on the provided text:
Acceptance Criteria and Device Performance
The acceptance criteria for the BioPlex 2200 Syphilis IgG device appear to be primarily focused on its agreement with established reference assays (RPR and TPPA, with treponemal EIA for discordant results) and its performance in various clinical populations. Specific quantitative acceptance criteria are not explicitly stated as distinct thresholds in the document, but rather are demonstrated through the reported percentages in the comparative testing tables.
Here’s a table summarizing the reported device performance in relation to the implied acceptance of sufficient agreement and sensitivity:
| Acceptance Criteria / Performance Category | Specific Metric (as reported) | Reported Device Performance (%) | 95% Confidence Interval (where available) |
|---|---|---|---|
| Comparative Testing (Syphilis Test Ordered) | |||
| Positive Agreement (vs. Reference Assays) | Serum Samples From Patients Who Had a Syphilis Test Ordered (N=500), Positive Reference Assays Result | 100% (46/46) | 92.0 - 100% |
| Negative Agreement (vs. Reference Assays) | Serum Samples From Patients Who Had a Syphilis Test Ordered (N=500), Negative Reference Assays Result | 93.8% (426/454) | 91.2 - 95.7% |
| Comparative Testing (Unselected Pregnant Women) | |||
| Positive Agreement (vs. Reference Assays) | Unselected Serum Samples From Pregnant Women (N=497), Positive Reference Assays Result | 100% (5/5) | 56.5 - 100% |
| Negative Agreement (vs. Reference Assays) | Unselected Serum Samples From Pregnant Women (N=497), Negative Reference Assays Result | 99.8% (491/492) | 98.9 - 100% |
| Comparative Testing (RPR/TPPA Reactive Samples) | |||
| Positive Agreement (vs. Reference Assays) | Serum Samples Requested to be RPR and TPPA Reactive (N=250), Positive Reference Assays Result | 100% (249/249) | 98.5 - 100% |
| Comparative Testing (Pregnant Women - Treponemal Assay Positive) | |||
| Positive Agreement (vs. Reference Assays) | Serum Samples From Pregnant Women Requested to be Treponemal Assay Positive (N=183), Positive Reference Assays Result | 100% (166/166) | 97.7-100% |
| Comparative Testing (Pregnant Women - RPR/TPPA Nonreactive) | |||
| Negative Agreement (vs. Reference Assays) | Serum Samples From Pregnant Women Requested to be RPR/TPPA Nonreactive (N=250), Negative Reference Assays Result | 98.8% (247/250) | 96.5 - 99.6% |
| Comparative Testing (Medically Diagnosed Syphilis) | |||
| Positive Agreement (vs. Reference Assays) | Serum Samples From Patients Medically Diagnosed With Syphilis (N=70), Positive Reference Assays Result | 100.0% (63/63) | 94.2 - 100% |
| Clinical Sensitivity (Combined CDC Panel + Prospectively Collected Samples) | |||
| Overall Clinical Sensitivity | BioPlex 2200 Syphilis IgG vs. Combined CDC Panel and Prospectively Collected Samples With Known Clinical Status (N=150), Total Reactive | 92.0% (138/150) | 86.5 - 95.4% |
| Overall Reference Assay Clinical Sensitivity | BioPlex 2200 Syphilis IgG vs. Combined CDC Panel and Prospectively Collected Samples With Known Clinical Status (N=150), Total Reactive by Reference Assay | 89.5% (137/150) | 86.5 - 94.9% |
| Overall Positive Agreement (by Known Clinical Status) | BioPlex 2200 Syphilis IgG vs. Reference Assays: Combined Percent Agreement By Known Clinical Status (N=150), Total | 100% (137/137) | 97.3 - 100% |
The studies provided demonstrate a high degree of concordance and sensitivity of the BioPlex 2200 Syphilis IgG assay with established methods and known clinical status, supporting its intended use. Reproducibility and precision data also indicate reliable performance.
Study Details
Here's the detailed information about the studies:
2. Sample sizes used for the test set and the data provenance:
The document describes several test sets used for comparative testing, reproducibility, precision, cross-reactivity, and clinical sensitivity.
- Comparative Testing (Primary Evaluation):
- Serum Samples From Patients Who Had a Syphilis Test Ordered: N=500, sourced from three (3) U.S. clinical testing sites. (Provenence: U.S., likely retrospective, as they are "banked serum samples").
- Unselected Serum Samples From Pregnant Women: N=497, sourced from three (3) U.S. clinical testing sites. (Provenance: U.S., likely retrospective, as they are "banked serum samples").
- Serum Samples Requested to be RPR and TPPA Reactive: N=250, sourced from three (3) U.S. clinical testing sites. (Provenance: U.S., likely retrospective, as they are "banked purchased samples").
- Serum Samples From Pregnant Women Requested to be Treponemal Assay Positive: N=183, sourced from three (3) U.S. clinical testing sites. (Provenance: U.S., likely retrospective, as they are "banked purchased samples").
- Serum Samples From Pregnant Women Requested to be RPR/TPPA Nonreactive: N=250, sourced from three (3) U.S. clinical testing sites. (Provenance: U.S., likely retrospective, as they are "banked purchased samples").
- Serum Samples From Patients Medically Diagnosed With Syphilis: N=70, sourced from three (3) U.S. clinical testing sites. (Provenance: U.S., "banked prospectively collected samples").
- Reproducibility Studies: Seven (7) panel members per recombinant protein (15kD, 17kD, 47kD) were prepared by Bio-Rad Laboratories. Each member was tested in duplicate (x2) on 2 runs per day for 3 days at 3 U.S. testing facilities, resulting in 36 replicates per panel member. (Provenance: U.S., prospective testing of prepared panels).
- Precision Studies: Nine (9) panel members per recombinant protein (15kD, 17kD, 47kD) were prepared by Bio-Rad Laboratories. Each member was tested in duplicate (x2) on 2 runs per day for 20 days, resulting in 80 replicates per panel member. (Provenance: Bio-Rad Laboratories, prospective testing of prepared panels).
- Correlation with Known HIV-1 Positive Samples: N=220, banked known HIV-1 positive serum samples. (Provenance: Not explicitly stated, but implies retrospective collection).
- Correlation with Disease Stages and Clinical Sensitivity (CDC Panel): N=140, CDC panel of banked frozen characterized sera. (Provenance: CDC, likely retrospective).
- Correlation with Disease Stages and Clinical Sensitivity (Prospectively Collected): N=10, banked prospectively collected serum samples from treated and untreated patients with primary and secondary syphilis infections. (Provenance: One clinical testing site, prospective collection specified).
- Cross-Reactivity: Ten (10) specimens positive for each cross-reactant (except E. Coli, N=4, and Lyme IgG Afzelli / Garnii, N=49). (Provenance: Not explicitly stated, likely sourced from various collections).
3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts:
The document does not specify the number of experts or their qualifications for establishing the ground truth. It refers to "Reference Assays Result" (RPR, TPPA, and treponemal EIA for discordant results) and "Known Clinical Status" (for the CDC panel and prospectively collected samples from medically diagnosed patients). The interpretation of these reference assays or the clinical diagnosis would typically involve trained laboratory personnel and/or clinicians, but specific details on their number or qualifications are absent.
4. Adjudication method for the test set:
For comparative testing, the document states: "Discordant samples were further tested by a treponemal EIA kit to determine the final result." This indicates an adjudication method where discrepant results between the RPR and TPPA assays were resolved using a third treponemal EIA.
For the initial comparison between the BioPlex 2200 and the reference assays, if the reference assays yielded an equivocal result, it was assigned to the opposite result than that of the corresponding BioPlex 2200 Syphilis IgG result for the purposes of percent agreement calculation.
5. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:
This device is an in vitro diagnostic (IVD) assay, not an AI-powered image analysis or diagnostic tool involving "human readers" in the typical sense of an MRMC study. Therefore, no MRMC comparative effectiveness study was done to assess human reader improvement with or without AI assistance. The performance is that of the automated system.
6. If a standalone (i.e., algorithm only without human-in-the loop performance) was done:
Yes, the studies described are standalone performance studies of the BioPlex 2200 Syphilis IgG kit on the BioPlex 2200 Multi-Analyte Detection System. The device is an automated IVD system that performs the assay and provides a result (Reactive, Equivocal, Nonreactive) without human interpretation steps that would define a "human-in-the-loop" performance study. The human intervention would be in operating the machine and interpreting the instrument's final declared result, but not in the diagnostic decision process itself as if it were an AI diagnostic aid.
7. The type of ground truth used:
The ground truth for the test sets was primarily established through a combination of expert consensus based on established reference laboratory assays (RPR, TPPA, and treponemal EIA) and known clinical status/medical diagnosis.
- For the main comparative testing studies, the "Reference Assays Result" was determined by an algorithm combining RPR and TPPA results, with discordant results resolved by treponemal EIA.
- For the clinical sensitivity studies (CDC panel and prospectively collected samples), the ground truth was based on "Known Clinical Status" or "medically diagnosed with syphilis infections." While the document doesn't explicitly state "expert consensus" as the method for clinical diagnosis, medical diagnosis inherently involves professional judgment and consensus within the medical community.
8. The sample size for the training set:
The document does not specify a distinct "training set" or its sample size. This is typical for traditional IVD assays, where method development and optimization (analogous to training in machine learning) occur, but a formal, separate "training set" as understood in AI/ML development is not usually detailed in regulatory submissions for these types of devices. The "training" here refers to the internal development and calibration of the assay.
9. How the ground truth for the training set was established:
As no explicit training set is described in the context of AI/ML, the question of how its ground truth was established is not directly applicable. However, the development of the BioPlex 2200 Syphilis IgG kit would have involved internal validation and optimization using characterized samples to define reactivities and cut-off values. This process would rely on previously established reference methods and clinical samples.
§ 866.3830
Treponema pallidum treponemal test reagents.(a)
Identification. Treponema pallidum treponemal test reagents are devices that consist of the antigens, antisera and all control reagents (standardized reagents with which test results are compared) which are derived from treponemal sources and that are used in the fluorescent treponemal antibody absorption test (FTA-ABS), theTreponema pallidum immobilization test (T.P.I.), and other treponemal tests used to identify antibodies toTreponema pallidum directly from infecting treponemal organisms in serum. The identification aids in the diagnosis of syphilis caused by bacteria belonging to the genusTreponema and provides epidemiological information on syphilis.(b)
Classification. Class II (performance standards).