(90 days)
For the qualitative determination of total (IgG and IgM) antibodies to Treponema pallidum (TP) specific antigens in human serum and plasma using the VITROS 5600 Integrated System.
The presence of antibodies to Treponema pallidum (TP) specific antigens, in conjunction with non-treponemal laboratory tests and clinical findings may aid in the diagnosis of syphilis infection.
The VITROS Syphilis test is not intended for blood and tissue donor screening.
The VITROS Immunodiagnostic Products Syphilis test is performed using the VITROS Immunodiagnostic Products Syphilis Reagent Pack and VITROS Immunodiagnostic Products Syphilis Calibrator on the VITROS 5600 Integrated System.
An immunometric technique is used; this involves a two-stage reaction. In the first stage antibodies to Syphilis TP specific antigens present in the sample bind with biotinylated recombinant Syphilis TP antigens immobilized on streptavidin coated wells. Unbound sample is removed by washing. In the second stage conjugate reagent containing horseradish peroxidase (HRP)-labeled recombinant Syphilis TP antigens is added. The conjugate binds specifically to any antibody to Syphilis TP specific antigens captured on the well in the first stage. Unbound conjugate is removed by washing.
The bound HRP conjugate is measured by a luminescent reaction. A reagent containing luminogenic substrates (a luminol derivative and a peracid salt) and an electron transfer agent is added to the wells. The HRP in the bound conjugate catalyzes the oxidation of the luminol derivative, producing light. The electron transfer agent (a substituted acetanilide) increases the level of light produced and prolongs its emission. The light signals are read by the system.
Here's a detailed breakdown of the acceptance criteria and the study proving the device meets these criteria, based on the provided text:
1. Table of Acceptance Criteria and Reported Device Performance
The acceptance criteria are implied by the clinical performance results meeting high percentage agreements. While explicit numeric acceptance criteria (e.g., "PPA must be >X%") are not directly stated in a dedicated section, the sponsor presents the results and concludes substantial equivalence, indicating that these results were considered acceptable.
| Performance Metric | Acceptance Criteria (Implied) | Reported Device Performance |
|---|---|---|
| All Prospective Subgroups | ||
| Positive Percent Agreement (PPA) | High agreement with comparator | 99.5% (218/219) |
| Negative Percent Agreement (NPA) | High agreement with comparator | 97.6% (688/705) |
| Routine Syphilis (Prospective) | ||
| PPA | High agreement with comparator | 98.8% (83/84) |
| NPA | High agreement with comparator | 99.2% (527/531) |
| Pregnant Women (Prospective) | ||
| PPA | High agreement with comparator | 100.0% (1/1) |
| NPA | High agreement with comparator | 97.8% (45/46) |
| HIV Positive (Prospective) | ||
| PPA | High agreement with comparator | 100.0% (134/134) |
| NPA | High agreement with comparator | 90.6% (116/128) |
| All Retrospective Specimens | ||
| PPA | High agreement with comparator | 100.0% (213/213) |
| NPA | High agreement with comparator | 98.8% (330/334) |
| Pregnant Women (Retrospective) | ||
| PPA | High agreement with comparator | 100.0% (31/31) |
| NPA | High agreement with comparator | 100.0% (212/212) |
| HIV Positive (Retrospective) | ||
| PPA | High agreement with comparator | 100.0% (30/30) |
| NPA | High agreement with comparator | 96.7% (118/122) |
| Pre-selected Positive (Retrospective) | ||
| PPA | High agreement with comparator | 100.0% (152/152) |
| NPA | N/A (denominator is zero) | N/A |
| Medically Diagnosed (Syphilis) Reactivity | Consistent detection | 100% reactive (151/151) across all stages and treatment statuses |
| Apparently Healthy Individuals Reactivity | Low false positive rate | 2.0% reactive (4/201), with 3 of these confirmed positive by final comparator |
2. Sample size used for the test set and the data provenance
- Test Set Sample Size:
- Prospective Specimens: 924 total specimens.
- 615 subjects for routine syphilis testing (from 6 sites in the United States).
- 47 pregnant women.
- 262 HIV positive subjects.
- Retrospective Specimens: 547 total specimens.
- 243 samples from pregnant women.
- 152 HIV positive samples.
- 152 pre-selected positive samples.
- Medically Diagnosed Individuals: 151 samples.
- Apparently Healthy Individuals: 201 prospective specimens (from 3 sites in the United States).
- Total Clinical Samples: 924 (prospective) + 547 (retrospective) + 201 (apparently healthy) = 1672 specimens for primary clinical evaluation. The medically diagnosed group is a subset likely included within the prospective/retrospective groups or as additional targeted samples.
- Prospective Specimens: 924 total specimens.
- Data Provenance:
- Country of Origin: United States (for prospective and apparently healthy specimens). Retrospective samples are "purchased," implying they may originate from various sources but were tested at sites.
- Retrospective or Prospective: Both retrospective and prospective clinical studies were conducted.
3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts
The document does not specify the number of experts or their qualifications for establishing the ground truth. Instead, the ground truth was established using a "composite testing algorithm."
4. Adjudication method for the test set
The adjudication method relied on a composite testing algorithm using:
- A treponemal electrochemiluminescence immunoassay (TP-ECLIA)
- A Rapid Plasma Reagin (RPR) non-treponemal assay
- A Treponema pallidum Particle Agglutination (TP-PA) Treponema-specific assay
Cases were categorized as "Positive for Syphilis" or "Negative for Syphilis" based on the results of these FDA-cleared comparator assays. The document describes specific combinations of results from these three assays that led to a final comparator result. For example, "Non-reactive TP-ECLIA + Non-reactive RPR" was deemed "Negative," while "Reactive TP-ECLIA + Reactive RPR (and N/A TP-PA)" was "Positive." Discordant results with the final comparator were further analyzed (e.g., 14 discordant prospective samples were found reactive by another FDA-cleared TP antibody test).
5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance
No, an MRMC comparative effectiveness study involving human readers and AI assistance was not conducted or reported. This device is an in vitro diagnostic test (IVD) for laboratory use, not an AI-powered image analysis or diagnostic aid for human readers.
6. If a standalone (i.e. algorithm only without human-in-the loop performance) was done
Yes, the studies presented are standalone performance evaluations of the VITROS Immunodiagnostic Products Syphilis test (an algorithm-driven instrument/reagent system) without human-in-the-loop performance being a variable in the reported clinical accuracy. The assay itself performs the determination.
7. The type of ground truth used
The ground truth used was a composite testing algorithm based on multiple FDA-cleared comparator assays (TP-ECLIA, RPR, and TP-PA). This is a form of clinical surrogate ground truth derived from established diagnostic tests, rather than direct pathology, biopsy, or long-term outcomes data, which are typically used for definitive diagnosis in some other medical fields.
8. The sample size for the training set
The document does not provide information on the sample size for a training set. This is because the device described is an immunoassay, a biochemical test, not a machine learning or AI algorithm that typically requires a separate training set. The reported studies evaluate the locked-down analytical and clinical performance of the manufactured diagnostic kit.
9. How the ground truth for the training set was established
Since there is no training set mentioned for an AI/machine learning algorithm, the concept of establishing ground truth for a training set is not applicable here. The assay's internal calibration and optimization would have been performed during product development, but this is distinct from "training" an AI model in the context of imaging or predictive analytics.
§ 866.3830
Treponema pallidum treponemal test reagents.(a)
Identification. Treponema pallidum treponemal test reagents are devices that consist of the antigens, antisera and all control reagents (standardized reagents with which test results are compared) which are derived from treponemal sources and that are used in the fluorescent treponemal antibody absorption test (FTA-ABS), theTreponema pallidum immobilization test (T.P.I.), and other treponemal tests used to identify antibodies toTreponema pallidum directly from infecting treponemal organisms in serum. The identification aids in the diagnosis of syphilis caused by bacteria belonging to the genusTreponema and provides epidemiological information on syphilis.(b)
Classification. Class II (performance standards).