(109 days)
The Access Syphilis assay is a paramagnetic particle, chemiluminescent immunoassay for the qualitative detection of total antibodies to Treponema pallidum in human serum and plasma using the Access lmmunoassay Systems. It is intended to be used as an aid in the diagnosis of syphilis or in conjunction with a nontreponemal laboratory test and clinical findings to aid in the diagnosis of syphilis infection. The Access Syphilis assay is not intended for blood and tissue donor screening.
The Access Syphilis assay is a two-step enzyme immunoassay. A sample is added to a reaction vessel with buffer, paramagnetic particles coated with recombinant Treponema pallidum antigens Tp17 and Tp47, and Tp47, and biotinylated Treponema Tp17 & Tp47 antigens. After incubation in a reaction vessel, materials bound to the solid phase are held in a magnetic field while unbound materials are washed away. Alkaline phosphatase conjugates are added, and the conjugates bind to the immunoglobulin captured on the particles. A chemilyminescent substrate is added to the vessel and light generated by the reaction is measured with a luminometer. The light production is proportional to the amount of Treponema pallidum antibodies in the sample. The light quantity measured for a sample allows a determination of the presence of the analyte by comparison with a cut-off value defined during the assay calibration on the instrument. The Access Syphilis reagents are provided in liquid ready-to-use format designed for optimal performance on the Beckman Coulter Access Immunoassay Systems. Each reagent kit contains two reagent packs.
The Access Syphilis assay is a qualitative immunoassay for detecting total antibodies to Treponema pallidum in human serum and plasma, used as an aid in diagnosing syphilis. The device was evaluated for its clinical performance on two systems: the Access 2 Immunoassay System and the Dxl 9000 Access Immunoassay Analyzer. The acceptance criteria and performance data are primarily based on percent agreement with a composite reference method.
Here's a breakdown of the requested information:
1. Table of Acceptance Criteria and Reported Device Performance:
The document does not explicitly state pre-defined acceptance criteria values for Positive Percent Agreement (PPA) and Negative Percent Agreement (NPA). However, the results presented are implicitly the "performance" that aims to demonstrate substantial equivalence. The following table summarizes the reported performance in key clinical evaluation cohorts:
| Performance Metric | Acceptance Criteria (Implicit) | Reported Device Performance (Access 2 & Dxl 9000) |
|---|---|---|
| PPA (Overall Intended Use Population) | High agreement expected, generally >95% | 100% (184/184) [95% CI: 98.0 - 100%] |
| NPA (Overall Intended Use Population) | High agreement expected, generally >95% | 96.7% (890/920) [95% CI: 95.4 - 97.7%] |
| PPA (Retrospective Specimens) | High agreement expected, generally >95% | 100% (398/398) [95% CI: 99.0 - 100%] |
| NPA (Retrospective Specimens) | High agreement expected, generally >95% | 25.0% (1/4) [95% CI: 4.6 - 69.9%] * |
| PPA (High-Risk Individuals) | High agreement expected, generally >95% | 100% (20/20) [95% CI: 83.9 - 100%] |
| NPA (High-Risk Individuals) | High agreement expected, generally >95% | 80.0% (24/30) [95% CI: 62.7 - 90.5%] |
*Note on NPA for Retrospective Specimens: The low NPA for retrospective specimens is due to a very small number of non-reactive specimens in the comparator (only 4), making the percentage highly sensitive to a single misclassification. The 31 reactive results from Access Syphilis where the comparator was nonreactive need further investigation, which the document attributes to "3 specimens were reactive by treponemal immunoassay and nonreactive by RPR and TPPA". This suggests potential discordance with the composite comparator definition for these few cases rather than a broad failing of the device's negative detection.
2. Sample Size for the Test Set and Data Provenance:
The study involved a total of 1,910 specimens for clinical performance evaluation. These specimens were broadly characterized as:
- 1,104 prospectively collected specimens from the intended use population. The age range was 12 to >89 years, with 63.8% female and 36.2% male. This cohort included:
- 399 patients sent for syphilis testing
- 405 pregnant women
- 300 HIV positive patients
- 402 retrospective specimens from patients (including 22 from pregnant females).
- 204 prospectively collected specimens from apparently healthy individuals.
- 150 retrospective specimens from patients with medically diagnosed syphilis (primary, secondary, and latent stages).
- 50 retrospective specimens from individuals at high-risk of sexually transmitted disease.
The provenance of the data is multicenter, meaning specimens were collected from multiple clinical sites. Both retrospective and prospective data were used. The document does not explicitly state the country of origin, but the submission to the U.S. FDA suggests a U.S. or international clinical setting adhering to FDA standards.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of those Experts:
The document does not mention the use of human experts to establish the ground truth for the test set in the traditional sense (e.g., radiologists interpreting images). Instead, the "final comparator result" (ground truth) was established using a composite testing algorithm of multiple FDA-approved laboratory assays, which is standard practice for in vitro diagnostic devices.
4. Adjudication Method for the Test Set:
The adjudication method used a composite testing algorithm as the "final comparator result." This algorithm consisted of:
- An FDA-approved predicate treponemal immunoassay.
- A non-treponemal assay (RPR - Rapid Plasma Reagin).
- A second treponemal assay (TPPA - Treponema Pallidum Particle Agglutination).
The document does not detail a specific "2+1" or "3+1" adjudication process involving human review for discordant results beyond the internal algorithmic comparison. However, for discordant results in the HIV positive patient subpopulation (where the Access Syphilis assay showed lower NPA), an additional FDA-cleared electrochemiluminescent immunoassay was used to further evaluate 28 discordant specimens.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was done:
No, an MRMC comparative effectiveness study was not done. This device is an in vitro diagnostic assay, not an imaging AI device that involves human readers interpreting images with or without AI assistance. The performance is assessed by comparing the device's output to established laboratory reference methods, not human interpretation.
6. If a Standalone (i.e. algorithm only without human-in-the-loop performance) was done:
Yes, the clinical performance study is a standalone performance evaluation. The Access Syphilis assay is an automated immunoassay system that provides results independently. Its performance (reactive/non-reactive) is directly compared against the composite reference standard without human interpretation of the assay's raw output.
7. The type of ground truth used:
The ground truth was established using a composite testing algorithm consisting of results from:
- An FDA-approved predicate treponemal immunoassay.
- A non-treponemal assay (RPR).
- A second treponemal assay (TPPA).
This acts as a "reference standard" or "expert consensus" in the context of laboratory diagnostics, where consensus among multiple established tests determines the final disease status. In some cases, "medically diagnosed syphilis" for specific retrospective cohorts also contributed to the understanding of the samples.
8. The sample size for the training set:
The document does not specify a separate training set size for the Access Syphilis assay. As a chemiluminescent immunoassay, it is a biochemical test, not an AI/machine learning algorithm that typically requires a discrete "training set" in the same way. The assay's parameters would have been optimized during its development phase using internal studies, but these are not referred to as a "training set" in the context of typical AI device submissions.
9. How the ground truth for the training set was established:
Since a distinct "training set" in the AI/ML sense is not described, the method for establishing its ground truth is not applicable in the document. The development of an immunoassay involves extensive analytical and clinical validation, which ensures the reagents and system accurately detect the target antibodies. This process implicitly refines the assay's performance against known positive and negative samples, similar to how a training set might function for an algorithm.
§ 866.3830
Treponema pallidum treponemal test reagents.(a)
Identification. Treponema pallidum treponemal test reagents are devices that consist of the antigens, antisera and all control reagents (standardized reagents with which test results are compared) which are derived from treponemal sources and that are used in the fluorescent treponemal antibody absorption test (FTA-ABS), theTreponema pallidum immobilization test (T.P.I.), and other treponemal tests used to identify antibodies toTreponema pallidum directly from infecting treponemal organisms in serum. The identification aids in the diagnosis of syphilis caused by bacteria belonging to the genusTreponema and provides epidemiological information on syphilis.(b)
Classification. Class II (performance standards).