(118 days)
Immunoassay for the in vitro qualitative detection of total antibodies (IgG and IgM) to Treponema pallidum in human serum and plasma. The test is intended as an aid in the diagnosis of syphilis infection with clinical signs and symptoms.
The Elecsys Syphilis immunoassay is not intended for use in screening blood or tissue donors. The effectiveness of this assay in testing blood or tissue donors has not been established.
The electrochemiluminescence immunoassay "ECLIA" is intended for use on the cobas e 411 analyzer.
PreciControl Syphilis is intended for the quality control of the Elecsys Syphilis immunoassay on the cobas e 411 analyzer.
The Elecsys Syphilis assay is a fully automated qualitative assay detecting IgG and IgM antibodies to Treponema pallidum, the causative agent of syphilis. Assay results, in conjunction with other laboratory results and clinical information, may be used to provide presumptive evidence of active or previous infection with Treponema pallidum in persons with signs and symptoms of syphilis, as well as in patients at risk for syphilis infection. This assay does not determine the stage of infection or associated disease.
PreciControl Syphilis is a lyophilized control based on human serum. It is used for monitoring the accuracy of the Elecsys Syphilis immunoassay.
Here's a breakdown of the acceptance criteria and study details for the Elecsys Syphilis assay, based on the provided document:
1. Table of Acceptance Criteria and Reported Device Performance
The document doesn't explicitly state quantitative acceptance criteria for sensitivity and specificity. Instead, it presents the performance of the Elecsys Syphilis assay compared to a "final comparator algorithm" or "medically diagnosed individuals" and implies that "good agreement" and "substantial equivalence" are the overarching criteria.
Assuming the implicit acceptance criteria are high levels of positive percent agreement (PPA) and negative percent agreement (NPA) with the established comparator, here's a table summarizing the reported performance:
| Performance Metric | Acceptance Criteria (Implicit) | Reported Device Performance (Elecsys Syphilis) | Study Context |
|---|---|---|---|
| PPA (Sensitivity) | High agreement (e.g., >95%)* | 100% (228/228) | Prospective Cohorts (Total) |
| 95.00 to 99.83% (95% CI) | Staged (Retrospective) | ||
| 98.7% (155/157) | Overall Retrospective Cohorts | ||
| 100% (15/15) | Pregnant (Retrospective) | ||
| NPA (Specificity) | High agreement (e.g., >95%)* | 99.2% (2038/2054) | Prospective Cohorts (Total) |
| 99.40 to 99.96% (95% CI) | Routine Syphilis (Prospective) | ||
| 92.58 to 97.63% (95% CI) | HIV (Prospective) | ||
| 100% (301/301) | Pregnant (Prospective) | ||
| 100% (12/12) | Staged (Retrospective) | ||
| 100% (12/12) | Overall Retrospective Cohorts |
*Note: The document states "good agreement" and "substantially equivalent," which generally implies high sensitivity and specificity for diagnostic assays. The specific quantitative thresholds are not explicitly defined as "acceptance criteria" but are demonstrated by the reported results.
2. Sample Sizes and Data Provenance
-
Test Set Sample Sizes:
- Prospective Cohorts: Total of 2282 samples.
- Routine Syphilis testing: 1524 subjects
- HIV positive subjects: 457 subjects
- Pregnant women: 301 subjects
- Retrospective Medically Diagnosed Individuals: Total of 169 specimens.
- Pregnant positive women: 15
- Subjects medically diagnosed with syphilis at different stages: 154
- Apparently Healthy Individuals: 209 specimens.
- Analytical Specificity: 266 human clinical samples from patients with medical conditions unrelated to Syphilis.
- Prospective Cohorts: Total of 2282 samples.
-
Data Provenance: The document generally refers to "human serum and plasma" samples.
- Prospective Cohorts: "prospectively collected samples of the intended use population." A multi-center study was conducted in the EU and the US for cut-off validation, implying global data.
- Retrospective Medically Diagnosed Individuals: "pre-selected retrospective cohort."
- The "apparently healthy individuals" and "analytical specificity" samples are also clinical.
-
Retrospective or Prospective: Both retrospective and prospective data were used for clinical performance evaluation.
3. Number of Experts and Qualifications for Ground Truth
The document does not specify the "number of experts" or their "qualifications" used to establish the ground truth.
4. Adjudication Method for the Test Set
The ground truth for the clinical performance studies was established using a "composite algorithm" rather than expert adjudication.
- Composite Algorithm: For the prospective cohorts, "all samples were tested according to a composite algorithm using FDA-cleared tests that included the predicate Syphilis immunoassay, the Rapid Plasma Reagin (RPR) non-treponemal specific assay and the Treponema pallidum Particle Agglutination, treponemal-specific assay."
- For the retrospective medically diagnosed individuals, the ground truth was based on "medically diagnosed individuals" and the "final comparator results" from the composite algorithm.
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
No, a Multi-Reader Multi-Case (MRMC) comparative effectiveness study comparing human readers with and without AI assistance was not conducted. This device is an in-vitro diagnostic (IVD) immunoassay, not an AI-assisted diagnostic imaging or interpretation tool for human readers.
6. Standalone (Algorithm Only) Performance
Yes, the studies presented are for standalone (algorithm only) performance. The Elecsys Syphilis assay is an automated immunoassay that provides a qualitative result (reactive or non-reactive) without human interpretation in the loop for the primary result generation. Its performance is measured against established diagnostic algorithms and clinical diagnoses.
7. Type of Ground Truth Used
The primary ground truth used for evaluating clinical performance was a composite algorithm of FDA-cleared tests (predicate immunoassay, RPR, and TPPA). Additionally, a medical diagnosis was used for the retrospective cohort.
8. Sample Size for the Training Set
The document does not explicitly state the sample size for a "training set." This type of IVD immunoassay typically involves extensive assay development and optimization rather than a "training set" in the machine learning sense. The "cut-off determination" process likely utilized a set of samples, but these are not explicitly termed a "training set" with a specified size.
- Cut-off Determination: "native human serum samples were measured with a prototype lot of the Elecsys Syphilis assay and a preliminary cut-off was set."
- Final Validation: "Final validation of the cut-off was done in multi-center studies conducted in the EU and the US."
9. How the Ground Truth for the Training Set Was Established
As there's no explicitly defined "training set" in the machine learning context, the ground truth for cut-off determination would have been established through a combination of:
- Known positive and negative samples: These would have been derived from individuals with confirmed syphilis infections (using reference methods or clinical diagnosis) and confirmed non-infected individuals.
- Preliminary cut-off setting: Based on the distribution of results from these known samples.
- Comparability with commercially available assays: Used to refine and validate the cut-off.
§ 866.3830
Treponema pallidum treponemal test reagents.(a)
Identification. Treponema pallidum treponemal test reagents are devices that consist of the antigens, antisera and all control reagents (standardized reagents with which test results are compared) which are derived from treponemal sources and that are used in the fluorescent treponemal antibody absorption test (FTA-ABS), theTreponema pallidum immobilization test (T.P.I.), and other treponemal tests used to identify antibodies toTreponema pallidum directly from infecting treponemal organisms in serum. The identification aids in the diagnosis of syphilis caused by bacteria belonging to the genusTreponema and provides epidemiological information on syphilis.(b)
Classification. Class II (performance standards).