(227 days)
The ASI Automated RPR (rapid plasma reagin) Test for Syphilis, for use on the ASI Evolution Automated Syphilis Analyzer, is a qualitative nontreponemal flocculation test for the detection of reagin antibodies in human serum and plasma as a screening test for serological evidence of syphilis. All reactive RPR test samples should be further tested with a treponemal test. The ASI Automated RPR Test for Syphilis for use on the ASI Evolution produces only a reactive or non-reactive result and does not report RPR titers for reactive samples. The ASI Automated RPR Test for Syphilis is for professional use only. The test is intended to be used for in vitro diagnostic testing. The ASI Evolution is intended to be used as a fully automated analyzer to objectively interpret the results of the ASI automated RPR test for syphilis. The ASI Evolution is designed to provide standardized test interpretation and to provide for storage, retrieval, and transmittal of the test results. It is intended to be acquired, possessed and used only by health care professionals. The ASI Evolution analyzer, in conjunction with the ASI Automated RPR Test is intended to be used for in vitro diagnostic testing. The ASI Automated RPR Test for Syphilis for use on the ASI Evolution produces only a reactive or non-reactive result and does not report RPR titers for reactive samples.
The ASI Evolution is an integrated digital particle analyzer designed to objectively interpret certain slide agglutination tests manufactured by Arlington Scientific, Inc. (ASI). The ASI Evolution fully automates the sample and reagent handling steps of the test procedure. Qualitative tests are performed by laboratory professionals who use the ASI Evolution to provide standardized test interpretation using criteria that define reactive and nonreactive aqqlutination reactions. The ASI Evolution employs a camera that uses light reflectance to create a highly sensitive and high-resolution image of the agglutination immunoassay. This image is then analyzed by the proprietary software algorithm to interpret the agglutination pattern. The ASI Evolution further provides tools that enable the creation, storage, retrieval and transmittal of the test results. The ASI Automated RPR Test for Syphilis reagents include the following: CARBON ANTIGEN - 0.003% cardiolipin, 0.020-0.022% lecithin, 0.09% cholesterol, charcoal (activated) as visual enhancer, phosphate buffer, 0.1% sodium azide as preservative and stabilizers. CONTROLS (REACTIVE, WEAK REACTIVE, NONREACTIVE) - Human serum or defibrinated plasma (liquid), with 0.1% sodium azide as preservative. Reagents have two-year expiration dating from date of manufacture. The specific expiration date is located on the label on the vial.
This document describes the regulatory approval for the ASI Automated RPR Test for Syphilis ASI Evolution. The product is a qualitative nontreponemal flocculation test for the detection of reagin antibodies in human serum and plasma for syphilis screening. It is designed to be used with the ASI Evolution Automated Syphilis Analyzer, which objectively interprets test results as reactive or non-reactive.
Here is a summary of the acceptance criteria and study information:
1. Table of Acceptance Criteria and Reported Device Performance
The acceptance criteria for the ASI Automated RPR Test for Syphilis ASI Evolution are based on demonstrating substantial equivalence to its predicate device, the ASI RPR Card Test for Syphilis on the ASiManager-AT. The key performance metrics are positive and negative agreement rates.
| Performance Metric | Acceptance Criteria (Implicitly based on predicate equivalence) | Reported Device Performance |
|---|---|---|
| Prospective Serum Samples (N=1068) | ||
| Positive Agreement (Reactive) | High agreement with predicate device (ASiManager-AT) | 99.13% (95% CI: 95.25% - 99.98%) |
| Negative Agreement (Nonreactive) | High agreement with predicate device (ASiManager-AT) | 99.9% (95% CI: 99.42% - 100%) |
| Retrospective Serum Samples (N=10) | ||
| Positive Agreement (Reactive) | High agreement with predicate device (ASiManager-AT) | 100% (95% CI: 59.04% - 100%) |
| Negative Agreement (Nonreactive) | High agreement with predicate device (ASiManager-AT) | 100% (95% CI: 29.24% - 100%) |
| Retrospective Plasma Samples (N=1003) | ||
| Positive Agreement (Reactive) | High agreement with predicate device (ASiManager-AT) | 100% (95% CI: 69.15% - 100%) |
| Negative Agreement (Nonreactive) | High agreement with predicate device (ASiManager-AT) | 100% (95% CI: 99.63% - 100%) |
| Precision | 100% repeatability for a variety of reactive and nonreactive samples | 100% for all 10 tested samples |
| Reproducibility (3 sites, 7 samples) | 100% agreement with expected results across sites and operators | 100% for all 7 tested samples across all 3 sites |
| Reproducibility (3 sites, 448 samples) | 100% agreement with expected results across sites | RPR Nonreactive: 100% (144/144) |
| RPR Reactive: 100% (1200/1200) | ||
| Cross-Reactivity/Interfering Substances | No interference or cross-reactivity with specified conditions | No interference observed in all tested categories |
| Carry-Over | No contamination of nonreactive samples from adjacent reactive samples | All results as expected (no carry-over) |
| On-board Stability | 100% agreement for samples and reagent for 8 hours on instrument | 100% agreement for all samples and operators for 8 hours |
| Frozen vs. Refrigerated Testing | 100% agreement between frozen and refrigerated samples | 100% agreement for all samples |
2. Sample Sizes Used for the Test Set and Data Provenance
- Prospective Samples: 1,068 individual samples. Collected at two different Departments of Public Health Labs. Data provenance is prospective.
- Retrospective Serum Samples (Agreement Study): 10 individual samples. Collected from various reference labs and serum and plasma vendors from across the United States. Data provenance is retrospective.
- Retrospective Plasma Samples (Agreement Study): 1,003 individual samples. Collected from various reference labs and serum and plasma vendors from across the United States. Data provenance is retrospective.
- ASI Evolution Characterized Specimen Testing (Clinical Diagnosis): Total of 1,068 + 1,013 = 2,081 samples (Total samples from prospective and retrospective studies)
- Prospective Random Samples:
- Site a: 567 Serum samples (country not specified, likely US based on other descriptions)
- Site b: 501 Serum samples (country not specified, likely US based on other descriptions)
- Retrospective Samples:
- Site c: 17 Known infected samples (serum & plasma, from across the United States)
- Site c: 996 Known uninfected samples (serum & plasma, from across the United States)
- Total Retrospective samples: 1013
- Prospective Random Samples:
- Performance with Samples from Pregnant Women: 250 non-reactive and 30 reactive samples from pregnant women. Serum samples. Country not specified, likely US.
- Precision Testing: 10 samples (3 nonreactive, 7 reactive at various titers), each tested 192 times (total 1920 tests).
- Reproducibility Testing (7 samples): 7 samples (2 nonreactive, 5 reactive at various titers), tested at 3 sites, each sample tested 180 times across various conditions (total 1260 tests).
- Reproducibility Testing (448 samples): 448 retrospective serum samples (48 nonreactive, 400 reactive), tested at 3 sites (total 1344 tests). Samples collected from various reference labs and serum and plasma vendors from across the United States.
- Cross Reactivity/Interfering Substances: 368 samples across various categories (e.g., ANA (+), HIV (+), pregnant women, etc.).
- Carry-Over: 48 aliquots per run (24 reactive, 24 nonreactive). Run over 5 days.
- On-board Stability Testing: 4 samples (1 nonreactive, 3 reactive at various titers), each run in triplicate morning and afternoon for 5 days by 2 operators (total 120 tests).
- Frozen vs. Refrigerated Testing: Total of 55 samples (20 non-reactive, 35 reactive at various titers) tested as both frozen and refrigerated.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications
The document does not explicitly state the "number of experts" or their "qualifications" for establishing the ground truth for the test set in the comparative studies.
- For the comparative studies against the ASiManager-AT, the predicate device's results (interpreted by "trained laboratory professionals") served as the reference. These professionals are implicitly qualified by their training to interpret RPR tests.
- For the "ASI Evolution Characterized Specimen Testing," the "Expected results are based on known clinical diagnosis." This implies that clinical experts (e.g., physicians, specialists) established the diagnosis, which then served as a proxy for ground truth.
4. Adjudication Method for the Test Set
The document does not explicitly describe an adjudication method like 2+1 or 3+1. For the comparative studies, the results of the ASiManager-AT, interpreted by trained laboratory professionals, were considered the comparator. In cases of discrepancies or challenges, no specific adjudication process is detailed for these studies. For the Characterized Specimen Testing, the "known clinical diagnosis" itself serves as the ground truth, implying no further adjudication on the test results against the diagnosis.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was done
No, a traditional MRMC comparative effectiveness study, where multiple human readers interpret cases with and without AI assistance to measure improvement in human performance, was not performed. The study focused on the standalone performance of the ASI Evolution device compared to the predicate device (ASiManager-AT) and known clinical diagnoses. When human operators were involved (e.g., for ASiManager-AT interpretation), they were trained professionals, but the study did not measure their improvement with AI assistance.
6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) was done
Yes, the studies primarily assessed the standalone performance of the ASI Evolution. The device is described as a "fully automated analyzer to objectively interpret the results" of the RPR test. The comparison against the ASiManager-AT (which involves human interpretation of results) and known clinical diagnoses evaluates the ASI Evolution's direct output.
7. The Type of Ground Truth Used
The ground truth used varied:
- Comparative Studies: The results obtained from the predicate device, the ASI RPR Card Test for Syphilis on the ASiManager-AT, interpreted by trained laboratory professionals, served as the comparator/reference.
- Characterized Specimen Testing: "Known clinical diagnosis" was used as the ground truth. This implies diagnoses established by medical professionals through various clinical and laboratory indicators.
- Precision, Reproducibility, Cross-Reactivity, Carry-Over, On-board Stability, Frozen vs. Refrigerated Testing: Expected results for controls and characterized samples (e.g., "RPR nonreactive," "RPR reactive 1:1 titered samples") were used as the ground truth. These expected results are typically established during the characterization and manufacturing of the control materials themselves.
8. The Sample Size for the Training Set
The document does not explicitly state the sample size used for the training set of the ASI Evolution's algorithm. It mentions that the "same proprietary interpretive algorithm used in the predicate device (ASiManager-AT) is used in the ASI Evolution." This suggests that the algorithm was developed and potentially trained using data prior to this submission, possibly with data collected for the ASiManager-AT validation. However, details of that training are not provided here.
9. How the Ground Truth for the Training Set Was Established
As the document does not specify a training set for the ASI Evolution's algorithm, it also does not detail how the ground truth for such a set was established. Given the claim that the algorithm is the "same proprietary interpretive algorithm" as in the predicate device, it is likely that the ground truth for any initial algorithm development would have been established through expert human interpretation of RPR flocculation patterns over a large set of samples, potentially correlated with confirmatory treponemal tests or clinical diagnoses.
§ 866.3820
Treponema pallidum nontreponemal test reagents.(a)
Identification. Treponema pallidum nontreponemal test reagents are devices that consist of antigens derived from nontreponemal sources (sources not directly associated with treponemal organisms) and control sera (standardized sera with which test results are compared) used in serological tests to identify reagin, an antibody-like agent, which is produced from the reaction of treponema microorganisms with body tissues. The identification aids in the diagnosis of syphilis caused by microorganisms belonging to the genusTreponema and provides epidemiological information on syphilis.(b)
Classification. Class II (performance standards).