(142 days)
The ASI Automated RPR (rapid plasma reagin) Test for Syphilis, for use on the ASI Evolution Automated Analyzer, is a qualitative and semiquantitative flocculation test for the detection of nontreponemal antibodies in human serum and plasma to aid in the diagnosis of syphilis. All reactive RPR test samples should be further tested with a treponemal test to determine serological evidence of syphilis infection. The test is intended to be used for in vitro diagnostic testing.
The ASI Evolution is an integrated digital particle analyzer designed to objectively interpret certain slide agglutination tests manufactured by Arlington Scientific. Inc. (ASI). The ASI Evolution fully automates the sample and reagent handling steps of the test procedure. Qualitative and semiquantitative tests are performed by laboratory professionals who use the ASI Evolution to provide standardized test interpretation using criteria that define reactive and nonreactive agglutination reactions.
The ASI Evolution employs a camera that uses light reflectance to create a highly sensitive and high-resolution image of the agglutination immunoassay. This image is then analyzed by the proprietary software algorithm to interpret the agglutination pattern.
The ASI Evolution further provides tools that enable the creation, storage, retrieval and transmittal of the test results.
The ASI Automated RPR Test for Syphilis reagents include the following:
CARBON ANTIGEN - 0.003% cardiolipin, 0.020-0.022% lecithin, 0.09% cholesterol, charcoal (activated) as visual enhancer, phosphate buffer, 0.1% sodium azide as preservative and stabilizers.
CONTROLS (REACTIVE, WEAK REACTIVE, NONREACTIVE) - Human serum or defibrinated plasma (liquid), with 0.1% sodium azide as preservative.
Reagents have two-year expiration dating from date of manufacture. The specific expiration date is located on the label on the vial.
Here's a breakdown of the acceptance criteria and the study proving the device meets them, based on the provided FDA 510(k) summary:
This submission is for a new algorithm for an existing device, the ASI Evolution Automated Syphilis Analyzer, used with the ASI Automated RPR Test for Syphilis. The study aims to demonstrate substantial equivalence of the new algorithm to the original algorithm. Therefore, the acceptance criteria and study design are geared towards showing comparable performance rather than de novo validation against a clinical gold standard.
Acceptance Criteria and Reported Device Performance
The acceptance criteria are implicitly defined by the goal of demonstrating substantial equivalence to the original algorithm. This means the new algorithm's performance (positive agreement and negative agreement) should be comparable to or better than the original algorithm's. While explicit numerical thresholds for acceptance are not stated for agreement, the 95% Confidence Intervals are provided, which typically are evaluated against a pre-defined acceptance range (e.g., lower bound of 90% for positive agreement). The reproducibility section demonstrates 100% agreement, which is a strong indicator of meeting a high standard for reproducibility.
Table of Acceptance Criteria (Implicit) and Reported Device Performance (Calculated from provided data)
| Performance Metric | Implicit Acceptance Criteria (Comparable to original algorithm, with high confidence) | Reported Device Performance (New Algorithm vs. Original Algorithm) |
|---|---|---|
| Retrospective Serum Samples | ||
| Serum Positive Agreement | High agreement (e.g., >90% lower bound 95% CI) | 100% (95% Cl = 96.03% - 100%) |
| Serum Negative Agreement | High agreement (e.g., >90% lower bound 95% CI) | 99.23% (95% Cl = 98.34% - 99.72%) |
| Retrospective Plasma Samples (Total) | ||
| Total Plasma Positive Agreement | High agreement (e.g., >90% lower bound 95% CI) | 95.97% (95% Cl = 90.91% - 98.27%) |
| Total Plasma Negative Agreement | High agreement (e.g., >90% lower bound 95% CI) | 99.87% (95% Cl = 99.27% - 100.00%) |
| Reproducibility | Consistent results across runs and operators | 100% reproducibility for all tested samples (60/60 for each sample, 95% CI 94.04-100) |
| End-point Titer Testing (Semiquantitative) | Within +/- 1 titer of expected result for reactive samples; Nonreactive samples must be nonreactive. | All samples (80/80 data points per sample) met the criteria. |
Details of the Study Proving Device Meets Acceptance Criteria
1. Sample Sizes Used for the Test Set and Data Provenance:
- Retrospective Study Comparing Algorithms:
- Serum Samples: 872 individual retrospective samples.
- Plasma Samples: 890 individual retrospective samples.
- Pregnant Women Testing: 280 samples (30 reactive, 250 nonreactive).
- Total Samples for Algorithm Comparison: 872 (serum) + 890 (plasma) + 280 (pregnant women) = 2042 samples.
- Reproducibility Study:
- 7 samples (2 nonreactive, 2 reactive 1:2, 1 reactive 1:4, 1 reactive 1:8, 1 reactive 1:16).
- Each sample tested in duplicate within the panel, for 5 non-consecutive days, producing 60 data points per sample (60/60 reported). Total 7 samples x 60 data points = 420 data points.
- End-point Titer Testing:
- 9 samples (2 nonreactive, 7 reactive with varying titers).
- Each sample tested in 8 replicates on 10 different days, resulting in 80 data points for each sample. Total 9 samples x 80 data points = 720 data points.
- Data Provenance: Retrospective samples, with identifiers removed, collected from different Departments of Public Health Labs and Blood Banks. No specific country of origin is mentioned, but "U.S. Food & Drug Administration" implies U.S. data.
2. Number of Experts Used to Establish Ground Truth for the Test Set and Qualifications of Those Experts:
- For Algorithm Equivalence Study: The "ground truth" for the algorithm comparison study was the results interpreted by the original ASI Evolution algorithm (K173376 and K182391). This device was already cleared by the FDA, implying its outputs were considered reliable. No human experts were explicitly stated to establish ground truth for this direct comparison between two algorithms. However, for discordant results:
- The 6 discordant serum results (new algorithm nonreactive, original algorithm reactive) were investigated and tested with a treponemal test and found to be reactive. This implies confirmation by an independent, more specific test, which is a common form of "ground truth" for syphilis diagnosis.
- The 6 discordant plasma results (1 new reactive/original nonreactive, 5 new nonreactive/original reactive) were investigated. The new reactive/original nonreactive sample was tested with a treponemal test and found to be nonreactive. The 5 new nonreactive/original reactive samples were attributed to bubbles or artifacts in the test well. This also points to investigation and confirmation using a more definitive test or root cause analysis, serving as a form of expert adjudication or outcome-based truth.
- For Reproducibility and End-point Titer Testing: The "Expected Result" for these tests was established either by the manual interpretation method prior to testing (for end-point titer samples) or by the known characteristics of the control samples used. This implies laboratory or subject matter expert consensus or established reference values.
3. Adjudication Method for the Test Set:
- For the algorithm comparison (serum and plasma), discordant results between the new and original algorithms were investigated using a treponemal test (for biological confirmation) or attributed to technical issues (e.g., bubbles/artifacts). This serves as an adjudication method based on a more definitive test or root cause analysis.
- For the pregnant women testing, the comparison was made against the "ASI RPR Card Test for Syphilis on the ASiManager-AT Result," which acts as the reference.
- No explicit "2+1" or "3+1" human expert adjudication method was described as the primary ground truth establishment for the algorithm comparison, as the goal was algorithm-to-algorithm equivalence rather than algorithm-to-human.
4. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was done:
- No, an MRMC comparative effectiveness study was not performed as the primary validation for this submission. The study focused on demonstrating substantial equivalence between two automated algorithms (new vs. original ASI Evolution algorithm), not on comparing human readers with and without AI assistance.
- Effect size of human readers improving with AI vs. without AI assistance: Not applicable to this study design.
5. If a Standalone (i.e., algorithm only without human-in-the-loop performance) was done:
- Yes, the core of the submission is a standalone performance study comparing the new algorithm's interpretation capabilities against the existing, cleared original algorithm on the same instrument. The "Performance Data" section explicitly states, "A comparison of the digital interpretation of the results from the ASI Evolution using the original interpretation algorithm... to establish substantial equivalence to the interpretation made by the ASI Evolution using the new interpretation algorithm was conducted."
6. The Type of Ground Truth Used:
- Algorithm vs. Algorithm (Predicate Device as Reference): The primary "ground truth" for the main algorithm comparison study was the output of the predicate device's algorithm (original ASI Evolution algorithm).
- Confirmatory Treponemal Test: For discrepant results in the algorithm comparison studies, a treponemal test was used as a more definitive clinical ground truth to resolve ambiguities.
- Manual Interpretation Methods / Known Reactivity: For reproducibility and end-point titer testing, the "expected result" was based on prior manual interpretation methods or known characteristics of control samples.
- No pathology or direct outcomes data was cited as the primary ground truth for the device clearance.
7. The Sample Size for the Training Set:
- Not specified in the provided document. The document describes a study comparing the new algorithm's performance against the old one (test set). It does not provide details about if or how the new algorithm itself was "trained" using specific data. Given the context of medical device clearance, it's presumed that the algorithm development (training, if any) would have occurred prior to this validation study.
8. How the Ground Truth for the Training Set was Established:
- Not specified in the provided document. As the training set details are not provided, neither is the method for establishing its ground truth.
§ 866.3820
Treponema pallidum nontreponemal test reagents.(a)
Identification. Treponema pallidum nontreponemal test reagents are devices that consist of antigens derived from nontreponemal sources (sources not directly associated with treponemal organisms) and control sera (standardized sera with which test results are compared) used in serological tests to identify reagin, an antibody-like agent, which is produced from the reaction of treponema microorganisms with body tissues. The identification aids in the diagnosis of syphilis caused by microorganisms belonging to the genusTreponema and provides epidemiological information on syphilis.(b)
Classification. Class II (performance standards).