(77 days)
The assay is intended for use in detecting antibodies to Scl-70 antigen in a single human serum sample. The results of the assay are to be used as an aid in the diagnosis of autoimmune disorders.
The Is-anti-Scl-70 Test Kit System is an enzyme-linked immunosorbent assay (ELISA) for the detection and semi-quantitation of IgG to Scl-70 antigen in human serum.
Here's an analysis of the provided text regarding the acceptance criteria and study for the Is-anti-Scl-70 Test System:
This document focuses on the performance characteristics of an in vitro diagnostic device (IVD), specifically an Enzyme-Linked Immunosorbent Assay (ELISA) for detecting antibodies to Scl-70 antigen. For IVDs, "acceptance criteria" are usually demonstrated through tests of sensitivity, specificity, accuracy (agreement), precision, linearity, and and cross-reactivity compared to a predicate device or established reference methods. The "study" described is the validation study performed to demonstrate these characteristics.
1. Table of Acceptance Criteria and Reported Device Performance
The document doesn't explicitly state "acceptance criteria" as distinct numerical thresholds for approval. Instead, it presents performance characteristics and compares them to a predicate device. For IVDs, the reported performance is what demonstrates the device is "safe and effective for its intended use" as compared to the predicate.
Given the context of a 510(k) summary, the implicit acceptance criteria would be substantial equivalence to the predicate device. This means the new device performs at least as well as, or comparably to, the legally marketed predicate device.
| Performance Metric | Acceptance Criteria (Implicit: Comparable to Predicate) | Reported Device Performance (Manual) | Reported Device Performance (MAGO) |
|---|---|---|---|
| Relative Sensitivity | High agreement with predicate device for positive samples. | 88% (21/24) with 95% CI: 68-97% | 92% (22/24) with 95% CI: 73-99% |
| Relative Specificity | High agreement with predicate device for negative samples. | 100% (138/138) with 95% CI: 97-100% | 99% (138/139) with 95% CI: 96-100% |
| Agreement | High overall agreement with predicate device. | 98% (159/162*) with 95% CI: 95-100% | 98% (160/163) with 95% CI: 95-100% |
| Linearity | High degree of linearity throughout the testing range. | R-squared = 0.9873 | R-squared = 0.9888 |
| Precision (Intra-CV%) | Acceptable variability for both negative and positive samples. (Implicit: typically <15-20% for negative, <10% for positive) | Ranged from 1.5% to 12.6% | Ranged from 1.8% to 20.2% |
| Precision (Inter-CV%) | Acceptable variability for both negative and positive samples. (Implicit: typically <15-20% for negative, <10% for positive) | Ranged from 3.1% to 14.3% | Ranged from 3.5% to 29.4% |
| Cross-reactivity | No significant positive results for non-Scl-70 specificities. | No false positives reported for SSA, SSB, Sm, RNP, Jo-1 samples. | Not explicitly stated separately from manual results in Table 3, assumed to be negligible. |
| Correlation (Manual vs. MAGO) | High correlation between manual and automated methods. | Not Applicable | Pearson correlation = 0.99 |
Note: The precision for MAGO for "NEG CTRL" has an Inter-CV% of 29.4%, which might be considered higher than ideal for some IVD applications, but is presented without further commentary on its acceptability against specific criteria.
2. Sample Size Used for the Test Set and Data Provenance
-
Test Set Sample Size:
- Comparison Testing (Sensitivity, Specificity, Agreement): 162 sera (24 autoimmune patients + 138 normal blood donors). One equivocal sample was excluded from calculations, leading to calculations based on 162 or 163 samples depending on the specific calculation.
- Linearity: The Calibrator, tested after serial two-fold dilutions. The number of individual samples is not specified, but it's a test of the assay's range performance.
- Precision Testing: 6 different sera (2 negative, 4 positive), a calibrator, a positive control, and a negative control. Each was tested 10 times in one run and 10 times over three different days.
- Cross-reactivity: 24 sera (positive for six different autoimmune specificities: SSA, SSB, Sm, RNP, Jo-1, Scl-70).
- Expected Values (Clinical Samples): 50 clinically characterized sera + 100 normal donor sera.
- Correlation (Manual vs. MAGO): 150 samples.
-
Data Provenance:
- The document implies the data was collected for this study, making it prospective in nature, as new testing was performed against the predicate.
- Country of Origin: For "Expected Values," 100 normal donor sera were collected in South Florida, USA. No other specific country of origin is mentioned for the other sample sets, but the applicant (Diamedix Corporation) is based in Miami, Florida, suggesting the studies were likely conducted in the USA.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts
- The document does not specify the number of experts used, nor their specific qualifications (e.g., radiologist with X years of experience).
- For a serological test like this, the "ground truth" for patient samples (autoimmune vs. normal) would typically be established based on clinical diagnosis by physicians following standard diagnostic criteria, often supported by various laboratory tests, including the predicate device or a "gold standard" method. The document mentions "clinically characterized sera" for expected values and "autoimmune patients" for comparison testing, implying such clinical determination.
4. Adjudication Method for the Test Set
- The document does not describe an explicit adjudication method (e.g., 2+1, 3+1).
- For the comparison testing, if the predicate device serves as the primary comparator, any discrepancies might have been resolved by a "third method" as mentioned: "Three sera negative by Is-anti-Scl-70 (manual) and positive by the comparative method were negative when tested by a third method." This implies a form of discrepancy testing/resolution rather than a formal expert adjudication of images/readings.
5. If a Multi Reader Multi Case (MRMC) Comparative Effectiveness Study Was Done, If So, What Was the Effect Size of How Much Human Readers Improve with AI vs Without AI Assistance
- No, this application describes the validation of an in vitro diagnostic (IVD) immunoassay, not an AI-assisted diagnostic imaging device or a device involving human readers interpreting outputs for improved performance. Therefore, an MRMC study and a human-in-the-loop effect size are not applicable to this device type.
6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) Was Done
- Yes, this is a standalone IVD device. The test system (Is-anti-Scl-70 Test Kit) generates qualitative (positive/negative) and semi-quantitative (EU/ml) results.
- The "MAGO" system mentioned in the results appears to be an automated analyzer for processing the immunoassay, but it is still part of the standalone test system and does not involve human interpretation of complex data (like images) that would typically benefit from "human-in-the-loop" AI. The comparison between "Manual" and "MAGO" results demonstrates the performance of the assay itself, with the MAGO representing an automated reading of the assay's output.
7. The Type of Ground Truth Used
- Clinical Diagnosis / Reference Method:
- For the comparative study (sensitivity, specificity, agreement), the predicate device (Helix Diagnostics Enzyme Immunoassay Anti-Scl-70 Antibody Test Kit) served as the primary reference for defining positive and negative samples for establishing relative performance.
- For discordant results, a "third method" was used for resolution, acting as a secondary reference or adjudicator.
- For "Expected Values," samples were categorized as "normal donor sera" and "clinically characterized sera," implying their status was determined by clinical evaluation and/or established diagnostic criteria, not solely by the test results themselves.
- For "Cross-reactivity," samples were "positive for the six autoimmune specificities," indicating their status was established by prior diagnostic testing/clinical diagnosis.
8. The Sample Size for the Training Set
- The document does not describe a separate "training set" in the context of machine learning or AI. This is a traditional IVD assay.
- For an IVD like this, assay development and initial characterization (e.g., reagent optimization, cutoff determination) would occur using various panels of known positive and negative samples, but these are part of assay development rather than a formally designated "training set" for an algorithm in the AI sense. The samples used for linearity, precision, and cross-reactivity could be seen as part of the overall characterization of the assay's behavior.
9. How the Ground Truth for the Training Set Was Established
- As there is no explicitly defined "training set" in the context of an AI/ML algorithm, this question is not applicable in the same way.
- However, for the development of the assay itself and the establishment of its cut-offs, the "ground truth" for samples used in that phase would similarly be established through clinical diagnosis, expert clinical consensus, and/or comparison to established reference methods or predicate devices to define positive and negative cases of Scl-70 antibodies.
{0}------------------------------------------------
510(k) Summary of Safety and Effectiveness
This summary of sefety and effectiveness information is being submitted in accordance with the requirements of SMDA 1990 and 21 CFR 807.92.
The assigned 510(k) number is:
Applicant Information:
| Date Prepared: | January 15, 1997 |
|---|---|
| Name: | Diamedix Corporation |
| Address: | 2140 N. Miami AvenueMiami, FL 33127 |
| Contact Person: | Dr. Lynne Stirling |
|---|---|
| Phone Number: | 305-324-2354 |
| Fax Number: | 305-324-2585 |
Device Information:
| Trade Name: | Is-(immunosimplicity)-anti-Scl-70 Test System |
|---|---|
| Common Name: | Anti-Scl-70 EIA Test |
| Classification Name: | Extractable Antinuclear Antibody |
Equivalent Device:
Helix Diagnostics Enzyme Immunoassay Anti-Scl-70 Antibody Test Kit
Device Description: The Is-anti-Scl-70 Test Kit System is an enzyme-linked immunosorbent assay (ELISA) for the detection and semi-quantitation of IgG to Scl-70 antigen in human serum.
Intended use: The assay is intended for use in detecting antibodies to Scl-70 antigen in a single human serum sample. The results of the assay are to be used as an aid in the diagnosis of autoimmune disorders.
Comparison to Predicate Device:
The Is-anti-Scl-70 Test System is an enzyme-linked immunosorbent assay to detect IgG to Scl-70 in human serum. Purified Scl-70 antigen is attached to a solid phase microtiter well. Diluted test sera are added to each well. If antibodies which recognize the Scl-70 antigen are present in the patient sample, they will bind to the antigen in the well. After incubation, the wells are washed to remove unbound antibody. An enzyme-labeled anti-human immunoglobulin (conjugate) is added to each test well. If antibody is present the enzyme-linked antibody will bind to it. After incubation, the wells are washed to remove unbound conjugate. A substrate solution is then added to each well. If enzyme is present from the prior step, the substrate will be converted to produce a colored product. The reaction is stopped and the color intensity is measured photometrically providing an indirect measure of the specific antibody present in the patient sample.
{1}------------------------------------------------
Summary of Safety and Effectiveness
Performance Characteristics
A. Comparison Testing
The Diamedix Is-anti-Scl-70 Test Kit was evaluated relative to another commercially available anti-Scl-70 The Diamedia Is-anti-OSF 70 Test In normal blood donors and 63 sera from autoimmune patients. The results are summarized in Table 1 below.
| Table 1 | Manual | MAGO | ||||
|---|---|---|---|---|---|---|
| Number ofSera | % | 95%Confidence | Number ofSera | % | 95%Confidence | |
| RelativeSensitivity | 21/24 | 88 | 68-97 | 22/24 | 92 | 73-99 |
| RelativeSpecificity | 138/138 | 100 | 97-100 | 138/139 | 99 | 96-100 |
| Agreement | 159/162* | 98 | 95-100 | 160-163 | 98 | 95-100 |
- One equivocal sample was excluded from the calculations.
Three sera negative by Is-anti-Scl-70 (manual) and positive by the comparative method were negative when tested by a third method. Two sera negative by Is-anti-Scl-70 (MAGO) and positive by the when wethod were negative when tested by a third method. One serum positive by Is-anti-Scl-70 (MAGO) and negative by the comparative method was positive when tested by a third method. Two of the discordant sera were from patient samples.
B. Linearity
Figures 1 and 2 show typical examples of Is-anti-Scl-70 Test Kit linearity. The figures depict the results of the Calibrator tested by Is-anti-Scl-70 after serial two-fold manual dilutions in Sample Diluent. Separate dilutions were tested both manually and with MAGO. The results demonstrate a high degree of linearity for the Is-anti-Scl-70 Test Kit throughout the testing range.
Image /page/1/Figure/9 description: The image contains two line graphs, both titled "Is-anti-Scl-70 Calibrator Linearity". Both graphs plot "Absorbance" on the y-axis versus "Dilution" on the x-axis. The graph on the left has an R-squared value of 0.9873, while the graph on the right has an R-squared value of 0.9888.
Figure 1 Manual Linearity
Figure 2 MAGO Linearity
{2}------------------------------------------------
C. Precision Testing
The precision of the Is-anti-Scl-70 Test Kit was determined at Diamedix by testing six different sera and The precision of the Is and Oo. To 100 itthree different days. The intra- and interassay precision is shown in Table 2 below.
| Table 2 | anti-Scl-70 PRECISION | ||||
|---|---|---|---|---|---|
| OverallMEAN EU/ml | MANUAL | MAGO | |||
| SERUM | INTRA-CV% | INTER-CV% | INTRA-CV % | INTER-CV % | |
| 1 (NEG) | 2.3 | 5.7 | 8.7 | 18.3 | 18.2 |
| 2 (NEG) | 3.9 | 5.8 | 10.3 | 12.4 | 13.2 |
| 3 (POS) | 50.5 | 3.1 | 5.0 | 1.8 | 5.4 |
| 4 (POS) | 29.0 | 4.3 | 4.4 | 5.3 | 9.4 |
| 5 (POS) | 79.6 | 2.7 | 4.5 | 2.1 | 3.5 |
| 6 (POS) | 103.1 | 1.8 | 3.6 | 4.7 | 6.1 |
| CAL | 106.0 | 5.4 | 7.3 | 3.3 | 6.7 |
| POS CTRL | 52.3 | 1.5 | 3.1 | 3.9 | 6.6 |
| NEG CTRL | 1.6 | 12.6 | 14.3 | 20.2 | 29.4 |
D. Crossreactivity
Twenty-four sera positive for the six autoimmune specificities were tested in Is- anti-Scl-70 Test Kit. The results are shown in Table 3.
| ﻧ1• |
|---|
| ------------- |
| Table 3 Crossreactivity | |||
|---|---|---|---|
| Sample | Is-anti-Scl-70 EU/ml | Interp | Specificity |
| 1 | 7.2 | NEG | SSA |
| 2 | 3.7 | NEG | SSA |
| 3 | 1.5 | NEG | SSA |
| 4 | 3.0 | NEG | SSA |
| 5 | 4.0 | NEG | SSB |
| 6 | 2.6 | NEG | SSB |
| 7 | 3.1 | NEG | SSB |
| 8 | 3.6 | NEG | SSB |
| 9 | 3.0 | NEG | Sm |
| 10 | 3.2 | NEG | Sm |
| 11 | 2.5 | NEG | Sm |
| 12 | 4.1 | NEG | Sm |
| 13 | 4.4 | NEG | RNP |
| 14 | 3.0 | NEG | RNP |
| 15 | 4.2 | NEG | RNP |
| 16 | 4.7 | NEG | RNP |
| 17 | 3.3 | NEG | Jo-1 |
| 18 | 5.1 | NEG | Jo-1 |
| 19 | 1.9 | NEG | Jo-1 |
| 20 | 2.9 | NEG | Jo-1 |
| 21 | 134.2 | POS | Scl-70 |
| 22 | 108.0 | POS | Scl-70 |
| 23 | 110.1 | POS | Scl-70 |
| 24 | 165.6 | POS | Scl-70 |
{3}------------------------------------------------
E. Expected Values
The expected values in the normal population were determined by assaying 100 normal donor sera collected in South Florida. Figures 3 and 5 show the distribution of Sm results in the normal population performed manually and on MAGO respectively.
The distribution of EU/ml values for 50 clinically characterized sera along with the 100 normal donor sera are shown in Figures 4 and 6 performed manually and on MAGO respectively.
Image /page/3/Figure/3 description: The image is a graph titled "Is-anti-Scl-70 Normals". The x-axis is labeled "Number of Sera" and ranges from 0 to 100. The y-axis is labeled "EU/ml" and ranges from 0.0 to 30.0. There is a horizontal line at 20.0 on the y-axis, and a curve that increases from approximately 0.0 to 5.0, with one outlier at approximately 18.0.
Image /page/3/Figure/4 description: The image is a graph titled "Is-anti-Scl-70 Expected Values". The x-axis is labeled "Number of Sera" and ranges from 0 to 150. The y-axis is labeled "EU/ml" and ranges from 0.0 to 160.0. The graph shows a line that is relatively flat until the number of sera reaches approximately 140, at which point the line increases sharply.
Figure 3 Manual Normals
Image /page/3/Figure/6 description: The image shows a graph titled "Is-anti-Scl-70 Normals". The x-axis is labeled "Number of Sera" and ranges from 0 to 100. The y-axis is labeled "EU/ml" and ranges from 0.0 to 30.0, with a horizontal line at approximately 20.0. There is also a line that starts near 0 and gradually increases to approximately 4.
Figure 4 Manual Expected Values
Image /page/3/Figure/8 description: The image is a graph titled "Is-anti-Sci-70 Expected Values". The x-axis is labeled "Number of Sera" and ranges from 0 to 150. The y-axis is labeled "EU/ml" and ranges from 0.0 to 140.0. The graph shows a horizontal line at approximately 18 EU/ml, and a curve that rises sharply after the number of sera reaches approximately 140.
Figure 5 MAGO Normals
Figure 6 MAGO Expected Values
F. Correlation of Manual and MAGO Results
Numerical comparison of EU/ml values, between manual and MAGO results for 150 samples in the Is-anti-Scl-70 Test Kit showed a correlation of 0.99 (Pearson).
§ 866.5100 Antinuclear antibody immunological test system.
(a)
Identification. An antinuclear antibody immunological test system is a device that consists of the reagents used to measure by immunochemical techniques the autoimmune antibodies in serum, other body fluids, and tissues that react with cellular nuclear constituents (molecules present in the nucleus of a cell, such as ribonucleic acid, deoxyribonucleic acid, or nuclear proteins). The measurements aid in the diagnosis of systemic lupus erythematosus (a multisystem autoimmune disease in which antibodies attack the victim's own tissues), hepatitis (a liver disease), rheumatoid arthritis, Sjögren's syndrome (arthritis with inflammation of the eye, eyelid, and salivary glands), and systemic sclerosis (chronic hardening and shrinking of many body tissues).(b)
Classification. Class II (performance standards).