(265 days)
The Gold Standard Diagnostics Antibody (ANA) Screen ELISA Test Kit is a qualitative assay for the detection of ANAs in human serum. The assay collectively detects in one well ANAs against double stranded DNA (dsDNA), SSA (Ro60 and Ro52), SSB (La), Sm, Sm/RNP, Scl-70, Jo-1, Ribosomal P, and Centromeric antibodies along with sera positive for immunofluorescent HEp-2 ANAs.
The assay is used as an aid in the diagnosis of Systemic Lupus Erythematosus, Mixed Connective Tissue Disease, Sjögren's Syndrome, Progressive Systemic Sclerosis, and Polymyositis/Dermatomyositis, and should be used in conjunction with other laboratory tests and clinical findings.
The assay requires a total of 90 minutes incubation time. The test uses antigen coated on microtiter wells. Serum is added to each well and incubated for 30 minutes at room temperature. If antibodies are present they will bind to the antigen in the well. Unbound serum is removed by washing the wells three times. An HRP-conjugated anti-human IgG is then added to each well and incubated for 30 minutes at room temperature. If antibody is present, it will bind to the antibody attached to the antigen on the well. The wells are again washed three times to remove any unbound conjugate. A TMB substrate is added to each well and incubated for 30 minutes at room temperature. If enzyme is present, it will react with the substrate to generate a colored product. After the incubation period, the reaction is stopped with a Stop Solution and the color intensity is measured spectrophotometrically.
The provided text describes the performance evaluation of the "Gold Standard Diagnostics Anti-Nuclear Antibody (ANA) Screen ELISA Test Kit" and its comparison to a predicate device.
Here's a breakdown of the requested information based on the document:
1. Table of Acceptance Criteria and Reported Device Performance
The document does not explicitly state pre-defined acceptance criteria (e.g., "The device must achieve >X% sensitivity and >Y% specificity"). Instead, it presents the device's performance in comparison to a predicate device and provides clinical sensitivity and specificity for various conditions.
However, we can infer performance targets by looking at the results that demonstrate equivalence or acceptable clinical utility. For the purpose of this analysis, I will present the reported performance of the device, especially concerning agreement with the predicate and clinical sensitivity/specificity.
| Performance Metric | Reported Device Performance (When Equivocal Treated as Positive) | Reported Device Performance (When Equivocal Treated as Negative) |
|---|---|---|
| Agreement with Predicate Device (Overall Results) | ||
| Percent Positive Agreement | 94.9% (C.I. 91.5% - 97.2%) | 92.3% (C.I. 88.4% - 95.2%) |
| Percent Negative Agreement | 92.2% (C.I. 89.7% - 94.2%) | 96.2% (C.I. 94.3% - 97.6%) |
| Overall Agreement | 93.0% (C.I. 91.1% - 95.7%) | 94.9% (C.I. 93.2% - 96.3%) |
| Clinical Performance (against clinical diagnosis - CTD vs. Non-CTD & Others) | ||
| Sensitivity (for Connective Tissue Disease (CTD)) | 83.5% (95% C.I. = 79.1% - 86.0%) | 79.2% (95% C.I. = 75.4% - 82.7%) |
| Specificity (for Non-Connective Tissue Disease & Others) | 96.7% (95% C.I. = 93.6% - 98.6%) | 96.7% (95% C.I. = 93.6% - 98.6%) |
| Overall Agreement (Clinical) | 87.8% (95% C.I. = 85.2% - 90.0%) | 84.9% (95% C.I. = 82.1% - 87.4%) |
| Reproducibility (Qualitative Result Matching Expected) | ||
| Sample 1 (Negative) | 100% (Site 1, 2, 3) | |
| Sample 2 (Equivocal) | 95% (Site 1, 2), 100% (Site 3) | |
| Sample 3 (Positive) | 100% (Site 1, 2, 3) | |
| Precision (Qualitative Result Matching Expected) | ||
| Positive Samples (3) | 100% | |
| Equivocal Sample (1) | 90% | |
| Negative Samples (2) | 90% (one sample), 100% (one sample) | |
| Lot-to-Lot Comparison (Qualitative Result Matching Expected) | ||
| Positive Sample | 100% (all 3 lots) | |
| Equivocal Sample | 100% (all 3 lots) | |
| Negative Sample | 100% (all 3 lots) | |
| Interfering Substances (Mean Percent Inhibition) | ||
| Hemoglobin | -2.1% | |
| Bilirubin | -3.0% | |
| Rheumatoid Factor | 8.8% | |
| Triglycerides | -1.4% | |
| Heterophile (multiple concentrations) | 5.4%, 2.2%, -0.6% | |
| Prozone (Hook) Effect (Max ratio calculated without observing prozone effect) | 12 |
2. Sample size used for the test set and the data provenance
- Test Set for Equivalence to Predicate Device: 848 samples
- Test Set for Clinical Sensitivity/Specificity: 753 samples (510 CTD samples, 243 Non-CTD & others)
- Test Set for Analyte Specificity (initial comparison): 55 samples (5 samples for each of 11 analytes)
- Test Set for Analytical Specificity (CDC & AMLI samples): 20 samples (10 CDC, 10 AMLI)
- Test Set for Precision: 7 samples
- Test Set for Reproducibility: 3 samples tested across 3 sites
- Test Set for Lot-to-Lot Comparison: 3 samples tested across 3 lots
- Test Set for Expected Values (Normal Donors): 99 normal blood donors (51 males, 48 females)
The data provenance is not explicitly stated in terms of country of origin, but it is indicated that the clinical comparison was conducted at three different sites. The samples for clinical sensitivity and specificity were "clinically diagnosed connective tissue and non-connective tissue disease sera." The study is retrospective in the sense that existing samples were tested, not samples collected specifically for a prospective clinical trial.
3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts
The document does not specify the number or qualifications of experts for establishing ground truth for the clinical samples. For the comparison to the predicate device, the "ground truth" was essentially the result from the "commercially available Anti-Nuclear Antibody ELISA test," which is the predicate device itself. For the clinical sensitivity/specificity, the ground truth was based on "Clinical Diagnosis" (e.g., Systemic Lupus Erythematosus (SLE), Systemic Sclerosis (SSc)), but the process of how these clinical diagnoses were definitively established is not detailed, nor is the number or qualifications of the clinicians involved.
4. Adjudication method for the test set
The document does not describe an explicit adjudication method (like 2+1 or 3+1) for the test sets. For the comparison to the predicate, it presents the agreement directly. For clinical sensitivity and specificity, it relies on existing clinical diagnoses. For the analytical specificity with CDC and AMLI samples, it states, "All CDC and AMLI samples gave their expected results," implying predefined expected results without mentioning an adjudication process for these specific samples.
5. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance
This section is not applicable. The device is an ELISA-based diagnostic test kit, not an AI-assisted diagnostic tool that involves human readers interpreting images or data with and without AI assistance. Therefore, an MRMC study comparing human readers with and without AI assistance was not conducted.
6. If a standalone (i.e., algorithm only without human-in-the-loop performance) was done
This concept is not directly applicable in the context of this ELISA test kit. The "device" itself (the ELISA kit) is a standalone test that produces a result (OD Ratio, then converted to units and interpreted as Positive, Equivocal, or Negative). The performance reported for agreement with the predicate and clinical sensitivity/specificity is the standalone performance of the test kit against established comparative methods or clinical diagnoses. There isn't an "algorithm" in the typical sense of AI, but the assay procedure and interpretation rules define its standalone performance.
7. The type of ground truth used (expert consensus, pathology, outcomes data, etc.)
The ground truth used depends on the specific study:
- For comparison with the predicate device: The results from the legally marketed predicate device (Aesku, Inc. Aeskulisa ANA HEp-2) served as the comparative "ground truth."
- For clinical sensitivity and specificity: Clinical diagnoses (e.g., Systemic Lupus Erythematosus (SLE), Mixed Connective Tissue Disease (MCTD), Rheumatoid Arthritis (RA)) were used. It's implied these diagnoses were established through standard clinical practice, though the specifics of that establishment (e.g., expert consensus, pathology, specific diagnostic criteria) are not detailed in this document.
- For analytical specificity (CDC and AMLI samples): These were reference sera with known specificities, so their predefined "expected results" acted as the ground truth.
8. The sample size for the training set
The document does not explicitly mention a "training set" in the context of an algorithm. For an ELISA kit, development typically involves optimizing reagents and protocols. The studies described are performance evaluation studies (test sets) rather than algorithm training.
9. How the ground truth for the training set was established
As there is no "training set" in the sense of machine learning algorithms, this question is not applicable. The development of an ELISA kit involves laboratory optimization and validation, rather than learning from a labeled training dataset.
§ 866.5100 Antinuclear antibody immunological test system.
(a)
Identification. An antinuclear antibody immunological test system is a device that consists of the reagents used to measure by immunochemical techniques the autoimmune antibodies in serum, other body fluids, and tissues that react with cellular nuclear constituents (molecules present in the nucleus of a cell, such as ribonucleic acid, deoxyribonucleic acid, or nuclear proteins). The measurements aid in the diagnosis of systemic lupus erythematosus (a multisystem autoimmune disease in which antibodies attack the victim's own tissues), hepatitis (a liver disease), rheumatoid arthritis, Sjögren's syndrome (arthritis with inflammation of the eye, eyelid, and salivary glands), and systemic sclerosis (chronic hardening and shrinking of many body tissues).(b)
Classification. Class II (performance standards).