(266 days)
The ImmuGlo HEp-2 Elite IFA is an indirect immunofluorescence antibody test for the qualitative or semi-quantitative detection of anti-nuclear antibodies (ANA) of the IgG isotype in human serum utilizing standard HEp-2 cells and engineered HEp-2 cells as a substrate. The ImmuGlo HEp-2 Elite IFA is intended for use as an aid in the diagnosis of systemic rheumatic diseases in conjunction with other clinical and laboratory findings.
The ImmuGlo™ HEp-2 Elite IFA is an indirect immunofluorescence antibody test for the qualitative detection of anti-nuclear antibodies (ANA) of the IgG isotype in human serum utilizing standard HEp-2 cells as a substrate. The HEp-2 Elite sides provided with this kit contain a 1:9 mixture of standard HEp-2 cells and engineered HEp-2 cells with the PSP1 gene knocked out. The engineered HEp-2 cells are able to detect all ANA specificities with the exception of the DF570/AC-2 pattern resulting from autoantibodies associated with PSP2. In DF570 positive reactions standard HEp-2 cells provide a positive reaction while engineered HEp-2 cells do not. The HEp-2 Elite substrate thereby provides additionality to aid in discriminating homogeneous, speckled, and dense fine speckled (DFS70/AC-2) patterns during the screening phase.
The provided text describes the ImmuGlo HEp-2 Elite IFA device, which is an indirect immunofluorescence antibody test for the detection of anti-nuclear antibodies (ANA). It aims to aid in the diagnosis of systemic rheumatic diseases.
Here's an analysis of the acceptance criteria and the study data:
1. Table of Acceptance Criteria and Reported Device Performance
The document does not explicitly state formal acceptance criteria in a dedicated section for all tests. However, performance metrics are provided in the "Non-clinical Tests" and "Precision" sections. Based on the "Repeatability" and "Reproducibility" sections, the acceptance criteria for precision studies are implicitly defined.
| Performance Metric | Acceptance Criteria (Stated or Implied) | Reported Device Performance |
|---|---|---|
| Qualitative Agreement (Pos/Neg) | Not explicitly stated as acceptance criteria, but implied performance goal for substantial equivalence. | Positive % Agreement: 99.7% (95% CI 97.8% - 100.0%) |
| Negative % Agreement: 98.3% (95% CI 95.8% - 99.4%) | ||
| Overall % Agreement: 99.3% (95% CI 97.8% - 99.5%) (vs. Predicate device) | ||
| Pattern Agreement Study (Overall) | Not explicitly stated as acceptance criteria, but implied performance goal compared to the predicate device. | Consensus (2/3 Readers) PPA: 88.9% (83.1-92.9%) |
| Consensus (2/3 Readers) NPA: 76.2% (63.5-85.6%) | ||
| Pattern Agreement Study (DFS70) | Not explicitly stated. | Consensus (2/3 Readers) PPA: 100.0% (84.0-100.0%) |
| Consensus (2/3 Readers) NPA: 93.1% (88.6-95.9%) | ||
| Repeatability | 90% qualitative agreement (positive), fluorescence intensity agreement (± 1 fluorescence value within one run), and pattern agreement. | All samples met the 90% acceptance criteria for qualitative agreement (positive), fluorescence intensity agreement (± 1 fluorescence value within one run) and pattern agreement. (Total 60 replicates per sample, two operators, 120 total reads for 8 negative and 17 positive samples) |
| Reproducibility | AQL: ± 1 fluorescence value, ± 1 titer, and a consensus pattern agreement. Acceptance criteria: 95% or greater agreement (titer and fluorescence within ± 1 value, with consensus pattern). | Overall agreement was 99.75%. Two samples (6 and 8 for Homogeneous pattern and Titer respectively) did not meet AQL due to subjective reader error, but other patterns showed 93-100% Pattern Agreement, 93-100% Titer Agreement, and 100% Fluorescence Intensity Agreement. (For 11 samples, tested in triplicate in 10 runs at 3 sites, total 90 replicates per sample). |
| ANA Associated Disease Sensitivity | Not explicitly stated as acceptance criteria, but implied performance goal for clinical utility. | 78.1% (95% C.I. 72.5-82.9%) |
| ANA Associated Disease Specificity | Not explicitly stated as acceptance criteria, but implied performance goal for clinical utility. | 75.1% (95% C.I. 71.0-78.7%) |
2. Sample Size Used for the Test Set and Data Provenance
-
Qualitative Agreement (Pos/Neg) Study (vs. Predicate):
- Sample Size: 591 samples (298 positive, 293 negative by predicate).
- Data Provenance: Not explicitly stated, but the "Method Comparison" implies these are clinical samples tested against the predicate device. No details on country of origin or retrospective/prospective collection are provided.
-
Pattern Agreement Study:
- Sample Size: 243 samples with known ANA patterns.
- Data Provenance: Not explicitly stated. The samples were "randomized." No details on country of origin or retrospective/prospective collection are provided.
-
Repeatability Study:
- Sample Size: 8 negative and 17 positive samples. Each sample was tested in 6 replicates in 10 runs (over 5 days), resulting in 60 replicates for each sample. Read by 2 operators, total N = 120 per sample series.
- Data Provenance: Not specified.
-
Reproducibility Study:
- Sample Size: 10 positive samples with various patterns and intensities. Each sample was tested in triplicate in 10 runs (2 runs/day for 5 days) at 3 different sites, resulting in 90 replicates per sample.
- Data Provenance: Not specified, other than "three different sites."
-
Clinical Study (Sensitivity/Specificity):
- Sample Size: 256 ANA-associated disease samples and 497 autoimmune and infectious disease controls. (Total 753 samples, excluding normal human sera and DFS70 suspected samples for these specific calculations).
- Data Provenance: Not explicitly stated regarding country of origin or retrospective/prospective collection. These are referred to as "sets of clinical samples."
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts
- Qualitative Agreement Study: Ground truth was established by comparison to a "Predicate" device. No human experts are explicitly mentioned for ground truth.
- Pattern Agreement Study: Three independent evaluators provided reads. The ground truth for the comparison was established by "Consensus (2/3 Readers)." The qualifications of these evaluators are not specified (e.g., if they are expert medical professionals like radiologists or pathologists, or their years of experience).
- Precision (Repeatability and Reproducibility) Studies: The repeatability study mentions "2 operators." The reproducibility study mentions testing at "three different sites" and implies readers/operators at those sites. However, the qualifications of these operators/readers are not specified.
- Clinical Study (Sensitivity/Specificity): The classification of samples into "ANA-associated autoimmune disease" or "control" implies a clinical diagnosis was used as ground truth, likely established by referring clinicians. However, the exact method of ground truth establishment (e.g., based on clinical criteria, pathology, or expert consensus) and the qualifications of those who established it are not detailed.
4. Adjudication Method for the Test Set
- Qualitative Agreement (Pos/Neg) Study: The comparison is made against a "Predicate" device. No explicit human adjudication method for the ground truth of the samples themselves is stated; the predicate device's results are used as the reference.
- Pattern Agreement Study: The adjudication method for comparing the Elite device against the predicate was based on "Consensus (2/3 Readers)." This means that for a result to be considered consensus, at least two out of the three independent evaluators had to agree.
- Precision (Repeatability and Reproducibility) Studies: For reproducibility, "consensus pattern agreement" was part of the AQL. This indicates an adjudication process was used, likely involving agreement among multiple readers/operators, but the specific number or method isn't detailed beyond the term "consensus."
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done
- Yes, a form of MRMC study was done for pattern agreement. The "Pattern Agreement Study" involved 243 samples and 3 independent evaluators (readers) who were blinded to each other's results and the samples. The data was analyzed by comparing the Elite device vs. the Predicate device based on reader consensus (at least 2 of 3 readers).
- Effect Size of Human Readers Improvement with AI vs. without AI assistance: This information is not applicable as the device (ImmuGlo HEp-2 Elite IFA) is a diagnostic kit read by human operators (microscopists/technologists) and is not described as an AI-powered image analysis tool. The study assesses the agreement between patterns identified using the new device and the predicate by human readers, not the effect of AI assistance on human reader performance.
6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) Was Done
- No, the device described is an "indirect immunofluorescence antibody test kit" (IFA). This is a laboratory diagnostic method that requires human-in-the-loop performance for slide preparation, reading under a fluorescence microscope, and interpretation of patterns and titers. There is no mention of an algorithm-only or AI-driven standalone component in the provided text.
7. The Type of Ground Truth Used
- Qualitative Agreement Study: The ground truth for this comparison was the results obtained from the predicate device.
- Pattern Agreement Study: The ground truth for comparison in the pattern study was established by expert consensus among 2 out of 3 independent evaluators.
- Clinical Study (Sensitivity/Specificity): The ground truth was based on a clinical diagnosis of "ANA-associated autoimmune disease" or "control" conditions. This typically reflects real-world clinical outcomes and physician diagnoses, potentially supported by other laboratory and clinical data. There is no mention of pathology or other specific "gold standard" laboratory tests cited as the primary ground truth.
8. The Sample Size for the Training Set
- The document describes performance studies for a diagnostic kit (ImmuGlo HEp-2 Elite IFA). This typically does not involve "training sets" in the context of machine learning or AI models, but rather clinical samples are used for method comparison, precision, and clinical performance evaluation.
- Therefore, a "training set" for an algorithm is not applicable to this device as described. The samples mentioned in the clinical study (753 samples) are test sets for evaluating the performance of the device itself.
9. How the Ground Truth for the Training Set Was Established
- As a "training set" in the AI or machine learning sense is not applicable to this device, the method for establishing its ground truth is not relevant here.
§ 866.5100 Antinuclear antibody immunological test system.
(a)
Identification. An antinuclear antibody immunological test system is a device that consists of the reagents used to measure by immunochemical techniques the autoimmune antibodies in serum, other body fluids, and tissues that react with cellular nuclear constituents (molecules present in the nucleus of a cell, such as ribonucleic acid, deoxyribonucleic acid, or nuclear proteins). The measurements aid in the diagnosis of systemic lupus erythematosus (a multisystem autoimmune disease in which antibodies attack the victim's own tissues), hepatitis (a liver disease), rheumatoid arthritis, Sjögren's syndrome (arthritis with inflammation of the eye, eyelid, and salivary glands), and systemic sclerosis (chronic hardening and shrinking of many body tissues).(b)
Classification. Class II (performance standards).