(197 days)
AESKUSLIDES® nDNA (Crithidia luciliae) is an indirect immunofluorescence assay utilizing Crithidia luciliae coated slides as a substrate for the qualitative and/or semi-quantitative determination of antibodies to native double stranded DNA (dsDNA) in human serum. This in vitro diagnostic assay is used as an aid for the diagnosis of Systemic Lupus Erythematosus (SLE) in conjunction with other clinical and laboratory findings. The assay can be processed manually and analyzed at the microscope or processed and analyzed with HELIOS® AUTOMATED IFA SYSTEM. All suggested results obtained with the HELIOS® AUTOMATED IFA SYSTEM must be confirmed by trained personnel.
AESKUSLIDES® nDNA (Crithidia luciliae) is an indirect immunofluorescence assay utilizing Crithidia luciliae coated slides as a substrate for the qualitative and/or semiquantitative determination of antibodies to native double stranded DNA (dsDNA) in human serum.
Each kit contains (Quantity depends on product variant):
- -Slides, each containing 10 wells coated with Crithidia Luciliae cells
- 4.0 ml vial containing Fluorescein (FITC) labelled Anti-human Antibody lgG conjugate in a solution of BSA, ready for use
- -0.5 ml vial of positive control containing human serum (diluted), ready for Use
- 0.5 ml vial of negative control containing diluted human serum, ready for use -
- -8.0 ml vial of mounting medium containing a solution of glycerol and PBS, ready for use
- 70 ml bottle of sample buffer, containing BSA, PBS and ready for use -
- -100 ml bottle of wash buffer, concentrated buffer 1:10 in distilled water, containing BSA, PBS.
Here's a breakdown of the acceptance criteria and study information for the AESKUSLIDES® nDNA (Crithidia luciliae) device, based on the provided document:
Acceptance Criteria and Device Performance Study for AESKUSLIDES® nDNA (Crithidia luciliae)
1. Table of Acceptance Criteria and Reported Device Performance
The document presents several studies with specific acceptance criteria. Below is a summary for the Between-Lab Precision Study and the Lot-to-Lot Precision Study, as these provide clear, quantitative acceptance criteria and corresponding results. Other studies, like Serum Stability and Carryover, also met their respective criteria (e.g., 100% agreement, no significant deviations).
Between-Lab Precision Study (Excluding Borderline Samples - Method B: Reader Confirmation)
| Type of Agreement | Acceptance Criteria | Reported Device Performance (All Sites, excluding borderline) |
|---|---|---|
| Positive Agreement | > 90% | 97.9% (96.9 - 98.5)% CI |
| Negative Agreement | > 90% | 99.4% (98 - 99.8)% CI |
| Overall Agreement | > 90% | 98.2% (97.4 - 98.8)% CI |
Between-Lab Precision Study (Excluding Borderline Samples - Method C: Manual)
| Type of Agreement | Acceptance Criteria | Reported Device Performance (All Sites, excluding borderline) |
|---|---|---|
| Positive Agreement | > 90% | 99.5% (99 - 99.8)% CI |
| Negative Agreement | > 90% | 99.7% (98.4 - 100)% CI |
| Overall Agreement | > 90% | 99.6% (99.1 - 99.8)% CI |
| Fluorescence Intensity Agreement | > 90% | 97.5% (96.6 - 98.1)% CI |
Lot-to-Lot Precision Study (Combined Readers)
| Type of Agreement | Acceptance Criteria | Reported Device Performance (Combined Readers) |
|---|---|---|
| Positive Agreement | > 90% | 100% (99.3 - 100)% CI |
| Negative Agreement | > 90% | 100% (96.9 - 100)% CI |
| Overall Agreement | > 90% | 100% (99.4 - 100)% CI |
| Fluorescence Intensity Agreement | > 90% | 100% (99.4 - 100)% CI |
2. Sample Size Used for the Test Set and Data Provenance
The primary clinical evaluation and method comparison studies used 776 clinical samples.
- Test Set Size: 776 samples.
- Data Provenance:
- 746 samples were obtained from 10 US BioBanks (BioChain, BioReclamationIVT, Bioserve, ConversantBio, Cureline, DiscoveryLifeSciences, iSpecimen, Precision for Medicine, ProMedDx, and Vitrologic).
- 30 serum samples were from a German University Hospital (used to complement rare diagnoses like Vasculitis).
- Retrospective/Prospective: The samples were collected from BioBanks, indicating they are retrospective samples with pre-existing diagnoses.
- Sample Characteristics: The samples were selected to reflect various diagnoses relevant to the study (e.g., 297 SLE, 479 other diseases) and different ethnic groups in the US population (White, Black/Black African, Asian, Hispanic). Serum stability over multiple freeze-thaw cycles and long-term storage was also evaluated and deemed acceptable.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications
The term "ground truth" for the diagnosis of the clinical samples (used in the method comparison studies) was established based on diagnostic standards used in the U.S. and Germany, such as ACR criteria. A written statement from different serum suppliers confirmed this.
For evaluation of the device performance within the studies (e.g., precision studies, clinical study), the ground truth for individual results was typically established by:
- Two independent readers.
- Qualifications of experts: The document consistently refers to "trained personnel" and "trained readers" to perform manual microscopy and confirm automated results. Specific certifications or years of experience (e.g., "Radiologist with 10 years of experience") are not explicitly provided in this document.
4. Adjudication Method for the Test Set
For studies involving human readers:
- Results were analyzed by two independent readers.
- The document implies a consensus-based approach or separate reporting of each reader's results for evaluation, but a formal adjudication method (like "2+1" or "3+1") where a third reader resolves discrepancies is not explicitly stated. For instance, in the Between-Lab and Within-Lab Precision studies, results for two readers were calculated separately and then combined. In cases of discrepancy (e.g., sample S1 in Within-Lab precision for Method B, where Reader 1 found 30 Negatives and Reader 2 found 6 Negatives and 24 Positives), the combined percentage reflects this disagreement rather than a formal adjudication.
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
The document does not explicitly describe a formal MRMC comparative effectiveness study in the sense of comparing human reader performance with and without AI assistance to quantify an "effect size" of improvement.
Instead, it evaluates the agreement between:
- Method C (Manual): Manual processing and manual reading by human readers.
- Method B (Reader Confirmation): Automated processing and automated imaging, followed by manual reading of digital images by human readers.
- Method A (HELIOS): Automated processing, automated imaging, and automated software interpretation (with required human confirmation).
The study compared the performance and agreement between these methods. It highlights that Method A (software-only interpretation) had lower agreement and sensitivity compared to human interpretation (Method B and C), underscoring the need for human confirmation. However, it does not quantify how much human readers improve when using AI as an assistance tool, but rather assesses the standalone performance of the AI component and the human element with and without automated processing.
6. Standalone (Algorithm Only) Performance
Yes, standalone performance was done:
- Method A (HELIOS): This method represents the algorithm's standalone performance, where the "HELIOS DNA Pattern Plus Software" performs positive/negative classification.
- Results: For Method A (HELIOS suggestion, excluding borderline samples) in the Between-Lab Precision Study:
- Positive Agreement: 72.7% (69.1 - 76)% CI (across all sites)
- Negative Agreement: 90% (84.7 - 93.6)% CI (across all sites)
- Overall Agreement: 76.5% (73.5 - 79.3)% CI (across all sites)
The document notes that acceptance criteria for Method A were met for negative and overall agreement, but positive agreement was sometimes below 70% for individual site comparisons, explaining that this was due to out-of-focus images that the software couldn't interpret as positive. This reinforces the requirement for human confirmation.
7. Type of Ground Truth Used (Clinical Samples)
For the 776 clinical samples used in the method comparison studies, the "ground truth" for patient diagnosis (e.g., SLE, other rheumatic diseases, autoimmune liver diseases, infections, leukemia) was established based on diagnostic standards used in the U.S. and Germany (e.g., ACR criteria). This implies a clinical diagnosis based on a combination of clinical findings and existing laboratory tests, rather than a single definitive gold standard like pathology or outcome data.
For validation within the performance studies (e.g., precision, stability), the "expected result" (positive/negative, fluorescence intensity) of specific control or serum samples was pre-defined or derived from manual readings by trained personnel, which can be considered an expert consensus type of ground truth for analytical performance.
8. Sample Size for the Training Set
The document does not provide information regarding the sample size used for training the HELIOS DNA Pattern Plus Software (Method A). It mainly focuses on the performance evaluation of the device in various settings.
9. How the Ground Truth for the Training Set Was Established
Since information on the training set sample size is absent, how its ground truth was established is also not specified in this document.
§ 866.5100 Antinuclear antibody immunological test system.
(a)
Identification. An antinuclear antibody immunological test system is a device that consists of the reagents used to measure by immunochemical techniques the autoimmune antibodies in serum, other body fluids, and tissues that react with cellular nuclear constituents (molecules present in the nucleus of a cell, such as ribonucleic acid, deoxyribonucleic acid, or nuclear proteins). The measurements aid in the diagnosis of systemic lupus erythematosus (a multisystem autoimmune disease in which antibodies attack the victim's own tissues), hepatitis (a liver disease), rheumatoid arthritis, Sjögren's syndrome (arthritis with inflammation of the eye, eyelid, and salivary glands), and systemic sclerosis (chronic hardening and shrinking of many body tissues).(b)
Classification. Class II (performance standards).