(275 days)
NOVA Lite® DAPI ANCA (Ethanol) Kit is an indirect immunofluorescence assay for the qualitative detection and semiquantitative determination of anti-neutrophil cytoplasmic antibodies (ANCA) of IgG isotypes in human serum by manual fluorescence microscopy or with NOVA View Automated Fluorescence Microscope. The presence of ANCA, in conjunction with other serological, radiological, histological, and clinical findings aids in the diagnosis of ANCA associated vasculitides. A trained operator must confirm results when generated with the NOVA View device.
NOVA Lite® DAPI ANCA (Formalin) Kit is an indirect immunofluorescence assay for the qualitative detection and semi-quantitative determination of anti-neutrophil cytoplasmic antibodies (ANCA) of IgG istoypes in human serum by manual fluorescence microscopy or with NOVA View Automated Fluorescence Microscope. The presence of ANCA, in conjunction with other serological, radiological, histological, and clinical findings aids in the diagnosis of ANCA associated vasculitides. A trained operator must confirm results when generated with the NOVA View device. ANCA Formalin test is not intended to be used by itself, but in conjunction with ANCA Ethanol test.
The NOVA Lite DAPI ANCA (Ethanol) and ANCA (Formalin) Kits are indirect immunofluorescence assays for the qualitative detection and semi-quantitative determination of anti-neutrophil cytoplasmic antibodies of IgG isotypes in human serum
Kit components:
- . ANCA (Formalin Fixed Human Neutrophils) Slides; 12 wells/slide, with desiccant, or ANCA (Ethanol Fixed Human Neutrophils) Slides; 12 wells/slide, with desiccant
- FITC IgG Conjugate with DAPI, containing 0.09% sodium azide; ready to use. ●
- Positive Controls: cANCA and pANCA; human serum with antibodies to PR3 and MPO antigen, ● containing 0.09% sodium azide; pre-diluted, ready to use.
- Negative Control: IFA System Negative Control, diluted human serum with no ANCA present, containing 0.09% sodium azide; pre-diluted, ready to use.
- PBS II (40x) Concentrate, sufficient for making 2000 mL of 1x PBS II.
- Mounting Medium, containing 0.09% sodium azide
- Coverslips
This document outlines the acceptance criteria and study results for the NOVA Lite Dapi ANCA Ethanol Kit and Formalin Kit. This is a medical device, and the information is presented in the context of an FDA 510(k) premarket notification.
Acceptance Criteria and Reported Device Performance
The device is evaluated based on its precision performance (within laboratory imprecision, between lots reproducibility, between sites/instruments reproducibility, and between operators reproducibility), interference resistance, cross-reactivity with other conditions, and clinical sensitivity and specificity.
Precision Performance:
The acceptance criteria for precision studies consistently revolve around:
- Qualitative agreement: ≥ 90% (for NOVA View, Digital, and Manual readings)
- Grade agreement: ≥ 90% within ± 1 reactivity grade (for Digital and Manual readings)
- Pattern agreement: ≥ 90% (for Digital and Manual readings), or ≥ 80% after excluding positive/negative discrepancies for NOVA View
The studies generally show the device meets these targets. For example:
- Within-laboratory imprecision: "grades were within ± one reactivity grade within one run (within triplicates), and the average grade was no more than one reactivity grade different between runs."
- Between lots reproducibility: All qualitative agreements for Ethanol ANCA (NOVA View, Manual, Digital) were ≥ 97.0%. For Formalin ANCA, agreements ranged from 90.9% to 100%. Grade agreements were 100% within ± 1 grade for both Ethanol and Formalin ANCA for manual and digital readings. Pattern agreements were 100% for manual and digital Ethanol ANCA, and ranged from 90.9% to 100% for manual and digital Formalin ANCA.
- Between sites/instruments reproducibility:
- Ethanol ANCA: Qualitative agreement (Total) ranged from 90.9% to 96.1% for NOVA View, 86.4% to 91.5% for Manual, and 90.1% to 96.1% for Digital. Grade agreement was ≥ 96.0% for both Manual and Digital. Pattern agreement (excluding pos/neg disagreement) was ≥ 90.0% for NOVA View, ≥ 98.0% for Manual, and ≥ 98.0% for Digital.
- Formalin ANCA: Qualitative agreement (Total) ranged from 93.7% to 94.8% for NOVA View, 90.2% to 91.3% for Manual, and 92.7% to 95.5% for Digital. Grade agreement was ≥ 91.0% for both Manual and Digital. Pattern agreement (excluding pos/neg disagreement) was ≥ 93.0% for NOVA View, ≥ 98.0% for Manual, and ≥ 99.0% for Digital.
- Between operators reproducibility: Overall agreement (Positive/Negative) for Ethanol ANCA ranged from 94.4% to 100% for Manual reading and 97.2% to 100% for Digital Image Reading. For Formalin ANCA, it ranged from 94.0% to 100.0% for Manual reading and 91.6% to 99.6% for Digital Image Reading.
Interference:
- Acceptance Criteria: Grades obtained on samples with interfering substances are within ± 1 reactivity grade of those obtained on the control samples, spiked with diluent.
- Reported Performance: "No interference was detected with bilirubin up to 100 mg/dL, hemoglobin up to 200 mg/dL, triglycerides up to 1000 mg/dL, cholesterol up to 224.3 mg/dL, RF IgM up to 56 IU/mL, Human Immunoglobulin up to 35 mg/dL, Rituximab up to 7.6 mg/mL, Methylprednisolone up to 0.85 mg/mL, Cyclophosphamide up to 4.1 mg/mL, Methotrexate up to 0.01 mg/mL and Azathioprine up to 0.03 mg/mL. Reactivity grades of samples containing the interfering substance were within ± one grade of the control samples with both manual and digital reading."
Cross-reactivity:
The document reports cross-reactivity rates for various autoimmune and infectious conditions (e.g., Infectious Disease, Autoimmune thyroid disease, Celiac, Rheumatoid Arthritis). It also specifically examines cross-reactivity with known ANA positive samples. This is presented as information rather than having explicit numerical acceptance criteria in the provided text. The document acknowledges that ANA can interfere and states: "ANA positive samples may react with the nuclei of ethanol-fixed neutrophils, masking or mimicking ANCA. Positive IIF results should be confirmed by antigen specific solid phase assay for anti-MPO and anti-PR3."
Conjugate Comparison (to predicate device):
-
Acceptance Criteria: Qualitative agreement: ≥ 90%, Grade agreement: ≥ 90% within ± 1 reactivity grade, Endpoint dilution is within ±1 dilution step between the two conjugates.
-
Reported Performance: "The qualitative agreement and the grade agreement between the result obtained with the predicate and the new conjugate was 100% on both Ethanol and Formalin ANCA slides." Endpoint titers were within ±1 dilution step.
-
Method Comparison (to predicate device):
- Acceptance Criteria: Qualitative agreement: ≥ 80% (for manual and digital), Grade agreement: ≥ 90% within ± 1 reactivity grade, Pattern agreement: ≥ 80% between manual and digital interpretation.
- Reported Performance: Ethanol ANCA qualitative agreement ranged from 80.3% to 91.8% for manual and digital vs. predicate manual. Grade agreement (within ±2 grades) was 99.6%. Pattern agreement was ≥ 80.1%. Formalin ANCA qualitative agreement ranged from 79.0% to 94.4%. Grade agreement (within ±2 grades) was 100%. Pattern agreement was ≥ 86.9%.
Clinical Sensitivity and Specificity:
These are reported for various ANCA Associated Vasculitides (AAV) subgroups (GPA, MPA, eGPA) and overall AAV, as well as for control populations across multiple sites and interpretation methods (Digital, Manual, NOVA View). No explicit numerical acceptance criteria are given for these performance characteristics in this document, but the results are presented as the device's performance.
1. Table of Acceptance Criteria and Reported Device Performance
| Category | Acceptance Criteria | Reported Device Performance (Summary) |
|---|---|---|
| Precision | ||
| - Within-lab Imprecision | Diff. within run ± 1 reactivity grade; Avg. diff. between runs ± 1 reactivity grade. | Met: "grades were within ± one reactivity grade within one run... and the average grade was no more than one reactivity grade different between runs." (p. 11) |
| - Between-lots | Qualitative agreement ≥ 90%; Grade agreement ≥ 90% (within ± 1 grade); Pattern agreement ≥ 90%. | Met: Qualitative agreements ≥ 90.9% (NOVA View), 100% (Manual/Digital). Grade agreements 100% (within ± 1 grade). Pattern agreements ≥ 71.9% (NOVA View), 100% (Manual), ≥ 90.9% (Digital). (p. 16-20) |
| - Between Sites/Instruments | Qualitative agreement ≥ 85%; Grade agreement ≥ 90% (within ± 1 grade); Pattern agreement ≥ 80% (excl. pos/neg disc.). | Met: Qualitative agreements ≥ 86.4% (Manual), ≥ 90.1% (Digital), ≥ 90.9% (NOVA View). Grade agreements ≥ 91.0%. Pattern agreements ≥ 90.0% (NOVA View), ≥ 98.0% (Manual/Digital ethanol), ≥ 93.0% (NOVA View), 99.0%-100% (Manual/Digital formalin). (p. 24-27) |
| - Between Operators | (Implicitly part of between-sites/instruments, but separate summary provided) | Positive/Negative overall agreement 94.0-100% (Manual), 91.6-100% (Digital) for Ethanol and Formalin ANCA. (p. 30, 33) |
| Interference | Grades obtained on samples with interfering substances are within ± 1 reactivity grade of controls. | Met: "No interference was detected with bilirubin... hemoglobin... triglycerides... cholesterol... RF IgM... Human Immunoglobulin... Rituximab... Methy |
| Conjugate Comparison | Qualitative agreement ≥ 90%; Grade agreement ≥ 90% (within ± 1 grade); Endpoint within ±1 dilution step. | Met: Qualitative and Grade agreement 100%. Endpoint titers within ±1 dilution step. (p. 37) |
| Method Comparison | Qualitative agreement ≥ 80% (Manual/Digital); Grade agreement ≥ 90% (within ± 1 grade); Pattern agreement ≥ 80% (Manual/Digital). | Met: Ethanol ANCA Qualitative 80.3-91.8% (Manual/Digital). Grade agreement 99.6% (within ±2 grades), Pattern 80.1-89.9%. Formalin ANCA Qualitative 79.0-94.4%. Grade agreement 100% (within ±2 grades), Pattern 86.9-92.1%. (p. 38-40) |
| SWT Function | SWT is within ± 2 dilution steps of manual titer AND digital titer. | Met: 97.7% of SWT results were within ± 2 dilution steps of both the manual and digital titer (for in-house validation). 100% for external sites. (p. 47) |
2. Sample sizes used for the test set and data provenance
- Precision (within lab): 16 samples (3 negative, 13 positive - 7 anti-MPO, 6 anti-PR3) tested in triplicates across 10 runs (30 data points per sample). (p. 11) Data provenance is in-house (Inova Diagnostics). This was a prospective study.
- Precision (between lots): 33 clinically and/or analytically characterized samples. (p. 16) Data provenance is in-house (Inova Diagnostics). This was a prospective study.
- Precision (between sites/instruments): 287 clinically characterized samples were tested at three sites (Inova's laboratory and two external US clinical laboratories). Additionally, the two external sites each tested 100 routine clinical samples. The internal study uses 287 samples, with a clinical cohort of n=238 after excluding 49 analytically characterized samples. (p. 21) Data provenance includes US clinical laboratories (prospective, as samples were "tested").
- Precision (between operators): 10 samples (2 negative, 4 P-ANCA, 4 C-ANCA positive) tested at each of three sites, for 5 days in 5 replicates (25 data points per sample). (p. 28) Data provenance includes US clinical laboratories (prospective).
- Interference: 5 specimens (1 negative, 1 low MPO, 1 strong MPO, 1 low PR3, 1 strong PR3). These were spiked with various interferents and tested in triplicates. (p. 34) Data provenance is likely in-house. This was a prospective study.
- Cross-reactivity: 151 clinical patient samples (Infectious Disease, Autoimmune thyroid disease, Celiac, Rheumatoid Arthritis) and 25 analytically characterized ANA positive samples. (p. 35) Data provenance includes a clinical patient population. This was likely a retrospective analysis from collected samples.
- Conjugate Comparison: 36 specimens (analytically characterized serum samples and controls) plus a diluent blank. Endpoint titration on 6 positive samples. (p. 37) Data provenance is likely in-house. This was a prospective study.
- Method Comparison (vs. predicate): 100 samples (50 P-ANCA, 50 C-ANCA) and various disease control groups (Infectious Disease, Systemic Lupus Erythematosus, Progressive Systemic Sclerosis, Rheumatoid arthritis and Chronic Kidney Disease) for a total of 267 samples. (p. 38) Data provenance is in-house (Inova Diagnostics), likely retrospective from collected samples.
- Clinical Performance (Sensitivity/Specificity): 653 clinically or analytically characterized serum samples. A subset of 287 samples was also tested across three sites, with a clinical cohort of 238 after excluding analytically characterized samples. (p. 41) Data provenance covers a combined population, including US clinical samples and characterized samples. Likely a mix of retrospective and prospective.
- Expected Values: 89 samples from apparently healthy subjects. (p. 45) Data provenance is not explicitly stated as US or international but is part of the broader clinical validation. Likely a retrospective analysis.
3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts
The document does not explicitly state the number of experts or their specific qualifications (e.g., "radiologist with 10 years of experience") used to establish the ground truth for the test sets.
For "clinically characterized samples" and "analytically characterized MPO/PR3" samples, it is implied that a reference standard (e.g., diagnosis of ANCA Associated Vasculitis, characterized anti-MPO/PR3 status) was used as ground truth. However, the exact process of how this ground truth was established, who established it, and their qualifications are not detailed.
The manual readings and digital image interpretations are performed by "trained operators."
4. Adjudication method (e.g., 2+1, 3+1, none) for the test set
The document does not describe an explicit adjudication method like "2+1" or "3+1" for discrepancies in the test sets.
- For manual and digital readings in precision and method comparison studies, results from different operators/sites are compared.
- The "NOVA View interpretation" results are expected to be reviewed and confirmed by a "trained operator." (p. 9, 45) This implies a human-in-the-loop confirmation process as a form of adjudication for the automated results.
5. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance
While there are studies involving multiple sites and operators comparing manual reading, digital image reading (human reading of images captured by the automated system), and NOVA View software interpretation, the document does not describe a formal MRMC comparative effectiveness study in the sense of measuring the improvement of human readers with AI assistance vs. without AI assistance.
The studies compare the performance of human readers (manual and digital) and the NOVA View software alone, but not the synergistic effect or comparative effectiveness of AI-assisted human reading against unassisted human reading. The "digital image reading" is human reading of images captured by the NOVA View, which is an output of the system, but the document does not present data on how this assistance (providing the images) improves human readers compared to traditional manual microscopy.
6. If a standalone (i.e., algorithm only without human-in-the loop performance) was done
Yes, a standalone performance (algorithm only) was done. The "NOVA View software interpretation" is explicitly compared throughout the document to "Manual reading" (traditional microscopy) and "Digital reading" (human interpretation of the NOVA View generated digital images).
For example, in the Clinical Sensitivity and Specificity section (p. 42-44), separate results are provided for "NOVA View" (software interpretation), "Digital" (human interpretation of digital images), and "Manual" (traditional microscopy). This clearly indicates a standalone performance evaluation of the NOVA View software. The statement "A trained operator must confirm results when generated with the NOVA View device" (p. 2-3, 45) specifies the intended use model, but the software's performance without this confirmation step is also reported.
7. The type of ground truth used (expert consensus, pathology, outcomes data, etc.)
The types of ground truth used include:
- Analytically characterized samples: These are identified as "anti-MPO/PR3 positive" (p. 41), implying a biochemical or molecular characterization. This forms a strong ground truth for the presence of specific antibodies.
- Clinically characterized serum samples: These are often categorized by "Diagnosis" (e.g., ANCA Associated Vaculitidies (AAV), Infectious Disease, Rheumatoid Arthritis, etc.) (p. 21-23, 41). The establishment of these diagnoses would likely be based on a combination of clinical findings, laboratory tests, histology, and possibly expert consensus from treating physicians. The document does not specify the exact diagnostic criteria or who established these clinical diagnoses.
- Apparently healthy subjects: Used as a negative control group. (p. 41, 45)
8. The sample size for the training set
The document does not explicitly state the sample size for a "training set" for the NOVA View AI algorithm itself. It mentions that the "SWT function was established on 10 anti-MPO (P-ANCA) and 10 anti-PR3 (C-ANCA) positive samples" to establish LIU curves, which could be considered a form of training or calibration data for that specific function. However, a general training set size for the core ANCA detection and pattern recognition algorithm is not provided.
9. How the ground truth for the training set was established
As the document does not explicitly detail a separate "training set" and its ground truth establishment, the information is limited. For the 20 samples (10 anti-MPO, 10 anti-PR3) used to establish the SWT function's LIU curves, the ground truth was "manually titrated" and "results were interpreted by NOVA View and by manual reading." (p. 47) This implies that the manual titration and reading served as the reference for establishing the LIU curves.
§ 866.5660 Multiple autoantibodies immunological test system.
(a)
Identification. A multiple autoantibodies immunological test system is a device that consists of the reagents used to measure by immunochemical techniques the autoantibodies (antibodies produced against the body's own tissues) in serum and other body fluids. Measurement of multiple autoantibodies aids in the diagnosis of autoimmune disorders (disease produced when the body's own tissues are injured by autoantibodies).(b)
Classification. Class II (performance standards).