(115 days)
NOVA View® Automated Fluorescence Microscope is an automated system consisting of a fluorescence microscope and software that acquires, stores and displays digital images of stained indirect immunofluorescent slides. It is intended as an aid in the detection and classification of certain antibodies by indirect immunofluorescence technology. The device can only be used with cleared or approved in vitro diagnostic assays that are indicated for use with the device. A trained operator must confirm results generated with the device.
NOVA View® is an automated fluorescence microscope. The instrument does not process samples. The instrument acquires digital images of representative areas of indirect immunofluorescent slides.
Hardware components:
- PC and monitor
- Keyboard and mouse ●
- Microscope
- Microscope control unit
- Slide stage
- LED illumination units
- Handheld LED display unit ●
- Camera ●
- Two fans ●
- Printer (optional) ●
- UPS (optional) or surge protector
- Handheld barcode scanner (optional) ●
1. Acceptance Criteria and Reported Device Performance
The device under evaluation is the NOVA View® Automated Fluorescence Microscope and its performance for the NOVA Lite® DAPI ANA Kit, as presented in the accuracy and reproducibility studies. The acceptance criteria are implicitly derived from the comparisons to manual reading (the reference standard) and digital reading by a human operator, with targets often expressed as agreement percentages or consistent classification and pattern recognition. The accuracy study assesses sensitivity and specificity, while the reproducibility study examines agreement within and between sites and operators.
Here's a summary of the reported device performance against these implicit acceptance criteria, focusing on key metrics from the provided text:
| Acceptance Criteria Category | Specific Metric | Acceptance Criteria (Implicit from context) | Reported Device Performance |
|---|---|---|---|
| Accuracy (Detection) | Agreement between NOVA View® and Manual reading for Positive/Negative classification | High agreement (e.g., >80-90%) for positive and negative classifications, indicating that the NOVA View® system's automated calls align well with human expert interpretation. | Site #1: Positive Agreement: 88.3% (82.5-92.7), Negative Agreement: 90.4% (86.4-93.5), Total Agreement: 89.6% (86.5-92.3). Site #2: Positive Agreement: 80.5% (74.2-85.9), Negative Agreement: 96.3% (93.4-98.2), Total Agreement: 89.8% (86.7-92.4). Site #3: Positive Agreement: 86.1% (80.7-90.5), Negative Agreement: 87.8% (83.1-91.6), Total Agreement: 87.0% (83.6-90.0). |
| Sensitivity for specific disease conditions (e.g., SLE) | The device's performance (NOVA View® and Digital Read) should be comparable to or ideally better than Manual Read. | Site #1: Manual: 72.0% (SLE), 62.9% (CTD+AIL); Digital: 80.0% (SLE), 69.9% (CTD+AIL); NOVA View®: 80.0% (SLE), 69.4% (CTD+AIL). Site #2: Manual: 70.7% (SLE), 65.6% (CTD+AIL); Digital: 73.3% (SLE), 62.9% (CTD+AIL); NOVA View®: 72.0% (SLE), 62.9% (CTD+AIL). Site #3: Manual: 82.7% (SLE), 71.0% (CTD+AIL); Digital: 81.3% (SLE), 69.4% (CTD+AIL); NOVA View®: 82.7% (SLE), 72.0% (CTD+AIL). | |
| Specificity (excluding healthy subjects) | The device's performance (NOVA View® and Digital Read) should be comparable to or ideally better than Manual Read. | Site #1: Manual: 74.1%; Digital: 72.4%; NOVA View®: 75.3%. Site #2: Manual: 67.2%; Digital: 75.3%; NOVA View®: 77.0%. Site #3: Manual: 67.2%; Digital: 71.3%; NOVA View®: 69.0%. | |
| Accuracy (Pattern Recognition) | Agreement between NOVA View® and Manual for Pattern Identification | High agreement (e.g., >70% for definitive patterns) for pattern recognition, indicating that the automated system can accurately classify patterns as interpreted by human experts. | Accuracy Study: Site #1: 76.0%; Site #2: 86.3%; Site #3: 72.7%. Reproducibility Study: Site #1: 78.9%; Site #2: 83.3%; Site #3: 80.4%. |
| Precision/Reproducibility | Repeatability (internal consistency) - Positive/Negative Classification | High consistency (e.g., >95%) for samples not near the cut-off. | For samples away from the cut-off, NOVA View® output showed 100% positive or negative classification. For samples near the cut-off, variability was observed. |
| Repeatability (internal consistency) - Pattern Consistency | 100% consistency for pattern determination in positive samples. | Pattern determination was consistent for 100% of replicates for positive samples (digital image reading and manual reading). NOVA View® pattern classification was correct for >80% of cases (excluding unrecognized). | |
| Within-Site Reproducibility (Operator and Method Agreement) | High total agreement (e.g., >90-95%) between operators and different reading methods within a site. | Site #1: NOVA View® vs Manual: 99.2%; Digital vs Manual: 99.2%; Digital vs NOVA View®: 100.0%. Site #2: NOVA View® vs Manual: 96.7%; Digital vs Manual: 95.8%; Digital vs NOVA View®: 95.8%. Site #3: NOVA View® vs Manual: 96.7%; Digital vs Manual: 96.7%; Digital vs NOVA View®: 98.3%. | |
| Between-Site Reproducibility (Method Agreement across sites) | High overall agreement (e.g., >90-95%) across different sites for all reading methods. | Manual: Site #1 vs #2: 90.7%; Site #1 vs #3: 85.7%; Site #2 vs #3: 87.3%. Digital: Site #1 vs #2: 92.0%; Site #1 vs #3: 93.1%; Site #2 vs #3: 92.0%. NOVA View®: Site #1 vs #2: 92.7%; Site #1 vs #3: 89.6%; Site #2 vs #3: 87.9%. | |
| Single Well Titer (SWT) | SWT accuracy compared to Manual and Digital endpoints | High agreement, with estimated titer within ±1 or ±2 dilution steps. | SWT results were within ±2 dilution steps from manual endpoint for 96% (48/50) of samples and from digital endpoint for 98% (49/50) of samples in the initial validation. In the clinical study, it was within ±2 dilution steps for all 20 samples at all three locations. |
2. Sample Sizes and Data Provenance
- Test Set (Accuracy Study): 463 clinically characterized samples.
- Data Provenance: The study was conducted retrospectively or prospectively, and at three different locations: one internal (Site 1) and two external (Sites 2 and 3). The countries of origin of the data are not explicitly stated, but the mention of "U.S. sites" in special controls (2)(ii)(B) suggests primary relevance to the US context.
- Test Set (Reproducibility Study): 120 samples per location (total of 360 unique or overlapping samples if shared across sites as described).
- Data Provenance: Conducted at Inova Diagnostics (internal; Site#1) and two external sites (Sites #2 and #3). The same cohort of samples was processed at each location.
- Test Set (Repeatability Study 1): 13 samples (3 negative, 10 positive), tested in triplicate across 10 runs (30 data points per sample).
- Test Set (Repeatability Study 2): 22 samples (20 borderline/cut-off, 2 high intensity), tested in triplicate across 10 runs (30 data points per sample).
- Test Set (Repeatability Study 3): 8 samples, tested in triplicate or duplicate across 5 runs (10-15 data points per sample).
- SWT Validation Study 1: 50 ANA positive samples.
- SWT Validation Study 2: 20 ANA positive samples at each of the three locations (total 60 data points, if unique).
3. Number of Experts and Qualifications for Ground Truth
- Accuracy Study, Reproducibility Study, and SWT Validation: For "Manual" reading (the reference standard), "trained human operators" performed the interpretations. For "Digital" reading, "trained human operators" interpreted the software-generated images, blinded to automated results.
- Qualifications: The document consistently refers to "trained operators" and "trained human operators." Specific professional qualifications (e.g., "radiologist with 10 years of experience") are not explicitly provided, but the context implies experienced clinical laboratory personnel proficient in indirect immunofluorescence microscopy.
4. Adjudication Method for the Test Set
- Accuracy Study: Not explicitly stated as a formal adjudication. The comparison was described as a "three-way method comparison of NOVA View® automated software-driven result (NOVA View®) compared to the Digital image reading... by a trained operator who was blinded to the automated result (Digital) and compared to the reference standard of conventional IIF manual microcopy (Manual)." The "Manual" reading served as the key reference. Clinical truth for sensitivity/specificity was determined independently from the three reading methods.
- Reproducibility Study: No formal adjudication process is detailed between the different reading methods or operators. Agreement simply refers to concordance between the specified interpretations. For between-operator agreement, multiple operators at each site interpreted the same digital images.
- SWT Validation Studies: The "manual endpoint" determined by an operator using a traditional microscope served as a primary reference for comparison with the SWT application's endpoint.
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
- Yes, a form of MRMC was conducted for reproducibility. The reproducibility study involved multiple operators (referred to as "Operator #1" and "Operator #2" at each site) interpreting the same digital images and comparing their results.
- Effect Size of Human Readers with AI vs. without AI: The document does not provide a direct "effect size" in terms of how much human readers improve with AI assistance versus without. Instead, it measures the agreement between manual reading (without AI assistance, as the traditional method) and digital image reading (human-in-the-loop with AI-provided images).
- For example, in the Accuracy Study, comparing "Digital vs. Manual" total agreement was: Site #1 (91.4%), Site #2 (92.2%), Site #3 (92.2%). This indicates a high level of agreement between human interpretation of digital images (AI-assisted display) and manual microscopy, suggesting that the digital images are comparable to traditional microscopy.
- Furthermore, "Between Operator Agreement" for digital image reading showed very high agreement (e.g., 99.2% for Site #1 Op #1 vs. Site #1 Op #2), indicating consistency among human readers using the digital system.
6. Standalone (Algorithm Only) Performance
- Yes, a standalone performance was done for various aspects.
- The "NOVA View®" results explicitly refer to "results obtained with the NOVA View® Automated Fluorescence Microscope, such as Light Intensity Units (LIU), positive/negative classification and pattern information without operator interpretation." This represents the algorithm's standalone output before human review.
- The accuracy and reproducibility tables compare "NOVA View®" (standalone algorithm) directly against "Manual" reading (reference standard) and "Digital" reading (human-in-the-loop).
7. Type of Ground Truth Used
- Expert Consensus / Clinical Diagnosis / Reference Standard:
- For the Accuracy Study, the "Manual" reading by trained operators using a traditional fluorescence microscope served as the primary reference standard for comparing the digital and automated methods. Additionally, clinical sensitivity and specificity were determined by comparing the results from all three methods (Manual, Digital, NOVA View®) to a "clinical truth" derived from a "cohort of clinically characterized samples." This clinical truth would likely be established through a combination of clinical criteria and other diagnostic tests, representing a form of expert consensus or outcomes data.
- For the Reproducibility/Repeatability Studies, the "Manual" reading served as the reference standard for evaluating consistency.
- For the SWT Validation, the "manual endpoint" titer determined by trained operators using traditional microscopy was the reference.
8. Sample Size for the Training Set
- The document does not explicitly state the sample size used for training the NOVA View® algorithm. The studies described are performance evaluations (test sets) rather than detailing the algorithm's development or training data.
- For the Single Well Titer (SWT) function, it states that "The NOVA View® SWT function was established [using] 38 ANA positive samples," which could be considered a form of "calibration" or establishment data for that specific algorithm feature, rather than a general training set for the primary classification.
9. How the Ground Truth for the Training Set Was Established
- Since the training set size is not provided, the method for establishing its ground truth is also not detailed in this document.
- For the SWT function's establishment data (38 ANA positive samples), the text implies that the "software application automatically performs the calculations based on the predetermined dilution curve, the LIU produced by the sample, and the pattern of the ANA." This suggests these 38 samples were used to define or fine-tune this "predetermined dilution curve" and pattern-based calculations, likely referencing expert-determined ANA patterns and traditional titration results for those samples.
{0}------------------------------------------------
EVALUATION OF AUTOMATIC CLASS III DESIGNATION FOR NOVA View® Automated Fluorescence Microscope DECISION SUMMARY
A. DEN Number:
B. Purpose for Submission:
De novo request for evaluation of automatic class III designation for the Inova NOVA View® Automated Fluorescence Microscope
C. Measurands:
Not applicable. Measurands are dependent on the assay indicated for use with the device.
D. Type of Test or Tests Performed:
Qualitative and/or semi-quantitative indirect immunofluorescence (IIF) assays
E. Applicant and Instrument Name:
Inova Diagnostics, Inc.
NOVA View® Automated Fluorescence Microscope
F. Proprietary and Established Names:
NOVA View® Automated Fluorescence Microscope
G. Regulatory Information:
-
- Regulation section:
21 CFR 866.4750
- Regulation section:
-
- Classification:
Class II (special controls)
- Classification:
-
3 Product code:
PIV, Automated indirect immunofluorescence microscope and software-assisted system for clinical use
{1}------------------------------------------------
4. Panel:
Immunology (82)
H. Intended Use:
-
- Intended use(s):
NOVA View® Automated Fluorescence Microscope is an automated system consisting of a fluorescence microscope and software that acquires, stores and displays digital images of stained indirect immunofluorescent slides. It is intended as an aid in the detection and classification of certain antibodies by indirect immunofluorescence technology. The device can only be used with cleared or approved in vitro diagnostic assays that are indicated for use with the device. A trained operator must confirm results generated with the device.
- Intended use(s):
-
- Indication(s) for use:
Same as intended use.
- Indication(s) for use:
-
- Special Conditions for Use Statement(s):
-
- For prescription use only.
-
- This device is only for use with reagents that are indicated for use with the device.
-
- The device is for use by a trained operator in a clinical laboratory setting.
-
- All software-aided results must be confirmed by the trained operator.
I. System Descriptions:
-
- Device Description:
NOVA View® is an automated fluorescence microscope. The instrument does not process samples. The instrument acquires digital images of representative areas of indirect immunofluorescent slides.
- Device Description:
Hardware components:
- PC and monitor
- Keyboard and mouse ●
- Microscope
- Microscope control unit
- Slide stage
- LED illumination units
- Handheld LED display unit ●
- Camera ●
- Two fans ●
{2}------------------------------------------------
- Printer (optional) ●
- UPS (optional) or surge protector
- Handheld barcode scanner (optional) ●
2. Principles of Operation:
NOVA View® Automated Fluorescence Microscope (NOVA View®) is an automated system consisting of a fluorescence microscope and software that acquires, analyzes, stores and displays digital images of stained indirect immunofluorescent (IIF) slides. The NOVA View® device consists of an inverted fluorescence microscope with LED light source, motorized microscope stage, a CCD camera, computer, keyboard and mouse, monitor and assay specific software. The device's capabilities include acquiring and digitizing high resolution images from IIF microscope slides, storing and managing the resulting digital images, retrieving and displaying the digital images (including enlarging and overlaying), and providing facilities for annotating digital images (i.e., entering comments). Image analysis capabilities include the ability to detect and quantify fluorescent light intensity of certain cellular structures. The digital image viewing capabilities of the system support reading digital images on a computer monitor, enabling the trained operator to make clinically relevant decisions analogous to those they would make using a conventional microscope.
All images are taken through a DAPI and a FITC filter. FITC (fluorescein isothiocyanate) is a green fluorescent dye that is chemically bound to the anti-human IgG conjugate. DAPI (4', 6-diamidino-2-phenylindole) is a blue fluorescent dye added to the conjugate solution. DAPI strongly binds to DNA; therefore, it stains cell nuclei, regardless of antibody and conjugate binding. NOVA View® reads the slides through the two filters consecutively, enabling the detection of both DAPI and FITC fluorescence. DAPI fluorescence is used by the instrument to identify nuclei within the cell to take nuclear light intensity measurements. At the same time digital images are taken. NOVA View® measures the FITC light intensity of the cells that are included in the region. NOVA View® reports the measured average nuclear fluorescence intensity in units of Light Intensity Units (LIU).
The Single Well Titer (SWT) is a software application that estimates the endpoint titer (e.g., the highest dilution that would give positive result) for wells with a positive reaction, based on the LIU and pattern. The Single Well Titer function is not automatic; the Single Well Titer button has to be selected by the trained operator to display the calculated endpoint titer. SWT can only be generated on un-confirmed results. The result will appear in the Endpoint field in the Results tab.
The software requires competent human intervention for the analysis and results reporting process. To facilitate the interpretation, the NOVA View® provides the trained operator with the acquired digital images and the following supportive information:
- Average nuclear LIU value
- Negative/positive classification based on predetermined LIU cut-off .
{3}------------------------------------------------
-
. Pattern information (homogeneous, speckled, centromere, nucleolar, nuclear dots, or unrecognized for positive results only)
The trained operator reviews the images taken by the NOVA View®. During the review process, the trained operator has the option to: -
Switch between images
-
. Overlay DAPI and FITC images to help identify nuclei and other cellular structures
-
. Enlarge images to examine details
-
. Review "Standard" and "Optimal" images to identify patterns
The trained operator will confirm the results by any combination of the following procedures, and then clicking the "Confirm" button on the screen:
- . Accepting the classification suggested by NOVA View® (negative/positive and pattern), and then clicking the "Confirm" button on the screen
- . Revising the suggested NOVA View® classification (negative to positive and vice versa pattern), and then clicking the "Confirm" button on the screen
- Adding comments and then clicking the "Confirm" button on the screen .
All results must be reviewed and confirmed (or revised and confirmed) by the trained operator. The trained operator confirmed result is the final result. No patient report can be created if the trained operator does not confirm the result.
The instrument cover encloses the microscope, camera, computer, microscope control unit, LED illumination unit (consisting of an LED unit and a collimator attachment to the microscope). NOVA View® includes an Olympus IX83 inverted fluorescence microscope with 4X, 10X, and 40X objectives, and dual band DAPI FITC/HC filters. The microscope is housed under the NOVA View® cover while the slide stage is fixed above the microscope objectives, atop the cover. The microscope is powered by an Olympus Control Unit 1X83-CBH. NOVA View® includes an industrial computer with Windows 7 operating system. The stage includes a slide stage cover that is closed during scanning. The stage is fitted with a slide carrier that can hold up to five standard microscope slides. Digital images are captured by a Kappa Zelos 285M GigE digital camera. The camera is connected to the microscope by an adapter. NOVA View@'s light source is provided by a CoolLed PreciseExcite LED with excitation wavelengths of 400nm (DAPI) and 490nm (FITC). A handheld LED display unit displays activity of the LED unit and is externally accessible.
-
- Modes of Operation:
Does the applicant's device contain the ability to transmit data to a computer, webserver, or mobile device?
- Modes of Operation:
Yes __________________________________________________________________________________________________________________________________________________________________________
{4}------------------------------------------------
Does the applicant's device transmit data to a computer, webserver, or mobile device using wireless transmission?
Yes _________ or No __________________________________________________________________________________________________________________________________________________________
-
- Specimen Identification:
Sample identification (ID) is manually entered through the user interface when a new project is created. Sample information is linked to slide ID and well location in the NOVA View® software. Alternatively, slide and sample information can be transferred through middleware when the slides are processed with the automated slide processing instrument that is integrated into the laboratory information system (LIS), and sample ID and position is already linked to the slide through slide barcode.
- Specimen Identification:
-
- Specimen Sampling and Handling:
Not applicable. The device does not process samples.
- Specimen Sampling and Handling:
-
- Calibration:
The purpose of instrument calibration is to regulate the light intensity of the LED, as the intensity of the excitation light directly influences the intensity of the emitted light. The instrument measures and uses the light intensity information for analyzing results, and light intensity also influences the appearance of the digital images. In order to have consistent light intensity production between instruments, a calibration procedure was established. Calibration is performed at installation, and as part of the yearly preventative maintenance. The procedure uses green fluorescent beads fixed on glass microscope slides. The calibration slides' target fluorescent light intensity value is established by Inova. During the calibration procedure, the calibration slide is scanned by the instrument, and the fluorescent light intensity is measured. The obtained value is compared to the target value, and the LED is adjusted according to a formula that was established by Inova between LED intensity and emitted fluorescent light intensity.
- Calibration:
-
- Quality Control:
Quality Control material is included in the assay reagent kit and must to be included in every run according to the procedure described in the Direction Insert.
- Quality Control:
-
- Software:
FDA has reviewed applicant's Hazard Analysis and Software Development processes for this line of product types:
- Software:
Yes _X or No _________________________________________________________________________________________________________________________________________________________
{5}------------------------------------------------
J. Substantial Equivalence Information:
-
- Predicate Device Name(s) and 510(k) numbers:
Not applicable
- Predicate Device Name(s) and 510(k) numbers:
-
- Comparison with Predicate Device:
Not applicable
- Comparison with Predicate Device:
K. Special Control/Guidance Document Referenced (if applicable):
Guidance for Industry and FDA Staff: Recommendations for Anti-Nuclear Antibody (ANA) Test System Premarket (510(k)) Submissions (January 22, 2009).
C28-A3, Defining, Establishing, and Verifying Reference Intervals in the Clinical Laboratory; Approved Guideline, Third Addition.
EP07-A2, Interference Testing in Clinical Chemistry; Approved Guideline, Second Edition.
EP09-A2IR, Method Comparison and Bias Estimation Using Patient Samples; Approved Guideline, Second Edition (Interim Revision) (used for matrix comparison).
L. Test Principles:
Not applicable. Test principles are assay specific and dependent on the assay indicated for use with the NOVA View® device.
M. Performance Characteristics:
-
- Analytical Performance:
Nomenclature used in studies:
- Analytical Performance:
-
. Throughout the submission, results obtained by manual reading of the same slides are used as reference method
-
"Manual" and "Manual reading" refer to results obtained by the operator reading and interpreting the slides with a traditional fluorescence microscope
-
"Digital", "Digital reading" and "Digital image" refers to results obtained by the trained operator reading the NOVA View® generated images on the computer monitor blinded to the suggested interpretation
-
. "NOVA View" refers to results obtained with the NOVA View® Automated Fluorescence Microscope, such as Light Intensity Units (LIU), positive/negative classification and pattern information without operator interpretation
-
a. Accuracy:
{6}------------------------------------------------
Accuracy for the NOVA View® is assay dependent. Accuracy for each assay run on this device will be assessed at the time of assay clearance.
The Accuracy study design was determined using the NOVA Lite® DAPI ANA Kit, K150155. Accuracy was based on a three-way method comparison of NOVA View® automated software-driven result (NOVA View®) compared to the Digital image reading of the software generated output by a trained operator who was blinded to the automated result (Digital) and compared to the reference standard of conventional IIF manual microcopy (Manual). The results from all three outcome methods (NOVA View®, Digital, and Manual) were compared to clinical truth to determine clinical sensitivity and specificity.
A cohort of 463 clinically characterized samples tested the accuracy and clinical sensitivity and specificity of the NOVA Lite® DAPI ANA kit as scanned and interpreted by the NOVA View®. Digital images were independently interpreted and confirmed by trained human operators. Additionally, each slide was read with a traditional manual fluorescence microscope by the same operator. The same slides were read at three different locations, one internal (site 1) and two external (sites 2 and 3).
| Sample type | Number ofsamples | Number of positive samples | Percent Positive Samples | ||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Healthy control | 75 | Manual | Digital | NOVA View | Manual | Digital | NOVA View | ||||||||||||||||
| HBV | 20 | Sample type | N | Site1 | Site2 | Site3 | Site1 | Site2 | Site3 | Site1 | Site2 | Site3 | Sample type | N | Site1 | Site2 | Site3 | Site1 | Site2 | Site3 | Site1 | Site2 | Site3 |
| HCV | 5 | Healthy control | 75 | 4 | 7 | 13 | 5 | 2 | 8 | 4 | 2 | 19 | Healthycontrol | 75 | 5.3% | 9.3% | 17.3% | 6.7% | 2.7% | 10.7% | 5.3% | 2.7% | 25.3% |
| HIV | 5 | HBV | 20 | 5 | 4 | 2 | 3 | 3 | 1 | 1 | 1 | 4 | HBV | 20 | 25.0% | 20.0% | 10.0% | 15.0% | 15.0% | 5.0% | 5.0% | 5.0% | 20.0% |
| Syphilis | 5 | HCV | 5 | 0 | 1 | 2 | 2 | 1 | 2 | 2 | 1 | 2 | HCV | 5 | 0.0% | 20.0% | 40.0% | 40.0% | 20.0% | 40.0% | 40.0% | 20.0% | 40.0% |
| Systemic Lupus Erythematosus (SLE) | 75 | HIV | 5 | 0 | 0 | 2 | 2 | 1 | 2 | 2 | 2 | 5 | HIV | 5 | 0.0% | 0.0% | 40.0% | 40.0% | 20.0% | 40.0% | 40.0% | 40.0% | 100.0% |
| Systemic Sclerosis (SSC) | 20 | Syphilis | 5 | 0 | 0 | 3 | 3 | 0 | 3 | 3 | 1 | 3 | Syphilis | 5 | 0.0% | 0.0% | 60.0% | 60.0% | 0.0% | 60.0% | 60.0% | 20.0% | 60.0% |
| Sjögren's syndrome (SS) | 20 | SLE | 75 | 54 | 53 | 62 | 60 | 55 | 61 | 60 | 54 | 62 | SLE | 75 | 72.0% | 70.7% | 82.7% | 80.0% | 73.3% | 81.3% | 80.0% | 72.0% | 82.7% |
| Autoimmune Liver Disease (AIL) | 20 | SSc | 20 | 19 | 19 | 19 | 19 | 19 | 19 | 19 | 19 | 19 | SSc | 20 | 95.0% | 95.0% | 95.0% | 95.0% | 95.0% | 95.0% | 95.0% | 95.0% | 95.0% |
| Rheumatoid arthritis (RA) | 20 | SS | 20 | 9 | 11 | 14 | 13 | 9 | 14 | 12 | 9 | 15 | SS | 20 | 45.0% | 55.0% | 70.0% | 65.0% | 45.0% | 70.0% | 60.0% | 45.0% | 75.0% |
| Mixed Connective Tissue Disease (MCTD) | 21 | AIL | 20 | 16 | 18 | 17 | 20 | 17 | 18 | 20 | 17 | 20 | AIL | 20 | 80.0% | 90.0% | 85.0% | 100.0% | 85.0% | 90.0% | 100.0% | 85.0% | 100.0% |
| Autoimmune myositis | 26 | RA | 20 | 11 | 15 | 15 | 14 | 14 | 13 | 15 | 13 | 13 | RA | 20 | 55.0% | 75.0% | 75.0% | 70.0% | 70.0% | 65.0% | 75.0% | 65.0% | 65.0% |
| Fibromyalgia | 25 | MCTD | 21 | 10 | 10 | 8 | 10 | 8 | 8 | 10 | 8 | 8 | MCTD | 21 | 47.6% | 47.6% | 38.1% | 47.6% | 38.1% | 38.1% | 47.6% | 38.1% | 38.1% |
| Anti-MPO/anti-PR3 | 26 | Autoimmune myositis | 26 | 7 | 9 | 10 | 6 | 7 | 7 | 6 | 8 | 8 | Autoimmunemyositis | 26 | 26.9% | 34.6% | 38.5% | 23.1% | 26.9% | 26.9% | 23.1% | 30.8% | 30.8% |
| Crohn's/Inflammatory bowel disease | 20 | Fibromyalgia | 25 | 9 | 11 | 9 | 6 | 6 | 10 | 6 | 5 | 8 | Fibromyalgia | 25 | 36.0% | 44.0% | 36.0% | 24.0% | 24.0% | 40.0% | 24.0% | 20.0% | 32.0% |
| Autoimmune thyroiditis | 24 | Anti-MPO/anti-PR3 | 26 | 3 | 5 | 4 | 4 | 4 | 4 | 1 | 5 | 5 | Anti-MPO/anti-PR3 | 26 | 11.5% | 19.2% | 15.4% | 15.4% | 15.4% | 15.4% | 3.8% | 19.2% | 19.2% |
| Celiac disease | 24 | Crohn's/Inflammatorybowel disease | 20 | 9 | 8 | 8 | 8 | 7 | 7 | 8 | 6 | 7 | Crohn's/Inflammatorybowel disease | 20 | 45.0% | 40.0% | 40.0% | 40.0% | 35.0% | 35.0% | 40.0% | 30.0% | 35.0% |
| Drug induced lupus (DIL) | 25 | Autoimmune thyroiditis | 24 | 4 | 6 | 5 | 3 | 2 | 4 | 2 | 3 | 5 | Autoimmunethyroiditis | 24 | 16.7% | 25.0% | 20.8% | 12.5% | 8.3% | 16.7% | 8.3% | 12.5% | 20.8% |
| Other | 7 | Celiac disease | 24 | 4 | 7 | 7 | 3 | 5 | 4 | 3 | 3 | 2 | Celiac disease | 24 | 16.7% | 29.2% | 29.2% | 12.5% | 20.8% | 16.7% | 12.5% | 12.5% | 8.3% |
| Total | 463 | Drug induced Lupus (DIL) | 25 | 5 | 5 | 7 | 5 | 5 | 5 | 4 | 5 | 4 | Drug inducedLupus (DIL) | 25 | 20.0% | 20.0% | 28.0% | 20.0% | 20.0% | 20.0% | 16.0% | 20.0% | 16.0% |
| Other | 7 | 2 | 1 | 2 | 1 | 1 | 1 | 1 | 1 | 2 | Other | 7 | 28.6% | 14.3% | 28.6% | 14.3% | 14.3% | 14.3% | 14.3% | 14.3% | 28.6% | ||
| Total | 463 | 171 | 190 | 209 | 187 | 166 | 191 | 179 | 163 | 211 | Total | 463 |
The number and distribution of the samples are shown below:
{7}------------------------------------------------
Number Positive and Percent Positivity rates in the various disease cohorts by method (NOVA View®, Manual read or Digital read) at the three locations are listed below:
{8}------------------------------------------------
Because of concerns about sample quality, 21 of the 25 drug induced lupus (DIL) samples have not been included in the sensitivity calculations, but were included in the agreement calculations. The remaining four DIL samples were included in the sensitivity calculations.
{9}------------------------------------------------
Sensitivity was calculated at each site on SLE separately, and on the combination of the connective tissue diseases (CTD; SLE + systemic sclerosis + Sjögren's + MCTD + autoimmune myositis + DIL) plus autoimmune liver disease (AIL) population. Specificity was calculated on the total control population (excluding healthy subjects).
| Site | 1: | |
|---|---|---|
| Site 1 | Sensitivity % (95% CI) | Specificity%(95% CI)no healthyN=174 | |
|---|---|---|---|
| SLEN=75 | CTD+AILN=186 | ||
| ManualRead | 72.0(60.4 to 81.8) | 62.9(55.5 to 69.9) | 74.1(67.0 to 80.5) |
| DigitalRead | 80.0(69.2 to 88.4) | 69.9(62.8 to 76.4) | 72.4(65.1 to 78.9) |
| NOVAView | 80.0(69.2 to 88.4) | 69.4(62.2 to 75.9) | 75.3(68.2 to 81.5) |
Site 2:
| Site 2 | Sensitivity % (95% CI) | Specificity%(95% CI) | |
|---|---|---|---|
| SLEN=75 | CTD+AILN=186 | no healthyN=174 | |
| ManualRead | 70.7(59.0 to 80.6) | 65.6(58.3 to 72.4) | 67.2(59.7 to 74.2) |
| DigitalRead | 73.3(61.9 to 82.9) | 62.9(55.5 to 69.9) | 75.3(68.2 to 81.5) |
| NOVAView | 72.0(60.4 to 81.8) | 62.9(55.5 to 69.9) | 77.0(70.0 to 83.0) |
| Site | 3: | |||
|---|---|---|---|---|
| ------ | -- | -- | ---- | -- |
| Site 3 | Sensitivity % (95% CI) | Specificity %(95% CI) | |
|---|---|---|---|
| SLEN=75 | CTD+AILN=186 | no healthyN=174 | |
| ManualRead | 82.7 | 71.0 | 67.2 |
| (72.2 to 90.4) | (63.9 to 77.4) | (59.7 to 74.2) | |
| DigitalRead | 81.3 | 69.4 | 71.3 |
| (70.7 to 89.4) | (62.2 to 75.9) | (63.9 to 77.9) | |
| NOVAView | 82.7 | 72.0 | 69.0 |
| (72.2 to 90.4) | (65.0 to 78.4) | (61.5 to 75.7) |
Agreement results between NOVA View® classification, digital image reading and manual reading were calculated within each testing location and between locations:
{10}------------------------------------------------
| N=463 | PositiveAgreement %(95% CI) | NegativeAgreement %(95% CI) | TotalAgreement %(95% CI) | |
|---|---|---|---|---|
| Site#1 | NOVA Viewvs. Manual | 88.3(82.5-92.7) | 90.4(86.4-93.5) | 89.6(86.5-92.3) |
| Digital vs.Manual | 93.0(88.1-96.3) | 90.4(86.4-93.5) | 91.4(88.4-93.8) | |
| Digital vs.NOVA View | 94.1(89.7-97.0) | 98.9(96.9-99.8) | 97.0(95.0-98.3) | |
| Site#2 | NOVA Viewvs. Manual | 80.5(74.2-85.9) | 96.3(93.4-98.2) | 89.8(86.7-92.4) |
| Digital vs.Manual | 84.2(78.2-89.1) | 97.8(95.3-99.2) | 92.2(89.4-94.5) | |
| Digital vs.NOVA View | 94.0(89.2-97.1) | 97.6(95.2-99.0) | 96.3(94.2-97.8) | |
| Site#3 | NOVA Viewvs. Manual | 86.1(80.7-90.5) | 87.8(83.1-91.6) | 87.0(83.6-90.0) |
| Digital vs.Manual | 87.1(81.8-91.3) | 96.5(93.4-98.4) | 92.2(89.4-94.5) | |
| Digital vs.NOVA View | 95.8(91.9-98.2) | 89.7(85.5-93.0) | 92.2(89.4-94.5) |
Within-Site Agreement:
Between Site Overall Agreement by interpretation method:
| Between Site Overall Agreement N = 463 | ||
|---|---|---|
| Manual | Site#1 Manual | Site#2 Manual |
| Site#2 Manual | 90.7 (87.7–93.2) | |
| Site#3 Manual | 85.7 (82.2–88.8) | 87.3 (83.9–90.2) |
| Digital | Site#1 Digital | Site#2 Digital |
| Site#2 Digital | 92.0 (89.2–94.3) | |
| Site#3 Digital | 93.1 (90.4–95.2) | 92.0 (89.2–94.3) |
| NOVA View | Site#1 NOVA View | Site#2 NOVA View |
| Site#2 NOVA View | 92.7 (89.9–94.9) | |
| Site#3 NOVA View | 89.6 (86.5–92.3) | 87.9 (84.6–90.7) |
Within-Site Pattern Agreement:
Pattern agreement was assessed in a pair-wise comparison between manual reading,
{11}------------------------------------------------
NOVA View® results, and digital image reading. Only definitive patterns (homogeneous, speckled, centromere, nucleolar, nuclear dots) were considered pattern agreement. NOVA View® reported "Unrecognized" patterns and user reported "Other" patterns were not considered as an agreement.
Out of the 463 clinical samples, there were 171 positive samples at Site #1, 190 at Site #2, and 209 at Site #3 by manual reading (reference method). Agreement between digital image reading and manual reading was above 90% at all three testing sites.
| N=463 | % of samples with pattern agreement* | ||
|---|---|---|---|
| Site#1 | Site#2 | Site#3 | |
| Digital vs Manual | 94.7% | 94.7% | 94.7% |
| NOVA View vs Manual | 76.0% | 86.3% | 72.7% |
| Digital vs NOVA View | 69.6% | 69.6% | 69.6% |
Summary table of pattern percent agreement is shown below:
- As percentage of samples that were positive with manual interpretation.
b. Precision/Reproducibility:
NOVA View® precision is assay-dependent and should be performed with each IVD assay intended for use with the device.
To assess variability within the NOVA View® performance, repeatability and reproducibility studies were conducted using the NOVA Lite® DAPI ANA kit. Results were also generated and compared to the Digital image reading of the software generated output by a trained operator who was blinded to the automated result (Digital) and compared to the reference standard of conventional IIF manual microcopy (Manual).
Repeatability:
To assess repeatability of the NOVA Lite® DAPI ANA kit using both the NOVA View® and a manual microscope, three different studies were performed. For each study, samples were diluted for each run separately; therefore, if 10 runs were performed, 10 dilutions were prepared at the beginning of the slide processing. Within one run, the same dilution was tested in triplicates. The three repeatability studies used the same reagent lot.
In the first study, three negative and 10 positive samples with various patterns and intensities were stained with NOVA Lite® DAPI ANA kit, and tested in triplicate, in 10 runs (two runs per day), resulting in 30 data points for each sample. The slides were scanned by the NOVA View®, and the resulting digital images were interpreted by the operator. The slides in this study were not read with a manual microscope; i.e., only two set of results were generated: NOVA View® output and digital image reading results. The percentage of positive/negative calls are presented below:
{12}------------------------------------------------
| Sample ID | N | Mean LIU | NOVA View output | Digital Reading | ||
|---|---|---|---|---|---|---|
| % Negative | % Positive | % Negative | % Positive | |||
| NVB012 | 30 | 4.7 | 100% | 0% | 100% | 0% |
| NVB007 | 30 | 7.6 | 100% | 0% | 100% | 0% |
| NVB063 | 30 | 7.9 | 100% | 0% | 100% | 0% |
| NVB111 | 30 | 38.5 | 63.3% | 36.7% | 3.3% | 96.7% |
| NVB079 | 30 | 91.6 | 13.3% | 86.7% | 3.3% | 96.7% |
| NVB009 | 30 | 229.1 | 0% | 100% | 0% | 100% |
| NVB029 | 30 | 233.8 | 0% | 100% | 0% | 100% |
| NVB017 | 30 | 310.5 | 0% | 100% | 0% | 100% |
| NVB087 | 30 | 310.6 | 0% | 100% | 0% | 100% |
| NVB023 | 30 | 715.5 | 0% | 100% | 0% | 100% |
| NVB004 | 30 | 933.3 | 0% | 100% | 0% | 100% |
| NVB118 | 30 | 1300.1 | 0% | 100% | 0% | 100% |
| NVB037 | 30 | 2217.7 | 0% | 100% | 0% | 100% |
A second study cohort of samples was selected to challenge the cut-off LIU of the NOVA View® System. Twenty-two samples covering all patterns identified by the NOVA View® which included 20 samples considered borderline/LIU values around cut-off, and 2 samples with 3+ average grade intensity level. Samples were tested in three replicates, in 10 runs (2 runs per day), resulting in 30 data points for each sample. Samples were diluted to target low-positive samples and challenge the NOVA View® LIU and then diluted a second time at 1:80 per the kit instructions for use. The slides were scanned with the NOVA View®, and digital images were interpreted. The same slides were then read with a manual microscope. In total, three set of results were generated: NOVA View® output, digital image reading results and manual reading results.
| Sample ID | N | Mean LIU | NOVA View output | Manual Reading | Digital Reading | |||
|---|---|---|---|---|---|---|---|---|
| % Negative | % Positive | % Negative | % Positive | % Negative | % Positive | |||
| NV20 | 30 | 3.5 | 100% | 0% | 100% | 0% | 100% | 0% |
| NV16 | 30 | 10.2 | 100% | 0% | 100% | 0% | 100% | 0% |
| NV2 | 30 | 11.4 | 100% | 0% | 100% | 0% | 100% | 0% |
| NV8 | 30 | 13.5 | 100% | 0% | 100% | 0% | 100% | 0% |
| NV15 | 30 | 16.6 | 100% | 0% | 13.3% | 86.7% | 16.7% | 83.3% |
| NV9 | 30 | 19.1 | 100% | 0% | 46.7% | 53.3% | 100% | 0% |
| SB24216 | 30 | 31.4 | 86.7% | 13.3% | 0% | 100% | 0% | 100% |
| NV22 | 30 | 33.6 | 76.7% | 23.3% | 96.7% | 3.3% | 76.7% | 23.3% |
| NV26 | 30 | 38.4 | 90.0% | 10.0% | 60.0% | 40.0% | 53.3% | 46.7% |
| NV14 | 30 | 38.8 | 43.3% | 56.7% | 6.7% | 93.3% | 0% | 100% |
| NV13 | 30 | 40.5 | 66.7% | 33.3% | 0% | 100% | 6.7% | 93.3% |
{13}------------------------------------------------
| Sample ID | N | Mean LIU | NOVA View output | Manual Reading | Digital Reading | |||
|---|---|---|---|---|---|---|---|---|
| % Negative | % Positive | % Negative | % Positive | % Negative | % Positive | |||
| NV5 | 30 | 40.7 | 66.7% | 33.3% | 0% | 100% | 16.7% | 83.3% |
| NVB440 | 30 | 43.8 | 73.3% | 26.7% | 33.3% | 66.7% | 46.7% | 53.3% |
| NV4 | 30 | 57.5 | 43.3% | 56.7% | 0% | 100% | 0% | 100% |
| NVB201 | 30 | 62.8 | 26.7% | 73.3% | 0% | 100% | 0% | 100% |
| NVB074 | 30 | 63.8 | 16.7% | 83.3% | 0% | 100% | 0% | 100% |
| NV12 | 30 | 64.8 | 36.7% | 63.3% | 0% | 100% | 0% | 100% |
| NVB369 | 30 | 72.4 | 23.3% | 76.7% | 3.3% | 96.7% | 13.3% | 86.7% |
| NV7 | 30 | 74.1 | 10.0% | 90.0% | 0% | 100% | 0% | 100% |
| NV10 | 30 | 128.5 | 30.0% | 70.0% | 0% | 100% | 0% | 100% |
| NV23 | 30 | 822.4 | 0% | 100% | 0% | 100% | 0% | 100% |
| NV6 | 30 | 903.9 | 0% | 100% | 0% | 100% | 0% | 100% |
A third, separate study was also performed with samples tested in triplicates or duplicates, in five runs, resulting in 15 or 10 data points for each sample. The slides were scanned with the NOVA View®, and digital images were interpreted. Slides were also read with a manual microscope. Three set of results were generated: NOVA View® output, digital image reading results and manual reading results.
| Sample ID | N | Mean LIU | NOVA View output | Manual Reading | Digital Reading | |||
|---|---|---|---|---|---|---|---|---|
| % Negative | % Positive | % Negative | % Positive | % Negative | % Positive | |||
| PMDx 5087 | 15 | 24.6 | 100% | 0% | 100% | 0% | 100% | 0% |
| SS-A Monospecific 08203 | 15 | 103.6 | 0% | 100% | 0% | 100% | 0% | 100% |
| AMA 930328 | 15 | 882.6 | 0% | 100% | 0% | 100% | 0% | 100% |
| Centromere 120571 | 10 | 1052.9 | 0% | 100% | 0% | 100% | 0% | 100% |
| Nucleolar 120559 | 10 | 1339.8 | 0% | 100% | 0% | 100% | 0% | 100% |
| DNA PS0007 520847 | 15 | 1375.6 | 0% | 100% | 0% | 100% | 0% | 100% |
| ANA DNA 420530 | 10 | 1607.8 | 0% | 100% | 0% | 100% | 0% | 100% |
| SmRNP 220951 | 10 | 2811.2 | 0% | 100% | 0% | 100% | 0% | 100% |
For NOVA View®, positive/negative classification was consistent (except for samples around the cut-off). Pattern (for positive samples only) was correct for >80% of the cases (excluding unrecognized patterns).
{14}------------------------------------------------
For both digital image reading and manual reading for study one and study two, intensity grades were within ± 1 reactivity grade within one run (within triplicates), and the average grade was no more than one reactivity grade different between runs. Pattern determination was consistent for 100% of the replicates (for positive samples only).
Reproducibility:
To assess between-operator and between-instrument variability, a reproducibility study was performed at Inova Diagnostics (internal; Site#1) and at two external sites (Sites #2 and #3) using the same sample cohort.
A cohort of 120 samples at each location was processed with NOVA Lite® DAPI ANA kit, and scanned with NOVA View®. Digital images were interpreted and confirmed. Additionally, a second operator read and interpreted the same digital images at each location. Altogether, six digital image datasets were generated (three locations, two operators at each site). The same digital images were read by the two operators at each site, but different slides were read at all three locations.
The 120 samples were selected to represent approximately 50% negative and 50% positive samples with various patterns. All major patterns were represented, and reactivity grades ranged from 0 to +4.
| I I THITTI KITH T THA T TET SATTIATIV | ||||||
|---|---|---|---|---|---|---|
| Positive | Negative | Total | ||||
| N=120 | Agreement % | Agreement % | Agreement % | |||
| (95% CI) | (95% CI) | (95% CI) | ||||
| NOVA View | 100.0 | 98.4 | 99.2 | |||
| vs Manual | (93.7-100.0) | (91.5-100.0) | (95.4-100.0) | |||
| Digital vs | 100.0 | 98.4 | 99.2 | |||
| Site#1 | Manual | (93.7-100.0) | (91.5-100.0) | (95.4-100.0) | ||
| Digital vs | 100.0 | 100.0 | 100.0 | |||
| NOVA View | (93.8—100.0) | (94.2-100.0) | (97.0-100.0) | |||
| NOVA View | તે 95.0 | 98.3 | 96.7 | |||
| vs Manual | (81.6-99.0) | (91.1-100.0) | (91.7-99.1) | |||
| Digital vs | 96.7 | 95.0 | 95.8 | |||
| Site#2 | Manual | (88.5-99.6) | (86.1-99.0) | (90.5-98.6) | ||
| Digital vs | 93.4 | 98.3 | તેર જ | |||
| NOVA View | (84.1-98.2) | (90.9-100.0) | (90.5-98.6) | |||
| Site#3 | NOVA View | 94.6 | 98.4 | 96.7 | ||
| vs Manual | (85.1-96.8) | (91.6-100.0) | (91.7-99.1) | |||
| Digital vs | 92.9 | 100.0 | 96.7 | |||
| Manual | (82.7-98.0) | (94.4-100.0) | (91.7-99.1) | |||
| Digital vs | 100.0 | 97.1 | 98.3 | |||
| NOVA View | (93.2-100.0) | (89.9-99.6) | (94.1-99.8) |
Within Site Reproducibility: Within-Site Agreement:
{15}------------------------------------------------
Within-Site Pattern Agreement across method:
Pattern agreement was assessed in pair-wise comparison between manual reading, NOVA View® results, and digital image reading at each site. Only definitive patterns (Homogeneous, Speckled, Centromere, Nucleolar, Nuclear dots) were considered as pattern agreement. NOVA View® reported "Unrecognized" patterns and user reported "Other" patterns were not considered as an agreement.
Out of the 120 samples in the reproducibility cohort, there were 57 positive samples at Site #1, 60 at Site #2 and 56 at Site #3 by manual reading (reference method). A summary table of the pattern agreement is shown below:
| Percent of samples with pattern agreement* | |||
|---|---|---|---|
| N=120 | Site#1 | Site#2 | Site#3 |
| Digital vs Manual | 96.5% | 95.0% | 96.4% |
| NOVA View vs Manual | 78.9% | 83.3% | 80.4% |
| Digital vs NOVA View | 77.2% | 80.0% | 80.4% |
- As percentage of samples that were positive with manual interpretation.
Fluorescent intensity (grade) agreement:
Fluorescence intensity grades were within ± one grade from each other between manual reading and digital image reading, as shown below:
| N=120 | Percent of samples within ± one grade | ||
|---|---|---|---|
| Site#1 | Site#2 | Site#3 | |
| Digital vs Manual | 98.3% | 99.2% | 99.2% |
Between-Site Reproducibility:
Between-site reproducibility was assessed by calculating average positive, average negative and total agreement between NOVA View® generated results, digital image reading result and manual (traditional) reading results between the three sites. Confidence intervals were determined using bootstrap analysis. Results are shown below:
| Manual ReadingN=120 | Site #1 vs.Site #2 | Site #1 vs.Site #3 | Site #2 vs.Site #3 |
|---|---|---|---|
| Average PositiveAgreement % (95%CI) | 97.4(94.0-100.0) | 99.1(97.0-100.0) | 96.6(92.7-99.2) |
| Average NegativeAgreement % (95%CI) | 97.6(94.3-100.0) | 99.2(97.4-100.0) | 96.8(93.1-99.3) |
| Overall Agreement %(95%CI) | 97.5(92.9-99.5) | 99.2(95.4-100.0) | 96.9(91.7-99.1) |
Manual Reading Between-Site
{16}------------------------------------------------
| Digital Reading*N=120 | Site #1 vs.Site #2 | Site #1 vs.Site #3 | Site #2 vs.Site #3 |
|---|---|---|---|
| Average PositiveAgreement % (95%CI) | 95.8(91.6–99.2) | 94.5(89.6–98.3) | 92.0(86.2–96.7) |
| Average NegativeAgreement % (95%CI) | 95.9(97.1–99.2) | 95.4(91.2–98.6) | 92.9(87.7–97.1) |
| Overall Agreement %(95%CI) | 95.8(90.5–98.6) | 95.0(89.4–98.1) | 925(86.2–96.5) |
Digital Reading Between-Site:
*Considering only Operator #1 results. Operator #2 had similar results (see betweenoperator agreement below).
NOVA View® Interpretation Between-Site:
| NOVA ViewN=120 | Site #1 vs.Site #2 | Site #1 vs.Site #3 | Site #2 vs.Site #3 |
|---|---|---|---|
| Average PositiveAgreement % (95%CI) | 100.0(100.0-100.0) | 96.4(92.4-99.2) | 96.4(92.4-99.2) |
| Average NegativeAgreement % (95%CI) | 100.0(100.0-100.0) | 96.6(93.3-99.3) | 96.6(93.3-99.3) |
| Overall Agreement %(95%CI) | 100.0(97.0-100.0) | 96.7(91.7-99.1) | 96.7(91.7-99.1) |
Between Operator Agreement:
Between operators, total agreement was > 90% in each of the 15 pair-wise comparisons, as shown in the matrix below:
| % Overall Agreement (Positive/Negative)for digital image reading across all operators | ||||||
|---|---|---|---|---|---|---|
| Site #1Op #1 | Site #1Op #2 | Site #2Op #1 | Site #2Op #2 | Site #3Op #1 | Site #3Op #2 | |
| Site #1 Op #1 | N/A | 99.2 | 95.8 | 100.0 | 95.0 | 94.2 |
| Site #1 Op #2 | N/A | N/A | 95.0 | 99.2 | 94.2 | 93.3 |
| Site #2 Op #1 | N/A | N/A | N/A | 95.8 | 92.5 | 91.7 |
| Site #2 Op #2 | N/A | N/A | 95.0 | 94.2 | ||
| Site #3 Op #1 | N/A | 99.2 |
c. Linearity:
NOVA View® based on LIU is assay-dependent and should be performed with each IVD assay intended for use with the device, if applicable.
The NOVA Lite® DAPI ANA Kit using the NOVA View® is run only at a 1:80 dilution and does not have a formal analytical measuring range and there is no upper
{17}------------------------------------------------
LIU limit. The reported LIU values range from 0 to approximately 3000 LIU for highly positive samples.
-
d. Carryover:
Not applicable. Sample processing is not a function of the device. -
e. Interfering Substances:
Not applicable. Interference for each assay run on this system is assessed during the clearance of the assay.
2. Other Supportive Instrument Performance Data Not Covered Above:
- a. Single Well Titer (SWT):
The Single Well Titer (SWT) is a proprietary application of the NOVA View® that estimates the endpoint titer (the highest dilution that produces a positive result) of a sample containing ANA. The SWT application uses two pieces of information: the measured LIU value of the sample, and the user confirmed pattern information.
Single Well Titer Establishment:
The NOVA View® SWT function was established 38 ANA positive samples representing the five major patterns. The software application automatically performs the calculations based on the predetermined dilution curve, the LIU produced by the sample, and the pattern of the ANA. If the LIU value is above the highest value of the dilution curve, the titer will be described as greater than or equal to (≥) the highest titer that can be calculated from that curve.
| Pattern | Maximum titer calculatedby the dilution curve |
|---|---|
| Homogeneous | ≥ 2560 |
| Speckled | ≥ 2560 |
| Nucleolar | ≥ 5120 |
| Centromere | ≥ 5120 |
| Nuclear Dots | ≥ 5120 |
The maximum SWT values are listed belows
Single Well Titer Validation:
Two sets of validation studies were performed.
The SWT application was originally validated at Inova. Fifty (50) samples, representing all five major patterns were titered with the traditional method (i.e., using two fold dilution series) starting at 1:80 and diluted to 1:40,960. All dilutions
{18}------------------------------------------------
were tested with the NOVA Lite® DAPI ANA kit, and read with NOVA View®. Digital images were interpreted and confirmed by the operator. Additionally, slides were read with traditional manual microscope by the same operator. Endpoint titer was determined by both manual and digital reading. The SWT application was initiated from the well that contained the 1:80 serum dilutions.
SWT results were compared to the endpoints obtained with manual microscopy, and to the endpoint obtained with the digital reading of NOVA View® images at each testing site as shown below. The endpoint titer obtained with the SWT application was within ± 2 dilution steps from the manual endpoint and the digital endpoint for 48 (96%) and 49 (98%) out of 50 samples, respectively. Moreover, 86% and 90% of the samples (compared to manual and digital endpoint) we were within the ± 1 dilution step endpoint difference.
| Number of samples | |||
|---|---|---|---|
| Dilution | SWT vs manual | SWT vs digital | |
| Steps | endpoint | endpoint | |
| ±0 | 26 | રૂટ | |
| +1 | 17 | 10 | |
| ± 2 | ನ | 4 | |
| > ± 2 | 2 | 1 | |
| Total | 50 | 50 |
The SWT application was also validated during the clinical study at the two external sites as well as at Inova Diagnostics. Altogether, 20 ANA positive samples (the same 20 samples at each location) with various intensities and patterns were titered with the traditional method (i.e., using two-fold dilution series) up to 1:5120. All dilutions were processed with the NOVA Lite® DAPI ANA kit, and scanned with NOVA View®. Digital images were interpreted and confirmed by the operator. Additionally, slides were read with traditional manual microscope by the same operator. Endpoint titer was determined by both manual and digital reading. The SWT application was initiated from the well that contained the 1:80 serum dilution.
Endpoint titer obtained with the SWT application was within ± 2 dilution steps from the manual and the digital endpoint for all 20 samples at all three locations. SWT results at the three sites were within ± 1 dilution steps from each other for 19 out of the 20 samples.
- b. Refer to the K150155 for additional NOVA View® performance parameters that are specific for the NOVA Lite® DAPI ANA Kit.
N. Proposed Labeling:
The labeling is sufficient and it satisfies the requirements of 21 CFR Parts 801 and 809 and the special controls.
{19}------------------------------------------------
O. Identified Risks to Health and Required Mitigations:
The device (instrument and software system) might be used with a variety of disease indications and associated analytes that must be cleared for use with the device. Risks may vary depending on the indications for use of the specific assay used with the device. The primary risks of this device are related to the consequences of clinical decisions based on false negative and false positive results for a patient due to inaccurate test results or failure to correctly interpret test results. For a false positive result, the risks could include unnecessary testing or inappropriate treatment related to an inaccurate result. For a false negative result, the risk could include a missed or delayed diagnosis. Assay specific performance studies outlined in the special controls will further mitigate risk associated with the device. The identified risks and required mitigations associated with the device type are summarized in the Table below.
| Identified Risks to Health | Required Mitigations |
|---|---|
| Inaccurate test results that provide falsepositive or false negative results. | Special controls (1), (2), and (3) |
| Failure to correctly interpret test results canlead to false positive or false negative results | Special controls (1), (2)(i), (2)(ii)(A), (2)(ii)(B),(2)(iii), and (3) |
P. Benefit/Risk Analysis:
| Summary | |
|---|---|
| Summary ofthe Benefit(s) | This is the first automated IIF system to be commercially available in the US,providing an unmet medical need. |
| The benefit of an automated IIF system is the reduction of intra- and inter-laboratoryvariability and higher throughput in laboratory workflow. | |
| This is of benefit to patients by providing potentially greater accuracy and more timelytest results, which could ultimately lead to earlier diagnosis and initiation ofappropriate therapy. |
{20}------------------------------------------------
| Summary ofthe Risk(s) | The primary risks are related to the consequences of clinical decisions based on falsenegative and false positive results for a patient due to inaccurate test results or failureto correctly interpret test results. For a false positive result, the risks could includeunnecessary testing or inappropriate treatment related to an inaccurate result. For afalse negative result, the risk could include a missed or delayed diagnosis. The resultsfrom this test would be used with results from other diagnostic tests, otherautoantibodies, and clinical signs and symptoms, which would mitigate these risks.The risk of false negative and false positive results are also mitigated by statements inthe Intended Use for the automated instrument which state that the device can only beused with reagents that are indicated for use on the system, and that the resultsgenerated by the automated instrument must be confirmed by a trained operator. Therisks are further mitigated by the special controls.The test requires that a blood sample be obtained during routine phlebotomy. This isstandard procedure in clinical care, and the risk is minimal. |
|---|---|
| Summary ofOther Factors | None |
| ConclusionsDo theprobablebenefitsoutweigh theprobable risks? | Given the well characterized performance characteristics, statements in theIntended Use for the device and special controls, including performance data,references to the legally marketed assays intended for use with the device, andwarnings required in the labeling, the probable benefits outweigh the probablerisks for this device. |
Q. Conclusion:
The information provided in this de novo submission is sufficient to classify this device into class II under regulation 21 CFR 866.4750. FDA believes that special controls, along with the applicable general controls, provide reasonable assurance of the safety and effectiveness of the device type. The device is classified under the following:
Product Code: PIV
Automated indirect immunofluorescence microscope and software-Device Type: assisted system.
Class: II (special controls).
Regulation: 21 CFR 866.4750
{21}------------------------------------------------
- (a) Identification. The automated indirect immunofluorescence (IIF) microscope and softwareassisted system is a device that acquires, analyzes, stores, and displays digital images of indirect immunofluorescent slides. It is intended to be used as an aid in the determination of antibody status in clinical samples. The device may include a fluorescence microscope with light source, a motorized microscope stage, dedicated instrument controls, a camera, a computer, a sample processor, or other hardware components. The software may include fluorescent signal acquisition and processing software, data storage and transferring mechanisms, or assay specific algorithms to suggest results. A trained operator must confirm results generated with the device.
- (b) Classification. Class II (special controls). Automated indirect immunofluorescence (IIF) microscope and software-assisted system must comply with the following special controls:
- (1) The labeling for the device must reference legally marketed assays intended for use with the device.
- (2) Premarket notification submissions must include the following information:
- (i) A detailed description of the device that includes:
- (A) A detailed description of instrumentation and equipment, and illustrations or photographs of non-standard equipment or methods, if applicable.
- Detailed documentation of the software, including, but not limited to, (B) standalone software applications and hardware-based devices that incorporate software, if applicable.
- A detailed description of appropriate internal and external quality (C) controls that are recommended or provided. The description must identify those control elements that are incorporated into the recommended testing procedures.
- Detailed description and specifications for sample preparation, processing (D) and storage, if applicable.
- Methodology and protocols for detecting fluorescence and visualizing (E) results.
- Detailed specification of the criteria for test results interpretation and (F) reporting.
- Data demonstrating the performance characteristics of the device, which (ii) must include:
- (A) A comparison study of the results obtained with the conventional manual method (i.e., reference standard), the device, and the reading of the digital image without aid of the software, using the same set of patient samples for each. The study must use a legally marketed assay intended for use with the device. Patient samples must be from the assay-specific intended use population and differential diagnosis population. Samples must also cover the assay measuring range, if applicable.
- Device clinical performance established by comparing device results at (B)
- (i) A detailed description of the device that includes:
{22}------------------------------------------------
multiple U.S. sites to the clinical diagnostic standard used in the U.S., using patient samples from the assay-specific intended use population and the differential diagnosis population. For all samples, the diagnostic clinical criteria and the demographic information must be collected and provided. Clinical validation must be based on the determination of clinical sensitivity and clinical specificity using the test results (e.g., antibody status based on fluorescence to include pattern and titer, if applicable) compared to the clinical diagnosis of the subject from whom the clinical sample was obtained. The data must be summarized in tabular format comparing the result generated by automated, manual, and digital only interpretation to the disease status.
- (C) Device precision/reproducibility data generated from within-run, between-run, between-day, between-lot, between-operator, betweeninstruments, between-site, and total precision for multiple nonconsecutive days (as applicable) using multiple operators, multiple instruments and at multiple sites. A well-characterized panel of patient samples or pools from the associated assay specific intended use population must be used.
- Device linearity data generated from patient samples covering the (D) assay measuring range, if applicable.
- Device analytical sensitivity data, including limit of blank, limit of (E) detection and limit of quantitation, if applicable.
- Device assay specific cut-off, if applicable. (F)
- Device analytical specificity data, including interference by (G) endogenous and exogenous substances, if applicable.
- (H) Device instrument carryover data, if applicable.
- Device stability data including real-time stability under various storage (1) times and temperatures, if applicable.
- Information on traceability to a reference material and description of (1) value assignment of calibrators and controls, if applicable.
- (iii) Identification of risk mitigation elements used by the device, including description of all additional procedures, methods, and practices incorporated into the directions for use that mitigate risks associated with testing.
- (3) Your 21 CFR 809.10 compliant labeling must include:
- A warning statement that reads "The device is for use by a trained operator (i) in a clinical laboratory setting."
- (ii) A warning statement that reads "All software-aided results must be confirmed by the trained operator."
- (iii) A warning statement that reads "This device is only for use with reagents that are indicated for use with the device."
- (iv) A description of the protocol and performance studies performed in accordance with special control (2)(ii) and a summary of the results, if applicable.
§ 866.4750 Automated indirect immunofluorescence microscope and software-assisted system.
(a)
Identification. An automated indirect immunofluorescence microscope and software assisted system is a device that acquires, analyzes, stores, and displays digital images of indirect immunofluorescent slides. It is intended to be used as an aid in the determination of antibody status in clinical samples. The device may include a fluorescence microscope with light source, a motorized microscope stage, dedicated instrument controls, a camera, a computer, a sample processor, or other hardware components. The device may use fluorescent signal acquisition and processing software, data storage and transferring mechanisms, or assay specific algorithms to suggest results. A trained operator must confirm results generated with the device.(b)
Classification. Class II (special controls). The special controls for this device are:(1) The labeling for the device must reference legally marketed assays intended for use with the device.
(2) Premarket notification submissions must include the following information:
(i) A detailed description of the device that includes:
(A) A detailed description of instrumentation and equipment, and illustrations or photographs of non-standard equipment or methods, if applicable;
(B) Detailed documentation of the software, including, but not limited to, stand-alone software applications and hardware-based devices that incorporate software, if applicable;
(C) A detailed description of appropriate internal and external quality controls that are recommended or provided. The description must identify those control elements that are incorporated into the recommended testing procedures;
(D) Detailed description and specifications for sample preparation, processing, and storage, if applicable;
(E) Methodology and protocols for detecting fluorescence and visualizing results; and
(F) Detailed specification of the criteria for test results interpretation and reporting.
(ii) Data demonstrating the performance characteristics of the device, which must include:
(A) A comparison study of the results obtained with the conventional manual method (
i.e., reference standard), the device, and the reading of the digital image without aid of the software, using the same set of patient samples for each. The study must use a legally marketed assay intended for use with the device. Patient samples must be from the assay-specific intended use population and differential diagnosis population. Samples must also cover the assay measuring range, if applicable;(B) Device clinical performance established by comparing device results at multiple U.S. sites to the clinical diagnostic standard used in the United States, using patient samples from the assay-specific intended use population and the differential diagnosis population. For all samples, the diagnostic clinical criteria and the demographic information must be collected and provided. Clinical validation must be based on the determination of clinical sensitivity and clinical specificity using the test results (
e.g., antibody status based on fluorescence to include pattern and titer, if applicable) compared to the clinical diagnosis of the subject from whom the clinical sample was obtained. The data must be summarized in tabular format comparing the result generated by automated, manual, and digital only interpretation to the disease status;(C) Device precision/reproducibility data generated from within-run, between-run, between-day, between-lot, between-operator, between-instruments, between-site, and total precision for multiple nonconsecutive days (as applicable) using multiple operators, multiple instruments and at multiple sites. A well-characterized panel of patient samples or pools from the associated assay specific intended use population must be used;
(D) Device linearity data generated from patient samples covering the assay measuring range, if applicable;
(E) Device analytical sensitivity data, including limit of blank, limit of detection, and limit of quantitation, if applicable;
(F) Device assay specific cutoff, if applicable;
(G) Device analytical specificity data, including interference by endogenous and exogenous substances, if applicable;
(H) Device instrument carryover data, if applicable;
(I) Device stability data including real-time stability under various storage times and temperatures, if applicable; and
(J) Information on traceability to a reference material and description of value assignment of calibrators and controls, if applicable.
(iii) Identification of risk mitigation elements used by the device, including description of all additional procedures, methods, and practices, incorporated into the directions for use that mitigate risks associated with testing.
(3) Your 21 CFR 809.10 compliant labeling must include:
(i) A warning statement that reads “The device is for use by a trained operator in a clinical laboratory setting”;
(ii) A warning statement that reads “All software-aided results must be confirmed by the trained operator”;
(iii) A warning statement that reads “This device is only for use with reagents that are indicated for use with the device”; and
(iv) A description of the protocol and performance studies performed in accordance with paragraph (b)(2)(ii) of this section and a summary of the results, if applicable.