(197 days)
Yes
The summary mentions "automated HELIOS interpretation" and provides performance metrics for this automated method (Method A) compared to manual methods. While it doesn't explicitly use the terms AI or ML, automated interpretation of complex biological images like immunofluorescence is highly suggestive of the use of such technologies, especially in the context of a system like HELIOS which is designed for automated IFA processing and analysis. The lower agreement of the automated method compared to manual methods also aligns with the typical performance characteristics of early or less mature AI/ML models in image analysis, requiring human confirmation.
No.
This device is an in vitro diagnostic (IVD) assay designed for the qualitative and/or semi-quantitative determination of antibodies to native double-stranded DNA (dsDNA) in human serum, used as an aid for the diagnosis of Systemic Lupus Erythematosus (SLE). It does not treat or prevent disease, thus it is not a therapeutic device.
Yes
This device is described as an "in vitro diagnostic assay" used "as an aid for the diagnosis of Systemic Lupus Erythematosus (SLE)." Its intended use is clearly diagnostic, providing information for the diagnosis of a disease.
No
The device description clearly lists several physical components included in the kit, such as slides, vials of reagents, and buffers. While it mentions the use of the HELIOS® AUTOMATED IFA SYSTEM (a separate device) for processing and analysis, the core device being described is a physical in vitro diagnostic assay kit.
Yes, this device is an IVD (In Vitro Diagnostic).
Here's why:
- Intended Use/Indications for Use: The document explicitly states "This in vitro diagnostic assay is used as an aid for the diagnosis of Systemic Lupus Erythematosus (SLE) in conjunction with other clinical and laboratory findings." This directly aligns with the definition of an in vitro diagnostic device, which is intended for use in the diagnosis of disease or other conditions, including a determination of the state of health, in order to cure, mitigate, treat, or prevent disease or its sequelae.
- Device Description: The description details a kit containing reagents and slides used to perform a test on human serum, which is a sample taken from the human body. This is characteristic of an IVD.
- Intended User/Care Setting: The intended user is a "trained operator in a clinical laboratory setting," which is a typical environment for performing IVD tests.
- Performance Studies: The document includes detailed performance studies (analytical and clinical) demonstrating the device's ability to detect antibodies in human serum and its comparison to a predicate device. This type of testing is required for IVD devices to demonstrate their safety and effectiveness.
- Predicate Device: The mention of a predicate device (K880742; NOVA Lite dsDNA Crithidia luciliae) further confirms that this device is being compared to an already legally marketed IVD.
No
The letter does not explicitly state that the FDA has reviewed, approved, or cleared a PCCP for this specific device.
Intended Use / Indications for Use
AESKUSLIDES® nDNA (Crithidia luciliae) is an indirect immunofluorescence assay utilizing Crithidia luciliae coated slides as a substrate for the qualitative and/or semi-quantitative determination of antibodies to native double stranded DNA (dsDNA) in human serum. This in vitro diagnostic assay is used as an aid for the diagnosis of Systemic Lupus Erythematosus (SLE) in conjunction with other clinical and laboratory findings. The assay can be processed manually and analyzed at the microscope or processed and analyzed with HELIOS® AUTOMATED IFA SYSTEM. All suggested results obtained with the HELIOS® AUTOMATED IFA SYSTEM must be confirmed by trained personnel.
Product codes
LSW
Device Description
AESKUSLIDES® nDNA (Crithidia luciliae) is an indirect immunofluorescence assay utilizing Crithidia luciliae coated slides as a substrate for the qualitative and/or semiquantitative determination of antibodies to native double stranded DNA (dsDNA) in human serum.
Each kit contains (Quantity depends on product variant):
- -Slides, each containing 10 wells coated with Crithidia Luciliae cells
- 4.0 ml vial containing Fluorescein (FITC) labelled Anti-human Antibody lgG conjugate in a solution of BSA, ready for use
- -0.5 ml vial of positive control containing human serum (diluted), ready for Use
- 0.5 ml vial of negative control containing diluted human serum, ready for use -
- -8.0 ml vial of mounting medium containing a solution of glycerol and PBS, ready for use
- 70 ml bottle of sample buffer, containing BSA, PBS and ready for use -
- -100 ml bottle of wash buffer, concentrated buffer 1:10 in distilled water, containing BSA, PBS.
Mentions image processing
Yes
Mentions AI, DNN, or ML
Not Found
Input Imaging Modality
Fluorescence microscopy
Anatomical Site
Not Found
Indicated Patient Age Range
Not Found
Intended User / Care Setting
trained operator in a clinical laboratory setting
Description of the training set, sample size, data source, and annotation protocol
Not Found
Description of the test set, sample size, data source, and annotation protocol
776 Clinical samples have been used for the Clinical Evaluation and Method Comparisons studies described herein.
An exhaustive search for US Clinical Samples from 10 BioBanks (BioChain, BioReclamationIVT, Bioserve, ConversantBio, Cureline, DiscoveryLifeSciences, iSpecimen, Precision for Medicine, ProMedDx, and Vitrologic) has been performed and 746 Clinical Samples have been found. The samples have been selected according to their diagnosis, to reflect all important diagnosis for this study (see table). The sample set has been completed with 30 serum samples from a German University Hospital to complement some rare diagnosis (Vasculitis).
Sample Definition | Diagnosis | n | |
---|---|---|---|
Target | |||
Diagnosis | SLE | SLE | 277 |
Differential | |||
Diagnosis | APS | APS | 19 |
Other rheumatic | |||
diseases | Mixed connective tissue disease (MCTD) | 31 | |
Undifferentiated connective tissue disease (UCTD) | 21 | ||
Systemic sclerosis (SSc) | 44 | ||
Sjögren's syndrome | 30 | ||
Dermatomyositis (DM) and polymyositis (PM) | 37 | ||
Rheumatoid arthritis (RA) | 58 | ||
Autoimmune Liver | |||
Diseases | Autoimmune Hepatitis | 21 | |
Primary biliary cholangitis (PBC) | 20 | ||
Primary sclerosing cholangitis (PSC) | 10 | ||
Vasculitis | AAVs | 44* | |
Fibromyalgia | Fibromyalgia | 30 | |
Infections | HCV | 44 | |
HBV | 34 | ||
EBV | 25 | ||
Leukemia | Lymphoma, Myeloma | 31 | |
Sum | 776 |
*30 samples from Germany.
The diagnosis criteria of the different samples have been made in agreement with diagnostic standards used in the U.S and Germany. A written statement from different serum suppliers is available on request. Standard criteria are for example ACR criteria.
The US sample set was also selected to contain different ethnic groups (e.g. White, Black/Black African, Asian, Hispanic) to reflect the ethnic composition of the US population as good as possible.
The samples have been checked for purity and volume by visual inspection and for further contaminations by an in house prescreening before inclusion into the study. All 776 sera passed and were determined suitable for the study.
All serum samples which have been used for this studies have been shipped and stored at -20°C. Upon arrival from the Biobanks they were thawed once to aliquot them for the different studies. Subsequently, samples were frozen again. Therefore, samples underwent 2-3 freezing-thawing cycles before they were used in the studies.
Annotation protocol: Not Found
Summary of Performance Studies
Study Type: Analytical performance, Precision/Reproducibility (Within-Lab, Between-Lab, Lot to Lot), Carryover, Time Extension, Analytical Specificity (Interfering substances), Method Comparison with Predicate Device, Method Comparison of Method A, B, and C, Endpoint Titer Comparison, Normal Range Study
Sample Size:
- Serum Stability: 8 serum samples (7 positive, 1 negative)
- Long Term Serum Stability: 7 different serum samples (4 positive and 3 negative)
- Within-Lab Precision: 11 samples
- Between-Lab Precision: 11 samples, tested on three different study sites
- Lot to Lot Precision: 11 sera
- Carryover: 3 high positive samples, 1 negative sample
- Time Extension: 10 samples
- Analytical Specificity: 8 serum samples
- Method comparison with predicate device: 776 serum samples (297 SLE, 479 other diseases)
- Method Comparison of Method A, B, and C: 776 serum samples (297 SLE, 479 other diseases) on 3 different study sites
- Endpoint Titer Comparison: 5 serum samples
- Normal Range Study: 164 sera from healthy donors
AUC: Not Found
MRMC: Not Found
Standalone performance:
- Serum Stability: 100% Positive/Negative/Overall Agreement, 100% FI agreement.
- Long Term Serum Stability: Positive, negative, and FI results remained within acceptance criteria over 14 months.
- Analytical Specificity: 100% Positive Agreement (except for 20 mg/ml Triglycerides with Reader 1 (97%)), 100% Negative Agreement, Overall Agreement ranged from 97.9% to 100%, FI Agreement was 100%.
- Carryover: No carryover observed.
- Time Extension: All positive samples identified as positive, all negative as negative. FI did not deviate more than +/-1 level.
Key results:
- Precision/Reproducibility:
- Within-Lab Precision: All pre-determined acceptance criteria were met for Method A, B, and C.
- Between-Lab Precision: All acceptance criteria were met for Method C (Manual) and Method B (Reader Confirmation) with agreements >90%. For Method A (HELIOS suggestion), acceptance criteria were met for negative and overall agreement (>70%) for all study sites. Positive agreement for Method A was >70% for some site comparisons and overall across all three sites. Overall-Between-Lab agreements ranged from 72.6% to 80.2% for Method A, 97.7% to 99.2% for Method B, and 96.9% to 98.4% for Method C.
- Between-Operator Agreement: Overall-Between-Operator agreements ranged from 96.3% to 99.3% for Method B and 98.9% to 100% for Method C.
- Single-Operator Agreement: Overall-Single-Operator agreements ranged from 95.2% to 100% for Method B and 98.1% to 100% for Method C.
- Instrument Precision (HELIOS - Method A): Overall-Instrument agreements ranged from 69.3% to 84.4%. Acceptance criteria (>70%) were met for Site 1 and Site 2. Site 3 showed slightly lower positive agreement (61.9%) due to out-of-focus images, emphasizing the need for trained reader confirmation.
- Lot to Lot Precision: Overall lot to lot agreement for AESKUSLIDES nDNA was 100%. All acceptance criteria were fulfilled.
- Method comparison with predicate device: AESKUSLIDES nDNA showed lower diagnostic sensitivity (24.6%) but higher specificity (94.8%) compared to the predicate (37% sensitivity, 85.8% specificity). Overall Agreement between the two assays was 85.7%, and negative agreement was 97.8%. The lower positive agreement (44.9%) was attributed to the predicate detecting more positives, many of which were not confirmed by ELISA.
- Method Comparison of Method A, B, and C:
- Method C (Manual) vs. Method B (Reader Confirmation): Agreements were >85% (Positive: 85.1-91%, Negative: 98.1-98.3%, Overall: 96.7-97.5%), indicating substantial equivalence.
- Method B vs. Method A, and Method C vs. Method A: Acceptance criteria (>85%) were not met for positive agreement (44.8-55.4%) or overall agreement (80.3-85.9%). This was because Method A detected only around 50% of the positive samples found by manual readers. However, diagnostic sensitivities were comparable (around 20-22.6%). Method A had slightly lower specificity (81-88.1%) due to more false positives. Confirmed the need for trained reader confirmation of HELIOS results.
- Endpoint Titer Comparison: Sera titrated on HELIOS and manually resulted in comparable endpoint titers, with maximum 1 titer level difference.
- Normal Range Study: Only 1 out of 164 healthy samples was found positive by one reader and negative by another, showing a low number of false positives in healthy population, correlating with literature.
Key Metrics
- Sensitivity (Diagnostic):
- AESKUSLIDES nDNA: 24.6% (20 - 29.8) for SLE
- Predicate Assay: 37% (31.7 - 42.7) for SLE
- Specificity (Diagnostic):
- AESKUSLIDES nDNA: 94.8% (92.4 - 96.4) for Other Diseases
- Predicate Assay: 85.8% (82.4 - 88.6) for Other Diseases
- Predictive Value of a Positive Test Result (PPV):
- AESKUSLIDES nDNA: 74.5%
- Predicate Assay: 61.8%
- Predictive Value of a Negative Test Result (NPV):
- AESKUSLIDES nDNA: 67.0%
- Predicate Assay: 68.7%
- Agreement Metrics (from reproducibility and method comparison studies):
- Overall Agreement (Method C vs B): 96.7% to 97.5%
- Overall Agreement (Method B vs A): 80.3% to 85.9%
- Overall Agreement (Method C vs A): 80.3% to 85.9%
- Overall Agreement (Method with Predicate): 85.7%
- Negative Agreement (Method with Predicate): 97.8%
Predicate Device(s)
Reference Device(s)
Predetermined Change Control Plan (PCCP) - All Relevant Information
Not Found
§ 866.5100 Antinuclear antibody immunological test system.
(a)
Identification. An antinuclear antibody immunological test system is a device that consists of the reagents used to measure by immunochemical techniques the autoimmune antibodies in serum, other body fluids, and tissues that react with cellular nuclear constituents (molecules present in the nucleus of a cell, such as ribonucleic acid, deoxyribonucleic acid, or nuclear proteins). The measurements aid in the diagnosis of systemic lupus erythematosus (a multisystem autoimmune disease in which antibodies attack the victim's own tissues), hepatitis (a liver disease), rheumatoid arthritis, Sjögren's syndrome (arthritis with inflammation of the eye, eyelid, and salivary glands), and systemic sclerosis (chronic hardening and shrinking of many body tissues).(b)
Classification. Class II (performance standards).
0
Please wait...
If this message is not eventually replaced by the proper contents of the document, your PDF viewer may not be able to display this type of document.
You can upgrade to the latest version of Adobe Reader for Windows®, Mac, or Linux® by visiting http://www.adobe.com/go/reader_download.
For more assistance with Adobe Reader visit http://www.adobe.com/go/acrreader.
Windows is either a registered trademark of Microsoft Corporation in the United States and/or other countries. Mac is a trademark of Apple Inc., registered in the United States and other countries. Linux is the registered trademark of Linus Torvalds in the U.S. and other countries.
1
510(k) SUBSTANTIAL EQUIVALENCE DETERMINATION DECISION MEMORANDUM ASSAY AND INSTRUMENT COMBINATION TEMPLATE
- A. 510(k) Number: K172348
- B. Purpose for Submission: New Device
- C. Measurand: native double helix DNA (dsDNA)
- D. Type of Test: Qualitative and/or semi-quantitative, indirect immunofluorescence
- E. Applicant: AESKU.Diagnostics GmbH & Co. KG
- F. Proprietary and Established Names: AESKUSLIDES® nDNA (Crithidia luciliae)
G. Regulatory Information:
-
Regulation section: §CFR 866.5100 - Antinuclear antibody immunological test system
-
Classification: Class II
-
Product code: LSW, Anti-DNA Antibody, Antigen and Control
-
Clinical use Immunology (82)
2
H. Intended Use:
-
- Intended use(s):
AESKUSLIDES® nDNA (Crithidia luciliae) is an indirect immunofluorescence assay utilizing Crithidia luciliae coated slides as a substrate for the qualitative and/or semi-quantitative determination of antibodies to native double stranded DNA (dsDNA) in human serum. This in vitro diagnostic assay is used as an aid for the diagnosis of Systemic Lupus Erythematosus (SLE) in conjunction with other clinical and laboratory findings. The assay can be processed manually and analyzed at the microscope or processed and analyzed with HELIOS® AUTOMATED IFA SYSTEM. All suggested results obtained with the HELIOS® AUTOMATED IFA SYSTEM must be confirmed by trained personnel.
- Intended use(s):
-
- Indication(s) for use:
Same as intended use
- Indication(s) for use:
-
- Special conditions for use statement(s):
-
- For prescription use only
-
This device is only for use with reagents that are indicated for use with the device.
-
The device is for use by a trained operator in a clinical laboratory setting. 4. All software-aided results must be confirmed by the trained operator. 5.For use only by manual microscopy or with HELIOS® AUTOMATED IFA SYSTEM.
l. Device Description:
AESKUSLIDES® nDNA (Crithidia luciliae) is an indirect immunofluorescence assay utilizing Crithidia luciliae coated slides as a substrate for the qualitative and/or semiquantitative determination of antibodies to native double stranded DNA (dsDNA) in human serum.
Each kit contains (Quantity depends on product variant):
- -Slides, each containing 10 wells coated with Crithidia Luciliae cells
- 4.0 ml vial containing Fluorescein (FITC) labelled Anti-human Antibody lgG conjugate in a solution of BSA, ready for use
- -0.5 ml vial of positive control containing human serum (diluted), ready for Use
- 0.5 ml vial of negative control containing diluted human serum, ready for use -
- -8.0 ml vial of mounting medium containing a solution of glycerol and PBS, ready for use
- 70 ml bottle of sample buffer, containing BSA, PBS and ready for use -
- -100 ml bottle of wash buffer, concentrated buffer 1:10 in distilled water, containing BSA, PBS.
3
Standard Ref. | Description | Tests |
---|---|---|
53.100.US | nDNA (Crithidia luciliae) (10 | |
wells) | 100 | |
53.100.US.Demo | nDNA (Crithidia luciliae) (10 | |
wells) Demo kit | 20 | |
53.100.US.Bulk5 | nDNA (Crithidia luciliae) (10 | |
wells) bulk kit x5 | 500 | |
53.100.US.Bulk10 | nDNA (Crithidia luciliae) (10 | |
wells) bulk kit x10 | 1000 |
Not provided in the kit:
| Ref. | Reagent | Quantity
/ Volume | | Description | Ready
to use |
|-------|-----------------------|----------------------|-----|--------------------------------------------------------------------------------------------------------------------------------|-----------------|
| EBIFA | Evans
Blue
0.2% | 1x | 3ml | Capped white: Blue coloured
solution
Containing: PBS, Evans
Blue.
Dilute the Evans Blue 0.2%
1:3000 in 1x WBIFA | NO |
HELIOS AUTOMATED IFA SYSTEM (K153117) or equivalent manual microscope.
J. Substantial Equivalence Information:
-
- Predicate device name(s): NOVA Lite dsDNA Crithidia luciliae
-
- Predicate 510(k) number(s): K880742
4
July 28, 2017
3. Comparison with predicate device:
| | Item | Predicate
NOVA Lite dsDNA Crithidia luciliae | AESKUSLIDES® nDNA (Crithidia luciliae) |
|--------------|-------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Similarities | Intended Use | NOVA Lite® dsDNA Crithidia luciliae is an
indirect immunofluorescent assay for the
screening and semi-quantitative
determination of anti-double stranded
DNA (dsDNA) in human serum. The
presence of anti-double stranded DNA can
be used in conjunction with other
serological tests and clinical findings to aid
in the diagnosis of systemic lupus
erythematosus (SLE) | AESKUSLIDES® nDNA (Crithidia luciliae) is an
indirect immunofluorescence assay utilizing Crithidia
luciliae coated slides as a substrate for the qualitative
and/or semi-quantitative determination of antibodies to
native double stranded DNA (dsDNA) in human serum.
This in vitro diagnostic assay is used as an aid for the
diagnosis of Systemic Lupus Erythematosus (SLE) in
conjunction with other clinical and laboratory findings.
The assay can be processed manually and analyzed at
the microscope or processed and analyzed with
HELIOS® AUTOMATED IFA SYSTEM. All suggested
results obtained with the HELIOS® AUTOMATED IFA
SYSTEM must be confirmed by trained personnel. |
| | Methology | Immunofluoreszenz assay (IFA) | same as Predicate |
| | Procedure | Standard IFA technique | same as Predicate |
| | Results | qualitative and semi-quantitative titer | same as Predicate |
| | Samples
Matrix | Serum | same as Predicate |
| | Analyte | dsDNA | same as Predicate |
| | Antigen | dsDNA Crithidia luciliae cells | same as Predicate |
| | Fluorescence
Marker | FITC | same as Predicate |
| | Controls | one positive control, one negative control | same as Predicate |
| | conjugate
Screening | anti-human IgG | same as Predicate |
| Differences | Screening
dilution | 1:10 | same as Predicate |
| | Storage | 2 - 8 °C | same as Predicate |
| | Shelf life | 24 months | same as Predicate |
| | Slides | 12 wells coated with antigen | 10 wells coated with antigen |
| | Manual
Interpretation
of result | Manual fluorescense microscopy | Manual fluorescense microscopy or with HELIOS w/
trained operator verification |
| | Automated
interpretation
of results | N/A | HELIOS w/ trained operator verification |
Table 6: comparison with predicate device
5
K. Standard/ Guidance Document Referenced (if applicable):
Table 7: List of Standards / Guidance Documents
# | Standards Title | Version |
---|---|---|
1 | ISO 14971 - Medical Devices - Application of risk management to medical devices | Second Edition |
2 | IEC 62366 - Medical Devices - Part 1: Application Of Usability Engineering To Medical Devices [Including CORRIGENDUM 1 (2016)] | Edition 1.0 |
3 | 15223-1- Medical Devices - Symbols To Be Used With Medical Device Labels, Labelling, and Information to be supplied - Part 1: General Requirements | Second Edition |
4 | Interference Testing in Clinical Chemistry | EP07-A2 |
5 | Evaluation of Detection Capability for Clinical Laboratory Measurement Procedures | EP17-A2 |
6 | Evaluation of Stability of In Vitro Diagnostic Reagents | EP25-A |
7 | Defining, Establishing and Verifying Reference Intervals in the Clinical Laboratory | EP28-A3c |
8 | Evaluation of the Linearity of Quantitative Measurement Procedures A Statistical Approach | EP06-A |
9 | Procedures for the Handling and Processing of Blood Specimens for Common Laboratory Tests; Approved Guideline-Fourth Edition | GP44-A4 |
10 | Guidance for Industry and FDA Staff: Recommendations for Anti-Nuclear Antibody (ANA) Test System Premarket (510k) Submissions | - |
11 | Factors to Consider Regarding Benefit-Risk in Medical Device Product Availability, Compliance, and Enforcement Decisions | - |
12 | Applying Human Factors and Usability Engineering to Medical Devices | - |
L. Test Principle
The AESKUSLIDES nDNA assay utilizes indirect immunofluorescent antibody assay techniques. Patient sera are diluted in wash/sample buffer and applied to a well on the slide. nDNA antibodies, if present, will bind to antigens coated on the slide. After washing with wash/sample buffer, a conjugate specific for human IgG is applied which binds to the nDNA antibodies immobilized on the slide surface. After a final wash to remove excess conjugate, the slide is mounted and read as soon as possible using a fluorescence microscope for manual microscopy or HELIOS AUTOMATED IFA SYSTEM.
Manual interpretation of test results
The fluorescence intensity level is the intensity of the specific fluorescence expressed as a numeric value. These values, if present, are reported as a number between "O" (no specific flurescence) and "4+" (very strong visible reaction).
6
July 28, 2017
Intensity | Interpretation | |
---|---|---|
4+ | high positive | maximal fluorescence, very strong |
visible reaction; brilliant yellow-green | ||
3+ | positive | strong visible reaction; less brilliant as |
4+; yellow-green fluorescence | ||
2+ | positive | moderate visible reaction; definite but |
dull yellow-green fluorescence | ||
1+ | positive | weak visible reaction, very dim |
subdued fluorescence | ||
0 | negative | no specific fluorescence |
AESKU recommends a screening dilution of 1:10, followed by serial dilution for semiquantitative determinations and suggests each laboratory establish its own screening dilution and titration scheme based on its population and instrumentation.
Qualitative evaluation
A serum dilution is considered negative for nDNA antibodies if the cells exhibit 85%.
- -FI is allowed to differ maximum + or - 1 from the expected value.
- -FI Agreement: > 85%.
Results
- All samples fulfilled the above criteria. No differences were observed between sera thawed a single time and sera that had undergone 4 cycles of freeze/thawing:
- -Positive/Negative/Overall Agreement was 100% for AESKUSLIDES nDNA. All positive samples have been found positive, all negative samples have been found negative.
- FI agreement was 100% for AESKUSLIDES nDNA. No deviations of fluorescence intensities greater than +/-1 from the initial value have been observed.
Raw data are in Attachment 10_Section 12_Raw data serum stability
Conclusions
All criteria are fulfilled in each test. The results show that multiple freeze-thaw cycles have no effect on test results.
Long Term Serum Stability AESKUSLIDES nDNA (Crithidia luciliae)
To show that long term storage of serum samples at -20°C has no effect on performance and results of AESKUSLIDES nDNA.
Procedure:
Seven different serum samples (4 positive and 3 negative) have been aliquoted and stored at -20°C for a time period of at least 14 months. Frozen aliquots of the different sera have been thawed and assayed on AESKUSLIDES nDNA at the indicated time points. All tests have been performed manually according to the IFU and subsequently analyzed at the microscope
| Sample
ID | Expected Result | |
---|---|---|
Pos/Neg | FI | |
Sample 1 | Pos | 3 |
Sample 2 | Pos | 2 |
Sample 3 | Pos | 4 |
Sample 4 | Neg | 0 |
Sample 5 | Neg | 0 |
Sample 6 | Neg | 0 |
Sample 7 | Pos | 2 |
12
Acceptance Criteria:
Positive sera have to be found positive and negative sera have to be found negative throughout the whole testing period. The fluorescence intensity (FI) is allowed to differ maximum +/- 1 level from the expected value at each test time point.
Results:
Positive samples have been found positive, negative samples have been found negative at all test time points. FI did not differ more than +/- 1 level from the expected values.
Sample ID | month 0 | month 4 | month 8 | month 10 | month 12 | month 14 | ||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
Pos/Neg | FI | Pos/Neg | FI | Pos/Neg | FI | Pos/Neg | FI | Pos/Neg | FI | Pos/Neg | FI | |
Sample 1 | Pos | 3 | Pos | 3 | Pos | 4 | Pos | 3 | Pos | 3 | Pos | 3 |
Sample 2 | Pos | 2 | Pos | 2 | Pos | 2 | Pos | 2 | Pos | 1 | Pos | 2 |
Sample 3 | Pos | 4 | Pos | 3 | Pos | 4 | Pos | 3 | Pos | 4 | Pos | 3 |
Sample 4 | Neg | 0 | Neg | 0 | Neg | 0 | Neg | 0 | Neg | 0 | Neg | 0 |
Sample 5 | Neg | 0 | Neg | 0 | Neg | 0 | Neg | 0 | Neg | 0 | Neg | 0 |
Sample 6 | Neg | 0 | Neg | 0 | Neg | 0 | Neg | 0 | Neg | 0 | Neg | 0 |
Sample 7 | Pos | 2 | Pos | 2 | Pos | 2 | Pos | 2 | Pos | 2 | Pos | 2 |
Table 10: Results long term serum stability
Conclusion:
All acceptance criteria have been fulfilled for the tested sera. The results show that long term storage of sera at -20°C over a time period of at least 14 months has no effect on performance and test results of AESKUSLIDES nDNA.
a) Precision/Reproducibilty
Within-Lab Precision AESKUSLIDES nDNA (Crithida luciliae)
To assess Within-Lab Precision of AESKUSLIDES nDNA (Crithidia luciliae) processed on HELIOS (Method A - HELIOS suggestions and Method B - Reader Confirmation of HELIOS images) and processed manually (Method C), based on CLSI Guideline EP12-A2.
Sample Set
11 samples were assayed on AESKUSLIDES nDNA. The sample set includes borderline, low, medium, high positive, and negative samples. The samples are characterized in the following table:
Sample ID | Result | Grading |
---|---|---|
1 | Pos | borderline |
2 | Pos | low positive |
3 | Pos | low positive |
4 | Pos | medium positive |
5 | Pos | medium positive |
6 | Pos | medium positive |
Table 11: Sample table for Within-Lab Precision
13
July 28, 2017
7 | Pos | high positive |
---|---|---|
8 | Pos | high positive |
9 | Pos | high positive |
10 | Neg | negative |
11 | Neg | negative |
Procedure
11 samples have been tested on five days, two runs per day, three replicates per sample per run, resulting in 30 data points for each sample.
This Within-lab precision study was done for three different Methods:
| Method A (HELIOS) | Processing of slides and image recording at HELIOS +
HELIOS result suggestion |
|-----------------------------------|-------------------------------------------------------------------------------------------------|
| Method B (Reader
Confirmation) | HELIOS images recorded in Method A + result analysis
by two independent readers |
| Method C (Manual) | Manual processing of slides and result analysis at the
microscope by two independent readers |
All runs have been performed according to the respective IFUs. 1 positive and 1 negative control (kit controls) were included in each run. Samples were tested at 1:10 dilution. Results were analyzed by two independent readers.
Data Analysis
Positive/Negative Classification was recorded for each sample in each run and for each Method. Pos/Neg Classification in Method A (HELIOS) is provided by the nDNA Pattern Plus software tool.
For Method C (Manual performance) fluorescence intensity was also reported.
As an estimate of the imprecision of the three methods A, B and C the percentages of positive and neqative results will be calculated for each sample:
- % Positive (number of positive calls divided by the total number of data points for each sample)
• % Negative (number of negative calls divided by the total number of data points per sample)
Results of the two readers (Method B/C) will be calculated and displayed separately as well as combined.
Acceptance Criteria
· Positive sera have to be found positive, and negative sera have to be found negative*.
· Reported FI is allowed to differ max. ± 1 level within the study
14
- borderline samples are samples with analyte concentration near the cutoff. These samples are very low positive. They can be evaluated also negative in certain cases, depending on the reader, the subjective manner of result analysis and normal test variances.
For Method A we accept lower percentage of positive/negative results for positive and negative samples, respectively, because we state in the IFU that all results generated by the HELIOS (Method A) must be confirmed by trained personnel.
Results
The results are presented in % Positive and % Negative tables of each sample. Percentages were calculated for the two readers separately (30 replicates for each sample) and for both readers combined (60 data points per sample).
Raw Data can be found in 2 Report Within-Lab Precision re-evaluation nDNA
Table 12: Results - Within Lab Precision AESKUSLIDES nDNA - Method A, B, and C number of positive and negative results
Sample ID | N | Method A | Method B | Method C | |||||||
---|---|---|---|---|---|---|---|---|---|---|---|
n Neg | n Pos | n Neg | n Pos | n Neg | n Pos | ||||||
Reader 1 | Reader 2 | Reader 1 | Reader 2 | Reader 1 | Reader 2 | Reader 1 | Reader 2 | ||||
S1 | 30 | 30 | 0 | 30 | 6 | 0 | 24 | 4 | 5 | 26 | 25 |
S2 | 30 | 10 | 20 | 0 | 0 | 30 | 30 | 0 | 0 | 30 | 30 |
S3 | 30 | 17 | 13 | 0 | 3 | 30 | 27 | 0 | 0 | 30 | 30 |
S4 | 30 | 14 | 16 | 4 | 1 | 26 | 29 | 0 | 0 | 30 | 30 |
S5 | 30 | 0 | 30 | 0 | 0 | 30 | 30 | 0 | 0 | 30 | 30 |
S6 | 30 | 2 | 28 | 0 | 0 | 30 | 30 | 0 | 0 | 30 | 30 |
S7 | 30 | 0 | 30 | 0 | 0 | 30 | 30 | 0 | 0 | 30 | 30 |
S8 | 30 | 0 | 30 | 0 | 0 | 30 | 30 | 0 | 0 | 30 | 30 |
S9 | 30 | 1 | 29 | 0 | 0 | 30 | 30 | 0 | 0 | 30 | 30 |
S10 | 30 | 25 | 5 | 30 | 30 | 0 | 0 | 29 | 30 | 1 | 0 |
S11 | 30 | 21 | 9 | 30 | 30 | 0 | 0 | 30 | 30 | 0 | 0 |
15
Sample ID | N | Method A | Method B | Method C | |||||||
---|---|---|---|---|---|---|---|---|---|---|---|
% Negative | % Positive | % Negative | % Positive | % Negative | % Positive | ||||||
Reader 1 | Reader 2 | Reader 1 | Reader 2 | Reader 1 | Reader 2 | Reader 1 | Reader 2 | ||||
S1 | 30 | 100 | 0 | 100 | 20.0 | 0 | 80.0 | 13.3 | 16.7 | 86.7 | 83.3 |
S2 | 30 | 33.3 | 66.7 | 0 | 0 | 100 | 100 | 0 | 0 | 100 | 100 |
S3 | 30 | 56.7 | 43.3 | 0 | 10 | 100 | 90 | 0 | 0 | 100 | 100 |
S4 | 30 | 46.7 | 53.3 | 13.3 | 3.3 | 86.7 | 96.7 | 0 | 0 | 100 | 100 |
S5 | 30 | 0 | 100 | 0 | 0 | 100 | 100 | 0 | 0 | 100 | 100 |
S6 | 30 | 6.7 | 93.3 | 0 | 0 | 100 | 100 | 0 | 0 | 100 | 100 |
S7 | 30 | 0 | 100 | 0 | 0 | 100 | 100 | 0 | 0 | 100 | 100 |
S8 | 30 | 0 | 100 | 0 | 0 | 100 | 100 | 0 | 0 | 100 | 100 |
S9 | 30 | 3.3 | 96.7 | 0 | 0 | 100 | 100 | 0 | 0 | 100 | 100 |
S10 | 30 | 83.3 | 16.7 | 100 | 100 | 0 | 0 | 96.7 | 100 | 3.3 | 0 |
S11 | 30 | 70.0 | 30.0 | 100 | 100 | 0 | 0 | 100 | 100 | 0 | 0 |
Table 13: Results – Within Lab Precision AESKUSLIDES nDNA – Method A, B and C – percentages of positive and negative results
Table 14: Results – Within Lab Precision AESKUSLIDES nDNA – Method A, B and C – percentages of positive and negative results for both readers combined
| Sample ID | N
(Method A) | Method A | | N
(Method B/C) | Method B | | Method C | |
|-----------|-----------------|---------------|---------------|-------------------|---------------|---------------|---------------|---------------|
| | | %
Negative | %
Positive | | %
Negative | %
Positive | %
Negative | %
Positive |
| S1 | 30 | 100 | 0 | 60 | 60.0 | 40.0 | 15.0 | 85.0 |
| S2 | 30 | 33.3 | 66.7 | 60 | 0 | 100 | 0 | 100 |
| S3 | 30 | 56.7 | 43.3 | 60 | 5.0 | 95.0 | 0 | 100 |
| S4 | 30 | 46.7 | 53.3 | 60 | 8.3 | 91.7 | 0 | 100 |
| S5 | 30 | 0 | 100 | 60 | 0 | 100 | 0 | 100 |
| S6 | 30 | 6.7 | 93.3 | 60 | 0 | 100 | 0 | 100 |
| S7 | 30 | 0 | 100 | 60 | 0 | 100 | 0 | 100 |
| S8 | 30 | 0 | 100 | 60 | 0 | 100 | 0 | 100 |
| S9 | 30 | 3.3 | 96.7 | 60 | 0 | 100 | 0 | 100 |
| S10 | 30 | 83.3 | 16.7 | 60 | 100 | 0 | 98.3 | 1.7 |
| S11 | 30 | 70.0 | 30.0 | 60 | 100 | 0 | 100 | 0 |
For manual testing (Method C) fluorescence intensities differed at maximum ±1 level within one run and between the runs.
Conclusion
AESKUSLIDES nDNA (Crithidia luciliae)
510(k) Traditional Submission
All pre-determined acceptance criteria were met.
16
Between-Lab Precision AESKUSLIDES nDNA (Crithida luciliae)
To assess Between-Lab Precision of AESKUSLIDES nDNA (Crithidia luciliae) processed on HELIOS (Method A - HELIOS suggestions and Method B - Reader Confirmation of HELIOS images) and processed manually (Method C).
Sample Set
11 samples were assayed on AESKUSLIDES nDNA. The sample set includes borderline, low, medium, high positive, and negative samples. The samples are characterized in the following table:
Sample ID | AESKUSLIDES nDNA | ||
---|---|---|---|
Result | Grading | Fluorescence Intensity (FI) | |
1 | Pos | borderline positive | 1 |
2 | Pos | low positive | 2 |
3 | Pos | borderline positive | 1 |
4 | Pos | medium positive | 3 |
5 | Pos | medium positive | 3 |
6 | Pos | medium positive | 3 |
7 | Pos | high positive | 4 |
8 | Pos | high positive | 4 |
9 | Pos | high positive | 4 |
10 | Neg | negative | 0 |
11 | Neg | negative | 0 |
Table 15: Sample table for Precision study |
---|
Procedure
11 samples have been tested on five days, two runs per day, three replicates per sample per run, on three different study sites. Two study sites were in the US, one study site was in Germany, with one HELIOS device per study site. Results were analyzed by two different readers per study site, resulting in a total of 90 data points per sample per reader.
This Between-lab precision study was done for three different Methods:
| Method A (HELIOS) | Processing of slides and image recording at HELIOS +
HELIOS result suggestion |
|-----------------------------------|-------------------------------------------------------------------------------------------------|
| Method B (Reader
Confirmation) | HELIOS images recorded in Method A + result analysis
by two independent readers |
| Method C (Manual) | Manual processing of slides and result analysis at the
microscope by two independent readers |
All runs have been performed according to the respective IFUs. 1 positive and 1 negative control (kit controls) were included in each run. Samples were tested at 1:10 dilution. Results were analyzed by two independent readers.
17
Data Analysis
Positive/Negative Classification was recorded for each sample in each run and for each Method. Pos/Neg Classification in Method A (HELIOS) is provided by the nDNA Pattern Plus software tool.
For Method C (Manual performance) fluorescence intensity was also reported.
The following % Agreements including 95% CI (confidence intervals) will be calculated for each Method A, B and C:
- Positive % Agreement (across all positive samples: number of correctly found samples divided through number of total positive samples)
- -Negative % Agreement (across all negative samples: number of correctly found samples divided through number of total negative samples)
- -Overall % Agreement (across all positive and negative samples: number of correctly found samples divided through number of total samples)
- -Fluorescence Intensity % Agreement (across all samples, only for Method C: number of correctly found samples divided through number of total samples)
Results of the two readers will be calculated separately as well as combined.
Acceptance Criteria
- -Positive sera have to be found positive, and negative sera have to be found negative.
- Reported FI is allowed to differ max. ± 1 level from the expected value. -
- -Agreements should meet the following criteria:
Table 16: Acceptance criteria for between-lab precision study
| Type of Agreement (borderline positive
samples excluded*) | Method A | Method B | Method C |
---|---|---|---|
Positive Agreement | > 70% | > 90% | > 90% |
Negative Agreement | > 70% | > 90% | > 90% |
Overall Agreement | > 70% | > 90% | > 90% |
Fluorescence Intensity Agreement | > 90% |
- borderline samples are very low positive samples that can be evaluated also negative in certain cases, depending on the reader, the subjective manner of result analysis and normal test variances.
For Method A we accept a lower agreement (>70%), because we state in the IFU that all results generated by the HELIOS (Method A) must be confirmed by trained personnel.
From the results of those study, the following will be calculated:
- Between-Lab Precisions, 2. Between-Operator Precisions, 3. Single-Operator Precisions, and 4. Instrument Precision
"The e-Copy is an exact duplicate of the paper copy"
18
Results
Results are presented in the tables below. Agreements were calculated 1. including the results of the two borderline positive samples, 2. excluding the results of those two samples.
All predetermined acceptance criteria were met.
Between-Site Agreement
Manual (Method C) Between-Site Agreement (Calculation including borderline samples):
Table 17: Between-Site Agreement (Calculation including borderline samples) Number of Correctly/Incorrectly found samples Method C
| Number of
Samples | Site 1 vs Site 2 | Site 2 vs Site 3 | Site 1 vs Site 3 | All Sites | ||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
Found | ||||||||||||
Correctly | Found | |||||||||||
Incorrectly | Expected | |||||||||||
Numbers | Found | |||||||||||
Correctly | Found | |||||||||||
Incorrectly | Expected | |||||||||||
Numbers | Found | |||||||||||
Correctly | Found | |||||||||||
Incorrectly | Expected | |||||||||||
Numbers | Found | |||||||||||
Correctly | Found | |||||||||||
Incorrectly | Expected | |||||||||||
Numbers | ||||||||||||
Positive | ||||||||||||
Agreement | 1071 | 9 | 1080 | 1069 | 11 | 1080 | 1060 | 20 | 1080 | 1600 | 20 | 1620 |
Negative | ||||||||||||
Agreement | 239 | 1 | 240 | 240 | 0 | 240 | 239 | 1 | 240 | 359 | 1 | 360 |
Overall | ||||||||||||
Agreement | 1310 | 10 | 1320 | 1309 | 11 | 1320 | 1299 | 21 | 1320 | 1959 | 21 | 1980 |
FI Agreement | 1303 | 17 | 1320 | 1289 | 31 | 1320 | 1286 | 34 | 1320 | 1939 | 41 | 1980 |
Table 18: Between-Site Agreement (Calculation including borderline samples) % Agreements (95% CI) Method C
| % Agreement
(95% CI) | Site 1 vs Site 2 | Site 2 vs Site 3 | Site 1 vs Site 3 | All Sites |
---|---|---|---|---|
Positive | ||||
Agreement | 99.2 | |||
(98.4 - 99.6) | 99 | |||
(98.2 - 99.4) | 98.1 | |||
(97.2 - 98.8) | 98.8 | |||
(98.1 - 99.2) | ||||
Negative | ||||
Agreement | 99.6 | |||
(97.7 - 99.9) | 100 | |||
(98.4 - 100) | 99.6 | |||
(97.7 - 99.9) | 99.7 | |||
(98.4 - 100) | ||||
Overall | ||||
Agreement | 99.2 | |||
(98.6 - 99.6) | 99.2 | |||
(98.5 - 99.5) | 98.4 | |||
(97.6 - 99) | 98.9 | |||
(98.4 - 99.3) | ||||
FI Agreement | 98.7 | |||
(97.9 - 99.2) | 97.7 | |||
(96.7 - 98.3) | 97.4 | |||
(96.4 - 98.2) | 97.9 | |||
(97.2 - 98.5) |
Manual (Method C) Between-Site Agreement (Calculation excluding borderline samples):
19
Table 19: Between-Site Agreement (Calculation excluding borderline samples) Number of Correctly/Incorrectly found samples Method C
| Number of
Samples | Site 1 vs Site 2 | Site 2 vs Site 3 | Site 1 vs Site 3 | All Sites | ||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
Found | ||||||||||||
Correctly | Found | |||||||||||
Incorrectly | Expected | |||||||||||
Numbers | Found | |||||||||||
Correctly | Found | |||||||||||
Incorrectly | Expected | |||||||||||
Numbers | Found | |||||||||||
Correctly | Found | |||||||||||
Incorrectly | Expected | |||||||||||
Numbers | Found | |||||||||||
Correctly | Found | |||||||||||
Incorrectly | Expected | |||||||||||
Numbers | ||||||||||||
Positive | ||||||||||||
Agreement | 840 | 0 | 840 | 834 | 6 | 840 | 834 | 6 | 840 | 1254 | 6 | 1260 |
Negative | ||||||||||||
Agreement | 239 | 1 | 240 | 240 | 0 | 240 | 239 | 1 | 240 | 359 | 1 | 360 |
Overall | ||||||||||||
Agreement | 1079 | 1 | 1080 | 1074 | 6 | 1080 | 1073 | 7 | 1080 | 1613 | 7 | 1620 |
FI Agreement | 1063 | 17 | 1080 | 1049 | 31 | 1080 | 1046 | 34 | 1080 | 1579 | 41 | 1620 |
Table 20: Between-Site Agreement (Calculation excluding borderline samples) % Agreements (95% Cl) Method C
| % Agreement
(95% CI) | Site 1 vs Site
2 | Site 2 vs Site
3 | Site 1 vs Site
3 | All Sites |
|-------------------------|-----------------------|-----------------------|-----------------------|-----------------------|
| Positive
Agreement | 100
(99.5 - 100) | 99.3
(98.5 - 99.7) | 99.3
(98.5 - 99.7) | 99.5
(99 - 99.8) |
| Negative
Agreement | 99.6
(97.7 - 99.9) | 100
(98.4 - 100) | 99.6
(97.7 - 99.9) | 99.7
(98.4 - 100) |
| Overall
Agreement | 99.9
(99.5 - 100) | 99.4
(98.8 - 99.7) | 99.4
(98.7 - 99.7) | 99.6
(99.1 - 99.8) |
| FI Agreement | 98.4
(97.5 - 99) | 97.1
(96 - 98) | 96.9
(95.6 - 97.7) | 97.5
(96.6 - 98.1) |
20
July 26, 201
Reader Confirmation (Method B) Between-Site Agreement (Calculation including borderline samples):
Table 21: Between-Site Agreement (Calculation including borderline samples) Number of Correctly/Incorrectly found samples Method B
| Number of
Samples | Site 1 vs Site 2 | Site 2 vs Site 3 | Site 1 vs Site 3 | All Sites | ||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
Found | ||||||||||||
Correctly | Found | |||||||||||
Incorrectly | Expected | |||||||||||
Numbers | Found | |||||||||||
Correctly | Found | |||||||||||
Incorrectly | Expected | |||||||||||
Numbers | Found | |||||||||||
Correctly | Found | |||||||||||
Incorrectly | Expected | |||||||||||
Numbers | Found | |||||||||||
Correctly | Found | |||||||||||
Incorrectly | Expected | |||||||||||
Numbers | ||||||||||||
Positive | ||||||||||||
Agreement | 995 | 85 | 1080 | 912 | 168 | 1080 | 909 | 171 | 1080 | 1408 | 212 | 1620 |
Negative | ||||||||||||
Agreement | 238 | 2 | 240 | 238 | 2 | 240 | 240 | 0 | 240 | 358 | 2 | 360 |
Overall | ||||||||||||
Agreement | 1233 | 87 | 1320 | 1150 | 170 | 1320 | 1149 | 171 | 1320 | 1766 | 214 | 1980 |
FI Agreement | na | na | na | na | na | na | na | na | na | na | na |
Table 22: Between-Site Agreement (Calculation including borderline samples) % Agreements (95% Cl) Method B
| % Agreement
(95% CI) | Site 1 vs Site 2 | Site 2 vs Site 3 | Site 1 vs Site 3 | All Sites |
---|---|---|---|---|
Positive | ||||
Agreement | 92.1 | |||
(90.4 - 93.6) | 84.4 | |||
(82.2 - 86.5) | 84.2 | |||
(81.9 - 86.2) | 86.9 | |||
(85.2 - 88.5) | ||||
Negative | ||||
Agreement | 99.2 | |||
(97 - 99.8) | 99.2 | |||
(97 - 99.8) | 100 | |||
(98.4 - 100) | 99.4 | |||
(98 - 99.8) | ||||
Overall | ||||
Agreement | 93.4 | |||
(91.9 - 94.6) | 87.1 | |||
(85.2 - 88.8) | 87 | |||
(85.1 - 88.8) | 89.2 | |||
(87.7 - 90.5) | ||||
FI Agreement | na | na | na | na |
21
Reader Confirmation (Method B) Between-Site Agreement (Calculation excluding borderline samples):
Table 23: Between-Site Agreement (Calculation excluding borderline samples) Number of Correctly/Incorrectly found samples Method B
| Number of
Samples | Site 1 vs Site 2 | Site 2 vs Site 3 | Site 1 vs Site 3 | All Sites | ||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
Found | ||||||||||||
Correctly | Found | |||||||||||
Incorrectly | Expected | |||||||||||
Numbers | Found | |||||||||||
Correctly | Found | |||||||||||
Incorrectly | Expected | |||||||||||
Numbers | Found | |||||||||||
Correctly | Found | |||||||||||
Incorrectly | Expected | |||||||||||
Numbers | Found | |||||||||||
Correctly | Found | |||||||||||
Incorrectly | Expected | |||||||||||
Numbers | ||||||||||||
Positive | ||||||||||||
Agreement | 833 | 7 | 840 | 818 | 22 | 840 | 815 | 25 | 840 | 1233 | 27 | 1260 |
Negative | ||||||||||||
Agreement | 238 | 2 | 240 | 238 | 2 | 240 | 240 | 0 | 240 | 358 | 2 | 360 |
Overall | ||||||||||||
Agreement | 1071 | 9 | 1080 | 1056 | 24 | 1080 | 1055 | 25 | 1080 | 1591 | 29 | 1620 |
FI Agreement | na | na | na | na | na | na | na | na | na | na | na | na |
Table 24: Between-Site Agreement (Calculation excluding borderline samples) % Agreements (95% Cl) Method B
| % Agreement
(95% CI) | Site 1 vs Site 2 | Site 2 vs Site 3 | Site 1 vs Site 3 | All Sites |
---|---|---|---|---|
Positive | ||||
Agreement | 99.2 | |||
(98.3 - 99.6) | 97.4 | |||
(96.1 - 98.3) | 97 | |||
(95.6 - 98) | 97.9 | |||
(96.9 - 98.5) | ||||
Negative | ||||
Agreement | 99.2 | |||
(97 - 99.8) | 99.2 | |||
(97 - 99.8) | 100 | |||
(98.4 - 100) | 99.4 | |||
(98 - 99.8) | ||||
Overall | ||||
Agreement | 99.2 | |||
(98.4 - 99.6) | 97.8 | |||
(96.7 - 98.5) | 97.7 | |||
(96.6 - 98.4) | 98.2 | |||
(97.4 - 98.8) | ||||
FI Agreement | na | na | na | na |
22
July 28, 2017
HELIOS (Method A) Between-Site Agreement (Calclation including borderline samples):
Table 25: Between-Site Agreement (Calculation including borderline samples) Number of Correctly/Incorrectly found samples Method A
| Number of
Samples | Site 1 vs Site 2 | Site 2 vs Site 3 | Site 1 vs Site 3 | All Sites | ||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
Found | ||||||||||||
Correctly | Found | |||||||||||
Incorrectly | Expected | |||||||||||
Numbers | Found | |||||||||||
Correctly | Found | |||||||||||
Incorrectly | Expected | |||||||||||
Numbers | Found | |||||||||||
Correctly | Found | |||||||||||
Incorrectly | Expected | |||||||||||
Numbers | Found | |||||||||||
Correctly | Found | |||||||||||
Incorrectly | Expected | |||||||||||
Numbers | ||||||||||||
Positive | ||||||||||||
Agreement | 341 | 199 | 540 | 282 | 258 | 540 | 321 | 219 | 540 | 472 | 338 | 810 |
Negative | ||||||||||||
Agreement | 105 | 15 | 120 | 114 | 6 | 120 | 105 | 15 | 120 | 162 | 18 | 180 |
Overall | ||||||||||||
Agreement | 446 | 214 | 660 | 396 | 264 | 660 | 426 | 234 | 660 | 634 | 356 | 990 |
FI Agreement | na | na | na | na | na | na | na | na | na | na | na |
Table 26: Between-Site Agreement (Calculation including borderline samples) % Agreements (95% Cl) Method A
| % Agreement
(95% CI) | Site 1 vs Site
2 | Site 2 vs Site
3 | Site 1 vs Site
3 | All Sites |
|-------------------------|-----------------------|---------------------|-----------------------|-----------------------|
| Positive
Agreement | 63.1
(59 - 67.1) | 52.2
(48 - 56.4) | 59.4
(55.3 - 63.5) | 58.3
(54.8 - 61.6) |
| Negative
Agreement | 87.5
(80.4 - 92.3) | 95
(89.5 - 97.7) | 87.5
(80.4 - 92.3) | 90
(84.7 - 93.6) |
| Overall
Agreement | 67.6
(63.9 - 71) | 60
(56.2 - 63.7) | 64.5
(60.8 - 68.1) | 64
(61 - 67) |
| FI Agreement | na | na | na | na |
23
HELIOS (Method A) Between-Site Agreement (Calclation excluding borderline samples):
Table 27: Between-Site Agreement (Calculation excluding borderline samples) Number of Correctly/Incorrectly found samples Method A
Site 1 vs Site 2 | Site 2 vs Site 3 | Site 1 vs Site 3 | All Sites | |||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
Number of | ||||||||||||
Samples | Found | |||||||||||
Correctly | Found | |||||||||||
Incorrectly | Expected | |||||||||||
Numbers | Found | |||||||||||
Correctly | Found | |||||||||||
Incorrectly | Expected | |||||||||||
Numbers | Found | |||||||||||
Correctly | Found | |||||||||||
Incorrectly | Expected | |||||||||||
Numbers | Found | |||||||||||
Correctly | Found | |||||||||||
Incorrectly | Expected | |||||||||||
Numbers | ||||||||||||
Positive | ||||||||||||
Agreement | 328 | 92 | 420 | 278 | 142 | 420 | 310 | 110 | 420 | 458 | 172 | 630 |
Negative | ||||||||||||
Agreement | 105 | 15 | 120 | 114 | 6 | 120 | 105 | 15 | 120 | 162 | 18 | 180 |
Overall | ||||||||||||
Agreement | 433 | 107 | 540 | 392 | 148 | 540 | 415 | 125 | 540 | 620 | 190 | 810 |
FI Agreement | na | na | na | na | na | na | na | na | na | na | na | na |
Table 28: Between-Site Agreement (Calculation excluding borderline samples) % Agreements (95% Cl) Method A
% Agreement (95% CI) | Site 1 vs Site 2 | Site 2 vs Site 3 | Site 1 vs Site 3 | All Sites |
---|---|---|---|---|
Positive Agreement | 78.1 (73.9 - 81.8) | 66.2 (61.5 - 70.5) | 73.8 (69.4 - 77.8) | 72.7 (69.1 - 76) |
Negative Agreement | 87.5 (80.4 - 92.3) | 95 (89.5 - 97.7) | 87.5 (80.4 - 92.3) | 90 (84.7 - 93.6) |
Overall Agreement | 80.2 (76.6 - 83.3) | 72.6 (68.7 - 76.2) | 76.9 (73.1 - 80.2) | 76.5 (73.5 - 79.3) |
FI Agreement | na | na | na | na |
All acceptance critieria were met for Method C (Manual) and Method B (Reader Confirmation). All agreements were >90%. For Method A (HELIOS suggestion) acceptance criteria were met for negative and overall agreement for all study sites (>70%). Positive agreement for Method A was also >70% for site to site precision of site 1 vs site 2 and site 1 vs 3, but not for site 2 vs site 3, where a slightly lower agreement could be observed (66.2%). However, positive agreement met acceptance criteria when calculated across all three sites (72.7%).
Overall-Between-Lab agreements ranged from 72.6% to 80.2% for Method A (HELIOS), from 97.7% to 99.2% for Method B (Reader Confirmation), and from 96.9% to 98.4% for Method C (Manual).
24
Between-Operator Agreement
Manual (Method C) Between-Operator Agreement (Calculation including borderline samples):
Table 29: Between-Operator Agreement (Calculation including borderline samples) Number of Correctly/Incorrectly found samples Method C
Site 1 | Site 2 | Site 3 | |||||||
---|---|---|---|---|---|---|---|---|---|
Number of Samples | Found Correctly | Found Incorrectly | Expected Numbers | Found Correctly | Found Incorrectly | Expected Numbers | Found Correctly | Found Incorrectly | Expected Numbers |
Positive Agreement | 531 | 9 | 540 | 540 | 0 | 540 | 529 | 11 | 540 |
Negative Agreement | 119 | 1 | 120 | 120 | 0 | 120 | 120 | 0 | 120 |
Overall Agreement | 650 | 10 | 660 | 660 | 0 | 660 | 649 | 11 | 660 |
FI Agreement | 650 | 10 | 660 | 653 | 7 | 660 | 636 | 24 | 660 |
Table 30: Between-Operator Agreement (Calculation including borderline samples) % Agreements (95% CI) Method C
| % Agreement
(95% CI) | Site 1 | Site 2 | Site 3 |
---|---|---|---|
Positive | |||
Agreement | 98.3 | ||
(96.9 - 99.1) | 100 | ||
(99.3 - 100) | 98 | ||
(96.4 - 98.9) | |||
Negative | |||
Agreement | 99.2 | ||
(95.4 - 99.9) | 100 | ||
(96.9 - 100) | 100 | ||
(96.9 - 100) | |||
Overall | |||
Agreement | 98.5 | ||
(97.2 - 99.2) | 100 | ||
(99.4 - 100) | 98.3 | ||
(97 - 99.1) | |||
FI Agreement | 98.5 | ||
(97.2 - 99.2) | 98.9 | ||
(97.8 - 99.5) | 96.4 | ||
(94.6 - 97.5) |
25
Manual (Method C) Between-Operator Agreement (Calculation excluding borderline samples):
Table 31: Between-Operator Agreement (Calculation excluding borderline samples) Number of Correctly/Incorrectly found samples Method C
Site 1 | Site 2 | Site 3 | |||||||
---|---|---|---|---|---|---|---|---|---|
Number of | |||||||||
Samples | Found | ||||||||
Correctly | Found | ||||||||
Incorrectly | Expected | ||||||||
Numbers | Found | ||||||||
Correctly | Found | ||||||||
Incorrectly | Expected | ||||||||
Numbers | Found | ||||||||
Correctly | Found | ||||||||
Incorrectly | Expected | ||||||||
Numbers | |||||||||
Positive | |||||||||
Agreement | 420 | 0 | 420 | 420 | 0 | 420 | 414 | 6 | 420 |
Negative | |||||||||
Agreement | 119 | 1 | 120 | 120 | 0 | 120 | 120 | 0 | 120 |
Overall | |||||||||
Agreement | 539 | 1 | 540 | 540 | 0 | 540 | 534 | 6 | 540 |
FI Agreement | 530 | 10 | 540 | 533 | 7 | 540 | 516 | 24 | 540 |
Table 32: Between-Operator Agreement (Calculation excluding borderline samples) % Agreements (95% Cl) Method C
| % Agreement
(95% CI) | Site 1 | Site 2 | Site 3 |
---|---|---|---|
Positive Agreement | 100 | ||
(99.1 - 100) | 100 | ||
(99.1 - 100) | 98.6 | ||
(96.9 - 99.3) | |||
Negative Agreement | 99.2 | ||
(95.4 - 99.9) | 100 | ||
(96.9 - 100) | 100 | ||
(96.9 - 100) | |||
Overall Agreement | 99.8 | ||
(99 - 100) | 100 | ||
(99.3 - 100) | 98.9 | ||
(97.6 - 99.5) | |||
FI Agreement | 98.1 | ||
(96.6 - 99) | 98.7 | ||
(97.3 - 99.4) | 95.6 | ||
(93.5 - 97) |
26
borderline samples):
Reader Confirmation (Method B) Between-Operator Agreement (Calculation including
Table 33: Between-Operator Agreement (Calculation including borderline samples) Number of Correctly/Incorrectly found samples Method B
| Number of
Samples | Site 1 | Site 2 | Site 3 | ||||||
---|---|---|---|---|---|---|---|---|---|
Found | |||||||||
Correctly | Found | ||||||||
Incorrectly | Expected | ||||||||
Numbers | Found | ||||||||
Correctly | Found | ||||||||
Incorrectly | Expected | ||||||||
Numbers | Found | ||||||||
Correctly | Found | ||||||||
Incorrectly | Expected | ||||||||
Numbers | |||||||||
Positive | |||||||||
Agreement | 496 | 44 | 540 | 499 | 41 | 540 | 413 | 127 | 540 |
Negative | |||||||||
Agreement | 120 | 0 | 120 | 118 | 2 | 120 | 120 | 0 | 120 |
Overall | |||||||||
Agreement | 616 | 44 | 660 | 617 | 43 | 660 | 533 | 127 | 660 |
FI Agreement | na | na | na | na | na | na | na | na | na |
Table 34: Between-Operator Agreement (Calculation including borderline samples) % Agreements (95% CI) Method B
% Agreement (95% CI) | Site 1 | Site 2 | Site 3 |
---|---|---|---|
Positive Agreement | 91.9 | ||
(89.2 - 93.9) | 92.4 | ||
(89.9 - 94.4) | 76.5 | ||
(72.7 - 79.9) | |||
Negative Agreement | 100 | ||
(96.9 - 100) | 98.3 | ||
(94.1 - 99.5) | 100 | ||
(96.9 - 100) | |||
Overall Agreement | 93.3 | ||
(91.2 - 95) | 93.5 | ||
(91.3 - 95.1) | 80.8 | ||
(77.6 - 83.6) | |||
FI Agreement | na | na | na |
27
Reader Confirmation (Method B) Between-Operator Agreement (Calculation excluding borderline samples):
Table 35: Between-Operator Agreement (Calculation excluding borderline samples) Number of Correctly/Incorrectly found samples Method B
| Number of
Samples | Site 1 | Site 2 | Site 3 | ||||||
---|---|---|---|---|---|---|---|---|---|
Found | |||||||||
Correctly | Found | ||||||||
Incorrectly | Expected | ||||||||
Numbers | Found | ||||||||
Correctly | Found | ||||||||
Incorrectly | Expected | ||||||||
Numbers | Found | ||||||||
Correctly | Found | ||||||||
Incorrectly | Expected | ||||||||
Numbers | |||||||||
Positive | |||||||||
Agreement | 415 | 5 | 420 | 418 | 2 | 420 | 400 | 20 | 420 |
Negative | |||||||||
Agreement | 120 | 0 | 120 | 118 | 2 | 120 | 120 | 0 | 120 |
Overall | |||||||||
Agreement | 535 | 5 | 540 | 536 | 4 | 540 | 520 | 20 | 540 |
FI Agreement | na | na | na | na | na | na | na | na |
Table 36: Between-Operator Agreement (Calculation excluding borderline samples) % Agreements (95% CI) Method B
% Agreement (95% CI) | Site 1 | Site 2 | Site 3 |
---|---|---|---|
Positive Agreement | 98.8 | ||
(97.2 - 99.5) | 99.5 | ||
(98.3 - 99.9) | 95.2 | ||
(92.8 - 96.9) | |||
Negative Agreement | 100 | ||
(96.9 - 100) | 98.3 | ||
(94.1 - 99.5) | 100 | ||
(96.9 - 100) | |||
Overall Agreement | 99.1 | ||
(97.9 - 99.6) | 99.3 | ||
(98.1 - 99.7) | 96.3 | ||
(94.3 - 97.6) | |||
FI Agreement | na | na | na |
Overall-Between-Operator agreements ranged from 96.3% to 99.3% for Method B (Reader Confirmation), and from 98.9% to 100% for Method C (Manual).
28
Single-Operator Agreement
Manual (Method C) Single-Operator Agreement (Calculation including borderline samples):
Table 37: Single-Operator Agreement (Calculation including borderline samples) Number of Correctly/Incorrectly found samples Method C
Site 1 | Site 2 | Site 3 | ||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Number of Samples | Reader 1 | Reader 2 | Reader 3 | Reader 4 | Reader 5 | Reader 6 | ||||||||||||
Found Correctly | Found Incorrectly | Expected Numbers | Found Correctly | Found Incorrectly | Expected Numbers | Found Correctly | Found Incorrectly | Expected Numbers | Found Correctly | Found Incorrectly | Expected Numbers | Found Correctly | Found Incorrectly | Expected Numbers | Found Correctly | Found Incorrectly | Expected Numbers | |
Positive Agreement | 266 | 4 | 270 | 265 | 5 | 270 | 270 | 0 | 270 | 270 | 0 | 270 | 266 | 4 | 270 | 263 | 7 | 270 |
Negative Agreement | 59 | 1 | 60 | 60 | 0 | 60 | 60 | 0 | 60 | 60 | 0 | 60 | 60 | 0 | 60 | 60 | 0 | 60 |
Overall Agreement | 325 | 5 | 330 | 325 | 5 | 330 | 330 | 0 | 330 | 330 | 0 | 330 | 326 | 4 | 330 | 323 | 7 | 330 |
FI Agreement | 329 | 1 | 330 | 321 | 9 | 330 | 329 | 1 | 330 | 324 | 6 | 330 | 322 | 8 | 330 | 314 | 16 | 330 |
Table 38: Single-Operator Agreement (Calculation including borderline samples) % Agreements (95% CI) Method C
| % Agreement
(95% CI) | Site 1 | Site 2 | Site 3 | |||
---|---|---|---|---|---|---|
Reader 1 | Reader 2 | Reader 3 | Reader 4 | Reader 5 | Reader 6 | |
Positive | ||||||
Agreement | 98.5 | |||||
(96.3 - 99.4) | 98.1 | |||||
(95.7 - 99.2) | 100 | |||||
(98.6 - 100) | 100 | |||||
(98.6 - 100) | 98.5 | |||||
(96.3 - 99.4) | 97.4 | |||||
(94.7 - 98.7) | ||||||
Negative | ||||||
Agreement | 98.3 | |||||
(91.1 - 99.7) | 100 | |||||
(94 - 100) | 100 | |||||
(94 - 100) | 100 | |||||
(94 - 100) | 100 | |||||
(94 - 100) | 100 | |||||
(94 - 100) | ||||||
Overall | ||||||
Agreement | 98.5 | |||||
(96.5 - 99.4) | 98.5 | |||||
(96.5 - 99.4) | 100 | |||||
(98.8 - 100) | 100 | |||||
(98.8 - 100) | 98.8 | |||||
(96.9 - 99.5) | 97.9 | |||||
(95.7 - 99) | ||||||
FI Agreement | 99.7 | |||||
(98.3 - 99.9) | 97.3 | |||||
(94.9 - 98.6) | 99.7 | |||||
(98.3 - 99.9) | 98.2 | |||||
(96.1 - 99.2) | 97.6 | |||||
(95.3 - 98.8) | 95.2 | |||||
(92.3 - 97) |
29
Manual (Method C) Single-Operator Agreement (Calculation excluding borderline samples):
Table 39: Single-Operator Agreement (Calculation excluding borderline samples) Number of Correctly/Incorrectly found samples Method C
Site 1 | Site 2 | Site 3 | |||||||
---|---|---|---|---|---|---|---|---|---|
Number of | |||||||||
Samples | Reader 1 | Reader 2 | Reader 3 | Reader 4 | Reader 5 | Reader 6 | |||
Found | |||||||||
Correctly | Found | ||||||||
Incorrectly | Expected | ||||||||
Numbers | Found | ||||||||
Correctly | Found | ||||||||
Incorrectly | Expected | ||||||||
Numbers | Found | ||||||||
Correctly | Found | ||||||||
Incorrectly | Expected | ||||||||
Numbers | |||||||||
Positive | |||||||||
Agreement | 210 | 0 | 210 | 210 | 0 | 210 | 209 | 1 | 210 |
Negative | |||||||||
Agreement | 59 | 1 | 60 | 60 | 0 | 60 | 60 | 0 | 60 |
Overall | |||||||||
Agreement | 269 | 1 | 270 | 270 | 0 | 270 | 269 | 1 | 270 |
FI | |||||||||
Agreement | 269 | 1 | 270 | 261 | 9 | 270 | 262 | 8 | 270 |
Found | |||||||||
Correctly | Found | ||||||||
Incorrectly | Expected | ||||||||
Numbers | Found | ||||||||
Correctly | Found | ||||||||
Incorrectly | Expected | ||||||||
Numbers | Found | ||||||||
Correctly | Found | ||||||||
Incorrectly | Expected | ||||||||
Numbers | |||||||||
Positive | |||||||||
Agreement | 210 | 0 | 210 | 210 | 0 | 210 | 205 | 5 | 210 |
Negative | |||||||||
Agreement | 60 | 0 | 60 | 60 | 0 | 60 | 60 | 0 | 60 |
Overall | |||||||||
Agreement | 270 | 0 | 270 | 270 | 0 | 270 | 265 | 5 | 270 |
FI | |||||||||
Agreement | 269 | 1 | 270 | 264 | 6 | 270 | 254 | 16 | 270 |
Table 40: Single-Operator Agreement (Calculation excluding borderline samples) % Agreements (95% CI) Method C
| % Agreement
(95% CI) | Site 1 | Site 2 | Site 3 | |||
---|---|---|---|---|---|---|
Reader 1 | Reader 2 | Reader 3 | Reader 4 | Reader 5 | Reader 6 | |
Positive | ||||||
Agreement | 100 | |||||
(98.2 - 100) | 100 | |||||
(98.2 - 100) | 100 | |||||
(98.2 - 100) | 100 | |||||
(98.2 - 100) | 99.5 | |||||
(97.4 - 99.9) | 97.6 | |||||
(94.5 - 99) | ||||||
Negative | ||||||
Agreement | 98.3 | |||||
(91.1 - 99.7) | 100 | |||||
(94 - 100) | 100 | |||||
(94 - 100) | 100 | |||||
(94 - 100) | 100 | |||||
(94 - 100) | 100 | |||||
(94 - 100) | ||||||
Overall | ||||||
Agreement | 99.6 | |||||
(97.9 - 99.9) | 100 | |||||
(98.6 - 100) | 100 | |||||
(98.6 - 100) | 100 | |||||
(98.6 - 100) | 99.6 | |||||
(97.9 - 99.9) | 98.1 | |||||
(95.7 - 99.2) | ||||||
FI Agreement | 99.6 | |||||
(97.9 - 99.9) | 96.7 | |||||
(93.8 - 98.2) | 99.6 | |||||
(97.9 - 99.9) | 97.8 | |||||
(95.2 - 99) | 97 | |||||
(94.3 - 98.5) | 94.1 | |||||
(90.6 - 96.3) |
30
July 28, 2017
Reader Confirmation (Method B) Single-Operator Agreement (Calculation including borderline samples):
Table 41: Single-Operator Agreement (Calculation including borderline samples) Number of Correctly/Incorrectly found samples Method B
Site 1 | Site 2 | Site 3 | ||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Number of | ||||||||||||||||||
Samples | Reader 1 | Reader 2 | Reader 3 | Reader 4 | Reader 5 | Reader 6 | ||||||||||||
Found | ||||||||||||||||||
Correctly | Found | |||||||||||||||||
Incorrectly | Expected | |||||||||||||||||
Numbers | Found | |||||||||||||||||
Correctly | Found | |||||||||||||||||
Incorrectly | Expected | |||||||||||||||||
Numbers | Found | |||||||||||||||||
Correctly | Found | |||||||||||||||||
Incorrectly | Expected | |||||||||||||||||
Numbers | Found | |||||||||||||||||
Correctly | Found | |||||||||||||||||
Incorrectly | Expected | |||||||||||||||||
Numbers | Found | |||||||||||||||||
Correctly | Found | |||||||||||||||||
Incorrectly | Expected | |||||||||||||||||
Numbers | Found | |||||||||||||||||
Correctly | Found | |||||||||||||||||
Incorrectly | Expected | |||||||||||||||||
Numbers | ||||||||||||||||||
Positive | ||||||||||||||||||
Agreement | 236 | 34 | 270 | 260 | 10 | 270 | 257 | 13 | 270 | 242 | 28 | 270 | 209 | 61 | 270 | 204 | 66 | 270 |
Negative | ||||||||||||||||||
Agreement | 60 | 0 | 60 | 60 | 0 | 60 | 60 | 0 | 60 | 58 | 2 | 60 | 60 | 0 | 60 | 60 | 0 | 60 |
Overall | ||||||||||||||||||
Agreement | 296 | 34 | 330 | 320 | 10 | 330 | 317 | 13 | 330 | 300 | 30 | 330 | 269 | 61 | 330 | 264 | 66 | 330 |
FI | ||||||||||||||||||
Agreement | na | na | na | na | na | na | na | na | na | na | na | na | na | na | na | na | na | na |
Table 42: Single-Operator Agreement (Calculation including borderline samples) % Agreements (95% CI) Method B
| % Agreement
(95% CI) | Site 1 | Site 2 | Site 3 | |||
---|---|---|---|---|---|---|
Reader 1 | Reader 2 | Reader 3 | Reader 4 | Reader 5 | Reader 6 | |
Positive | ||||||
Agreement | 87.4 | |||||
(82.9 - 90.8) | 96.3 | |||||
(93.3 - 98) | 95.2 | |||||
(91.9 - 97.2) | 89.6 | |||||
(85.4 - 92.7) | 77.4 | |||||
(72.1 - 82) | 75.6 | |||||
(70.1 - 80.3) | ||||||
Negative | ||||||
Agreement | 100 | |||||
(94 - 100) | 100 | |||||
(94 - 100) | 100 | |||||
(94 - 100) | 96.7 | |||||
(88.6 - 99.1) | 100 | |||||
(94 - 100) | 100 | |||||
(94 - 100) | ||||||
Overall | ||||||
Agreement | 89.7 | |||||
(85.9 - 92.5) | 97 | |||||
(94.5 - 98.3) | 96.1 | |||||
(93.4 - 97.7) | 90.9 | |||||
(87.3 - 93.6) | 81.5 | |||||
(77 - 85.3) | 80 | |||||
(75.4 - 84) | ||||||
FI Agreement | na | na | na | na | na | na |
31
July 28, 2017
Reader Confirmation (Method B) Single-Operator Agreement (Calculation excluding borderline samples):
Table 43: Single-Operator Agreement (Calculation excluding borderline samples) Number of Correctly/Incorrectly found samples Method B
| Number of
Samples | Site 1 | Site 2 | Site 3 | |||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Reader 1 | Reader 2 | Reader 3 | Reader 4 | Reader 5 | Reader 6 | |||||||||||||
Found | ||||||||||||||||||
Correctly | Found | |||||||||||||||||
Incorrectly | Expected | |||||||||||||||||
Numbers | Found | |||||||||||||||||
Correctly | Found | |||||||||||||||||
Incorrectly | Expected | |||||||||||||||||
Numbers | Found | |||||||||||||||||
Correctly | Found | |||||||||||||||||
Incorrectly | Expected | |||||||||||||||||
Numbers | Found | |||||||||||||||||
Correctly | Found | |||||||||||||||||
Incorrectly | Expected | |||||||||||||||||
Numbers | Found | |||||||||||||||||
Correctly | Found | |||||||||||||||||
Incorrectly | Expected | |||||||||||||||||
Numbers | Found | |||||||||||||||||
Correctly | Found | |||||||||||||||||
Incorrectly | Expected | |||||||||||||||||
Numbers | ||||||||||||||||||
Positive | ||||||||||||||||||
Agreement | 206 | 4 | 210 | 209 | 1 | 210 | 210 | 0 | 210 | 208 | 2 | 210 | 203 | 7 | 210 | 197 | 13 | 210 |
Negative | ||||||||||||||||||
Agreement | 60 | 0 | 60 | 60 | 0 | 60 | 60 | 0 | 60 | 58 | 2 | 60 | 60 | 0 | 60 | 60 | 0 | 60 |
Overall | ||||||||||||||||||
Agreement | 266 | 4 | 270 | 269 | 1 | 270 | 270 | 0 | 270 | 266 | 4 | 270 | 263 | 7 | 270 | 257 | 13 | 270 |
FI | ||||||||||||||||||
Agreement | na | na | na | na | na | na | na | na | na | na | na | na | na | na | na | na | na | na |
Table 44: Single-Operator Agreement (Calculation excluding borderline samples) % Agreements (95% CI) Method B
| % Agreement
(95% CI) | Site 1 | Site 2 | Site 3 | ||||
---|---|---|---|---|---|---|---|
Reader 1 | Reader 2 | Reader 3 | Reader 4 | Reader 5 | Reader 6 | ||
Positive | |||||||
Agreement | 98.1 | 99.5 | 100 | 99 | 96.7 | 93.8 | |
(95.2 - 99.3) | (97.4 - 99.9) | (98.2 - 100) | (96.6 - 99.7) | (93.3 - 98.4) | (89.7 - 96.3) | ||
Negative | |||||||
Agreement | 100 | 100 | 100 | 96.7 | 100 | 100 | |
(94 - 100) | (94 - 100) | (94 - 100) | (88.6 - 99.1) | (94 - 100) | (94 - 100) | ||
Overall | |||||||
Agreement | 98.5 | 99.6 | 100 | 98.5 | 97.4 | 95.2 | |
(96.3 - 99.4) | (97.9 - 99.9) | (98.6 - 100) | (96.3 - 99.4) | (94.7 - 98.7) | (91.9 - 97.2) | ||
Fl Agreement | na | na | na | na | na | na |
Overall-Single-Operator agreements ranged from 95.2% to 100% for Method B (Reader Confirmation), and from 98.1% to 100% for Method C (Manual).
32
Instrument Precision
HELIOS (Method A) Instrument Agreement (Calculation including borderline samples):
Table 45: Instrument Precision (Calculation including borderline samples) Number of Correctly/Incorrectly found samples Method A
| Number of
Samples | Site 1 | Site 2 | Site 3 | ||||||
---|---|---|---|---|---|---|---|---|---|
Found | |||||||||
Correctly | Found | ||||||||
Incorrectly | Expected | ||||||||
Numbers | Found | ||||||||
Correctly | Found | ||||||||
Incorrectly | Expected | ||||||||
Numbers | Found | ||||||||
Correctly | Found | ||||||||
Incorrectly | Expected | ||||||||
Numbers | |||||||||
Positive | |||||||||
Agreement | 190 | 80 | 270 | 151 | 119 | 270 | 131 | 139 | 270 |
Negative | |||||||||
Agreement | 48 | 12 | 60 | 57 | 3 | 60 | 57 | 3 | 60 |
Overall | |||||||||
Agreement | 238 | 92 | 330 | 208 | 122 | 330 | 188 | 142 | 330 |
FI Agreement | na | na | na | na | na | na | na | na |
Table 46: Instrument Precision (Calculation including borderline samples) % Agreements (95% Cl) Method A
| % Agreement
(95% CI) | Site 1 | Site 2 | Site 3 |
---|---|---|---|
Positive | |||
Agreement | 70.4 | ||
(64.7 - 75.5) | 55.9 | ||
(50 - 61.7) | 48.5 | ||
(42.6 - 54.5) | |||
Negative | |||
Agreement | 80 | ||
(68.2 - 88.2) | 95 | ||
(86.3 - 98.3) | 95 | ||
(86.3 - 98.3) | |||
Overall | |||
Agreement | 72.1 | ||
(67 - 76.7) | 63 | ||
(57.7 - 68.1) | 57 | ||
(51.6 - 62.2) | |||
FI Agreement | na | na | na |
33
HELIOS (Method A) Instrument Agreement (Calculation excluding borderline samples):
| Number of
Samples | Site 1 | Site 2 | Site 3 | ||||||
---|---|---|---|---|---|---|---|---|---|
Found | |||||||||
Correctly | Found | ||||||||
Incorrectly | Expected | ||||||||
Numbers | Found | ||||||||
Correctly | Found | ||||||||
Incorrectly | Expected | ||||||||
Numbers | Found | ||||||||
Correctly | Found | ||||||||
Incorrectly | Expected | ||||||||
Numbers | |||||||||
Positive | |||||||||
Agreement | 180 | 30 | 210 | 148 | 62 | 210 | 130 | 80 | 210 |
Negative | |||||||||
Agreement | 48 | 12 | 60 | 57 | 3 | 60 | 57 | 3 | 60 |
Overall | |||||||||
Agreement | 228 | 42 | 270 | 205 | 65 | 270 | 187 | 83 | 270 |
FI Agreement | na | na | na | na | na | na | na | na | na |
Table 47: Instrument Precision (Calculation excluding borderline samples) Number of Correctly/Incorrectly found samples Method A
Table 48: Instrument Precision (Calculation excluding borderline samples) % Agreements | |
---|---|
(95% Cl) Method A |
| % Agreement
(95% CI) | Site 1 | Site 2 | Site 3 |
---|---|---|---|
Positive | |||
Agreement | 85.7 | ||
(80.3 - 89.8) | 70.5 | ||
(64 - 76.2) | 61.9 | ||
(55.2 - 68.2) | |||
Negative | |||
Agreement | 80 | ||
(68.2 - 88.2) | 95 | ||
(86.3 - 98.3) | 95 | ||
(86.3 - 98.3) | |||
Overall | |||
Agreement | 84.4 | ||
(79.6 - 88.3) | 75.9 | ||
(70.5 - 80.6) | 69.3 | ||
(63.5 - 74.5) | |||
FI Agreement | na | na | na |
Acceptance criteria were met for site 1 and site 2. Positive, Neqative and Overall Agreements were >70%, At site 3, positive agreement was slightly below 70% which impacts also on the overall agreement at this site. Upon rechecking the respective HELIOS images we found that some of the images (approx. 11% of all images) that had been suggested false negative by HELIOS were out of focus. Therefore, the DNA Pattern Plus Software was not able to detect the sample as positive, although the readers still could find the sample as positive. This explains the observed lower positive agreement at site 3. This is also a reason why we clearly state that all results suggested by HELIOS have to be confirmed by a trained reader.
Overall, the three different HELIOS devices gave comparable results. Overall-Instrument agreements ranged from 69.3% to 84.4%.
Raw data see Attachment 12 Section 12 Raw data between lab Precision Study
34
Conclusions
Overall-Between-Lab agreements ranged from 72.6% to 80.2% for Method A (HELIOS), from 97.7% to 99.2% for Method B (Reader Confirmation), and from 96.9% to 98.4% for Method C (Manual).
Overall-Between-Operator aqreements ranged from 96.3% to 99.3% for Method B (Reader Confirmation), and from 98.9% to 100% for Method C (Manual).
Overall-Single-Operator agreements ranged from 95.2% to 100% for Method B (Reader Confirmation), and from 98.1% to 100% for Method C (Manual).
Overall-Instrument agreements ranged from 69.3% to 84.4% (Method A, HELIOS).
The results show that all results suggested by HELIOs have to be confirmed by a trained reader.
Lot to Lot Precision Study AESKUSLIDES nDNA (Crithidia luciliae)
A lot to lot reproducibility study was performed using three reagent lots each for AESKUSLIDES nDNA. 11 sera were tested:
The samples used for this study are the same as for Within Lab and Between Lab Precision Study AESKUSLIDES nDNA (Crithidia luciliae)
Each of the 11 serum samples was assayed 10 times on each reagent lot to give a total of 30 replicates per serum sample.
Slides were processed manually according to the IFU and subsequently analyzed at the microscope by two independent readers.
Agreements were calculated across all 3 reagent lots, for individual as well as for combined readers.
Acceptance Criteria:
- -Positive sera have to be found positive, and negative sera have to be found negative.
- -Correct pattern has to be found.
- -Reported FI is allowed to differ max. ± 1 level from the expected value.
- -Agreements should meet the following criteria:
Type of Agreement | % Agreement |
---|---|
Positive Agreement | > 90% |
Negative Agreement | > 90% |
Overall Agreement | > 90% |
Fluorescence Intensity Agreement | > 90% |
35
Results:
Table 49: Lot to lot agreement (across all 3 lots) AESKUSLIDES nDNA – Number of Samples | |||
---|---|---|---|
----------------------------------------------------------------------------------------- | -- | -- | -- |
Number of | Reader 1 | Reader 2 | Combined Readers | ||||||
---|---|---|---|---|---|---|---|---|---|
Samples | Found | ||||||||
Correctly | Found | ||||||||
Incorrectly | Expected | ||||||||
Numbers | Found | ||||||||
Correctly | Found | ||||||||
Incorrectly | Expected | ||||||||
Numbers | Found | ||||||||
Correctly | Found | ||||||||
Incorrectly | Expected | ||||||||
Numbers | |||||||||
Positive | |||||||||
Agreement | 270 | O | 270 | 270 | 0 | 270 | 540 | 0 | 540 |
Negative | |||||||||
Agreement | 60 | O | 60 | 60 | 0 | 60 | 120 | 0 | 120 |
Overall | |||||||||
Agreement | 330 | O | 330 | 330 | 0 | 330 | 660 | 0 | 660 |
F | |||||||||
Agreement | 330 | O | 330 | 330 | 0 | 330 | 660 | 0 | 660 |
FI = Fluorescence Intensity
Table 50: Lot to lot agreement (across all 3 lots) AESKUSLIDES nDNA – % Agreement: | |
---|---|
| % Agreement
(95% CI) | Reader 1 | Reader 2 | Combined
Readers |
|-------------------------|---------------------|---------------------|---------------------|
| Positive Agreement | 100
(98.6 - 100) | 100
(98.6 - 100) | 100
(99.3 - 100) |
| Negative Agreement | 100
(94 - 100) | 100
(94 - 100) | 100
(96.9 - 100) |
| Overall Agreement | 100
(98.8 - 100) | 100
(98.8 - 100) | 100
(99.4 - 100) |
| FI Agreement | 100
(98.8 - 100) | 100
(98.8 - 100) | 100
(99.4 - 100) |
FI = Fluorescence Intensity
All agreements fulfilled the above criteria.
- -Positive agreement was 100% for AESKUSLIDES nDNA for both readers.
- -Negative agreement for nDNA was 100% for both readers.
- Overall agreement for nDNA was 100% for both readers. -
- FI agreement for nDNA was 100% for both readers. -
- No deviations of fluorescence intensities greater than +/-1 from the initial value have been observed.
Raw data see Attachment 13_Section 12_ Raw data Lot to Lot Precision Study
Conclusion:
Overall lot to lot agreement (across all 3 lots) for AESKUSLIDES nDNA was 100%. All acceptance criteria have been fulfilled.
36
b) Carryover:
To investigate and evaluate the carry over of samples from one reaction well to the following reaction well when slides are processed by HELIOS.
Three high positive samples were run on HELIOS in alternate with a negative serum sample. Results were confirmed by a reader. Tests were performed in accordance with the respective IFUs.
| Sample
ID | AESKUSLIDES nDNA
Pos/Neg | Grading |
|--------------|-----------------------------|---------------|
| 1 | Pos | high positive |
| 2 | Pos | high positive |
| 3 | Pos | high positive |
| 4 | Neg | negative |
Acceptance criteria:
- Pos/Neg/Overall Agreement: All positive sera have to be found positive and all negative sera have to be found negative.
- FI is allowed to differ maximum + or 1 from the expected value. -
Results:
Table 51: Results Carry Over
Sample ID | AESKUSLIDES nDNA | |
---|---|---|
Result | FI | |
1 | Pos | 3 |
4 | Neg | 0 |
4 | Neg | 0 |
2 | Pos | 3 |
4 | Neg | 0 |
4 | Neg | 0 |
3 | Pos | 3 |
4 | Neg | 0 |
4 | Neg | 0 |
All samples fulfilled the above criteria. No carry over has been observed from well to well. All positive samples were identified as positive. All negative samples were identified as negative.
Conclusions
No carry over could be observed.
c) Time Extension Study
To investigate the stability of AESKUSLIDES nDNA performance upon shortening and elongation of the recommended incubation times for samples and conjugate.
Procedure:
AESKUSLIDES nDNA was assayed with 10 samples on HELIOS with the following incubation times:
37
-
- 15 min sera incubation + 15min incubation of conjugate
-
- 30 min sera incubation + 30min incubation of conjugate
-
- 45 min sera incubation + 45min incubation of conjugate
The following samples were used:
Sample ID | Expected Results | |
---|---|---|
Result | Grading | |
Sample 1 | Pos | low positive |
Sample 2 | Pos | medium positive |
Sample 3 | Pos | low positive |
Sample 4 | Pos | low positive |
Sample 5 | Pos | medium positive |
Sample 6 | Pos | medium positive |
Sample 7 | Pos | high positive |
Sample 8 | Pos | high positive |
Sample 9 | Neg | negative |
Sample 10 | Neg | Neg |
Tests were performed in accordance with the respective IFUs. Results have been confirmed by a trained reader.
Results of the 30 min protocol will be compared to the results of the 15 min and 45 min protocol.
Acceptance criteria:
- Pos/Neg/Overall Agreement: All positive sera have to be found positive and all negative sera have to be found negative.
- -FI is allowed to differ maximum + or - 1 from the expected value.
Results
| Incubation
Times | 2 x 15 Minutes | 2 x 30 Minutes | 2 x 45 Minutes | |||
---|---|---|---|---|---|---|
Sample ID | Result | Fluorescence | ||||
Intensity | Result | Fluorescence | ||||
Intensity | Result | Fluorescence | ||||
Intensity | ||||||
PC | Pos | 3 | Pos | 3 | Pos | 3 |
NC | Neg | Neg | Neg | Neg | Neg | Neg |
Sample 1 | Pos | 1 | Pos | 1 | Pos | 1 |
Sample 2 | Pos | 2 | Pos | 2 | Pos | 2 |
Sample 3 | Pos | 1 | Pos | 1 | Pos | 2 |
Sample 4 | Pos | 1 | Pos | 1 | Pos | 2 |
Sample 5 | Pos | 2 | Pos | 2 | Pos | 2 |
Sample 6 | Pos | 2 | Pos | 2 | Pos | 3 |
Sample 7 | Pos | 3 | Pos | 3 | Pos | 3 |
Sample 8 | Pos | 3 | Pos | 3 | Pos | 3 |
Sample 9 | Neg | Neg | Neg | Neg | Neg | Neg |
Sample 10 | Neg | Neg | Neg | Neg | Neg | Neg |
Table 52: Results Time Extention
38
All acceptance criteria were fulfilled in each assay. All positive samples were identified as positive. All negative samples were identified as negative. Fluorescence intensities did not deviate more than +/-1 level.
Conclusions
The results demonstrate stability of the AESKUSLIDES nDNA during the tested incubation times of conjugate and sample.
d) Detection limit
Not applicable
e) Analytical specificity
Interfering substances:
The interference study was performed according to CLSI EP07-A2. Interference Testing in Clinical Chemistry; Approved Guideline - Second Edition. Interference by 8 different substances was assessed by testing 8 serum samples:
Sample ID | AESKUSLIDES nDNA | |
---|---|---|
Pos/Neg | Grading | |
1 | Pos | low positive |
2 | Pos | low positive |
3 | Pos | medium positive |
4 | Pos | medium positive |
5 | Pos | high positive |
6 | Pos | high positive |
7 | Neg | negative |
8 | Neg | negative |
The following substances have been tested with the indicated concentrations:
Table 53: Interfering substances with the indicated concentrations
| Interfering Substance | Minimum final
Concentration
tested | Maximum final
Concentration
tested |
|------------------------|------------------------------------------|------------------------------------------|
| Bilirubin conjugated | 0.1 mg/mL | 0.4 mg/mL |
| Bilirubin unconjugated | 0.1 mg/mL | 0.4 mg/mL |
| Hemoglobin | 2.5 mg/mL | 5.0 mg/mL |
| Triglycerides | 5 mg/mL | 20 mg/mL |
| RF IgM | 200 U/mL | 400 U/mL |
| Rituximab | 0.5 mg/mL | 2.0 mg/mL |
| Methylprednisolone | 0.2 mg/mL | 0.8 mg/mL |
| Cyclophosphamide | 1.0 mg/mL | 4.0 mg/mL |
| Methotrexate | 0.025 mg/mL | 0.1 mg/mL |
| Azathioprine | 0.0075 mg/mL | 0.03 mg/mL |
| Belimumab | 2 mg/mL | 8 mg/mL |
| Hydroxychloroquine | 0.006 mg/mL | 0.024 mg/mL |
| Mycophenolat | 0.012 mg/mL | 0.048 mg/mL |
| Ibuprofen | 0.5 mg/mL | 2 mg/mL |
| Naproxen | 0.5 mg/mL | 2 mg/mL |
"The e-Copy is an exact duplicate of the paper copy"
39
Interfering substances were spiked in each of the 8 samples in two different concentrations. Controls were prepared for each serum sample by spiking in only the respective amount of diluent without interfering substances. Spiked serum samples and their controls have been tested in triplicates for each concentration of interferent with AESKUSLIDES nDNA. All tests have been performed manually according to the IFU. Results have been analyzed by two independent readers at the microscope.
The results (positive/negative classification. fluorescence intensity (FI)) of the spiked samples were compared to the results of the respective controls. Agreements have been calculated by dividing the number of correctly found results through the number of total results for spiked samples.
Acceptance criteria:
- -Pos/Neg/Overall Agreement: All positive sera have to be found positive and all negative sera have to be found negative.
- -Overall Agreement: > 90%.
- FI is allowed to differ maximum + or 1 from the expected value. -
- FI Agreement: > 90%. -
Results
All samples fulfilled the above criteria. No significant differences were observed between sera spiked with different concentrations of the interferents and their controls:
- Positive Agreement was 100% for all tested substances and concentrations. except for 20 mg/ml Triglycerides with Reader 1 (97%). There. one out of three repetitions of sample 2 was found negative instead of low positive (FI 1). Reader 2 found the sample correctly. Therefore. we consider this single discrepancy as a result of normal test variances and the subjective nature of the result analysis.
- Negative Agreement was 100% for all tested interferents and concentrations.
- Overall Agreement ranged from 97.9% to 100%. It was 100% for all tested substances and concentrations. except for 20 mg/ml Triglycerides with Reader 1 (97.9%).
- -FI Agreement was 100% for all tested interferents and concentrations.
Raw Data see Attachment 14_Section 12_Raw data Interfering Substances
Conclusions
All acceptance criteria are fulfilled in each test. No interference was detected with the tested substances. up to the maximal concentrations indicated above.
40
f) Assay cut-off:
AESKU recommends a screening dilution of 1:10. followed by serial dilutions for semi-quantitative determinations. but suggests each laboratory establish its own screening dilution and titration scheme based on its population. The titers of 1:10 are considered low titers. 1:20 and 1:40 are considered medium titers, and 1:80 and greater are considered high titers.
2. Comparison studies
a. Method comparison with predicate device:
To show comparable performance of AESKUSLIDES nDNA and the predicate assay NOVA Lite dsDNA (Crithidia luciliae) of INOVA.
Procedure:
A clinical study with the same 776 serum samples (comprising 297 serum samples from patients with SLE and 479 samples from patients with other diseases) was performed with AESKUSLIDES nDNA and the predicate assay. A detailed description of the study sample can be found on page 12-50.
For each assay the sample set was processed manually and analyzed (positive/negative classification) at the microscope (Method C).
All assays were performed according to the respective IFUs. Sample were screened at a dilution of 1:10.
From the results of both assays the relative sensitivity and specificity / % positive, negative and overall agreements were calculated.
Furthermore, diagnostic sensitivity and specificity was calculated for both assays, and the cross reactivities in different diagnosis will be shown in a table.
Results:
Table 54: Method comparison with predicate device Diagnostic Sensitivity and Specificity Number of positive and negative results and % agreements:
| Diagnostic Sensitivity &
Specificity | Diagnosis | |||
---|---|---|---|---|
SLE | Other Diseases | Total | ||
AESKUSLIDES | ||||
nDNA | Pos | 73 | 25 | 98 |
Neg | 224 | 454 | 678 | |
Total | 297 | 479 | 776 | |
Predicate Assay | Pos | 110 | 68 | 178 |
Neg | 187 | 411 | 598 | |
Total | 297 | 479 | 776 |
41
| % Diagnostic
Sensitivity &
Specificity (95%
CI) | % Sensitivity
(95% CI) | % Specificity
(95% CI) | Predictive Value
of a Positive Test
Result (PPV) | Predictive Value
of a Negative
Test Result
(NPV) |
|----------------------------------------------------------|---------------------------|---------------------------|--------------------------------------------------------|-----------------------------------------------------------|
| | SLE (n= 297) | OD (n=479) | | |
| AESKUSLIDES
nDNA | 24.6
(20 - 29.8) | 94.8
(92.4 - 96.4) | 74.5% | 67.0% |
| Predicate Assay | 37
(31.7 - 42.7) | 85.8
(82.4 - 88.6) | 61.8% | 68.7% |
Sensitivity of AESKUSLIDES nDNA is a little bit lower (24%.6) than that calculated for the predicate assay (37%) in this study. On the other hand, specificity of AESKU is higher (94.8%) than that of the predicate (85%.8). Those values for sensitivity correlate well with values reported in the literature, where sensitivities of nDNA IFA assays for SLE are described from 13% up to 67%. Anti dsDNA antibodies are associated with disease activity and also have been shown to disappear upon treatment. The SLE cohort tested in this study was not further selected for inactive or active patients, or for treatment naïve patients. Treatment naïve patients are very rare, therefore we expect to have a greater proportion of patients under treatment and also inactive patients in the cohort. This explains why in this study the sensitivities are at the lower end of the reported sensitivities in the literature.
The higher specificity and lower sensitivity of AESKUSLIDES nDNA goes along with a higher PPV (74.5% compared to 61.8% for the predicate). NPVs of both assays are comparable (67% vs 68.7%).
The results for both assays are shown in detail in the crossreactivity table below. With AESKUSLIDES nDNA fewer crossereactivities are seen. Crossreactivities could be detected in APS (8.3%), in Dermato-/Polymyositis (16.1%), RA (17.2%), MCTD (12.5%), AIH (9.5%), PBC (11.1%), and Vasculitis (4.5%). The predicate assay detected positives in almost all diagnosis tested and also higher numbers of positives.
42
Image /page/42/Picture/0 description: The image is a logo for AESKU.DIAGNOSTICS. The logo features a green square with a white symbol inside, followed by the text "AESKU.DIAGNOSTICS" in gray. Below the text, there is a tagline that reads "THE DIAGNOSTIC TOOL THAT WORKS" in a smaller font size and lighter shade of gray.
CONFIDENTIAL
AESKUSLIDES nDNA (Crithidia luciliae) 510(k) Traditional Submission
July 28, 2017
Diagnosis | n | AESKUSLIDES nDNA | Predicate | ||||||||
---|---|---|---|---|---|---|---|---|---|---|---|
n Pos | %Pos | n Neg | %Neg | n Pos | %Pos | n Neg | %Neg | ||||
Target disease | Systemic Lupus Erythematosus | 297 | 73 | 24.6% | 224 | 75.4% | 110 | 37.0% | 187 | 63.0% | |
Antiphospholipid Syndrome | 12 | 1 | 8.3% | 11 | 91.7% | 4 | 33.3% | 8 | 66.7% | ||
Sjoegren's Syndrome | 30 | 0 | 0.0% | 30 | 100.0% | 5 | 16.7% | 25 | 83.3% | ||
Systemic Sclerosis | 44 | 0 | 0.0% | 44 | 100.0% | 5 | 11.4% | 39 | 88.6% | ||
Other Rheumatic | |||||||||||
Diseases | Dermatomyositis/Polymyositis | 31 | 5 | 16.1% | 26 | 83.9% | 3 | 9.7% | 28 | 90.3% | |
Rheumatoid Arthritis | 58 | 10 | 17.2% | 48 | 82.8% | 15 | 25.9% | 43 | 74.1% | ||
Mixed Connective Tissue Diseases | 24 | 3 | 12.5% | 21 | 87.5% | 5 | 20.8% | 19 | 79.2% | ||
Undifferentiated Connective Tissue Diseases | 21 | 0 | 0.0% | 21 | 100.0% | 0 | 0.0% | 21 | 100.0% | ||
Control | |||||||||||
diseases | Autoimmune Liver | ||||||||||
Diseases | Autoimmune Hepatitis | 21 | 2 | 9.5% | 19 | 90.5% | 7 | 33.3% | 14 | 66.7% | |
Primary Biliary Cirrhosis | 20 | 2 | 11.1% | 18 | 88.9% | 5 | 44.4% | 15 | 55.6% | ||
Primary Sclerosing Cholangitis | 10 | 0 | 0.0% | 10 | 100.0% | 0 | 0.0% | 10 | 100.0% | ||
Vasculitis | ANCA associated Vasculitis | 44 | 2 | 4.5% | 42 | 95.5% | 1 | 2.3% | 43 | 97.7% | |
Epstein-Barr Virus Infection | 25 | 0 | 0.0% | 25 | 100.0% | 4 | 16.0% | 21 | 84.0% | ||
Infectious | |||||||||||
Diseases | Hepatitis B Virus Infection | 34 | 0 | 0.0% | 34 | 100.0% | 4 | 11.8% | 30 | 88.2% | |
Hepatitis C Virus Infection | 44 | 0 | 0.0% | 44 | 100.0% | 2 | 4.5% | 42 | 95.5% | ||
Leukemia | Lymphoma | 31 | 0 | 0.0% | 31 | 100.0% | 7 | 22.6% | 24 | 77.4% | |
Other | Fibromyalgia | 30 | 0 | 0.0% | 30 | 100.0% | 1 | 3.3% | 29 | 96.7% | |
Total | 776 | 98 | 12.6% | 678 | 87.4% | 178 | 22.9% | 598 | 77.1% |
Table 55: Method comparison with predicate device Number and percent positive samples in the different tested diagnosis
43
Image /page/43/Picture/1 description: The image is a logo for AESKU.DIAGNOSTICS. The logo features a green square with a white image inside. To the right of the square is the text "AESKU.DIAGNOSTICS" in gray, with the word "AESKU" being larger than the word "DIAGNOSTICS". Below the text is the phrase "THE DIAGNOSTIC TOOL THAT WORKS" in a smaller, lighter gray font.
Table 56: Method comparison with predicate device, Relative sensitivity and specificity Number of positive and negative results and % agreements:
Method Comparison | Predicate | |||
---|---|---|---|---|
Pos | Neg | Total | ||
AESKUSLIDES | ||||
nDNA | Pos | 80 | 13 | 93 |
Neg | 98 | 585 | 683 | |
Total | 178 | 598 | 776 |
| Method Comparison
Predicate vs AESKU | % Agreement
(95% CI) |
|-----------------------------------------|-------------------------|
| Positive Agreement | 44.9
(37.8 - 52.3) |
| Negative Agreement | 97.8
(96.3 - 98.7) |
| Overall Agreement | 85.7
(83.1 - 88) |
Overall Agreement between both assays was good (85.7%), and negative agreement was very good with 97.8%. The rather low positive agreement of 44.9% is due to the fact that the predicate assay detects more positive results.
For the interpretation of the discrepant results we also tested the respective samples by ds DNA ELISA (IgG). The results are shown in the following tables.
Table 57: Method comparison with predicate device, Discrepant samples in the Non-Healthy Control group:
| Sample
ID | Diagnosis | Predicate | | AESKUSLIDES
nDNA | | AESKULISA
dsDNA-G |
|--------------|--------------|-----------|----|---------------------|----|----------------------|
| | | Result | FI | Result | FI | U/ml |
| 688 | DM/PM | Neg | 0 | Pos | 3 | 4,9 |
| 690 | DM/PM | Neg | 0 | Pos | 3 | 7,4 |
| 691 | DM/PM | Neg | 0 | Pos | 2 | 8,7 |
| 67 | RA | Neg | 0 | Pos | 1 | 1,8 |
| 694 | RA | Neg | 0 | Pos | 1 | 4,3 |
| 596 | Vasculitis | Neg | 0 | Pos | 1 | 0,7 |
| 632 | APS | Pos | 1 | Neg | 0 | 0,9 |
| 758 | APS | Pos | 1 | Neg | 0 | 28,3 |
| 759 | APS | Pos | 1 | Neg | 0 | 3,1 |
| 243 | AIH | Pos | 1 | Neg | 0 | 4,3 |
| | | | | | | |
| 246 | AIH | Pos | 1 | Neg | 0 | 3,1 |
| 322 | AIH | Pos | 1 | Neg | 0 | 9,9 |
| 361 | AIH | Pos | 1 | Neg | 0 | 3,4 |
| 234 | PBC | Pos | 1 | Neg | 0 | 6,5 |
| 237 | PBC | Pos | 1 | Neg | 0 | 46,6 |
| 240 | PBC | Pos | 1 | Neg | 0 | 4,8 |
| 209 | EBV | Pos | 1 | Neg | 0 | 3,2 |
| 212 | EBV | Pos | 1 | Neg | 0 | 5,3 |
| 220 | EBV | Pos | 1 | Neg | 0 | 12,2 |
| 221 | EBV | Pos | 1 | Neg | 0 | Max |
| 200 | SLE | Pos | 1 | Neg | 0 | 1,9 |
| 201 | SLE | Pos | 1 | Neg | 0 | 2,4 |
| 207 | SLE | Pos | 1 | Neg | 0 | 20,3 |
| 317 | SLE | Pos | 1 | Neg | 0 | 28,8 |
| 323 | SLE | Pos | 1 | Neg | 0 | 8,5 |
| 324 | SLE | Pos | 1 | Neg | 0 | 62,3 |
| 328 | SLE | Pos | 1 | Neg | 0 | 3,9 |
| 371 | SLE | Pos | 1 | Neg | 0 | 4,8 |
| 381 | SLE | Pos | 1 | Neg | 0 | 301,0 |
| 383 | SLE | Pos | 1 | Neg | 0 | 2,1 |
| 386 | SLE | Pos | 1 | Neg | 0 | 14,0 |
| 432 | SLE | Pos | 2 | Neg | 0 | 11,3 |
| 433 | SLE | Pos | 1 | Neg | 0 | 13,1 |
| 434 | SLE | Pos | 1 | Neg | 0 | 4,6 |
| 447 | SLE | Pos | 1 | Neg | 0 | 1,8 |
| 451 | SLE | Pos | 1 | Neg | 0 | 52,5 |
| 456 | SLE | Pos | 1 | Neg | 0 | 1,8 |
| 461 | SLE | Pos | 1 | Neg | 0 | 6,2 |
| 466 | SLE | Pos | 1 | Neg | 0 | 57,0 |
| 467 | SLE | Pos | 1 | Neg | 0 | 4,3 |
| 482 | SLE | Pos | 1 | Neg | 0 | 61,5 |
| 499 | SLE | Pos | 1 | Neg | 0 | 7,1 |
| 505 | SLE | Pos | 1 | Neg | 0 | 8,3 |
| 513 | SLE | Pos | 1 | Neg | 0 | 3,6 |
| 514 | SLE | Pos | 1 | Neg | 0 | 14,9 |
| 672 | SLE | Pos | 2 | Neg | 0 | 84,7 |
46
Image /page/46/Picture/1 description: The image is a logo for AESKU.DIAGNOSTICS. The logo features a green square with a white medical symbol inside. To the right of the square is the company name, "AESKU.DIAGNOSTICS," in gray text. Below the company name is the tagline, "THE DIAGNOSTIC TOOL THAT WORKS," in a lighter gray font.
July 28, 2017
47
42 of the 97 of the discrepant samples were found in the SLE cohort. Of these 42 samples, 2 have been reported positive by the AESKU assay and negative by the predicate. However, the positive results could not be confirmed by ELISA.
40 of these 42 samples have been reported positive by the predicate assay and negative by the AESKU assay. Almost all positive samples had a low fluorescence intensity (FI=1). Only 12 samples of those 40 could be confirmed positive by ELISA. Therefore, we think that the predicate outcompetes AESKU in those 12 cases. This is only 4% of the total SLE samples. 28 of the 40 samples could not be confirmed by ELISA which is generally more sensitive than the nDNA immunofluorescence assays.
Overall, the predicate assay detected more positives in the SLE cohort, but most of those results could not be confirmed by ELISA. This matches the observation that INOVA has a lower specificity than AESKU.
Generally, Crithidia luciliae immunofluorescence tests provided by different manufactures on the market are known to deliver different results. This could be explained by differences in the manufacturing process, e.g. cultivation medium, cultivation period, and the fixation method.
Conclusions
AESKUSLIDES nDNA showed a lower diagnostic sensitivity (24.6%) compared to the predicate assay NOVA Lite provided by INOVA (37%). However, the AESKU assay is characterized by a higher specificity (94.8% vs. 85.8%), because of less crossreactivities with other disease groups.
Raw data see Attachment 15_Section 12_Raw data Method comparison with predicate device
b. Method Comparison of Method A, B, and C and clinical study To show comparable performance of AESKUSLIDES nDNA when processed manually and on HELIOS.
Procedure:
A clinical study with the same 776 serum samples (comprising 297 serum samples from patients with SLE and 479 samples from patients with other diseases) was performed on 3 different study sites (2 US sites, one German site). A detailed description of the study sample can be found on page 12-50.
At each study site the sample set was processed
-
- manually and analyzed (positive/negative classification) at the microscope by two independent readers (Method C)
-
- on HELIOS and HELIOS images were analyzed by two independent readers at the PC (Method B)
"The e-Copy is an exact duplicate of the paper copy"
48
-
- on HELIOS and positive/negative classification was done by the HELIOS DNA Pattern Plus Software (Method A)
All assays were performed according to the IFU.
- on HELIOS and positive/negative classification was done by the HELIOS DNA Pattern Plus Software (Method A)
For each study site, the results of the single and combined readers with the different Methods were compared to each other and % agreements were calculated:
- Method C (Manual) vs Method B (Reader Confirmation)
- Method B (Reader Confirmation) vs Method A (HELIOS)
- Method C (Manual) vs Method A (HELIOS).
Furthermore, the number of positive samples found for each tested diagnosis for all study sites and all Methods will be compared as well as the calculated diagnostic sensitivities and specificities.
Acceptance Criteria:
Positive, Negative, and Overall Agreements between the different Methods should be >85%. The analysis of diagnostic immunofluorescence assays and especially of nDNA assays depends strongly on the readers experience and is very subjective. Therefore, we expect and accept variances up to 15% in this clinical study.
Results:
| % Agreement
(95% CI) | Method C vs B | Method B vs A | Method C vs A | ||||
---|---|---|---|---|---|---|---|
Reader 1 | Reader 2 | Reader 1 | Reader 2 | Reader 1 | Reader 2 | ||
Site | |||||||
1 | Positive | ||||||
Agreement | 85.1 | ||||||
(76.1 - 91.1) | 85.5 | ||||||
(75.9 - 91.7) | 51.8 | ||||||
(41.3 - 62.1) | 50.6 | ||||||
(40 - 61.2) | 48.3 | ||||||
(38.1 - 58.6) | 51.3 | ||||||
(40.3 - 62.2) | |||||||
Negative | |||||||
Agreement | 98.4 | ||||||
(97.2 - 99.1) | 97.7 | ||||||
(96.3 - 98.6) | 84.4 | ||||||
(81.5 - 86.9) | 84 | ||||||
(81.1 - 86.6) | 84 | ||||||
(81.1 - 86.6) | 83.9 | ||||||
(80.9 - 86.4) | |||||||
Total | |||||||
Agreement | 96.9 | ||||||
(95.4 - 97.9) | 96.5 | ||||||
(95 - 97.6) | 80.8 | ||||||
(77.9 - 83.4) | 80.5 | ||||||
(77.6 - 83.2) | 80 | ||||||
(77.1 - 82.7) | 80.7 | ||||||
(77.7 - 83.3) | |||||||
Site | |||||||
2 | Positive | ||||||
Agreement | 91.3 | ||||||
(83 - 95.7) | 90.7 | ||||||
(82 - 95.4) | 54.8 | ||||||
(44.1 - 65) | 56.1 | ||||||
(45.3 - 66.3) | 53.8 | ||||||
(42.9 - 64.3) | 54.7 | ||||||
(43.4 - 65.4) | |||||||
Negative | |||||||
Agreement | 98.4 | ||||||
(97.2 - 99.1) | 98 | ||||||
(96.7 - 98.8) | 86.3 | ||||||
(83.5 - 88.6) | 86.3 | ||||||
(83.6 - 88.7) | 85.9 | ||||||
(83.1 - 88.3) | 85.7 | ||||||
(83 - 88.1) | |||||||
Total | |||||||
Agreement | 97.7 | ||||||
(96.4 - 98.5) | 97.3 | ||||||
(95.9 - 98.2) | 82.9 | ||||||
(80 - 85.3) | 83.1 | ||||||
(80.3 - 85.6) | 82.6 | ||||||
(79.8 - 85.1) | 82.7 | ||||||
(79.9 - 85.2) | |||||||
Site | |||||||
3 | Positive | ||||||
Agreement | 85.5 | ||||||
(75.9 - 91.7) | 87 | ||||||
(77 - 93) | 48 | ||||||
(37.1 - 59.1) | 47.3 | ||||||
(36.3 - 58.5) | 43.4 | ||||||
(32.9 - 54.6) | 46.4 | ||||||
(35.1 - 58) | |||||||
Negative | |||||||
Agreement | 98.6 | ||||||
(97.4 - 99.2) | 98 | ||||||
(96.7 - 98.8) | 90.6 | ||||||
(88.2 - 92.5) | 90.5 | ||||||
(88.1 - 92.4) | 90.1 | ||||||
(87.7 - 92.1) | 90.1 | ||||||
(87.7 - 92.1) | |||||||
Total | |||||||
Agreement | 97.3 | ||||||
(95.9 - 98.2) | 97 | ||||||
(95.6 - 98) | 86.5 | ||||||
(83.9 - 88.7) | 86.3 | ||||||
(83.7 - 88.6) | 85.6 | ||||||
(82.9 - 87.9) | 86.2 | ||||||
(83.6 - 88.5) |
Table 59: Method Comparison of Method A, B and C - Single Reader Agreements
49
Image /page/49/Picture/1 description: The image shows the logo for AESKU.DIAGNOSTICS. The logo consists of a green square with a white image inside, followed by the text "AESKU.DIAGNOSTICS" in gray. Below the text is the phrase "THE DIAGNOSTIC TOOL THAT WORKS" in a smaller, lighter gray font. The image is a professional and modern design.
Table 60: Method Comparison of Method A, B and C – Combined Reader Agreements
| % Agreement (95%
CI) | | Method C vs
B | Method B vs
A | Method C vs
A |
|-------------------------|-----------------------|-----------------------|-----------------------|-----------------------|
| Site 1 | Positive
Agreement | 85.3
(79 - 89.9) | 51.2
(43.7 - 58.7) | 49.7
(42.1 - 57.3) |
| | Negative
Agreement | 98.1
(97.2 - 98.7) | 84.2
(82.2 - 86) | 83.9
(81.9 - 85.8) |
| | Total
Agreement | 96.7
(95.7 - 97.5) | 80.7
(78.6 - 82.6) | 80.3
(78.3 - 82.2) |
| Site 2 | Positive
Agreement | 91
(85.4 - 94.5) | 55.4
(47.8 - 62.8) | 54.2
(46.3 - 61.8) |
| | Negative
Agreement | 98.2
(97.4 - 98.8) | 86.3
(84.4 - 88) | 85.8
(83.9 - 87.6) |
| | Total
Agreement | 97.5
(96.6 - 98.2) | 83
(81 - 84.8) | 82.7
(80.7 - 84.5) |
| Site 3 | Positive
Agreement | 86.2
(79.7 - 90.9) | 47.7
(39.8 - 55.6) | 44.8
(37 - 53) |
| | Negative
Agreement | 98.3
(97.5 - 98.9) | 90.5
(88.9 - 91.9) | 90.1
(88.5 - 91.6) |
| | Total
Agreement | 97.2
(96.2 - 97.9) | 86.4
(84.6 - 88) | 85.9
(84.1 - 87.5) |
50
Image /page/50/Picture/1 description: The image shows the logo for AESKU.DIAGNOSTICS. The logo consists of a green square with a white symbol inside, followed by the text "AESKU.DIAGNOSTICS" in gray. Below the text, there is a tagline that reads "THE DIAGNOSTIC TOOL THAT WORKS" in a smaller, lighter gray font. The logo appears to be for a company that specializes in diagnostic tools.
July 28, 2017
Table 61: Method comparison Number and percent positive samples in the different tested diagnosis
Cross Reactivity | Site 1 | Site 2 | Site 3 | ||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Manual Performance | Reader Confirmation at HELIOS | HELIOS suggestion | Manual Performance | Reader Confirmation at HELIOS | HELIOS suggestion | Manual Performance | Reader Confirmation at HELIOS | HELIOS suggestion | |||||||||||||||||||||||
Diagnosis | n | Reader 1 | |||||||||||||||||||||||||||||
n Pos | % Pos | Reader 2 | |||||||||||||||||||||||||||||
n Pos | % Pos | Reader 1 | |||||||||||||||||||||||||||||
n Pos | % Pos | Reader 2 | |||||||||||||||||||||||||||||
n Pos | % Pos | n Pos | % Pos | Reader 3 | |||||||||||||||||||||||||||
n Pos | % Pos | Reader 4 | |||||||||||||||||||||||||||||
n Pos | % Pos | Reader 3 | |||||||||||||||||||||||||||||
n Pos | % Pos | Reader 4 | |||||||||||||||||||||||||||||
n Pos | % Pos | n Pos | % Pos | Reader 5 | |||||||||||||||||||||||||||
n Pos | % Pos | Reader 6 | |||||||||||||||||||||||||||||
n Pos | % Pos | Reader 5 | |||||||||||||||||||||||||||||
n Pos | % Pos | Reader 6 | |||||||||||||||||||||||||||||
n Pos | % Pos | n Pos | % Pos | ||||||||||||||||||||||||||||
SLE | 297 | 62 | 20.9 | 59 | 19.9 | 63 | 21.2 | 60 | 20.2 | 61 | 20.5 | 67 | 22.6 | 62 | 20.9 | 63 | 21.2 | 67 | 22.6 | 65 | 21.9 | 65 | 21.9 | 56 | 18.9 | 57 | 19.2 | 57 | 19.2 | 45 | 15.2 |
APS | 12 | 0 | 0.0 | 0 | 0.0 | 1 | 8.3 | 0 | 0.0 | 5 | 41.7 | 0 | 0.0 | 0 | 0.0 | 1 | 8.3 | 0 | 0.0 | 5 | 41.7 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 |
Sjögren | 30 | 0 | 0.0 | 0 | 0.0 | 1 | 3.3 | 1 | 3.3 | 10 | 33.3 | 0 | 0.0 | 0 | 0.0 | 1 | 3.3 | 0 | 0.0 | 4 | 13.3 | 0 | 0.0 | 0 | 0.0 | 1 | 3.3 | 1 | 3.3 | 4 | 13.3 |
SSc | 44 | 1 | 2.3 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 6 | 13.6 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 7 | 15.9 | 0 | 0.0 | 0 | 0.0 | 1 | 2.3 | 1 | 2.3 | 7 | 15.9 |
Myositis | 31 | 1 | 3.2 | 1 | 3.2 | 1 | 3.2 | 1 | 3.2 | 6 | 19.4 | 1 | 3.2 | 1 | 3.2 | 1 | 3.2 | 2 | 6.5 | 4 | 12.9 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 1 | 3.2 |
RA | 58 | 8 | 13.8 | 7 | 12.1 | 7 | 12.1 | 8 | 13.8 | 15 | 25.9 | 6 | 10.3 | 6 | 10.3 | 7 | 12.1 | 6 | 10.3 | 12 | 20.7 | 5 | 8.6 | 8 | 13.8 | 7 | 12.1 | 6 | 10.3 | 7 | 12.1 |
MCTD | 24 | 2 | 8.3 | 2 | 8.3 | 3 | 12.5 | 2 | 8.3 | 10 | 41.7 | 2 | 8.3 | 2 | 8.3 | 3 | 12.5 | 2 | 8.3 | 8 | 33.3 | 2 | 8.3 | 1 | 4.2 | 2 | 8.3 | 2 | 8.3 | 4 | 16.7 |
UCTD | 21 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 2 | 9.5 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 3 | 14.3 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 2 | 9.5 |
AIH | 21 | 2 | 9.5 | 2 | 9.5 | 2 | 9.5 | 2 | 9.5 | 5 | 23.8 | 2 | 9.5 | 2 | 9.5 | 2 | 9.5 | 1 | 4.8 | 3 | 14.3 | 1 | 4.8 | 0 | 0.0 | 2 | 9.5 | 2 | 9.5 | 3 | 14.3 |
AIH/PBC | 9 | 0 | 0.0 | 0 | 0.0 | 1 | 11.1 | 0 | 0.0 | 2 | 22.2 | 0 | 0.0 | 0 | 0.0 | 1 | 11.1 | 1 | 11.1 | 1 | 11.1 | 0 | 0.0 | 1 | 11.1 | 0 | 0.0 | 1 | 11.1 | 1 | 11.1 |
PBC | 11 | 3 | 27.3 | 0 | 0.0 | 3 | 27.3 | 3 | 27.3 | 4 | 36.4 | 0 | 0.0 | 0 | 0.0 | 3 | 27.3 | 1 | 9.1 | 4 | 36.4 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 1 | 9.1 |
PSC | 10 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 3 | 30.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 |
Vasculitis | 44 | 5 | 11.4 | 5 | 11.4 | 2 | 4.5 | 2 | 4.5 | 11 | 25.0 | 1 | 2.3 | 1 | 2.3 | 2 | 4.5 | 1 | 2.3 | 4 | 9.1 | 2 | 4.5 | 1 | 2.3 | 2 | 4.5 | 2 | 4.5 | 6 | 13.6 |
EBV | 25 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 3 | 12.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 1 | 4.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 3 | 12.0 |
HBV | 34 | 2 | 5.9 | 0 | 0.0 | 1 | 2.9 | 1 | 2.9 | 1 | 2.9 | 1 | 2.9 | 1 | 2.9 | 1 | 2.9 | 1 | 2.9 | 4 | 11.8 | 1 | 2.9 | 0 | 0.0 | 2 | 5.9 | 1 | 2.9 | 5 | 14.7 |
HCV | 44 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 3 | 6.8 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 5 | 11.4 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 2 | 4.5 |
Fibromyalgia | 30 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 3 | 10.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 1 | 3.3 | 0 | 0.0 | 0 | 0.0 | 4 | 13.3 |
Leukemia | 31 | 1 | 3.2 | 0 | 0.0 | 0 | 0.0 | 1 | 3.2 | 5 | 16.1 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 8 | 25.8 | 0 | 0.0 | 1 | 3.2 | 1 | 3.2 | 1 | 3.2 | 7 | 22.6 |
51
| | % Diagnostic Sensitivity &
Specificity (95% CI) | | % Sensitivity
(95% CI) | % Specificity
(95% CI) | | % Sensitivity
(95% CI) | % Specificity
(95% CI) |
|--------|----------------------------------------------------|----------|---------------------------|---------------------------|---------------------|---------------------------|---------------------------|
| | | | SLE (n= 297) | OD (n=479) | | SLE (n= 297) | OD (n=479) |
| | Manual | Reader 1 | 20.9
(16.6 - 25.9) | 94.8
(92.4 - 96.4) | Combined
Readers | 20.4
(17.3 - 23.8) | 95.6
(94.1 - 96.7) |
| | | Reader 2 | 19.9
(15.7 - 24.8) | 96.5
(94.4 - 97.8) | | | |
| Site 1 | Reader
Confirmation
at HELIOS | Reader 1 | 21.2
(16.9 - 26.2) | 95.4
(93.1 - 96.9) | Combined
Readers | 20.7
(17.6 - 24.1) | 95.5
(94 - 96.7) |
| | | Reader 2 | 20.2
(16 - 25.1) | 95.6
(93.4 - 97.1) | | | |
| | HELIOS | | 20.5
(16.3 - 25.5) | 81
(77.2 - 84.3) | | | |
| | | | | | | | |
| | | | | | | | |
| Site 2 | Manual | Reader 3 | 22.6
(18.2 - 27.6) | 97.3
(95.4 - 98.4) | Combined
Readers | 21.7
(18.6 - 25.2) | 97.3
(96.1 - 98.1) |
| | | Reader 4 | 20.9
(16.6 - 25.9) | 97.3
(95.4 - 98.4) | | | |
| | Reader
Confirmation
at HELIOS | Reader 3 | 21.2
(16.9 - 26.2) | 95.4
(93.1 - 96.9) | Combined
Readers | 21.9
(18.7 - 25.4) | 96.1
(94.7 - 97.2) |
| | | Reader 4 | 22.6
(18.2 - 27.6) | 96.9
(94.9 - 98.1) | | | |
| | HELIOS | | 21.9
(17.6 - 26.9) | 84.1
(80.6 - 87.1) | | | |
| Site 3 | Manual | Reader 5 | 21.9
(17.6 - 26.9) | 97.7
(95.9 - 98.7) | Combined
Readers | 20.4
(17.3 - 23.8) | 97.5
(96.3 - 98.3) |
| | | Reader 6 | 18.9
(14.8 - 23.7) | 97.3
(95.4 - 98.4) | | | |
| | Reader
Confirmation
at HELIOS | Reader 5 | 19.2
(15.1 - 24.1) | 97.3
(95.4 - 98.4) | Combined
Readers | 19.2
(16.2 - 22.6) | 96.9
(95.6 - 97.8) |
| | | Reader 6 | 19.2
(15.1 - 24.1) | 96.5
(94.4 - 97.8) | | | |
| | HELIOS | | 15.2
(11.5 - 19.7) | 88.1
(84.9 - 90.7) | | | |
Table 62: Diagnostic Sensitivity and Specificity
Summary of Results:
Agreements for Method Comparison between Method C (Manual) and Method B (Reader Confirmation of HELIOS images) were > 85% and acceptance criteria were met.
Positive Agreement of Method C vs B ranged from 85.3% to 91%.
Negative Agreement of Method C vs B ranged 98.1% to 98.3%.
Overall Agreement of Method C vs B ranged from 96.7% to 97.5%.
The results show that Methods C (Manual) and B (Reader Confirmation of HELIOS images) give equivalent results. Agreements were > 85%. Furthermore, result of the 3 study site were comparable with each other. We accept an agreement of 85% for AESKUSLIDES nDNA in this study, since the analysis of nDNA slides is very subjective and strongly depends on the experience of the reader.
52
Image /page/52/Picture/1 description: The image shows the logo for AESKU.DIAGNOSTICS. The logo features a green square with a white image inside. To the right of the square is the text "AESKU.DIAGNOSTICS" in gray. Below the text is the phrase "THE DIAGNOSTIC TOOL THAT WORKS" in a smaller, lighter gray font.
Agreements for Comparison between Method B (Reader Confirmation of HELIOS images) and Method A (HELIOS positive/negative classification by the DNA Pattern Plus software) were as follows:
Positive Agreement of Method B vs A ranged from 47.7% to 55.4%.
Negative Agreement of Method B vs A ranged from 84.2% to 90.5%.
Overall Agreement of Method B vs A ranged from 80.3% to 85.9%.
According to this acceptance criteria are not met for comparison of Method B vs A in this study. The same holds true for the comparison of Method C (Manual) vs A (HELIOS suggested results).
Positive Agreement of Method C vs A ranged from 44.8% to 54.2%.
Negative Agreement of Method C vs A ranged from 83.9% to 90.1%.
Overall Agreement of Method C vs A ranged from 80.3% to 85.9%.
The reason why acceptance criteria were not met for comparison of Methods B vs A and C vs A is due to the fact that Method A (HELIOS DNA Pattern Plus software) detected only around 50% of the positive samples that had been detected by the readers at the microscope and at the screen.
However, if one takes into account the results of the calculated sensitivities and specificities for the different study sites and methods it can be seen, that the results for sensitivities are very similar between the different methods and study sites:
At site 1, sensitivities ranged from 20.4% to 20.7%.
At site 2. sensitivities ranged from 21.7% to 21.9%.
At site 2, sensitivities ranged from 15.2% to 20.4%.
All sensitivities were around 20% except for site 3, Method A. where only 15.2% were achieved.
Overall these results show, that Method A, B, and C give comparable results for the diagnostic sensitivity. The reason why the positive agreement between Method B vs A and Method C vs A is only around 50% is that in total, the DNA Pattern Plus software detected a similar number of positive samples in the SLE group, but 50% of those were different samples than those detected by the readers in Method B and C.
Furthermore, the calculated specificities for Method C and B were >95% for all three study sites. The specificity for Method A was slightly lower, it ranged from 81% to 88.1%. This is due to the fact that the DNA Pattern Plus Software detected more "false positives" in the non-healthy control group.
The described results are also reflected by the Cross Reactivity Table. Overall, for Method C and B there were only few crossreactivities which is also reflected by the high specificity of this methods (>95%). The most crossreactivities could be seen in RA (up to
"The e-Copy is an exact duplicate of the paper copy"
53
Image /page/53/Picture/2 description: The image is a logo for AESKU.DIAGNOSTICS. The logo features a green square with a white symbol inside, followed by the text "AESKU.DIAGNOSTICS" in gray. Below the text, there is a tagline that reads "THE DIAGNOSTIC TOOL THAT WORKS" in a smaller, lighter gray font. The logo is clean and professional, with a focus on the company's name and its diagnostic tools.
13.8%), MCTD (up to 12.5%), in the Autoimmune Liver Disease Group (AIH, PBC, and PCS: up to 11.8%), and in Vasculitis (up to 11.4%). These results reflect values reported in the literature. Method A has a lower specificity as described and therefore, higher crossreactivities can be seen.
Conclusion:
Method C and Method B show substantial equivalence. AESKUSLIDES nDNA performs substantially equivalent when processed manually and on HELIOS with Reader Confirmation of HELIOS images, as shown by the method comparison study, the calculated diagnostic sensitivities and specificities, and the cross reactivity table.
For Method A in summary, when single samples were compared to each other like has been done in the method comparison described, we found a lower agreement of Method A, when compared to Method B and C. But when diagnostic sensitivities are compared, results are comparable between all three Methods. It has been shown that with Method A additional samples could be detected in the SLE group, compared to Method B and C, with a slightly lower specificity.
Our results show that results generated at HELIOS must always be confirmed by trained readers. The DNA Pattern Plus software can support the reader by providing additional information, and might encourage to recheck further samples.
Raw data see Attachment 16 Section 12 Raw data Method comparison of Method A, B, and C and Clinical Study
c. Endpoint Titer Comparison AESKUSLIDES nDNA (Crithidia luciliae)
To show that titration of samples results in the same endpoint titer when assayed manually and on HELIOS.
Procedure:
Five serum samples were assayed in duplicates and analyzed on HELIOS (Method B, Reader Confirmation) and manually at the microscope (Method C) by two independent readers:
| Sample ID | Pos/Neg | Expected Endpoint
Titer |
|-----------|---------|----------------------------|
| 1 | Pos | 1280 |
| 2 | Pos | 640 |
| 3 | Pos | 40 |
| 4 | Pos | 320 |
| 5 | Pos | 80 |
54
Image /page/54/Picture/1 description: The image contains text that describes a submission related to AESKUSLIDES nDNA (Crithidia luciliae). The text indicates that it is a 510(k) Traditional Submission. The text is written in a clear, sans-serif font and is horizontally aligned.
Image /page/54/Picture/2 description: The image contains the logo for AESKU.DIAGNOSTICS. The logo features a green square with a white symbol inside, resembling a stylized "Y" with horizontal lines. To the right of the square, the text "AESKU.DIAGNOSTICS" is displayed in gray, with the word "AESKU" being larger and bolder than "DIAGNOSTICS". Below the company name, the tagline "THE DIAGNOSTIC TOOL THAT WORKS" is written in a smaller, lighter gray font.
All assays were performed according to the respective IFUs.
Determination of endpoint titer:
The reciprocal of the highest dilution in which both replicates were reported as positive was defined as the endpoint titer of the sample.
Example: Highest dilution in which both replicates are detected as positive = 1:160
=> Endpoint Titer = 160
Acceptance Criteria:
The reported endpoint titer of a sample is allowed to differ maximum 1 titer level between Method B and C and Reader 1 and 2.
Results:
Table 63: Endpoint titer comparison
Sample ID | Method B | Method C | ||
---|---|---|---|---|
Reader 1 | Reader 2 | Reader 1 | Reader 2 | |
Serum 1 | 1280 | 1280 | 1280 | 1280 |
Serum 2 | 640 | 1280 | 1280 | 1280 |
Serum 3 | 80 | 80 | 40 | 40 |
Serum 4 | 160 | 160 | 320 | 320 |
Serum 5 | 80 | 80 | 40 | 80 |
The reported endpoint titers for the tested sera differ maximally 1 titer level between the different methods and readers.
Raw data see Attachment 17_Section 12_ Raw data Endpoint titer comparison
Conclusion:
Acceptance criteria were met. Sera titrated on HELIOS and manually result in comparable endpoint titers.
3. Clinical cut-off:
See assay cut-off
55
Image /page/55/Picture/1 description: The image contains text that describes a submission related to AESKUSLIDES nDNA (Crithidia luciliae). The text indicates that it is a 510(k) Traditional Submission. The text is left-aligned and appears to be part of a document or presentation slide.
Image /page/55/Picture/2 description: The image is a logo for AESKU.DIAGNOSTICS. The logo features a green square with a white image inside of what appears to be a vest with a rope around it. To the right of the square is the company name in gray, with the words "THE DIAGNOSTIC TOOL THAT WORKS" in a smaller font below.
4. Expected values/Reference range
Normal Range Study AESKUSLIDES nDNA (Crithidia luciliae)
Expected values for nDNA were analyzed with a panel of 164 sera from healthy donors: 114 from Germany (63 females, 51 males, mean age 32 years, range of 18-58 years, Ethnicity: German) and 50 from the US (17 females, 33 males, mean age 34 years, range of 19-62 years, Ethnicity: 30 Black, 10 Hispanic, 10 White).
Slides were processed manually according to the IFU and subsequently analyzed at the microscope by two independent readers.
Results:
With AESKUSLIDES nDNA, there was 1 positive result found by reader 1 and no positive result found by reader 2.
Table 64: Normal Range Study Number and Percent positive and negative samples:
| Normal Range
Study | Pos (%) | Neg (%) | Total |
---|---|---|---|
Reader 1 | 1 (0.6) | 163 (99.4) | 164 |
Reader 2 | 0 (0) | 164 (100) | 164 |
Table 65: Normal Range Study Positive samples
Normal Range Study | Reader 1 | Reader 2 | |||
---|---|---|---|---|---|
Sample ID | Sample Origin | Result | FI | Result | FI |
NC 072 | Germany | Pos | 1 | Neg | 0 |
FI = Fluorescence Intensitiy
Only one sample (NC 072) has been found positive in the AESKUSLIDES nDNA assay (FI = 1) by reader 1.
Raw data see Attachment 18_Section 12_Raw data Normal Range Study
Conclusion:
The low number of positive samples we found with AESKUSLIDES nDNA in the tested healthy population correlate well with numbers reported in the literature.
56
Image /page/56/Picture/1 description: The image shows the logo for AESKU.DIAGNOSTICS. The logo features a green square with a white symbol inside, followed by the text "AESKU.DIAGNOSTICS" in gray. Below the company name, there is a tagline that reads "THE DIAGNOSTIC TOOL THAT WORKS" in a smaller, lighter gray font.
N. Instrument Name
HELIOS AUTOMATED IFA SYSTEM
O. System Description
The HELIOS AUTOMATED IFA SYSTEM is an automated system including pipetting unit with microscope and software that acquires. interprets. stores and displays digital images of stained indirect immunofluorescence slides. The HELIOS DEVICE SOFTWARE is designed to support input of results from the AESKUSLIDES into electronic laboratory data management systems.
The HELIOS AUTOMATED IFA SYSTEM should only be used with AESKUSLIDES assays that are cleared or approved for use on the instrument. All suggested results obtained with the HELIOS DEVICE SOFTWARE must be confirmed by trained personnel.
1. Modes of Operation:
HELIOS: automated processing with automated imaging and automated reading HELIOS User Eval: automated processing with automated imaging and manual reading HELMED Mode: automated processing with manual imaging and manual reading Manual Mode: manual processing with manual imaging and manual reading
Does the applicant's device contain the ability to transmit data to a computer. webserver. or mobile device?
Yes
Does the applicant's device transmit data to a computer. webserver. or mobile device using wireless transmission?
No
2. Software:
General
The HELIOS AUTOMATED IFA SYSTEM is an automated system for immunofluorescence processing with an integrated camera with an optic (microscope) and software for routine laboratory use by professional users under controlled environmental conditions.
The HELIOS camera with an optic (microscope) automatically focuses on the areas of biological material on the slides. Pictures are stored and evaluated by the HELIOS DEVICE SOFTWARE. The software performs a positive/negative of the cells. The software suggests the result as a qualitative result (positive. negative). All suggested
57
Image /page/57/Picture/1 description: The image is a logo for AESKU.DIAGNOSTICS. The logo features a green square with a white symbol inside, followed by the text "AESKU.DIAGNOSTICS" in gray. Below the text, there is a tagline that reads "THE DIAGNOSTIC TOOL THAT WORKS" in a smaller, lighter gray font.
results obtained with the HELIOS AUTOMATED IFA SYSTEM and the HELIOS DEVICE SOFTWARE must be confirmed by trained personnel before the result is finalized.
Level of Concern
The Level of concern was determined according to the "Guidance for the Content of Premarket Submissions for Software Contained in Medical Devices" dated May 11, 2005. Following the recommendations of this standard. the HELIOS DEVICE SOFTWARE is assigned moderate level of concern.
FDA has reviewed applicant's Hazard Analysis and software development processes for this line of product types.
-
- Specimen Identification: Manual sample identification and/or Barcode
-
- Specimen Sampling and Handling: Not applicable
-
- Calibration: There is no calibration of the instrument by the user.
-
- Quality Control:
Positive and negative controls are supplied with the assay reagents (see assay description above).
P. Other Supportive Instrument Performance Characteristics Data Not Covered In The "Performance Characteristics" Section above:
Not applicable
Q. Proposed Labeling:
-
- AESKUSLIDES nDNA Package Labels: See Section 13
-
- AESKUSLIDES nDNA Package Insert: See Section 13
- HELIOS Label - "For Use with AESKUSLIDES nDNA (Crithidia luciliae)" See Section 13
-
- HELIOS Instruction Manual Draft Section 13
-
- All other Labels see Section 13
58
February 16, 2018
Image /page/58/Picture/1 description: The image contains the logo of the U.S. Food and Drug Administration (FDA). On the left is the Department of Health & Human Services logo. To the right of that is the FDA logo, which is a blue square with the letters "FDA" in white. To the right of the blue square is the text "U.S. FOOD & DRUG ADMINISTRATION" in blue.
Aesku Diagnostics GmbH & Co. KG Sandra Reuter Regulatory Affairs Manager Mikro-Forum-Ring 2 Wendeslheim, 55234 DE
Re: K172348
Trade/Device Name: AESKUSLIDES nDNA (Crithidia luciliae), AESKUSLIDES nDNA (Crithidia luciliae) Demo Kit, AESKUSLIDES nDNA (Crithidia luciliae) Bulk kit x5, AESKUSLIDES nDNA (Crithidia luciliae) Bulk kit x10 Regulation Number: 21 CFR 866.5100 Regulation Name: Antinuclear antibody immunological test system Regulatory Class: Class II Product Code: LSW, PIV Dated: July 28, 2017 Received: August 3, 2017
Dear Sandra Reuter:
We have reviewed your Section 510(k) premarket notification of intent to market the device referenced above and have determined the device is substantially equivalent (for the indications for use stated in the enclosure) to legally marketed predicate devices marketed in interstate commerce prior to May 28, 1976, the enactment date of the Medical Device Amendments, or to devices that have been reclassified in accordance with the provisions of the Federal Food, Drug, and Cosmetic Act (Act) that do not require approval of a premarket approval application (PMA). You may, therefore, market the device, subject to the general controls provisions of the Act. The general controls provisions of the Act include requirements for annual registration, listing of devices, good manufacturing practice, labeling, and prohibitions against misbranding and adulteration. Please note: CDRH does not evaluate information related to contract liability warranties. We remind you, however, that device labeling must be truthful and not misleading.
If your device is classified (see above) into either class II (Special Controls) or class III (PMA), it may be subject to additional controls. Existing major regulations affecting your device can be found in the Code of Federal Regulations, Title 21, Parts 800 to 898. In addition, FDA may publish further announcements concerning your device in the Federal Register.
Please be advised that FDA's issuance of a substantial equivalence determination does not mean that FDA has made a determination that your device complies with other requirements of the Act or any Federal statutes and regulations administered by other Federal agencies. You must comply with all the Act's
59
requirements, including, but not limited to: registration and listing (21 CFR Part 807); labeling (21 CFR Part 801 and Part 809); medical device reporting of medical device-related adverse events) (21 CFR 803); good manufacturing practice requirements as set forth in the quality systems (OS) regulation (21 CFR Part 820); and if applicable, the electronic product radiation control provisions (Sections 531-542 of the Act); 21 CFR 1000-1050.
Also, please note the regulation entitled, "Misbranding by reference to premarket notification" (21 CFR Part 807.97). For questions regarding the reporting of adverse events under the MDR regulation (21 CFR Part 803), please go to http://www.fda.gov/MedicalDevices/Safety/ReportaProblem/default.htm for the CDRH's Office of Surveillance and Biometrics/Division of Postmarket Surveillance.
For comprehensive regulatory information about mediation-emitting products, including information about labeling regulations, please see Device Advice (https://www.fda.gov/MedicalDevices/DeviceRegulationandGuidance/) and CDRH Learn (http://www.fda.gov/Training/CDRHLearn). Additionally, you may contact the Division of Industry and Consumer Education (DICE) to ask a question about a specific regulatory topic. See the DICE website (http://www.fda.gov/DICE) for more information or contact DICE by email (DICE@fda.hhs.gov) or phone (1-800-638-2041 or 301-796-7100).
Sincerely,
For Lea Carrington Director Division of Immunology and Hematology Devices Office of In Vitro Diagnostics and Radiological Health Center for Devices and Radiological Health
Enclosure