(92 days)
The FilmArray® Global Fever Panel External Control Kit contains Positive External Controls intended for use as assayed quality controls to monitor the performance of in vitro diagnostic laboratory nucleic acid testing procedures for the qualitative detection of FilmArray® Global Fever Panel targets on FilmArray® 2.0 systems. The Global Fever Panel External Control Kit is designed for and intended to be used solely with the FilmArray Global Fever Panel. This product does not replace manufacturer internal controls provided as part of the Global Fever Panel device.
The FilmArray Global Fever Panel External Control Kit contains Positive and Negative External Controls. The Positive External Control has been optimized to be detected by all pathogen assays contained in the Global Fever Panel (Table 1). The Negative External Control contains no nucleic acid and a successful run will be negative for all assays on the panel. These controls are not intended to replace the internal FilmArray Global Fever Panel pouch controls (RNA process control and second stage PCR array control). The Global Fever Panel External Control Kit contains no biological hazards and is 100% non-infectious.
The External Controls are referenced in the Ouick Guide and product literature as Control Injection Vials. The use of room-temperature stable External Controls contained within an injection vial simplifies the workflow and allows for use of the External Controls in settings where access to refrigeration may be limited. Each individually packaged, ready-to-use FilmArray Global Fever External Control is processed separately according to the Instructions for Use, and follows the procedure as outlined in the Quick Guide for the FilmArray Global Fever External Control Kit. Each External Control Injection Vial is intended for a single use.
The Global Fever Panel External Control Kit is designed to mitigate the risk of control contamination and misuse when evaluating clinical samples on the FilmArray 2.0 System.
- Negative External Controls are tested using the Negative External Control protocol, which monitors for contamination from both external control material and target pathogens; it will fail if either is detected.
- The Positive External Contains DNA sequences that produce signature amplicon melting temperature (Tm) values distinct from the amplicon Tm values produced by each of the pathogens detected by the Global Fever Panel. By design, the Positive ECM will not be detected when using the Global Fever Panel Whole Blood Protocol, and reciprocally, amplified pathogen-specific nucleic acid will not be detected when using the Positive External Control Protocol. Through modification of the sequence between the inner primers for each ECM target, the Tm value of the amplicon is shifted to higher or lower Tm values relative to the expected Global Fever Panel target amplicon while running the same Global Fever Panel pouches with the same physical pouch manipulation in the FilmArray 2.0 Instrument. Positive External Control-specific pouch module software detects the expected shifted Tm values as being from ECM amplicon, thereby evaluating the performance of the FilmArray 2.0 System. Also, the modification of the ECM sequence mitigates possible contamination events and does not cause false positives in clinical samples. In the unlikely event that Positive ECM or ECM amplicon is introduced into a patient sample, the resulting amplification Tm value(s) is not detected within the pathogen(s) Tm window in the Global Fever Panel Whole Blood Protocol. where different Tm windows are used to detect amplified pathogen sequence.
Here's an analysis of the provided text regarding the acceptance criteria and study for the FilmArray Global Fever Panel External Control Kit:
Note: This document describes an external control kit for an in vitro diagnostic device, not a diagnostic device itself. As such, some of the typical criteria for diagnostic devices (like sensitivity, specificity for patient outcomes, or MRMC studies) are not directly applicable or reported in the same way. The focus here is on the performance of the control kit in monitoring the primary diagnostic device.
Acceptance Criteria and Device Performance for FilmArray Global Fever Panel External Control Kit
1. Table of Acceptance Criteria and Reported Device Performance
| Acceptance Criterion (Implicit) | Reported Device Performance (FilmArray Global Fever Panel External Control Kit) |
|---|---|
| Clinical Performance (Field Use) | |
| Positive External Control "Passed" Rate ≥ 95% (based on expected results) | 99.4% (159/160 completed with expected result, one user error suspected for the failed case) |
| Negative External Control "Passed" Rate ≥ 95% (based on expected results) | 98.7% (155/157 completed with expected result, two failed; one identified pathogen contamination, one user error suspected). |
| Overall External Control "Passed" Rate ≥ 95% (based on expected results) | 99.1% (314/317) - Calculated from the percentages for positive and negative controls provided. The document also states 98.7% separately for the sum, likely indicating the sum of successful passes (313) from total (317). |
| Repeatability (Within-Lab Consistency) | |
| Positive External Control "Passed" Rate = 100% | 100% (45/45) |
| Negative External Control "Passed" Rate = 100% | 100% (45/45) |
| Positive External Control Tm Standard Deviation (implicitly low) | 0.1-0.2°C (for all target assays) |
| Positive External Control Coefficient of Variation (CV) (implicitly low) | 0.1-0.3% (for all target assays) |
| Multi-Site Reproducibility (Between-Lab Consistency) | |
| Positive External Control "Passed" Rate ≥ 95% for each site and overall | Site 1: 93.3% (42/45 - one site below 95%) |
| Site 2: 100% (45/45) | |
| Site 3: 100% (45/45) | |
| All Sites Overall: 97.8% (132/135 - meets ≥95% requirement) | |
| Negative External Control "Passed" Rate ≥ 95% for each site and overall | Site 1: 100% (45/45) |
| Site 2: 97.8% (44/45) | |
| Site 3: 97.8% (44/45) | |
| All Sites Overall: 98.5% (133/135 - meets ≥95% requirement) | |
| Overall Agreement with Expected Result for both Positive and Negative Controls Across all sites | 98.1% (265/270), 95% CI [95.7-99.2%] |
| Negative External Control detecting contamination (implicitly, ability to detect contamination) | Identified one instance of pathogen contamination out of 157 tests in clinical testing. In reproducibility, two failures due to pathogen detection (not ECM contamination), suspected laboratory contamination. (Demonstrates ability to detect contamination as intended.) |
| Positive External Control not causing false positives in clinical samples (implicitly, design efficacy) | The modification of the ECM sequence mitigates possible contamination events and does not cause false positives in clinical samples. In the unlikely event of ECM introduction into a patient sample, resulting amplification Tm values are not detected within pathogen Tm windows. (Design feature to prevent false positives in clinical samples due to the control material itself.) |
Explanations of Implicit Criteria:
- For a quality control material, the primary acceptance criteria are its ability to consistently produce the expected positive or negative results, and for negative controls, its ability to detect contamination.
- The document implies that a "passed" rate of ≥95% is acceptable for field performance and multi-site reproducibility. For repeatability, 100% pass rates are shown, indicating a high standard for within-lab consistency.
- Low Tm standard deviation and CV for positive controls are implicit acceptance criteria for consistent amplification and melting characteristics.
2. Sample Size Used for the Test Set and Data Provenance
- Clinical Testing (Field Use):
- Sample Size: 317 External Control tests (160 Positive, 157 Negative).
- Data Provenance: Prospective, collected between July 2019 and January 2020, from "six sites" (implied to be clinical laboratories). The country of origin is not explicitly stated but assumed to be the US based on FDA submission.
- Repeatability:
- Sample Size: 90 External Control tests (45 Positive, 45 Negative).
- Data Provenance: In-house study, one operator, one kit lot, one instrument, over 14 days. Retrospective based on experimental design.
- Multi-Site Reproducibility:
- Sample Size: 270 External Control tests (135 Positive, 135 Negative).
- Data Provenance: Multi-site study involving "three sites," "three External Control kits," "three reagent pouch lots," "two operators and three FilmArray 2.0 instruments at each site." This is a prospective experimental study.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts
- This device is an external control kit, not a diagnostic device for patient samples. Therefore, the "ground truth" is based on the expected output of the control material (i.e., Positive Control = "Passed" with expected amplicon melts, Negative Control = "Passed" with no amplicon melts/no pathogen detection).
- The "truth" is inherent in the design of the control material (synthetic DNA designed to produce specific signals or no signal) and the proper functioning of the FilmArray 2.0 system. No human experts were used to establish ground truth in the traditional sense of disease diagnosis for these control materials. The system itself determines "passed" or "failed" based on pre-programmed criteria for the control.
4. Adjudication Method for the Test Set
- Not applicable in the traditional sense. The device (FilmArray 2.0 System) with its specific "External Control-specific protocols" internally adjudicates whether a run "passed" or "failed" based on expected amplicon melt curves (for positive) or absence of melts and pathogen detection (for negative). The results are "overall passed or failed results."
- For the clinical testing study, if a control failed, the site "immediately tested a new External Control and obtained the expected result," suggesting internal site-level re-testing but not a formal expert adjudication of the initial failed result. The cause of failure (e.g., user error, contamination) was noted where apparent.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance
- No MRMC study was done. This is an in vitro diagnostic control kit, not an AI-assisted diagnostic imaging or pathology device. The "reader" here is the automated FilmArray 2.0 system. The study focuses on the performance and reliability of the control kit itself.
6. If a Standalone (i.e. algorithm only without human-in-the-loop performance) was done
- The performance data presented could be considered standalone algorithm performance in the context of the FilmArray 2.0 instrument processing the control material. The instrument's software interprets the results of the control runs and determines if they "passed" or "failed" based on programmed criteria for the specific control and panel. Human intervention is primarily in performing the test, not in interpreting the primary outcome of the control run.
7. The Type of Ground Truth Used
- The ground truth used is expert design specification and expected performance for a quality control material.
- Positive Controls: Designed with "dried synthetic DNA segments" to produce expected amplicon melting temperature (Tm) values, specific for the External Control melt range and distinct from pathogen Tm values. Ground truth is that these segments should amplify and produce these specific Tm values.
- Negative Controls: Designed to contain "no DNA" and be non-reactive. Ground truth is that these should not produce any amplification or pathogen detection.
- This is not pathology, outcomes data, or expert consensus on clinical findings, but rather a verification that the control material itself performs as designed and intended to monitor the system's performance.
8. The Sample Size for the Training Set
- Not applicable / No specific training set mentioned. For an in vitro diagnostic control kit, the "training" typically involves the development and calibration of the control material and the associated instrument software during the design phase. The document describes verification and validation studies (clinical, repeatability, reproducibility), which assess the final product's performance, rather than a separate training set for a machine learning algorithm.
9. How the Ground Truth for the Training Set Was Established
- Not applicable. As the device is not an AI/ML algorithm that undergoes a training process with a specific ground truth dataset, this question does not directly apply. The "ground truth" for the development of the control kit and its interpretation by the FilmArray system would have been established internally during product development based on molecular biology principles, assay design, synthetic DNA characteristics, and instrument calibration standards.
§ 866.3920 Assayed quality control material for clinical microbiology assays.
(a)
Identification. An assayed quality control material for clinical microbiology assays is a device indicated for use in a test system to estimate test precision or to detect systematic analytical deviations that may arise from reagent or analytical instrument variation. This type of device consists of single or multiple microbiological analytes intended for use with either qualitative or quantitative assays.(b)
Classification. Class II (special controls). The special controls for this device are:(1) Premarket notification submissions must include detailed device description documentation and information concerning the composition of the quality control material, including, as appropriate:
(i) Analyte concentration;
(ii) Expected values;
(iii) Analyte source;
(iv) Base matrix;
(v) Added components;
(vi) Safety and handling information; and
(vii) Detailed instructions for use.
(2) Premarket notification submissions must include detailed documentation, including line data as well as detailed study protocols and a statistical analysis plan used to establish performance, including:
(i) Description of the process for value assignment and validation.
(ii) Description of the protocol(s) used to establish stability.
(iii) Line data establishing precision/reproducibility.
(iv) Where applicable, assessment of matrix effects and any significant differences between the quality control material and typical patient samples in terms of conditions known to cause analytical error or affect assay performance.
(v) Where applicable, identify or define traceability or relationship to a domestic or international standard reference material and/or method.
(vi) Where applicable, detailed documentation related to studies for surrogate controls.
(3) Premarket notification submissions must include an adequate mitigation (e.g., real-time stability program) to the risk of false results due to potential modifications to the assays specified in the device's 21 CFR 809.10 compliant labeling.
(4) Your 21 CFR 809.10 compliant labeling must include the following:
(i) The intended use of your 21 CFR 809.10(a)(2) and (b)(2) compliant labeling must include the following:
(A) Assayed control material analyte(s);
(B) Whether the material is intended for quantitative or qualitative assays;
(C) Stating if the material is a surrogate control; and
(D) The system(s), instrument(s), or test(s) for which the quality control material is intended.
(ii) The intended use in your 21 CFR 809.10(a)(2) and (b)(2) compliant labeling must include the following statement: “This product is not intended to replace manufacturer controls provided with the device.”
(iii) A limiting statement that reads “Quality control materials should be used in accordance with local, state, federal regulations, and accreditation requirements.”