(87 days)
FilmArray RP EZ Control Panel M265 is intended for use as an external positive and negative assayed quality control to monitor the performance of in vitro laboratory nucleic acid testing procedures for the qualitative detection of Adenovirus, Coronavirus, Human Metapneumovirus, Human Rhinovirus, Influenza A, Influenza A subtype H1, Influenza A subtype H3, Influenza A subtype H1-2009, Influenza Virus, Respiratory Syncytial Virus, Bordetella pertussis, Chlamydophila pneumoniae, and Mycoplasma pneumoniae on the FilmArray RP EZ assay performed on the FilmArray systems. FilmArray RP Positive Control is composed of synthetic RNA transcripts specifically designed for and intended to be used solely with the FilmArray RP EZ assay. This product is not intended to replace manufacturer controls provided with the device.
FilmArray RP EZ Control Panel M265, P/N M265, is a quality control panel consisting of 2 controls, FilmArray RP EZ Positive Control, P/N M266, and FilmArray RP EZ Negative Control. P/N M267. The Positive Contains non-infectious surrogate control material: a solution of synthetic RNA transcripts in buffers, stabilizers and preservatives. The RNA carries segments of all respiratory pathogens detected by the FilmArray RP EZ assay (Table 1. below) on the FilmArray 2.0 EZ Configuration system. The RNA in the Negative Control is non-specific RNA in buffers, stabilizers and preservatives. Each liquid control of FilmArray RP EZ Control Panel M265 is processed separately according to FilmArray RP EZ assay manufacturer's Instructions for Use for patient samples (nasopharyngeal swabs (NPS) obtained from individuals suspected of respiratory tract infection and placed in Viral Transport Media (VTM)).
The document describes the FilmArray RP EZ Control Panel M265, an assayed quality control material for clinical microbiology assays. The study aims to demonstrate substantial equivalence to a predicate device by evaluating the performance of the control panel.
1. Table of Acceptance Criteria and Reported Device Performance
The acceptance criteria for this device are implied by the "Correct Positive Control Result" and "Correct Negative Control Result" percentages in the performance studies. While explicit numerical thresholds for acceptance are not stated for each study, the consistently high percentages (often 100% or very close to it) demonstrate that the device met the manufacturer's internal performance expectations for these quality control materials.
| Acceptance Criteria (Implied) | Reported Device Performance |
|---|---|
| Positive Control Performance: All respiratory pathogen analytes in the positive control should be correctly detected. | All Test Results Summary (4 sites - Internal & External): 100% Correct Positive Control Result (129/129 valid tests). |
| External Site Testing (3 sites): 100% Correct Positive Control Result (90/90 valid tests). | |
| Within-run Reproducibility: 6/6 correct results for Positive Control Lot A06JAN16A. | |
| Lot-to-Lot Testing: 4/4 correct (M40DEC15A), 3/3 correct (A28DEC15A), 4/4 correct (A06JAN16A after 1 invalid retest). | |
| Precision Testing: 100% Correct Results for all 3 positive control lots (12/12, 13/13, 14/14 valid tests). | |
| Negative Control Performance: No respiratory pathogens should be detected. | All Test Results Summary (4 sites - Internal & External): 98.6% Correct Negative Control (137/139 valid tests) with 2 incorrect results (false positives). |
| External Site Testing (3 sites): 97.8% Correct Negative Control (88/90 valid tests) with 2 incorrect results (false positives, specifically 2 from Site 3). | |
| Within-run Reproducibility: 5/5 correct results for Negative Control Lot M41DEC15A. | |
| Precision Testing: 100% Correct Results for all 3 negative control lots (16/16, 16/16, 17/17 valid tests). | |
| Reproducibility (within-run, lot-to-lot, precision): Consistent and correct results across different testing conditions (lots, operators, days). | All reproducibility tests (within-run, lot-to-lot, and precision) reported "All results were correct and reproducible" or 100% correct results for valid tests. |
2. Sample Sizes Used for the Test Set and Data Provenance
-
All Test Results:
- Total Tests: 271 (3 invalid, 268 included in analysis). This includes both internal and external studies.
- Positive Controls: 129 valid tests.
- Negative Controls: 139 valid tests.
- Provenance: Data was generated from an "internal study" at MMQCI (Saco, Maine) and an "external study" at "2 CLIA-waived sites and 1 external site under simulated CLIA-waived conditions." The study appears to be prospective, as it involved manufacturing and testing according to a pre-defined protocol.
-
External Site Testing:
- Total Tests: 182 (2 invalid, 180 included in analysis).
- Positive Controls: 90 valid tests.
- Negative Controls: 90 valid tests.
- Provenance: Generated from two CLIA-waived clinical settings and one simulated CLIA-waived test setting. Prospective.
-
Reproducibility (Within-run):
- Positive Control: 6 tests.
- Negative Control: 5 tests.
- Provenance: MMQCI (Saco, Maine). Prospective.
-
Reproducibility (Lot-to-Lot):
- Total Tests: 12 (1 invalid, 11 included in analysis).
- Provenance: Not explicitly stated but inferred to be similar to other MMQCI internal testing. Prospective.
-
Precision Testing:
- Total Tests: 89 (1 invalid, 88 included in analysis).
- Positive Controls: 39 valid tests.
- Negative Controls: 49 valid tests.
- Provenance: MMQCI (Saco, Maine). Prospective.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts
This information is not provided in the document. For a quality control material, the "ground truth" is typically inherent in the design and formulation of the control itself (e.g., the positive control is designed to be positive for specific targets, and the negative control is designed to be negative for those targets). The performance of the control is then verified against the expected outcome using the diagnostic assay it's intended to monitor. The document describes how the control panel was formulated (synthetic RNA transcripts for positive control, non-specific RNA for negative control).
4. Adjudication Method for the Test Set
This information is not explicitly provided. The testing results indicate some "invalid" results due to "internal control pouch failures" which were excluded and re-tested. This implies a protocol for handling such technical failures but does not describe an adjudication method by experts for discrepancies in the detection of analytes. Given that this is a quality control device's performance assessment, results are binary (detected/not detected) against a known composition, so expert adjudication for ground truth may not be relevant in the same way as for diagnostic devices analyzing patient samples.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was done
No, an MRMC comparative effectiveness study was not done. This type of study typically assesses human reader performance with and without an AI system for diagnostic tasks. This device is a quality control material, not a diagnostic AI algorithm.
6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) was done
Yes, in essence. The entire study describes the "standalone" performance of the FilmArray RP EZ Control Panel M265 when processed by the FilmArray RP EZ assay on the FilmArray systems. While human operators are involved in running the tests, the performance being measured is of the control material itself, validating its ability to produce expected results. There is no "human-in-the-loop" AI involved here; it's about the control material's interaction with a diagnostic assay.
7. The Type of Ground Truth Used
The ground truth for this device is based on the known synthetic composition of the control panel.
- Positive Control: Composed of synthetic RNA transcripts corresponding to all respiratory pathogens detected by the FilmArray RP EZ assay. The ground truth is that these specific targets should be detected.
- Negative Control: Composed of non-specific RNA. The ground truth is that the target respiratory pathogens should not be detected.
8. The Sample Size for the Training Set
This information is not applicable/not provided in the context of this document. This device is a quality control material, not an algorithm that requires a "training set" for machine learning. The studies described are validation and performance testing of a physical product.
9. How the Ground Truth for the Training Set was Established
This information is not applicable/not provided as there is no "training set" for this quality control material. The ground truth is established by the controlled, synthetic formulation of the positive and negative controls themselves.
§ 866.3920 Assayed quality control material for clinical microbiology assays.
(a)
Identification. An assayed quality control material for clinical microbiology assays is a device indicated for use in a test system to estimate test precision or to detect systematic analytical deviations that may arise from reagent or analytical instrument variation. This type of device consists of single or multiple microbiological analytes intended for use with either qualitative or quantitative assays.(b)
Classification. Class II (special controls). The special controls for this device are:(1) Premarket notification submissions must include detailed device description documentation and information concerning the composition of the quality control material, including, as appropriate:
(i) Analyte concentration;
(ii) Expected values;
(iii) Analyte source;
(iv) Base matrix;
(v) Added components;
(vi) Safety and handling information; and
(vii) Detailed instructions for use.
(2) Premarket notification submissions must include detailed documentation, including line data as well as detailed study protocols and a statistical analysis plan used to establish performance, including:
(i) Description of the process for value assignment and validation.
(ii) Description of the protocol(s) used to establish stability.
(iii) Line data establishing precision/reproducibility.
(iv) Where applicable, assessment of matrix effects and any significant differences between the quality control material and typical patient samples in terms of conditions known to cause analytical error or affect assay performance.
(v) Where applicable, identify or define traceability or relationship to a domestic or international standard reference material and/or method.
(vi) Where applicable, detailed documentation related to studies for surrogate controls.
(3) Premarket notification submissions must include an adequate mitigation (e.g., real-time stability program) to the risk of false results due to potential modifications to the assays specified in the device's 21 CFR 809.10 compliant labeling.
(4) Your 21 CFR 809.10 compliant labeling must include the following:
(i) The intended use of your 21 CFR 809.10(a)(2) and (b)(2) compliant labeling must include the following:
(A) Assayed control material analyte(s);
(B) Whether the material is intended for quantitative or qualitative assays;
(C) Stating if the material is a surrogate control; and
(D) The system(s), instrument(s), or test(s) for which the quality control material is intended.
(ii) The intended use in your 21 CFR 809.10(a)(2) and (b)(2) compliant labeling must include the following statement: “This product is not intended to replace manufacturer controls provided with the device.”
(iii) A limiting statement that reads “Quality control materials should be used in accordance with local, state, federal regulations, and accreditation requirements.”