(103 days)
Amplichek II is intended for use as an external assayed quality control material to monitor the performance of in vitro laboratory nucleic acid testing procedures for the qualitative detection of Methicillin Resistant Staphylococcus aureus, Methicillin Sensitive Staphylococcus aureus, Clostridium difficile and Vancomycin-resistant Enterococci performed on Cepheid GeneXpert Systems. This product is not intended to replace manufacturer controls provided with the device. This product is only for use with assays and instruments listed in the Representative Results Chart in this labeling.
Amplichek II (Assayed Microbiology Control) is manufactured at three levels, Levels 1, 2 and 3, for each analyte indicated in the package insert. Individual analyte values are listed in the package insert and are specific for the instrument system or method utilized. Each control is prepared in liquid form in a buffer solution with preservatives including 5chloro-2-methyl-2H-isothiazol-3-one at a concentration of 0.1%, stabilizers and added preparations of purified intact microorganisms grown in microbial culture. Source materials are chemically treated and processed to inactivate infectious agents.
The Amplichek II is an assayed quality control material for clinical microbiology assays. Its purpose is to monitor the performance of in vitro laboratory nucleic acid testing procedures for the qualitative detection of Methicillin-Resistant Staphylococcus aureus (MRSA), Methicillin-Sensitive Staphylococcus aureus (MSSA), Clostridium difficile (Cdiff), and Vancomycin-resistant Enterococci (VRE) on Cepheid GeneXpert Systems.
Here's an analysis of the acceptance criteria and the study that proves the device meets them:
1. Table of Acceptance Criteria and Reported Device Performance
The primary acceptance criteria for the Amplichek II, as demonstrated by the reproducibility studies, is the percent agreement with the expected result for each analyte at various levels (Negative, Level 1, Level 2, Level 3).
| Acceptance Criteria (Implicit) | Reported Device Performance (Reproducibility Studies) |
|---|---|
| High percent agreement (ideally 100%) for expected positive and negative results across different lots, operators, days, and sites. | Study 1 (Single Lab): 100% agreement for all analytes (SPA, SCCmec, mecA, Binary Toxin, Toxin B, TcdC, Van A) across all Amplichek II levels (Negative, Level 1, Level 2, Level 3) for both product lots (Lot #1 and Lot #2), resulting in 100% total agreement by samples. (e.g., 80/80 agreements) |
| Consistency in Cycle Threshold (Ct) values and low Coefficient of Variation (CV%) across different lots, operators, days, and sites for positive controls. | Study 2 (Three Labs): Ct Mean and Ct %CV provided for positive controls across all analytes and levels for both Lot #1 and Lot #2. The Ct %CV values are generally low, mostly ranging from 1.4% to 9.1%, indicating good consistency. Note that Negative and some Level 1 controls had "N/A" for Ct values as they were expected to be negative. |
| Stability of the material until the expiration date when stored correctly. | An accelerated stability study was performed to establish shelf-life stability claims. Real-time stability studies are ongoing to support product claims. The protocols and acceptance criteria for these studies were reviewed and found acceptable. |
| The labeling is sufficient and satisfies regulatory requirements. | The labeling was deemed sufficient and satisfies the requirements of 21 CFR Parts 801 and 809 and the special controls for this device type. |
2. Sample Size Used for the Test Set and Data Provenance
The document describes two reproducibility studies acting as the test sets:
-
Study 1 (Single Lab):
- Sample Size: For each gene/analyte at each Amplichek II level (Negative, Level 1, Level 2, Level 3), there were 40 tests per lot (2 replicates per run x 2 runs per day x 10 days). With two lots, this totals 80 tests per gene/analyte/level. For instance, for the Cepheid Xpert SA Nasal Complete (MSSA/MRSA), there were 80 tests for SPA, 80 for SCCmec, and 80 for mecA at the Negative level, and similarly for Level 1, Level 2, and Level 3.
- Data Provenance: The data was generated through prospective testing at one laboratory site in the United States (implied by the FDA De Novo application and regulatory context). The study incorporated different operators and different days.
-
Study 2 (Three Labs):
- Sample Size: For each gene/analyte at each Amplichek II level, there were 9 tests per lot (different operators x 3 different days). With two lots, this totals 18 tests per gene/analyte/level.
- Data Provenance: The data was generated through prospective testing at three laboratory sites in the United States (implied by the FDA De Novo application and regulatory context). The study incorporated different operators and different days.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications
The Amplichek II is a quality control material intended to monitor the performance of in vitro laboratory nucleic acid testing. The "ground truth" for the test results (i.e., whether a control should be positive or negative for a specific analyte and its approximate Ct value range) is inherent to the design and composition of the Amplichek II material itself. It is not established by human experts in the way clinical diagnostic ground truth might be (e.g., through pathologist review or clinical outcomes).
Instead, the expected results (positive/negative and Ct range) are predetermined by Bio-Rad Laboratories based on their characterization of the control material's content and concentration of purified intact microorganisms. The studies then verify that the Cepheid GeneXpert Systems accurately detect these known compositions.
Therefore, no external experts were used to establish the ground truth for the test set; the ground truth is defined by the manufacturer's formulation.
4. Adjudication Method for the Test Set
Since the ground truth is defined by the known composition of the control material, and the output is a qualitative (positive/negative) or semi-quantitative (Ct value) result from a machine, there is no adjudication method involving human experts for the test set. The device performance is a direct comparison of the instrument's output against the expected result of the control material.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done
No, a Multi-Reader Multi-Case (MRMC) comparative effectiveness study was not done. This type of study (comparing human readers with and without AI assistance on various cases) is typically applied to diagnostic imaging interpretation or other scenarios where human interpretation is the primary method being evaluated.
The Amplichek II is a quality control material for an automated molecular diagnostic system (Cepheid GeneXpert Systems). Its function is to verify the correct operation of these automated systems, not to assist human readers in interpreting complex cases. Therefore, the concept of human readers improving with or without AI assistance does not apply here.
6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) Was Done
The device itself, Amplichek II, is a physical quality control material, not an algorithm. However, the performance of the Cepheid GeneXpert system when using the Amplichek II control material is effectively a standalone performance evaluation of the GeneXpert system's ability to correctly process and interpret the known control substance. The studies assessed the Cepheid GeneXpert system's output (positive/negative and Ct values) without human intervention in the interpretation of the control material results. The Amplichek II simply serves as the standardized input.
Given that the Amplichek II is designed for automated molecular diagnostic systems, the evaluation of its performance (meaning how effectively it allows monitoring of an automated system) inherently relied on the standalone performance of the GeneXpert system.
7. The Type of Ground Truth Used
The ground truth used is based on the known, pre-defined composition and concentration of microorganisms within the Amplichek II control material. Bio-Rad Laboratories engineers and develops the control material to contain specific analytes (e.g., MRSA, MSSA, Cdiff, VRE) at defined levels (Negative, Level 1, Level 2, Level 3).
Therefore, the expected result for each test (e.g., "Positive" for SCCmec at Level 2, "Negative" for Toxin B at Level 1) is a known characteristic of the manufactured control. The studies verify that the Cepheid GeneXpert Systems produce results that agree with these known characteristics.
8. The Sample Size for the Training Set
The document does not explicitly describe a separate training set for the Amplichek II or the Cepheid GeneXpert System in the context of this submission.
The Amplichek II is a quality control material, not an algorithm that requires a training set. The Cepheid GeneXpert Systems are the devices being monitored by the Amplichek II. The GeneXpert systems themselves would have undergone extensive validation and potentially "training" during their development, but this information is not detailed in the Amplichek II's de novo submission.
The studies described are performance evaluations of the control material and how well it helps monitor the performance of the GeneXpert systems.
9. How the Ground Truth for the Training Set Was Established
As noted above, there is no explicit "training set" for the Amplichek II control material itself. The ground truth for the performance evaluation (test set) is the known, manufactured composition of the control material. This is established through the Bio-Rad Laboratories' internal manufacturing processes, formulation, and characterization of the specific microorganisms and their target concentrations within each level of the Amplichek II product.
§ 866.3920 Assayed quality control material for clinical microbiology assays.
(a)
Identification. An assayed quality control material for clinical microbiology assays is a device indicated for use in a test system to estimate test precision or to detect systematic analytical deviations that may arise from reagent or analytical instrument variation. This type of device consists of single or multiple microbiological analytes intended for use with either qualitative or quantitative assays.(b)
Classification. Class II (special controls). The special controls for this device are:(1) Premarket notification submissions must include detailed device description documentation and information concerning the composition of the quality control material, including, as appropriate:
(i) Analyte concentration;
(ii) Expected values;
(iii) Analyte source;
(iv) Base matrix;
(v) Added components;
(vi) Safety and handling information; and
(vii) Detailed instructions for use.
(2) Premarket notification submissions must include detailed documentation, including line data as well as detailed study protocols and a statistical analysis plan used to establish performance, including:
(i) Description of the process for value assignment and validation.
(ii) Description of the protocol(s) used to establish stability.
(iii) Line data establishing precision/reproducibility.
(iv) Where applicable, assessment of matrix effects and any significant differences between the quality control material and typical patient samples in terms of conditions known to cause analytical error or affect assay performance.
(v) Where applicable, identify or define traceability or relationship to a domestic or international standard reference material and/or method.
(vi) Where applicable, detailed documentation related to studies for surrogate controls.
(3) Premarket notification submissions must include an adequate mitigation (e.g., real-time stability program) to the risk of false results due to potential modifications to the assays specified in the device's 21 CFR 809.10 compliant labeling.
(4) Your 21 CFR 809.10 compliant labeling must include the following:
(i) The intended use of your 21 CFR 809.10(a)(2) and (b)(2) compliant labeling must include the following:
(A) Assayed control material analyte(s);
(B) Whether the material is intended for quantitative or qualitative assays;
(C) Stating if the material is a surrogate control; and
(D) The system(s), instrument(s), or test(s) for which the quality control material is intended.
(ii) The intended use in your 21 CFR 809.10(a)(2) and (b)(2) compliant labeling must include the following statement: “This product is not intended to replace manufacturer controls provided with the device.”
(iii) A limiting statement that reads “Quality control materials should be used in accordance with local, state, federal regulations, and accreditation requirements.”