(432 days)
The BinaxNOW® PBP2a Test is a qualitative, in vitro immunochromatographic assay for the rapid detection of penicillin-binding protein 2a (PBP2a) present in methicillinresistant Staphylococcus aureus (MRSA). The test is performed directly on blood culture samples positive for S. aureus.
The BinaxNOW® PBP2a Test is not intended to diagnose MRSA nor to guide or monitor treatment for MRSA infections. Subculturing positive blood cultures is necessary to recover organisms for susceptibility testing or epidemiological typing.
The BinaxNOW® PBP2a Test is a rapid immunochromatographic membrane assay that uses highly sensitive monoclonal antibodies to detect the PBP2a protein directly from blood cultures which have been identified as being positive for S. aureus. These antibodies and a control antibody are immobilized onto a test strip as two distinct lines and combined with other reagents/pads. This test strip is mounted inside a cardboard, book-shaped hinged test device.
Specimens are aliquots from blood cultures which have been identified as positive for Staphylococcus aureus. After the sample is prepared, it is added to the sample pad at the top of the test strip and the device is closed. Results are read at 10 minutes.
Acceptance Criteria and Study for BinaxNOW® PBP2a Test
1. Table of Acceptance Criteria and Reported Device Performance
The acceptance criteria for the BinaxNOW® PBP2a Test, as derived from the provided document, are presented in the table below, alongside the reported device performance.
| Performance Metric | Acceptance Criteria (Implied) | Reported Device Performance |
|---|---|---|
| Clinical Performance (Positive Agreement) | ||
| Cefoxitin (30 µg) disc diffusion | High (e.g., >95%) | 96.9% (62/64) (89.3 - 99.1% CI) |
| Oxacillin (1 µg) disc diffusion | High (e.g., >95%) | 96.5% (55/57) (88.1 - 99.0% CI) |
| Automated Antimicrobial Susceptibility | High (e.g., >95%) | 97.6% (41/42) (87.7 - 99.6% CI) |
| Clinical Performance (Negative Agreement) | ||
| Cefoxitin (30 µg) disc diffusion | High (e.g., >99%) | 100.0% (67/67) (94.6 - 100.0% CI) |
| Oxacillin (1 µg) disc diffusion | High (e.g., >99%) | 100.0% (58/58) (93.8 - 100.0% CI) |
| Automated Antimicrobial Susceptibility | High (e.g., >99%) | 100.0% (29/29) (88.3 - 100.0% CI) |
| Overall Clinical Performance | High (e.g., >95% for positive, >99% for negative) | 97.1% positive agreement, 100.0% negative agreement (for overall 199 samples) |
| Analytical Reactivity (MRSA strains) | All tested strains positive | All listed MRSA strains (NARSA and ATCC) tested positive. |
| Analytical Specificity (MSSA strains) | All tested strains negative | All listed MSSA strains tested negative. |
| Analytical Specificity (Other Staphylococcal strains) | All tested strains negative (except for expected cross-reactivity) | All tested strains negative except Staphylococcus sciuri. |
| Analytical Specificity (Non-Staphylococcal strains) | All tested strains negative (except for expected cross-reactivity) | All tested strains negative except Cryptococcus neoformans. |
| Interfering Substances | No interference | All 20 listed substances produced appropriate results. |
| Analytical Sensitivity (Limit of Detection) | Specific CFU/mL value expected | 2.5 x 10^7 cells/mL (turbidity 0.03) / 2.36 x 10^7 CFU/mL (from ATCC BAA44) |
| Reproducibility | 100% agreement expected | 100% (599/599) agreement with expected results. No significant differences. |
Note: The document does not explicitly state numerical acceptance criteria. The "Implied Acceptance Criteria" are inferred from the demonstrated performance and the context of a 510(k) submission, where high agreement with established methods is required for substantial equivalence.
2. Sample Size Used for the Test Set and Data Provenance
- Sample Size for Clinical Performance Test Set: 199 S. aureus samples.
- Data Provenance: Multi-center clinical study conducted in 2008-09 at four geographically diverse hospital laboratories within the United States. The study was prospective in nature, as samples were "evaluated in the BinaxNOW® PBP2a Test and compared to standard methods."
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts
The document states that the ground truth for the clinical performance was established by "standard methods used routinely by the laboratories: Cefoxitin (30 µg) disc diffusion, Oxacillin (1 µg) disc diffusion, and automated Minimum Inhibitory Concentration (MIC) Systems." It also mentions "Individual samples were evaluated by multiple laboratory methods, and in all cases there was 100% agreement between the reference methods."
While specific "experts" for establishing ground truth are not explicitly named or quantified, the ground truth was determined by multiple, routinely used laboratory methods performed by qualified laboratory personnel within the four hospital laboratories. The qualifications of these individuals would typically include clinical microbiologists or medical technologists with experience in performing and interpreting these standard susceptibility tests. The document implies that the "experts" were the laboratory staff routinely performing these validated reference methods.
4. Adjudication Method for the Test Set
The document notes that "Individual samples were evaluated by multiple laboratory methods, and in all cases there was 100% agreement between the reference methods." This indicates that if there were any discrepancies between results from the Cefoxitin disc diffusion, Oxacillin disc diffusion, and automated MIC systems, they were resolved or did not occur. The fact that only three clinical samples (1.5%) produced "discrepant results" overall (compared to the BinaxNOW test) suggests that there was an established reference standard, and these three were just discordant with the device, not necessarily with the reference methods themselves. No specific formal adjudication method (e.g., 2+1, 3+1) for resolving conflicts between the reference methods is described, as 100% agreement among them was reported. Discrepancies between the device and the reference methods were simply noted.
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
No, a multi-reader multi-case (MRMC) comparative effectiveness study was not done. The study focused on the diagnostic accuracy of the device against standard laboratory methods, not on how human readers' performance improved with or without AI assistance. The BinaxNOW® PBP2a Test is a rapid immunochromatographic assay, meaning it is a diagnostic test kit that provides a direct result, not an AI system designed to assist human readers.
6. Standalone Performance
Yes, a standalone performance study was done. The entire clinical performance study section describes the algorithm's (the BinaxNOW® PBP2a Test's) performance "alone" against established reference methods. The results presented in the table are the direct output of the BinaxNOW® PBP2a Test without human-in-the-loop intervention in the interpretation of the enzymatic reaction as positive or negative. While a human reads the test, the test itself provides a direct biochemical result that is interpreted.
7. Type of Ground Truth Used
The type of ground truth used for the clinical performance study was expert consensus (of established laboratory methods). Specifically, it relied on the results from:
- Cefoxitin (30 µg) disc diffusion
- Oxacillin (1 µg) disc diffusion
- Automated Minimum Inhibitory Concentration (MIC) Systems
The document states there was "100% agreement between the reference methods" for individual samples, indicating these methods collectively formed the definitive truth.
8. Sample Size for the Training Set
The document does not specify a sample size for a "training set." The BinaxNOW® PBP2a Test is described as an immunochromatographic assay, which is a chemical reaction-based test, not a machine learning or AI model that typically requires a separate training set. The various analytical studies (reactivity, specificity, sensitivity, interfering substances) might be considered part of the development and "training" (calibration/validation) of the assay itself, but these are based on known reference strains and substances rather than patient data used in a typical machine learning training set.
9. How the Ground Truth for the Training Set Was Established
As noted above, there is no explicit "training set" in the context of a machine learning model. However, if we consider the development and validation of the assay, the "ground truth" for analytical studies was established using known, well-characterized bacterial strains from sources like the American Type Culture Collection (ATCC) and the Network on Antimicrobial Resistance in Staphylococcus aureus (NARSA). For example, MRSA strains from these collections were expected to test positive, and MSSA, other Staphylococcal, and non-Staphylococcal strains were expected to test negative. These strains have established classifications regarding their methicillin resistance.
§ 866.1640 Antimicrobial susceptibility test powder.
(a)
Identification. An antimicrobial susceptibility test powder is a device that consists of an antimicrobial drug powder packaged in vials in specified amounts and intended for use in clinical laboratories for determining in vitro susceptibility of bacterial pathogens to these therapeutic agents. Test results are used to determine the antimicrobial agent of choice in the treatment of bacterial diseases.(b)
Classification. Class II (performance standards).