(92 days)
The GenePOC"M Strep A assay, performed on the revogene"14 instrument, is an automated, qualitative in vitro diagnostic test that utilizes real-time polymerase chain reaction (PCR) for the direct detection of Streptococcus pyogenes (Group A B-hemolytic Streptococcus) nucleic acids from throat swab specimens obtained from patients with signs and symptoms of pharyngitis. The GenePOC™ Strep A assay is intended for use as an aid in the diagnosis of Group A Streptococcus infection.
The GenePOC™ Strep A assay is a single-use test for qualitative detection of Streptococcus pyogenes (group A Streptococcus - GAS) nucleic acids from throat swab specimens obtained from patients with signs and symptoms of pharyngitis. The GenePOCTM Strep A assay kit is comprised of the disposable Strep A microfluidic cartridge (PIE), Sample Buffer Tube (SBT), and Disposable Transfer Tool (DTT). These components are used to suspend the sample, extract, amplify, and detect Streptococcus pyogenes (S. pyogenes) nucleic acid.
A Process Control (PrC) is also incorporated into each PIE to verify sample processing and amplification steps. The PrC allows for the verification of potential inhibitor substances as well as microfluidic, instrument or reagent failure. The GenePOC™ Strep A assay is designed to be used on the revogene™. The revogene™ is an instrument that automates sample homogenization, sample dilution, cell lysis, DNA amplification and detection of the amplified PCR products.
Each GenePOC™ Strep A assay kit provides components for twenty-four (24) tests. User intervention is required for sample preparation, transferring throat swab specimen into the SBT, using the DTT to transfer the sample into the PIE, and loading the PIE into the revogene™ carousel. Each PIE is a completely integrated closed device in which a sample is dispensed and processed through different microfluidic chambers and channels that allow for the sample processing and subsequent real-time PCR steps.
During the run and at run completion, the results are computed by the revogene™ from measured fluorescent signals and embedded calculation algorithms. The output results include positive, negative, indeterminate, and unresolved. An Early Positive Result Outcome (E-PRO) feature provides positive result if the signal from the target DNA reaches a predetermined threshold before the full PCR cycles have been completion of a run, the user removes the used cartridges and disposes of them in normal biological waste. Results may be viewed, printed, transferred, and/or stored by the user.
The provided text describes the acceptance criteria and the study proving the GenePOC™ Strep A device meets these criteria. Since the document is a 510(k) summary for an in vitro diagnostic (IVD) assay, the acceptance criteria are generally related to analytical performance (e.g., precision, detection limit, inclusivity, specificity, interference) and clinical performance (sensitivity and specificity compared to a reference method). The study design is focused on demonstrating the reliability and accuracy of the diagnostic test in detecting Streptococcus pyogenes.
Here's the breakdown of the information requested based on the provided text:
1. Table of Acceptance Criteria and Reported Device Performance
For an IVD such as this, the acceptance criteria are typically implicit in the "Performance Characteristics" section, where the manufacturer demonstrates that the device performs reliably and accurately for its intended use. There are no explicit pass/fail acceptance values stated for each measured characteristic, but the reported performance values are the data presented to demonstrate adequacy.
| Performance Characteristic | Acceptance Criteria (Implied) | Reported Device Performance |
|---|---|---|
| Precision/Reproducibility | Overall Qualitative Agreement | |
| Within-Laboratory Precision | High agreement for negative, low positive, and moderate positive samples | Negative: 97.5% (95% CI: [91.3%-99.7%]) (78/80 detected negative) |
| Low Positive: 100% (95% CI: [96.3%-100%]) (80/80 detected positive) | ||
| Moderate Positive: 100% (95% CI: [96.3%-100%]) (80/80 detected positive) | ||
| Between-Laboratory Reproducibility | High agreement across multiple sites for low positive, moderate positive, and negative samples | Low Positive: 100% (95% CI: [96.7%-100%]) (90/90 detected positive) |
| Moderate Positive: 100% (95% CI: [96.7%-100%]) (90/90 detected positive) | ||
| Negative: 100% (95% CI: [97.5%-100%]) (120/120 detected negative) | ||
| Between-Lot Reproducibility | High agreement across multiple reagent lots for low positive, moderate positive, and negative samples | Low Positive: 97.8% (95% CI: [92.2-99.7%]) (88/90 detected positive) |
| Moderate Positive: 100% (95% CI: [96.7-100%]) (90/90 detected positive) | ||
| Negative: 100% (95% CI: [97.5-100%]) (120/120 detected negative) | ||
| Limit of Detection (LoD) | To establish the lowest detectable concentration of S. pyogenes | Ranges from 333 to 1,333 CFU/mL of Sample Buffer (SB) depending on the strain. Confirmed with various swab types. |
| Analytical Reactivity/Inclusivity | To detect a broad range of clinically relevant Streptococcus pyogenes strains with high positivity | 8 out of 9 strains detected with 100% positivity at 999 CFU/mL SB. One strain (ATCC® 49399™) detected with 100% positivity at 1667 CFU/mL SB. (9/9 replicates for each) |
| Analytical Specificity/Cross-Reactivity | No cross-reactivity with common throat/mouth microorganisms, phylogenetically related species, or human gDNA | No cross-reactivity observed among 50 non-specific analytes tested (concentrations up to ≥10^6 CFU/mL or cp/mL). Bioinformatic analysis also showed no significant homology with primers/probe. |
| Carry-over and Cross-Contamination | No false positive results due to carry-over or cross-contamination | No false positive results detected (n=80 in cross-contamination, n=80 in carry-over). |
| Assay Interference | Minimal to no interference from endogenous/exogenous substances or other microorganisms at relevant concentrations | Some substances (Analgesic/Antipyretic, NSAID, Bronchodilator, Whole Blood, Mucin) showed inhibitory effect at high concentrations (4.3% w/v or v/v) but no reportable interference at lower, clinically relevant concentrations (0.1-0.4% w/v or v/v). A combinatory effect of specific Streptococcus species or S. dysgalactiae was noted as potentially inhibitory. |
| Clinical Performance | High sensitivity and specificity for direct detection of S. pyogenes in throat swab specimens compared to the reference method (culture). | Sensitivity: 98.1% (151/154), 95% CI: [94.4% - 99.3%] |
| Specificity: 94.7% (426/450), 95% CI: [92.2% - 96.4%] |
2. Sample Sizes Used for the Test Set and Data Provenance
-
Clinical Performance Test Set (Clinical Studies):
- Sample Size: 604 fully compliant prospective specimens.
- Data Provenance:
- Country of Origin: Geographically diverse clinical trial sites; two (2) in Canada and six (6) in the United States.
- Retrospective or Prospective: Prospective multicenter trial. Throat swab specimens were collected from patients with signs and symptoms of pharyngitis.
-
Analytical Performance Test Sets (Examples):
- Within-Laboratory Precision: 240 samples (80 low positive, 80 moderate positive, 80 negative samples) tested over 20 days.
- Between-Laboratory Reproducibility: 300 samples (90 low positive, 90 moderate positive, 120 negative samples) tested across 3 sites over 5 days.
- Between-Lot Reproducibility: 300 samples (90 low positive, 90 moderate positive, 120 negative samples) tested across 3 reagent lots over 15 days.
- Limit of Detection: 24 replicates per concentration per strain (3 strains) with 3 kit lots; confirmation with 20 replicates per strain.
- Analytical Reactivity/Inclusivity: 9 replicates per strain (12 strains total).
- Analytical Specificity: 50 analytes, concentrations tested for each.
- Carry-over and Cross-Contamination: 80 samples for cross-contamination, 80 samples for carry-over.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications
- Clinical Performance Test Set: The ground truth for the clinical study was established by a "Reference Method" which is explicitly stated as culture. The text does not mention the number or qualifications of human experts (e.g., microbiologists, lab technicians) involved in performing or interpreting the reference culture method, as culture is considered the gold standard and its results are objective.
- For discrepancies in the clinical study, an "alternative PCR with bi-directional sequencing" was performed on the discordant samples. This suggests a secondary method for adjudication rather than expert consensus on the primary ground truth.
4. Adjudication Method for the Test Set
- Clinical Performance Test Set:
- The primary ground truth was established by the Reference Method (culture).
- For discordant results between the GenePOC™ Strep A assay and the culture, an alternative PCR with bi-directional sequencing was used for further investigation. This is a form of discrepancy resolution or adjudication for the clinical performance data. The results of this alternative PCR are footnoted in the clinical performance table (e.g., "17 of 24 were Strep A Positive" for samples positive by GenePOC but negative by culture).
- The text does not indicate a system like 2+1 or 3+1 involving human experts directly adjudicating case outcomes for the primary study endpoint; rather, it uses a technical discrepancy resolution method.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was Done, What was the Effect Size of How Much Human Readers Improve with AI vs. Without AI Assistance
- Not applicable. This device is an automated, qualitative in vitro diagnostic test for direct detection of nucleic acids. It does not involve human "readers" or image interpretation tasks where AI assistance would directly improve human reader performance. The device provides a direct positive/negative/indeterminate result.
6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) was Done
- Yes, implicitly. The device, the GenePOC™ Strep A assay on the revogene™ instrument, is described as an "automated, qualitative in vitro diagnostic test." Once the sample is loaded, "No operator intervention is necessary once the PIE is loaded onto the revogene™." The results are "computed by the revogene™ from measured fluorescent signals and embedded calculation algorithms."
- The clinical performance study directly evaluates this standalone performance (assay results vs. culture ground truth). The human element is in sample preparation and loading, but the detection and result interpretation are automated by the device.
7. The Type of Ground Truth Used
- Clinical Performance Test Set: Culture (for Streptococcus pyogenes). This is the traditional "gold standard" for bacterial identification in clinical microbiology.
- Analytical Performance Test Sets: Defined scientific standards, such as known concentrations of specific bacterial strains (CFU/mL), absence of target (negative matrix), and specific interfering substances, or bioinformatic analysis for sequence specificity.
8. The Sample Size for the Training Set
- This document is a 510(k) summary for an IVD test, not a machine learning or AI algorithm summary. The "device" here refers to a diagnostic assay kit used on an automated instrument.
- The assay's "embedded calculation algorithms" determine the results based on fluorescent signals and pre-set thresholds/cut-offs for RNA/DNA detection. There is no mention of a "training set" in the context of machine learning model training. The cut-offs were determined by testing n=509 native and contrived samples, which could be considered an "assay development" or "validation" dataset rather than a "training set" for a continually learning algorithm.
9. How the Ground Truth for the Training Set Was Established
- As noted above, there isn't a traditional "training set" in the machine learning sense. The assay cut-offs were established using a mix of "native (throat swab specimens) and contrived samples" (n=509). The ground truth for these would logically be established by the same reliable reference method (culture) or by known spiked concentrations for contrived samples.
§ 866.2680
Streptococcus spp. nucleic acid-based assay.(a)
Identification. AStreptococcus spp. nucleic acid-based assay is a qualitative in vitro diagnostic device intended to simultaneously detect and identify variousStreptococcus spp. nucleic acids extracted directly from clinical specimens. The device detects specific nucleic acid sequences for organism identification. The identification aids in the diagnosis of diseases caused by bacteria belonging to the genusStreptococcus and provides epidemiological information on these diseases. Pathogenic streptococci are associated with infections, such as sore throat, impetigo (an infection characterized by small pustules on the skin), urinary tract infections, rheumatic fever, and kidney disease.(b)
Classification. Class II (special controls). The special controls for this device are:(1) Premarket notification submissions must include detailed device description documentation, including the device components, ancillary reagents required but not provided, and a detailed explanation of the methodology including primer/probe sequence, design, and rationale for sequence selection.
(2) Premarket notification submissions must include detailed documentation from the following analytical and clinical performance studies: Analytical sensitivity (Limit of Detection), reactivity, inclusivity, precision, reproducibility, interference, cross reactivity, carry-over, and cross contamination.
(3) Premarket notification submissions must include detailed documentation from a clinical study. The study, performed on a study population consistent with the intended use population, must compare the device performance to results obtained from well-accepted reference methods.
(4) Premarket notification submissions must include detailed documentation for device software, including, but not limited to, software applications and hardware-based devices that incorporate software.
(5) Premarket notification submissions must include database implementation methodology, construction parameters, and quality assurance protocols, as appropriate.
(6) The device labeling must include limitations regarding the need for culture confirmation of negative specimens, as appropriate.
(7) A detailed explanation of the interpretation of results and acceptance criteria must be included in the device's 21 CFR 809.10(b)(9) compliant labeling.
(8) Premarket notification submissions must include details on an end user device training program that will be offered while marketing the device, as appropriate.