(167 days)
The Liat™ Strep A Assay, performed on the Liat™ Analyzer, is a qualitative in vitro diagnostic test for the detection of Streptococus pyogenes (Group A 8-hemolytic Streptococcus, Strep A) in throat swab specimens from patients with signs and symptoms of pharyngitis.
The Liat™ Strep A Assay utilizes nucleic acid purification and polymerase chain reaction (PCR) technology to detect Streptococcus pyogenes by targeting a segment of the Streptococcus pyogenes genome.
The Liat™ Strep A Assay, performed on the Liat™ Analyzer, is a rapid, automated in vitro diagnostic test for the qualitative detection of Streptococcus pyogenes (Group A B-hemolytic Streptococcus, Strep A) DNA in throat swab specimens in Amies medium.
The Liat™ Strep A Assay targets a well-conserved region of Strep A genome. An Internal Process Control (IPC) is also included. The IPC is present to control for adequate processing of the target bacteria through all steps of the assay process and to monitor the presence of inhibitors in the sample preparation and PCR. The sample-to-result time is ~15 minutes.
The assay utilizes a single-use disposable Liat™ Tube that holds the sample purification and PCR reagents, and hosts the sample preparation and PCR processes. The Liat™ Tube contains all required unit dose reagents pre-packed in tube segments, separated by peelable seals, in the order of reagent use.
The Liat™ Analyzer automates and integrates sample purification, nucleic acid amplification, and detection of the target sequence in biological samples. The Liat™ Analyzer performs all assay steps from clinical sample and reports assay result automatically. During the testing process, multiple sample processing actuators of the analyzer compress the Liat™ Tube to selectively release reagents from tube segments, move the sample from one segment to another, and control reaction volume, temperature, and time to conduct sample preparation, nucleic acid extraction, target enrichment, inhibitor removal, nucleic acid elution and real-time PCR. An embedded microprocessor controls and coordinates the actions of these sample processors to perform all required assay processes within the closed Liat™ Tube.
Positive and negative controls are provided in the Liat™ Strep A Assay Quality Control Kit. The positive control comprises inactivated Strep A bacteria in a dried format. The negative control comprises Amies medium.
To perform the Liat™ Strep A Assay, an operator first collects a throat swab and places the swab into Amies transport medium. The operator transfers the sample into the Liat™ Strep A Assay tube using a transfer pipette, and scans the tube barcode to identify the test and the sample barcode to code the sample ID with the assay run on the Liat™ Analyzer. The Liat™ Tube is then inserted into the Liat™ Analyzer. The analyzer performs all the test steps and outputs interpreted results (e.g. Strep A Detected, Strep A Not Detected) in ~15 minutes. A report of the interpreted results can be viewed on the Liat™ Analyzer's LCD screen, and printed directly through a USB or network connected printer. No reagent preparation or additional steps are required other than adding the sample to the Liat™ Tube. Because all the reagents are contained within the Liat™ assay tube and no sample or reagent needs to be removed from the tube, crosscontamination between samples is minimized.
The results are interpreted by the Liat™ Analyzer software from measured fluorescent signals and real time curve recognition algorithm.
Here's a breakdown of the acceptance criteria and study information for the Liat™ Strep A Assay, based on the provided document:
1. Table of Acceptance Criteria and Reported Device Performance
The document does not explicitly state pre-defined acceptance criteria for clinical performance, but rather presents the computed clinical performance metrics. For analytical performance, criteria are implied by the results (e.g., 95% detection for LOD, absence of cross-reactivity).
| Metric / Category | Acceptance Criteria (Implied/Deduced) | Reported Device Performance |
|---|---|---|
| Analytical Performance | ||
| Strep A Reproducibility | High agreement for all samples across sites, operators, days, analyzers, and lots. | Total agreement: 99.7% (359/360 runs) for Strep A |
| IPC Reproducibility | High agreement for all samples across sites, operators, days, analyzers, and lots. | Total agreement: 100% (360/360 runs) for IPC |
| Limit of Detection (LOD) | 95% detection at lowest bacterial concentration | 5-20 CFU/mL (1-4 CFU/test) across 4 strains tested |
| Analytical Specificity (Reactivity) | Detection of all Strep A strains tested at specified concentrations. | Detected all 5 Strep A strains tested at 20-80 CFU/mL |
| Analytical Specificity (Cross-reactivity) | No cross-reactivity with non-Strep A microorganisms. | No cross-reactivity with 72 tested microorganisms |
| Interfering Microorganisms | No interference with Strep A detection from other microorganisms. | No interference from 72 tested microorganisms with Strep A detection at 3x LOD |
| Interfering Substances | No interference with Strep A detection from common throat substances. | No interference from 28 tested substances with Strep A detection at 3x LOD |
| Carry-over/Cross-contamination | No false positives from negative samples following high positive samples. | 0% carry-over/cross-contamination (40/40 negative samples correctly reported) |
| Clinical Performance | ||
| Clinical Sensitivity | (Not explicitly stated, but typically high for diagnostic tests of this type) | 98.3% (170/173) with 95% CI: 95.0% - 99.4% |
| Clinical Specificity | (Not explicitly stated, but typically high for diagnostic tests of this type) | 94.2% (374/397) with 95% CI: 91.5% - 96.1% |
| Accuracy | (Not explicitly stated) | 95.4% (544/570) with 95% CI: 93.4% - 96.9% |
| Invalid Rate | Low rate of invalid/indeterminate/aborted results. | 1.2% (7/577) Rate (all re-tested specimens gave valid results) |
2. Sample Size Used for the Test Set and Data Provenance
- Clinical Test Set Sample Size: 570 throat swab specimens were analyzed for clinical performance. An additional 7 specimens initially yielded invalid/indeterminate/aborted results, bringing the total to 577 specimens processed.
- Data Provenance: The clinical study was conducted at six clinical sites in the United States (geographically distinct regions) between December 2013 and April 2014. This indicates a prospective collection of real-world clinical samples.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts
The document does not specify the number or qualifications of experts used to establish the ground truth. It states that "Performance characteristics of the assay were determined by comparison to culture and latex agglutination for Strep A typing. Discordant results were investigated using PCR and bi-directional sequencing based on published methods." This implies laboratory personnel with expertise in microbiology, culture techniques, latex agglutination, PCR, and sequencing.
4. Adjudication Method for the Test Set
The adjudication method for the clinical test set was:
- Initial comparison: Liat™ Strep A Assay results vs. culture and latex agglutination.
- Discordant results investigation: PCR and bi-directional sequencing.
- For the 23 Liat positive, culture negative specimens, all 23 were confirmed Strep A positive by PCR/sequencing.
- For the 3 Liat negative, culture positive specimens, all 3 were confirmed Strep A positive by PCR/sequencing. (Re-testing with Liat also yielded positive results for these 3).
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
No, a multi-reader multi-case (MRMC) comparative effectiveness study was not done for this device. The Liat™ Strep A Assay is an automated in vitro diagnostic test for the qualitative detection of DNA, not an imaging device requiring human interpretation, so MRMC studies are not applicable in this context. It's a standalone device that provides an automated result.
6. Standalone (Algorithm Only) Performance
Yes, the study presents standalone performance. The Liat™ Strep A Assay is an automated system where the analyzer performs all test steps and outputs interpreted results (e.g., "Strep A Detected," "Strep A Not Detected") directly. The results are interpreted by the Liat™ Analyzer software using measured fluorescent signals and a real-time curve recognition algorithm.
7. Type of Ground Truth Used
The ground truth for the clinical study was established using a combination of methods:
- Microbial Culture: The primary comparative method for detecting Streptococcus pyogenes (Group A ß-hemolytic Streptococcus) in throat swab specimens.
- Latex Agglutination: Used for Strep A typing in conjunction with culture.
- PCR and Bi-directional Sequencing: Used as a reference method to resolve discordant results between the Liat™ assay and the primary culture/latex agglutination method. This serves as a highly specific molecular confirmation.
8. Sample Size for the Training Set
The document does not explicitly mention a separate "training set" or its size. For in vitro diagnostic devices like the Liat™ Strep A Assay, method development and initial algorithm design would typically involve a range of known positive and negative samples, and potentially spiked samples, to establish parameters like cut-offs and curve recognition algorithms.
The analytical performance studies (LOD, reactivity, cross-reactivity, interfering substances, etc.) used various biological and chemical samples to characterize the assay's behavior. For instance, the assay cut-offs were determined through "analysis of a combination of negative clinical samples that were spiked with different strains of S. pyogenes at the LOD target level." This could be considered part of the internal development and tuning process, which might be analogous to a training or development dataset in some contexts, but not typically reported as a formalized "training set" in the same way as machine learning models.
9. How the Ground Truth for the Training Set (or assay development) Was Established
As noted above, a formal "training set" with ground truth establishment is not detailed. However, for the development of cut-offs and algorithms, the ground truth was established by:
- Spiking studies: Negative clinical samples were spiked with known concentrations of different strains of S. pyogenes at target LOD levels. This engineered ground truth allows the system to learn to differentiate true positives from negatives and establish appropriate thresholds (Ct value and endpoint amplitude cut-offs) and curve recognition algorithms using known concentrations of bacteria.
- Controlled experiments: Analytical studies like LOD, reactivity, and cross-reactivity used pre-characterized strains and samples, where the presence or absence of specific microbes and their concentrations were known.
§ 866.2680
Streptococcus spp. nucleic acid-based assay.(a)
Identification. AStreptococcus spp. nucleic acid-based assay is a qualitative in vitro diagnostic device intended to simultaneously detect and identify variousStreptococcus spp. nucleic acids extracted directly from clinical specimens. The device detects specific nucleic acid sequences for organism identification. The identification aids in the diagnosis of diseases caused by bacteria belonging to the genusStreptococcus and provides epidemiological information on these diseases. Pathogenic streptococci are associated with infections, such as sore throat, impetigo (an infection characterized by small pustules on the skin), urinary tract infections, rheumatic fever, and kidney disease.(b)
Classification. Class II (special controls). The special controls for this device are:(1) Premarket notification submissions must include detailed device description documentation, including the device components, ancillary reagents required but not provided, and a detailed explanation of the methodology including primer/probe sequence, design, and rationale for sequence selection.
(2) Premarket notification submissions must include detailed documentation from the following analytical and clinical performance studies: Analytical sensitivity (Limit of Detection), reactivity, inclusivity, precision, reproducibility, interference, cross reactivity, carry-over, and cross contamination.
(3) Premarket notification submissions must include detailed documentation from a clinical study. The study, performed on a study population consistent with the intended use population, must compare the device performance to results obtained from well-accepted reference methods.
(4) Premarket notification submissions must include detailed documentation for device software, including, but not limited to, software applications and hardware-based devices that incorporate software.
(5) Premarket notification submissions must include database implementation methodology, construction parameters, and quality assurance protocols, as appropriate.
(6) The device labeling must include limitations regarding the need for culture confirmation of negative specimens, as appropriate.
(7) A detailed explanation of the interpretation of results and acceptance criteria must be included in the device's 21 CFR 809.10(b)(9) compliant labeling.
(8) Premarket notification submissions must include details on an end user device training program that will be offered while marketing the device, as appropriate.