Search Results
Found 2 results
510(k) Data Aggregation
(150 days)
QBH
The Unyvero LRT BAL Application is a qualitative nucleic acid multiplex test intended for the simultaneous detection and identification of nucleic acid sequences from the following microorganisms (N = 20) and antibiotic resistance markers (N = 10) in bronchoalveolar lavage (BAL)-like specimens (BAL or mini-BAL) from adult hospitalized patients with suspected lower respiratory tract infections.
The Unyvero LRT BAL Application performed on the Unyvero System is indicated as an aid in the diagnosis of lower respiratory tract infection in adult hospitalized patients with signs and symptoms of lower respiratory infection; results should be used in conjunction with other clinical and laboratory findings. As BAL specimens may contain colonizing microorganisms, detection of Unyvero LRT BAL microbial targets does not indicate that the microorganism is the disease. Unyvero positive results do not rule out co-infection with other microorganisms. Negative results do not preclude lower respiratory infection, as the causative agent may be a microorganism not detected by this test.
A negative result for any antibiotic resistance marker does not indicate that detected microorganisms are susceptible to applicable antimicrobial agents. Detected antibiotic resistance markers cannot be definitively linked to specific microorganisms, and may be present in organisms that are not detected by the Unyvero LRT BAL Application.
Microbiology cultures of BALs should be performed to obtain isolates for species identification and antimicrobial susceptibility testing and to identify potential microorganisms not targeted by the Unyvero LRT BAL Application.
The Unyvero LRT BAL Application automates and integrates DNA purification and eight parallel multiplex endpoint PCR reactions. It provides qualitative detection of nucleic acids from multiple lower respiratory pathogens using hybridization on PCR chamber arrays in a single use cartridge from a single bronchoalveolar lavage (BAL)-like specimen (BAL or mini-BAL).
The Unyvero LRT BAL Application identifies 20 microorganisms and 10 antibiotic resistance markers as listed in the Intended Use Statement, below.
The Unyvero LRT BAL Application uses a multiplex PCR approach following array hybridization which targets 30 individual analytes (microorganisms (N = 20) and antibiotic resistance markers (N = 10) divided into eight separate PCR reactions that are performed in individual reaction chambers simultaneously on a Unyvero LRT BAL Application cartridge. Multiplex compositions are designed to avoid any expected common occurrence of certain analytes within the same multiplex to largely reduce competitive PCR inhibition. Individual analyte assays of the Unyvero LRT BAL panel are designed to exhibit low or absent cross-reactivity with the relevant bronchoalveolar lavage (BAL)like specimens (BAL or mini-BAL) sample matrix or 'close neighbor' strains. Array oligonucleotides are designed for similar hybridization and melting temperatures (approx. 65 - 80 ℃, varying by amplicon). Hybridization and melting temperatures are used to exclude non-specific hybridization signals for improved signal specificity.
The instrumentation consists of one (or more) Unyvero L4 Lysator, one (or more) Unyvero A50 Analyzer, a Unyvero C8 Cockpit, and four single-use consumables: the Unyvero LRT BAL Cartridge, the Unyvero Sample Tube, Sample Tube Cap and the Unyvero Master Mix. A Unyvero Sample Tube Holder is supplied as accessory to simplify the sample filling step.
- Unyvero LRT BAL Cartridge contains DNA isolation and purification reagents, . primers, hybridization and wash buffers, and oligonucleotides for detection.
- Unyvero T1 Sample Tube and Transport Cap contains glass beads and buffers to lyse ● microorganisms and liquefy the sample.
- . Unyvero T1 Sample Tube Cap seals the Unyvero Sample Tube and contains Proteinase K and a synthetic control gene for process monitoring.
- Unyvero M1 Master Mix Tube contains reagents for DNA amplification. ●
An internal control (a synthetic gene without any homology to known sequences) is processed in every chamber in order to verify the DNA purification, array hybridization, and detection.
Other than the built-in controls, no external materials are supplied with Unyvero LRT BAL Application devices and consumables. Good laboratory practice recommends running external positive and negative controls using samples cultured in the laboratory.
No additional reagents are required to perform the Unyvero LRT BAL Application; all reagents are supplied within the cartridge or within the other consumables with the exception of the polymerase Master Mix, which is provided separately (frozen).
How to perform a Unyvero LRT BAL test:
-
- Remove the Unyvero Sample Tube from its packaging and slide it in the Unyvero Sample Tube Holder in the upright position with the barcoded end at the bottom.
-
- Remove the Transport Cap from the Sample Tube by pulling it upward.
-
- Pipette 180 uL of vortexed patient specimen into the Sample Tube and close it using the Unyvero Cap provided in the LRT BAL kit: align the small nodules on the neck of the Sample Tube with the Tube with the openings on the Cap and press down to lock in place.
-
- Scan the Sample Tube and place it into the Lysator. Close the Lysator lid to start the Lysator.
-
- Remove Master Mix from freezer and thaw at room temperature (15 ℃ 25 ℃) for approximately 30 minutes.
-
- When lysis is complete, remove the Sample Tube from the Lysator and place it into the labeled position on the left-hand side of the Unyvero LRT BAL Cartridge.
-
- Place the thawed Master Mix into the labeled position on the right-hand side of the Cartridge.
-
- Scan the Cartridge on the Cockpit and insert it into the position indicated on the Analyzer. The software provides on-screen instructions to start the test.
-
- View results after completion of the run.
During the automated analysis (Step 8), which is entirely controlled by the A50 Analyzer, the sample is mixed with ethanol and then transferred onto the DNA purification column, where buffers purify and elute the DNA. Eluted DNA is transferred to a chamber, where mixing with the Master Mix takes place. This mixture is distributed into eight separate PCR reaction chambers each containing multiple primer pairs, consisting of one labeled and one non-labeled primer for the respective multiplex endpoint PCR. After specific amplification, PCR products are hybridized to the corresponding array probes. Each array has been manufactured with probes corresponding to the amplicons for the targeted microorganism sequences described above. A total of up to 49 spots per array allows for redundant detection with at least four spots per analyte, as well as spots for intensity calibration, and orientation markers for the image processing software. Binding of amplicons to specific probes is detected by analyzing fluorescence images of the arrays. Result data are displayed on the C8 Cockpit for visualization, printout, temporary storage and electronic data export.
A test run is completed after 4 to 5 hrs, and results for panel microorganisms and corresponding antibiotic resistance markers are displayed on the Cockpit screen. Four screens are provided:
- . The Result Summary screen provides a quick overview of all detected LRT BAL panel microorganisms, together with all detected corresponding antibiotic resistance markers.
- . The Microorganisms screen provides a list of all panel microorganisms grouped in Grampositive bacteria, non-fermenting bacteria, Enterobacteriaceae and other microorganisms together with the analysis result (reported as detected, not detected or invalid).
- The Result Details screen provides a list of all analyzed microorganisms and the ● corresponding antibiotic resistance markers together with the analysis result (reported as detected, not detected, invalid or NA).
- Test Details screen showing user name, lot numbers of used consumables, expiration dates of ● the consumables and start and stop times and dates of the test.
Results can be reviewed on the cockpit or, optionally, be printed out. All results are saved in a database on the Unyvero Cockpit for later review and printing.
The Unyvero software is designed to:
- Manage analysis workflow (Cockpit) ●
- Carry out sample lysis (Lysator) ●
- Execute the analysis and generate the analytical result (Analyzer) .
- Manage communication among units (Cockpit, Lysator, Analyzer) ●
- Monitor internal mechanical / electrical actuators (Lysator, Analyzer) ●
- Present analysis results (Cockpit) .
- Store analysis results (Cockpit) ●
Each device (Cockpit, Lysator. Analyzer) is a subsystem within the overall system, and each consists of hardware and software components. The different devices are interconnected by an Ethernet based communication interface, and system functionality is provided by the interaction of all three device types. Only the Cockpit presents a rich user interface and allows interaction with the operator. The Lysator and Analyzer units include a simple display for showing device status. Optional HIS/LIS connectivity allows transferring results to a hospital or laboratory information system.
This looks like a medical device submission, specifically a 510(k) summary for the "Unyvero LRT BAL Application." Let's break down the acceptance criteria and study data provided.
Acceptance Criteria and Device Performance (Summary derived from the provided text)
The document doesn't explicitly state "acceptance criteria" numerical thresholds in a single, consolidated table. However, regulatory submissions often imply acceptance criteria based on the demonstrated performance that is deemed sufficient for clearance. Based on the performance tables provided, particularly the "Comparison to Composite Comparator Reference" (Table 28), we can infer the device's demonstrated performance for each microorganism. For antibiotic resistance markers, the "Comparison of antibiotic resistance markers to corresponding molecular reference assays" (Table 29) shows reported performance.
Inferred Acceptance Criteria (Implicit from demonstrated performance):
- For Microorganism Detection (PPA & NPA values compared to a Composite Comparator): The FDA generally expects high Positive Percent Agreement (PPA) and Negative Percent Agreement (NPA) to ensure the device is effective and safe. While no explicit thresholds are given, the achieved percentages, predominantly in the range of 80-100% for PPA and 90-100% for NPA across various microorganisms, suggest these levels were acceptable to the FDA.
- For Antibiotic Resistance Marker Detection (PPA & NPA values compared to Molecular Reference): Similar high PPA and NPA values were demonstrated (e.g., 88.9-100% for PPA and 69.5-100% for NPA), indicating these ranges were likely acceptable.
- Reproducibility: High agreement rates (over 90% for most cases, 100% for negative samples) for both moderate and low concentrations.
- Inclusivity & Exclusivity: Detection of target strains at or near LoD, and no cross-reactivity with non-target or common flora organisms.
- Limit of Detection (LoD): Demonstrated specific LoD values for each target (as listed in Tables 2 and 3).
Reported Device Performance (from Tables 28 & 29 for clinical accuracy):
Microorganism | PPA (%) (95% CI) | NPA (%) (95% CI) | PPV (%) (95% CI) | NPV (%) (95% CI) |
---|---|---|---|---|
Acinetobacter spp. | 88.9 (67.2 - 96.9) | 99.5 (98.8 - 99.8) | 76.2 (54.9 - 89.4) | 99.8 (99.3 - 99.9) |
Chlamydia pneumoniae | NA | 100.0 (99.6 - 100.0) | NA | 100.0 (99.6 - 100.0) |
Citrobacter freundii | 100.0 (20.7 - 100.0) | 99.7 (99.1 - 99.9) | 25.0 (4.6 - 69.9) | 100.0 (99.6 - 100.0) |
Enterobacter cloacae complex | 81.8 (61.5 - 92.7) | 99.8 (99.3 - 99.9) | 90.0 (69.9 - 97.2) | 99.6 (99.0 - 99.8) |
Escherichia coli | 82.4 (66.5 - 91.7) | 98.1 (97.0 - 98.8) | 59.6 (45.3 - 72.4) | 99.4 (98.7 - 99.7) |
Haemophilus influenzae | 90.9 (72.2 - 97.5) | 96.4 (95.0 - 97.4) | 35.7 (24.5 - 48.8) | 99.8 (99.2 - 99.9) |
Klebsiella oxytoca | 80.0 (49.0 - 94.3) | 99.4 (98.7 - 99.7) | 57.1 (32.6 - 78.6) | 99.8 (99.3 - 99.9) |
Klebsiella pneumoniae | 84.0 (65.3 - 93.6) | 99.1 (98.3 - 99.5) | 70.0 (52.1 - 83.3) | 99.6 (99.0 - 99.8) |
Klebsiella variicola | 0.0 (0.0 - 65.8) | 99.9 (99.4 - 100.0) | 0.0 (0.0 - 79.3) | 99.8 (99.3 - 99.9) |
Legionella pneumophila | 100.0 (20.7 - 100.0) | 100.0 (99.6 - 100.0) | 100.0 (20.7 - 100.0) | 100.0 (99.6 - 100.0) |
Moraxella catarrhalis | 75.0 (50.5 - 89.8) | 99.7 (99.1 - 99.9) | 80.0 (54.8 - 93.0) | 99.6 (99.0 - 99.8) |
Morganella morganii | 100.0 (43.9 - 100.0) | 100.0 (99.6 - 100.0) | 100.0 (43.9 - 100.0) | 100.0 (99.6 - 100.0) |
Mycoplasma pneumoniae | 100.0 (43.9 - 100.0) | 99.7 (99.1 - 99.9) | 50.0 (18.8 - 81.2) | 100.0 (99.6 - 100.0) |
Pneumocystis jirovecii | 80.0 (60.9 - 91.1) | 99.8 (99.3 - 99.9) | 90.9 (72.2 - 97.5) | 99.5 (98.8 - 99.8) |
Proteus spp. | 100.0 (72.3 - 100.0) | 100.0 (99.6 - 100.0) | 100.0 (72.3 - 100.0) | 100.0 (99.6 - 100.0) |
Pseudomonas aeruginosa | 93.5 (87.1 - 96.8) | 98.7 (97.7 - 99.2) | 89.3 (82.2 - 93.8) | 99.2 (98.4 - 99.6) |
Serratia marcescens | 93.8 (71.7 - 98.9) | 99.8 (99.3 - 99.9) | 88.2 (65.7 - 96.7) | 99.9 (99.4 - 100.0) |
Staphylococcus aureus | 83.3 (74.6 - 89.5) | 97.4 (96.1 - 98.2) | 76.9 (68.0 - 84.0) | 98.2 (97.2 - 98.9) |
Stenotrophomonas maltophilia | 82.9 (68.7 - 91.5) | 99.3 (98.5 - 99.7) | 82.9 (68.7 - 91.5) | 99.3 (98.5 - 99.7) |
Streptococcus pneumoniae | 100.0 (74.1 - 100.0) | 99.8 (99.3 - 99.9) | 84.6 (57.8 - 95.7) | 100.0 (99.6 - 100.0) |
Antibiotic Resistance Markers (Table 29):
Antibiotic Resistance Marker | PPA (%) (95% CI) | NPA (%) (95% CI) |
---|---|---|
ctx-M | 88.9 (56.5 - 98.0) | 99.5 (97.2 - 99.9) |
kpc | 100.0 (43.9 - 100.0) | 99.5 (97.3 - 99.9) |
mecA | 100.0 (85.1 - 100.0) | 69.5 (58.9 - 78.4) |
ndm | 100.0 (20.7 - 100.0) | 100.0 (98.2 - 100.0) |
oxa-23 | 100.0 (43.9 - 100.0) | 100.0 (82.4 - 100.0) |
oxa-24 | 100.0 (43.9 - 100.0) | 94.1 (73.0 - 99.0) |
oxa-48 | 100.0 (20.7 - 100.0) | 100.0 (96.7 - 100.0) |
oxa-58 | NA | 100.0 (84.5 - 100.0) |
tem | 100.0 (70.1 - 100.0) | 85.1 (72.3 - 92.6) |
vim | NA | 99.5 (97.3 - 99.9) |
Study Details
Here's a breakdown of the information regarding the studies:
-
Sample size used for the test set and the data provenance:
- Prospective Study: 1,016 bronchoalveolar lavage (BAL) or mini-BAL specimens from hospitalized adult patients with suspected lower respiratory tract infections. Data provenance: Collected at nine US clinical sites between June 2015 and July 2016 for a previous study, stored frozen. Retrospective collection of prospectively collected samples.
- Archived Study: 392 lavage specimens (BAL or mini-BAL) from patients (age 18 or older) with suspected lower respiratory infection and at least one positive LRT BAL panel microorganism by standard-of-care (SoC). Data provenance: Collected at 11 US clinical sites between 2015 and 2019, complemented by a few specimens from other sites. This included 197 specimens from a previous study and 195 new specimens. Retrospective collection.
- Contrived Study: Used to supplement rare analytes. Consisted of 60 contrived specimens per target analyte (total not explicitly stated for all analytes combined, but implies 60 for each of the 13 microorganisms and relevant antibiotic resistance markers used in this component). Prepared by spiking pooled microorganisms into negative lavage specimens.
-
Number of experts used to establish the ground truth for the test set and the qualifications of those experts:
- The document describes "Standard-of-Care (SoC) culture results" as one reference method for microorganisms and "molecular multiplex PCR comparator assay for which all positive PCR results were followed by bi-directional sequencing" as another.
- For "atypical microorganisms," a combination of two different validated PCR comparator assays (each targeting different genetic loci) was used.
- For antibiotic resistance markers, "PCR assays corresponding to each LRT BAL antibiotic resistance marker assay were included into the multiplex PCR comparator assay as a (single) molecular reference. Positive PCRs were followed by bi-directional sequencing."
- "Cultured isolates had been collected for positive prospective specimens whenever possible. Isolates were regrown and evaluated by MALDI-TOF to confirm strain identities and by whole genome sequencing using a next generation sequencing (NGS) approach to screen for presence or absence of LRT BAL panel antibiotic resistance markers."
- "Phenotypic AST results for positive specimens as reported by SoC culture were collected to correlate antibiotic resistance markers detected by the LRT BAL Application to resistance phenotypes."
- The document does NOT explicitly mention the number or specific qualifications of "experts" (e.g., medical doctors, microbiologists, or other specialists) routinely involved in establishing the ground truth measurements (like interpreting SoC culture results, performing PCR/sequencing or AST). It refers to "standard-of-care" clinical laboratory procedures and "molecular reference assays," implying trained laboratory personnel, but no explicit "expert panel" or count is given as typically seen for imaging studies.
-
Adjudication method (e.g., 2+1, 3+1, none) for the test set:
- For microbiology results: Discrepant results for false positives were analyzed by performing "singleplex PCRs/bi-directional sequencing using primer pairs targeting different genetic loci compared to the corresponding LRT BAL assays on specimen DNA extracts for each discrepant LRT BAL analyte." This indicates a form of molecular adjudication for discrepancies, rather than a consensus by human experts in the traditional sense, but still a specific method for resolving differences.
- For antibiotic resistance markers, "Correlation of Detected Antibiotic Resistance Markers to Strain Genotypes and Phenotypes" involved comparing the device's results to both molecular sequencing of isolates and phenotypic antimicrobial susceptibility testing (AST) results. This also serves as a multi-modal "adjudication" or reference verification process.
- No explicit "2+1" or "3+1" human reader adjudication method, common in imaging studies, is mentioned here, as it's a diagnostic test based on molecular/culture evidence.
-
If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:
- No, this is not applicable. The Unyvero LRT BAL Application is a qualitative nucleic acid multiplex test for microorganism and resistance marker detection. It is a standalone diagnostic device, not an AI-assisted interpretation tool for human readers (like radiologists interpreting images). Therefore, an MRMC comparative effectiveness study involving human reader improvement with/without AI is not relevant to this device.
-
If a standalone (i.e., algorithm only without human-in-the-loop performance) was done:
- Yes, this was a standalone performance study. The device itself performs automated DNA purification, multiplex PCR, hybridization, and interpretation. The clinical performance data presented (PPA, NPA, etc.) represent the performance of the "Unyvero LRT BAL Application" as a full system, comparing its direct output to the established reference methods (SoC culture, molecular comparator assays, sequencing). There is no "human-in-the-loop" aspect to the test's result generation.
-
The type of ground truth used (expert consensus, pathology, outcomes data, etc.):
- For "typical" microorganisms: The primary ground truth for the prospective study was Standard-of-Care (SoC) culture results, supplemented by a composite comparator (SoC culture combined with molecular multiplex PCR and bi-directional sequencing) to address limitations of culture.
- For "atypical" microorganisms: The primary ground truth was a composite comparator consisting of two different validated PCR comparator assays (each with bi-directional sequencing).
- For antibiotic resistance markers: Ground truth involved comparison to molecular reference assays (PCR/sequencing) for the specific marker and phenotypic antimicrobial susceptibility testing (AST) results for cultured isolates.
-
The sample size for the training set:
- The document describes performance studies (prospective, archived, contrived) used for validation. It does NOT specify a distinct "training set" or its size. This is typical for a traditional (non-AI/machine learning) diagnostic device validation, where the focus is on a comprehensive validation dataset rather than a separate training and test split. The LoD, inclusivity, exclusivity, interference, and reproducibility studies use various laboratory-prepared samples.
-
How the ground truth for the training set was established:
- Not applicable, as a specific "training set" is not mentioned in the context of this device. The validation studies utilized reference methods as described in point 6 above.
Ask a specific question about this device
(204 days)
QBH
The Unyvero LRT Application is a qualitative nucleic acid multiplex test intended for the simultaneous detection and identification of nucleic acid sequences from the following microorganisms and antibiotic resistance markers in endotracheal aspirates from adult hospitalized patients with suspected lower respiratory tract infections.
The Unyvero LRT Application performed on the Unyvero System is indicated as an aid in the diagnosis of lower respiratory tract infection in adult hospitalized patients with signs and symptoms of lower respiratory infection: results should be used in conjunction with other clinical and laboratory findings. As tracheal aspirates commonly contain colonizing microorganisms, detection of Unyvero LRT microbial targets does not indicate that the microorganism is the cause of the disease. Unyvero positive results do not rule out co-infection with microorganisms not detected by the Unyvero LRT Application. Negative results do not preclude lower respiratory infection, as the causative agent may be a microorganism not detected by this test.
A negative result for any antibiotic resistance marker does not indicate that detected microorganisms are susceptible to applicable antimicrobial agents. Detected resistance markers cannot be definitively linked to specific microorganisms, and may be present in organisms that are not detected by the Unyvero LRT Application such as organisms present as colonizing or normal flora.
Microbiology cultures of aspirates should be performed to obtain isolates for species identification and antimicrobial susceptibility testing, to differentiate quantities of identified microorganisms as well as normal flora present in the specimen and to identify potential microorganisms not targeted by the Unyvero LRT Application.
The Unyvero LRT Application is a qualitative test that includes specimen processing, genomic bacterial DNA isolation and purification, multiplex PCR and array hybridization and detection. The Unyvero LRT Application performed using the Unyvero System detects specific nucleic acid sequences from microorganisms and resistance markers in tracheal aspirates collected from patients with signs and symptoms of lower respiratory infection.
The Unyvero LRT Application consists of the following components:
- Unyvero LRT Cartridge: Contains DNA isolation and purification reagents, a DNA isolation column, eight separate PCR chambers with eight corresponding detection arrays. The Cartridge also contains fluorescently-labeled primers, hybridization and wash buffers and oligonucleotide probes for detection of targeted PCR products using array hybridization technology.
- Unyvero T1 Sample Tube: Contains glass beads and buffers to lyse bacteria and liquefy the sample.
- Unyvero T1 Sample Tube Cap (with Internal Control): Contains proteinase K and a synthetic internal control gene for process monitoring. The T1 Sample Tube Cap seals the Unyvero Sample Tube after which the internal control is combined with each patient specimen. The internal control DNA sequence does not have significant homology to targeted sequences and is amplified independently in each of the eight PCR chambers and the amplified internal control product is hybridized on each array.
- Unyvero M1 Master Mix: Contains reagents for DNA amplification.
- Unyvero T1 Transfer Tool: The Transfer tool can be used to transfer viscous specimens from the primary sample container to the Unyvero Sample Tube.
The Unyvero System consists of the following components:
- Unyvero Lysator: The Lysator lyses the specimen and can process up to four specimens simultaneously in four separate slots.
- Unyvero Analyzer: The Analyzer automates DNA purification, amplification and detection. Each Analyzer can simultaneously process up to two Unyvero Cartridges with each slot available using random access.
- Unyvero Cockpit: The Cockpit provides the main user interface for the Unyvero System, guides the user through the steps to run the Unvvero LRT Application and automatically generates and displays test results. The Cockpit is equipped with a high-resolution touch screen and a barcode reader.
- Unyvero Sample Tube Holder: The Sample Tube holder holds the Sample Tube securely while the specimen is transferred into the Sample Tube.
Here's a breakdown of the acceptance criteria and study proving the device meets them, based on the provided text:
Device: Unyvero Lower Respiratory Tract (LRT) Application and Unyvero System (Qualitative nucleic acid amplification assay)
Purpose: Simultaneous detection and identification of nucleic acid sequences from specified microorganisms and antibiotic resistance markers in endotracheal aspirates from adult hospitalized patients with suspected lower respiratory tract infections.
1. Table of Acceptance Criteria and Reported Device Performance
The document does not explicitly state pre-defined acceptance criteria in a quantitative table format (e.g., "PPA must be >90%"). However, it describes the performance observed in the validation studies and then concludes that this performance is "acceptable" in conjunction with various limitations and recommended practices. The implicit acceptance criteria appear to be the demonstrated performance characteristics from the clinical studies.
Below is a table summarizing the reported device performance for organism detection (compared to a composite comparator, which is generally the most stringent comparison) and resistance marker detection (with software masking). The document doesn't provide a single, overarching table of "acceptance criteria" but rather details performance metrics across various studies.
Table of Reported Device Performance (Key Clinical Metrics)
Target (Type) | Performance Metric (vs. Composite Comparator) | Value (95% CI) | Notes |
---|---|---|---|
Microorganism Targets | (Prospective Study, comparison to Composite Comparator unless otherwise noted) | ||
Acinetobacter spp. | PPA | 95.8% (79.8 - 99.3%) | Also 100% PPA vs. Culture for numerous, moderate, abundant categories; 37.0% PPV vs. Culture (likely due to DNA vs. viable organism) |
Chlamydia pneumoniae | PPA | na (0/0 positive cases) | 100.0% NPA (99.4 - 100.0%) for 603 cases |
Citrobacter freundii | PPA | 16.7% (3.0 - 56.3%) | Very low PPA in prospective study; complemented by 86.0% PPA in contrived study. |
Enterobacter cloacae complex | PPA | 94.4% (74.2 - 99.0%) | |
Escherichia coli | PPA | 97.4% (86.5 - 99.5%) | |
Haemophilus influenzae | PPA | 88.2% (65.7 - 96.7%) | |
Klebsiella oxytoca | PPA | 77.8% (45.3 - 93.7%) | Complemented by 89.3% PPA in contrived study. |
Klebsiella pneumoniae | PPA | 90.9% (76.4 - 96.9%) | |
Klebsiella variicola | PPA | 100.0% (34.2 - 100.0%) | Low number of positive cases (2/2 prospectively); complemented by 96.4% PPA in contrived study. |
Legionella pneumophila | PPA | 100.0% (34.2 - 100.0%) | Low number of positive cases (2/2 prospectively); complemented by 100.0% PPA in contrived study. |
Moraxella catarrhalis | PPA | 52.2% (33.0 - 70.8%) | Notably low PPA in prospective study; 96.0% PPA in contrived study. High FP rate against culture. |
Morganella morganii | PPA | 85.7% (48.7 - 97.4%) | Complemented by 95.6% PPA in contrived study. |
Mycoplasma pneumoniae | PPA | 100.0% (34.2 - 100.0%) | Low number of positive cases (2/2 prospectively); complemented by 100.0% PPA in contrived study. |
Proteus spp. | PPA | 96.0% (80.5 - 99.3%) | |
Pseudomonas aeruginosa | PPA | 89.4% (81.1 - 94.3%) | |
Serratia marcescens | PPA | 87.5% (69.0 - 95.7%) | |
Staphylococcus aureus | PPA | 91.6% (85.2 - 95.4%) | |
Stenotrophomonas maltophilia | PPA | 89.3% (78.5 - 95.0%) | |
Streptococcus pneumoniae | PPA | 62.5% (38.6 - 81.5%) | Notably low PPA in prospective study. |
Antibiotic Resistance Markers | (Prospective Study, comparison to molecular comparator with software masking applied for host microorganism) | ||
ctx-M | PPA | 93.8% (71.7 - 98.9%) | |
kpc | PPA | 100.0% (61.0 - 100.0%) | |
ndm | PPA | na (0/0 positive cases) | |
oxa-23 | PPA | 85.7% (48.7 - 97.4%) | |
oxa-24 | PPA | 100.0% (34.2 - 100.0%) | Low number of positive cases (2/2) |
oxa-48 | PPA | na (0/0 positive cases) | |
oxa-58 | PPA | na (0/0 positive cases) | |
tem | PPA | 100.0% (67.6 - 100.0%) | This is for tem detected when H. influenzae is also detected. Overall tem PPA (without masking) was 100.0% (93.4 - 100.0%) for 54/54 cases. |
vim | PPA | 100.0% (34.2 - 100.0%) | Low number of positive cases (2/2) |
mecA | PPA | 91.5% (81.6 - 96.3%) | This is for mecA detected when S. aureus is also detected. Overall mecA PPA (without masking) was 87.1% (80.1 - 91.9%) for 108/124 cases. |
2. Sample Size Used for the Test Set and Data Provenance
- Test Set (Clinical Studies):
- Prospective Study: 603 evaluable tracheal aspirate specimens.
- Provenance: Multi-center study at nine clinical sites in the United States.
- Nature: Prospectively collected.
- Retrospective (Archived) Study: 369 evaluable archived specimens (211 from US study sites, 158 from other US or European sites tested in-house at Curetis).
- Provenance: US and European sites.
- Nature: Previously frozen, retrospective.
- Contrived Clinical Study: Ranged from 21 to 50 specimens for each microorganism or resistance marker.
- Provenance: Prepared and tested at three sites in the United States and in-house at Curetis.
- Nature: Contrived specimens (spiked into natural tracheal aspirate matrices).
- Prospective Study: 603 evaluable tracheal aspirate specimens.
3. Number of Experts Used to Establish Ground Truth for the Test Set and Qualifications of those Experts
The document does not specify the number or qualifications of experts involved in establishing the ground truth for the clinical studies.
- For "typical" microorganisms in the prospective study, the primary ground truth was standard-of-care (SoC) tracheal aspirate culture. For the composite comparator, it also included independent and validated multiplexed PCR assays with bi-directional sequencing.
- For "atypical" microorganisms, the ground truth was two independent and validated multiplex PCR assays with bi-directional sequencing.
- For antibiotic resistance markers, the ground truth was validated multiplex PCR assays followed by bi-directional sequencing.
- For comparative phenotypic analysis of mecA, phenotypic antimicrobial susceptibility (AST) testing results from established culture methods were used.
The validation of these comparator methods (LoDs, inclusivity) is mentioned, implying expert oversight, but specific expert count or detailed qualifications are not provided.
4. Adjudication Method for the Test Set
The document describes the "Composite Comparator" method as an adjudication of sorts:
- For 'typical' microorganisms: A specimen was considered positive if culture was positive OR if the validated comparator PCR and follow-up bi-directional sequencing was positive. Any specimen negative by both culture and PCR was considered negative. This is a form of "consensus" or "best clinical practice" ground truth, where molecular confirmation is used to bolster culture results, or establish positivity where culture might miss (e.g., non-viable organisms, difficult to culture).
- For 'atypical' microorganisms: Specimens positive by either of the two PCR/sequencing assays were considered positive. Those negative by both were considered negative. This implies a "molecular consensus" approach.
- For 'false positives' against culture: False positive LRT results were analyzed with molecular assays (PCR/bi-directional sequencing) using sample DNA extracts to confirm presence or absence of microorganisms. This acts as a re-adjudication/confirmation step for discordant results.
There is no mention of human expert adjudication panels in the way it might be done for image-based AI devices (e.g., 2+1 radiologist review). The adjudication is inherently built into the definition of the reference/comparator method, often relying on molecular methods to resolve discrepancies or provide more sensitive ground truth than traditional culture.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was done
No, an MRMC comparative effectiveness study was not done for this device. This device is a diagnostic assay for molecular detection, not an AI-assisted diagnostic aid for human readers/interpreters in the typical MRMC study design (e.g., radiology AI). The study proves the standalone performance of the device.
6. If a Standalone (algorithm only without human-in-the-loop performance) was done
Yes, the study primarily evaluates the standalone performance of the Unyvero LRT Application. The reported PPA and NPA values for specific targets are measures of the device's accuracy in detecting nucleic acid sequences in the specimens, independent of human interpretation of the device's raw output. The device itself (Unyvero System with LRT Application) automates DNA purification, amplification, and detection, and automatically generates and displays results. The human involvement is in sample preparation and loading, and interpreting the final qualitative "positive/negative" result the device provides, rather than interpreting raw data alongside or without AI assistance.
7. The Type of Ground Truth Used
The ground truth used varied depending on the target:
- For 'typical' microorganisms:
- Primary Reference: Standard-of-care (SoC) tracheal aspirate culture.
- Composite Comparator: A combination of SoC culture PLUS independent and validated multiplexed PCR assays with bi-directional sequencing. This represents a more sensitive and comprehensive "true presence" ground truth, especially for organisms that may be difficult to culture or present as non-viable DNA.
- For 'atypical' microorganisms: Two independent and validated multiplex PCR assays with bi-directional sequencing.
- For antibiotic resistance markers: Validated multiplex PCR assays followed by bi-directional sequencing.
- For genotypic linkage and phenotypic correlation: Molecular sequencing of cultured isolates for the presence of the resistance marker gene, and phenotypic antimicrobial susceptibility testing (AST) results from those isolates.
8. The Sample Size for the Training Set
The document does not explicitly mention a "training set" in the context of machine learning or AI models. Given that this is a PCR-based diagnostic assay, it's developed through traditional molecular assay design and optimization, not typically through supervised machine learning with distinct training and test sets as seen in AI imaging. The "development" or "optimization" of the assay (e.g., primer design, LoD determination, inclusivity/exclusivity testing) would constitute its "training" or optimization phase, but no sample size for this phase is provided. The data presented relates to the validation of the final assay.
9. How the Ground Truth for the Training Set Was Established
As noted above, there's no explicitly defined "training set" in the machine learning sense. The ground truth for the analytical studies and clinical validation studies (which would inform refinement if developmental iterations were considered "training") was established as follows:
- Analytical Studies (LoD, Reproducibility, Inclusivity, Exclusivity, Interference): Samples were contrived by spiking known concentrations of well-characterized reference strains (ATCC, NCTC, DSM, JMI, Micromyx, NRZ, UCLA, RKI, clinical isolates) into artificial respiratory matrix (ARM) or PBS. The "ground truth" here is the known identity and concentration of the spiked microorganisms/resistance markers.
- Clinical Studies (Prospective and Retrospective): As detailed in point 7, ground truth was established by:
- Standard-of-care (SoC) tracheal aspirate culture.
- Validated multiplex PCR assays with bi-directional sequencing.
- A composite of SoC culture and PCR/sequencing.
- Molecular sequencing of isolates and phenotypic AST.
This means the "ground truth" was established by conventional microbiological and molecular diagnostic methods, which were considered the gold standard for comparing the performance of the new Unyvero system.
Ask a specific question about this device
Page 1 of 1