Search Results
Found 4 results
510(k) Data Aggregation
(106 days)
Intelligent Medical Devices, Inc.
The IMDx HSV-1/2 for Abbott m2000 assay is an in vitro diagnostic test for the direct, qualitative detection and differentiation of herpes simplex virus type 1 (HSV-1) and type 2 (HSV-2) DNA from male and female skin lesions from anogenital or oral sites. The test is intended for use as an aid in the diagnosis of HSV infection in symptomatic patients. The assay is intended to be run on the Abbott m2000 instrument system.
Warning: The IMDx HSV-1/2 for Abbott m2000 assay is not FDA cleared for use with cerebrospinal fluid (CSF). The assay is not intended for prenatal screening.
The IMDx FISV-1/2 for Abbott m2000 assay uses PCR to generate amplified product from HSV-1 and HSV-2 present in clinical specimens. The presence of HSV-1 and/or HSV-2 target DNA is indicated by the fluorescent signal generated through the use of fluorescently labeled oligonucleotide probes on the Abbott m2000rt instrument. The probes do not generate a signal unless they are specifically bound to the amplified product. The amplification cycle at which fluorescent signal is detected by the Abbott m2000rt is inversely proportional to the HSV-1 and/or HSV-2 DNA target concentration present in the original specimen. A plasmid containing DNA unrelated to HSV-1 and HSV-2 is introduced into each specimen during sample preparation to serve as an internal control. The internal control is amplified in the same reaction as the HSV-1 and HSV-2 DNA targets, and serves to demonstrate that the sample preparation and amplification proceeded correctly for each specimen. A preparation of intact, inactivated HSV-1 and HSV-2 virus is included as the positive control in the IMDx HSV-1/2 for Abbott m2000 assay. Run as a separate control, the positive control serves to demonstrate that the HSV-1/2 PCR reagents are functional. In addition, the positive control functions as a process control to demonstrate that sample preparation has proceeded correctly during the run. A negative control consisting of M4RT viral transport medium is included in each run to independently verify the absence of contaminating target material in assay reagents.
Acceptance Criteria and Device Performance Study for IMDx HSV-1/2 for Abbott m2000
The IMDx HSV-1/2 for Abbott m2000 assay is an in vitro diagnostic test for the direct, qualitative detection and differentiation of herpes simplex virus type 1 (HSV-1) and type 2 (HSV-2) DNA from male and female skin lesions from anogenital or oral sites.
The study that proves the device meets the acceptance criteria involved Prospective Clinical Performance Characteristics and Retrospective Clinical Performance Characteristics studies, as well as several analytical performance studies. The primary comparison for clinical performance was against the ELVIS® (Enzyme Linked Virus Inducible System) HSV ID and D3 Typing Test System.
1. Table of Acceptance Criteria and Reported Device Performance
The document does not explicitly state pre-defined acceptance criteria in terms of target sensitivity and specificity percentages. However, the study results, particularly from the prospective clinical performance, would implicitly represent the performance deemed acceptable for FDA clearance through substantial equivalence. If acceptance criteria were required, they would typically be set at a high level given the diagnostic nature of the device. For this summary, we will present the reported performance from the primary prospective study as the "met acceptance criteria".
Parameter | Acceptance Criteria (Implied by Study Results) | Reported Device Performance (Prospective Study) |
---|---|---|
HSV-1 Anogenital | ||
Sensitivity | High (e.g., >90%) | 99.0% (95% CI: 94.7%-99.8%) |
Specificity | High (e.g., >85%) | 96.3% (95% CI: 94.4%-97.6%) |
HSV-2 Anogenital | ||
Sensitivity | High (e.g., >90%) | 97.5% (95% CI: 93.8% - 99.0%) |
Specificity | High (e.g., >80%) | 89.5% (95% CI: 86.9% - 91.6%) |
HSV-1 Oral | ||
Sensitivity | High (e.g., >90%) | 100.0% (95% CI: 90.6% - 100.0%) |
Specificity | High (e.g., >70%) | 77.8% (95% CI: 69.1% - 84.6%) |
HSV-2 Oral | ||
Sensitivity | (Due to low positive count, this is challenging to set a high bar, but detection is important) | 0.0% (95% CI: 0.0% - 65.8%) (Very low N, limited conclusions) |
Specificity | High (e.g., >95%) | 98.6% (95% CI: 95.1% - 99.6%) |
Note on HSV-2 Oral Sensitivity: The reported sensitivity of 0.0% in the prospective study for oral HSV-2 is based on only 2 reference method positive cases. This low sample size limits definitive conclusions, and a contrived specimen study was performed to provide additional data for HSV-2 oral detection. This contrived study showed 100% detection of HSV-2 in spiked oral samples across various concentrations.
2. Sample Size Used for the Test Set and Data Provenance
- Prospective Studies (Clinical Performance Characteristics):
- Test Set Size: 954 prospective specimens (807 anogenital and 147 oral).
- Data Provenance: From four geographically diverse locations within the United States.
- Retrospective Studies:
- Test Set Size: 54 retrospective specimens (27 anogenital and 27 oral).
- Data Provenance: Not explicitly stated, but "historical results" for the ELVIS system suggests previously collected specimens.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of those Experts
The ground truth was established by the ELVIS® (Enzyme Linked Virus Inducible System) HSV ID and D3 Typing Test System. This is a commercially available viral culture system, which is a recognized reference method for HSV detection and typing. The document does not specify the number of human experts or their qualifications involved in interpreting the ELVIS results for establishing ground truth. The ELVIS system itself is the "expert" or gold standard in this context.
4. Adjudication Method for the Test Set
The document mentions "Discordant analysis" was performed.
- For anogenital HSV-1, 20 initial IMDx positive / ELVIS negative results were analyzed; 6 were confirmed positive for HSV-1, 11 remained discordant.
- For anogenital HSV-2, 68 initial IMDx positive / ELVIS negative results were analyzed; 55 were confirmed positive for HSV-2, 7 remained discordant.
- For oral HSV-1, 24 initial IMDx positive / ELVIS negative results were analyzed; 14 were confirmed positive for HSV-1, 10 remained discordant.
- For oral HSV-2, 2 initial IMDx positive / ELVIS negative results were analyzed; both were confirmed positive for HSV-2.
The specific "adjudication method" beyond re-testing or further analysis of discordant samples with an unspecified method is not detailed (e.g., whether a third, independent gold standard was used, or if expert review of discrepant results was performed). However, the results presented in the tables incorporate the outcomes of this discordant analysis by adjusting the counts (e.g., "HSV-1 was detected in 6 of the 17 specimens" changes the true positive count for HSV-1).
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was done
No, a Multi-Reader Multi-Case (MRMC) comparative effectiveness study was not done. This device is an in vitro diagnostic assay (a lab test), not an imaging or interpretation assistance device for human readers. Therefore, the concept of "human readers improve with AI vs without AI assistance" is not applicable here. The device provides a direct qualitative detection and differentiation of HSV-1/2 DNA.
6. If a Standalone (i.e. algorithm only without human-in-the loop performance) was done
Yes, a standalone performance study was done. The entire set of clinical performance characteristics (prospective and retrospective studies) evaluates the performance of the IMDx HSV-1/2 for Abbott m2000 assay directly against the reference method (ELVIS viral culture). This is an "algorithm only" or "device only" performance without human interpretation being the primary variable. The results are based on the device's output.
7. The Type of Ground Truth Used
The primary ground truth used for clinical performance studies was the ELVIS® (Enzyme Linked Virus Inducible System) HSV ID and D3 Typing Test System. This is a viral culture-based method, which is a widely accepted laboratory gold standard for direct detection of viable virus.
8. The Sample Size for the Training Set
The document does not explicitly describe a separate "training set" for the IMDx HSV-1/2 for Abbott m2000 assay. This is a PCR-based diagnostic test, not a machine learning model that typically requires a distinct training phase with a large dataset. The assay's design and optimization (e.g., primer and probe selection, reaction conditions) would have occurred during development, likely using various characterized samples, but these are not formally presented as a "training set" in the context of this 510(k) submission. Therefore, the concept of a separate "training set" as understood in machine learning is not directly applicable or reported for this type of IVD.
9. How the Ground Truth for the Training Set Was Established
As mentioned above, no explicit "training set" is described for this PCR-based IVD. The ground truth for the analytical and clinical validation of the device relies on established laboratory culture methods (ELVIS system for clinical specimens) and characterized viral strains/isolates for analytical studies (e.g., limit of detection, analytical reactivity, cross-reactivity). These reference methods serve as the "ground truth" to evaluate the accuracy of the IMDx assay.
Ask a specific question about this device
(85 days)
Intelligent Medical Devices, Inc.
The IMDx C. difficile for Abbott m2000 assay is an in vitro diagnostic assay that uses real-time polymerase chain reaction (PCR) amplification for the qualitative detection of nucleic acids encoding the toxin A gene (tcdA) and toxin B gene (tcdB ) sequences of toxigenic strains of Clostridium difficile in human liquid or soft stool specimens collected from patients suspected of having symptoms of Clostridium difficile infection.
The assay is intended to be performed on the Abbott m2000 System (which comprises the Abbott m2000sp and m2000rt instruments) and is indicated for use as an aid in the diagnosis of Clostridium difficile infection. The test is intended to be used directly on liquid or soft stool specimens (unpreserved stool, or stool preserved in Cary Blair transport medium). Negative results do not preclude toxigenic C. difficile infection and should not be used as the sole basis for treatment or other patient management decisions. The IMDx C. difficile for Abbott m2000 assay is intended for professional use. The device is not intended for point-of-care use.
The IMDx C. difficile for Abbott m2000 assay uses PCR to generate amplified product from the tcdA and tcdB/tcdBv genes in toxigenic C. difficile DNA in clinical specimens. The presence of a toxigenic C. difficile target sequence is indicated by the fluorescent signal generated through the use of fluorescently labeled oligonucleotide probes on the Abbott m2000rt instrument. The probes do not generate a signal unless they are specifically bound to the amplified product. The amplification cycle at which fluorescent signal is detected by the Abbott m2000rt is inversely proportional to the toxigenic C. difficile DNA target concentration present in the original specimen. A bacterial species unrelated to toxigenic C. difficile is introduced into each specimen during sample preparation to serve as a process control. The process control bacteria are lysed simultaneously with toxigenic C. difficile in the specimen, and amplified in the same reaction as the C. difficile targets using PCR, and serve to demonstrate that the entire assay process has proceeded correctly for each specimen.
Here's a breakdown of the acceptance criteria and the study details for the IMDx C. difficile for Abbott m2000 assay, based on the provided 510(k) summary:
Acceptance Criteria and Device Performance
The acceptance criteria are not explicitly stated in a defined section. However, based on the "Clinical Performance Characteristics" section, we can infer the acceptance criteria are related to the clinical agreement (Sensitivity, Specificity, Positive Predictive Value, and Negative Predictive Value) when compared to the Bartels® Cytotoxicity Assay for Clostridium difficile Toxin.
Here's a summary of the reported device performance:
Performance Metric | Acceptance Criteria (Inferred) | Reported Performance (Unpreserved Stool) | Reported Performance (Cary Blair Preserved Stool) |
---|---|---|---|
Clinical Sensitivity | Not explicitly stated | 86.1% (95% CI: 79.4% - 90.9%) | 91.3% (95% CI: 73.2% - 97.6%) |
Clinical Specificity | Not explicitly stated | 92.5% (95% CI: 90.7% - 93.9%) | 93.5% (95% CI: 90.5% - 95.7%) |
Positive Predictive Value | Not explicitly stated | 59.9% (95% CI: 52.9% - 66.5%) | 47.7% (95% CI: 33.8% - 62.1%) |
Negative Predictive Value | Not explicitly stated | 98.1% (95% CI: 97.0% - 98.8%) | 99.4% (95% CI: 97.8% - 99.8%) |
Note: For analytical performance metrics like "Precision/Reproducibility" and "Analytical Sensitivity (LoD)", the tables present the results, but specific numerical acceptance criteria (e.g., minimum % agreement for reproducibility or maximum CFU/mL for LoD) are not explicitly outlined as "acceptance criteria." However, the data presented implies that these results were deemed acceptable to demonstrate adequate analytical performance. For example, for "Precision/Reproducibility", achieving high percentage agreement, especially for positive and low positive samples, suggests successful criteria were met.
Study Details
-
Sample sizes used for the test set and the data provenance:
- Total Test Set Sample Size: 1,565 specimens.
- 1,186 unpreserved stool specimens
- 379 Cary Blair preserved stool specimens
- Data Provenance: The study was conducted at "seven (7) geographically diverse locations within the United States from 2011 to 2013." This indicates prospective data collection for a clinical trial.
- Total Test Set Sample Size: 1,565 specimens.
-
Number of experts used to establish the ground truth for the test set and the qualifications of those experts (e.g., radiologist with 10 years of experience):
- The document does not specify the number of experts or their qualifications for establishing the ground truth. The ground truth was established by a reference assay.
-
Adjudication method (e.g., 2+1, 3+1, none) for the test set:
- The document implies a form of discrepancy analysis rather than adjudication by multiple human readers for initial ground truth establishment. For discrepant results between the IMDx C. difficile for Abbott m2000 assay and the Bartels® Cytotoxicity Assay, "sequencing" was used to resolve some of these discrepancies (e.g., for unpreserved stool: 16 samples were sequenced, 13 resolved as negative, 3 remained discrepant; 53 samples were sequenced, 40 resolved as positive, 9 remained discrepant, 4 indeterminate). This suggests a molecular method was used for further investigation of conflicting results, rather than expert consensus on the initial test.
-
If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:
- No, a multi-reader multi-case (MRMC) comparative effectiveness study was not performed. This device is an in vitro diagnostic assay for molecular detection, not an imaging or interpretive device that would typically involve human readers and AI assistance.
-
If a standalone (i.e., algorithm only without human-in-the-loop performance) was done:
- Yes, a standalone performance study was done. The "Clinical Performance Characteristics" section reports the performance of the IMDx C. difficile for Abbott m2000 assay (the algorithm/device) directly against the reference method. There is no mention of human interpretation influencing the assay's output.
-
The type of ground truth used (expert consensus, pathology, outcomes data, etc.):
- The primary ground truth was established by comparison to the Bartels® Cytotoxicity Assay for Clostridium difficile Toxin. For discrepant results, sequencing was used for further resolution, serving as a secondary, more definitive ground truth for those specific cases.
-
The sample size for the training set:
- The document does not explicitly state a sample size for a training set. This is a molecular diagnostic assay, and while it uses PCR amplification, the summary focuses on analytical validation and clinical validation against a reference method, rather than a machine learning model that would typically have distinct training and test sets in the same way. The analytical studies (e.g., LoD, reactivity, cross-reactivity) might be seen as foundational "training" for the assay's design, but not in the sense of a machine learning training set with labeled data.
-
How the ground truth for the training set was established:
- As mentioned above, a "training set" in the context of machine learning is not described. For the development and analytical validation of the assay, ground truth would have been established through controlled laboratory experiments using well-characterized C. difficile strains and other microorganisms, as described in the Analytical Reactivity and Cross-Reactivity sections. The document indicates these strains were identified and confirmed (e.g., ATCC strains), which serves as their established "ground truth" for analytical purposes.
Ask a specific question about this device
(82 days)
INTELLIGENT MEDICAL DEVICES, INC
The IMDx Flu A/B and RSV for Abbott m2000 assay performed on the Abbott m2000 System is a qualitative in vitro diagnostic test designed for the detection of influenza B, and RSV RNA directly from nasopharyngeal swabs from patients with signs and symptoms of respiratory infection. The assay detects RNA from influenza A. influenza B, and RSV (A and B) using real-time, reverse transcription polymerase chain reaction (rRT-PCR) and fluorogenic target specific hybridization for unique sequences in the viral target genomes. The IMDx Flu A/B and RSV for Abbott m2000 assay is intended for use as an aid in diagnosing influenza A and/or RSV infection.
Negative results for influenza B, or RSV do not preclude influenza virus or RSV infection and should not be used as the sole basis for diagnosis, treatment, or patient management decisions. Conversely, positive results do not rule-out bacterial infection or co-infection with other viruses. The agent(s) detected may not be the definite cause of disease. The use of additional laboratory testing and clinical presentation must be considered when diagnosing respiratory viral infection.
Performance characteristics for influenza A were established during the 2011-2013 influenza seasons when Influenza A/2009 H1N1 and Influenza A/H3 were the predominant influenza A viruses in circulation. When other influenza A viruses are emerging, performance characteristics may vary,
If infection with a novel influenza A virus is suspected based on current clinical and epidemiological screening criteria recommended by public health authorities, specimens should be collected with appropriate infection control precautions for novel virulent influenza viruses and sent to state or local health department for testing. Viral culture should not be attempted in these cases unless a BSL 3+ facility is available to receive and culture specimens.
The IMDx Flu A/B and RSV for Abbott m2000 assay uses nucleic acid extraction and purification technology, performed on the Abbott m2000sp. combined with rRT-PCR. performed with the Abbott m2000rt, to generate and detect amplified products from influenza A, influenza B, and RSV RNA that is isolated from clinical specimens.
The assay targets the influenza A matrix (M) gene, influenza B non-structural protein (NS1) gene, and RSV A and RSV B fusion (F) gene. The presence of a viral RNA target sequence is indicated by the fluorescent signal generated through the use of fluorescently labeled oligonucleotide probes on the Abbott m2000rt instrument. The probes do not generate a signal unless they are specifically bound to the amplification cycle at which fluorescent signal is detected by the Abbott m2000rt is inversely proportional to the viral RNA target concentration present in the original sample.
An RNA bacteriophage species unrelated to influenza B, or RSV is introduced into each specimen during sample preparation to serve as a process control bacteriophage is lysed simultaneously with influenza A, influenza B and RSV B in the sample, and amplified in the same reaction as the viral RNA targets using rRT-PCR. The process control serves to demonstrate that the entire assay process has proceeded correctly for each sample.
Here's a breakdown of the acceptance criteria and the study details for the IMDx Flu A/B and RSV for Abbott m2000 assay:
1. Table of Acceptance Criteria and Reported Device Performance
The acceptance criteria for diagnostic devices often involve achieving a certain level of sensitivity and specificity against a defined ground truth. Based on the provided summary, the FDA cleared the device, implying that the observed performance met the agency's criteria. While explicit "acceptance criteria" are not listed as target percentages, the "Clinical Agreement Summary" tables present the device's performance metrics.
Target Organism | Performance Metric | Acceptance Criteria (Implied by Clearance) | Reported Device Performance (95% CI) |
---|---|---|---|
Influenza A | Sensitivity | Met FDA clearance standards | 97.6% (94% - 99%) |
Specificity | Met FDA clearance standards | 95.7% (94% - 97%) | |
Influenza B | Sensitivity | Met FDA clearance standards | 97.1% (90% - 99%) |
Specificity | Met FDA clearance standards | 97.2% (96% - 98%) | |
RSV | Sensitivity | Met FDA clearance standards | 97.2% (92% - 99%) |
Specificity | Met FDA clearance standards | 93.0% (91% - 95%) |
2. Sample Size Used for the Test Set and Data Provenance
- Test Set Sample Size: A total of 932 nasopharyngeal swab (NPS) samples were tested and analyzed for performance across two influenza seasons (2011-2012 and 2012-2013).
- Data Provenance: The samples were prospectively collected from four (2011-2012 season) and three (2012-2013 season) geographically diverse test sites within the United States. These were nasopharyngeal swabs collected for routine influenza testing. The study period was February-April 2012 and March-April 2013.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications
The document does not specify the number of experts or their qualifications for establishing the ground truth. It states that the performance was assessed "compared to viral culture followed by direct fluorescent antibody (DFA) detection." This implies laboratory techniques were used as the reference standard, rather than expert clinical consensus or interpretation of images by experts.
4. Adjudication Method for the Test Set
The document mentions that "Discordant samples were tested using the Nanosphere Verigene® Respiratory Virus Plus Nucleic Acid Test (RV+) on the Verigene® System and subsequent results are documented in the footnote." This indicates that discordant results (where the IMDx assay result did not match the initial ground truth) were adjudicated using a "third-party" FDA-cleared molecular assay (the predicate device).
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
- No, an MRMC comparative effectiveness study was not conducted. This device is an in vitro diagnostic test for detecting viral RNA, not an imaging device or a tool that directly assists human readers in interpreting clinical information.
6. Standalone (Algorithm Only) Performance Study
- Yes, a standalone performance study was done. The entire study described under "Clinical Agreement" and "Analytical Performance" evaluates the algorithm's performance in detecting viral RNA without human interpretation or intervention in the diagnostic process beyond running the assay. The results presented are exclusively for the device's analytical detection capabilities.
7. Type of Ground Truth Used
- Clinical Agreement: The primary ground truth for the clinical agreement study was viral culture followed by direct fluorescent antibody (DFA) detection. For discordant samples, a FDA cleared molecular assay (Nanosphere Verigene® Respiratory Virus Plus Nucleic Acid Test) was used for adjudication.
- Analytical Performance: For analytical studies (reproducibility, analytical sensitivity, reactivity, specificity, interference), the ground truth generally refers to precisely prepared "spiked" samples with known concentrations of viral strains or challenging substances.
8. Sample Size for the Training Set
The document does not provide information regarding the sample size for a training set. This is typical for FDA 510(k) summaries for IVD assays. The development and training of such assays often involve proprietary internal validation data not typically released in a 510(k) summary, which focuses on clinical validation for market clearance.
9. How the Ground Truth for the Training Set Was Established
As with the training set sample size, the document does not provide information on how the ground truth for any potential training set was established. This information is typically considered proprietary to the manufacturer's development process and not part of the public 510(k) summary.
Ask a specific question about this device
(223 days)
INTELLIGENT MEDICAL DEVICES, INC
The IMDx VanR for Abbott m2000 assay is an in vitro diagnostic assay that uses polymerase chain reaction (PCR) amplification for the qualitative detection of nucleic acids encoding the vancomycin resistance genes vanA and/or vanB. The assay is performed directly on human perirectal swabs, rectal swabs, or stool specimens from patients at risk for Vancomycin Resistant Enterococcus (VRE) colonization. The IMDx VanR for Abbott m2000 assay detects the presence of vanA and vanB genes that can be associated with vancomycin-resistant enterococci. The IMDx VanR for Abbott m2000 assay can be used as an aid to identify. prevent and control vancomycin-resistant colonization in healthcare settings. The IMDx VanR for Abbott m2000 assay is not intended to diagnose VRE infection nor to guide or monitor treatment of infection. Culture methods are necessary to recover organisms for epidemiology typing and confirmation testing.
The IMDx VanR for Abbott m2000 assay is a PCR-based assay that targets regions unique to the vanA and vanB vancomycin resistance genes that may be associated with vancomycin resistant Enterococcus (VRE). Detection of the vanA and vanB genes is measured by the presence of fluorescently-labeled oligonucleotide probes that generate a fluorescent signal when specifically bound to amplified vanA and/or vanB PCR products. Differentiation of vanA from vanB is attained by labeling the oligonucleotide probes with different colored fluorescent dyes. The amplification cycle at which fluorescent signal is detected by the Abbott m2000rt is inversely proportional to the vanA and vanB DNA target level present in the sample.
The IMDx VanR for Abbott m2000 assay includes reagents for the detection of the assay process control, which contains inactivated bacteria, unrelated to enterococci, and is introduced into each specimen during sample preparation. The process control (also acting as an internal control (IC)) is co-extracted with the specimen and co-amplified in the same PCR reaction as the vanA and vanB targets. The IC demonstrates that the entire assay process has proceeded within specification.
Here's an analysis of the provided information, focusing on the acceptance criteria and the study proving the device meets them:
Acceptance Criteria and Device Performance for IMDx VanR for Abbott m2000
The document provided, K123753, describes the IMDx VanR for Abbott m2000 assay, an in vitro diagnostic device for detecting vancomycin resistance genes. The acceptance criteria are implicitly derived from the performance goals demonstrated in the clinical studies, as explicit pre-defined acceptance criteria are not directly stated in the summary. The performance is reported as sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) based on comparison to a "gold standard" method.
1. Table of Acceptance Criteria and Reported Device Performance
Performance Metric | Acceptance Criteria (Implied) | Reported Device Performance (95% CI) |
---|---|---|
Peri-rectal Swab Specimens | ||
Sensitivity | High sensitivity for detecting vanA and/or vanB genes. | 94.0% (83.8% - 97.9%) |
Specificity | High specificity for differentiating between detected vanA/vanB and negative samples. | 93.9% (91.5% - 95.6%) |
Positive Predictive Value | Adequate PPV for indicating presence of vanA/vanB. | 58.8% (47.8% - 68.9%) |
Negative Predictive Value | High NPV for ruling out vanA/vanB presence. | 99.4% (98.3% - 99.8%) |
Rectal Swab Specimens (Prospective) | ||
Sensitivity | High sensitivity for detecting vanA and/or vanB genes. | 96.8% (89.1% - 99.1%) |
Specificity | High specificity for differentiating between detected vanA/vanB and negative samples. | 91.7% (88.3% - 94.2%) |
Positive Predictive Value | Adequate PPV for indicating presence of vanA/vanB. | 68.5% (58.3% - 77.2%) |
Negative Predictive Value | High NPV for ruling out vanA/vanB presence. | 99.4% (97.7% - 99.8%) |
Rectal Swab Specimens (Retrospective) | ||
Positive Percent Agreement | High agreement with the reference method for positive samples. | 100.0% (91.6% - 100.0%) |
Negative Percent Agreement | High agreement with the reference method for negative samples. | 0.0% (0.0% - 65.8%) Note: This low NPA is due to the specific composition of the retrospective cohort, where all "negative" samples by the IMDx assay were also negative by the reference standard. The total NEG for the IMDx assay was 0, meaning no calculation was possible. This requires careful interpretation. |
Stool Specimens | ||
Sensitivity | High sensitivity for detecting vanA and/or vanB genes. | 87.0% (77.0% - 93.0%) |
Specificity | High specificity for differentiating between detected vanA/vanB and negative samples. | 85.8% (82.0% - 88.8%) |
Positive Predictive Value | Adequate PPV for indicating presence of vanA/vanB. | 51.3% (42.3% - 60.2%) |
Negative Predictive Value | High NPV for ruling out vanA/vanB presence. | 97.4% (95.2% - 98.6%) |
Note on Acceptance Criteria: The document does not explicitly state numerical acceptance criteria (e.g., "Sensitivity must be >X%"). Instead, the clinical performance study aims to demonstrate that the device performs comparably and effectively for its intended use, implying that the reported percentages of sensitivity, specificity, PPV, and NPV are deemed acceptable for supporting substantial equivalence to the predicate device.
2. Sample Size Used for the Test Set and Data Provenance
- Test Set (Clinical Performance Study):
- Peri-rectal swabs: 587 samples
- Rectal swabs: 444 samples (further broken down into prospective and retrospective collections)
- Prospective: 400 samples
- Retrospective: 44 samples
- Stool specimens: 469 samples
- Total Clinical Samples: 587 + 444 + 469 = 1500 samples
- Data Provenance: Samples were collected from "five geographically diverse test sites within the United States." The study included both prospective and retrospective collection for rectal swabs.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of those Experts
The document does not specify the number or qualifications of experts used to establish the ground truth. It states that the ground truth was "enriched culture combined with confirmation of the molecular basis of vancomycin resistance of isolates using an alternate PCR method." This implies laboratory personnel with expertise in microbiology and molecular diagnostics, but no specific number or detailed qualifications are provided.
4. Adjudication Method for the Test Set
The document does not describe a formal adjudication method (e.g., 2+1, 3+1, none) for the test set. The ground truth was established by a combination of enriched culture and an alternate PCR method, implying a direct comparison against these established laboratory techniques rather than a human consensus process.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done
No, an MRMC comparative effectiveness study was not done. This study focuses on the in vitro diagnostic performance of the assay itself (device vs. reference method) rather than evaluating human reader performance with and without AI assistance.
6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) Was Done
Yes, the clinical performance study evaluates the IMDx VanR for Abbott m2000 assay in a standalone manner. The assay is an automated PCR-based diagnostic test, and its results are compared directly to the ground truth (enriched culture + alternative PCR). There is no human interpretation of images or other data being assisted by the algorithm; the assay itself generates the result.
7. The Type of Ground Truth Used
The ground truth used was a combination of enriched culture combined with confirmation of the molecular basis of vancomycin resistance of isolates using an alternate PCR method. This is a highly robust ground truth for detecting the presence of vanA and/or vanB genes and is considered a reference standard in microbiology.
8. The Sample Size for the Training Set
The document does not explicitly state the sample size for a "training set." This type of molecular diagnostic assay typically undergoes extensive analytical validation (e.g., LOD, reactivity, cross-reactivity) using characterized strains and spiked samples, and then its clinical performance is evaluated on a distinct clinical test set. It's possible that data from these extensive analytical studies could be considered analogous to training data for method optimization, but a defined "training set" in the context of machine learning is not applicable here as it's a rule-based PCR assay, not an AI/ML algorithm.
9. How the Ground Truth for the Training Set Was Established
As noted above, a traditional "training set" as understood in AI/ML is not described. However, for the analytical studies (e.g., Analytical Reactivity, Challenge Study), the ground truth was established using "well-characterized vancomycin-resistant Enterococcus strains and/or clinical isolates" and other organisms. These strains would have their vanA/vanB status, and other genetic characteristics, confirmed through standard microbiological and molecular methods (e.g., sequencing, biochemical tests, established PCR assays).
Ask a specific question about this device
Page 1 of 1