(72 days)
The Xpert HCV test, performed on the GeneXpert Xpress System, is an automated in vitro reverse transcription polymerase chain reaction (RT-PCR) test for the qualitative detection of hepatitis C virus (HCV) RNA in human fingerstick K2-EDTA whole blood from adult individuals at risk and/or with signs and symptoms of HCV infection with or without antibody evidence of HCV infection. Detection of HCV RNA indicates that the virus is replicating and therefore is evidence of active infection. Detection of HCV RNA does not discriminate between acute and chronic states of infection.
The Xpert HCV test is not intended for monitoring patients undergoing treatment or for use in screening blood, plasma, or tissue donors.
The Xpert HCV test, is an automated qualitative in vitro reverse transcription polymerase chain reaction (RT-PCR) test. The Xpert HCV test is performed on the GeneXpert Xpress System. With this system an operator can run the test by performing four steps: 1) mix the specimen. 2) transfer the liquid sample to the cartridge with a transfer pipette, 3) run the test on the instrument, and 4) read the results.
The GeneXpert Xpress System (Hub configuration) consists of a GeneXpert IV instrument that conducts the sample preparation, nucleic acid amplification and real-time fluorescent signal detection for the test, and a GeneXpert Hub with preloaded GeneXpert Xpress software for running the tests and viewing the results. The GeneXpert Hub accessory integrates the computer, touchscreen monitor and barcode scanner. Each of the GeneXpert modules in the GeneXpert IV instrument can perform separate sample preparation and testing. The module contains a syringe drive for dispensing fluids (i.e., the syringe drive activates the plunger that works in concert with the rotary valve in the cartridge to move fluids between chambers), an ultrasonic horn for lysing cells or spores, and a proprietary I-CORE (Intelligent Cooling/Heating Optical Reaction) thermocycler for performing real-time PCR and RT-PCR and detection.
The Xpert HCV test requires the use of a single-use disposable GeneXpert cartridge that contains all necessary reagents for the detection of HCV RNA. Because the cartridges are self-contained, the risk of cross-contamination between samples is minimized. The Xpert HCV test includes reagents for the detection of HCV RNA in clinical specimens as well as a sample processing control (SPC) and internal control high (IC-H) used to control for adequate processing of the target and to monitor the presence of inhibitor(s) in the RT and PCR reactions. The Probe Check Control (PCC) verifies reagent rehydration. PCR tube filling in the cartridge, probe integrity, and dye stability. The Sample Volume Adequacy (SVA) control ensures the sample was correctly added to the cartridge and verifies that the correct volume of sample has been added to the sample chamber.
The Xpert HCV test is designed for use with human K2-fingerstick EDTA whole blood. The BD Microtainer for capillary whole blood collection was validated for use with the Xpert HCV test. After collecting human fingerstick EDTA whole blood in the BD Microtainer, a 100μl aliquot of the specimen is transferred to the sample chamber of the Xpert HCV cartridge using the transfer pipette supplied in the Xpert HCV kit.
The sample results are interpreted by the GeneXpert Xpress System from measured fluorescent signals and embedded calculation algorithms and are shown in the View Results window. It also reports if the test has encountered an instrument error or produces no result and needs to be repeated.
This document describes the evaluation of the Xpert HCV test for an automatic Class III designation. The test is an automated qualitative reverse transcription polymerase chain reaction (RT-PCR) test for the qualitative detection of hepatitis C virus (HCV) RNA.
Here's a breakdown of the requested information:
1. Table of Acceptance Criteria and Reported Device Performance
The acceptance criteria for the Xpert HCV test are implied by the reported performance in both analytical and clinical studies. For analytical performance, the agreement rates for precision and reproducibility are presented. For clinical performance, Positive Percent Agreement (PPA) and Negative Percent Agreement (NPA) compared to a Patient Infected Status (PIS) algorithm are key metrics.
| Acceptance Criteria (Implied) | Reported Device Performance |
|---|---|
| Analytical Performance | |
| Precision Percent Agreement (e.g., for 1.5x LoD and 3.0x LoD) | Range: 95.0% - 100% (Table 2, Precision Study) |
| Reproducibility Percent Agreement (e.g., for 1.5x LoD and 3.0x LoD) | Range: 96.7% - 100% (Table 4, Multi-Site Reproducibility) |
| Analytical Specificity: No cross-reactivity with common organisms | None of 27 tested organisms showed cross-reactivity. |
| Interference: No interference from common endogenous/exogenous substances or autoimmune conditions | None of the tested substances or conditions interfered. |
| Limit of Detection (LoD) | Genotype 1a: 35.0 IU/mL; Genotype 1b: 41.5 IU/mL; etc. (Table 10) |
| Non-determinate rate | Initial: 6.0%; Overall (after retest): 1.2% |
| Clinical Performance | |
| Positive Percent Agreement (PPA) | 93.44% (95% CI: 87.59% – 96.64%) (Table 14) |
| Negative Percent Agreement (NPA) | 99.77% (95% CI: 99.16% – 99.94%) (Table 14) |
2. Sample Sizes Used for the Test Set and Data Provenance
- Precision Study Test Set: The precision study used a total of 60 replicates per panel member (e.g., negative, GT1a 1.5x LoD, etc.). The study was conducted in a blinded and randomized manner.
- Multi-Site Reproducibility Study Test Set: A total of 90 replicates per panel member were tested across three sites. The study was blinded and randomized.
- Clinical Study Test Set: 1,012 fingerstick whole blood (FS) specimens were initially collected. After exclusions due to protocol deviations, unresolved non-determinate results, and non-evaluable comparator test results, 982 FS samples were included in the performance calculations (Table 13, 14).
- Non-Viral Hepatitis Samples Study: 78 FS samples from individuals with non-viral hepatitis were included, with 68 being evaluable for performance comparison (Table 15).
- Data Provenance for Clinical Study: Prospective, all-comers blinded clinical study conducted at 15 CLIA waived sites located in the US.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts
The document does not specify the number or qualifications of experts used to establish the ground truth for the test set. Instead, it states that the "patient infected status (PIS) algorithm based on results from an FDA approved HCV RNA test and an antibody test" served as the comparator/ground truth for the clinical study. This implies that the ground truth was established by the performance of these approved reference tests, rather than by a panel of human experts directly reviewing the samples.
4. Adjudication Method for the Test Set
The document does not describe an explicit adjudication method (e.g., 2+1, 3+1) involving human experts for the test set. The clinical study's ground truth was established by comparing the Xpert HCV test results to the "patient infected status (PIS) algorithm based on results from an FDA approved HCV RNA test and an antibody test." Any discrepancies would likely be resolved by the established reference method rather than expert adjudication of the fingerstick samples themselves. The retesting of "Two (2) specimens (1 false positive and 1 false negative) with suspicion of specimen handling and testing errors" suggests a method for investigating discrepancies, but not a general expert adjudication for all cases.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance
A Multi-Reader Multi-Case (MRMC) comparative effectiveness study comparing human readers with and without AI assistance was not done. This device is an automated qualitative RT-PCR test, and its output (HCV DETECTED/NOT DETECTED) is a direct result from the instrument, not interpreted by human readers in the traditional sense that an AI would assist. The "operators" mentioned in the analytical studies are those performing the physical test steps, not interpreting diagnostic images or complex medical data.
6. If a Standalone (i.e. algorithm only without human-in-the-loop performance) was done
Yes, a standalone performance evaluation was done. The Xpert HCV test is an automated system where the "sample results are interpreted by the GeneXpert Xpress System from measured fluorescent signals and embedded calculation algorithms" and results are "displayed in the View Results window." The clinical study (Section VI.C.3) directly evaluates the performance of this automated system against the patient infected status algorithm. The operators in the study are described as "untrained," further emphasizing the standalone nature of the device's interpretation.
7. The Type of Ground Truth Used
The ground truth used for the clinical study was the Patient Infected Status (PIS) algorithm, which was "based on results from an FDA approved HCV RNA test and an FDA approved HCV antibody test." This constitutes a form of established clinical diagnosis reference standard (or "outcomes data" in a broader sense of established clinical status).
8. The Sample Size for the Training Set
The document does not specify a sample size for a training set. As an RT-PCR diagnostic device, the "training" of the device involves the internal development and calibration of the assay (e.g., primer and probe design, thermal cycling conditions, fluorescent signal thresholds, embedded calculation algorithms) by the manufacturer based on analytical studies and known HCV characteristics, rather than a classical machine learning training phase on a large clinical dataset.
9. How the Ground Truth for the Training Set Was Established
Since no explicit training set is mentioned in the context of machine learning, there is no description of how ground truth for a training set was established. The development of the assay likely involved extensive analytical studies (e.g., determining LoD for various genotypes, specificity, interference) using well-characterized HCV strains and clinical samples to establish the assay's performance parameters and the embedded algorithms' decision rules. These characterization efforts would form the basis of the device's "knowledge," but it's not a "training set" in the sense of supervised learning for an AI algorithm that learns from labeled data.
N/A