Search Results

The Joint Biological Agent Identification and Diagnostic System (JBAIDS) Plague Detection Kit is a real-time polymerase chain reaction (PCR) test kit intended for the qualitative in vitro diagnostic (IVD) detection of target DNA sequences of Yersinia pestis. The kit can be used to test human whole blood collected in sodium citrate or sputum collected aseptically from individuals greater than 18 years of age suspected of having septic or pneumonic plague. In addition, positive blood cultures and colonies may be tested. The JBAIDS Plague Target 2 assay is used as a supplementary test only after a positive result with the Target I Assay.

The JBAIDS Plague Target 1 and Target 2 assays are run on the JBAIDS instrument using the Diagnostic Wizard. Results are for the presumptive identification of Y. pestis in conjunction with culture and other laboratory tests. The definitive identification of Y. pestis from colony growth, liquid blood culture growth, or from blood or sputum specimens requires additional testing and confirmation procedures in consultation with public health or other authorities for whom reports are required.

The diagnosis of plague must be made based on history, signs, symptoms, exposure likelihood, and other laboratory evidence in addition to the identification of Y. pestis from cultures or directly from whole blood or sputum specimens.

The JBAIDS Plague Detection Kit is intended for use by trained clinical laboratory personnel who have received specific training on the use of the JBAIDS Plague Detection Kit. The level of Y. pestis that would be present in blood or sputum from individuals with early systemic infection is unknown. Due to the difficulty in obtaining clinical specimens, these assays were not evaluated with blood or sputum from individuals with septic or pneumonic plague.

The Joint Biological Agent Identification and Diagnostic System (JBAIDS) Plague Detection System is a fully integrated IVD system composed of the portable JBAIDS instrument, laptop computer and software and the JBAIDS Plague Detection Kit with two different freeze-dried PCR assays for detection of Yersinia pestis DNA. The system has been validated using four different sample preparation kits for isolating DNA from wole blood (IT 1-2-3TM Platinum Path and QFLOWdana Sample Purification Kits), sputum (IT 1-2-3TM Platinum Path and IT 1-2-3TM VIBE), positive blood cultures (IT 1-2-3TM SWIPE Sample Purification Kit), and plate cultures (IT 1-2-3™ Platinum Path and SWIPE Sample Purification Kits). Use of the JBAIDS DNA Extraction Control Kit is also recommended.

Prior to testing, specimens are processed using BioFire Diagnostic's IT 1-2-3 Sample Purification Kits. The resulting purified sample is added to Target 1 Unknown and Target 1 Inhibition Control vials, along with reconstitution buffer. Target 1 Positive Control and Negative Control vials are prepared using reconstitution buffer and water. When Y. pestis DNA is present, a fragment of Y. pestis DNA is amplified. The amplicon is detected by fluorescence using a specific hydrolysis probe. Each probe is labeled on one end with a fluorescent reporter moiety (6-carboxyfluorescein (6-FAM)) and elsewhere with a quencher moiety (carboxy tetramethylrhodamine (TAMRA)). When the probe is intact, the quencher absorbs the light emitted by the reporter moiety. During PCR, the probe hybridizes to the target sequence before the exonuclease activity of Taq polymerase hydrolyzes the probe, separating the fluorophore from the quencher and permitting detection of the fluorescent signal generated by the reporter. The fluorescent signal increases as additional templates are amplified and more probes are hydrolyzed.

JBAIDS Software analyzes the fluorescence amplification curves and reports results as positive, negative, uncertain or inhibited. A failure of the Positive or Negative Control will result in the entire run being called invalid. Retesting is required to resolve uncertain, invalid or inhibited results. The Target 2 assay is used as a supplementary test only after a positive result is obtained with the Target 1 assay.

This document describes the validation of the JBAIDS Plague Detection Kit, specifically the modification for use with the IT 1-2-3™ Platinum Path Sample Purification Kit Accessory. The study aims to demonstrate that the new purification method provides equivalent performance to previously validated methods.

1. Table of Acceptance Criteria and Reported Device Performance

The acceptance criteria are implied by the "equivalence" study design, where the performance of the new extraction method (Platinum Path) is compared to the previously validated methods (QFLOWdna for whole blood, VIBE for sputum). The goal is to show that the Platinum Path method performs equivalently to the existing methods. For the clinical performance, the reported metrics are Positive Percent Agreement (PPA) and Negative Percent Agreement (NPA). For Limit of Detection (LoD), the acceptance criterion is the confirmation of the previously established LoD. For reproducibility, the criteria are defined by the agreement with expected results and consistency across sites.

2. Sample Sizes used for the Test Set and Data Provenance

• Clinical Performance (Whole Blood):
  • Test Set Sample Size: 100 surrogate whole blood samples (50 spiked with inactivated Y. pestis, 50 unspiked).
  • Data Provenance: Prospectively collected specimens from febrile volunteers in the USA (November 2012 - April 2013). This is surrogate data, not from actual plague patients.
• Clinical Performance (Sputum):
  • Test Set Sample Size: 100 surrogate sputum samples (50 spiked with inactivated Y. pestis, 50 unspiked).
  • Data Provenance: Residual frozen sputum specimens (presumably from the USA). This is surrogate data.
• Limit of Detection:
  • Whole Blood LoD Confirmation: 20 independent whole blood specimens.
  • Sputum LoD Confirmation: 20 independent sputum specimens.
• Reproducibility:
  • Whole Blood: A panel of 12 blood samples (4 medium positive, 4 low positive, 4 negative) tested twice a day for 4 days at 3 sites = 12 samples * 2 tests/day * 4 days * 3 sites = 288 individual tests for each target.
  • Sputum: A panel of 9 sputum samples (3 medium positive, 3 low positive, 3 negative) tested twice a day for 5 days at 3 sites = 9 samples * 2 tests/day * 5 days * 3 sites = 270 individual tests for each target.
• Detection of Direct Culture Samples: 10 Y. pestis colonies and 10 non-Y. pestis colonies.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications

The ground truth for the clinical performance and LoD studies was established by spiking samples with known quantities of inactivated Y. pestis. In the clinical performance comparison, the result obtained from the previously validated extraction method (QFLOWdna for blood, VIBE for sputum) was considered the "correct result" to which the new method (Platinum Path) was compared.
For the reproducibility study, the samples were prepared with known spike levels, making the expected outcome clear.
For the direct culture detection, the ground truth was the known identity of the bacterial colonies (Y. pestis vs. non-Y. pestis).

This is a molecular diagnostic test, so there isn't an "expert" interpreting images or clinical signs to establish ground truth. The ground truth is determined by the controlled experimental design (spiking, known bacterial cultures) and the performance of the predicate device.

4. Adjudication Method for the Test Set

Not applicable in the traditional sense for a molecular diagnostic device measuring defined targets in spiked samples or known cultures. The "adjudication" in the clinical equivalency studies was implicitly done by comparing the new method's results to those of the previously validated predicate method. JBAIDS operators were blinded to the analyte content of the samples.

5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance

Not applicable. This is a molecular diagnostic test, not an AI-powered diagnostic imaging device requiring human interpretation of results. The device provides a qualitative "positive," "negative," "uncertain," or "inhibited" result.

6. If a standalone (i.e., algorithm only without human-in-the-loop performance) was done

Yes, the device operates autonomously to detect target DNA sequences and report results (positive, negative, uncertain, or inhibited) based on its pre-programmed algorithms. The performance data presented (PPA, NPA, LoD confirmation, reproducibility, direct culture detection) are all standalone performance metrics of the device with the new extraction method. Human operators perform the sample preparation and initiate the test, but the interpretation of the PCR amplification curves and the final result determination are automated by the JBAIDS software.

7. The Type of Ground Truth Used

• Spiking studies (Clinical Performance, LoD, Reproducibility): Ground truth was established by spiking clinically relevant matrices (whole blood, sputum) with known quantities of inactivated Y. pestis. For the clinical performance comparison, the results from the previously cleared extraction methods served as the comparator "ground truth" to determine equivalency.
• Direct Culture Detection: Ground truth was established by using known bacterial colonies (Y. pestis vs. non-Y. pestis).

8. The Sample Size for the Training Set

The document does not explicitly mention a "training set" in the context of developing the JBAIDS Plague Detection Kit or its modification. This is a PCR-based assay, not a machine learning algorithm that typically requires a large training dataset. The device utilizes pre-defined molecular targets and amplification curves for detection. Any 'training' would refer to the initial development and optimization of the PCR primers/probes and reaction conditions, rather than a data-driven machine learning training set described here.

9. How the Ground Truth for the Training Set Was Established

As noted above, a "training set" in the machine learning sense is not applicable here. The development of the JBAIDS Plague Detection Kit and its components (like the PCR assays and sample purification methods) would involve:

• Design and optimization: Identifying specific DNA targets for Y. pestis.
• Analytical validation: Testing the assay with characterized strains of Y. pestis and related/unrelated organisms at various concentrations.
• Limit of Detection (LoD) determination: Empirically determining the lowest concentration consistently detected.
• Specificity and Sensitivity testing: Challenging the assay with known positive and negative samples.

These processes would utilize known samples and cultures where the presence/absence and concentration of Y. pestis are precisely controlled.

Ask a specific question about this device