Search Results

The BioFire Joint Infection (JI) Panel is a multiplexed nucleic-acid-based, in vitro diagnostic test intended for use with BioFire FilmArray 2.0 and BioFire FilmArray Torch Systems for the simultaneous qualitative detection and identification of multiple bacterial and yeast nucleic acids and select antimicrobial resistance genes from synovial fluid obtained from individuals suspected to have a joint infection.

The following organisms are identified using the BioFire JI Panel: Anaerococcus prevotii/vaginalis, Bacteroides fragilis, Candida spp., Candida albicans, Citrobacter, Clostridium perfringens, Cutibacterium avidum/granulosum, Enterobacter cloacae complex, Enterococcus faecalis, Enterococcus faecium, Escherichia coli, Fingoldia magna, Haemophilus influenzae, Kingella kingae, Klebsiella aerogenes, Klebsiella pneumoniae group, Morganella morganii. Neisseria gonorrhoeae, Parvimonas micra, Peptoniphilus, Peptostreptococcus anaerobius, Proteus spp., Pseudomonas aeruginosa, Salmonella spp., Serratia marcescens, Staphylococcus aureus, Staphylococcus lugdunensis, Streptococcus spp., Streptococcus agalactiae, Streptococcus pneumoniae, and Streptococcus pyogenes.

The BioFire JI Panel contains assays for the detection of genetic determinants associated with S. aureus resistance to methicillin (mecA/C) in conjunction with the SCCmec right extremity junction (MREJ), enterococcal resistance to vancomycin (vanA and vanB), and some mechanisms of gram-negative bacterial resistance ß-lactams including penicillins, cephalosporins, monobactams, and carbapenems (blactx.M, blakec, blaNDM, blaOXA-48-like; blavin). Detection of these genetic determinants can aid in the identification of potentially antimicrobial-resistant organisms in synovial fluid samples. The antimicrobial resistance gene or marker detected may or may not be associated with the agent responsible for disease. Negative results for these select antimicrobial resistance gene assays do not indicate susceptibility, as multiple mechanisms of resistance to methicillin, vancomycin, and ß-lactams exist.

The BioFire JI Panel is indicated as an aid in the diagnosis of specific agents of joint infection and results should be used in conjunction with other clinical and laboratory findings. Negative results may be due to infection with pathogens that are not detected by this test, pathogens present below the limit of detection of the assay, or infection that may not be detected in a synovial fluid specimen. Positive results do not rule out co-infection with other organisms. The BioFire JI Panel is not intended to monitor treatment for joint infections.

Culture of synovial fluid is necessary to recover organisms for susceptibility testing and epidemiological typing, to identify organisms in the synovial fluid that are not detected by the BioFire JI Panel, and to further identify species in the genus, complex or group results.

The BioFire Joint Infection (JI) Panel is designed to simultaneously identify 39 different bacteria, yeast, and select genetic determinants of antimicrobial resistance from synovial fluid specimens. The BioFire JI Panel is compatible with BioFire's PCR-based in vitro diagnostic BioFire FilmArray 2.0 and BioFire FilmArray Torch Systems for infectious disease testing. A panel-specific software module (i.e., BioFire JI Panel pouch module software) is used to perform BioFire JI Panel testing on these systems.

A test is initiated by loading Hydration Solution into one port of the BioFire JI Panel pouch and the synovial fluid sample mixed with the provided Sample Buffer into the other port of the BioFire JI Panel pouch and placing it in a FilmArray instrument. The pouch contains all of the reagents required for specimen testing and analysis in a freeze-dried format; the addition of Hydration Solution and Sample/Buffer Mix rehydrates the reagents. After the pouch is prepared, the BioFire Software guides the user through the steps of placing the pouch into the instrument, scanning the pouch barcode, entering the sample identification, and initiating the run.

The FilmArray instrument contains a coordinated system of inflatable bladders and seal points. which act on the pouch to control the movement of liguid between the pouch blisters. When a bladder is inflated over a reagent blister, it forces liquid from the blister into connecting channels. Alternatively, when a seal is placed over a connecting channel it acts as a valve to open or close a channel. In addition, electronically-controlled pneumatic pistons are positioned over multiple plungers in order to deliver the rehydrated reagents into the blisters at the appropriate times. Two Peltier devices control heating and cooling of the pouch to drive the PCR reactions and the melt curve analysis.

Nucleic acid extraction occurs within the FilmArray pouch using mechanical and chemical lysis followed by purification using standard magnetic bead technology. After extracting and purifying nucleic acids from the unprocessed sample, the Film Array performs a nested multiplex PCR that is executed in two stages. During the first stage, the FilmArray performs a single, large volume, highly multiplexed reverse transcription PCR (rt-PCR) reaction. The products from first stage PCR are then diluted and combined with a fresh, primer-free master mix and a fluorescent double stranded DNA binding dye (LC Green Plus, BioFire Diagnostics). The solution is then distributed to each well of the array. Array wells contain sets of primers designed specifically to amplify sequences internal to the PCR products generated during the first stage PCR reaction. The 2nd stage PCR, or nested PCR, is performed in singleplex fashion in each well of the array. At the end of the 2nd stage PCR, the array is interrogated by melt curve analysis for the detection of signature amplicons denoting the presence of specific targets. A digital camera placed in front of the 2nd stage PCR captures fluorescent images of the PCR reactions and software interprets the data.

The FilmArray Software automatically interprets the results of each DNA melt curve analysis and combines the data with the results of the internal pouch controls to provide a test result for each organism on the panel.

This document describes the BioFire Joint Infection (JI) Panel, a multiplexed nucleic-acid-based in vitro diagnostic test for the simultaneous qualitative detection and identification of multiple bacterial and yeast nucleic acids and select antimicrobial resistance genes from synovial fluid.

Here's an analysis of the acceptance criteria and the studies that prove the device meets these criteria:

1. Table of Acceptance Criteria and Reported Device Performance

The core acceptance criteria for this device, based on the provided document, are generally demonstrated through analytical and clinical performance studies, specifically in terms of Sensitivity (Positive Percent Agreement - PPA) and Specificity (Negative Percent Agreement - NPA) for detecting various analytes (bacteria, yeast, and antimicrobial resistance genes). While explicit numerical "acceptance criteria" for PPA and NPA are not stated in the document as a single, overarching threshold, the individual performance metrics for each analyte in the clinical and contrived studies implicitly define the expected performance. The document does state: "The validation results met the predefined acceptance criteria of >95% accuracy as compared to expert annotation." for individual melt curves and assay calls, which is a key technical acceptance criterion. We will, therefore, summarize the clinical performance for a representative selection of analyt analytes as reported in the prospective clinical study.

Note: The reported performance above focuses on the prospective clinical study (Table 56) for general analytes and AMR genes (Table 58). Additional archived specimen study (Table 79) and contrived specimen testing (Tables 80 and 81) were conducted to supplement prevalence. The "Overall" rows in these tables represent broader performance. A key technical acceptance criterion was ">95% accuracy as compared to expert annotation" for individual melt curves (99.59% sensitivity, 99.98% specificity) and assay calls (99.49% sensitivity and 99.97% specificity).

2. Sample sizes used for the test set and the data provenance

Test Set for Clinical Performance:

• Sample Size: 1544 synovial fluid specimens (after exclusions from an initial 1591).
• Data Provenance:
  • Country of Origin: U.S. and Europe (13 geographically distinct study sites).
  • Retrospective or Prospective: Primarily prospective clinical study. A separate "Archived Specimen Study" was also conducted with 134 retrospective frozen archived specimens, and "Contrived Specimen Testing" was performed using residual clinical samples (N=1235).

Test Set for Reproducibility:

• Sample Size: 480 valid runs overall (20 replicates per sample and system, across 3 sites and 12 different samples/conditions).
• Data Provenance: The document implies these were contrived samples tested at multiple laboratory locations; the geographical origin and whether they were retrospective or prospective are not explicitly stated, but they are laboratory-prepared samples.

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts

The document refers to "expert annotation" for validating the Melt Detector and Analysis Software. However, it explicitly states:

• "Annotations (positive and negative calls) for all melt curves and assay calls were determined by the sponsor."
• It does not specify the number or qualifications of these "experts" in detail. It suggests the sponsor (BioFire Diagnostics) established these expert annotations internally using well-characterized samples from clinical and analytical testing.

For the clinical performance studies, the ground truth was established by comparator methods:

• Bacterial and Yeast Analytes: Standard of Care (SoC) culture, supplemented by a single PCR assay followed by quantitative molecular assay that included sequencing (tMol).
• AMR Genes: Single PCR assay followed by sequencing (from the specimen) and a separate PCR on cultured isolates at BioFire. Standard manual and automated phenotypic AST of appropriate cultured isolates was performed at the study sites as SoC testing.

The "experts" in this context are the clinicians and laboratory personnel performing the SoC testing and the molecular testing for ground truth establishment. Their specific qualifications are not detailed beyond "standard of care" and "sequencing-based comparator methods."

4. Adjudication method for the test set

For the clinical performance evaluation (Sections V.C.11, 12):

• Discrepancy Investigation: "Samples for which false positive and/or false negative results (i.e., discrepant results) were obtained when comparing the BioFire Joint Infection Panel results to the comparator method results were further investigated."
• Adjudication Process:
  1. Discrepant bacterial and yeast samples were first examined by an independent molecular assay performed directly on the specimen.
  2. If the discrepancy remained, the study site was queried to ensure accurate data entry in the Case Report Form (CRF).
  3. If still unresolved, results from additional laboratory testing were considered.
• Outcome: "Results from the discrepancy testing did not change the final performance estimates." This suggests that even after investigation, the initial comparison results stood for the presented performance metrics. The underlying molecular comparator was considered the definitive ground truth.

5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance

No, a multi-reader multi-case (MRMC) comparative effectiveness study was NOT done.

This device is an automated, qualitative nucleic acid amplification assay (PCR-based IVD). It does not involve human readers interpreting images or data for diagnosis in the way an AI-assisted diagnostic device might for radiology, pathology, or ophthalmology. The "results" are automatically interpreted by the FilmArray Software. Therefore, the concept of "human readers improving with AI vs. without AI assistance" is not applicable to this type of device. The interpretation is performed algorithmically.

6. If a standalone (i.e., algorithm only without human-in-the-loop performance) was done

Yes, a standalone performance evaluation of the algorithm was inherently conducted.

The BioFire JI Panel is designed to be an automated system. The "FilmArray Software automatically interprets the results of each DNA melt curve analysis and combines the data with the results of the internal pouch controls to provide a test result for each organism on the panel."

The "Melt Detector" and "Analysis Software" performance against "expert annotation" (as described in section VI.A.7) directly reflects the standalone performance of the algorithm in interpreting raw data into assay calls.

• Melt Detector (individual melt curves): Sensitivity 99.59%, Specificity 99.98%
• Analysis Software (assay calls): Sensitivity 99.49%, Specificity 99.97%

The clinical performance studies (prospective, archived, contrived) also demonstrate the standalone performance of the "algorithm only" device against established ground truth (SoC culture and molecular methods). Human input is limited to specimen preparation and loading, and initiation/monitoring of the run; interpretation of the complex raw data is automated by the software.

7. The type of ground truth used (expert consensus, pathology, outcomes data, etc.)

The ground truth used depends on the analyte:

• Bacterial analytes and yeast analytes: Standard of Care (SoC) culture was the primary comparator, supplemented by a single PCR assay for the organism of interest followed by a quantitative molecular assay that included sequencing (tMol). A specimen was considered positive if bi-directional sequencing data met predefined quality acceptance criteria and matched NCBI GenBank sequences. If two PCR comparator assays were used, a negative result from both was considered Negative.
• Antimicrobial Resistance (AMR) genes: A single PCR assay (from the specimen) followed by sequencing. A separate PCR was also performed on cultured isolates. Additionally, Standard manual and automated phenotypic AST (Antimicrobial Susceptibility Testing) of appropriate cultured isolates was performed at the study sites as SoC testing, which was used for comparison in Tables 75-78.

Therefore, the ground truth primarily relies on a combination of:

• Culture (SoC)
• Molecular methods (PCR + Sequencing)
• Phenotypic AST

8. The sample size for the training set

The document does not explicitly specify a "training set" in the context of machine learning, as this device is a PCR-based assay with rule-based software interpretation, not a machine learning model that typically undergoes iterative training and validation phases with separate datasets.

The development and "optimization parameters" of the FilmArray system and its Melt Detector/Analysis Software would have involved extensive internal data and potentially diverse sets of isolates to establish appropriate thresholds and rules. However, these are not presented as a "training set" in the common AI/ML sense. The document refers to "well-characterized samples, collected as part of the clinical evaluation and analytic testing of the Bone and Joint Panel" being used to determine sensitivity and specificity of the Melt Detector and Analysis Software as compared to expert annotation, which implies these were part of the testing/validation phase rather than algorithm training.

9. How the ground truth for the training set was established

As explained in point 8, a "training set" in the machine learning sense is not explicitly detailed. The "expert annotation" described in section VI.A.7 for validating the Melt Detector and Analysis Software "were determined by the sponsor." This would involve internal BioFire experts reviewing raw melt curve data and possibly other molecular characterization data to define true positive/negative signals.

For the analytical performance studies, which involve method development and optimization, the ground truth for various isolates (e.g., in Analytical Reactivity, Precision, LoD) would be based on:

• Reference strains/isolates: e.g., ATCC strains, AR Bank isolates, NCTC, DSM, CCUG (as seen in Tables 7-48).
• Known concentrations: "contrived samples containing known concentrations of organisms" (section VI.A.6).
• In silico analysis of sequences available in public databases (for analytical reactivity/specificity, section VI.A.3).

Ask a specific question about this device