Search Results

The Joint Biological Agent Identification and Diagnostic System (JBAIDS) Influenza A Subtyping Kit is intended for the in vitro qualitative detection and differentiation of seasonal Influenza A/H1, seasonal Influenza A/H3, and 2009 H1N1Influenza viral nucleic acids isolated and purified from nasopharyngeal swab (NPS) and nasopharyngeal wash (NPW) specimens from human patients with signs and symptoms of respiratory infection, in conjunction with clinical and epidemiological risk factors. The JBAIDS Influenza A Subtyping Kit contains reverse transcriptase real-time polymerase chain reaction (rRT-PCR) assays for use on the JBAIDS instruments. The Flu A H1. Flu A H3. and Flu A H1 2009 assays of the JBAIDS Influenza A Subtyping Kit target a region of the hemagglutinin (HA) gene of the respective Influenza A virus. The Flu A Sw assay of the JBAIDS Influenza A Subtyping Kit targets a region of the nucleocapsid protein (NP) gene of the 2009 H1N1 Influenza virus, as well as some other Influenza A viruses of swine lineage. This kit is not intended to detect Influenza B or Influenza C viruses.

A negative result for all assays in the JBAIDS Influenza A Subtyping Kit is a presumptive negative result for Influenza A. These results should be confirmed using the JBAIDS Influenza A & B Detection Kit.

Test results are to be used in conjunction with other clinical and epidemiological information. Negative results do not preclude influenza virus infection and should not be used as the sole basis for treatment or other patient management decisions.

Due to low seasonal prevalence, performance characteristics for detection of seasonal Influenza A/H1 were established primarily with retrospective and contrived clinical specimens.

All users, analysts, and any person reporting diagnostic results from use of this device should be trained to perform and interpret the results from this procedure by JBAIDS instructors or designees prior to use. Use of this device is limited to designated Department of Defense (DoD) laboratories equipped with the JBAIDS instruments.

If infection with a novel influenza A virus is suspected based on current clinical and epidemiological screening criteria recommended by public health authorities, specimens should be collected with appropriate infection control precautions for novel virulent influenza viruses and sent to state or local health departments for testing. Viral culture should not be attempted in these cases unless a biosafety laboratory (BSL) 3+ facility is available to receive and culture specimens.

The JBAIDS Influenza A Subtyping Kit is a real time RT-PCR test kit, which, when used with the JBAIDS instrument and software, allows the qualitative in vitro detection and identification of influenza A subtypes H1, H3, and H1 2009 (swine lineage) viral RNA. The assays have been optimized as freeze-dried assays with primer and fluorescent-probe sets for the detection of influenza A/H1, A/H3, and A/2009 H1 viral RNA. In particular, the Flu A H1, Flu A H3, and Flu A H1 2009 assays target distinct regions of the hemagglutinin gene specific to those subtypes, and the Flu A Sw assay targets a region of the nucleocapsid protein gene as a secondary target for the influenza A/2009 H1(swine lineage) virus. The tests are performed using the JBAIDS instrument and software.

Here's an analysis of the acceptance criteria and study details for the JBAIDS Influenza A Subtyping Kit based on the provided text:

Preamble Regarding Acceptance Criteria:
The document provided, a 510(k) Summary, details performance data but does not explicitly state pre-defined "acceptance criteria" in the format of pass/fail thresholds for the clinical study. Instead, it presents the achieved performance metrics (PPA and NPA with 95% Confidence Intervals) from the clinical and analytical studies, implicitly demonstrating that these achieved results were deemed sufficient for substantial equivalence. For the purpose of this response, I will present the reported performance as the "acceptance criteria met," recognizing that the specific numerical targets for acceptance are not explicitly listed in this summary.

1. Table of Acceptance Criteria and Reported Device Performance

As described above, explicit acceptance criteria (i.e., specific numerical targets prior to testing) were not provided in the 510(k) summary. However, the reported performance from the clinical studies served as the basis for the device's clearance. The table below summarizes the reported clinical performance.

2. Sample Size Used for the Test Set and Data Provenance

• Clinical Performance (Prospective Study):

  • Sample Size: 795 valid specimens (312 NPS, 483 NPW) were analyzed. 6% (44/795) required retesting due to invalid/inconclusive/unsubtypeable initial results.
  • Data Provenance: Prospective, collected from 5 geographically separated military clinical sites in the U.S. during the 2010-2011 influenza season (December 2010 to April 2011).

• Testing of Preselected Archived Samples (for Seasonal Influenza A/H1):

  • Sample Size: 51 NPS specimens (30 known positive seasonal Influenza A/H1, 21 influenza-negative).
  • Data Provenance: Retrospective, pre-selected archived clinical NPS specimens obtained and confirmed at two different clinical study sites in the U.S.

• Testing of Surrogate Clinical Specimens (for Seasonal Influenza A/H1):

  • Sample Size: 136 individual influenza-negative clinical specimens (68 NPS, 68 NPW) were spiked with known concentrations of seasonal Influenza A/H1 virus. Valid results obtained for 128 (62 NPW, 66 NPS).
  • Data Provenance: Contrived clinical samples generated from residual influenza-negative NPS and NPW samples. Sent to two different clinical trial sites in the U.S.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications

The document does not explicitly state the number or qualifications of individual experts who established the ground truth. However, the ground truth for the clinical and archived samples was established using a reference method:

• "The reference method was the CDC rRT-PCT Flu Panel influenza A and influenza B assays, followed by subtype-specific PCR and bi-directional sequencing of amplicons."
• "the presence of Influenza A/H1 viral RNA was confirmed using 'validation' PCR assays. The validation PCR assays were identical to the comparator assays that were used for the prospective clinical evaluation study."

This implies that the "experts" were the personnel performing and interpreting the CDC assays and sequencing results, which would typically be highly trained laboratory professionals and molecular biologists proficient in these techniques, likely adhering to CDC protocols. No specific number of such experts is given.

4. Adjudication Method for the Test Set

The document states: "JBAIDS Software analyzes the fluorescence amplification curves and reports results as positive, negative, or uncertain. A failure of the Positive or Negative Control will result in the entire run being called invalid. Retesting is required to resolve uncertain or invalid results."

For the clinical study, 6% of specimens (44/795) required retesting. Specifically, "Invalid" (32/44), "Inconclusive" (1/44), and "Unsubtypeable" (1/44). "Forty (40) out of 44 samples resolved upon a 1st retest and the remaining 4 samples required a re-extraction and retest and resolved." This indicates a retesting and re-extraction protocol for initial indeterminate or invalid results rather than a multi-expert adjudication panel for interpretive discrepancies. The ground truth itself (CDC rRT-PCR + sequencing) would have its own internal validation/adjudication processes, but this is not detailed for external review.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

No multi-reader, multi-case (MRMC) comparative effectiveness study comparing human readers with and without AI assistance was performed. This device is a diagnostic kit (real-time RT-PCR assay) that provides automated qualitative results, not an imaging AI diagnostic aid for human interpretation. Therefore, the concept of "human readers improve with AI vs without AI assistance" does not directly apply to this type of device.

6. Standalone Performance Study (Algorithm Only Without Human-in-the-Loop Performance)

Yes, the studies described (clinical performance, archived samples, surrogate samples) represent standalone performance of the algorithm/device. The "JBAIDS Software analyzes the fluorescence amplification curves and reports results as positive, negative, or uncertain." While human operators perform the physical tests, the interpretation of the raw PCR data into a final qualitative result (positive/negative/uncertain) is automated by the device's software (algorithm). The clinical performance data presented (PPA and NPA) are a direct measure of this standalone performance against a defined ground truth.

7. Type of Ground Truth Used

The ground truth used was:

• Expert Consensus/Reference Method: For prospective clinical specimens and archived samples, the reference method was the CDC rRT-PCR Flu Panel influenza A and influenza B assays, followed by subtype-specific PCR and bi-directional sequencing of amplicons. This is a highly robust and accepted laboratory method for viral identification and subtyping.
• Known Spiked Concentration: For surrogate clinical specimens, the ground truth for positive samples was established by spiking influenza-negative clinical samples with known concentrations of seasonal Influenza A/H1 virus. Un-spiked negative samples also served as ground truth negatives.

8. Sample Size for the Training Set

The document does not provide details on a specific "training set" sample size or data for the development of the device's algorithm, as would typically be described for machine learning or AI models. This device is a PCR-based assay, where the "algorithm" is primarily the pre-defined thresholds and analysis logic within the JBAIDS software to interpret PCR amplification curves. Performance characteristics are primarily established through analytical validation (LoD, inclusivity, exclusivity) and clinical validation with independent test sets, rather than an explicit training-validation split in the context of an adaptive or learning algorithm. The design of the primers and probes, and the associated software interpretation, would have been iteratively developed and optimized, but a formal "training set" as understood in AI/ML is not mentioned.

9. How the Ground Truth for the Training Set Was Established

As noted above, a formal "training set" for an AI/ML algorithm is not described for this diagnostic kit. The underlying principles of the PCR assay (primer/probe design, reaction conditions) are established through extensive analytical studies to ensure specificity and sensitivity. The "ground truth" for developing the analytical performance characteristics (like LoD, exclusivity, inclusivity) would involve testing well-characterized viral strains and non-target organisms with known concentrations, typically obtained from reference collections. This process establishes the analytical sensitivity and specificity that the device is designed to achieve when interpreting a sample.

Ask a specific question about this device

The Joint Biological Agent Identification and Diagnostic System (JBAIDS) Influenza A & B Detection Kit is intended for use on the JBAIDS instruments, for the in vitro qualitative detection of Influenza A and Influenza B viral nucleic acids isolated and purified from nasopharyngeal swab (NPS) and nasopharyngeal wash (NPW) specimens from human patients with signs and symptoms of respiratory infection. The JBAIDS Influenza A & B Detection Kit contains reverse transcriptase real-time polymerase chain reaction (rRT-PCR) assays that target the Matrix protein gene of Influenza A viruses, and the Non-structural protein gene of Influenza B viruses. This kit is not intended to detect Influenza C viruses.

Test results are to be used in conjunction with other clinical and epidemiological information. Negative results do not preclude influenza virus infection and should not be used as the sole basis for treatment or other patient management decisions.

Performance characteristics for detection of influenza A were established when 2009 H1N1 Influenza, Influenza A H1N1, and Influenza A H3N2 were the predominant influenza A viruses in circulation. Due to low seasonal prevalence, performance characteristics for detection of seasonal Influenza A/H1 were established primarily with retrospective and surrogate clinical specimens. When other influenza A viruses are present, performance characteristics may vary.

All users, analysts, and any person reporting diagnostic results from use of this device should be trained to perform and interpret the results from this procedure by JBAIDS instructors or designees prior to use. Use of this device is limited to designated Department of Defense (DoD) laboratories equipped with the JBAIDS instruments.

If infection with a novel influenza A virus is suspected based on current clinical and epidemiological screening criteria recommended by public health authorities, specimens should be collected with appropriate infection control precautions for novel virulent influenza viruses and sent to state or local health departments for testing. Viral culture should not be attempted in these cases unless a biosafety laboratory (BSL) 3+ facility is available to receive and culture specimens.

The JBAIDS Influenza A & B Detection Kit is a rRT-PCR test kit, which, when used with the JBAIDS instrument and software, allows the qualitative in vitro detection of influenza A and B viral RNA. These two assays have been optimized as freeze-dried assays with primer and fluorescent-probe sets for the detection of influenza A and B viral RNA. In particular, the influenza A assay targets a region of the matrix gene specific to the influenza A virus genera, and the influenza B assay targets a region of the non-structural gene specific to the influenza B virus genera. The tests are performed using the previously FDA-cleared JBAIDS instrument and software. A human gene target assay (RNAseP target) will be used as an inhibition and extraction control.

Here's a breakdown of the acceptance criteria and the study proving the device meets them, based on the provided text:

1. Table of Acceptance Criteria and Reported Device Performance

For the JBAIDS Influenza A & B Detection Kit, the primary performance metrics are Positive Percent Agreement (PPA) and Negative Percent Agreement (NPA) compared to a reference method. The document does not explicitly state pre-defined acceptance criteria (e.g., "PPA must be >90%"). However, the reported performance is presented to demonstrate substantial equivalence, implying that these agreement percentages were considered acceptable for clearance.

2. Sample Sizes and Data Provenance

• Clinical Performance (Prospective Study):
  • Test Set Sample Size: 804 valid specimens (314 NPS, 490 NPW).
  • Data Provenance: Prospective study conducted at 5 geographically separated military clinical sites in the U.S. during the 2010-2011 influenza season (December 2010 to April 2011).
• Testing of Preselected Archived Specimens:
  • Test Set Sample Size: 51 NPS specimens (30 known positive seasonal Influenza A/H1, 21 influenza-negative).
  • Data Provenance: Retrospective, pre-selected archived clinical NPS specimens. Country of origin not explicitly stated but implied to be US-based given the context of US military sites.
• Testing of Surrogate Clinical Specimens:
  • Test Set Sample Size: 136 individual influenza-negative clinical specimens (68 NPS, 68 NPW) that were spiked. 128 produced valid test results for analysis.
  • Data Provenance: Contrived clinical samples created by spiking residual influenza-negative NPS and NPW samples. Origin of the original negative specimens is clinical, likely from the US military sites or similar sources.
• Analytic Studies (LoD, Inclusivity, Exclusivity, Reproducibility):
  • These studies used distinct sample sizes, generally involving multiple replicates (e.g., 20 replicates for LoD, 30-90 results per spike level for reproducibility). The samples were either live, quantified virus strains or simulated NPS/NPW matrices spiked with specific organisms.

3. Number of Experts and Qualifications for Ground Truth

The document does not mention the use of human experts to establish ground truth for the test sets in the clinical performance study.

• Clinical Performance (Prospective, Archived, and Surrogate Studies): The ground truth was established by a comparator method, the CDC rRT-PCR Flu Panel influenza A and influenza B assays. This is a laboratory-based reference standard, not a human expert consensus.
• Analytic Studies: The ground truth for these (e.g., LoD, inclusivity, exclusivity) was based on the known presence and concentration of specific viruses or bacteria in the spiked samples.

4. Adjudication Method

There is no mention of an adjudication method in the studies described, as the comparator method (CDC rRT-PCR Flu Panel) served as the direct reference standard. The results of the JBAIDS kit were compared to this objective method.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

No MRMC comparative effectiveness study was performed or described. This device is an in-vitro diagnostic (IVD) assay designed for laboratory use, not for direct interpretation by human readers in the same way an image-based diagnostic AI might be. Therefore, the concept of human readers improving with AI assistance is not applicable in this context. The device's results are intended to be interpreted by trained laboratory personnel.

6. Standalone Performance

Yes, a standalone performance study was done. The entire set of clinical and analytical performance evaluations (PPA, NPA, LoD, inclusivity, exclusivity, reproducibility) directly measures the performance of the JBAIDS Influenza A & B Detection Kit algorithm and associated testing procedure in isolation (without human-in-the-loop performance influencing the assay's output). The device's software automatically analyzes fluorescence amplification curves and reports results (positive, negative, or uncertain).

7. Type of Ground Truth Used

• Clinical Performance (Prospective, Archived, Surrogate): The ground truth was based on the results from a legally marketed comparator device: the CDC rRT-PCR Flu Panel influenza A and influenza B assays. This is a highly sensitive and specific laboratory-based molecular diagnostic method, often considered a "gold standard" for viral nucleic acid detection.
• Analytical Studies (LoD, Inclusivity, Exclusivity, Reproducibility): The ground truth was based on known concentrations of specific viral or bacterial strains in the spiked samples.

8. Sample Size for the Training Set

The document does not specify a separate training set for the JBAIDS Influenza A & B Detection Kit. As an IVD based on rRT-PCR, the "training" analogous to machine learning models is typically inherent in the assay design and optimization process (e.g., primer and probe design, reaction condition optimization), rather than a distinct data-driven training phase on labeled samples as seen in AI/ML products. The mentioned studies focus on the validation of the optimized kit.

9. How the Ground Truth for the Training Set was Established

As no explicit training set is described in the context of an AI/ML model, this question is not directly applicable. If one considers the development and optimization of the rRT-PCR assays as an analogous process, the "ground truth" during this phase would involve using well-characterized viral isolates and clinical samples with confirmed presence/absence of influenza to optimize the assay's sensitivity and specificity parameters.

Ask a specific question about this device

The Joint Biological Agent Identification and Diagnostic System (JBAIDS) Q Fever Detection Kit is a qualitative real-time polymerase chain reaction (PCR) test kit intended to identify and detect target DNA sequence from Coxiella burnetii in serum collected from individuals suspected of having acute Q fever, typically 7-10 days after onset of symptoms or before antibody formation. This in vitro diagnostic (IVD) test is intended to aid in the diagnosis of Q fever in individuals presenting with signs and symptoms of acute Q fever when used in conjunction with other clinical and laboratory findings. This kit is only intended to aid in the diagnosis of Q fever of patients presenting in the acute stage of the disease. Negative results do not preclude C. burnetii infection and should not be used as the sole basis for diagnosis, treatment or other management decisions.

The JBAIDS Q Fever assay is run on the JBAIDS instrument using the Diagnostic Wizard. Results are for the presumptive identification of C. burnetii in conjunction with serology and/or other laboratory tests. The following considerations also apply:

• The diagnosis of acute Q fever must be made based on history, signs, symptoms, exposure likelihood, and other laboratory evidence, in addition to the identification of C. burnetii from serum specimens.
• Sensitivity is decreased by ~25-40%, with no change in specificity, if the sample is collected after the patient has formed specific antibodies to C. burnetii, typically 7-10 days after onset of symptoms.
• The definitive identification of C. burnetii from serum specimens requires additional testing and confirmation procedures in consultation with public health or other authorities for whom reports are required.

The Joint Biological Agent Identification and Diagnostic System (JBAIDS) Q Fever Detection Kit is a fully integrated in-vitro diagnostic (IVD) system composed of the JBAIDS instrument with laptop computer, software, a freeze-dried reagent assay for the qualitative detection of pathogenic Coxiella burnetii, and 2 different sample preparation protocols for isolating target DNA from serum. Use of the JBAIDS DNA Extraction Control kit (not included) is also recommended.

The JBAIDS instrument, using Polymerase Chain Reaction (PCR) technology, is a portable thermocycler and real-time fluorimeter. The JBAIDS Q Fever Detection Kit is specially designed for PCR in glass capillaries using the JBAIDS instrument and hydrolysis probes for sequence-specific detection of the C. burnetii IS1111 DNA target.

The reagent kit contains 3 different types of freeze-dried reagent vials: Positive Controls, Negative Controls, and Unknowns (used for testing the patient sample). Each JBAIDS assay requires a Positive and Negative Control, and each sample is tested using an Unknown reagent vial which contains a multiplexed target and inhibition control (IC) assay.

Prior to testing, serum samples are purified using the Idaho Technology IT 1-2-3™ QFLOW or IT 1-2-3™ Platinum Path Sample Purification Kit. The resulting purified sample is added to an Unknown reagent vial, along with reconstitution buffer. A Positive Control and a Negative Control vial are prepared using reconstitution buffer and reagent grade water. Aliquots from each reagent vial are transferred to 2 reaction capillaries that are tested together in the JBAIDS instrument. The instrument is programmed to perform heating and cooling cycles that drive the PCR process. The heating and cooling cycles are generated using a heating coil and varying fan speeds. Fluorescence emission is monitored over 1 of 3 wavelengths, and the instrument software interprets the change in fluorescence to determine whether the target DNA is present.

When the organism is present, a fragment of C. burnetii DNA is amplified using specific primers. The amplicon is detected by fluorescence using a specific hydrolysis probe. The hydrolysis probe contains a short oligonucleotide that hybridizes to an internal sequence of the amplified fragment during the annealing phase of the PCR cycle. This probe has the 5' and 3' ends labeled with a reporter dye and a quenching dye, respectively. When the probe hybridizes to the specific DNA target, the Taq polymerase enzyme replicating the targetspecific DNA hydrolyzes the probe, which separates the two fluorophores, thus allowing the reporter dye to fluoresce. Fluorescence is monitored at a wavelength of 530 nm. Inhibition is monitored using an internal inhibition control. The inhibition control consists of a linearized plasmid containing an artificial intervening sequence flanked by target assay primer sequences. Similar to the target, the IC probe also has the 5' and 3' ends labeled with a reported and quenching dye, respectively. Hydrolysis of the IC probe during amplification is monitored at a wavelength of 705 nm.

The level of fluorescence from each unknown sample and control is measured by the JBAIDS instrument. JBAIDS Software analyzes fluorescence amplification curves and reports results as Positive, Negative, Inhibited, or Uncertain. A failure of the Positive or Negative Control will result in the entire run being called Invalid. Failure of the Inhibition Control when no target amplification is observed yields an Inhibited result for the associated sample and requires retesting of that sample. A positive result for the target will override an inhibited result.

Here's a breakdown of the acceptance criteria and study details for the JBAIDS Q Fever Detection Kit, based on the provided text:

1. Table of Acceptance Criteria and Reported Device Performance

The document does not explicitly state formal "acceptance criteria" in a quantitative, pre-defined manner for the clinical study. Instead, it presents performance characteristics (sensitivity and specificity) from a clinical trial. The analytical performance (LoD and exclusivity) directly reports observed results without listing specific target criteria.

• 10/10 (100%) of isolates in C. burnetii inclusivity panel correctly identified at determined system LoD. |
  | • Exclusivity (Specificity) | No cross-reactivity with phylogenetically related or commonly found organisms. | - 22 of 22 non-C. burnetii strains tested in exclusivity panel were negative at high concentration.
• In silico evaluation of Ehrlichia and Neorickettsia indicated no cross-reactivity. |
  | • Reproducibility | Highly reproducible test results at or above LoD (Implied: High agreement with expected positive results; low %CV for Cp values). | - Agreement with Expected Positive Results (Detection ≥ LoD): 100% (204/204) across all sites and both purification kits (95% CI: 98.2-100.0%).
• Agreement at LoD/15 (low concentration): 52% (53/102) across all sites and both purification kits (95% CI: 41.8-62.0%).
• Average Cp %CV: Low %CV values for 5X LoD, LoD, and LoD/15, indicating consistent Cycle threshold values. Percent CV for 5X LoD and LoD was generally below 3%. |
  | Clinical Studies | | |
  | • Clinical Sensitivity | Detect C. burnetii in samples from acute Q fever patients (Implied: Reasonable detection rate, especially prior to antibody formation). | For samples prior to seroconversion (Acute Q Fever, Seronegative):
• Site 1: Platinum Path: 47% (9/19; 95% CI=24-71%); QFLOWdna: 29% (5/17; 95% CI=10-56%).
• Site 2: Platinum Path: 77% (20/26; 95% CI=56-91%); QFLOWdna: 81% (22/27; 95% CI=62-94%). (Note: Sensitivity decreased significantly in seropositive samples). |
  | • Clinical Specificity | No false positives in samples without C. burnetii (Implied: High negative predictive value). | For serology negative samples (no antibody to C. burnetii):
• Site 1: Platinum Path: 100% (40/40; 95% CI=91-100%); QFLOWdna: 100% (39/39; 95% CI=91-100%).
• Site 2: Platinum Path: 100% (71/71; 95% CI=95-100%); QFLOWdna: 100% (72/72; 95% CI=95-100%).
• Overall, 306/307 (99.7%) of serology negative samples gave expected negative results. One false positive was observed from a sample that failed to show a four-fold antibody titer rise. |

2. Sample Size Used for the Test Set and Data Provenance

• Test Set Description: Banked, frozen serum specimens.
• Data Provenance: The specimens were sequentially received during a specified time period. The clinical study involved three independent laboratories located in distinct geographical regions: Australia, France, and the Netherlands. This indicates the data is retrospective.
• Total Subjects Evaluated: 749 subjects.
• Subjects Included in Analysis: 465 subjects.
• Exclusions:
  • 86 for inconclusive or incomplete serology.
  • 21 due to environmental contamination.
  • 7 for inconclusive JBAIDS results.
  • All 170 samples tested at Site 3 due to deviations from the study protocol.
• Total Final Test Results (after purification):
  • Platinum Path: 324 results (170 from serology positive, 154 from serology negative).
  • QFLOWdna: 324 results (171 from serology positive, 153 from serology negative).
    (Note: 183 specimens had adequate volume for purification by both methods, contributing to the total results.)
• Breakdown for Sensitivity/Specificity Calculation:
  • Seronegative, No Antibody to C. burnetii (Specificity):
    • Site 1: 40 (Platinum Path) + 39 (QFLOWdna) = 79 samples
    • Site 2: 71 (Platinum Path) + 72 (QFLOWdna) = 143 samples
    • Total: 222 samples
  • Seronegative, Failed 4-fold Antibody Rise (where 1 false positive occurred): 85 samples
  • Seropositive (Sensitivity, where seroconversion occurred):
    • Site 1: 19 (Platinum Path) + 17 (QFLOWdna) = 36 samples
    • Site 2: 26 (Platinum Path) + 27 (QFLOWdna) = 53 samples
    • Total: 89 samples (for samples exhibiting seroconversion)

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts

The ground truth for the clinical study was established by standard paired serological testing for Q fever, which had been previously performed on the banked specimens.
The document does not specify the number or qualifications of experts who performed or interpreted this serological testing. It refers to "standard paired serological testing," implying established laboratory procedures.

4. Adjudication Method for the Test Set

The document does not describe an explicit "adjudication method" involving independent review of results for the clinical study. The ground truth was based on the outcome of "standard paired serological testing." The JBAIDS results were compared against this serological ground truth. There is no mention of multiple experts reviewing individual cases or a conciliation process if disagreements arose.

5. If a Multi Reader Multi Case (MRMC) Comparative Effectiveness Study was Done, If So, What was the effect size of how much human readers improve with AI vs without AI assistance

No, an MRMC comparative effectiveness study involving human readers and AI assistance was not done.
The JBAIDS Q Fever Detection Kit is a real-time PCR assay, an in vitro diagnostic (IVD) device, not an AI-powered diagnostic imaging or interpretation system that would typically involve human "readers." The device provides automated test interpretation and report generation.

6. If a Standalone (i.e. algorithm only without human-in-the-loop performance) was done

Yes, a standalone performance evaluation was done. The JBAIDS Q Fever Detection Kit is described as a "fully integrated in-vitro diagnostic (IVD) system" with "automated test interpretation and report generation." The clinical and analytical studies measure the performance of this system operating in a standalone capacity, producing "Positive, Negative, Inhibited, or Uncertain" results based on fluorescence amplification curves. While human operators are involved in sample preparation and loading, the interpretation of the PCR data is handled by the instrument's software algorithms, without human interpretative input into individual test results.

7. The Type of Ground Truth Used

The ground truth used for the clinical study was based on standard paired serological testing for Q fever.
Specifically, for establishing sensitivity, it relied on samples exhibiting a four-fold rise in antibodies (indicating seroconversion, a definitive sign of infection). For establishing specificity, it relied on samples with no antibody to C. burnetii confirmed by serology.

8. The Sample Size for the Training Set

The document does not specify the sample size for a training set. As this is a molecular diagnostic kit (PCR), the "training" would typically involve optimization and validation of primers, probes, and reaction conditions rather than machine learning on a dataset with ground truth. The performance data presented (LoD, exclusivity, reproducibility, clinical studies) are evaluations of the finalized kit.

9. How the Ground Truth for the Training Set was Established

Since no explicit "training set" for an algorithm is mentioned or implied, the question of how its ground truth was established is not applicable in the typical sense of machine learning. The kit's design and analytical performance (e.g., selection of primers and probes for C. burnetii IS1111 DNA sequences) would have been developed against known strains and samples, but these are part of assay development and analytical validation, not a separate "training set" with ground truth in the context of an AI/ML device.

Ask a specific question about this device

The Joint Biological Agent Identification and Diagnostic System (JBAIDS) Plague Detection Kit is a real-time polymerase chain reaction (PCR) test kit intended for the qualitative in vitro diagnostic (IVD) detection of target DNA sequences of Yersinia pestis. The kit can be used to test human whole blood collected in sodium citrate or sputum collected aseptically from individuals greater than 18 years of age suspected of having septic or pneumonic plague. In addition, positive blood cultures and colonies may be tested. The JBAIDS Plague Target 2 assay is used as a supplementary test only after a positive result with the Target 1 Assay.
The JBAIDS Plague Target 1 and Target 2 assays are run on the JBAIDS instrument using the Diagnostic Wizard. Results are for the presumptive identification of Y. pestis in conjunction with culture and other laboratory tests. The definitive identification of Y. pestis from colony growth, liquid blood culture growth, or from blood or sputum specimens requires additional testing and confirmation procedures in consultation with public health or other authorities for whom reports are required.

The diagnosis of plague must be made based on history, signs, symptoms, exposure likelihood, and other laboratory evidence in addition to the identification of Y. pestis from cultures or directly from whole blood or sputum specimens.

The JBAIDS Plague Detection Kit is intended for use by trained clinical laboratory personnel who have received specific training on the use of the JBAIDS Plague Detection Kit. The level of Y. pestis that would be present in blood or sputum from individuals with early systemic infection is unknown. Due to the difficulty in obtaining clinical specimens, these assays were not evaluated with blood or sputum from individuals with septic or pneumonic plague.

The Joint Biological Agent Identification and Diagnostic System (JBAIDS) Plague Detection Kit is a real-time polymerase chain reaction (PCR) reagent kit, which, when used with the JBAIDS instrument and software, allows the qualitative in vitro diagnostic (IVD) detection of target DNA sequences within the pathogenic bacterium, Yersinia pestis, the causative agent of plague. The kit contains two assays, Plague Target 1 and Target 2, each of which consists of oligonucleotide primers and a fluorescent-labeled target assay probe that specifically detect Y. pestis DNA. The Target 2 assay is reserved for samples that test positive with the Target 1 assay. The kit is designed for use with the JBAIDS instrument, a portable thermocycler and real-time fluorimeter that performs PCR in glass capillaries.

Before testing, samples are purified using Idaho Technology's 1-2-3TM Sample Purification Kits (or validated equivalent). The resulting purified sample is added to an Unknown reagent vial and an Inhibition Control reagent vial, along with reconstitution buffer. When the organism is present, a fragment of Y. pestis DNA is amplified. The amplicon is detected by fluorescence using a specific hydrolysis probe. The hydrolysis probe contains a short oligonucleotide that hybridizes to an internal sequence of the amplified fragment during the annealing phase of the PCR cycle. This probe has the 5' and 3' ends labeled with a reporter dye and a quenching dye, respectively. When the probe hybridizes to the specific DNA target, the Taq polymerase enzyme replicating the target-specific DNA hydrolyzes the probe, separating the two fluorophores and allowing the reporter dye to fluoresce.

The JBAIDS instrument measures the level of fluorescence from each unknown sample and control. JBAIDS Software analyzes the fluorescence amplification curves and reports results as positive, negative, inhibited, or uncertain. A failure of the Positive or Negative Control will result in the entire run being called invalid. Failure of the Inhibition Control yields an inhibited result when the associated sample has a negative result for the target assay and requires retesting of that sample.

Due to the nature of the provided document (a 510(k) summary for a diagnostic test kit), the study performed is a diagnostic accuracy study rather than a study evaluating the performance of an AI/ML device. Therefore, many of the requested fields (MRMC study, effect size of human readers improving with AI, number of experts for ground truth establishment, etc.) are not applicable and will be marked as "N/A".

Here's an analysis of the provided text based on your request:

JBAIDS Plague Detection Kit Performance Study Analysis

1. Table of Acceptance Criteria and Reported Device Performance

The document does not explicitly state "acceptance criteria" for the clinical trial in terms of specific performance metrics that needed to be met for FDA clearance. However, it details the design of the clinical specificity study and the resulting performance. For analytical studies, it provides "LOD" (Limit of Detection) and inclusivity/exclusivity data which serve as performance measures.

2. Sample Size Used for the Test Set and Data Provenance

• Analytical Test Set:
  • Inclusivity: 18 isolates of virulent Y. pestis (subtypes and strains)
  • Exclusivity: A "panel" of 24 isolates (phylogenetically related and unrelated organisms)
  • Data Provenance: Not explicitly stated, but typically research laboratory settings for analytical studies. Retrospective in nature.
• Clinical Test Set:
  • Whole Blood: 132 samples
  • Sputum: 36 samples
  • Data Provenance: The study was a "multisite clinical trial". The samples were obtained from "subjects with clinical signs and symptoms consistent with systemic plague and for whom a blood and/or sputum culture had been ordered." Since clinical plague is rare, and the document explicitly states "Due to the difficulty in obtaining clinical specimens, these assays were not evaluated with blood or sputum from individuals with septic or pneumonic plague," these samples were likely from individuals suspected of plague but ultimately found to be negative. Thus, these are clinical samples, but representative of a "negative" population in the context of plague in humans. Retrospective in nature.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications

• N/A. For this in vitro diagnostic (IVD) device, the ground truth for analytical studies is established by known bacterial stocks and for clinical studies, by established laboratory culture methods.
• The ground truth for the clinical study was "laboratory culture results" which were considered the "gold standard." These methods are typically performed by trained microbiologists or clinical laboratory scientists, not "experts" in the sense of physicians adjudicating medical imaging or complex interpretations.

4. Adjudication Method for the Test Set

• N/A. For analytical studies, the ground truth (presence/absence of specific bacteria, concentration) is determined by established microbiological methods and known samples, without adjudication.
• For the clinical specificity study, the ground truth was "laboratory culture results." These are typically definitive for bacterial presence/absence. No adjudication among experts is mentioned or typically needed for this type of ground truth.

5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done

• No. This is an in vitro diagnostic (IVD) device, not an AI/ML device requiring human-in-the-loop performance evaluation. The device provides an automated "positive, negative, inhibited, or uncertain" result.
• Effect size of how much human readers improve with AI vs. without AI assistance: N/A.

6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) Was Done

• Yes, in essence. The JBAIDS system operates as a standalone diagnostic test. The "JBAIDS Software analyzes the fluorescence amplification curves and reports results as positive, negative, inhibited, or uncertain." This is the standalone performance of the analytical system.
• While the kit is "intended for use by trained clinical laboratory personnel," their role is to operate the instrument and interpret the automated results within a clinical context, not to perform a subjective interpretation of a raw signal that the algorithm then enhances.

7. The Type of Ground Truth Used

• Analytical Studies: Known strains/isolates of Y. pestis and other bacteria, with confirmed identification and concentrations (e.g., CFU/mL).
• Clinical Studies: "Laboratory culture results," which are considered the "gold standard" for bacterial identification in clinical samples.

8. The Sample Size for the Training Set

• Not explicitly stated in terms of a "training set." For IVDs, assay development and optimization (which can be considered analogous to training) involve numerous preliminary experiments with various dilutions, strains, and conditions, but these aren't typically documented as a formal "training set" with a defined size in the same way an AI/ML model would. The document describes the assay's design and then its validation/testing.

9. How the Ground Truth for the Training Set Was Established

• N/A (as no formal "training set" is described). However, for the development of PCR assays (like this kit), the ground truth for optimizing primers and probes would be established by:
  • Using purified genomic DNA of Yersinia pestis and other closely related and common organisms.
  • Sequencing to confirm target regions.
  • Utilizing known bacterial cultures and dilutions with confirmed concentrations.
  • These foundational truths are based on established molecular biology and microbiology techniques.

Ask a specific question about this device

The Joint Biological Agent Identification and Diagnostic System (JBAIDS) Tularemia Detection Kit is a real-time polymerase chain reaction (PCR) test system intended for the qualitative in vitro diagnostic (IVD) detection of target DNA sequences of Francisella tularensis. The system can be used to test human whole blood collected in sodium citrate or sputum collected aseptically from individuals greater than 18 years of age suspected of having tularemia. In addition, positive blood cultures and colonies may be tested. This assay is intended to aid in diagnosis of individuals presenting with signs and symptoms of pneumonic or typhoidal tularemia. It is not intended to aid in the diagnosis of glandular, ulceroglandular, oculoglandular, or oropharyngeal tularemia.

The JBAIDS Tularemia Detection Kit is run on the JBAIDS instrument using the Diagnostic Wizard. Results are for the presumptive identification of F. tularensis, in conjunction with culture and other laboratory tests. The definitive identification of F. tularensis from colony growth, liquid blood culture, blood specimens, or sputum specimens requires additional testing and confirmation procedures in consultation with public health or other authorities for whom reports are required.

The diagnosis of tularemia infection must be made based on history, signs, symptoms, exposure likelihood, and other laboratory evidence, in addition to the identification of the target either from colonies, blood culture, whole blood specimens, or sputum specimens.

The JBAIDS Tularemia Detection Kit is intended for use by trained clinical laboratory personnel who have received specific training on the use of the JBAIDS Tularemia Detection kit. The level of F. tularensis that would be present in blood or sputum of individuals with early systemic or pneumonic infection is unknown. Due to the difficulty in obtaining clinical specimens, the assay was not evaluated with blood or sputum from individuals presenting with signs and symptoms of tularemia and who subsequently developed pneumonic or typhoidal tularemia.

The Joint Biological Agent Identification and Diagnostic System (JBAIDS) Tularemia Detection Kit is a real-time polymerase chain reaction (PCR) reagent kit, which, when used with the JBAIDS instrument and software, allows the qualitative in vitro diagnostic (IVD) detection of a target DNA sequence within the pathogenic bacterium, Francisella tularensis, the causative agent of tularemia. Key components of the kit include oligonucleotide primers and a fluorescent-labeled target assay probe that specifically detects F. tularensis DNA. The kit is designed for use with the JBAIDS instrument, a portable thermocycler and real-time fluorimeter that performs PCR in glass capillaries.
Before testing, samples are purified using Idaho Technology's 1-2-37M Sample Purification Kits (or validated equivalent). The resulting purified sample is added to an Unknown reagent vial and an Inhibition Control reagent vial, along with reconstitution buffer. When the organism is present, a fragment of F. tularensis DNA is amplified. The amplicon is detected by fluorescence using a specific hydrolysis probe. The hydrolysis probe contains a short oligonucleotide that hybridizes to an internal sequence of the amplified fragment during the annealing phase of the PCR cycle. This probe has the 5' and 3' ends labeled with a reporter dye and a quenching dye, respectively. When the probe hybridizes to the specific DNA target, the Taq polymerase enzyme replicating the target-specific DNA hydrolyzes the probe, separating the two fluorophores and allowing the reporter dye to fluoresce.

The JBAIDS instrument measures the level of fluorescence from each unknown sample and control. JBAIDS Software analyzes the fluorescence amplification curves and reports results as positive, negative, inhibited, or uncertain. A failure of the Positive or Negative Control will result in the entire run being called invalid. Failure of the Inhibition Control yields an inhibited result when the associated sample has a negative result for the target assay and requires retesting of that sample.

Here's an analysis of the provided text regarding the acceptance criteria and study for the JBAIDS Tularemia Detection Kit, structured according to your request:

1. Table of Acceptance Criteria and Reported Device Performance

2. Sample Size Used for the Test Set and Data Provenance

• Analytical Testing (Inclusivity): 27 isolates of F. tularensis (subtypes and strains).
• Analytical Testing (Exclusivity): 24 organisms (phylogenetically related and unrelated).
• Clinical Specificity Testing:
  • Whole Blood: 132 samples
  • Sputum: 36 samples
• Data Provenance: The document does not explicitly state the country of origin for the clinical samples. It indicates a "multisite clinical trial was conducted," suggesting multiple sites, likely within the United States given it's an FDA submission. The clinical data is retrospective in nature, as it was limited to assessing clinical specificity using samples from individuals suspected of tularemia and for whom culture had been ordered. The submission notes the difficulty in obtaining clinical specimens from individuals with confirmed pneumonic or typhoidal tularemia.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts

The document does not specify the number or qualifications of experts used to establish ground truth for the test set.

4. Adjudication Method for the Test Set

The document does not explicitly describe an adjudication method for the test set. For the analytical and clinical studies, the reference method (culture for clinical specificity, identification of known organisms for analytical studies) served as the comparator without detailing an adjudication process for discrepancies.

5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was Done

No, a Multi-Reader Multi-Case (MRMC) comparative effectiveness study was not done. The device is an in vitro diagnostic (IVD) PCR kit, designed for automated laboratory analysis, not human interpretation of images or other data requiring multiple readers. The study primarily focused on the device's accuracy against a "gold standard" (culture for clinical samples, known strains/organisms for analytical studies).

6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) was done

Yes, a standalone performance study was conducted. The JBAIDS Tularemia Detection Kit is an automated PCR system. The JBAIDS Software "analyzes the fluorescence amplification curves and reports results as positive, negative, inhibited, or uncertain." This indicates that the algorithm itself (the software within the JBAIDS instrument) provides the primary result interpretation, making it a standalone system. The stated advantage of the JBAIDS is that "the JBAIDS software automatically interprets the assay results, reducing the opportunity for user error."

7. The Type of Ground Truth Used

• Analytical Studies (Inclusivity & Exclusivity): Known bacterial isolates and purified DNA from various F. tularensis strains and other organisms. The ground truth was established by the identity of these reference strains/samples.
• Clinical Studies (Specificity): Laboratory culture was used as the "gold standard technique" for the clinical specificity assessment.

8. The Sample Size for the Training Set

The document does not explicitly mention a "training set" in the context of machine learning. As a PCR diagnostic kit, performance characteristics are typically established through analytical validation (inclusivity, exclusivity, precision, etc.) and clinical validation studies, rather than a machine learning training/test set paradigm. Therefore, there's no specified training set for an AI/ML algorithm.

9. How the Ground Truth for the Training Set Was Established

Since no specific training set for an AI/ML algorithm is described, this question is not applicable in the context of this 510(k) submission for a PCR kit. Ground truth for the analytical and clinical validation was established as described in section 7.

Ask a specific question about this device