Search Results
Found 22 results
510(k) Data Aggregation
(50 days)
Healgen Scientific LLC
The Healgen® AccuFluor Fentanyl Fluorescence Immunoassay (FIA) Test Kit-Qualitative is a fluorescence immunoassay intended for the qualitative detection of fentanyl in human urine at a cutoff concentration of 1.0 ng/mL. The assay is intended for use with Healgen® Immunofluorescence analyzer OG-H180. This in vitro diagnostic device is for prescription use only.
This assay provides only a preliminary analytical test result. A more specific alternate chemical method must be used to obtain a confirmed analytical result. Gas Chromatography-Mass Spectrometry (GC-MS) and Liquid Chromatography-Mass Spectrometry (LC-MS) are the preferred confirmatory methods. Clinical consideration and professional judgment should be applied to a Fentanyl test result, particularly when a preliminary positive result is obtained.
The Healgen® Immunofluorescence analyzer OG-H180 is a portable fluorescence instrument for in vitro diagnostic use only. The analyzer is designed to detect test results from in vitro diagnostic tests on clinical specimens. This analyzer can be used in a laboratory or point-of-care setting.
The AccuFluor Fentanyl FIA Test Kit-Qualitative is a rapid fluorescence immunoassay based on the principle of competitive binding, which uses fluorescent microspheres-labeled antibody as the indicator marker to qualitatively detect fentanyl in human urine. Drugs which may be present in the urine specimen compete against the drug conjugate for binding sites on the antibody.
During testing, a urine specimen migrates upward by capillary action. Fentanyl, if present in the urine specimen below 1.0 ng/mL, will not saturate the binding sites of antibody-coated particles in the test device. The antibody coated fluorescence particles will then be captured by immobilized Fentanyl conjugate, and the signal will be detected in the test line (T) region to show a negative result. The signal will not be detected in the test line (T) region if the Fentanyl level exceeds 1.0 ng/mL because all the binding sites for the anti-Fentanyl antibodies will be saturated and the result will show as positive. To serve as a procedural control, a signal will be detected at the control line (C) region indicating the proper volume of specimen has been added and membrane wicking has occurred. The test is interpreted by the Healgen® Immunofluorescence analyzer OG-H180 and the result will be interpreted by the analyzer.
The provided FDA 510(k) clearance letter pertains to the Healgen® AccuFluor Fentanyl Fluorescence Immunoassay (FIA) Test Kit - Qualitative and the Healgen® Immunofluorescence Analyzer (OG-H180). This document outlines the general regulatory approval and provides some performance characteristics, but it is not a comprehensive study report detailing all aspects of the acceptance criteria and the full study that proves the device meets those criteria.
Specifically, the document does not explicitly state "acceptance criteria" as a defined set of metrics and thresholds prior to presenting performance data. Instead, it presents results from various analytical performance studies which are implicitly used to demonstrate equivalence to a predicate device. Similarly, it does not describe "human expert ground truth establishment," "adjudication methods," or "MRMC comparative effectiveness studies" because these are typically relevant for AI/ML-based diagnostic devices utilizing image interpretation or complex decision support, which is not the primary function described for this immunoassay and analyzer.
This device is an in vitro diagnostic (IVD) test for qualitative detection of fentanyl in urine, which relies on a chemical reaction read by an analyzer. Therefore, the "study" described is a series of analytical performance tests, rather than a clinical study with human readers and ground truth established by medical experts in the way that would be done for an AI radiology device, for example.
Despite these limitations in the provided text for certain categories, I will extract and infer information where possible based on the provided document and common IVD device clearance practices.
Acceptance Criteria and Device Performance for Healgen® AccuFluor Fentanyl FIA Test Kit
1. Table of Acceptance Criteria and Reported Device Performance
As noted, the document does not explicitly list pre-defined "acceptance criteria" with specific numerical thresholds for all metrics. However, based on the provided performance data, here's an interpretation of the implied criteria and the reported performance. The "acceptance criteria" inferred here are based on what constitutes successful demonstration of performance for an IVD device of this type, often aiming for high accuracy, precision, and lack of interference, especially around the cutoff concentration.
Performance Characteristic | Implied Acceptance Criteria (Inferred) | Reported Device Performance |
---|---|---|
Analytical Precision (Around Cutoff) | High agreement (low false negatives/positives) at concentrations near the 1.0 ng/mL cutoff. | At -25% Cutoff (0.75 ng/mL): 56-57 out of 60 negative readings across 3 lots (93.3% - 95% negative agreement). |
At Cutoff (1.0 ng/mL): 22-24 negative and 36-38 positive readings out of 60 total across 3 lots. This demonstrates the expected transition around the cutoff. | ||
At +25% Cutoff (1.25 ng/mL): 60 out of 60 positive readings across 3 lots (100% positive agreement). | ||
At -100%, -75%, -50% Cutoff: 60 out of 60 negative readings. | ||
At +50%, +75%, +100% Cutoff: 60 out of 60 positive readings. | ||
Analytical Specificity (Cross-Reactivity) | Minimal to no cross-reactivity with common related compounds (e.g., other opioids, metabolites) or other substances found in urine, beyond expected concentrations where some cross-reactivity is acceptable and quantified. | Norfentanyl: 0.003% cross-reactivity at 30,000 ng/mL. |
Carfentanil: 0.013% cross-reactivity at 8,000 ng/mL. | ||
Cyclopropyl fentanyl, Para-fluoro fentanyl, Acetyl fentanyl: 100% cross-reactivity at 1 ng/mL. | ||
Many other fentanyl analogs: various quantified cross-reactivity percentages. | ||
Numerous unrelated compounds/metabolites: No cross-reactivity at 100 µg/mL. | ||
Interference | No significant interference from common substances (physiological or exogenous) found in human urine at specified concentrations. | Many compounds (e.g., Acetaminophen, Ethanol, Glucose, Albumin, Hemoglobin) showed no interference at high concentrations (e.g., 100µg/mL or 1% for Ethanol) for both negative and ±50% Cut-Off fentanyl spiked samples. |
Stability | Device maintains stated performance characteristics over its shelf life under specified storage conditions. | Stable at 2-30°C for 27 months based on real-time stability study. |
Effect of Urine Specific Gravity & pH | Test performance (positive/negative call) should remain consistent across a physiological range of urine specific gravity and pH. | Results were consistent (all positive for samples at/above +50% Cut-Off, all negative for samples at/below -50% Cut-Off) across specific gravity 1.000-1.035 and pH 4-9. |
Method Comparison (Clinical Samples) | High agreement (concordance) with a confirmed analytical method (LC-MS/MS) for clinical samples, especially for samples near the cutoff. | Overall Concordance: Across 3 sites, for 80 clinical samples (40 negative, 40 positive) compared to LC-MS/MS: |
True Negative Ranges: 7 negative, 19 low negative (less than -50% cutoff) correctly identified as negative. | ||
Near Cutoff Negative: 11-12 samples (between -50% and cutoff) correctly identified as negative, with 2-3 false positives. | ||
Discordant Results: 6 samples near cutoff showed discordance. For example, 3 samples (0.802, 0.841, 0.916 ng/mL) were LC-MS/MS negative but device positive. 3 samples (1.013, 1.092, 1.113 ng/mL) were LC-MS/MS positive but device negative. This indicates typical variability at the qualitative cutoff. | ||
Near Cutoff Positive: 20-21 samples (between cutoff and +50%) correctly identified as positive, with 2-3 false negatives. | ||
High Positive: 17 samples (greater than +50%) correctly identified as positive. |
2. Sample Size Used for the Test Set and Data Provenance
- Analytical Precision: 60 replicates per concentration (6 replicates/day for 10 days) per lot, across 9 concentration levels, for 3 device lots. Total: 60 * 9 * 3 = 1620 individual tests.
- Interference: Samples with various interfering substances were tested, each at both drug-free and ±50% Cut-Off spiked fentanyl concentrations, using three batches of device. (Exact number of tests not specified, but implies a comprehensive set).
- Specificity: Various drug metabolites and other compounds tested, each using three batches of device. (Exact number of tests not specified).
- Effect of Urine Specific Gravity and pH: Samples across the specified ranges were tested at -50% and +50% Cut-Off levels by three different operators using three device lots. (Exact number of tests not specified).
- Method Comparison (Clinical Samples): 80 unaltered clinical samples (40 negative, 40 positive). These samples were run at three different testing sites.
- Data Provenance: The document does not explicitly state the country of origin for the clinical samples. It does state they were "unaltered clinical samples," implying they were retrospective real-world samples collected from patients. It does not indicate if they were prospective.
3. Number of Experts Used to Establish Ground Truth for the Test Set and Qualifications
- Not Applicable in the traditional sense for this device. For this IVD device, the primary ground truth for its performance studies (precision, specificity, method comparison) is established by analytical gold standards, specifically:
- LC/MS-MS (Liquid Chromatography-Mass Spectrometry/Mass Spectrometry) for confirming fentanyl concentrations in precision studies and as the comparator method in the method comparison study.
- This is a highly accurate and precise laboratory method for quantifying drug concentrations, and its results are considered the "ground truth" for chemical concentration data.
- There were "three different operators" for the specific gravity/pH study, but these are not "experts" in the sense of medical professionals establishing a clinical diagnosis ground truth. They are laboratory personnel performing the test.
4. Adjudication Method for the Test Set
- Not Applicable in the traditional sense. Given that the ground truth is established by LC-MS/MS, there is no human "adjudication" process like consensus reading by multiple radiologists for image interpretation. The LC-MS/MS results serve as the definitive reference.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done, and Effect Size
- No, an MRMC comparative effectiveness study was not done. This type of study (comparing human readers with and without AI assistance on complex interpretation tasks) is not applicable to a qualitative immunoassay and analyzer like the Healgen AccuFluor Fentanyl FIA Test Kit, which determines the presence or absence of a substance based on a fluorescent signal. The device performance is assessed on its analytical accuracy against a gold standard method.
6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) Was Done
- Yes, the performance presented is primarily standalone. The Healgen® Immunofluorescence Analyzer (OG-H180) automatically interprets the fluorescent signal from the test kit. The performance data (precision, specificity, interference, method comparison) directly reflects the analytical capability of the device and test kit combination, without any human interpretation or intervention in the final "positive" or "negative" determination. A human loads the sample and the device performs the analysis and provides the result.
7. The Type of Ground Truth Used
- Analytical Gold Standard (LC-MS/MS): This is the primary method used to establish the true concentration of fentanyl in samples for precision studies and as the comparative reference for clinical samples.
- Spiked Samples: For analytical performance studies (precision, interference, specificity), known concentrations of fentanyl or interfering substances were added to negative urine samples, establishing a controlled ground truth.
8. The Sample Size for the Training Set
- Not explicitly stated in the document, and likely not applicable in the typical AI/ML sense. This device is an immunoassay, not an AI/ML diagnostic algorithm that undergoes a "training" phase with a large dataset. Immunoassays are based on biochemical principles and do not "learn" from data in the same way. Performance is optimized during development and validated analytically.
9. How the Ground Truth for the Training Set Was Established
- Not Applicable / Not Described. As it's not an AI/ML device relying on a training set, the concept of establishing ground truth for training does not apply here. The analytical performance is characterized through rigorous testing under controlled conditions and comparison to established reference methods (like LC-MS/MS).
Ask a specific question about this device
(49 days)
Healgen Scientific LLC
The Healgen® Accurate Multi-Drug Urine Screening Cup is a rapid lateral flow immunoassay for the qualitative detection of 6-Monoacetylmorphine, d-Amphetamine, Benzoylecgonine, Buprenorphine, EDDP, Norfentanyl, Methadone, d-Methamphetamine, d/l-Methylenedioxymethamine, Mortriptyline, Oxazepam, Oxycodone, Phencyclidine, d-Propoxyphene, Secobarbital, THC-COOH and Tramadol in human urine. The test cut-off concentrations and the compounds the tests are calibrated to are as follows:
Test | Calibrator | Cut-off (ng/mL) |
---|---|---|
6-MAM | 6-Monoacetylmorphine | 10 |
AMP | d-Amphetamine | 500 or 1000 |
BAR | Secobarbital | 300 |
BUP | Buprenorphine | 10 |
BZO | Oxazepam | 300 |
COC | Benzoylecgonine | 150 or 300 |
EDDP | 2-ethylidene-1,5-dimethyl-3,3- | |
diphenylpyrolidine | 300 | |
FEN or FYL | Norfentanyl | 5 |
MDMA | Methylenedioxymethamphetamine | 500 |
MET | d-Methamphetamine | 500 or 1000 |
MTD | Methadone | 300 |
OPI | Morphine | 300 or 2000 |
OXY | Oxycodone | 100 |
PCP | Phencyclidine | 25 |
PPX | d-Propoxyphene | 300 |
TCA | Nortriptyline | 1000 |
THC | 11-nor-Δ9-THC-COOH | 50 |
TRA or TMI | Tramadol | 100 |
The single or multi-test cups can consist of up to eighteen (18) of the above listed analytes in any combination with or without on-board adulteration/specimen validity tests (SVT).
The tests provide only a preliminary result. A more specific alternative chemical must be used to obtain a confirmed positive result. Gas Chromatography-Mass Spectrometry (GC-MS), Liquid Chromatography-Mass Spectrometry (LC-MS), and their tandem mass-spectrometer versions are the preferred confirmatory methods. Careful consideration and judgment should be applied to any drugs of abuse screen test result, particularly when evaluating preliminary positive results.
The Healgen® Accurate Home Multi-Drug Urine Test Cup is a rapid qualitative immunoassay. The device provides preliminary results for the detection of one or more of the following drugs
CODE | SUBSTANCE | Cut-off (ng/mL) |
---|---|---|
AMP | Amphetamine | 1000 or 500 |
BUP | Buprenorphine | 10 |
BAR | Secobarbital | 300 |
BZO | Oxazepam | 300 |
COC | Cocaine | 300 or 150 |
EDDP | EDDP | 300 |
FYL | Norfentanyl | 5 |
MET/mAMP | Methamphetamine | 1000 or 500 |
MDMA | Ecstasy | 500 |
OPI | Morphine | 2000 or 300 |
MTD | Methadone | 300 |
OXY | Oxycodone | 100 |
PCP | Phencyclidine | 25 |
PPX | Propoxyphene | 300 |
TCA | Nortriptyline | 1000 |
THC | Marijuana | 50 |
TML | Tramadol | 100 |
6-MAM | 6-Monoacetylmorphine | 10 |
This drug test cup may contain any combination of the drug tests listed in the table above.
This test provides only preliminary result. A more specific alternative chemical must be used to obtain a confirmed positive result. Gas Chromatography-Mass Spectrometry (GC-MS), Liquid Chromatography-Mass Spectrometry (LC-MS), and their tandem mass-spectrometer versions are the preferred confirmatory methods. Careful consideration and judgment should be applied to any drugs of abuse screen test result, particularly when evaluating preliminary positive results.
Healgen® Accurate Home Muti-Drug Urine Test Cup and Healgen® Accurate Muti-Drug Urine Drug Screen Cup are immunochromatographic assays that use a lateral flow system for the qualitative detection of single or multiple drugs in human urine.
The device is a cup format. Each test device is sealed with two sachets of desiccant in an aluminum pouch. The device is in a ready-to-use format and no longer requires assembly before use.
Acceptance Criteria and Device Performance for Healgen® Accurate Multi-Drug Urine Drug Screen Cup
This document outlines the acceptance criteria and the evidence provided to demonstrate that the Healgen® Accurate Multi-Drug Urine Drug Screen Cup meets these criteria. The information is extracted from the provided FDA 510(k) summary (K243365).
1. Table of Acceptance Criteria and Reported Device Performance
The acceptance criteria for this qualitative immunoassay are primarily demonstrated through a combination of precision, analytical specificity/interference, and a method comparison study. The precision study evaluates the device's ability to consistently produce correct results at various concentrations relative to the cutoff, while the method comparison study assesses its agreement with a gold standard (LC-MS/MS) using clinical samples. The layperson study demonstrates ease of use for over-the-counter applications.
Acceptance Criteria Category | Specific Criteria | Reported Device Performance |
---|---|---|
Precision/Reproducibility | Consistent qualitative results (+/- at least 95% at the cutoff concentration, and 100% agreement for samples far from the cutoff) across multiple lots and runs, at various concentrations relative to the cutoff (-100%, -75%, -50%, -25%, Cutoff, +25%, +50%, +75%, +100%). | For all drugs tested (AMP 1000/500, BAR 300, BUP 10, BZO 300, COC 300/150, EDDP 300, FYL 5, MDMA 500, MET 1000/500, OPI 300/2000, OXY 100, PCP 25, PPX 300, TCA 1000, THC 50, TML 100, 6-MAM 10) across 3 lots: |
- 100% agreement for concentrations at -100%, -75%, -50%, +50%, +75%, and +100% cutoff (e.g., 0-/50+ or 50-/0+ indicating no false negatives or positives respectively).
- At the cutoff concentration, results consistently showed a mix of positive and negative interpretations, typically near 50%/50% (e.g., 23-/27+, 25-/25+), demonstrating sensitivity around the cutoff.
- At -25% cutoff, the device predominantly showed negative results (e.g., 50-/0+ or 49-/1+), with a few false positives.
- At +25% cutoff, the device predominantly showed positive results (e.g., 0-/50+ or 1-/49+), with a few false negatives.
This indicates good precision around the cutoff with expected variability at borderline concentrations. |
| Analytical Specificity | No significant cross-reactivity with commonly encountered substances (drug metabolites, prescription medications, endogenous compounds) at relevant concentrations, and no interference from variations in pH and specific gravity within a physiological range. | Diverse lists of compounds were tested for cross-reactivity. The results show that: - Many cross-reacting compounds are identified with their minimum concentration needed to yield a positive result and their % Cross-Reactivity (e.g., Hydroxyamphetamine with AMP 1000, Amobarbital with BAR 300).
- Numerous compounds showed no detection or were negative at high concentrations (e.g., 100,000 ng/mL), indicating no cross-reactivity for those substances.
- pH levels of 4 to 9 and specific gravity levels of 1.000 to 1.035 were demonstrated to not affect the assay results. |
| Method Comparison | A high degree of concordance between the device's qualitative results and confirmatory Gas Chromatography-Mass Spectrometry (GC-MS) or Liquid Chromatography-Mass Spectrometry (LC-MS) results for clinical urine samples, particularly for samples at or near the cutoff. | For each drug and its cutoff configuration: - 100% agreement was observed for drug-free samples and "low negative" samples (less than -50% of cutoff).
- 100% agreement was primarily observed for "high positive" samples (greater than +50% of cutoff).
- For "near cutoff negative" (between -50% and cutoff) and "near cutoff positive" (between cutoff and +50%) samples, there was expected variability, with some samples interpreted differently by the device compared to LC-MS/MS, indicating the device's expected performance around the cutoff. The discordant results table details specific samples where device results differed from LC-MS/MS. Generally, the majority of samples near the cutoff were correctly identified by the device. |
| Lay Person Usability | The device should be easy to understand and use by laypersons as indicated by a high agreement in understanding instructions and performance on prepared samples across different drug classes. A reading level analysis of the instructions for use should demonstrate appropriate readability. | A study with 280 laypersons (mix of male/female, aged 21 to >50, diverse backgrounds) demonstrated: - 100% agreement for interpretation of drug-free and concentrations at -100%, -75%, -50% below cutoff.
- 100% agreement for concentrations at +50% and +75% above cutoff.
- Expected variability in agreement around the -25% and +25% cutoff concentrations, mostly ranging from 90% to 95%.
- All participants indicated the instructions were easy to understand and follow.
- The Flesch-Kincaid reading analysis yielded a Grade Level of 7, which is suitable for layperson use. |
| Stability | The device should maintain its performance characteristics for a specified duration under defined storage conditions. | The device is stable at 2-30°C for 36 months based on real-time stability studies. |
2. Sample Size Used for the Test Set and Data Provenance
Precision/Reproducibility Study:
- Sample Size: For each drug and each concentration level (total 9 levels per drug), 50 individual tests were performed (2 runs per day for 25 days). Given there are 17 distinct drug analytes, this amounts to 17 drugs * 9 concentrations * 50 tests/concentration = 7,650 tests.
- Data Provenance: Samples were prepared by spiking target drugs into drug-free urine samples. These samples were likely prepared in a laboratory setting, making the data provenance prospective and controlled. The document does not specify a country of origin for the samples/urine, but the study was conducted "in-house."
Method Comparison Study:
- Sample Size: For each drug, 80 unaltered urine clinical samples were used. This consisted of 40 negative and 40 positive samples. Given there are 17 distinct drug analytes, this amounts to 17 drugs * 80 samples/drug = 1,360 total clinical samples tested across all drugs.
- Data Provenance: Unaltered clinical urine samples. The document does not specify the country of origin but states the study was performed "in-house." The nature of "unaltered clinical samples" suggests these were collected from patients, and their retrospective or prospective nature is not explicitly stated, though being "blind labeled" for comparison suggests they were handled specifically for this study.
Lay Person Study:
- Sample Size: 280 lay persons participated. Urine samples were prepared at 7 concentration levels (-100%, +/-75%, +/-50%, +/-25% of cutoff). Each participant tested 1 blind labeled sample. While it states urine samples were prepared at 7 concentrations, the agreement tables show 7 concentrations and 20 tests per concentration for each drug. If we assume each of the 280 participants tested one sample for one drug, the sample sizes per concentration shown in the table (20 tests) indicate that multiple participants contributed data for each drug and concentration, but a single participant did not test all drugs or all concentrations.
- Data Provenance: Urine samples were prepared by spiking drugs into drug-free pooled urine specimens. The samples were prepared in a laboratory environment, indicating prospective and controlled data provenance. The lay persons were recruited for the study, making their involvement prospective. The study was performed at "three intended user sites."
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications
Precision/Reproducibility Study:
- Ground Truth: The "ground truth" for the spiked samples was established by the known concentrations of the spiked drugs, which were confirmed by LC-MS/MS.
- Experts/Qualifications: The document states that "Each drug concentration was confirmed by LC-MS/MS." LC-MS/MS is a highly accurate analytical method, and while specific "experts" (e.g., toxicologists, clinical chemists) are not explicitly named, the use of this method implies analysis by qualified laboratory personnel specializing in mass spectrometry. "One operator per lot" (for 3 lots) performed the tests on the device, but the LC-MS/MS confirmation implies separate, expert analysis for ground truth.
Method Comparison Study:
- Ground Truth: The ground truth for the clinical samples was established by LC-MS/MS results.
- Experts/Qualifications: Similar to the precision study, the ground truth was determined by LC-MS/MS, implying analysis by qualified laboratory professionals. No specific number or qualification of "experts" is provided beyond the analytical method itself.
Lay Person Study:
- Ground Truth: The "ground truth" for the spiked samples was established by the known concentrations of the spiked drugs, which were confirmed by LC-MS/MS.
- Experts/Qualifications: As above, the ground truth was derived from LC-MS/MS analysis, indicating reliance on qualified laboratory personnel.
4. Adjudication Method for the Test Set
Precision/Reproducibility Study:
- Adjudication Method: Not applicable in the traditional sense of expert consensus. The ground truth was based on the quantitative LC-MS/MS results of the spiked samples. The device's qualitative results were compared against these known concentrations relative to the cutoff. Discrepancies were noted directly.
Method Comparison Study:
- Adjudication Method: Not applicable/Not explicitly stated as an adjudication process involving multiple human readers. The device results were directly compared to the LC-MS/MS results, which served as the definitive "true" value (ground truth). The tables show the device's output (positive/negative) versus the LC-MS/MS quantification, which then categorizes samples as drug-free, low negative, near cutoff negative, near cutoff positive, or high positive.
Lay Person Study:
- Adjudication Method: Not applicable in terms of expert consensus. The laypersons' interpretations of the device results were compared against the established ground truth (LC-MS/MS confirmed concentrations of spiked samples). The study aimed to assess if laypersons could correctly interpret the device results against this objective ground truth, not for them to establish ground truth or adjudicate.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done
No, an MRMC comparative effectiveness study was not done in the context of comparing human readers with and without AI assistance, as this is an in-vitro diagnostic device (a test cup) not an AI-powered diagnostic imaging or interpretation system. The "operators" in the method comparison study refer to individuals performing the device test, not medical interpreters assisted by AI. The "lay person study" evaluated if laypersons could correctly interpret the device, not an AI system.
6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) Was Done
This is an in-vitro diagnostic (IVD) device (a lateral flow immunoassay test cup), not an AI algorithm. Therefore, the concept of "standalone (algorithm only)" performance without human-in-the-loop is not applicable in the typical sense for AI/software devices. The device itself provides a visual qualitative result (lines appearing or not appearing), which then requires human observation and interpretation. The performance characteristics described (precision, specificity, method comparison) are inherently "standalone" in that they assess the device's ability to produce correct visual results, which a human then reads. The layperson study specifically evaluates the human (layperson) interpretation of these visual results.
7. The Type of Ground Truth Used
The primary type of ground truth used across all analytical performance studies (Precision, Analytical Specificity, Method Comparison, and Lay Person Study where applicable) is Gas Chromatography-Mass Spectrometry (GC-MS) and Liquid Chromatography-Mass Spectrometry (LC-MS) and their tandem mass-spectrometer versions (LC-MS/MS).
These are considered confirmatory analytical methods and are widely accepted as the gold standard for quantitative drug detection in toxicology.
8. The Sample Size for the Training Set
As this device is a lateral flow immunoassay (a chemical/biological test kit), and not an AI/machine learning algorithm, there is no "training set" in the computational sense. The device's performance is based on its inherent physical and chemical properties and reagent formulations, which are developed and optimized through laboratory research and development, rather than machine learning training.
9. How the Ground Truth for the Training Set Was Established
Since there is no "training set" for this type of device, this question is not applicable. The development and optimization of the immunoassay undoubtedly involved extensive experimentation with samples of known drug concentrations, likely confirmed by GC-MS/LC-MS, but this is part of the product's research and development process, not analogous to establishing ground truth for a machine learning training set.
Ask a specific question about this device
(35 days)
Healgen Scientific LLC
The Healgen® Accurate Fentanyl Rapid Test Dip Card (Urine) is an immunoassay intended for the qualitative detection of fentanyl in human urine at a cutoff concentration of 1.0 ng/mL. This test provides only preliminary result. A more specific alternative chemical must be used to obtain a confirmed analytical result. GC/MS is the preferred confirmatory method. Evaluate preliminary positive results carefully.
The Healgen® Accurate Fentanyl Rapid Test Strip (Urine) is an immunoassay intended for the qualitative detection of fentanyl in human urine at a cutoff concentration of 1.0 ng/mL. This test provides only preliminary result. A more specific alternative chemical must be used to obtain a confirmed analytical result. GC/MS is the preferred confirmatory method. Evaluate preliminary positive results carefully.
The Healgen® Accurate Fentanyl Rapid Test Cassette (Urine) is an immunoassay intended for the qualitative detection of fentanyl in human urine at a cutoff concentration of 1.0 ng/mL. This test provides only preliminary result. A more specific alternative chemical must be used to obtain a confirmed analytical result. GC/MS is the preferred confirmatory method. Evaluate preliminary positive results carefully.
The Healgen® Accurate Rapid Fentanyl Test Dip Card (Urine) is an immunoassay intended for the qualitative detection of fentanyl in human urine at a cutoff concentration of 1.0 ng/mL. This test provides only a preliminary result. A more specific alternative chemical method must be used to obtain a confirmed presumptive positive result. Gas Chromatography-Mass Spectrometry (CC-MS), Liquid Chromatography-Mass Spectrometry (LC-MS), and their tandem mass-spectrometer versions are the preferred confirmatory methods. Careful consideration and judgment should be applied to any drugs of abuse screen test result, particularly when evaluating preliminary positive results.
The Healgen® Accurate Rapid Fentanyl Test Strip (Urine) is an immunoassay intended for the qualitative detection of fentanyl in human urine at a cutoff concentration of 1.0 ng/mL. This test provides only a preliminary result. A more specific alternative chemical method must be used to obtain a confirmed presumptive positive result. Gas Chromatography-Mass Spectrometry (C-MS), Liquid Chromatography-Mass Spectrometry (LC-MS), and their tandem mass-spectrometer versions are the preferred confirmatory methods. Careful consideration and judgment should be applied to any drugs of abuse screen test result, particularly when evaluating preliminary positive results.
The Healge® Accurate Rapid Fentanyl Test Cassette (Urine) is an immunoassay intended for the qualitative detection of fentanyl in human urine at a cutoff concentration of 1.0 ng/mL. This test provides only a preliminary result. A more specific alternative chemical method must be used to obtain a confirmed presumptive positive result. Gas Chromatography-Mass Spectrometry (C-MS), Liquid Chromatography-Mass Spectrometry (LC-MS), and their tandem mass-spectrometer versions are the preferred confirmatory methods. Careful consideration and judgment should be applied to any drugs of abuse screen test result, particularly when evaluating preliminary positive results.
The Healgen® Accurate Fentanyl Tests are immunoassays intended for the qualitative detection of fentanyl in human urine. Each Healgen® Accurate Fentanyl Test consists of a Test Device in format of Cassette or Dip Card or Strip, and a package insert. Each Test Device is sealed with sachets of desiccant in an aluminum pouch.
The document provided is a 510(k) summary for a urine drug test for Fentanyl. It details the performance characteristics of the Healgen® Accurate Fentanyl Rapid Test Dip Card (Urine), Healgen® Accurate Rapid Fentanyl Test Dip Card (Urine), Healgen® Accurate Fentanyl Rapid Test Strip (Urine), Healgen® Accurate Rapid Fentanyl Test Strip (Urine), Healgen® Accurate Fentanyl Rapid Test Cassette (Urine), and Healgen® Accurate Rapid Fentanyl Test Cassette (Urine).
This document describes a medical device (rapid diagnostic test), not an AI/ML algorithm or software as a medical device (SaMD). Therefore, many of the requested criteria related to AI/ML studies (e.g., number of experts for ground truth, adjudication methods, MRMC studies, training set size/ground truth) are not applicable.
However, I can extract the relevant information regarding the device's acceptance criteria and the studies performed to demonstrate its performance.
Here's the breakdown of the acceptance criteria and the study that proves the device meets them, focusing on what is applicable to a rapid diagnostic test:
Acceptance Criteria and Reported Device Performance
For rapid diagnostic tests like this one, "acceptance criteria" are typically defined by performance characteristics such as precision (accuracy at and around the cutoff), specificity (no cross-reactivity with other substances), interference (no effect from common urine constituents), and comparison to a gold standard method. The document presents data to demonstrate these characteristics.
1. Table of Acceptance Criteria and the Reported Device Performance (Summary of Key Performance)
The document implicitly defines the acceptance criteria through the presented performance data, showing that the device consistently performs as expected at and around the cutoff concentration of 1.0 ng/mL for fentanyl.
Performance Metric | Acceptance Criteria (Implicit from Study Design) | Reported Device Performance (Cassette, Dip Card, Strip formats show similar performance, so a representative example or combined summary is given) |
---|---|---|
Precision (Accuracy at specific concentrations vs. cutoff) | - Expected 100% negative results for concentrations below cutoff. |
- Expected 100% positive results for concentrations above cutoff.
- Mixed results expected at cutoff (ideally close to 50% positive/negative). | Cassette Example (across 3 lots, 60 tests per lot/concentration)
-100% to -50% cut off: 60-/0+ (100% Negative)
-25% cut off: 58-60 negative, 0-3 positive (e.g., Lot 2: 58-/2+, Lot 1: 60-/0+, Lot 3: 59-/1+)
Cut off: Mixed results (e.g., Lot 1: 38+/22-, Lot 2: 33+/27-, Lot 3: 36+/24-)
+25% to +100% cut off: 60+/0- (100% Positive)
(Similar performance observed for Dip Card and Strip formats) |
| Specificity (Cross-reactivity) | No significant interference from tested compounds at physiological/therapeutic concentrations, or clear cross-reactivity profiles are established. | Fentanyl Analogs/Metabolites: - Cyclopropyl fentanyl: 1 ng/mL (100% cross-reactivity)
- Acetyl fentanyl: 1 ng/mL (100% cross-reactivity)
- Acrylfentanyl: 0.9 ng/mL (111.11% cross-reactivity)
- 4-Fluoro-isobutyrylfentanyl: 5 ng/mL (20% cross-reactivity)
- Norfentanyl: 30,000 ng/mL (0.003% cross-reactivity)
(Extensive list of other non-fentanyl compounds tested at 100 µg/mL showed no cross-reactivity) |
| Interference | No interference from common endogenous or exogenous substances in urine. | No interference from a wide range of compounds (e.g., acetaminophen, albumin, glucose, ethanol, common drugs) at specified concentrations. |
| Effect of Urine Specific Gravity & pH | Performance should remain accurate across a range of urine specific gravity and pH values. | All samples (spiked at -50% and +50% Cut-Off levels) tested correctly across ranges of specific gravity (1.000 to 1.035) and pH (4 to 9). |
| Method Comparison with LC/MS | High concordance with GC/MS (or LC/MS) as the gold standard, especially for samples far from the cutoff. Discrepancies primarily expected around the cutoff. | Cassette Example (80 clinical samples per operator): - 7 Negative, 19 Low Negative: 100% agreement with LC/MS.
- Near Cutoff Negative (LC/MS 1.0 ng/mL): 20-21 Positive, 2-3 Negative (total 23 samples)
- 17 High Positive: 100% agreement with LC/MS.
(Similar results for Dip Card and Strip formats. Discordant results are concentrated around the cutoff as expected for qualitative tests.) |
| Lay-User Performance (For OTC products) | High percentage of correct results by untrained lay users following package insert instructions. | Cassette Example (140 lay persons total, 20 samples each concentration):
-100% to -25% Cutoff: 100% correct negative results.
+25% to +75% Cutoff: 100% correct positive results.
(Exceptions for Dip Card: 1 false positive at -25% Cutoff, resulting in 95% correct. Strip: 100% correct across all positive/negative points.) |
2. Sample Size Used for the Test Set and the Data Provenance
-
Precision Studies:
- Sample Size: For each of the nine concentrations tested (from -100% cutoff to +100% cutoff), 60 tests were performed per device lot. With 3 lots, this implies 180 tests per concentration per device format (Cassette, Dip Card, Strip). Total for precision studies: 9 concentrations * 60 tests/concentration/lot * 3 lots * 3 formats = 4860 tests.
- Data Provenance: Samples were prepared by spiking fentanyl into negative samples. Concentrations were confirmed by LC/MS. The document does not specify the country of origin, but generally, these types of lab studies for FDA submissions are conducted domestically or in compliant international facilities. These are prospective experiments.
-
Specificity and Interference Studies:
- Sample Size: "Three batches of each device" were used. The number of individual tests on each substance is not explicitly stated, but it's implied that sufficient replicates were performed to draw conclusions.
- Data Provenance: Substances were added to drug-free urine and target drug fentanyl urine. These are prospective lab experiments.
-
Method Comparison Studies (Clinical Samples):
- Sample Size: 80 "unaltered clinical samples" were used for each device format (Cassette, Dip Card, Strip). These 80 samples were split into categories: 7 Negative (0 ng/mL), 19 Low Negative, 14 Near Cutoff Negative, 23 Near Cutoff Positive, and 17 High Positive. Each sample was tested by three different operators.
- Data Provenance: "unaltered clinical samples." The document does not specify the country of origin, but implies real-world urine specimens. It is a retrospective evaluation against a confirmed gold standard.
-
Lay-User Study:
- Sample Size: 140 lay persons participated for each device format (Cassette, Dip Card, Strip). Each participant received 1 blind-labeled sample. The samples themselves were prepared with different fentanyl concentrations, with 20 samples per concentration level.
- Data Provenance: Urine samples were prepared by spiking fentanyl into drug-free pooled urine specimens. Concentrations were confirmed by LC/MS. This is a prospective study involving human subjects (lay users).
3. Number of Experts Used to Establish the Ground Truth for the Test Set and the Qualifications of Those Experts
- The ground truth for all performance studies (precision, specificity, interference, method comparison, lay-user study) was established by LC/MS (Liquid Chromatography-Mass Spectrometry) or GC/MS (Gas Chromatography-Mass Spectrometry). These are established analytical chemistry methods considered the "gold standard" for confirming drug concentrations in toxicology.
- While these methods require expert chemists/toxicologists to operate and interpret, the document does not specify the "number of experts" or their "qualifications" in the same way one would for clinical image interpretation, as the ground truth is obtained from an analytical instrument, not human interpretation of a test. The results from LC/MS are quantitative and definitive.
4. Adjudication Method for the Test Set
- For the analytical performance and method comparison studies, the rapid test results (positive/negative) were directly compared to the quantitative LC/MS results.
- For the method comparison study, the results of three different operators were compared for each sample. The tables present the agreement of each operator's reading with the LC/MS result. Discordant results are explicitly listed for each operator involved. There is no mention of an "adjudication method" in the sense of a committee for final decision-making, as the LC/MS result serves as the definitive reference.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done, If So, What Was the Effect Size of How Much Human Readers Improve with AI vs Without AI Assistance
- Not Applicable. This is a rapid diagnostic test (an in-vitro diagnostic device), not an AI/ML SaMD that assists human readers. No MRMC comparative effectiveness study was done for AI assistance, as AI is not part of this device.
6. If a Standalone (i.e. algorithm only without human-in-the loop performance) Was Done
- Not Applicable. This is a rapid diagnostic test, interpreted visually by a human (either a trained operator or a lay user). There is no "algorithm" component in this device in the sense of an AI/ML model for standalone performance evaluation. The "test" itself is the standalone device performance when read correctly by a human.
7. The Type of Ground Truth Used
- The primary ground truth used throughout the studies was LC/MS (Liquid Chromatography-Mass Spectrometry) or GC/MS (Gas Chromatography-Mass Spectrometry) results. These are highly accurate, quantitative analytical methods for identifying and quantifying substances in a sample.
8. The Sample Size for the Training Set
- Not Applicable. This device is a rapid immunoassay test, not an AI/ML algorithm that requires a "training set" in the computational learning sense. The device's "training" would be its manufacturing process and quality control.
9. How the Ground Truth for the Training Set Was Established
- Not Applicable. As there is no AI/ML training set, this question is not relevant to this device.
Ask a specific question about this device
(255 days)
Healgen Scientific LLC
The Healgen Rapid COVID-19 Antigen Test is a lateral flow immunochromatographic assay intended for the rapid, qualitative detection of SARS-CoV-2 nucleocapsid protein antigens directly in anterior nasal swabs specimens from individuals with signs and symptoms of upper respiratory infection within the first six (6) days of symptom onset. The test is intended for use as an aid in the diagnosis of SARS-CoV-2 infections (COVID-19) in symptomatic individuals when either: tested at least twice over three days with at least 48 hours between tests; or when tested once, and negative by the Healgen Rapid COVID-19 Antigen Test and followed with a molecular test
The test does not differentiate between SARS-CoV and SARS-CoV-2.
A negative test result is presumptive and it is recommended these results be confirmed by a molecular SARS-CoV-2 assay. Negative results do not preclude SARS-CoV-2 infections and should not be used as the sole basis for treatment or other patient management decisions.
Positive results do not rule out co-infection with other respiratory pathogens
Performance characteristics for SARS-CoV-2 were established from May 2022 to July 2022 when SARS-CoV-2 Omicron was the predominant SARS-CoV-2 variant in circulation. When other SARS-CoV-2 virus variants are emerging, performance characteristics may vary.
The Healgen Rapid COVID-19 Antigen Test is a lateral flow immunochromatographic assay that uses highly sensitive monoclonal antibodies to detect nucleocapsid protein from SARS-CoV-2 virus in nasal swab collected specimens. The test strip is composed of the following components: sample pad, reagent pad, reaction membrane, and absorbing pad housed within a test cassette. The reagent pad contains colloidal-gold-conjugated monoclonal antibody that recognizes and binds to the nucleocapsid protein of SARS-CoV-2; the reaction membrane in the test line (T) contains the second antibody that recognizes another epitope of the nucleocapsid protein of SARS-CoV-2.
External quality controls are required but not included with the test kit and are sold separately as the COVID-19 Antigen Control Kit. The control swabs should be processed according to the Instructions for Use (IFU) and are intended to be used as quality control samples to demonstrate that the test is performing and is being performed correctly.
The provided text describes the performance characteristics and acceptance criteria for the Healgen Rapid COVID-19 Antigen Test.
Here's an analysis to extract the requested information:
1. A table of acceptance criteria and the reported device performance:
The document doesn't explicitly state "acceptance criteria" as a separate, pre-defined set of thresholds in a table. Instead, it presents the results of various performance studies. The implicit acceptance criteria are that the device performance should be clinically acceptable and comparable to existing devices for substantial equivalence.
Here's a table summarizing the reported device performance, which can be interpreted as having met the (implicit) acceptance criteria:
Performance Metric | Acceptance Criteria (Implicit) | Reported Device Performance |
---|---|---|
Clinical Performance: | ||
Positive Percent Agreement (PPA) | Clinically acceptable | 85.4% (95% CI: 79.1% - 90%) |
Negative Percent Agreement (NPA) | Clinically acceptable | 99.7% (95% CI: 98.9% - 99.9%) |
Analytical Sensitivity (LoD): | ||
LoD (SARS-CoV-2 USA-WA1/2020) | Lowest detectable concentration | 5.75 x 10³ TCID₅₀/mL (2.875 x 10² TCID₅₀/swab) |
LoD (WHO International Standard) | Lowest detectable concentration | 250 IU/mL (12.5 IU/swab) |
Precision/Reproducibility: | ≥ 90% agreement | Overall: |
Negative (NPA) | 99.2% (134/135) | |
Weak Positive (PPA) | 100.0% (135/135) | |
Positive (PPA) | 100.0% (135/135) | |
Inclusivity (for tested variants): | Reliable detection | All tested variants (Alpha, Beta, Delta, Gamma, Kappa, Omicron) showed positive results at various concentrations. |
Cross-Reactivity/Interference: | No cross-reactivity/interference | No cross-reactivity or microbial interference observed with tested organisms. No interference from tested endogenous/exogenous substances. |
Hook Effect: | No detectable hook effect | No detectable hook effect up to 5.75 × 10⁶ TCID₅₀/mL |
External Controls Validation: | 100% agreement expected | 100% agreement for both positive and negative controls. |
Specimen Stability: | Stable for up to 4 hours | Samples stable for up to 4 hours at room temperature. |
2. Sample sized used for the test set and the data provenance (e.g. country of origin of the data, retrospective or prospective)
- Test Set Sample Size: 806 evaluable subjects.
- Data Provenance:
- Country of Origin: Not explicitly stated, but the submission is to the U.S. FDA, and the virus strain mentioned is "USA-WA1/2020," suggesting a U.S. context.
- Retrospective/Prospective: Prospective. The clinical performance study collected samples "prospectively collected from symptomatic subjects between May 2022 and July 2022 at six clinical point of care sites."
3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts (e.g. radiologist with 10 years of experience)
The ground truth for the clinical test set was established using RT-PCR assays, not expert human readers. Therefore, the concept of "number of experts" for ground truth establishment isn't applicable in the context of this device. The RT-PCR assays are considered the gold standard for detecting SARS-CoV-2 viral nucleic acid.
4. Adjudication method (e.g. 2+1, 3+1, none) for the test set
The adjudication method used for the comparator (RT-PCR) was a "2 out of 3 rules." This means that if the first two FDA-cleared, highly sensitive RT-PCR assays had discordant results, a third FDA-cleared RT-PCR comparator assay was performed, and the final result was determined by the majority.
5. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance
No MRMC comparative effectiveness study was done. This device is a rapid antigen test read visually by users (healthcare professionals), not an AI-powered diagnostic imaging device that assists human readers. The study focuses on the diagnostic accuracy of the device itself compared to a molecular gold standard.
6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done
The device itself is a standalone "algorithm" in the sense of a chemical diagnostic assay that produces a visual result. There is no separate "algorithm" being evaluated without human-in-the-loop, as the human (healthcare professional) is part of reading the visually interpreted result. The "performance characteristics" section details the standalone performance of the device (the antigen test) against the RT-PCR comparator, which represents the device's accuracy without a separate human interpretation study impacting the core result.
7. The type of ground truth used (expert consensus, pathology, outcomes data, etc)
The type of ground truth used for the clinical performance test set was molecular SARS-CoV-2 assay results, specifically composite RT-PCR assays. This is a highly sensitive and specific laboratory-based "diagnostic truth" for the presence of the virus.
8. The sample size for the training set
The document does not mention a training set in the context of machine learning or AI models. This device is a lateral flow immunochromatographic assay, a chemical test, not a software or AI-based diagnostic tool that would typically involve a separate training phase with a distinct dataset. The "development" of such a test involves chemical formulation and design, not typically data training in the AI sense.
9. How the ground truth for the training set was established
As there is no mention of a training set for an AI/ML model, this question is not applicable to the Healgen Rapid COVID-19 Antigen Test.
Ask a specific question about this device
(31 days)
Healgen Scientific LLC
The Healgen Accurate Urine Drug Screen Dip Card is a rapid lateral flow immunoassays for the qualitative detection of 6-Monoacetylmorphine, d-Amphetamine, Benzoylecgonine, EDDP, d/l-Methadone, d-Methamphetamine, d1-Methylenedioxymethamine, Mortriptyline, Oxazepam, Oxycodone, Phencyclidine, d-Propoxyphene, Secobarbital and THC-COOH in human urine. The test cut-off concentrations and the compounds the tests are calibrated to are as follows:
Test | Calibrator | Cut-off (ng/mL) |
---|---|---|
6-MAM | 6-Monoacetylmorphine | 10 |
AMP | d-Amphetamine | 500 / 1000 |
BAR | Secobarbital | 300 |
BUP | Buprenorphine | 10 |
BZO | Oxazepam | 300 |
COC | Benzoylecgonine | 150 / 300 |
EDDP | 2-ethylidene-1,5-dimethyl-3,3-diphenylpyrolidine | 300 |
MDMA | Methylenedioxymethamphetamine | 500 |
MET | d-Methamphetamine | 500 / 1000 |
MTD | d/l-Methadone | 300 |
OPI | Morphine | 300 / 2000 |
OXY | Oxycodone | 100 |
PCP | Phencyclidine | 25 |
PPX | d-Propoxyphene | 300 |
TCA | Nortriptyline | 1000 |
THC | 11-nor-Δ9-THC-COOH | 50 |
The single or multi-test panels can consist of up to sixteen (16) of the above listed analytes in any combination with or without on-board adulteration/specimen validity tests (SVT).
The tests provide only a preliminary result. A more specific alternative chemical method must be used to obtain a confirmed positive result. Gas Chromatography-Mass Spectrometry (GC-MS), Liquid Chromatography-Mass Spectrometry (LC-MS), and their tandem mass-spectrometer versions are the preferred confirmatory methods. Careful consideration and judgment should be applied to any drugs of abuse screen test result, particularly when evaluating preliminary positive results.
The Healgen Accurate Home Urine Drug Test Dip Card is a rapid qualitative immunoassay.
The device provides preliminary results for the detection of potential abuse of one or more at the cutoff concentrations of table below.
This is not a screening device to monitor prescription medication.
CODE | SUBSTANCE | CUT-OFF (ng/mL) |
---|---|---|
AMP | Amphetamine | 1000 or 500 |
BUP | Buprenorphine | 10 |
BAR | Secobarbital | 300 |
BZO | Oxazepam | 300 |
COC | Cocaine | 300 or 150 |
---------- | ---------------------- | ------------- |
EDDP | EDDP | 300 |
MET/mAMP | Methamphetamine | 1000 or 500 |
MDMA | Ecstasy | 500 |
OPI | Morphine | 2000 or 300 |
MTD | Methadone | 300 |
OXY | Oxycodone | 100 |
PCP | Phencyclidine | 25 |
PPX | Propoxyphene | 300 |
TCA | Nortriptyline | 1000 |
THC | Marijuana | 50 |
6-MAM | 6-Monoacetylmorphine | 10 |
This drug test dip card may contain any combination of the drug tests listed in the table above but only one cutoff concentration under same drug condition will be included per device.
This test provides only preliminary result. An alternative laboratory test must be used to confirm the results provided by this drug test. GC/MS is the preferred confirmatory method. Evaluate preliminary positive results carefully.
Healgen Accurate Home Urine Drug Test Dip Card and Healgen Accurate Urine Drug Screen Dip Card are immunochromatographic assays that use a lateral flow system for the qualitative detection of single or multiple drugs in human urine.
The device is a dip card format. Each test device is sealed with two sachets of desiccant in an aluminum pouch. The device is in a ready-to-use format and no longer requires assembly before use.
Here's a breakdown of the acceptance criteria and study information for the Healgen Accurate Urine Drug Screen Dip Card and Healgen Accurate Home Urine Drug Test Dip Card, based on the provided document:
Acceptance Criteria and Device Performance
The acceptance criteria are implied by the reported performance, specifically the agreement rates with LC-MS/MS and within the lay user study, particularly around the established cut-off concentrations. A device is considered to meet the acceptance criteria if its performance demonstrates accurate qualitative detection of the target drugs at and around the specified cut-off concentrations.
Table of Acceptance Criteria (Implied) and Reported Device Performance:
Assessment Category | Implied Acceptance Criteria | Reported Device Performance |
---|---|---|
Precision | Consistent positive/negative results at concentrations significantly above/below cutoff. Mixed results at and near cutoff. | Generally 100% agreement for concentrations ±50% to ±100% of the cutoff. Mixed results at cutoff and ±25% of cutoff, indicating appropriate sensitivity at the threshold. |
Analytical Specificity/Interference | No interference or cross-reactivity from specified non-structural compounds at 100 µg/mL. Acceptable cross-reactivity for structurally related analogs. | Extensive list of compounds showing "Not detected" or specific cross-reactivity percentages. pH (4-9) and specific gravity (1.000-1.035) showed no interference. |
Method Comparison (Clinical Samples) | High agreement (concordance) with LC-MS/MS results, especially for clearly negative and clearly positive samples. | High concordance with LC-MS/MS, with discordant results primarily located near the cutoff concentrations. For example, for AMP (1000ng/mL), 0 false positives were seen among "Drug-Free" and "Low Negative" samples, and 0 false negatives among "High Positive" samples across all operators. |
Lay Person Study | High agreement with expected results by lay users, demonstrating ease of use and accurate interpretation. Instructions should be easy to understand. | High agreement (mostly 90-100%) near and away from cutoff concentrations for various drugs for both configurations. All participants found instructions easy to understand, and the Flesch-Kincaid reading level was 7. |
Stability | Device remains stable and performs within specifications for the claimed shelf life. | 24 months at 2-30℃ based on real-time stability study. |
Detailed Study Information:
2. Sample Size Used for the Test Set and Data Provenance:
- Precision/Reproducibility Study:
- For each drug and each concentration (-100% cutoff, -75% cutoff, -50% cutoff, -25% cutoff, cutoff, +25% cutoff, +50% cutoff, +75% cutoff, +100% cutoff), 50 tests were performed per lot.
- With 3 lots, this amounts to 150 tests per concentration per drug.
- There are 16 different drugs/cut-offs for which precision was tested. So, 16 drugs * 9 concentrations * 50 tests/concentration * 3 lots = 21,600 individual test results.
- Data Provenance: Not explicitly stated, but implies laboratory-prepared spiked samples using drug-free urine.
- Analytical Specificity/Interference Study:
- "drug metabolites and other components that are likely to cross-react in urine samples were spiked into drug-free urine samples were tested using three lots of the device."
- "non-structurally related compounds were added to drug-free urine and to urine samples containing the target drugs at 50% below and 50% above each corresponding cutoff."
- Sample Size: Not explicitly given as a total number of tests for all compounds but a significant number of tests were performed for each compound listed.
- Data Provenance: Laboratory-prepared spiked samples using drug-free urine.
- Method Comparison Study:
- Sample Size: 80 "unaltered urine clinical samples" (40 negative and 40 positive) for each drug.
- With 16 different drugs/cut-offs, this amounts to 16 drugs * 80 samples = 1280 clinical samples.
- These samples were tested by 3 operators, meaning 1280 samples * 3 operators = 3840 individual test results.
- Data Provenance: "Unaltered urine clinical samples." The country of origin is not specified but is likely the US given the FDA submission.
- Lay Person Study:
- Sample Size: 280 lay persons.
- 78 male and 62 female tested Configuration 1 (140 participants).
- 76 male and 64 female tested Configuration 2 (140 participants).
- Urine samples were prepared at 7 concentrations: -100%, +/-75%, +/-50%, +/-25% of the cutoff. Each participant received one blind-labeled sample.
- For each drug, 20 samples were prepared for each of the 7 concentrations, meaning 140 samples per drug.
- With 16 drugs, 16 drugs * 140 samples = 2240 samples. Since there were different configurations, the actual number of unique samples tested is likely higher.
- Data Provenance: Laboratory-prepared spiked samples using drug-free urine, presented to lay users.
3. Number of Experts Used to Establish Ground Truth for the Test Set and Qualifications:
- Precision/Reproducibility, Analytical Specificity/Interference, and Lay Person Studies: The ground truth for these studies was established by LC-MS/MS confirming the spiked concentrations. No human experts were involved in establishing the ground truth for these specific tests.
- Method Comparison Study: The ground truth for this study was established using LC-MS/MS results. The document states that "The samples were blind labeled and compared to LC-MS/MS results." No human experts were used for establishing the ground truth beyond the analytical method itself.
4. Adjudication Method for the Test Set:
- Method Comparison Study: "The samples were blind labeled and compared to LC-MS/MS results." This suggests direct comparison to the LC-MS/MS ground truth, with no explicit human adjudication mentioned for discrepancies. The discordant results table shows instances where the dip card result differed from the LC-MS/MS result for individual operators, but there's no mention of a formal adjudication process involving multiple experts to re-evaluate the LC-MS/MS results or the dip card interpretations. The "Accurate Result" column in the discordant table appears to consistently follow the LC-MS/MS result.
- Lay Person Study: The results were generated by lay persons. The "Agreement (%)" indicates concordance with the known spiked concentration as confirmed by LC-MS/MS, not a consensus among lay users or experts evaluating their interpretations.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done, Effect Size of How Much Human Readers Improve with AI vs. Without AI Assistance:
- No MRMC comparative effectiveness study was done. This device is a rapid lateral flow immunoassay (a dip card), not an AI-powered diagnostic device. The studies evaluate the device's performance directly and its usability by laypersons, not the improvement of human readers with AI assistance.
6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) Was Done:
- This question is not applicable as the device is a manual, qualitative immunoassay. Its "standalone" performance is what is evaluated in the precision, specificity, and method comparison studies (excluding the lay person aspect). There is no algorithm or AI component. The interpretation is visual.
7. The Type of Ground Truth Used:
- LC-MS/MS (Liquid Chromatography-Mass Spectrometry), and its tandem mass-spectrometer versions (LC-MS/MS), were used as the preferred confirmatory method to establish ground truth for all analytical and method comparison studies. For the lay person study, the known spiked concentrations, confirmed by LC-MS/MS, served as the ground truth.
8. The Sample Size for the Training Set:
- This information is not provided in the document. As this is an immunoassay and not a machine learning model, the concept of a "training set" in the context of AI is not relevant. The device is developed and manufactured, and its performance is then validated.
9. How the Ground Truth for the Training Set Was Established:
- Not applicable, as there is no mention of a training set for an AI/ML model.
Ask a specific question about this device
(65 days)
Healgen Scientific LLC
The Healgen URS Test Strips are for the qualitative detection of Leukocytes (white blood cells) and nitrite in urine as an aid in the screening of a Urinary Tract infection (UTI). It is intended for over-the-counter home use only.
Healgen URS Test Strips are in vitro diagnostic test devices for qualitative detection of leukocyte and nitrite in urine. The device is composed of two color pads aligned on a strip. One pad is employed for testing leukocyte and the other for nitrite by visually reading the color change of the pad and comparing with the corresponding blocks on a color chart.
The document describes the performance of the Healgen URS Test Strips for the qualitative detection of Leukocytes and Nitrite in urine to aid in UTI screening for over-the-counter home use.
Here's a breakdown of the requested information:
1. A table of acceptance criteria and the reported device performance
The document does not explicitly state "acceptance criteria" for each performance characteristic in a formal table. However, it presents various study results that implicitly serve as an indication of meeting performance expectations for a Class I device intended for OTC use. The implied acceptance is based on the presented results demonstrating sufficient sensitivity, precision, and user-friendability.
Here's a table summarizing the reported device performance for key characteristics:
Performance Characteristic | Analyte | Reported Device Performance |
---|---|---|
Limit of Detection (LOD) | Leukocyte | LOD verified at 15 cells/uL (100% detection at 30 and 15 cells/uL, 47% at 10 cells/uL, 7% at 5 cells/uL). |
Nitrite | LOD verified at 0.05 mg/dL (100% detection at 0.1 and 0.08 mg/dL, 83% at 0.06 mg/dL, 50% at 0.05 mg/dL). | |
Precision/Reproducibility | Leukocyte | 100% agreement with expected results across Negative, Trace (15 cells/uL), Small (70 cells/uL), Moderate (125 cells/uL), and Large (500 cells/uL) controls (N=45 for each control level). |
Nitrite | 100% agreement with positive results for 0.1 mg/dL and 0.3 mg/dL controls (N=45 for each control level). | |
Analytical Specificity | Leukocyte | No interference from various substances (Albumin, Ammonium Chloride, Bilirubin, Ciprofloxacin, Creatinine, Fructose, Galactose, Glycine, Hemoglobin, Lactose, Oxalic Acid, Phenazopyridine, Phenolphthalein, Potassium Chloride, Riboflavin, Sodium Nitrate, Sodium Nitrite, Sodium Phosphate, Sulfamethoxazole, Theophylline, Urea) at specified concentrations. |
Potential interference: High glucose (≥1000 mg/dL), high ascorbic acid (≥150 mg/dL) may decrease readings. pH >8.0 may cause false positive readings. Specific gravity >1.035 may cause false negative readings. | ||
Nitrite | No interference from various substances (same list as Leukocyte). | |
Potential interference: High ascorbic acid (≥150 mg/dL) may cause false negative readings. No effect of pH or specific gravity on nitrite testing. | ||
Stability | Both | Stable for 24 months at 2-30℃ based on real-time stability studies. |
Lay-user Study (Method Comparison) | Leukocyte | +% Agreement (Exact Match): Large (90.9%), Moderate (92%), Small (89.7%), Trace (88.5%), Negative (98.3%). |
+% Agreement (+/- Color Block): 100% for all levels. | ||
Nitrite | +% Agreement (Exact Match): 100% for both Positive and Negative, with 100% overall agreement. | |
Lay-user Instruction Survey | Instructions | All lay users indicated that the device instructions can be easily followed. Flesch-Kincaid reading analysis showed a reading grade level less than 7. |
2. Sample size used for the test set and the data provenance
- Test Set Sample Sizes:
- Sensitivity Study (LOD): 30 samples/tests per concentration for each analyte (Leukocyte and Nitrite). Each sample was tested in duplicates.
- Precision/Reproducibility Study: 45 assays results for each of eight levels of control (total 3 lots x 3 sites x 2 operators x 5 days x 1 rep/day = 90 tests per control, though results are reported as N=45 for each control).
- Analytical Specificity: Not explicitly stated as a single test set size, but multiple urine samples (negative urine spiked with interfering substances) were tested with three device lots by three different operators for each substance.
- Lay-user Studies: 150 lay users with UTI symptoms.
- Data Provenance: The document does not explicitly state the country of origin for the data. However, for the precision study, it mentions "three clinical sites," implying clinical data from a real-world setting. The lay-user study involved recruitment of "150 lay users with UTI symptoms," suggesting prospective collection of real-world user data. The sensitivity and analytical specificity studies appear to be laboratory-based. It is not specified if any data was retrospective.
3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts
- LOD and Precision Studies: "Known concentrations" for spiked samples and "expected ranges/values" for controls are used as ground truth. The expertise involved in preparing these precisely calibrated samples and controls is implied but not explicitly detailed (e.g., analytical chemists, laboratory technicians). Operators are mentioned (5 for LOD, 2 at each of 3 sites for precision), but their role is to perform the tests, not establish ground truth.
- Analytical Specificity: Ground truth is established by "negative urine with different leukocyte and nitrite concentrations" which were likely precisely prepared laboratory samples.
- Lay-user Studies: The "comparison testing by healthcare professionals" served as the ground truth. The number and qualifications of these healthcare professionals are not specified beyond the general term "healthcare professionals."
4. Adjudication method (e.g., 2+1, 3+1, none) for the test set
- The document does not describe a formal adjudication method (like 2+1 or 3+1) for resolving discrepancies in the test results.
- For the lay-user study, the comparison is made between the layperson's results and the results obtained by healthcare professionals, implying the healthcare professional's result is the definitive ground truth for comparison.
5. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance
- No, an MRMC comparative effectiveness study was not done. This device is a test strip for visual interpretation by lay-users, not an AI-assisted diagnostic device evaluated by professional readers. The study focuses on direct comparison of lay-user performance to healthcare professional results for the test strip itself, not on the impact of AI assistance on human readers.
6. If a standalone (i.e., algorithm only without human-in-the-loop performance) was done
- No, this device is not an algorithm. It is a visual assay (colorimetric test strip) that requires human interpretation. Therefore, a standalone algorithm-only performance study is not applicable. The "standalone" performance here refers to the device's inherent chemical reaction and color development, which is then interpreted by a human.
7. The type of ground truth used (expert consensus, pathology, outcomes data, etc.)
- LOD, Precision, Analytical Specificity: Ground truth was established using known concentrations of spiked analytes in urine samples and control materials, presumably confirmed by established laboratory methods and validated reagents.
- Lay-user Study: Ground truth was established by results obtained by healthcare professionals using the same test strips. This implies the healthcare professionals' interpretation served as the 'expert' reference. It's not explicitly stated whether these healthcare professionals confirmed results with a gold standard like culture or microscopic analysis, or if their reading of the strip was considered the definitive truth for the purpose of the study.
8. The sample size for the training set
- The document describes performance studies for the Healgen URS Test Strips, which are a chemical assay (test strips) with visual interpretation. This is not a machine learning or AI-based device that typically requires a "training set." Therefore, the concept of a training set size is not applicable here.
9. How the ground truth for the training set was established
- As stated above, this is not an AI/ML device, so there is no training set or ground truth establishment for a training set in the context of machine learning. The studies described are for analytical and lay-user performance of a diagnostic test strip.
Ask a specific question about this device
(161 days)
Healgen Scientific LLC
The Healgen® Accurate Oral Fluid Drug Test COT is a lateral flow chromatographic immunoassay for the qualitative detection of COT in oral fluid at the cut-off concentration 30 ng/mL.
This assay provides only a preliminary result. An alternative laboratory test must be used to confirm the results provided by this drug test. Gas chromatography/mass spectrometry (GC/MS) is the preferred method confirmation test.
The Healgen® Accurate Oral Fluid Drug Test is a competitive binding lateral flow immunochromatographic assay for the qualitative and simultaneous detection of Marijuana (THC) and Cotinine in human oral fluid at the cutoff concentrations listed below and their metabolites.
Test | Calibrator | Cut-off (ng/mL) |
---|---|---|
Cotinine (COT) | (-) Cotinine | 30 |
Marijuana (THC) | Delta-9-Tetrahydrocannabinol | 40 |
This assay provides only a preliminary analytical test result. A more specific alternate chemical method must be used in order to obtain a confirmed analytical result. Gas chromatography mass spectrometry (GC/MS) and liquid chromatography mass spectrometry (LC/MS) are the preferred confirmatory methods.
The Healgen® Accurate Oral Fluid Drug Test COT is immunochromatographic assay that uses a lateral flow system for the qualitative detection of cotinine in human oral fluid. The Healeen® Accurate Oral Fluid Drug Test immunochromatographic assay for the qualitative and simultaneous detection of Marijuana (THC) and Cotinine in human oral fluid. The tests are the first step in a two-step process. The second step is to send the sample for laboratory testing if preliminary positive results are obtained.
This document describes the performance characteristics and studies for the Healgen® Accurate Oral Fluid Drug Test and Healgen® Accurate Oral Fluid Drug Test COT, which are qualitative immunoassays for detecting Cotinine (COT) and Marijuana (THC) in oral fluid.
Here's an analysis of the acceptance criteria and the study that proves the device meets them:
1. Table of Acceptance Criteria and Reported Device Performance
The acceptance criteria are not explicitly stated as numerical targets in the document. However, the "Precision-Reproducibility-Cut-Off" studies implicitly define the expected performance around the cut-off values. The "Comparison Studies" and "Layuser Studies" then demonstrate the agreement of the device with a gold standard (LC/MS/MS).
Performance Metric | General Acceptance Goal (Implicit) | Reported Device Performance (COT) | Reported Device Performance (THC) |
---|---|---|---|
Precision-Reproducibility-Cut-Off (Analytical) | High accuracy, especially near cut-off; consistent performance across lots. | COT: | |
-100% to -50% cut off: 50-/0+ (100% negative) | |||
-25% cut off: Lot 1: 50-/0+, Lot 2: 48-/2+, Lot 3: 49-/1+ (High negative agreement) | |||
Cut off: Lot 1: 24-/26+, Lot 2: 23-/27+, Lot 3: 24-/26+ (Balanced +/- at cut-off, as expected) | |||
+25% to +100% cut off: Near 100% positive | THC: | ||
-100% to -50% cut off: 50-/0+ (100% negative) | |||
-25% cut off: Lot 1: 47-/3+, Lot 2: 48-/2+, Lot 3: 49-/1+ (High negative agreement) | |||
Cut off: Lot 1: 24-/26+, Lot 2: 28-/22+, Lot 3: 24-/26+ (Balanced +/- at cut-off, as expected) | |||
+25% to +100% cut off: Near 100% positive | |||
Method Comparison Study (Expert Operator) | High percentage of correct results across all concentration ranges, especially for Drug-Free and High Positive samples. | COT: | |
Drug-Free: 100% negative (112/112) | |||
Less than Half Cutoff: 100% negative (93/93) | |||
Near Cutoff Negative: 88% correct (35/40) | |||
Near Cutoff Positive: 88% correct (28/32) | |||
High Positive: 100% positive (150/150) | THC: | ||
Drug-Free: 100% negative (35/35) | |||
Less than Half Cutoff: 100% negative (13/13) | |||
Near Cutoff Negative: 71.4% correct (5/7) | |||
Near Cutoff Positive: 60% correct (6/10) | |||
High Positive: 100% positive (61/61) | |||
Layuser Study (OTC Use) | High percentage of correct results (similar to expert), ease of use. | COT (Layuser): | |
Drug-Free: 100% negative (95/95) | |||
Less than Half Cutoff: 100% negative (39/39) | |||
Near Cutoff Negative: 89% correct (31/35) | |||
Near Cutoff Positive: 87% correct (34/39) | |||
High Positive: 100% positive (154/154) | |||
All layusers found instructions easy to understand and follow (Reading Grade Level 7). | Not applicable (Layuser study only reported for COT, which is for OTC use as per the Indications for Use). |
2. Sample Sizes Used for the Test Set and Data Provenance
-
Precision-Reproducibility-Cut-Off (Analytical Performance):
- For each of the 9 concentration levels and 3 device lots: 50 samples (2 runs per day for 25 days).
- Total samples = 9 concentrations * 50 samples/concentration * 3 lots = 1350 samples for COT, and 1350 samples for THC.
- Data Provenance: Samples were "spiked" with cotinine or marijuana in negative oral fluid samples, indicating a controlled laboratory setting. The origin (country/retrospective/prospective) is not specified, but the nature of spiking implies prospective sample preparation for the study.
-
Method Comparison Studies (Expert Operator):
- COT: 427 samples
- THC: 126 samples
- Data Provenance: Oral fluid samples. The data provenance (e.g., country of origin, retrospective or prospective collection) is not specified.
-
Layuser Studies:
- COT: 362 samples
- Data Provenance: Oral fluid samples. The data provenance is not specified.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts
- Ground Truth Establishment: The ground truth for all performance studies (Precision, Method Comparison, Layuser) was established by LC/MS/MS (Liquid Chromatography/Mass Spectrometry/Mass Spectrometry), which is stated as the "preferred method confirmation test."
- Qualifications of Experts: This implies that the ground truth was not established by human experts interpreting images or subjective data, but by highly accurate analytical laboratory equipment and the qualified personnel operating it. The document does not specify the number or qualifications of the lab personnel who performed the LC/MS/MS analysis, as GC/MS and LC/MS/MS are considered an objective gold standard in drug testing.
4. Adjudication Method for the Test Set
- LC/MS/MS serves as the objective ground truth, providing quantitative results against which the device's qualitative results are compared. Therefore, an "adjudication method" in the sense of reconciling disagreements between human readers is not applicable here. Any discrepancies between the device result and LC/MS/MS are reported as "discordant results."
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done, and the Effect Size of How Much Human Readers Improve with AI vs Without AI Assistance
- This is not an AI-assisted diagnostic device, but a rapid diagnostic test (lateral flow immunoassay). Therefore, an MRMC comparative effectiveness study involving human readers with and without AI assistance is not applicable.
- The "Layuser Studies" evaluate the performance of the device in the hands of untrained users, essentially assessing human performance with the device instructions. It shows that layusers can achieve comparable results to expert operators for COT detection.
6. If a Standalone (i.e., algorithm-only without human-in-the-loop performance) Was Done
- This question is not applicable as the device is a lateral flow immunoassay kit, not an algorithm. Its "standalone" performance is inherently demonstrated by the results it produces directly from the oral fluid sample, without further human-in-the-loop interpretation beyond reading the visual lines on the strip. The "Precision-Reproducibility-Cut-Off" and "Method Comparison Studies" characterize this inherent device performance.
7. The Type of Ground Truth Used
- The ground truth used was analytical confirmation by LC/MS/MS. This is an objective, quantitative laboratory method considered the gold standard for drug detection and quantification in biological samples.
8. The Sample Size for the Training Set
- The document describes performance studies, but it does not mention a "training set" in the context of machine learning. This is a traditional immunoassay device, which does not employ machine learning algorithms that require separate training and test sets. The studies described are for analytical and clinical validation of the device's performance.
9. How the Ground Truth for the Training Set Was Established
- As there is no mention of a "training set" for a machine learning model, this question is not applicable.
Ask a specific question about this device
(210 days)
Healgen Scientific, LLC
The Strep A Rapid Test Strip (Throat Swab) is a rapid chromatographic immunoassy for the qualitative detection of Streptococcus pyogenes (Group A B-hemolytic Streptococcus, Strep A) antigen from throat swab specimens of symptomatic patients to aid in the diagnosis of Group A Streptococcus bacterial infection.
All negative test results should be confirmed by bacterial culture because negative results do not prection with Group A Streptococcus and should not be used as the sole basis for treatment.
Healgen Strep A Rapid Test Strip (Throat Swab) is a qualitative, lateral flow immunoassay for the detection of Strep A antigen directly from a throat swab sample.
In this test, antibody specific to Strep A carbohydrate antigen is coated on the test line region of the test. During testing, the extracted throat swab specimen reacts with an antibody to Strep A that is coated onto particles. The mixture migrates up the membrane to react with the antibody to Strep A on the membrane and generate a color line in the test line region. The presence of this color line in the test line region indicates a positive result, while its absence indicates a negative result. To serve as a procedural control, a colored line will always appear in the control line region, indicating that proper volume of specimen has been added and membrane wicking has occurred.
Here's a breakdown of the acceptance criteria and study details for the Healgen Strep A Rapid Test Strip (Throat Swab):
1. Table of Acceptance Criteria and Reported Device Performance
Device Performance Metric | Acceptance Criteria (Implicit) | Reported Device Performance |
---|---|---|
Analytical Performance | ||
Precision/Reproducibility | Results should be reproducible across different users, sites, lots, and days. | Overall Positive Detection: |
- True negative sample: 0% (0/180)
- Moderate positive sample (1.8x10^4 CFU/mL): 100% (180/180)
- LoD sample (7.2x10^3 CFU/mL): 95.6% (172/180)
- Low negative sample (3.6x10^3 CFU/mL): 44.4% (80/180)
(Concluded: No significant differences between users, sites, lots, and days; results are reproducible with good precision.) |
| Limit of Detection (LoD) | The LoD should be clearly established for both clinical matrix and saline solution. | LoD in clinical matrix: 7.2x10^3 CFU/mL (equivalent to 360 bacteria on the swab) based on 95.2% detection (20/21) at 7.2x10^4 CFU/mL.
LoD in saline solution: 7.2x10^3 CFU/mL (equivalent to 360 bacteria on the swab) based on 95.2% detection (20/21) at 7.2x10^3 CFU/mL. |
| Interference | No false positive or false negative results with common interfering substances (blood, mucus, saliva, medications). | No false positive or false negative results observed with various interfering substances (blood, mucin, OTC mouthwashes, lozenges, throat sprays, cough syrups, active ingredients such as acetaminophen, ibuprofen, etc.) at tested concentrations (e.g., 20% vol/vol for liquids, 5mg/mL for solids). |
| Analytical Specificity | No cross-reactivity with other common respiratory tract organisms (bacteria and viruses). | No cross-reactivity found for a comprehensive list of organisms (e.g., Arcanobacterium haemolyticum, Bordetella pertussis, Candida albicans, Enterococcus faecalis, Escherichia coli, various Streptococcus species, Adenovirus, Cytomegalovirus, HSV, etc.) at tested concentrations. |
| Clinical Performance | | |
| Clinical Sensitivity | Performance comparable to the legally marketed predicate device (Predicate: 95% CI (88-98%)). | Overall Clinical Sensitivity: 97.1% (200/206) with 95% CI (93.7-98.8%) - Age 0-5: 97.4% (74/76)
- Age 5-21: 96.7% (119/123)
- Age 21+: 100% (7/7)
(No statistical differences between age groups.) |
| Clinical Specificity | Performance comparable to the legally marketed predicate device (Predicate: 98% CI (96-99%)). | Overall Clinical Specificity: 99.4% (161/162) with 95% CI (96.2-100.0%) - Age 0-5: 98.1% (52/53)
- Age 5-21: 100% (88/88)
- Age 21+: 100% (21/21)
(No statistical differences between age groups.) |
| Consistency | Performance across different age groups should be consistent. | No statistical differences in performance between age groups. |
2. Sample size used for the test set and the data provenance
- Test Set Sample Size:
- Clinical Study: 368 subjects (206 culture-positive, 162 culture-negative).
- Analytical Precision: 180 determinations per sample type (60 determinations per site across 3 sites).
- Analytical LoD: 21 results per dilution (7 operators x 3 lots).
- Analytical Interference: Multiple tests across 3 lots for each interfering substance, for both positive and negative specimens.
- Analytical Specificity (Cross-reactivity): Multiple tests across 3 lots for each organism by 3 professional users.
- Data Provenance:
- The document does not explicitly state the country of origin for the clinical data.
- The clinical study appears to be prospective/concurrent as it describes testing "subjects...exhibiting symptoms of pharyngitis by both the Healgen Strep A Rapid Test Strip (Throat Swab) and the culture studies." This implies collection of samples and testing using both methods at the time of study.
- The analytical studies (precision, LoD, interference, specificity) were laboratory-based, performed internally or by designated personnel.
3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts
- Clinical Study Ground Truth: The ground truth for the clinical study was established by bacterial culture. The document implies that the culture results were considered the reference standard. It does not explicitly state the number of experts or their qualifications involved in performing or interpreting these cultures. However, bacterial culture is a standard clinical laboratory method typically performed by trained medical technologists or microbiologists.
- Analytical Studies:
- Precision/Reproducibility: 6 professional operators (2 at each of 3 sites) participated. Their specific qualifications are not detailed beyond "professional operators."
- LoD: 7 operators performed the testing. Their specific qualifications are not detailed beyond "operators."
- Interference: 3 laboratory assistants with relevant experience performed the test.
- Analytical Specificity: 3 professional users performed the test.
4. Adjudication method (e.g., 2+1, 3+1, none) for the test set
- The document does not describe an adjudication method (like 2+1 or 3+1) for establishing the ground truth in the clinical study. The reference standard (bacterial culture) appears to have been used directly.
5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance
- No Multi-Reader Multi-Case (MRMC) comparative effectiveness study was done in the context of AI assistance. This device is a manual rapid diagnostic test strip, not an AI-assisted diagnostic tool. Therefore, the concept of human readers improving with AI assistance is not applicable here. The "operators" or "users" in the analytical studies are performing the manual test according to instructions.
6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done
- This question is not applicable as the device is a manual rapid test strip, not an algorithm or AI-driven system. It does not operate without human interaction.
7. The type of ground truth used (expert consensus, pathology, outcomes data, etc.)
- The ground truth for the clinical study was bacterial culture, which is considered the gold standard for diagnosing Group A Streptococcus bacterial infection.
- For the analytical studies (LoD, interference, specificity), the ground truth was based on known concentrations of target analytes (S. pyogenes) or known presence/absence of interfering/cross-reacting organisms.
8. The sample size for the training set
- The document does not mention a training set in the context of machine learning or AI. This is a traditional rapid diagnostic test, not a learning algorithm. The "training" of the device refers to its manufacturing and validation process, not data-driven machine learning.
9. How the ground truth for the training set was established
- This question is not applicable as there is no mention or indication of a "training set" for an AI or machine learning model in this submission.
Ask a specific question about this device
(216 days)
HEALGEN SCIENTIFIC LLC
Healgen Multi-Drug Urine Test Cup and Healgen Multi-Drug Urine Test Dip Card are competitive binding, lateral flow immunochromatographic assays for qualitative and simultaneous detection of Amphetamine, Oxazepam, Cocaine, Cannabinoids, Methamphetamine, Oxycodone, Secobarbital, Buprenorphine, Methylenedioxymethamphetamine. Phencyclidine. EDDP. Nortriptyline and Methadone in human urine at the cutoff concentrations of:
Drug(Identifier) | Cut-off level |
---|---|
Amphetamine | 500 ng/mL |
Oxazepam | 300 ng/mL |
Cocaine | 150 ng/mL |
Cannabinoids | 50 ng/mL |
Methamphetamine | 500 ng/mL |
Morphine | 300 ng/mL or 2000 ng/mL |
Oxycodone | 100 ng/mL |
Secobarbital | 300 ng/mL |
Buprenorphine | 10 ng/mL |
Methylenedioxy-methamphetamine | 500 ng/mL |
Phencyclidine | 25 ng/mL |
Methadone | 300 ng/mL |
Propoxyphene | 300 ng/mL |
Nortriptyline | 1000 ng/mL |
Configuration of the Healgen Multi-Drug Urine Test Cup and Healgen Multi-Drug Urine Test Dip Card can consist of any combination of the above listed drug analytes.
The test may yield positive results for the prescription drugs Buprenorphine, Oxazepam, Secobarbital, Nortriptyline, Propoxyphene and Oxycodone when taken at or above prescribed doses. It is not intended to distinguish between prescription use or abuse of these drugs. Clinical consideration and professional judgment should be exercised with any drug of abuse test result, particularly when the preliminary result is positive. The test provides only preliminary test results. A more specific alternative chemical must be used in order to obtain a confirmed analytical result. GC/MS or LC/MS is the preferred confirmatory method.
Healgen Multi-Drug Urine Test Cup and Healgen Multi-Drug Urine Test DipCard are competitive binding, lateral flow immunochromatographic assays for qualitative and simultaneous detection of Amphetamine, Oxazepam, Cocaine, Cannabinoids, Methamphetamine, Morphine, Oxycodone, Secobarbital, Buprenorphine, Methylenedioxy-methamphetamine, Phencyclidine, Propoxyphene, Nortriptyline and Methadone in human urine samples. Healgen Multi-Drug devices detect each of analytes on different strips.
A positive urine sample will not generate a colored-line for the specific drug tested in the designated region. A negative urine specimen or a urine sample containing Amphetamine, Oxazepam, Cocaine, Cannabinoids, Methamphetamine, Morphine, Oxycodone, Secobarbital, Buprenorphine, Methylenedioxy-methamphetamine, Phencyclidine, Propoxyphene, Nortriptyline and Methadone at the concentration below the designated cutoff levels will generate a colored line in the designated test region for the drug. To serve as a test control, a color line will always appear at the control region.
1. Table of Acceptance Criteria and Reported Device Performance
The acceptance criteria for each drug for the Healgen Multi-Drug Urine Test Cup and Healgen Multi-Drug Urine Test Dip Card in the lay user study can be inferred as achieving at least 85% agreement with GC/MS results for urine samples near the cutoff concentrations (+/- 25% of cutoff). For samples further away from the cutoff (±50%, ±75%, -100% of cutoff), the acceptance criterion appears to be 100% agreement.
Healgen Multi-Drug Urine Test Cup and Dip Card Performance (Lay User Study Data)
Drug | % of Cutoff | Number of Samples | % Agreement (Lay Person vs. GC/MS) |
---|---|---|---|
Methamphetamine | -100% | 20 | 100% |
-75% | 20 | 100% | |
-50% | 160 | 100% | |
-25% | 20 | 95% | |
+25% | 20 | 95% | |
+50% | 40 | 100% | |
+75% | 20 | 100% | |
Cocaine | -100% | 20 | 100% |
-75% | 20 | 100% | |
-50% | 160 | 100% | |
-25% | 20 | 95% | |
+25% | 20 | 90% | |
+50% | 40 | 100% | |
+75% | 20 | 100% | |
Cannabinoids (THC) | -100% | 20 | 100% |
-75% | 20 | 100% | |
-50% | 160 | 100% | |
-25% | 20 | 90% | |
+25% | 20 | 95% | |
+50% | 40 | 100% | |
+75% | 20 | 100% | |
Morphine (MOP 300) | -100% | 20 | 100% |
-75% | 20 | 100% | |
-50% | 160 | 100% | |
-25% | 20 | 85% | |
+25% | 20 | 95% | |
+50% | 40 | 100% | |
+75% | 20 | 100% | |
Morphine (MOP 2000) | -100% | 20 | 100% |
-75% | 20 | 100% | |
-50% | 160 | 100% | |
-25% | 20 | 95% | |
+25% | 20 | 90% | |
+50% | 40 | 100% | |
+75% | 20 | 100% | |
Oxazepam | -100% | 20 | 100% |
-75% | 20 | 100% | |
-50% | 160 | 100% | |
-25% | 20 | 95% | |
+25% | 20 | 85% | |
+50% | 40 | 100% | |
+75% | 20 | 100% | |
Amphetamine | -100% | 20 | 100% |
-75% | 20 | 100% | |
-50% | 160 | 100% | |
-25% | 20 | 90% | |
+25% | 20 | 90% | |
+50% | 40 | 100% | |
+75% | 20 | 100% | |
Oxycodone | -100% | 20 | 100% |
-75% | 20 | 100% | |
-50% | 160 | 100% | |
-25% | 20 | 90% | |
+25% | 20 | 95% | |
+50% | 40 | 100% | |
+75% | 20 | 100% | |
Methadone | -100% | 20 | 100% |
-75% | 20 | 100% | |
-50% | 160 | 100% | |
-25% | 20 | 85% | |
+25% | 20 | 95% | |
+50% | 40 | 100% | |
+75% | 20 | 100% | |
Secobarbital | -100% | 20 | 100% |
-75% | 20 | 100% | |
-50% | 160 | 100% | |
-25% | 20 | 95% | |
+25% | 20 | 95% | |
+50% | 40 | 100% | |
+75% | 20 | 100% | |
Buprenorphine | -100% | 20 | 100% |
-75% | 20 | 100% | |
-50% | 160 | 100% | |
-25% | 20 | 95% | |
+25% | 20 | 95% | |
+50% | 40 | 100% | |
+75% | 20 | 100% | |
Phencyclidine | -100% | 20 | 100% |
-75% | 20 | 100% | |
-50% | 160 | 100% | |
-25% | 20 | 90% | |
+25% | 20 | 90% | |
+50% | 40 | 100% | |
+75% | 20 | 100% | |
MDMA | -100% | 20 | 100% |
-75% | 20 | 100% | |
-50% | 160 | 100% | |
-25% | 20 | 95% | |
+25% | 20 | 90% | |
+50% | 40 | 100% | |
+75% | 20 | 100% | |
Propoxyphene | -100% | 20 | 100% |
-75% | 20 | 100% | |
-50% | 160 | 100% | |
-25% | 20 | 90% | |
+25% | 20 | 90% | |
+50% | 40 | 100% | |
+75% | 20 | 100% | |
Tricyclic Antidepressants | -100% | 20 | 100% |
(Nortriptyline) | -75% | 20 | 100% |
-50% | 160 | 100% | |
-25% | 20 | 95% | |
+25% | 20 | 95% | |
+50% | 40 | 100% | |
+75% | 20 | 100% |
The device meets these acceptance criteria, as all reported percentage agreements are at or above the specified thresholds for each drug and concentration level.
2. Sample Size Used for the Test Set and Data Provenance
- Sample Size for Test Set:
- For each drug analyte tested in the lay user study, the total number of samples evaluated was 300 per format (Cup and Dip Card) across different concentration levels.
- Specifically, for each drug analyte, the breakdown of samples by concentration was:
- -100% Cutoff: 20 samples
- -75% Cutoff: 20 samples
- -50% Cutoff: 160 samples
- -25% Cutoff: 20 samples
- +25% Cutoff: 20 samples
- +50% Cutoff: 40 samples
- +75% Cutoff: 20 samples
- Data Provenance: The document does not explicitly state the country of origin of the data. However, as it is a submission to the United States FDA, it is implied that the data is intended for use in the US market. The study utilized "drug free-pooled urine specimens" which were then spiked with drugs, indicating a controlled laboratory setting for sample preparation. The study was conducted at "three intended user sites" with "lay persons," suggesting a prospective evaluation of the device by untrained users.
3. Number of Experts Used to Establish Ground Truth for the Test Set and Qualifications
- Number of Experts: Not applicable in the context of this study. The ground truth was not established by human experts interpreting results from the device itself.
- Qualifications: Not applicable.
4. Adjudication Method for the Test Set
- Adjudication Method: Not applicable. The ground truth was established by Gas Chromatography/Mass Spectrometry (GC/MS), which is an objective chemical analysis method, not a subjective interpretation requiring adjudication among experts. The lay users independently read the results from the test devices.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done
- No, an MRMC comparative effectiveness study was not done. This study focuses on the agreement of the device's results (read by lay users) with a gold standard (GC/MS), rather than comparing the performance of human readers with and without AI assistance. The device itself is a lateral flow immunoassay, not an AI system.
6. If a Standalone (Algorithm Only Without Human-in-the-Loop Performance) Was Done
- Yes, a standalone study was done in the sense that the device's performance was compared directly against a definitive analytical method (GC/MS). The lay user study involved individuals interpreting the results of the device directly, without the intervention of an algorithm or AI to assist in that interpretation. The comparison is between the human interpretation of the device and the chemical gold standard, effectively evaluating the standalone performance of the rapid immunoassay device.
7. The Type of Ground Truth Used
- Type of Ground Truth: The ground truth for the test set was established using Gas Chromatography/Mass Spectrometry (GC/MS). The document explicitly states: "The concentrations of the samples were confirmed by GC/MS." GC/MS is a highly accurate and widely accepted method for confirming the presence and concentration of drugs in urine, thus serving as the gold standard.
8. The Sample Size for the Training Set
- The document does not explicitly mention a training set in the context of the lay user study or the development of this specific final product (Healgen Multi-Drug Urine Test Cup/Dip Card). The performance data cited (precision, cut-off, interference, specificity, and method comparison) refers to earlier, individual FDA-cleared predicate devices (K142280, K143187, etc.). It's possible that earlier development and validation work for those predicate devices constituted a "training" or development phase, but details are not provided here. This document primarily describes the validation of the multi-drug format for lay use against a gold standard.
9. How the Ground Truth for the Training Set Was Established
- As a training set is not explicitly described for this submission, the method for establishing its ground truth is not provided. However, for similar in-vitro diagnostic devices, the ground truth for training/development (if applicable) would typically be established using validated laboratory reference methods such as GC/MS or LC/MS, similar to the method used for the test set.
Ask a specific question about this device
(159 days)
HEALGEN SCIENTIFIC LLC
The Healgen Multi-Drug Urine Test Cup and Healgen Multi-Drug Urine Test Dip Card are competitive binding, lateral flow immunochromatographic assays for qualitative and simultaneous detection of Amphetamine, Oxazepam, Cocaine, Cannabinoids, Methamphetamine, Morphine, Oxycodone, Secobarbital, Buprenorphine, Methylenedioxymethamphetamine. Phencyclidine. EDDP. Nortriptyline and Methadone in human urine at the cutoff concentrations of:
Drug (identifier) | Cut-off level |
---|---|
Amphetamine | 1000 ng/mL |
Oxazepam | 300 ng/mL |
Cocaine | 300 ng/mL |
Cannabinoids | 50 ng/mL |
Methamphetamine | 1000 ng/mL |
Morphine | 300 or 2000 ng/mL |
Oxycodone | 100 ng/mL |
Secobarbital | 300 ng/mL |
Buprenorphine | 10 ng/mL |
Methylenedioxymethamphetamine | 500 ng/mL |
Phencyclidine | 25 ng/mL |
Methadone | 300 ng/mL |
EDDP | 300 ng/mL |
Nortriptyline | 1000 ng/mL |
Configurations of the Healgen Multi-Drug Urine Test Cup and Healgen Multi-Drug Urine Test Dip Card can consist of any combination of the above listed drug analytes up to a maximum of 14 analytes. Only one cutoff concentration will be included per analyte per device.
The test may yield positive results for the prescription drugs Buprenorphine, Oxazepam, Secobarbital, Nortriptyline and Oxycodone when taken at or above prescribed doses. It is not intended to distinguish between prescription use or abuse of these drugs. There are no uniformly recognized drug levels for Buprenorphine. Nortriptyline or Oxycodone in urine. Clinical consideration and professional judgment should be exercised with any drug of abuse test result, particularly when the preliminary result is positive. The test provides only preliminary test results. A more specific alternative chemical method must be used in order to obtain a confirmed analytical result. GC/MS is the preferred confirmatory method.
Healgen Multi-Drug Urine Test Cup and Healgen Multi-Drug Urine Test DipCard are competitive binding, lateral flow immunochromatographic assays for qualitative and simultaneous detection of Amphetamine, Oxazepam, Cocaine, Cannabinoids, Methamphetamine, Morphine, Oxycodone, Secobarbital, Buprenorphine, Methylenedioxy-methamphetamine, Phencyclidine, EDDP, Nortriptyline and Methadone in human urine samples. Healgen Multi-Drug devices detect each of analytes on different strips.
A positive urine sample will not generate a colored-line for the specific drug tested in the designated region. A negative urine specimen or a urine sample containing Amphetamine, Oxazepam, Cocaine, Cannabinoids, Methamphetamine, Morphine, Oxycodone, Secobarbital, Buprenorphine, Methylenedioxy-methamphetamine, Phencyclidine, EDDP, Nortriptyline and Methadone at the concentration below the designated cutoff levels will generate a colored line in the designated test region for the drug. To serve as a test control, a color line will always appear at the control region.
The Healgen Multi-Drug Urine Test Cup and Healgen Multi-Drug Urine Test Dip Card are devices for qualitative and simultaneous detection of various drugs in human urine. The study conducted was a lay user study to verify the device's performance when used by individuals without professional training.
Here's the breakdown of the acceptance criteria and study details:
1. Table of Acceptance Criteria and Reported Device Performance
The acceptance criteria are implicitly defined by the percentage agreement with GC/MS (Gas Chromatography/Mass Spectrometry) results for different drug concentrations relative to the cutoff level. While no explicit percentage agreement threshold is stated as an "acceptance criteria", standard practice in such studies would expect high agreement for samples at or significantly away from the cutoff. The reported performance shows high agreement, particularly at concentrations of -50% Cutoff, +50% Cutoff, and +75% Cutoff.
Reported Device Performance based on Lay User Study (Agreement with GC/MS):
Drug Category | Concentration (% of Cutoff) | Number of Samples | % Agreement (Lay Person vs. GC/MS) |
---|---|---|---|
For all Drugs Tested | -100% Cutoff | 20 | 100% |
-75% Cutoff | 20 | 100% | |
-50% Cutoff | 170 | 100% | |
-25% Cutoff | 20 | 85-95% (varied by drug/device) | |
+25% Cutoff | 20 | 85-95% (varied by drug/device) | |
+50% Cutoff | 50 | 100% | |
+75% Cutoff | 20 | 100% |
Note: The specific percentage agreement for -25% and +25% cutoff concentrations varied slightly across different drugs and device types (dip card vs. cup). For instance, Morphine at +25% Cutoff for the dip card and Oxycodone at +25% Cutoff for the cup showed 85% agreement, while others were 90-95%. Oxazepam at -25% Cutoff for the cup also showed 85% agreement.
2. Sample Sizes and Data Provenance
- Test Set Sample Size: For each drug and concentration level (e.g., Methamphetamine at -100% Cutoff), there were between 20 and 170 samples tested. Each device format (Cup and Dip Card) was tested with these samples across all drugs.
- For -100%, -75%, -25%, +25%, +75% Cutoff, there were 20 samples each.
- For -50% Cutoff, there were 170 samples.
- For +50% Cutoff, there were 50 samples.
- Total samples per drug per device format were 20+20+170+20+20+50+20 = 320 samples.
- Given there are 14 drugs, the total number of samples processed by lay users for each device format would be approximately 14 drugs * 320 samples/drug = 4480 samples.
- Data Provenance: Not explicitly stated regarding the country of origin of the data. The study was a prospective lay user study where urine samples were prepared and then tested by lay users.
3. Number of Experts and Qualifications for Ground Truth
- Number of Experts: Not applicable in the context of human readers for generating ground truth.
- Qualifications of Experts: Not applicable.
4. Adjudication Method for the Test Set
- Adjudication Method: Not applicable. The "ground truth" was established by an independent laboratory method (GC/MS, see point 7).
5. Multi Reader Multi Case (MRMC) Comparative Effectiveness Study
- MRMC Study: No, this was not a MRMC comparative effectiveness study involving human readers with and without AI assistance. This study focused on the performance of a point-of-care test when interpreted by lay users.
6. Standalone Performance
- Standalone Performance: Yes, the study evaluates the standalone performance of the device (Healgen Multi-Drug Urine Test Cup and Dip Card) when used directly by lay users. The outcome of the device (positive/negative line on the test strip) as interpreted by the lay user is compared against the GC/MS reference standard.
7. Type of Ground Truth Used
- Type of Ground Truth: The ground truth for the test set was established using GC/MS (Gas Chromatography/Mass Spectrometry), which is the preferred confirmatory method for drug concentration in urine samples.
8. Sample Size for the Training Set
- Training Set Sample Size: The document does not mention a separate training set. The device is a lateral flow immunochromatographic assay, not an AI/ML algorithm that typically requires a training set. The "performance data of precision, cut-off, interference, specificity and method comparison" were reported in predicate device submissions (K142280, K143187, K141647, K140546, K150791, K150096 and K151348), implying these established characteristics during the development of earlier versions.
9. How the Ground Truth for the Training Set Was Established
- Ground Truth for Training Set: Not applicable as there is no specific "training set" for the device in the context of AI/ML or expert interpretation. The performance characteristics of the assay would have been established through a series of analytical studies using known concentrations and confirmed methods (like GC/MS) during its development and prior submissions, which serve as the foundation for the assay's expected behavior.
Ask a specific question about this device
Page 1 of 3