(43 days)
The AssureTech Panel Dip Tests are competitive binding, lateral flow immunochromatographic assays for qualitative and simultaneous detection of Amphetamine, Oxazepam, Cocaine, Methamphetamine, Fentanyl, Norfentany], Morphine, Oxycodone, Secobarbital, Buprenorphine, Methylenedioxy-methamphetamine, Phencyclidine, Methadone, EDDP, Nortriptyline and d-Propoxyphene in human urine at the cutoff concentrations listed. The single or multi-test panels can consist of up to seventeen (17) of the above listed analytes in any combination. Clinical consideration and professional judgment should be exercised with any drug of abuse test result, particularly when the preliminary result is positive. The test provides only preliminary test results. A more specific alternative chemical method must be used in order to obtain a confirmed analytical result. GC/MS is the preferred confirmatory method. For in vitro diagnostic use only.
The AssureTech Quick Cup Tests are competitive binding, lateral flow immunochromatographic assays for qualitative and simultaneous detection of Amphetamine, Oxazepam, Cocaine, Methamphetamine, Fentanyl, Norfentany], Morphine, Oxycodone, Secobarbital, Buprenorphine, Methylenedioxy-methamphetamine, Phencyclidine, Methadone, EDDP, Nortriptyline and d-Propoxyphene in human urine at the cutoff concentrations listed. The single or multi-test panels can consist of up to seventeen (17) of the above listed analytes in any combination. Clinical consideration and professional judgment should be exercised with any drug of abuse test result, particularly when the preliminary result is positive. The test provides only preliminary test results. A more specific alternative chemical method must be used in order to obtain a confirmed analytical result. GC/MS is the preferred confirmatory method. For in vitro diagnostic use only.
The AssureTech Multi-drug Urine Test Panel are competitive binding, lateral flow immunochromatographic assays for qualitative and simultaneous detection of Amphetamine, Cocaine, Marijuana, Methamphetamine, Morphine, Fentanyl, Norfentanyl, Oxycodone, Secobarbital, Buprenorphine, Methylenedioxy-methamphetamine, Phencyclidine, Methadone, EDDP, Nortriptyline, d-Propoxyphene and adulterants in human urine at the cutoff concentrations listed. The single or multi-test panel can consist of up to seventeen (17) of the above listed analytes in any combination. It is for in vitro diagnostic use only. The test provides only preliminary test results. A more specific alternative chemical must be used in order to obtain a confirmed analytical result. GC/MS is the preferred confirmatory method. Clinical consideration and professional judgment should be exercised with any drug of abuse test result, particularly when the preliminary result is positive.
The AssureTech Multi-drug Urine Test Cup are competitive binding, lateral flow immunochromatographic assays for qualitative and simultaneous detection of Amphetamine, Oxazepam, Marijuana, Methamphetamine, Morphine, Fentanyl, Norfentanyl, Oxycodone, Secobarbital, Buprenorphine, Methylenedioxy-methamphetamine, Phencyclidine, Methadone, EDDP, Nortriptyline, d-Propoxyphene and adulterants in human urine at the cutoff concentrations listed. The single or multi-test cups can consist of up to seventeen (17) of the above listed analytes in any combination. It is for in vitro diagnostic use only. The test provides only preliminary test results. A more specific alternative chemical method must be used in order to obtain a confirmed analytical result. GC/MS is the preferred confirmatory method. Clinical consideration and professional judgment should be exercised with any drug of abuse test result, particularly when the preliminary result is positive.
The AssureTech Panel Dip Tests and AssureTech Quick Cup Tests are immunochromatographic assays that use a lateral flow system for the qualitative detection of Amphetamine, Oxazepam, Cocaine, Marijuana, Methamphetamine, Fentanyl, Morphine, Oxycodone, Secobarbital, Buprenorphine, Methylenedioxy-methamphetamine, Phencyclidine, Methadone, EDDP. Nortriptyline and Propoxyphene (target analytes) in human urine. The products are single use in vitro diagnostic devices, which come in the formats of Panel Dip Cards or Cups. Each test kit contains a Test Device (in one of the two formats), a package insert and a urine cup for sample collection. Each test device is sealed with a desiccant in an aluminum pouch.
The provided document describes the FDA 510(k) premarket notification for AssureTech Panel Dip Tests and Quick Cup Tests, which are in vitro diagnostic devices for qualitative and simultaneous detection of various drugs of abuse in human urine. The document focuses on demonstrating substantial equivalence to a predicate device (K181768).
Here's an analysis of the acceptance criteria and the study proving the device meets them, based on the provided text:
Acceptance Criteria and Reported Device Performance
The acceptance criteria for this type of device are primarily related to its analytical performance, specifically its ability to accurately detect the presence or absence of target drugs at specified cutoff concentrations. The device is a qualitative test, meaning it provides a "positive" or "negative" result, rather than a quantitative measurement.
The study demonstrates performance through:
- Precision: Consistency of results across multiple runs and lots, especially near the cutoff concentrations.
- Specificity: Ability to react only with the target drug/metabolite and not with other substances or structurally similar compounds.
- Interference: Lack of false positives/negatives due to common interfering substances in urine, or variations in urine specific gravity and pH.
- Method Comparison: Agreement of device results with a known, more precise reference method (LC/MS).
- Lay-user study: Evaluation of the device's performance when used by non-professionals, assessing ease of use and accuracy of interpretation.
Here is a table summarizing the reported device performance for Fentanyl (FYL), Norfentanyl (NFYL), and as an example for another drug, Amphetamine (AMP) from the "Lay-user study" data. The document does not explicitly state numerical "acceptance criteria" for each performance metric, but rather presents the results of the studies conducted to show sufficient performance for regulatory clearance. The implicit acceptance criterion for a qualitative test like this is generally very high accuracy, especially around the cutoff, and a low rate of false positives/negatives.
Table of Performance for Key Drugs (Fentanyl, Norfentanyl, Amphetamine)
Precision Study (Fentanyl - Panel Dip/Quick Cup, Norfentanyl - Panel Dip/Quick Cup):
The precision data is presented for three lots and various concentrations relative to the cutoff. The data shows very high consistency. For instance, for Fentanyl:
- At -100%, -75%, -50% cut off (negative range), all 50 tests across 3 lots consistently yielded negative results (50-/0+).
- At +25%, +50%, +75%, +100% cut off (positive range), all 50 tests consistently yielded positive results (50+/0-).
- At the cutoff concentration, the device shows variability, as expected for tests near the decision threshold. For Fentanyl Panel Dip, results were 28+/22-, 29+/23-, 28+/22- for Lot 1, 2, 3 respectively (meaning some tests were positive and some negative at the cutoff). This variability is inherent for qualitative tests around the cutoff and implies that some samples at the cutoff may read positive and others negative, which is acceptable performance for a qualitative test. Similar patterns are observed for Quick Cup Fentanyl, Panel Dip Norfentanyl, and Quick Cup Norfentanyl.
Method Comparison Study (Fentanyl - Panel Dip/Quick Cup, Norfentanyl - Panel Dip/Quick Cup):
This study compared the device results against LC/MS, the preferred confirmatory method. The results are presented in tables showing agreement across different concentration ranges (Negative, Low Negative, Near Cutoff Negative, Near Cutoff Positive, High Positive).
Example for FYL (Fentanyl) - Panel Dip, Operator 1:
- Negative (LC/MS 0): Device: 0 Positive, 1 Negative (1 discordant result here, sample 1484, LC/MS 0.78 ng/mL, Device: +)
- Low Negative (LC/MS +50%): Device: 20 Positive, 0 Negative
Lay-User Study (Selected data for AMP, FYL, NFYL):
This study evaluates the percentage of correct results when used by lay persons at various concentrations relative to the cutoff.
Example for AMP (Amphetamine):
- Negative (100% below cutoff): 100% correct (0 positive, 20 negative)
- Low Negative (-75% to -25% Cutoff): 100% correct negative for -75% and -50%, but 0 positive/20 negative for -25% cutoff.
- Positive (+25% to +75% Cutoff): Generally high correctness (95%-100%). For +25% cutoff, 95% correctness (19 positive, 1 negative).
| Drug (Identifier) | Cut-off Level | Reported Device Performance (Summary) |
|---|---|---|
| Fentanyl (FYL) | 1 ng/mL | Precision: At -100% to -50% cutoff, 100% negative calls (50-/0+ over 3 lots for Panel Dip & Quick Cup). At +25% to +100% cutoff, 100% positive calls (50+/0- over 3 lots for Panel Dip & Quick Cup). At cutoff, performance is mixed (e.g., Panel Dip Lot 1: 28+/22-). |
| Method Comparison: High concordance with LC/MS, especially for samples well above or below cutoff. Some discordant results near cutoff for both negative (e.g., sample 1484, LC/MS 0.78 ng/mL, device +) and positive (e.g., sample 5419, LC/MS 1.05 ng/mL, device -) as expected for qualitative tests. | ||
| Lay-User Study: All 20 negative samples at -100%, -75%, -50% cutoff were correctly identified as negative (100% correct). At -25% cutoff, 95% correct (19 negative, 1 positive). All 20 positive samples at +50%, +75% cutoff were correctly identified as positive (100% correct). At +25% cutoff, 100% correct (20 positive). | ||
| Norfentanyl (NFYL) | 5 ng/mL | Precision: At -100% to -50% cutoff, 100% negative calls (50-/0+ over 3 lots for Panel Dip & Quick Cup). At +25% to +100% cutoff, 100% positive calls (50+/0- over 3 lots for Panel Dip & Quick Cup). At cutoff, performance is mixed (e.g., Panel Dip Lot 1: 27+/23-). |
| Method Comparison: High concordance with LC/MS, with some discordance near cutoff (e.g., sample 4074, LC/MS 4.39 ng/mL, device +; sample 0687, LC/MS 5.05 ng/mL, device -). | ||
| Lay-User Study: All 20 negative samples at -100%, -75%, -50% cutoff were correctly identified as negative (100% correct). At -25% cutoff, 95% correct (19 negative, 1 positive). All 20 positive samples at +25%, +50%, +75% cutoff were correctly identified as positive (100% correct). | ||
| Amphetamine (AMP) | 500 ng/mL | Lay-User Study: All 20 negative samples at -100%, -75%, -50%, -25% cutoff were correctly identified as negative (100% correct). All 20 positive samples at +50%, +75% cutoff were correctly identified as positive (100% correct). At +25% cutoff, 95% correctness (19 positive, 1 negative). |
Detailed Study Information:
-
Sample sizes used for the test set and the data provenance:
- Precision Study: For Fentanyl and Norfentanyl, the reported data is for 3 lots, with 2 runs per day for 25 days, for 9 concentrations (e.g., -100% cutoff, -75% cutoff, etc.). This implies 50 tests per concentration per lot (2 runs * 25 days), leading to 450 tests per drug type per lot (9 concentrations * 50 tests), and 1350 tests per drug type across all 3 lots. The data provenance implies these samples were prepared by spiking known concentrations of drug into negative samples, indicating a controlled laboratory environment. Data for other analytes was "reported in K181768" (the predicate device documentation), so the exact sample sizes are not explicitly stated in this document but are assumed to be similar.
- Method Comparison Study: For Fentanyl and Norfentanyl, 80 unaltered clinical samples (40 negative and 40 positive based on LC/MS results) were used per drug. Each sample was tested by three laboratory assistants for each device type (Panel Dip and Quick Cup). This means 80 samples * 3 operators = 240 tests per drug for each device type. The data provenance is "in-house" and "unaltered clinical samples." The document does not specify the country of origin, but given the FDA submission, it's likely US-based or compliant with US standards. The study appears to be retrospective, using already collected clinical samples for comparison.
- Lay-user Study: 280 lay persons were used for each device format (Panel Dip and Quick Cup, though the results summarized apply to the overall device type). Urine samples were prepared at 7 different concentrations (negative, +/-25%, +/-50%, +/-75%, +/-100% of cutoff). Each participant received one blind-labeled sample and one device. Assuming each person tested one sample, this implies 280 samples were tested for each specific drug evaluated by lay-users on each format. The data provenance: "samples were prepared by spiking drugs into drug-free pooled urine specimens" and confirmed by LC/MS. This is a controlled experimental set-up rather than real-world patient samples.
-
Number of experts used to establish the ground truth for the test set and the qualifications of those experts:
- Precision Study: Ground truth was established by spiking known concentrations of drugs into negative samples and confirmed by LC/MS. No human experts were involved in establishing the ground truth directly for this part.
- Method Comparison Study: The ground truth was established by LC/MS (Liquid Chromatography-Mass Spectrometry), which is explicitly stated as the "preferred confirmatory method" and is considered a gold standard for drug detection and quantification in urine. No human expert readers established the ground truth; it was a laboratory instrument measurement. The study used three laboratory assistants to read the device results, but they were comparing their readings against the LC/MS truth, not establishing the truth themselves. Their qualifications are not specified beyond "laboratory assistants."
- Lay-user Study: The ground truth was established by spiking known concentrations of drugs confirmed by LC/MS. No human experts established the ground truth of the samples. The study assessed the lay-users' ability to interpret the device results against this known truth.
-
Adjudication method (e.g. 2+1, 3+1, none) for the test set:
- Precision Study: No adjudication method mentioned as samples were prepared with known concentrations.
- Method Comparison Study: No adjudication method was explicitly mentioned for the device results. Each of the three operators performed their own reads, and their individual results were compared to the LC/MS. Discordant results are noted for each operator.
- Lay-user Study: No adjudication method was mentioned. Each lay user tested one sample against a pre-defined truth.
-
If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:
- No, a multi-reader multi-case (MRMC) comparative effectiveness study was not conducted in this report. This device is a rapid diagnostic test (lateral flow immunoassay), not an AI-assisted diagnostic tool for interpretation of medical images or other complex data. Therefore, the concept of "human readers improve with AI vs without AI assistance" is not applicable here.
-
If a standalone (i.e. algorithm only without human-in-the-loop performance) was done:
- This is a lateral flow immunoassay, which is a physical diagnostic device producing a visual result (colored lines). It does not involve an "algorithm" in the sense of a software-based AI or computational algorithm. The device itself is the "standalone" diagnostic. Its performance characteristics (precision, specificity, interference) are essentially its "algorithm only" performance. The method comparison study is akin to assessing the device's standalone performance against a gold standard. The lay-user study assesses human interpretation of the device's standalone output.
-
The type of ground truth used (expert consensus, pathology, outcomes data, etc.):
- The primary ground truth for the analytical and method comparison studies was Liquid Chromatography-Mass Spectrometry (LC/MS), which is a highly accurate chemical method for detecting and quantifying substances.
- For the precision and lay-user studies, the ground truth was based on spiked urine samples with known drug concentrations, which were then confirmed by LC/MS.
-
The sample size for the training set:
- This document describes a 510(k) premarket notification for an in vitro diagnostic device (lateral flow immunoassay). Unlike AI/ML-driven devices that require extensive training data, such chemical-based devices are developed and optimized through chemical engineering and biological principles, not by "training" on datasets in the AI sense. Therefore, the concept of a "training set" with a statistical sample size as understood in machine learning is not applicable to this type of device. The development process involves chemical formulation and validation, not data training.
-
How the ground truth for the training set was established:
- Since the concept of a "training set" as it pertains to AI/ML devices is not applicable, the establishment of ground truth for a training set is also not relevant in this context. The "ground truth" for the performance evaluation of the device relied on LC/MS results and carefully prepared spiked samples with known drug concentrations.
§ 862.3100 Amphetamine test system.
(a)
Identification. An amphetamine test system is a device intended to measure amphetamine, a central nervous system stimulating drug, in plasma and urine. Measurements obtained by this device are used in the diagnosis and treatment of amphetamine use or overdose and in monitoring levels of amphetamine to ensure appropriate therapy.(b)
Classification. Class II (special controls). An amphetamine test system is not exempt if it is intended for any use other than employment or insurance testing or is intended for Federal drug testing programs. The device is exempt from the premarket notification procedures in subpart E of part 807 of this chapter subject to the limitations in § 862.9, provided the test system is intended for employment and insurance testing and includes a statement in the labeling that the device is intended solely for use in employment and insurance testing, and does not include devices intended for Federal drug testing programs (e.g., programs run by the Substance Abuse and Mental Health Services Administration (SAMHSA), the Department of Transportation (DOT), and the U.S. military).