(243 days)
First Sign® Drug of Abuse Cup Test Marijuana is a qualitative lateral flow immunoassay intended for the detection of Marijuana in human urine at cut-off concentration of 20 ng/mL. The tests provide only preliminary test results. A more specific alternative method must be used to obtain a confirmed analytical result. Gas Chromatography/Mass Spectrometry is the preferred confirmatory method. Clinical consideration and professional judgment should be used when you get any drug of abuse test result. It should be used particularly when the preliminary result is positive. For in vitro diagnostic use only. The test is intended for prescription use.
First Sign® Drug of Abuse Dip Card Test Marijuana is a qualitative lateral flow immunoassay intended for the detection of Marijuana in human urine at cut-off concentration of 20 ng/mL. The tests provide only preliminary test results. A more specific alternative method must be used to obtain a confirmed analytical result. Gas Chromatography/Mass Spectrometry is the preferred confirmatory method. Clinical consideration and professional judgment should be used when you get any drug of abuse test result. It should be used particularly when the preliminary result is positive. For in vitro diagnostic use only. The test is intended for prescription use.
First Sign Multi-Drug Cup Test is a qualitative lateral flow immunoassay intended for the detection of Amphetamine, Cocaine, and Methamphetamine in human urine at cut-off concentrations of 500 ng/mL, and 500 ng/mL, respectively. The tests provide only preliminary test results. A more specific alternative method must be used to obtain a confirmed analytical result. Gas Chromatography/Mass Spectrometry is the preferred confirmatory method. Clinical consideration and professional judgment should be used when you get any drug of abuse test result. It should be used particularly when the preliminary result is positive. For in vitro diagnostic use only. The tests are intended for over-the-counter use.
First Sign® Multi-Drug Dip Card Test is a qualitative lateral flow immunoassay intended for the detection of Amphetamine, Cocaine, and Methamphetamine in human urine at cut-off concentrations of 500 ng/mL, and 500 ng/mL, respectively. The tests provide only preliminary test results. A more specific alternative method must be used to obtain a confirmed analytical result. Gas Chromatography/Mass Spectrometry is the preferred confirmatory method. Clinical consideration and professional judgment should be used when you get any drug of abuse test result. It should be used particularly when the preliminary result is positive. For in vitro diagnostic use only. The tests are intended for over-the-counter use.
First Sign Multi-Drug Cup Test, First Sign Multi-Drug Dip Card Test, First Sign Drug of Abuse Cup Test Marijuana, and First Sign Drug of Abuse Dip Card Marijuana are lateral flow, immunochromatographic assays. The First Sign Multi-Drug Cup Test and the First Sign Multi-Drug Dip Card Test are for the qualitative detection of Amphetamine, Cocaine, and Methamphetamine in human urine. First Sign Drug of Abuse Cup Test Marijuana, and First Sign Drug of Abuse Dip Card Marijuana are for the qualitative detection of Marijuana in human urine. The products are single-use in vitro diagnostic devices, which come in the formats of Dip Cards or Cups. Each test kit contains a Test Device (in one of the two formats), a package insert and a urine cup for sample collection. Each test device is sealed with a desiccant in an aluminum pouch.
The provided document describes the analytical and user performance of various First Sign® drug tests (Multi-Drug Cup Test, Multi-Drug Dip Card Test, Drug of Abuse Cup Test Marijuana, and Drug of Abuse Dip Card Test Marijuana). These are qualitative lateral flow immunoassays intended for the detection of Amphetamine, Cocaine, Methamphetamine, and Marijuana in human urine. The document does not describe an AI medical device, but rather an in-vitro diagnostic device. Therefore, many of the requested points related to AI/MRMC studies are not applicable.
Here's an attempt to extract relevant information and apply it to the closest possible concepts for an AI device's acceptance criteria and study, based on the provided content:
Device Description:
The devices are lateral flow, immunochromatographic assays for the qualitative detection of Amphetamine, Cocaine, and Methamphetamine (Multi-Drug tests) and Marijuana (Drug of Abuse tests) in human urine. They are single-use in vitro diagnostic devices available in Dip Card or Cup formats.
Acceptance Criteria and Device Performance (Interpreted for a diagnostic device):
The acceptance criteria for these devices would typically be established based on their ability to correctly identify positive and negative samples around the defined cut-off concentrations for each drug. The "Precision" and "Lay-user study" data provide the closest insight into this, demonstrating the device's ability to consistently provide correct results across various concentrations relative to the cut-off.
Since this is a qualitative test (positive/negative), the performance is measured by the percentage of correct results at different concentrations, especially near the cut-off.
Table of Acceptance Criteria (Inferred) and Reported Device Performance:
| Performance Metric (Inferred Acceptance Criteria) | Device Performance (Reported) |
|---|---|
| Precision (All tests, across 3 lots and 3 operators) | |
| -100% Cutoff (Negative Prediction) | 100% Negative (e.g., AMP Cup Lot 1: 50-/0+) |
| -75% Cutoff (Negative Prediction) | 100% Negative (e.g., AMP Cup Lot 1: 50-/0+) |
| -50% Cutoff (Negative Prediction) | 100% Negative (e.g., AMP Cup Lot 1: 50-/0+) |
| -25% Cutoff (Negative Prediction) | 100% Negative (e.g., AMP Cup Lot 1: 50-/0+) |
| Cut-off (Mixed Positive/Negative) | Varies (e.g., AMP Cup Lot 1: 3-/47+ (94% Positive)) |
| +25% Cutoff (Positive Prediction) | 100% Positive (e.g., AMP Cup Lot 1: 50+/0-) |
| +50% Cutoff (Positive Prediction) | 100% Positive (e.g., AMP Cup Lot 1: 50+/0-) |
| +75% Cutoff (Positive Prediction) | 100% Positive (e.g., AMP Cup Lot 1: 50+/0-) |
| +100% Cutoff (Positive Prediction) | 100% Positive (e.g., AMP Cup Lot 1: 50+/0-) |
| Lay-User Study (ACCURACY for over-the-counter and prescription use) | |
| -100% to -25% Cutoff (Negative Prediction) | Generally 100% correct negative results. Some exceptions (e.g., MET DipCard -25% Cutoff: 95% correct, MET Cup -25% Cutoff: 95% correct). |
| +25% to +100% Cutoff (Positive Prediction) | Generally 100% correct positive results. Some exceptions (e.g., COC DipCard +25% Cutoff: 90% correct, AMP Cup +25% Cutoff: 95% correct, MET Cup +25% Cutoff: 95% correct). |
| Effect of Urine Specific Gravity and pH | All positive for samples at and above +25% Cut-Off and all negative for samples at and below -25% Cut-Off. |
Breakdown of Requested Information (Applying to this non-AI device where possible):
-
A table of acceptance criteria and the reported device performance:
- Acceptance Criteria (Inferred): For a qualitative drug test, the acceptance criteria are generally that samples below the cut-off should test negative, and samples at or above the cut-off should test positive, with high accuracy at concentrations away from the cut-off, and acceptable accuracy around the cut-off. Specifically, the data shows an expectation of 100% correct results for samples significantly above or below the cut-off (e.g., +/- 50% to +/- 100%) and a tolerance for some misclassification directly at or very close to the cut-off.
- Reported Device Performance: See the table above and the detailed "Precision" and "Lay-user study" results within the document (pages 8-20). For instance, in the precision study, samples at the cut-off typically showed a mix of positive and negative results (e.g., 3-/47+ or 2-/48+ out of 50 tests), indicating the expected performance around the threshold. In the lay-user study, performance at +25% cutoff for COC DipCard was 90% correct, and for AMP Cup and MET Cup at +25% cutoff, it was 95% correct, with near-perfect accuracy further from the cutoff.
-
Sample sizes used for the test set and the data provenance:
- Precision Study:
- Sample Size: For each drug and each format (e.g., AMP Cup, COC Dip Card), there were 9 concentration levels tested (from -100% to +100% of cut-off). For each concentration, there were 2 runs per day for 25 days, totalling 50 tests per concentration per lot. Since three lots were tested, this amounts to 150 tests per concentration level per drug per format.
- Data Provenance: Samples were "prepared by spiking drug in negative urine samples." This suggests laboratory-controlled, prospectively prepared samples. The origin of the negative urine samples (e.g., country) is not specified.
- Method Comparison Studies (Clinical Samples):
- Sample Size: "80 (40 negative and 40 positive) unaltered clinical samples for each target drug" were used. This means 80 clinical samples per drug per format. (e.g., 80 for AMP Dip Card, 80 for AMP Cup, etc.).
- Data Provenance: "clinical samples." This implies retrospective (or collected for study), human biological samples. Country of origin is not specified but implied to be in the testing region (likely USA given the FDA submission).
- Lay-user Study:
- Sample Size: "320 lay persons testing the devices." Urine samples were prepared at 8 concentration levels.
- Data Provenance: "Urine samples were prepared... by spiking drugs into drug free-pooled urine specimens." This indicates laboratory-prepared, prospective samples.
- Precision Study:
-
Number of experts used to establish the ground truth for the test set and the qualifications of those experts:
- Ground Truth Establishment:
- For the Precision Study and Lay-user study, the ground truth was established by the precise concentrations of spiked drugs confirmed by GC/MS (Gas Chromatography/Mass Spectrometry). GC/MS is considered the "preferred confirmatory method" and the gold standard for drug detection and quantification in urine.
- For the Method Comparison Studies, the ground truth for clinical samples was established by GC/MS results.
- Experts and Qualifications: The document does not mention human experts establishing the ground truth for the test sets. The ground truth relies on the analytical accuracy of GC/MS. The "three different operators" (or "laboratory assistants") mentioned in the Precision and Method Comparison studies were performing the device tests, not establishing the ground truth. No specific qualifications for these operators are provided beyond their role.
- Ground Truth Establishment:
-
Adjudication method (e.g., 2+1, 3+1, none) for the test set:
- For the analytical and method comparison studies, the results of the device were compared directly to the GC/MS ground truth. There was no human adjudication process described for discrepancies in the test results themselves, as the GC/MS is considered definitive. The "discordant results" tables show instances where the viewer disagreed with GC/MS, but there's no mention of a re-adjudication of the GC/MS reference result itself.
-
If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:
- N/A. This document describes an in-vitro diagnostic device (drug test kit), not an AI medical device. Therefore, no MRMC study involving AI assistance for human readers was conducted or is applicable.
-
If a standalone (i.e. algorithm only without human-in-the-loop performance) was done:
- N/A. This is not an AI algorithm. The device itself is a standalone test kit that provides a visual qualitative result (lines appearing on the dip card/cup). The "lay-user study" demonstrates the performance of the device when interpreted by its intended users (without "human-in-the-loop performance" in the AI sense, but rather human-as-user interpretation). The "Method Comparison Studies" also represent standalone performance, where "viewers" (operators) interpret the device results against a gold standard.
-
The type of ground truth used (expert consensus, pathology, outcomes data, etc.):
- The primary ground truth used for all performance studies was Gas Chromatography/Mass Spectrometry (GC/MS) results, which is an analytical gold standard for quantifying drug concentrations in urine.
-
The sample size for the training set:
- N/A. This is not a machine learning/AI device, so there is no "training set" in the computational sense. The device's performance is based on its chemical and biological components, not trained data.
-
How the ground truth for the training set was established:
- N/A. As there is no training set for an AI model, this question is not applicable.
§ 862.3100 Amphetamine test system.
(a)
Identification. An amphetamine test system is a device intended to measure amphetamine, a central nervous system stimulating drug, in plasma and urine. Measurements obtained by this device are used in the diagnosis and treatment of amphetamine use or overdose and in monitoring levels of amphetamine to ensure appropriate therapy.(b)
Classification. Class II (special controls). An amphetamine test system is not exempt if it is intended for any use other than employment or insurance testing or is intended for Federal drug testing programs. The device is exempt from the premarket notification procedures in subpart E of part 807 of this chapter subject to the limitations in § 862.9, provided the test system is intended for employment and insurance testing and includes a statement in the labeling that the device is intended solely for use in employment and insurance testing, and does not include devices intended for Federal drug testing programs (e.g., programs run by the Substance Abuse and Mental Health Services Administration (SAMHSA), the Department of Transportation (DOT), and the U.S. military).