(31 days)
First Sign™ Drug of Abuse Tests are immunochromatographic assays for the qualitative determination of Oxazepam , Methamphetamine, and Morphine in human urine at cut-off concentrations of 300 ng/mL, and 2000 ng/mL, respectively. The tests are available in a Cup format and a Dip Card format.
The tests may yield preliminary positive results even when prescription drug Oxazepam is ingested, at prescribed doses; it is not intended to distinguish between prescription use or abuse of this drug. There is no uniformly recognized cutoff concentration level for oxazepam in urine. The tests provide only preliminary test results. A more specific alternative chemical method must be used in order to obtain a confirmed analytical result. Gas Chromatography/Mass Spectrometry is the preferred confirmatory method. Clinical consideration and professional judgment should be exercised with any drug of abuse test result, particularly when the preliminary result is positive.
For in vitro diagnostic use only. The tests are intended for over-the-counter and for prescription use.
First Sign™ Drug of Abuse Tests are immunochromatographic assays. Each assay test is a lateral flow system for the qualitative detection of Oxazepam , Methamphetamine , and Morphine in human urine. The products are single-use in vitro diagnostic devices, which come in the formats of DipCards or Cups. Each test kit contains a Test Device (in one of the two formats), a package insert and a urine cup for sample collection. Each test device is sealed with a desiccant in an aluminum pouch.
Here's an analysis of the acceptance criteria and study detailed in the provided text:
1. Table of Acceptance Criteria and Reported Device Performance
The document does not explicitly state formal "acceptance criteria" in terms of specific quantitative benchmarks (e.g., "sensitivity must be >90%"). Instead, it describes performance characteristics and then presents the results of studies to demonstrate that the device performs acceptably. The implied acceptance criterion for these qualitative drug tests is that they generally agree with GC/MS results, especially at and significantly above/below cut-off values.
Based on the provided performance characteristics, here's a summary:
| Feature | Acceptance Criteria (Implied) | Reported Device Performance |
|---|---|---|
| Precision | Consistent results at various concentrations relative to the cut-off. | Oxazepam, Methamphetamine, Morphine (Dip/Cup): Samples at -100%, -75%, -50%, -25% cut-off showed 100% negative results. Samples at +25%, +50%, +75%, +100% cut-off showed 100% positive results (with very few exceptions noted at the exact cut-off concentration, e.g., 3-4/46-47 +/-). |
| Cut-off | Correct qualitative determination (positive/negative) around the defined cut-off. | Oxazepam (300 ng/mL), Methamphetamine (1000 ng/mL), Morphine (2000 ng/mL): All devices (all lots, all formats) were positive at and above +25% cut-off and negative at and below -25% cut-off. |
| Interference | No interference from common physiological substances at specified concentrations. | Numerous compounds (e.g., Acetamidophenol, Ibuprofen, Caffeine for all drugs; specific examples listed for each drug) showed no interference at 100 µg/mL. |
| Specificity | Cross-reactivity minimized for non-target drugs; appropriate reactivity for metabolites. | Oxazepam: Showed expected cross-reactivity with some benzodiazepine metabolites/analogs (e.g., Alprazolam 240%, Clonazepam 3%, Triazolam 12%). No detection of Methamphetamine or Morphine. |
| Methamphetamine: Showed cross-reactivity with some related compounds (e.g., (+/-)3,4-Methylenedioxy-n-ethylamphetamine (MDEA) 2%, (+/-)3,4-Methylenedioxymethamphetamine (MDMA) 13%, L-Methamphetamine 10%). No detection of Morphine or Oxazepam. | ||
| Morphine: Showed cross-reactivity with Codeine (200%), Ethylmorphine (357%), Hydrocodone (40%), Hydromorphone (27%), σ-Monoacetylmorphine (200%), Morphine 3-b-D-glucuronide (154%). No detection of Oxazepam or Methamphetamine. (Note: % cross-reactivity values are relative to the drug's own cut-off concentration). | ||
| Urine Specific Gravity & pH | Performance unaffected by normal variations in urine specific gravity and pH. | Results were all positive for samples at and above +25% cut-off and all negative for samples at and below -25% Cut-Off across a specific gravity range of 1.000-1.035 and a pH range of 4-9. |
| Method Comparison (Professional User) | High concordance with GC/MS results, especially for clearly positive/negative samples. | Oxazepam Dip/Cup: For 40 negative (incl. low and near cut-off) and 40 positive (incl. near cut-off and high) samples, there were very few discordant results (e.g., 2-4 negative calls for samples slightly above cut-off, or 1 negative call for a sample slightly above cut-off per viewer/format). Overall high agreement. |
| Methamphetamine Dip/Cup: Similar high concordance, with few discordant results (e.g., 1-2 negative calls for samples slightly above cut-off per viewer/format). | ||
| Morphine Dip/Cup: Similar high concordance, with few discordant results (e.g., 1 negative call for a sample slightly above cut-off per viewer/format). | ||
| Lay-user Study | High percentage of correct results by lay users, clear instructions. | Oxazepam (Dip/Cup): 100% correct for negative samples, 90-100% correct for positive samples, with minor discrepancies (-25% cutoff for cup, +25% cutoff for dip card). |
| Methamphetamine (Dip/Cup): 100% correct for negative samples, 95-100% correct for positive samples, with minor discrepancies (+25% cutoff). | ||
| Morphine (Dip/Cup): 95-100% correct for negative samples (one false positive at -25% Cutoff for dip card), 95-100% correct for positive samples (one false negative at +25% Cutoff for cup/dip card). | ||
| All lay users could easily follow instructions. |
2. Sample Size and Data Provenance (Test Set)
- Sample Size (Trained Professionals, Method Comparison):
- For each of the three drugs (Oxazepam, Methamphetamine, Morphine) and each format (Dip Card, Cup), 80 clinical samples were used.
- Total samples for method comparison: 3 drugs * 2 formats * 80 samples/drug/format = 480 samples.
- Breakdown of 80 samples: 10 Negative, 10 Low Negative, 20 Near Cutoff Negative, 15 Near Cutoff Positive, 25 High Positive.
- Sample Size (Lay User Study):
- 280 lay persons for Oxazepam devices.
- 280 lay persons for Methamphetamine devices.
- 280 lay persons for Morphine devices.
- Each lay person tested 1 blind labeled sample and a device.
- Total samples tested by lay users: Roughly 280 samples/drug * 3 drugs = 840 samples.
- Data Provenance (Method Comparison): "unaltered clinical samples" - implies retrospective collection, origin unknown based on the text.
- Data Provenance (Lay User Study): "Urine samples were prepared... by spiking drugs into drug free-pooled urine specimens." This indicates the samples were synthesized or spiked rather than naturally occurring clinical samples, and then blind-labeled.
3. Number of Experts and Qualifications for Ground Truth (Test Set)
- Number of Experts:
- Method Comparison (Professional User): "three different laboratory assistants for each format of the device." Total of 6 unique readers (3 for dip card, 3 for cup) if they were distinct, or 3 if the same 3 read both formats (the wording "Different set of operators tested each format" in the precision study suggests distinct operators, but here it states "for each format", which could mean the same set of 3 rotated). The data is presented as Viewer A, B, C for each.
- Lay User Study: 280 lay persons per drug. Not "experts" in the traditional sense, but the intended users.
- Qualifications of Experts: The "three different laboratory assistants" are not further qualified (e.g., radiologist with 10 years of experience).
4. Adjudication Method (Test Set)
- None specified for the professional user readings. The raw results from each "Viewer" (laboratory assistant) are presented individually, and then compared against the GC/MS result. They are not pooled or adjudicated to form a single "device" result.
- For the Lay User Study: There is no "adjudication" between lay users. Each lay user's individual result is recorded and compared to the GC/MS confirmed concentration.
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
- No MRMC comparative effectiveness study was done to measure human reader improvement with AI vs. without AI assistance. This device is a standalone qualitative diagnostic test (immunochromatographic assay), not an AI-assisted interpretation tool for human readers.
6. Standalone Performance (Algorithm Only)
- Yes, a standalone performance was done, but it's not an "algorithm" in the typical AI sense. The device itself (the immunochromatographic assay) is designed to provide a result (positive/negative) based on a chemical reaction, which can then be visually interpreted. The precision studies, cut-off studies, interference, specificity, and specific gravity/pH studies all demonstrate the standalone performance of the device without explicit human interpretation variability considered (though a human reads the test line).
- The "Method Comparison" and "Lay-user study" then introduce the human element (laboratory assistants and lay users, respectively) reading these standalone device results.
7. Type of Ground Truth Used (Test Set)
- Gas Chromatography/Mass Spectrometry (GC/MS) was explicitly used as the preferred confirmatory method for establishing ground truth for both the professional method comparison study and for confirming the concentrations of the spiked samples in the lay user study and precision/cut-off studies.
8. Sample Size for the Training Set
- The document describes performance characteristics and equivalence to a predicate device, but does not specify a separate "training set" for the development of the device itself or for any AI/algorithmic component (as there isn't one). The studies described are performance validation studies.
9. How the Ground Truth for the Training Set was Established
- Not applicable as no "training set" is mentioned in the context of device development. The ground truth for all performance evaluation studies (precision, cut-off, method comparison, lay user) was established using GC/MS.
§ 862.3610 Methamphetamine test system.
(a)
Identification. A methamphetamine test system is a device intended to measure methamphetamine, a central nervous system stimulating drug, in serum, plasma, and urine. Measurements obtained by this device are used in the diagnosis and treatment of methamphetamine use or overdose.(b)
Classification. Class II (special controls). A methamphetamine test system is not exempt if it is intended for any use other than employment or insurance testing or is intended for Federal drug testing programs. The device is exempt from the premarket notification procedures in subpart E of part 807 of this chapter subject to the limitations in § 862.9, provided the test system is intended for employment and insurance testing and includes a statement in the labeling that the device is intended solely for use in employment and insurance testing, and does not include devices intended for Federal drug testing programs (e.g., programs run by the Substance Abuse and Mental Health Services Administration (SAMHSA), the Department of Transportation (DOT), and the U.S. military).