(55 days)
SAFECARE® Multi-Drug Urine Test Dip Card is competitive binding, lateral flow immunochromatographic assays for qualitative and simultaneous detection of Amphetamine, Oxazepam, Marijuana, Methamphetamine, Morphine, Oxycodone, Secobarbital, Buprenorphine, Methylenedioxymethamine, Phencyclidine, Methadone, Nortriotyline and d-Propoxyphene in human urine at the cutoff concentrations of:
| Drug(Identifier) | Cut-off level |
|---|---|
| Amphetamine | 1000 ng/mL |
| Oxazepam | 300 ng/mL |
| Cocaine | 300 ng/mL |
| Marijuana | 50 ng/mL |
| Methamphetamine | 1000 ng/mL |
| Morphine | 2000 ng/mL |
| Oxycodone | 100 ng/mL |
| Secobarbital | 300 ng/mL |
| Buprenorphine | 10 ng/mL |
| Methylenedioxymethamphetamine | 500 ng/mL |
| Phencyclidine | 25 ng/mL |
| Methadone | 300 ng/mL |
| Nortriptyline | 1000 ng/mL |
| d-Propoxyphene | 300 ng/mL |
Configuration of SAFECARE® Multi-Drug Urine Test Dip Card can consist of any combination of the above listed drug analytes.
The test may yield positive results for the prescription drugs Buprenorphine. Oxazepam, Secobarbital. Propoxyphene and Oxycodone when taken at or above prescribed doses. It is not intended to distinguish between prescription use or abuse of these drugs. Clinical consideration and professional judgment should be exercised with any drug of abuse test result, particularly when the preliminary result is positive. The test provides only preliminary test results. A more specific alternative chemical must be used in order to obtain a confirmed analytical result. GCMS or LC/MS is the preferred confirmatory method.
The tests are intended for over-the-counter use.
SAFECARE® Multi-Drug Urine Test Cup is competitive binding, lateral flow immunichromatographic assays for qualitative and simultaneous detection of Amphetanine, Oxazepam, Methamphetamine, Morghine, Oxycodone, Secobarbital, Buprenorphine, Methylenedioxymethamphetamine, Phencyclidine, Methadone, Nortriptyline and d-Propoxyphene in human urine at the cutoff concentrations of:
| Drug(Identifier) | Cut-off level |
|---|---|
| Amphetamine | 1000 ng/mL |
| Oxazepam | 300 ng/mL |
| Cocaine | 300 ng/mL |
| Marijuana | 50 ng/mL |
| Methamphetamine | 1000 ng/mL |
| Morphine | 2000 ng/mL |
| Oxycodone | 100 ng/mL |
| Secobarbital | 300 ng/mL |
| Buprenorphine | 10 ng/mL |
| Methylenedioxymethamphetamine | 500 ng/mL |
| Phencyclidine | 25 ng/mL |
| Methadone | 300 ng/mL |
| Nortriptyline | 1000 ng/mL |
| d-Propoxyphene | 300 ng/mL |
Configuration of SAFECARE® Multi-Drug Urine Test Cup contination of the above listed drug analytes.
The test may yield positive results for the prescription drugs Buprenorphine. Oxazenam, Secobarbital, Propoxyphene and Oxycodone when taken at or above prescribed doses. It is not intended to distinguish between prescription use or abuse of these drugs. Clinical consideration and professional judgment should be exercised with any drug of abuse test result, particularly when the preliminary result is positive. The test provides only preliminary test results. A more specific alternative chemical method must be used in order to obtain a confirmed analytical result. GCMS or LC/MS is the preferred confirmatory method.
The tests are intended for over-the-counter use.
The SAFECARE Dip Card Tests and SAFECARE Cup Tests are immunochromatographic assays that use a lateral flow system for the qualitative detection of Amphetamine, Oxazepam, Cocaine, Cannabinoids, Methamphetamine, Morphine, Secobarbital, Methadone, Methylenedioxymethamphetamine, Oxycodone, Buprenorphine, Phencyclidine, Nortriptyline and Propoxyphene (target analytes) in human urine. The products are single-use in vitro diagnostic devices, which come in the formats of Dip Cards or Cups. Each test kit contains a Test Device (in one of the two formats), a package insert and a urine cup for sample collection. Each test device is sealed with a desiccant in an aluminum pouch.
The document describes the performance characteristics and studies for the SAFECARE Multi-Drug Urine Test Dip Card and SAFECARE Multi-Drug Urine Test Cup.
Here's an analysis of the acceptance criteria and study that proves the device meets them:
1. Table of Acceptance Criteria and Reported Device Performance
The document does not explicitly state "acceptance criteria" in a separate section. However, the performance data presented implies a very high standard for accuracy, particularly around the established cutoff concentrations. The "precision" studies demonstrate the device's ability to consistently produce correct results for samples at varying concentrations relative to the cutoff. The "comparison studies" compare the device's qualitative results to a quantitative gold standard (LC/MS) to establish concordance. The "lay-user study" assesses the ability of untrained individuals to correctly use and interpret the device.
Based on the provided data, the implied acceptance criteria for qualitative immunoassay devices for drug testing would typically be:
- 100% agreement for samples significantly below the cutoff (Negative).
- 100% agreement for samples significantly above the cutoff (Positive).
- High percentage agreement for samples near the cutoff (typically >90%).
For the Precision Study (Analytical Performance - "Test Set" Data):
The table below summarizes the reported performance for Methylenedioxymethamphetamine (MDMA), Oxycodone, Buprenorphine, Phencyclidine, Nortriptyline, and Propoxyphene (results from 3 lots, 50 tests per concentration per lot, per device type). The format is (Number of Negative Results)/(Number of Positive Results).
| Drug (Identifier) | Cut-off Level | Concentration | Reported Performance (Typical across 3 lots) | Implied Acceptance Criteria Met? |
|---|---|---|---|---|
| Methylenedioxymethamphetamine (MDMA) (Dip Card & Cup) | 500 ng/mL | -100% Cut-off | 50-/0+ | Yes (100% negative) |
| -75% Cut-off | 50-/0+ | Yes (100% negative) | ||
| -50% Cut-off | 50-/0+ | Yes (100% negative) | ||
| -25% Cut-off | 50-/0+ | Yes (100% negative) | ||
| Cut-off | 25-/25+ or 26-/24+ | Yes (near 50/50 split at cutoff) | ||
| +25% Cut-off | 50+/0- | Yes (100% positive) | ||
| +50% Cut-off | 50+/0- | Yes (100% positive) | ||
| +75% Cut-off | 50+/0- | Yes (100% positive) | ||
| +100% Cut-off | 50+/0- | Yes (100% positive) | ||
| Oxycodone (Dip Card & Cup) | 100 ng/mL | -100% Cut-off | 50-/0+ | Yes (100% negative) |
| -75% Cut-off | 50-/0+ | Yes (100% negative) | ||
| -50% Cut-off | 50-/0+ | Yes (100% negative) | ||
| -25% Cut-off | 50-/0+ | Yes (100% negative) | ||
| Cut-off | 24-/26+ or 26-/24+ or 25-/25+ | Yes (near 50/50 split at cutoff) | ||
| +25% Cut-off | 50+/0- | Yes (100% positive) | ||
| +50% Cut-off | 50+/0- | Yes (100% positive) | ||
| +75% Cut-off | 50+/0- | Yes (100% positive) | ||
| +100% Cut-off | 50+/0- | Yes (100% positive) | ||
| Buprenorphine (Dip Card & Cup) | 10 ng/mL | -100% Cut-off | 50-/0+ | Yes (100% negative) |
| -75% Cut-off | 50-/0+ | Yes (100% negative) | ||
| -50% Cut-off | 50-/0+ | Yes (100% negative) | ||
| -25% Cut-off | 50-/0+ | Yes (100% negative) | ||
| Cut-off | 27-/23+, 25-/25+, 24-/26+, 26-/24+ | Yes (near 50/50 split at cutoff) | ||
| +25% Cut-off | 50+/0- | Yes (100% positive) | ||
| +50% Cut-off | 50+/0- | Yes (100% positive) | ||
| +75% Cut-off | 50+/0- | Yes (100% positive) | ||
| +100% Cut-off | 50+/0- | Yes (100% positive) | ||
| Phencyclidine (Dip Card & Cup) | 25 ng/mL | -100% Cut-off | 50-/0+ | Yes (100% negative) |
| -75% Cut-off | 50-/0+ | Yes (100% negative) | ||
| -50% Cut-off | 50-/0+ | Yes (100% negative) | ||
| -25% Cut-off | 50-/0+ | Yes (100% negative) | ||
| Cut-off | 26-/24+, 25-/25+, 24-/26+ | Yes (near 50/50 split at cutoff) | ||
| +25% Cut-off | 50+/0- | Yes (100% positive) | ||
| +50% Cut-off | 50+/0- | Yes (100% positive) | ||
| +75% Cut-off | 50+/0- | Yes (100% positive) | ||
| +100% Cut-off | 50+/0- | Yes (100% positive) | ||
| Nortriptyline (Dip Card & Cup) | 1000 ng/mL | -100% Cut-off | 50-/0+ | Yes (100% negative) |
| -75% Cut-off | 50-/0+ | Yes (100% negative) | ||
| -50% Cut-off | 50-/0+ | Yes (100% negative) | ||
| -25% Cut-off | 50-/0+ | Yes (100% negative) | ||
| Cut-off | 25-/25+, 26-/24+, 24-/26+ | Yes (near 50/50 split at cutoff) | ||
| +25% Cut-off | 50+/0- | Yes (100% positive) | ||
| +50% Cut-off | 50+/0- | Yes (100% positive) | ||
| +75% Cut-off | 50+/0- | Yes (100% positive) | ||
| +100% Cut-off | 50+/0- | Yes (100% positive) | ||
| Propoxyphene (Dip Card & Cup) | 300 ng/mL | -100% Cut-off | 50-/0+ | Yes (100% negative) |
| -75% Cut-off | 50-/0+ | Yes (100% negative) | ||
| -50% Cut-off | 50-/0+ | Yes (100% negative) | ||
| -25% Cut-off | 50-/0+ | Yes (100% negative) | ||
| Cut-off | 23-/27+, 26-/24+, 24-/26+, 25-/25+ | Yes (near 50/50 split at cutoff) | ||
| +25% Cut-off | 50+/0- | Yes (100% positive) | ||
| +50% Cut-off | 50+/0- | Yes (100% positive) | ||
| +75% Cut-off | 50+/0- | Yes (100% positive) | ||
| +100% Cut-off | 50+/0- | Yes (100% positive) |
For the Lay-user Study:
The acceptance criteria for the lay-user study would typically be a high percentage of correct results, especially for samples clearly negative or positive, and a reasonable proportion around the cutoff. The document shows the "percentage of correct results (%)" for each concentration level. The implied acceptance criteria for OTC urine drug screens are usually very high concordance (e.g., >90%) with the ground truth for samples away from the cutoff, and some level of agreement for near-cutoff samples (where variability is expected). The device performance meets these generally implied high standards, especially for samples clearly below or above the cutoff (100% for most categories). For samples near +/-25% of cutoff, the accuracy ranges from 85-95%, which is typical for qualitative tests and acceptable for over-the-counter use where confirmation is recommended for positive results.
2. Sample Size Used for the Test Set and Data Provenance
- Precision Studies (Test Set):
- For each drug, device type (Dip Card/Cup), and each of the 9 concentration levels: 50 tests were performed per concentration.
- This was repeated for 3 different lots for each device type.
- Therefore, for a single drug and device type, the test set size for precision was: 9 concentrations * 50 tests/concentration * 3 lots = 1350 tests.
- Data Provenance: The samples were prepared by "spiking drug in negative samples" and confirmed by LC/MS. This suggests controlled, prospective laboratory-prepared samples rather than retrospective clinical samples from specific countries. The testing was "in-house."
- Comparison Studies (Test Set):
- For each drug and device type (Dip Card/Cup): 80 unaltered clinical samples (40 negative and 40 positive) were used.
- Data Provenance: "unaltered clinical samples." The document does not specify the country of origin, but given the manufacturer (China) and submission to US FDA, the samples could be from various global sources or acquired for the study. It's a retrospective analysis against independently confirmed samples.
- Lay-user Study (Test Set):
- For each device format (presumably Dip Card and Cup, though only combined results are shown in the given table): 300 lay persons participated.
- The total number of samples processed for each drug across the different concentration levels in the lay-user study tables is: 20 + 20 + 160 + 20 + 20 + 40 + 20 = 300 samples. (It seems 300 samples were tested for each drug type, probably with each of the 300 lay users testing one sample)
- Data Provenance: Urine samples were "prepared at the following concentrations; negative, +/-75%, +/-50%, +/-25% of the cutoff by spiking drugs into drug free-pooled urine specimens." These were laboratory-prepared samples. The study involved "three intended user sites."
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts
- Precision Studies: Ground truth was established by LC/MS (Liquid Chromatography-Mass Spectrometry), which is a definitive quantitative analytical method. It's an instrumental method, so human "experts" in interpretation are typically not specified in the same way as for imaging or clinical diagnosis. The samples were "blindly labeled by the person who prepared the samples," which indicates blinding to the LC/MS results during the device testing.
- Comparison Studies: Ground truth was established by LC/MS results. The document states, "The samples were blind labeled and compared to LC/MS results." These are definitive chemical analysis results.
- Lay-user Study: Ground truth for the samples was established by LC/MS confirmation of the spiked drug concentrations.
4. Adjudication Method for the Test Set
- Precision Studies: No explicit human adjudication method mentioned, as the device results are quantitative categories (positive/negative) and compared to definitive LC/MS values. The results are reported as counts (e.g., 25-/25+).
- Comparison Studies: The device results were read by "three laboratory assistants." For discordant results, the individual viewer's result is listed against the LC/MS truth, but no specific human adjudication process (e.g., 2+1 majority voting) is described for resolving discrepancies among the assistants or between the assistants and LC/MS. The LC/MS is the definitive ground truth here.
- Lay-user Study: The results are presented as aggregated counts of positive/negative readings by the lay users compared to the known ground truth (LC/MS confirmed concentrations). No human adjudication process is mentioned among the lay users.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance
- No, an MRMC comparative effectiveness study was not done. This type of study is relevant for AI-assisted diagnostic devices where human readers interpret images or complex data, and the AI serves as an aid.
- This device is a qualitative, over-the-counter urine drug test. Its performance is evaluated against chemical analytical ground truth, and its usability is assessed with lay users, not by evaluating how it assists expert readers.
6. If a Standalone (i.e. algorithm only without human-in-the loop performance) was done
- This device is a physical, qualitative immunoassay test kit, not a standalone algorithm/AI for analysis. The "performance" of the device itself (the reaction on the dip card/cup) is evaluated.
- However, the Comparison Studies and Precision Studies could be considered the "standalone" performance of the device when read by trained laboratory personnel (3 laboratory assistants), as it's the device's output itself being compared to LC/MS.
- The Lay-user study evaluates the device performance when interpreted by the intended "human-in-the-loop" (a lay user).
7. The Type of Ground Truth Used (expert consensus, pathology, outcomes data, etc.)
- The primary ground truth used for all performance studies (Precision, Comparison, Lay-user) is quantitative chemical analysis by LC/MS (Liquid Chromatography-Mass Spectrometry). This is a highly accurate and widely accepted method for confirming drug concentrations in urine.
- For the precision and lay-user studies, urine samples were spiked with known concentrations of drugs, and these concentrations were confirmed by LC/MS.
- For the comparison studies, "unaltered clinical samples" were used, with LC/MS providing the definitive truth for the presence and concentration of drugs.
8. The Sample Size for the Training Set
- This document describes premarket notification (510(k)) for a drug test kit, which is a physical diagnostic device based on immunochromatography, not an AI/Machine Learning algorithm.
- Therefore, there is no "training set" in the context of machine learning model development. The device's performance relies on its physical and chemical design, not on a trained algorithm.
9. How the Ground Truth for the Training Set Was Established
- As there is no AI/ML algorithm or training set, this question is not applicable.
§ 862.3610 Methamphetamine test system.
(a)
Identification. A methamphetamine test system is a device intended to measure methamphetamine, a central nervous system stimulating drug, in serum, plasma, and urine. Measurements obtained by this device are used in the diagnosis and treatment of methamphetamine use or overdose.(b)
Classification. Class II (special controls). A methamphetamine test system is not exempt if it is intended for any use other than employment or insurance testing or is intended for Federal drug testing programs. The device is exempt from the premarket notification procedures in subpart E of part 807 of this chapter subject to the limitations in § 862.9, provided the test system is intended for employment and insurance testing and includes a statement in the labeling that the device is intended solely for use in employment and insurance testing, and does not include devices intended for Federal drug testing programs (e.g., programs run by the Substance Abuse and Mental Health Services Administration (SAMHSA), the Department of Transportation (DOT), and the U.S. military).