(94 days)
Healgen THC One Step Marijuana Test is an immunochromatographic assay for the qualitative determination of 11-nor-A9-THC-9-COOH in human urine at a Cut-Off concentration of 50 ng/mL. The test is available in a Strip format, a Cassette format, a Dip Card format and a Cup format.
The test provides only preliminary test results. A more specific alternative chemical method must be used in order to obtain a confirmed analytical result. GC/MS is the preferred confirmatory method. Clinical consideration and professional judgment should be exercised with any drug of abuse test result, particularly when the preliminary result is positive.
For in vitro diagnostic use only. It is intended for over-the-counter and for prescription use.
Healgen mAMP One Step Methamphetamine Test is an immunochromatographic assay for the qualitative determination of methamphetamine in human urine at a Cut-Off concentration of 1000 ng/mL. The test is available in a Strip format, a Cassette format, a Dip Card format and a Cup format.
The test provides only preliminary test results. A more specific alternative chemical method must be used in order to obtain a confirmed analytical result. GC/MS is the preferred confirmatory method. Clinical consideration and professional judgment should be exercised with any drug of abuse test result, particularly when the preliminary result is positive. For in vitro diagnostic use only. It is intended for over-the-counter and for prescription use.
Immunochromatographic assays for Marijuana and Methamphetamine Urine Tests use a lateral flow, one step system for the qualitative detection of 11-nor-Δ9-THC-9-COOH and Methamphetamine (target analyte) in human urine. Each assay uses a monoclonal antibody-dye conjugate against drugs with gold chloride and fixed drug-protein conjugates and anti-mouse IgG polyclonal antibody in membranes.
The provided document describes the performance characteristics of the Healgen THC One Step Marijuana Test and the Healgen mAMP One Step Methamphetamine Test, and concludes that they are substantially equivalent to a predicate device.
Here's an analysis of the acceptance criteria and the studies that support it:
1. Table of Acceptance Criteria and Reported Device Performance
The acceptance criteria for these devices are implicitly defined by the results of the performance studies presented, primarily focused on their ability to correctly identify drug presence/absence at and around the specified cutoff concentrations. The studies demonstrate that for samples at or above +25% of the cutoff, the devices consistently show positive results, and for samples at or below -25% of the cutoff, they consistently show negative results. At the cutoff concentration, there is some variability, which is expected for qualitative tests.
Healgen THC One Step Marijuana Test (Cut-off: 50 ng/mL)
| Performance Characteristic | Acceptance Criteria (Implicit) | Reported Device Performance |
|---|---|---|
| Precision | Consistent results at defined concentrations (100% agreement for -100%/-75%/-50%, +25%/+50%/+75%/+100% cut off) | THC Strip Format: |
| -100%, -75%, -50% cut off: 50-/0+ (100% negative) | ||
| +25%, +50%, +75%, +100% cut off: 50+/0- (100% positive) | ||
| Cut off: 12-/38+ | ||
| THC Cassette Format: Similar results (e.g., 18-/32+ at cut off) | ||
| THC Cup Format: Similar results (e.g., 14-/36+ at cut off) | ||
| THC Dipcard Format: Similar results (e.g., 20-/30+ at cut off) | ||
| Cut-off Verification | All positive at and above +25% Cut-off; all negative at and below -25% Cut-off. | THC: All positive at and above +25% Cut-off; all negative at and below -25% Cut-off. |
| Interference | No interference from common physiological/pathological substances at specified concentrations. | No interference from a long list of compounds at 100 µg/mL. |
| Specificity | Predictable cross-reactivity with related compounds. | THC: 11-nor-Δ9-THC-9-COOH (100% at 50 ng/mL), Delta-9-THC (0.1% at 50,000 ng/mL), 11-nor-delta-9-THC-carboxyglucuronide (67% at 75 ng/mL), 11-Hydroxy-Δ9-THC (1% at 5,000 ng/mL), etc. |
| Effect of Urine Specific Gravity and pH | No impact on results at defined range. | All positive for samples at and above +25% Cut-Off and all negative for samples at and below -25% Cut-Off within pH 4-9 and specific gravity 1.000-1.035. |
| Method Comparison (Clinical Samples) | Agreement with GC/MS (concordance) | THC Strip, Cassette, Cup, Dip Card: High concordance for "Negative" and "High Positive" samples. Some discordant results (false negatives) for samples "Near Cutoff Positive" (52-53 ng/mL, i.e., slightly above the 50 ng/mL cutoff). |
| Lay User Study | High percentage of correct results, especially for samples further from the cutoff. | THC: 100% correct for -100%, -75%, -50%, +50%, +75% Cutoff samples. 95% correct for -25% Cutoff (1 false positive) and +25% Cutoff (1 false negative). |
Healgen mAMP One Step Methamphetamine Test (Cut-off: 1000 ng/mL)
| Performance Characteristic | Acceptance Criteria (Implicit) | Reported Device Performance |
|---|---|---|
| Precision | Consistent results at defined concentrations (100% agreement for -100%/-75%/-50%, +25%/+50%/+75%/+100% cut off) | MET Strip Format: |
| -100%, -75%, -50% cut off: 50-/0+ (100% negative) | ||
| +25%, +50%, +75%, +100% cut off: 50+/0- (100% positive) | ||
| Cut off: 14-/36+ | ||
| MET Cassette, Dip Card, Cup Formats: Similar results (e.g., 20-/30+ at cut off, 24-/26+ at cut off, 22-/28+ at cut off respectively) | ||
| Cut-off Verification | All positive at and above +25% Cut-off; all negative at and below -25% Cut-off. | MET: All positive at and above +25% Cut-off; all negative at and below -25% Cut-off. |
| Interference | No interference from common physiological/pathological substances at specified concentrations. | No interference from a long list of compounds at 100 µg/mL. |
| Specificity | Predictable cross-reactivity with related compounds. | MET: D-Methamphetamine (100% at 1000 ng/mL), MDEA (5% at 20,000 ng/mL), Procaine (1.7% at 60,000 ng/mL), MDMA (40% at 2500 ng/mL), Ephedrine (1% at 100,000 ng/mL), etc. |
| Effect of Urine Specific Gravity and pH | No impact on results at defined range. | All positive for samples at and above +25% Cut-Off and all negative for samples at and below -25% Cut-Off within pH 4-9 and specific gravity 1.000-1.035. |
| Method Comparison (Clinical Samples) | Agreement with GC/MS (concordance) | MET Strip, Cassette, Dip Card, Cup: High concordance for "Negative" and "High Positive" samples. Some discordant results (false negatives) for samples "Near Cutoff Positive" (1003-1008 ng/mL, i.e., slightly above the 1000 ng/mL cutoff). |
| Lay User Study | High percentage of correct results, especially for samples further from the cutoff. | MET: 100% correct for -100%, -75%, -50%, +50%, +75% Cutoff samples. 90% correct for -25% Cutoff (2 false positives) and 95% correct for +25% Cutoff (1 false negative). |
2. Sample Sizes and Data Provenance
- Precision Study: For each drug (THC and MET) and each format (Strip, Cassette, Cup, Dip Card), 8 concentrations (-100% cut off to +100% cut off) were tested. For each concentration, tests were performed two runs per day for 25 days. This implies 50 individual tests per concentration per lot, and 3 lots were used for each format.
- Example for THC Strip: 8 concentrations x 50 tests/concentration x 3 lots = 1200 tests.
- Data Provenance: The document does not explicitly state the country of origin but implies an in-house laboratory setting (referred to as "in-house" for comparison studies, and "prepared by spiking drug in negative samples" for precision, cut-off, and interference studies). The data is retrospective in the sense that samples were prepared and then tested. The urine samples were "negative samples" (for spiking) or "drug-free urine" (for interference), suggesting they were sourced specifically for testing purposes.
- Cut-off Verification Study: 150 samples were used, equally distributed at concentrations of -50%, -25%, Cut-Off, +25%, +50% Cut-Off. So, 30 samples per concentration.
- Data Provenance: Similar to precision, prepared by spiking.
- Interference Study: Urine samples (drug-free and spiked with target drugs +/-25% Cut-Off) were used for testing various interfering substances. The number of samples per substance is not specified, but it was tested using three batches of each device for all formats.
- Data Provenance: Assumed to be artificially created/spiked samples.
- Method Comparison Study: For each drug and each format, 80 unaltered clinical urine samples were used (40 negative, 40 positive).
- Data Provenance: The samples are described as "clinical samples," suggesting real-world patient samples. The country of origin is not specified, but the study was performed "in-house." This is retrospective data.
- Lay-user Study: 140 lay persons participated. Urine samples were prepared at 7 different concentrations (negative, +/-75%, +/-50%, +25% of cutoff). For each concentration, 20 samples were prepared.
- Data Provenance: Artificially prepared by spiking into drug-free pooled urine specimens.
3. Number of Experts and Qualifications for Ground Truth
- Precision, Cut-off, Interference, Specificity, Effect of Urine Specific Gravity and pH:
- The ground truth for these analytical performance studies was established by GC/MS (Gas Chromatography/Mass Spectrometry). This is a highly accurate and widely accepted gold standard method for quantifying drug concentrations in urine.
- The document states that "Each drug concentration was confirmed by GC/MS" and that the "concentrations of the samples were confirmed by GC/MS." No human experts are explicitly mentioned for establishing ground truth from GC/MS results, as GC/MS is an objective analytical method.
- Method Comparison Study:
- The ground truth for the 80 clinical samples was established by GC/MS results.
- No specific number or qualifications of human experts are mentioned for interpreting the GC/MS results or establishing the ground truth from them.
- Lay-user Study:
- Ground truth was established by GC/MS for the spiked urine samples.
4. Adjudication Method for the Test Set
- Precision, Cut-off, Interference, Specificity, Effect of Urine Specific Gravity and pH: For these analytical studies, GC/MS was the objective reference. The testing was done by unspecified personnel; "sample aliquots were blinded labeled by the person who prepared the samples and won't take part in the sample testing" (precision study). This suggests a blinding mechanism but no explicit multi-reader adjudication process for the actual device results.
- Method Comparison Study: Three laboratory assistants were the "operators" who ran the devices and presumably interpreted the results. The comparison was against GC/MS. The individual results of each viewer are reported, and then discordant results are listed. There is no explicit mention of an adjudication method (e.g., 2+1, 3+1 consensus among the three viewers) to arrive at a single device result per sample if their interpretations differed. Each viewer's interpretation is compared independently to the GC/MS ground truth.
- Lay-user Study: The study involved "140 lay persons" each testing "1 blind labeled sample and a device." The "Lay person results" table shows aggregate counts of positive/negative results. It does not describe an adjudication process for discordant interpretations among lay users on a single sample, as each lay user tested a unique sample (or set of samples for the overall study) and their individual interpretation was the data point.
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
No MRMC comparative effectiveness study was described. The studies focused on the performance of the device itself, observed by laboratory assistants (method comparison) or lay users (lay-user study), against an objective ground truth (GC/MS). There is no comparison of "human readers improve with AI vs without AI assistance" because the device is a simple, qualitative immunochromatographic assay, not an AI-powered diagnostic system requiring human interpretation of complex AI outputs.
6. Standalone Performance Study (Algorithm Only)
The device itself is a standalone, qualitative diagnostic test (immunochromatographic assay). All the performance studies (precision, cut-off, interference, specificity, method comparison, lay-user study) assess the standalone performance of the device without human interpretation being the primary variable. The "human-in-the-loop" component is essentially the reading and interpretation of the colored lines on the test strip as positive or negative, which is intrinsic to this type of device. There isn't an "algorithm" in the sense of a sophisticated computational model that could be evaluated separately from the physical test.
7. Type of Ground Truth Used
The primary and consistently used ground truth for all analytical and comparative studies was GC/MS (Gas Chromatography/Mass Spectrometry). This is an objective, highly accurate analytical method for drug concentration determination.
8. Sample Size for the Training Set
The document does not describe the development of an "algorithm" or "AI" system that would typically require a training set. This is an immunochromatographic assay, which relies on chemical and biological reactions rather than machine learning algorithms. Therefore, there is no "training set" in the context of AI. The product validation data described (precision, cut-off, interference, specificity, method comparison) serves to demonstrate the device's performance, not to "train" it.
9. How the Ground Truth for the Training Set Was Established
As there is no "training set" in the context of AI/algorithm development for this device, this question is not applicable. The device's inherent design and manufacturing processes are validated by the performance studies using GC/MS as the ground truth.
§ 862.3870 Cannabinoid test system.
(a)
Identification. A cannabinoid test system is a device intended to measure any of the cannabinoids, hallucinogenic compounds endogenous to marihuana, in serum, plasma, saliva, and urine. Cannabinoid compounds includedelta -9-tetrahydrocannabinol, cannabidiol, cannabinol, and cannabichromene. Measurements obtained by this device are used in the diagnosis and treatment of cannabinoid use or abuse and in monitoring levels of cannabinoids during clinical investigational use.(b)
Classification. Class II (special controls). A cannabinoid test system is not exempt if it is intended for any use other than employment or insurance testing or is intended for Federal drug testing programs. The device is exempt from the premarket notification procedures in subpart E of part 807 of this chapter subject to the limitations in § 862.9, provided the test system is intended for employment and insurance testing and includes a statement in the labeling that the device is intended solely for use in employment and insurance testing, and does not include devices intended for Federal drug testing programs (e.g., programs run by the Substance Abuse and Mental Health Services Administration (SAMHSA), the Department of Transportation (DOT), and the U.S. military).