(43 days)
Healgen MDMA (Ecstasy) Test is an immunochromatographic assay for the qualitative determination of Methylenedioxymethamphetamine in human urine at a Cut-Off concentration of 500 ng/mL. The test is available in a Strip format, a Cassette format, a Dip Card format and a Cup format.
The test provides only preliminary test results. A more specific alternative chemical method must be used in order to obtain a confirmed analytical result: GC/MS is the preferred confirmatory method. Clinical consideration and professional judgment should be exercised with any drug of abuse test result, particularly when the preliminary result is positive. For in vitro diagnostic use only. It is intended for prescription and for over-the-counter use.
Healgen Phencyclidine Test is an immunochromatographic assay for the qualitative determination of Phencyclidine in human urine at a Cut-Off concentration of 25 ng/mL. The test is available in a Strip format, a Cassette format, a Dip Card format and a Cup format.
The test provides only preliminary test results. A more specific alternative chemical method must be used in order to obtain a confirmed analytical result. GC/MS is the preferred confirmatory method. Clinical consideration and professional judgment should be exercised with any drug of abuse test result, particularly when the preliminary result is positive. For in vitro diagnostic use only. It is intended for prescription and for over-the-counter use.
Healgen MDMA (Ecstasy) Test and Healgen Phencyclidine Test are immunochromatographic assays for Methylenedioxymethamphetamine and Phencyclidine. Each assay test is a lateral flow system for the qualitative detection of Methylenedioxymethamphetamine and Phencyclidine (target analyte) in human urine. The products are in vitro diagnostic devices, which come in the form of: Strips, Cassettes, DipCards, or Cups. Each product contains a Test Device (in one of the four formats), and a package Each test device is sealed with a desiccant in an aluminum pouch. insert.
Here's a breakdown of the acceptance criteria and study information for the Healgen MDMA (Ecstasy) Test and Healgen Phencyclidine Test, based on the provided text:
1. Table of Acceptance Criteria and Reported Device Performance
The acceptance criteria are not explicitly stated in a single section as pass/fail thresholds. Instead, the performance characteristics studies demonstrate the device's accuracy and reliability against the established cut-off values as compared to GC/MS, which serves as the gold standard.
Healgen MDMA (Ecstasy) Test Performance Summary (Across all formats: Strip, Cassette, Cup, Dip Card)
| Acceptance Criteria (Implied) | Reported Device Performance (Summary from tables) |
|---|---|
| Precision: |
- Samples +25% Cut-off: Positive result
- Samples at Cut-off: Expected mix of positive/negative | MDMA (Ecstasy):
- Samples at -100%, -75%, -50%, -25% Cut-off: 50-/0+ (All negative)
- Samples at +25%, +50%, +75%, +100% Cut-off: 50+/0- (All positive)
- Samples at Cut-off: Varied (e.g., Strip, Cassette: 22-/28+; Dip Card: 24-/26+; Cup: 30-/20+) - Indicates appropriate sensitivity around the cut-off. |
| Cut-off Verification: - All samples at and above +25% cut-off: Positive
- All samples at and below -25% cut-off: Negative | MDMA (Ecstasy): Verified (all positive at/above +25% cut-off, all negative at/below -25% cut-off).
Phencyclidine: Verified (all positive at/above +25% cut-off, all negative at/below -25% cut-off). |
| Interference: No clinically significant interference from common substances at 100 µg/mL. | No differences observed for different formats. Numerous compounds listed showed no interference at 100 µg/mL. |
| Specificity: Cross-reactivity analysis as expected. | MDMA (Ecstasy): 3,4-Methylenedioxyamphetamine HCl (MDA) 17% cross-reactivity, 3,4-Methylenedioxyethylamphetamine (MDEA) 167% cross-reactivity. d-methamphetamine 20% cross-reactivity. d-, l-amphetamine, l-methamphetamine not detected at >100,000 ng/mL.
Phencyclidine: 4-Hydroxy Phencyclidine 28% cross-reactivity. |
| Effect of Urine Specific Gravity and pH: Accurate results across a range of specific gravities (1.000-1.035) and pH (4-9). | Results all positive for samples at and above +25% cut-off and all negative for samples at and below -25% Cut-Off. No differences observed for different formats. |
| Method Comparison (Professional User): High agreement with GC/MS. | Each format across MDMA and Phencyclidine tests showed high agreement with GC/MS, with discordant results primarily near the cut-off concentrations (as expected for qualitative tests). Example for MDMA Strip, Viewer A: 13/16 (81.25%) agreement at Near Cutoff Positive, 24/24 (100%) at High Positive. 10/10 (100%) at Negative, 15/15 (100%) at Low Negative. |
| Lay-User Performance: High percentage of correct results, easy-to-follow instructions. | MDMA (Ecstasy): - Samples at -100%, -75%, -50% Cut-off: 100% correct negative.
- Samples at -25% Cut-off: 95% correct negative.
- Samples at +50%, +75% Cut-off: 100% correct positive.
- Samples at +25% Cut-off: 95% correct positive.
Similar results for Phencyclidine with some formats showing 90% at -25% and +25% Cut-off.
All lay users indicated device instructions can be easily followed (Flesch-Kincaid Grade Level 7). |
Healgen Phencyclidine Test Performance Summary (Across all formats: Strip, Cassette, Cup, Dip Card)
| Acceptance Criteria (Implied) | Reported Device Performance (Summary from tables) |
|---|---|
| Precision: |
- Samples +25% Cut-off: Positive result
- Samples at Cut-off: Expected mix of positive/negative | Phencyclidine:
- Samples at -100%, -75%, -50%, -25% Cut-off: 50-/0+ (All negative)
- Samples at +25%, +50%, +75%, +100% Cut-off: 50+/0- (All positive)
- Samples at Cut-off: Varied (e.g., Strip: 20-/30+; Cassette: 18-/32+; Dip Card: 22-/28+; Cup: 16-/34+) - Indicates appropriate sensitivity around the cut-off. |
| Cut-off Verification: - All samples at and above +25% cut-off: Positive
- All samples at and below -25% cut-off: Negative | MDMA (Ecstasy): Verified (all positive at/above +25% cut-off, all negative at/below -25% cut-off).
Phencyclidine: Verified (all positive at/above +25% cut-off, all negative at/below -25% cut-off). |
| Interference: No clinically significant interference from common substances at 100 µg/mL. | No differences observed for different formats. Numerous compounds listed showed no interference at 100 µg/mL. |
| Specificity: Cross-reactivity analysis as expected. | MDMA (Ecstasy): 3,4-Methylenedioxyamphetamine HCl (MDA) 17% cross-reactivity, 3,4-Methylenedioxyethylamphetamine (MDEA) 167% cross-reactivity. d-methamphetamine 20% cross-reactivity. d-, l-amphetamine, l-methamphetamine not detected at >100,000 ng/mL.
Phencyclidine: 4-Hydroxy Phencyclidine 28% cross-reactivity. |
| Effect of Urine Specific Gravity and pH: Accurate results across a range of specific gravities (1.000-1.035) and pH (4-9). | Results all positive for samples at and above +25% cut-off and all negative for samples at and below -25% Cut-Off. No differences observed for different formats. |
| Method Comparison (Professional User): High agreement with GC/MS. | Each format across MDMA and Phencyclidine tests showed high agreement with GC/MS, with discordant results primarily near the cut-off concentrations (as expected for qualitative tests). Example for Phencyclidine Strip, Viewer A: 13/16 (81.25%) agreement at Near Cutoff Positive, 24/24 (100%) at High Positive. 10/10 (100%) at Negative, 15/15 (100%) at Low Negative. |
| Lay-User Performance: High percentage of correct results, easy-to-follow instructions. | MDMA (Ecstasy): - Samples at -100%, -75%, -50% Cut-off: 100% correct negative.
- Samples at -25% Cut-off: 95% correct negative.
- Samples at +50%, +75% Cut-off: 100% correct positive.
- Samples at +25% Cut-off: 95% correct positive.
Similar results for Phencyclidine with some formats showing 90% at -25% and +25% Cut-off.
All lay users indicated device instructions can be easily followed (Flesch-Kincaid Grade Level 7). |
2. Sample size used for the test set and the data provenance
-
Precision Study:
- Test set size: For each drug (MDMA/Phencyclidine) and each device format, 9 different concentration levels were tested. For each concentration level, 50 tests were performed (2 runs/day for 25 days). So, 9 concentrations * 50 tests/concentration = 450 tests per lot. Since there are 3 lots mentioned per format, this would be 3 lots * 450 tests/lot = 1350 tests per format for each drug.
- Data Provenance: Samples were prepared by spiking drug in negative urine samples, suggesting these are prospectively prepared, controlled samples. The text does not specify the country of origin, but the submission is to the FDA, implying studies likely conducted in or for the US market. The samples were confirmed by GC/MS.
-
Cut-off Verification Study:
- Test set size: 150 samples were tested for each drug, equally distributed at 5 concentrations (-50% cut-off, -25% cut-off, cut-off, +25% cut-off, +50% cut-off).
- Data Provenance: Samples were prepared by spiking drug in negative samples, indicating prospectively prepared, controlled samples.
-
Interference Study:
- Test set size: Not explicitly stated as a number of samples, but numerous potential interfering substances were added to drug-free urine and urine containing target drugs at 25% above cut-off levels. Each type of sample/interferent was tested using three batches of each device for all formats.
- Data Provenance: Prepared samples, suggesting prospectively prepared, controlled samples.
-
Specificity Study:
- Test set size: Not explicitly stated as a number of samples, but drug metabolites and other components were tested using three batches of each device for all formats.
- Data Provenance: Prepared samples, suggesting prospectively prepared, controlled samples.
-
Effect of Urine Specific Gravity and pH Study:
- Test set size: Urine samples across a range of specific gravities (1.000-1.035) and pH (4-9) were spiked with target drugs at 25% below and 25% above cut-off levels. These were tested using three batches of each device for all formats.
- Data Provenance: Prepared samples, suggesting prospectively prepared, controlled samples.
-
Comparison Studies (Professional Users):
- Test set size: For each drug (MDMA and Phencyclidine) and each format (Strip, Cassette, Cup, Dip Card), 80 unaltered clinical samples were used (40 negative and 40 positive). So, for MDMA, 4 formats * 80 samples/format = 320 samples. For Phencyclidine, 4 formats * 80 samples/format = 320 samples.
- Data Provenance: Unaltered clinical samples, which were blind labeled. The text does not specify the country of origin, but it is retrospective in nature given they are "unaltered clinical samples." (The samples were analyzed by GC/MS, meaning the true drug concentration was known retrospectively).
-
Lay-User Study:
- Test set size: 557 lay persons tested MDMA devices and 555 lay persons tested Phencyclidine devices. Total of 1112 individuals. For each drug, samples were prepared at 7 concentration levels (negative, +/-75%, +/-50%, +/-25% of cutoff). For each concentration, 19-20 samples were tested. Thus, for MDMA: (3 * 20 samples) + (1 * 19 samples for -50% cutoff) + (1 * 20 samples for -25% cutoff) + (1 * 20 samples for +25% cutoff) + (1 * 20 samples for +50% cutoff) + (1 * 19 samples for +75% cutoff) = ~139 unique samples tested across different concentrations for each format (though the table headers suggest 20 samples per concentration usually). The 557 and 555 refers to the number of users, implying each user got at least one sample.
- Data Provenance: Samples were prepared by spiking drugs into drug-free pooled urine specimens, making this a prospectively prepared, controlled sample study. The concentrations were confirmed by GC/MS. The study was performed at "three intended user sites," but country of origin is not specified.
3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts
-
Professional User Comparison Study:
- Ground Truth: Established by GC/MS (Gas Chromatography/Mass Spectrometry) results. GC/MS is a highly accurate analytical method, considered the gold standard for drug confirmation.
- Experts for Ground Truth: Not applicable in the traditional sense of human readers. The ground truth is established by a diagnostic laboratory method (GC/MS).
- Human Readers of the Device: Three "laboratory assistants" were used for each format. Their specific qualifications are not detailed beyond "laboratory assistants."
-
Precision, Cut-off, Interference, Specificity, Effect of Urine Specific Gravity and pH Studies:
- Ground Truth: Established primarily by GC/MS for confirmation of spiked drug concentrations.
- Experts for Ground Truth: Not applicable in the traditional sense.
-
Lay-User Study:
- Ground Truth: Established by GC/MS confirmation of the spiked urine sample concentrations.
- Experts for Ground Truth: Not applicable.
4. Adjudication method (e.g., 2+1, 3+1, none) for the test set
-
Precision, Cut-off, Interference, Specificity, Effect of Urine Specific Gravity and pH Studies:
- No adjudication method is described for interpreting the results of the device itself (e.g., if there were conflicting results). However, for the precision study, three different operators performed the tests, suggesting individual readings were recorded. The summary tables aggregate these results. The ground truth (spiked concentration) was confirmed by GC/MS prior to testing.
-
Comparison Studies (Professional Users):
- For the discordant results in the comparison studies (Table: Discordant Results of MDMA (Ecstasy) Strip, etc.), it shows individual "Viewer" results (Viewer A, B, C) compared to the GC/MS result. This implies no adjudication method was applied to the device readings to establish a single device result. Instead, the individual results of the three viewers were compared directly to the GC/MS ground truth. They are presenting individual agreement/disagreement.
-
Lay-User Study:
- Each lay person individually performed the test and interpreted the results. There was no adjudication method among lay users. Each lay user's result was compared to the GC/MS ground truth.
5. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance
- No, an MRMC comparative effectiveness study involving AI assistance was NOT done.
- This device is an in vitro diagnostic (IVD) test (immunochromatographic assay for drug detection in urine), not an AI-powered image analysis or diagnostic support system for human readers.
- The "Viewers" mentioned in the comparison studies are human laboratory assistants interpreting the visual lines on the test strips/cassettes, not radiologists or clinicians integrating AI outputs. Their performance alone, both individually and in aggregate, is being evaluated against the GC/MS gold standard.
6. If a standalone (i.e. algorithm only without human-in-the loop performance) was done
- Yes, in essence, the "device" itself is a standalone test.
- For an IVD like this, the "algorithm" is the biochemical reaction and visual detection mechanism. The testing (precision, cut-off, interference, specificity, specific gravity/pH) directly evaluates the device's performance in detecting the target analytes in urine samples. The "professional user" comparison and "lay user" study involve human interpretation, but the core performance characteristics are intrinsic to the device's design. There isn't a separate "algorithm" being evaluated beyond the physical test kit itself.
7. The type of ground truth used (expert consensus, pathology, outcomes data, etc.)
- The primary ground truth used for all performance studies (Precision, Cut-off, Interference, Specificity, Specific Gravity/pH Effect, Professional User Comparison, Lay-User Study) was Gas Chromatography/Mass Spectrometry (GC/MS).
- GC/MS is a highly reliable and recognized analytical method for confirming the presence and concentration of drugs in biological samples and is considered the gold standard for such applications.
8. The sample size for the training set
- There is no stated training set for this device as it is an immunochromatographic assay, not a machine learning or AI algorithm in the context of typical software device development. The device's performance is based on its biochemical design and manufacturing, not a learned model from a dataset.
9. How the ground truth for the training set was established
- Not applicable, as there is no training set for this type of IVD device.
§ 862.3610 Methamphetamine test system.
(a)
Identification. A methamphetamine test system is a device intended to measure methamphetamine, a central nervous system stimulating drug, in serum, plasma, and urine. Measurements obtained by this device are used in the diagnosis and treatment of methamphetamine use or overdose.(b)
Classification. Class II (special controls). A methamphetamine test system is not exempt if it is intended for any use other than employment or insurance testing or is intended for Federal drug testing programs. The device is exempt from the premarket notification procedures in subpart E of part 807 of this chapter subject to the limitations in § 862.9, provided the test system is intended for employment and insurance testing and includes a statement in the labeling that the device is intended solely for use in employment and insurance testing, and does not include devices intended for Federal drug testing programs (e.g., programs run by the Substance Abuse and Mental Health Services Administration (SAMHSA), the Department of Transportation (DOT), and the U.S. military).