(63 days)
The Healstone Accurate ™ Drug of Abuse Urine Test Cup is a rapid qualitative immunoassay. The device provides preliminary results for the detection of potential abuse of one or more drugs. This is not a screening device to monitor prescription medication. This drug test cup may contain any combination of the drug tests listed in the table above. This drug tests cup provides only a preliminary result. An alternative laboratory test must be used to confirm the results provided by this drug test. Gas chromatography/mass spectrometry (CCMS) is the preferred method confirmation test. For in vitro diagnostic use only.
The Healstone Accurate ™ Drug of Abuse Urine Test Cup Rx is a rapid qualitative immunoassay. The device provides preliminary results for the detection of potential abuse of one or more drugs. This is not a screening device to monitor prescription medication. This drug test cup may contain any combination of the drug tests listed in the table above. This drug tests cup provides only a preliminary result. An alternative laboratory test must be used to confirm the results provided by this drug test. Gas chromatography/mass spectrometry (GC/MS) is the preferred method confirmation test. It is intended for prescription use. For in vitro diagnostic use only.
The Healstone Accurate™ Drug of Abuse Urine Test Cup and Healstone Accurate™ Drug of Abuse Urine Test Cup Rx are rapid, single-use in vitro diagnostic devices. Each test kit contains a test device in one pouch. One pouch contains a test Accurate™ Cup and two desiccants, and a package insert. The Healstone Accurate™ Drug of Abuse Urine Test Cup is intended for over-the-counter use and the Healstone Accurate™ Drug of Abuse Urine Test Cup Rx is intended for prescription use.
The provided text describes the performance characteristics of the "Healstone Accurate™ Drug of Abuse Urine Test Cup" and "Healstone Accurate™ Drug of Abuse Urine Test Cup Rx", but does not specify explicit acceptance criteria in the form of numerical thresholds for sensitivity, specificity, accuracy, or other performance metrics. The tables presented show the reported device performance across various drug tests at different concentrations relative to the cutoff level.
Here's an attempt to extract the information requested, based on what is available in the document:
1. Table of Acceptance Criteria and Reported Device Performance
As noted above, no explicit acceptance criteria are stated in the document. However, the reported performance in both the precision studies and lay-user studies consistently shows a high level of agreement with expected outcomes at concentrations well below and well above the cutoff. At the cutoff concentration, the device exhibits approximately 50% positive and 50% negative results, which is a typical characteristic for qualitative immunoassays designed to detect substances at a specific threshold.
For the precision studies, for concentrations at -100%, -75%, -50% cut off and +50%, +75%, +100% cut off, the results were almost always 100% negative and 100% positive, respectively. For concentrations at -25% cut off, a small number of false positives (1-3 out of 50) were observed across different lots and drugs. For concentrations at +25% cut off, a small number of false negatives (1-3 out of 50) were observed. This indicates that the device has a narrow window around the cutoff where results can be variable, which is expected for such tests.
For the lay-user studies, the agreement percentages for concentrations clearly below (-100%, -75%, -50% cutoff) and clearly above (+50%, +75% cutoff) the cutoff are all 100%. At -25% cutoff, agreement is generally high (85-95% negative). At +25% cutoff, agreement is also high (85-95% positive).
Without explicit acceptance criteria, it's impossible to state whether these results "meet" them. However, they demonstrate consistent performance around the designed cutoff levels.
2. Sample Size and Data Provenance
-
Precision Test Set Sample Size: For each drug and each concentration level, 50 tests were performed (2 runs per day for 25 days). This was done for three different lots of test cups. So, for each drug and each concentration, the total number of tests across three lots is 50 * 3 = 150.
-
Comparison Studies (Professional Operator) Test Set Sample Size: For each drug, 80 unaltered urine samples were used (40 negative and 40 positive). This was tested by three operators.
-
Lay-user Study Test Set Sample Size:
- Configuration 1 (AMP 1000, MET 1000, MOP 300, COC 300): 140 participants (81 males, 59 females). Each participant tested one blind-labeled test solution. Given that there were 7 concentration levels tested per drug in the table, and 14 drugs are listed for configuration 1, this would imply (20 samples per concentration level, 7 concentration levels, for 14 drugs means) a total of 14 * 7 * 20 = 1960 samples for Configuration 1.
- Configuration 2 (AMP 500, MET 500, OPI 2000, COC 150): 140 participants (90 males, 50 females). Similar breakdown of sample numbers as Configuration 1.
- Total participants across both configurations = 280. Total samples tested would be approximately 1960 samples per configuration, totaling around 3920 samples.
-
Data Provenance: The document does not specify the country of origin for the data. The studies appear to be prospective as samples were prepared at specific concentrations and then blinded for testing.
3. Number of Experts and Qualifications for Ground Truth
- Comparison Studies (Professional Operator): The ground truth for the comparison studies was established using LC/MS or GC/MS results. These are laboratory-based, highly accurate analytical methods, widely considered the "gold standard" for drug concentration determination. The document does not specify the number of experts (e.g., lab technicians, chemists) that performed or interpreted the LC/MS/GC/MS results, nor their specific qualifications.
- Lay-user Study: The ground truth for the lay-user study was also based on the known concentrations established by LC/MS.
4. Adjudication Method for the Test Set
- Precision Studies: Not applicable. The results are reported directly (number of negative/positive results for the given concentration).
- Comparison Studies (Professional Operator): The results from the test device (Healstone Accurate™) were compared directly against the LC/MS or GC/MS results. Discordant results are noted for each drug and operator. This is a direct comparison rather than an adjudication among human readers.
- Lay-user Study: The lay users read the device results, and these were compared to the known concentrations (ground truth from LC/MS). The document does not describe any adjudication process for lay-user interpretations if they differed from a pre-determined "correct" visual interpretation.
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
- No MRMC comparative effectiveness study was done regarding human readers improving with AI vs without AI assistance. This device is a rapid qualitative immunoassay (a test strip/cup that shows lines), not an AI-powered diagnostic tool requiring human interpretation of AI-generated insights. The "lay-user study" involved human readers (lay individuals) interpreting the device directly, without AI assistance.
6. Standalone (Algorithm Only) Performance Study
- This device is a physical immunoassay test cup, not an algorithm. Therefore, a standalone algorithm-only performance study is not applicable. The performance described is the standalone performance of the physical device.
7. Type of Ground Truth Used
- Precision, Comparison, and Lay-user Studies: The ground truth was established by known concentrations of spiked drugs in urine samples, confirmed by LC/MS or GC/MS (laboratory-based analytical methods). LC/MS and GC/MS are considered definitive methods for identifying and quantifying substances.
8. Sample Size for the Training Set
- This information is not provided in the document. The document describes performance evaluation studies but does not detail the development or training of the immunoassay. Immunoassays are not "trained" in the same way machine learning algorithms are; their performance is inherent to their chemical and biological design.
9. How the Ground Truth for the Training Set Was Established
- As mentioned above, information on a "training set" and its ground truth is not provided in this document as it pertains to the development of an immunoassay, not an AI/ML model.
§ 862.3100 Amphetamine test system.
(a)
Identification. An amphetamine test system is a device intended to measure amphetamine, a central nervous system stimulating drug, in plasma and urine. Measurements obtained by this device are used in the diagnosis and treatment of amphetamine use or overdose and in monitoring levels of amphetamine to ensure appropriate therapy.(b)
Classification. Class II (special controls). An amphetamine test system is not exempt if it is intended for any use other than employment or insurance testing or is intended for Federal drug testing programs. The device is exempt from the premarket notification procedures in subpart E of part 807 of this chapter subject to the limitations in § 862.9, provided the test system is intended for employment and insurance testing and includes a statement in the labeling that the device is intended solely for use in employment and insurance testing, and does not include devices intended for Federal drug testing programs (e.g., programs run by the Substance Abuse and Mental Health Services Administration (SAMHSA), the Department of Transportation (DOT), and the U.S. military).