(48 days)
TCPI's One Step™ Urine Drug of Abuse Barbiturate Test is a rapid, qualitative, competitive binding immunoassay for the determination of Barbiturates in urine at the cutoff level of 200 ng/ml. The test provides only preliminary data which should be confirmed by other methods such as gas chromatography/mass spectrophotometry (GC/MS). Clinical considerations and professional judgment should be applied to any drug of abuse test result, particularly when preliminary positive results are indicated. TCPI's One Step™ Urine Drug of Abuse: Barbiturate Test is not intended to monitor drug levels, but only to screen urines for the presence of Barbiturate and its metabolites.
TCPI's One Step™ Urine Drug of Abuse: Barbiturate Test consists of a chromatographic absorbent device in which the drug or drug metabolites in the sample compete with a drug conjugate immobilized on a porous membrane support for the limited antibody sites. As the test sample flows up through the absorbent device, the labeled antibody-dye conjugate binds to the free drug in the specimen forming an antibody:antigen complex. This complex competes with immobilized antigen conjugate in the positive reaction zone and will not produce a magenta color band when the drug is above the cutoff level of 200 ng/ml. Unbound dye conjugate binds to the reagent in the control zone, producing a magenta color band, demonstrating that the reagents and device are functioning correctly.
Since this refers to a 510(k) submission for a laboratory test, the term "device" refers to the "One Step™ Urine Drug of Abuse Barbiturate Test." The "algorithm" is the test itself, which is a qualitative competitive binding immunoassay.
Here's an analysis of the provided text in relation to your questions:
1. Table of Acceptance Criteria and Reported Device Performance
| Performance Metric | Acceptance Criteria (Implied) | Reported Device Performance |
|---|---|---|
| In-House Testing | ||
| Relative Sensitivity (Positives) | High agreement with predicate | 1.00 (100%) |
| Relative Specificity (Negatives) | High agreement with predicate | 1.00 (100%) |
| Accuracy | High agreement with predicate | 100% |
| Clinical Trial | ||
| Relative Sensitivity (Positives) | High agreement with GC/MS | 97.96% |
| Relative Specificity (Negatives) | High agreement with GC/MS | 100% |
| Accuracy | High agreement with GC/MS | 98.99% |
Note: The document doesn't explicitly state quantitative acceptance criteria prior to reporting results (e.g., "sensitivity must be >95%"). Instead, it presents the results as evidence of performance compared to established methods.
2. Sample Size Used for the Test Set and Data Provenance
- Test Set Sample Size:
- In-house testing: Not explicitly stated, but performed against Syva EMIT® II on an unspecified number of "samples documented to be positive by GC/MS."
- Clinical Trial: 296 samples.
- Data Provenance: The document does not specify the country of origin. It indicates the data is retrospective as the samples were collected and then tested. The samples were "documented to be positive by GC/MS" (for in-house testing) or "All positive samples by either screening method were confirmed by GC/MS" (for the clinical trial), implying these were pre-existing samples.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts
There were no human "experts" establishing the ground truth in the traditional sense of medical image interpretation (e.g., radiologists). The ground truth was established by Gas Chromatography/Mass Spectrometry (GC/MS), which is an analytical chemistry technique considered the "gold standard" for drug confirmation. Therefore, the "qualification" of the expert is the GC/MS instrument and the analytical chemists operating it.
4. Adjudication Method for the Test Set
Not applicable in the human expert sense. For the clinical trial, any sample that was positive by either the One Step™ test or the predicate screening method (Syva EMIT® II) was confirmed by GC/MS. This acts as a definitive adjudicator for discrepancies or initial positives.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done, If So, What Was the Effect Size of How Much Human Readers Improve with AI vs. Without AI Assistance
No, an MRMC study was not done. This device is an in-vitro diagnostic test (a chemical assay), not an AI-powered diagnostic imaging tool that would typically involve human readers (like radiologists) interpreting results with or without AI assistance. The "reader" is the test itself, which provides a qualitative "positive" or "negative" result.
6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) Was Done
Yes, the performance reported for the "One Step™ Urine Drug of Abuse: Barbiturate Test" is its standalone performance. This device is a rapid, qualitative immunoassay that produces a visual result (color band absence/presence), effectively acting as an "algorithm" (chemical reaction yielding a result) without direct human interpretation beyond reading the positive/negative indicator. Humans don't "interpret" the raw chemical reaction; they observe the final, designed output.
7. The Type of Ground Truth Used
The primary ground truth used was Gas Chromatography/Mass Spectrometry (GC/MS) confirmation. For the in-house testing, it was also compared against the Syva EMIT® II assay as a predicate, with GC/MS as a confirmatory "ground truth" for those initial positives.
8. The Sample Size for the Training Set
The document does not explicitly mention a "training set" in the context of machine learning. This is a chemical immunoassay, not an AI/ML model that undergoes a training phase with a distinct dataset. The development and optimization of the assay would involve various experiments, but these are not typically referred to as a "training set" in this context.
9. How the Ground Truth for the Training Set Was Established
As explained above, there isn't a "training set" in the AI/ML sense for this device. The development of the assay would have been based on established chemical principles for antibody-antigen binding and optimized for specificity and sensitivity to barbiturates, calibrated against known concentrations of barbiturates, likely using GC/MS or other established analytical methods to confirm concentrations of these known samples.
§ 862.3150 Barbiturate test system.
(a)
Identification. A barbiturate test system is a device intended to measure barbiturates, a class of hypnotic and sedative drugs, in serum, urine, and gastric contents. Measurements obtained by this device are used in the diagnosis and treatment of barbiturate use or overdose and in monitoring levels of barbiturate to ensure appropriate therapy.(b)
Classification. Class II (special controls). A barbiturate test system is not exempt if it is intended for any use other than employment or insurance testing or is intended for Federal drug testing programs. The device is exempt from the premarket notification procedures in subpart E of part 807 of this chapter subject to the limitations in § 862.9, provided the test system is intended for employment and insurance testing and includes a statement in the labeling that the device is intended solely for use in employment and insurance testing, and does not include devices intended for Federal drug testing programs (e.g., programs run by the Substance Abuse and Mental Health Services Administration (SAMHSA), the Department of Transportation (DOT), and the U.S. military).