(229 days)
The Cozart RapiScan Oral Fluid Drug Test System - Opiates and Methadone is intended for Point of Care testing in a number of settings such as prescription workplace, drug dependency clinics, and criminal justice. It provides qualitative screening results for opiates and methadone in human oral fluid at a cut-off concentration equivalent to 40ng/mL and 30ng/mL respectively in neat oral fluid. The cozart collection system involves a 1:3 dilution of the oral fluid sample, this dilution factor is corrected for in the Cozart RapiScan Drug Test and therefore the remainder of this document will refer to the cut-offs as 13ng/mL for opiates and 10ng/ml for methadone.
This kit has to be used in conjunction with the Cozart RapiScan instrument. Please refer to the sampling and testing procedures in the instructions for use leaflet.
This assay is for professional use only and provides only a preliminary analytical test result. Clinical consideration and professional judgement must be applied to any drug of abuse test result, particularly in evaluating a preliminary positive result. In order to obtain a more confirmed analytical result a more specific alternative chemical method is needed. Gas Chromatography/Mass Spectrometry (GC/MS) is the preferred confirmatory method.
It is the responsibility of those organisations required to follow department of transportation (DOT) or the Substance Abuse and Mental Health Services Administration (SAMHSA) Workplace Drug Testing guidelines to determine that this product satisfies the criteria for workplace testing established under DOT and SAMHSA.
The Cozart RapiScan opiates and methadone test system is composed of three parts - the oral fluid collection device, the opiates and methadone test cartridge and the Cozart RapiScan instrument.
- Oral fluid Collection Device: composed of an oral fluid collector pad, a transport tube containing run buffer and a dispense filter tube.
- Opiate and Methadone Test Cartridge: composed of a white reaction strip with immobilized drug sites for opiates and methadone contained in a blue plastic casing. The cartridge also has a pad containing gold labeled anti-morphine and gold labeled anti-methadone antibodies. In addition, the cartridge has a built in control line of sheep anti-mouse IgG antibody.
- Cozart RapiScan Instrument: a battery-powered instrument whose function is to automate the analysis of the opiate/methadone test cartridge. It is a hand-held low voltage optical scanner.
Here's an analysis of the Cozart RapiScan Oral Fluid Drug Test System for Opiates and Methadone, based on the provided text:
Acceptance Criteria and Device Performance
The acceptance criteria for the Cozart RapiScan are primarily defined by its ability to accurately detect opiates and methadone in oral fluid at specified cut-off concentrations, as validated against GC/MS. The study demonstrates the device's performance in terms of sensitivity, specificity, precision, and stability under various conditions.
Table of Acceptance Criteria and Reported Device Performance
| Performance Metric | Acceptance Criteria (Implied) | Reported Device Performance |
|---|---|---|
| Opiates Detection | ||
| Cut-off Concentration | 40 ng/mL in neat oral fluid (equivalent to 13 ng/mL in diluted sample) | The device correctly identified opiates at this cut-off. |
| Positive Agreement | High agreement with GC/MS for positive samples | Out of 99 RapiScan positive samples for opiates, 93 were confirmed positive by GC/MS. |
| Negative Agreement | High agreement with GC/MS for negative samples | Out of 124 RapiScan negative samples for opiates, 123 were confirmed negative by GC/MS. |
| Near Cut-off Performance | Accurate detection for samples within +/- 50% of the cut-off concentration | 24 samples were within +/- 50% of the opiate cut-off (20-60ng/mL). The specific accuracy for these 24 samples is not explicitly stated in a consolidated manner, but the overall method comparison suggests acceptable performance. Precision studies indicated 40/40 positive and 0/40 negative for 13 ng/mL in laboratory settings. For point of care settings, 25/25 positive and 1/25 negative for 13 ng/ml. |
| Methadone Detection | ||
| Cut-off Concentration | 30 ng/mL in neat oral fluid (equivalent to 10 ng/mL in diluted sample) | The device correctly identified methadone at this cut-off. |
| Positive Agreement | High agreement with GC/MS for positive samples | Out of 116 RapiScan positive samples for methadone, 116 were confirmed positive by GC/MS. |
| Negative Agreement | High agreement with GC/MS for negative samples | Out of 115 RapiScan negative samples for methadone, 115 were confirmed negative by GC/MS. |
| Near Cut-off Performance | Accurate detection for samples within +/- 50% of the cut-off concentration | 24 samples were within +/- 50% of the methadone cut-off (15-45ng/mL). The specific accuracy for these 24 samples is not explicitly stated in a consolidated manner, but the overall method comparison suggests acceptable performance. Precision studies indicated 40/40 positive and 0/40 negative for 10 ng/mL in laboratory settings. For point of care settings, 24/25 positive and 2/26 negative for 10 ng/ml. |
| Sensitivity | Detect drugs at or below 25-50% of the cut-off (implied by experiment design) | Opiates: detection limit between 9.75ng/mL (-25% of cutoff) and 6.5ng/mL (-50% of cutoff). Methadone: detection limit between 7.5ng/mL (-25% of cutoff) and 5ng/mL (-50% of cutoff). |
| Specificity | Minimal cross-reactivity with common interfering substances and related compounds. | 28 potentially interfering substances tested at 100,000ng/mL (Buprenorphine at 10,000ng/mL) showed no cross-reactivity. Lowest concentration to produce a positive was determined for 13 opiate family members and 4 methadone-related substances across three different batches, demonstrating appropriate specificity for these related compounds. |
| Precision | Consistent results over time and across different operators, especially near the cut-off. | Laboratory: For opiates, spiked samples at 13 ng/mL (cutoff) showed 40/40 positive. For methadone, spiked samples at 10 ng/mL (cutoff) showed 40/40 positive. Consistent results for samples at 0 ng/mL, 6.5 ng/mL (19.5 ng/mL) for opiates and 0 ng/mL, 5 ng/mL (15 ng/mL) for methadone. Point-of-Care: Opiate: 25/25 for 13 ng/mL, 1/25 negative for 13 ng/mL. Methadone: 24/25 for 10 ng/mL, 2/26 negative for 10 ng/mL. This suggests good precision even in varied settings. |
| Sample Stability | Samples remain stable for accurate testing for a reasonable period. | Stable at both room temperature and 2-8°C for 28 days. |
| Freeze/Thaw Stability | Samples tolerate multiple freeze/thaw cycles without affecting results. | Samples can undergo up to six freeze/thaw cycles at -20°C prior to testing. |
| Interference | No interference from common oral fluid contaminants or patient activities. | No interference observed from sample adequacy indicator dye, hemoglobin, smoking, coffee, tea, water, food, orange juice, hard candy, chewing gum, and mouthwash. |
| Time of Day/Instrument/Operator Comparison | Results should be consistent regardless of time of day, instrument used, or operator. | Sampling at different times of day (morning, midday, evening after fortification at +/- 50% cutoff) had no effect. All three instruments produced unambiguous results. Inexperienced operators yielded unambiguous results for all three operators. |
| Accuracy/Recovery | Accurate detection of fortified samples at various concentrations around the cut-off. | All samples fortified at or above the cut-off concentration (0, 6.5, 13, 19.5 ng/mL for morphine and 0, 5, 10, 15 ng/mL for methadone) were positive, with the exception of the negative sample fortified at 50% below the cut-off as expected, indicating accurate recovery and detection across the relevant range. |
| Cut-off Concentration Effectiveness | Clear discrimination at the established cut-off. | All samples fortified at 50% below the cut-off were negative. All samples fortified at the cut-off, 50% above, and 100% above the cut-off were positive for both opiates and methadone. |
Study Details
-
Sample sizes used for the test set and the data provenance:
- Method Comparison Test Set:
- Total Samples: 454 oral fluid samples.
- Opiates: 99 RapiScan positive (93 GC/MS confirmed), 124 RapiScan negative (123 GC/MS confirmed). (Total: 223 samples for opiates analysis against GC/MS)
- Methadone: 116 RapiScan positive (116 GC/MS confirmed), 115 RapiScan negative (115 GC/MS confirmed). (Total: 231 samples for methadone analysis against GC/MS)
- Special Samples:
- 24 opiate samples within +/- 50% of cut-off (14 previously diluted with blank saliva).
- 24 methadone samples within +/- 50% of cut-off (20 previously diluted with blank saliva).
- Provenance: All samples were oral fluid.
- 99 opiate-positive samples and 116 methadone-positive samples were from "drug users enrolled at a drug rehabilitation clinic."
- 103 opiate-negative samples and 102 methadone-negative samples were from "drug free volunteers."
- 21 opiate-negative samples and 13 methadone-negative samples were from "known drug users."
- The samples were tested by Cozart Bioscience Ltd staff, and GC/MS performed by FSS Chepstow UK, University of Glasgow, Scotland, and The Analytical Laboratory at Cozart Bioscience Ltd, UK. This suggests a retrospective collection of samples from clinics and volunteers, followed by prospective testing with the RapiScan and GC/MS. The origin is primarily the UK.
- Precision Test Set (Laboratory): Four oral fluid samples (negative, cutoff, 50% above cutoff, 50% below cutoff) tested in duplicate every day for 20 days (4 samples * 2 duplicates * 20 days = 160 tests per drug).
- Precision Test Set (Point of Care - POC): Four oral fluid samples (spiked at 0, 6.5, 13, 19.5 ng/mL for opiates and 0, 5, 10, 15 ng/mL for methadone) from drug-free volunteers, tested up to 9 times at 3 POC sites.
- Sensitivity Test Set: Eight negative oral fluid samples, fortified at 50% above cut-off and serially diluted.
- Specificity Test Set: Not explicitly stated, but 28 potentially interfering substances and 13 opiate/4 methadone related substances were tested.
- Cutoff Concentration Test Set: 8 fortified oral fluid samples at various concentrations (cutoff, +/- 50% cutoff, +/- 100% cutoff).
- Interference Studies Test Set: Not explicitly stated, but various parameters (dye, hemoglobin, common consumables) were tested.
- Stability Studies Test Set: Four fortified samples (2 at 50% above cutoff, 2 at 50% below cutoff) and two negative samples from volunteers.
- Time of Day Comparison Test Set: Samples from 4 volunteers at 3 different times of day, fortified.
- Instrument Comparison Test Set: Samples from 4 volunteers, fortified, tested on 3 different instruments.
- Operator Comparison Test Set: Samples from 4 volunteers, fortified, tested by "inexperienced operators" (number not specified, but likely at least 3 given the results statement mentions "all three operators").
- Method Comparison Test Set:
-
Number of experts used to establish the ground truth for the test set and the qualifications of those experts:
- The ground truth for the test set (method comparison) was established by Gas Chromatography/Mass Spectrometry (GC/MS). While GC/MS is an analytical method, not human experts, it is considered the "preferred confirmatory method" and gold standard for drug testing. The performing laboratories were FSS Chepstow UK, University of Glasgow, Scotland, and The Analytical Laboratory at Cozart Bioscience Ltd, UK. The qualifications of the GC/MS operators are implied to be standard for analytical laboratory personnel competent in this method.
-
Adjudication method for the test set:
- Not applicable in the typical sense for medical imaging or clinical decision-making. The "adjudication" here is the direct comparison of the Cozart RapiScan qualitative result (positive/negative) against the quantitative GC/MS result, using the defined cut-off concentrations. Discrepancies would be identified based on whether the RapiScan result matched the GC/MS result relative to the cut-off.
-
If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:
- No, a multi-reader multi-case (MRMC) comparative effectiveness study was not done. This device is a point-of-care test read by an instrument, not by human readers interpreting complex images or data. The instrument automatically displays results as "positive or negative," eliminating human subjectivity in reading.
-
If a standalone (i.e. algorithm only without human-in-the-loop performance) was done:
- Yes, the performance study described is essentially a standalone algorithm performance study. The Cozart RapiScan instrument, which incorporates the device's "algorithm" for reading the test cartridge, operates without human interpretation of the test line. The human interaction involves initiating the test and reading the displayed digital result (positive or negative). The study evaluates the objective output of this system compared to the gold standard. The note that the "Cozart RapiScan instrument must be used to read the cartridges, this eliminates the subjectivity of reading by eye" further supports this being a standalone evaluation of the device's automated reading.
-
The type of ground truth used (expert consensus, pathology, outcomes data, etc.):
- The ground truth used was Gas Chromatography/Mass Spectrometry (GC/MS), which is the "preferred confirmatory method" and considered the analytical gold standard for drug detection and quantification in biological samples.
-
The sample size for the training set:
- The document does not explicitly state a separate training set size. The studies described are performance validation studies. It's common for such devices (especially those using immunoassay principles rather than complex machine learning algorithms) not to have a distinct "training set" in the sense of AI/ML models. The design and optimization of the immunoassay itself (e.g., antibody selection, cut-off determination) would be based on internal development and verification processes that precede the formal validation studies presented here.
-
How the ground truth for the training set was established:
- As no explicit training set is mentioned in the context of advanced machine learning, the concept of establishing ground truth for a training set in the typical AI/ML sense is not directly applicable. For the development of the immunoassay, the "ground truth" for calibrating the system and establishing the cut-off concentrations would likely have involved controlled samples with known concentrations of opiates and methadone, confirmed by analytical methods like GC/MS during the R&D phase of the test's design. The "Cutoff Concentration" study described within the document is part of the verification of these established cutoffs.
§ 862.3650 Opiate test system.
(a)
Identification. An opiate test system is a device intended to measure any of the addictive narcotic pain-relieving opiate drugs in blood, serum, urine, gastric contents, and saliva. An opiate is any natural or synthetic drug that has morphine-like pharmocological actions. The opiates include drugs such as morphine, morphine glucoronide, heroin, codeine, nalorphine, and meperedine. Measurements obtained by this device are used in the diagnosis and treatment of opiate use or overdose and in monitoring the levels of opiate administration to ensure appropriate therapy.(b)
Classification. Class II (special controls). An opiate test system is not exempt if it is intended for any use other than employment or insurance testing or is intended for Federal drug testing programs. The device is exempt from the premarket notification procedures in subpart E of part 807 of this chapter subject to the limitations in § 862.9, provided the test system is intended for employment and insurance testing and includes a statement in the labeling that the device is intended solely for use in employment and insurance testing, and does not include devices intended for Federal drug testing programs (e.g., programs run by the Substance Abuse and Mental Health Services Administration (SAMHSA), the Department of Transportation (DOT), and the U.S. military).