Search Results
Found 6 results
510(k) Data Aggregation
(188 days)
The Cozart® EIA Amphetamines Oral Fluid Microplate Kit is intended for use in clinical and forensic laboratories when used in conjunction with the Cozart® RapiScan Oral Fluid collection system. Using this collection system it provides qualitative screening results for amphetamines in human oral fluid at a cutoff concentration of 15ng/ml. This is equal to 45ng/ml in neat oral fluid as the collection system involves a 1:3 dilution of the sample.
This assay is for professional use only and provides only a preliminary analytical test result. Clinical consideration and professional judgement must be applied to any drug of abuse test result, particularly in evaluating a preliminary positive result. In order to obtain a more confirmed analytical result a more specific alternative chemical method is needed. Gas Chromatography/Mass Spectrometry (GCMS) is the preferred confirmatory method.
The Cozart® EIA Amphetamines Oral Fluid Microplate Kit is a laboratory based test for the detection of amphetamines in human oral fluid using a cutoff equivalent to 45ng/mL. The device detailed above was compared to Gas Chromatography/Mass Spectrometry (GC/MS).
As can be seen from the Principle of the Test section in the pack insert, the Cozart® EIA Amphetamines Oral Fluid Microplate Kit is a competitive ELISA for the detection of amphetamines in human oral fluid.
The Cozart® EIA Amphetamines Oral Fluid Microplate kit supplies the following reagents - a microtitre plate coated with antibody, enzyme conjugate reagent, wash buffer, substrate solution, stop solution and four calibrators (0, 2, 15 and 50ng/ml amphetamine in oral fluid matrix).
Here's an analysis of the provided text, outlining the acceptance criteria and the study data, as requested:
Acceptance Criteria and Device Performance Study for Cozart® EIA Amphetamines Oral Fluid Microplate Kit
1. Table of Acceptance Criteria and Reported Device Performance
The acceptance criteria are not explicitly stated as distinct thresholds in the provided document. Instead, the document presents comparison data against a "gold standard" (GC/MS) and a predicate device. Based on the performance reported, an implicit acceptance criterion can be inferred as achieving a high agreement with GC/MS. The precision (CV%) is explicitly stated as acceptable if below 10%.
| Parameter | Acceptance Criteria (Inferred/Explicit) | Reported Device Performance (Cozart® EIA Amphetamines Oral Fluid Microplate Kit) |
|---|---|---|
| Overall Agreement with GC/MS | High agreement with GC/MS (e.g., comparable to or better than predicate) | 96% |
| Precision (CV%) | < 10% (for qualitative manual ELISA assay) | 2.7-12.4% (total precision <13%) for the kit; <11% for precision study |
| Sensitivity | Sufficiently low to detect amphetamines at specified cutoff (45ng/ml or 15ng/ml diluted) | 1.2 ng/mL |
| Specificity (Cross-reactivity) | No cross-reactivity with common interferents; well-defined cross-reactivity with related compounds | 20 potential interferents tested - none cross-reacted. 7 of 11 related compounds showed cross-reactivity. |
| Cutoff Concentration Validation | Samples at -50% and +50% of cutoff are correctly classified. | 7.5ng/ml absorbances higher than 15ng/ml calibrator; 22.5ng/ml absorbances lower than 15ng/ml calibrator. |
| Interference | No interference from common oral fluid components/ingestions. | No interference observed from various parameters (alcohol, smoking, coffee, food, etc.). |
| Stopped Assay Stability | Absorbance readable within a specific timeframe. | Must be read within 15 minutes at 450nm. |
| Assay Drift | Limited change in results across sample addition times. | Sample addition must take place within 25 minutes. |
2. Sample Size Used for the Test Set and Data Provenance
- Sample Size for Test Set: 163 samples.
- Data Provenance: Not explicitly stated regarding country of origin or whether the data was retrospective or prospective. However, the context suggests these were human oral fluid samples collected for "clinical and forensic samples" for the purpose of this study.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts
Not applicable. The ground truth was established by Gas Chromatography/Mass Spectrometry (GC/MS), which is an analytical chemical method, not human expert consensus for this type of test.
4. Adjudication Method for the Test Set
Not applicable, as the ground truth was established by GC/MS, an objective chemical analysis method, not by human experts requiring adjudication.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance
Not applicable. This device is an immunoassay (ELISA) for detecting amphetamines. It is a laboratory-based automated or semi-automated test, not a medical imaging or diagnostic device requiring human reader interpretation, nor does it involve AI assistance in the context of human reader improvement.
6. If a Standalone (i.e. algorithm only without human-in-the-loop performance) was done
Yes, the performance study describes the standalone performance of the Cozart® EIA Amphetamines Oral Fluid Microplate Kit. The device (kit) was tested independently, and its results were compared directly against GC/MS. There is no human-in-the-loop component mentioned that would alter the device's output.
7. The Type of Ground Truth Used
The ground truth used was Gas Chromatography/Mass Spectrometry (GC/MS). The document explicitly states: "Gas Chromatography/Mass Spectrometry (GC/MS) is the preferred confirmatory method." and "The Cozart® EIA Amphetamines Oral Fluid Microplate Kit was compared to Gas Chromatography/Mass Spectrometry (GC/MS)."
8. The Sample Size for the Training Set
The document does not provide information about a separate "training set" or its size. The 163 samples described were used for the performance comparison study, which assesses the device's accuracy against the ground truth. As this is an immunoassay kit, not a machine learning algorithm that typically requires a large training set, the concept of a training set in this context is less relevant in the same way it would be for AI/ML device submissions. The kit would have been developed and optimized through internal R&D, but specific training set data is not typically disclosed in 510(k) summaries for such devices.
9. How the Ground Truth for the Training Set Was Established
As no "training set" is explicitly mentioned for the reported comparison study (see #8), this question is not directly applicable. If a training set were implied for the kit's development, its ground truth would also presumably be established by highly accurate confirmatory methods like GC/MS.
Ask a specific question about this device
(335 days)
The Cozart Bioscience EIA Cotinine Urine Kit is intended for laboratory based testing in clinical and analytical laboratories and for insurance assessment. It provides qualitative screening results for Cotinine in human urine at a cut-off concentration of 500ng/ml. The Cozart EIA Cotinine Urine Kit acts as an aid in the detection of cotinine after use of tobacco products or other products containing nicotine.
This assay is for professional use only and provides only a preliminary analytical test result. Clinical consideration and professional judgement must be applied to any drug of abuse test result, particularly in evaluating a preliminary positive result. In order to obtain a more confirmed analytical result a more specific alternative chemical method is needed. Gas Chromatography/Mass Spectrometry (GC/MS) is the preferred confirmatory method.
The Cozart EIA Cotinine Urine Kit test is a competitive ELISA for the detection of Cotinine in human urine. The kit supplies the following reagents - a microtitre plate coated with antibody, enzyme conjugate reagent, wash buffer, substrate solution, stop solution and four calibrators (0, 50, 500 and 5000ng/ml Cotinine in human urine).
Here's a breakdown of the acceptance criteria and the study details for the Cozart EIA Cotinine Urine Kit, based on the provided text:
Acceptance Criteria and Device Performance
| Parameter | Acceptance Criteria (Implied by Predicate/Performance Claims) | Reported Device Performance (Cozart EIA Cotinine Urine Kit) |
|---|---|---|
| Intended Use | Qualitative test for Cotinine in human urine with a 500ng/ml cutoff. Recommendation for GC/MS confirmation. | Qualitative test for Cotinine in human urine with a 500ng/ml cutoff. Recommendation for GC/MS confirmation. |
| Method Comparison | High agreement with GC/MS for positive and negative samples. | 98% agreement for positive results (56/57), 100% agreement for negative results (38/38) with GC/MS (based on 95 samples). |
| Precision | CV (%) for the assay should be acceptable for qualitative ELISA (e.g., <10% for internal controls, <11% for total precision). | CVs less than 10% for internal controls; total precision CVs less than 11%. |
| Sensitivity | A low detection limit for cotinine. | 1.2 ng/ml |
| Specificity | Minimal cross-reactivity with common interferents. | Only nicotine cross-reacted (at 10,000 ng/ml) out of 28 tested substances. No interference observed with pH, specific gravity, ascorbic acid, or protein. |
| Cutoff Concentration | Proper differentiation around the 500ng/mL cutoff. | Successfully differentiated between 375ng/mL (higher absorbance) and 625ng/mL (lower absorbance) compared to the 500ng/mL cutoff calibrator. |
| Sample Stability | Samples should remain stable for a reasonable period under various storage conditions. | Stable for 21 days at 2-8°C, 25°C, and 37°C. For longer storage, freeze at -20°C. |
| Stopped Assay Stability | Finished assay should be stable for a practical reading window. | Must be read within 15 minutes at 450nm. |
| Assay Drift | Minimal drift with variation in sample addition time. | Little change observed when samples added within 25 minutes. |
Study Details
-
Sample Size Used for the Test Set and Data Provenance:
- Sample Size: 95 urine specimens.
- Data Provenance: The samples were collected from drug dependency units in Coventry and Warwick, UK. The data is retrospective, as the samples were collected and then tested.
-
Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications:
- The ground truth was established by Gas Chromatography/Mass Spectrometry (GC/MS). While GC/MS is an analytical method rather than an "expert," the results were performed by the Analytical Services Laboratory at Cozart Bioscience Ltd. The qualifications of the personnel performing these analyses are not explicitly stated but are presumed to be qualified laboratory staff.
-
Adjudication Method for the Test Set:
- Not applicable in the traditional sense, as the ground truth was established by a single, definitive analytical method (GC/MS). The Cozart EIA result was compared directly against the GC/MS result for agreement. There was no "expert consensus" or "adjudication" between multiple interpretations of the ground truth.
-
If a Multi Reader Multi Case (MRMC) Comparative Effectiveness Study was done:
- No, an MRMC comparative effectiveness study was not done. The study focused on the performance of the Cozart EIA Cotinine Urine Kit against a gold standard analytical method (GC/MS) rather than comparing different human readers or human readers with and without AI assistance.
-
If a Standalone Study (algorithm only without human-in-the-loop performance) was done:
- Yes, a standalone study was performed. The Cozart EIA Cotinine Urine Kit is a laboratory-based immunoassay kit, and its performance was evaluated intrinsically as a diagnostic device, without human interpretation influencing the primary result generated by the kit itself. The results are read spectrophotometrically, which is an objective measurement.
-
The Type of Ground Truth Used:
- The ground truth used was Gas Chromatography/Mass Spectrometry (GC/MS), which is considered a highly specific and sensitive reference method for cotinine detection (often referred to as the "gold standard" in toxicology).
-
The Sample Size for the Training Set:
- The document does not explicitly mention a separate "training set" for the Cozart EIA Cotinine Urine Kit. For an immunoassay kit like this, the development process involves reagent formulation and optimization, calibration, and validation. The "training" in this context would typically refer to the internal development and optimization of the assay parameters, not a separate data set for a machine learning algorithm. The provided "Method Comparison" data is for the test set (validation of the final product).
-
How the Ground Truth for the Training Set Was Established:
- As no explicit "training set" is mentioned in the context of typical machine learning models, the concept of establishing ground truth for it doesn't directly apply here. For the development and optimization of the assay itself, known concentrations of cotinine (calibrators) and controls would have been used. The GC/MS method would likely have also been used during the R&D phase to define the kit's performance characteristics and optimize its components (e.g., antibody binding, enzyme conjugates, cutoff values).
Ask a specific question about this device
(95 days)
The Cozart EIA Cocaine Oral Fluid Kit is intended for use in clinical and analytical laboratories when used in conjunction with the Cozart RapiScan Oral Fluid collection system. Using this collection system it provides qualitative screening results for cocaine and metabolites in human oral fluid at a cutoff concentration of 10ng/ml. This is equal to 30ng/ml. in undiluted oral fluid as the collection system involves a 1:3 dilution of the sample.
This assay is for professional use only and provides only a preliminary analytical test result. Clinical consideration and professional judgement must be applied to any drug of abuse test result, particularly in evaluating a preliminary positive result. In order to obtain a more confirmed analytical result a more specific alternative chemical method is needed. Gas Chromatography/Mass Spectrometry (GC/MS) is the preferred confirmatory method.
The Cozart EIA Cocaine Oral Fluid Kit is a laboratory based test for the detection of cocaine and metabolites in human oral fluid using a cutoff equivalent to 30ng/mL. The Cozart EIA Cocaine Oral Fluid Kit is a competitive ELISA for the detection of cocaine and its metabolites in human oral fluid. The kit supplies the following reagents - a microtitre plate coated with antibody, enzyme conjugate reagent, wash buffer, substrate solution, stop solution and four calibrators (0, 5, 10 and 50ng/ml benzoylecgonine in oral fluid matrix).
Here's a summary of the acceptance criteria and study details for the Cozart EIA Cocaine Oral Fluid Kit, based on the provided text:
Acceptance Criteria and Device Performance
| Acceptance Criteria Category | Reported Device Performance | Comments / Context |
|---|---|---|
| Method Comparison | 96% overall agreement | Compared to GC/MS (Gas Chromatography/Mass Spectrometry), the gold standard. |
| Precision | CVs less than 9% (kit) | Acceptable for qualitative manual ELISA assay (CVs less than 10% acceptable). |
| Total precision (kit) CVs less than 11% | ||
| Sensitivity | 1.8 ng/mL | Detection limit for cocaine in oral fluid. |
| Specificity | No cross-reactivity with 29 potential interferents | Three out of four related compounds showed some cross-reactivity. |
| Cutoff Concentration Validation | Absorbances for 5 ng/mL higher than 10 ng/mL cutoff calibrator; Absorbances for 15 ng/mL lower than 10 ng/mL cutoff calibrator | Confirmed appropriate response around the 10 ng/mL cutoff. |
| Interference Studies | No interference observed | Tested against alcohol, sample adequacy indicator dye, hemoglobin, smoking, coffee, tea, water, food, orange juice, hard candy, chewing gum, and mouthwash. |
| Stopped Assay Stability | Must be read within 15 minutes at 450 nm | Absorbance readings at various time points (0, 5, 10, 15, 30, 45, 60 minutes) showed stability up to 15 minutes. |
| Assay Drift | Little change across the plate | Sample addition must occur within 25 minutes. |
Study Details
-
Sample Size Used for the Test Set and Data Provenance:
- Sample Size: 177 samples.
- Data Provenance: Not explicitly stated, but the context of clinical and forensic samples suggests human samples. The country of origin is not specified, but the manufacturer is based in the UK. The study appears to be retrospective, as existing samples were tested.
-
Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications:
- Not applicable for this type of in-vitro diagnostic (IVD) device. The ground truth was established by Gas Chromatography/Mass Spectrometry (GC/MS), which is an analytical method, not human expert consensus, for drug detection.
-
Adjudication Method for the Test Set:
- Not applicable. The reference method (GC/MS) served as the definitive ground truth, and the device's results were compared directly to it. There was no mention of human adjudication between different readers or methods beyond the GC/MS.
-
If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done:
- No, an MRMC study was not done. This device is an in-vitro diagnostic kit, not an imaging or diagnostic algorithm requiring human interpretation of output.
-
If a Standalone (i.e., algorithm only without human-in-the-loop performance) Was Done:
- Yes, this was a standalone performance study. The Cozart EIA Cocaine Oral Fluid Kit is a laboratory-based test that provides a qualitative result based on its biochemical reactions and spectrophotometric reading. Its performance was evaluated directly against GC/MS without human intervention in the interpretation of the kit's results.
-
The Type of Ground Truth Used:
- analytical reference method, specifically Gas Chromatography/Mass Spectrometry (GC/MS). The text explicitly states: "The device detailed above was compared to Gas Chromatography/Mass Spectrometry (GC/MS)." and "96% overall agreement as compared with GC/MS". GC/MS is widely considered the "gold standard" for confirmatory drug testing.
-
Sample Size for the Training Set:
- Not explicitly stated. The document describes a "method comparison" study as the primary performance evaluation. For a competitive ELISA kit, internal calibration and validation (which might involve a "training" equivalent) would have been performed by the manufacturer during its development, but details regarding a distinct "training set" sample size are not provided in this 510(k) summary.
-
How the Ground Truth for the Training Set Was Established:
- Not explicitly stated. Given that performance for the test set was established against GC/MS, it is highly probable that any internal development, calibration, or "training" (if applicable for this type of device) would also have referenced GC/MS or highly purified standards of cocaine and its metabolites at known concentrations.
Ask a specific question about this device
(101 days)
The Cozart EIA Opiates Oral Fluid Kit is intended for use in clinical and analytical laboratories when used in conjunction with the Cozart RapiScan Oral Fluid collection system. Using this collection system it provides qualitative screening results for Opiates in human oral fluid at a cutoff concentration of 10ng/ml. This is equal to 30ng/mL in undiluted oral fluid as the collection system involves a 1:3 dilution of the sample. This assay is for professional use only and provides only a preliminary analytical test result. Clinical consideration and professional judgement must be applied to any drug of abuse test result, particularly in evaluating a preliminary positive result. In order to obtain a more confirmed analytical result a more specific alternative chemical method is needed. Gas Chromatography/Mass Spectrometry (GC/MS) is the preferred confirmatory method.
Not Found
The provided document is a 510(k) premarket notification letter from the FDA to Cozart Bioscience Ltd. regarding the Cozart EIA Opiate Oral Fluid Kit. This letter confirms that the device has been found substantially equivalent to legally marketed predicate devices.
However, this document does not contain the acceptance criteria or a study that proves the device meets specific acceptance criteria. It primarily focuses on the regulatory approval process and the device's indications for use.
Therefore, I cannot extract the requested information regarding acceptance criteria, device performance, sample sizes, expert qualifications, and study methodologies from this specific document. The letter itself does not detail the technical studies or their results, but rather refers to them implicitly by stating that the device has met the requirements for substantial equivalence.
To answer your request, I would need access to the actual 510(k) submission and the studies referenced within it.
Ask a specific question about this device
(229 days)
The Cozart RapiScan Oral Fluid Drug Test System - Opiates and Methadone is intended for Point of Care testing in a number of settings such as prescription workplace, drug dependency clinics, and criminal justice. It provides qualitative screening results for opiates and methadone in human oral fluid at a cut-off concentration equivalent to 40ng/mL and 30ng/mL respectively in neat oral fluid. The cozart collection system involves a 1:3 dilution of the oral fluid sample, this dilution factor is corrected for in the Cozart RapiScan Drug Test and therefore the remainder of this document will refer to the cut-offs as 13ng/mL for opiates and 10ng/ml for methadone.
This kit has to be used in conjunction with the Cozart RapiScan instrument. Please refer to the sampling and testing procedures in the instructions for use leaflet.
This assay is for professional use only and provides only a preliminary analytical test result. Clinical consideration and professional judgement must be applied to any drug of abuse test result, particularly in evaluating a preliminary positive result. In order to obtain a more confirmed analytical result a more specific alternative chemical method is needed. Gas Chromatography/Mass Spectrometry (GC/MS) is the preferred confirmatory method.
It is the responsibility of those organisations required to follow department of transportation (DOT) or the Substance Abuse and Mental Health Services Administration (SAMHSA) Workplace Drug Testing guidelines to determine that this product satisfies the criteria for workplace testing established under DOT and SAMHSA.
The Cozart RapiScan opiates and methadone test system is composed of three parts - the oral fluid collection device, the opiates and methadone test cartridge and the Cozart RapiScan instrument.
- Oral fluid Collection Device: composed of an oral fluid collector pad, a transport tube containing run buffer and a dispense filter tube.
- Opiate and Methadone Test Cartridge: composed of a white reaction strip with immobilized drug sites for opiates and methadone contained in a blue plastic casing. The cartridge also has a pad containing gold labeled anti-morphine and gold labeled anti-methadone antibodies. In addition, the cartridge has a built in control line of sheep anti-mouse IgG antibody.
- Cozart RapiScan Instrument: a battery-powered instrument whose function is to automate the analysis of the opiate/methadone test cartridge. It is a hand-held low voltage optical scanner.
Here's an analysis of the Cozart RapiScan Oral Fluid Drug Test System for Opiates and Methadone, based on the provided text:
Acceptance Criteria and Device Performance
The acceptance criteria for the Cozart RapiScan are primarily defined by its ability to accurately detect opiates and methadone in oral fluid at specified cut-off concentrations, as validated against GC/MS. The study demonstrates the device's performance in terms of sensitivity, specificity, precision, and stability under various conditions.
Table of Acceptance Criteria and Reported Device Performance
| Performance Metric | Acceptance Criteria (Implied) | Reported Device Performance |
|---|---|---|
| Opiates Detection | ||
| Cut-off Concentration | 40 ng/mL in neat oral fluid (equivalent to 13 ng/mL in diluted sample) | The device correctly identified opiates at this cut-off. |
| Positive Agreement | High agreement with GC/MS for positive samples | Out of 99 RapiScan positive samples for opiates, 93 were confirmed positive by GC/MS. |
| Negative Agreement | High agreement with GC/MS for negative samples | Out of 124 RapiScan negative samples for opiates, 123 were confirmed negative by GC/MS. |
| Near Cut-off Performance | Accurate detection for samples within +/- 50% of the cut-off concentration | 24 samples were within +/- 50% of the opiate cut-off (20-60ng/mL). The specific accuracy for these 24 samples is not explicitly stated in a consolidated manner, but the overall method comparison suggests acceptable performance. Precision studies indicated 40/40 positive and 0/40 negative for 13 ng/mL in laboratory settings. For point of care settings, 25/25 positive and 1/25 negative for 13 ng/ml. |
| Methadone Detection | ||
| Cut-off Concentration | 30 ng/mL in neat oral fluid (equivalent to 10 ng/mL in diluted sample) | The device correctly identified methadone at this cut-off. |
| Positive Agreement | High agreement with GC/MS for positive samples | Out of 116 RapiScan positive samples for methadone, 116 were confirmed positive by GC/MS. |
| Negative Agreement | High agreement with GC/MS for negative samples | Out of 115 RapiScan negative samples for methadone, 115 were confirmed negative by GC/MS. |
| Near Cut-off Performance | Accurate detection for samples within +/- 50% of the cut-off concentration | 24 samples were within +/- 50% of the methadone cut-off (15-45ng/mL). The specific accuracy for these 24 samples is not explicitly stated in a consolidated manner, but the overall method comparison suggests acceptable performance. Precision studies indicated 40/40 positive and 0/40 negative for 10 ng/mL in laboratory settings. For point of care settings, 24/25 positive and 2/26 negative for 10 ng/ml. |
| Sensitivity | Detect drugs at or below 25-50% of the cut-off (implied by experiment design) | Opiates: detection limit between 9.75ng/mL (-25% of cutoff) and 6.5ng/mL (-50% of cutoff). Methadone: detection limit between 7.5ng/mL (-25% of cutoff) and 5ng/mL (-50% of cutoff). |
| Specificity | Minimal cross-reactivity with common interfering substances and related compounds. | 28 potentially interfering substances tested at 100,000ng/mL (Buprenorphine at 10,000ng/mL) showed no cross-reactivity. Lowest concentration to produce a positive was determined for 13 opiate family members and 4 methadone-related substances across three different batches, demonstrating appropriate specificity for these related compounds. |
| Precision | Consistent results over time and across different operators, especially near the cut-off. | Laboratory: For opiates, spiked samples at 13 ng/mL (cutoff) showed 40/40 positive. For methadone, spiked samples at 10 ng/mL (cutoff) showed 40/40 positive. Consistent results for samples at 0 ng/mL, 6.5 ng/mL (19.5 ng/mL) for opiates and 0 ng/mL, 5 ng/mL (15 ng/mL) for methadone. Point-of-Care: Opiate: 25/25 for 13 ng/mL, 1/25 negative for 13 ng/mL. Methadone: 24/25 for 10 ng/mL, 2/26 negative for 10 ng/mL. This suggests good precision even in varied settings. |
| Sample Stability | Samples remain stable for accurate testing for a reasonable period. | Stable at both room temperature and 2-8°C for 28 days. |
| Freeze/Thaw Stability | Samples tolerate multiple freeze/thaw cycles without affecting results. | Samples can undergo up to six freeze/thaw cycles at -20°C prior to testing. |
| Interference | No interference from common oral fluid contaminants or patient activities. | No interference observed from sample adequacy indicator dye, hemoglobin, smoking, coffee, tea, water, food, orange juice, hard candy, chewing gum, and mouthwash. |
| Time of Day/Instrument/Operator Comparison | Results should be consistent regardless of time of day, instrument used, or operator. | Sampling at different times of day (morning, midday, evening after fortification at +/- 50% cutoff) had no effect. All three instruments produced unambiguous results. Inexperienced operators yielded unambiguous results for all three operators. |
| Accuracy/Recovery | Accurate detection of fortified samples at various concentrations around the cut-off. | All samples fortified at or above the cut-off concentration (0, 6.5, 13, 19.5 ng/mL for morphine and 0, 5, 10, 15 ng/mL for methadone) were positive, with the exception of the negative sample fortified at 50% below the cut-off as expected, indicating accurate recovery and detection across the relevant range. |
| Cut-off Concentration Effectiveness | Clear discrimination at the established cut-off. | All samples fortified at 50% below the cut-off were negative. All samples fortified at the cut-off, 50% above, and 100% above the cut-off were positive for both opiates and methadone. |
Study Details
-
Sample sizes used for the test set and the data provenance:
- Method Comparison Test Set:
- Total Samples: 454 oral fluid samples.
- Opiates: 99 RapiScan positive (93 GC/MS confirmed), 124 RapiScan negative (123 GC/MS confirmed). (Total: 223 samples for opiates analysis against GC/MS)
- Methadone: 116 RapiScan positive (116 GC/MS confirmed), 115 RapiScan negative (115 GC/MS confirmed). (Total: 231 samples for methadone analysis against GC/MS)
- Special Samples:
- 24 opiate samples within +/- 50% of cut-off (14 previously diluted with blank saliva).
- 24 methadone samples within +/- 50% of cut-off (20 previously diluted with blank saliva).
- Provenance: All samples were oral fluid.
- 99 opiate-positive samples and 116 methadone-positive samples were from "drug users enrolled at a drug rehabilitation clinic."
- 103 opiate-negative samples and 102 methadone-negative samples were from "drug free volunteers."
- 21 opiate-negative samples and 13 methadone-negative samples were from "known drug users."
- The samples were tested by Cozart Bioscience Ltd staff, and GC/MS performed by FSS Chepstow UK, University of Glasgow, Scotland, and The Analytical Laboratory at Cozart Bioscience Ltd, UK. This suggests a retrospective collection of samples from clinics and volunteers, followed by prospective testing with the RapiScan and GC/MS. The origin is primarily the UK.
- Precision Test Set (Laboratory): Four oral fluid samples (negative, cutoff, 50% above cutoff, 50% below cutoff) tested in duplicate every day for 20 days (4 samples * 2 duplicates * 20 days = 160 tests per drug).
- Precision Test Set (Point of Care - POC): Four oral fluid samples (spiked at 0, 6.5, 13, 19.5 ng/mL for opiates and 0, 5, 10, 15 ng/mL for methadone) from drug-free volunteers, tested up to 9 times at 3 POC sites.
- Sensitivity Test Set: Eight negative oral fluid samples, fortified at 50% above cut-off and serially diluted.
- Specificity Test Set: Not explicitly stated, but 28 potentially interfering substances and 13 opiate/4 methadone related substances were tested.
- Cutoff Concentration Test Set: 8 fortified oral fluid samples at various concentrations (cutoff, +/- 50% cutoff, +/- 100% cutoff).
- Interference Studies Test Set: Not explicitly stated, but various parameters (dye, hemoglobin, common consumables) were tested.
- Stability Studies Test Set: Four fortified samples (2 at 50% above cutoff, 2 at 50% below cutoff) and two negative samples from volunteers.
- Time of Day Comparison Test Set: Samples from 4 volunteers at 3 different times of day, fortified.
- Instrument Comparison Test Set: Samples from 4 volunteers, fortified, tested on 3 different instruments.
- Operator Comparison Test Set: Samples from 4 volunteers, fortified, tested by "inexperienced operators" (number not specified, but likely at least 3 given the results statement mentions "all three operators").
- Method Comparison Test Set:
-
Number of experts used to establish the ground truth for the test set and the qualifications of those experts:
- The ground truth for the test set (method comparison) was established by Gas Chromatography/Mass Spectrometry (GC/MS). While GC/MS is an analytical method, not human experts, it is considered the "preferred confirmatory method" and gold standard for drug testing. The performing laboratories were FSS Chepstow UK, University of Glasgow, Scotland, and The Analytical Laboratory at Cozart Bioscience Ltd, UK. The qualifications of the GC/MS operators are implied to be standard for analytical laboratory personnel competent in this method.
-
Adjudication method for the test set:
- Not applicable in the typical sense for medical imaging or clinical decision-making. The "adjudication" here is the direct comparison of the Cozart RapiScan qualitative result (positive/negative) against the quantitative GC/MS result, using the defined cut-off concentrations. Discrepancies would be identified based on whether the RapiScan result matched the GC/MS result relative to the cut-off.
-
If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:
- No, a multi-reader multi-case (MRMC) comparative effectiveness study was not done. This device is a point-of-care test read by an instrument, not by human readers interpreting complex images or data. The instrument automatically displays results as "positive or negative," eliminating human subjectivity in reading.
-
If a standalone (i.e. algorithm only without human-in-the-loop performance) was done:
- Yes, the performance study described is essentially a standalone algorithm performance study. The Cozart RapiScan instrument, which incorporates the device's "algorithm" for reading the test cartridge, operates without human interpretation of the test line. The human interaction involves initiating the test and reading the displayed digital result (positive or negative). The study evaluates the objective output of this system compared to the gold standard. The note that the "Cozart RapiScan instrument must be used to read the cartridges, this eliminates the subjectivity of reading by eye" further supports this being a standalone evaluation of the device's automated reading.
-
The type of ground truth used (expert consensus, pathology, outcomes data, etc.):
- The ground truth used was Gas Chromatography/Mass Spectrometry (GC/MS), which is the "preferred confirmatory method" and considered the analytical gold standard for drug detection and quantification in biological samples.
-
The sample size for the training set:
- The document does not explicitly state a separate training set size. The studies described are performance validation studies. It's common for such devices (especially those using immunoassay principles rather than complex machine learning algorithms) not to have a distinct "training set" in the sense of AI/ML models. The design and optimization of the immunoassay itself (e.g., antibody selection, cut-off determination) would be based on internal development and verification processes that precede the formal validation studies presented here.
-
How the ground truth for the training set was established:
- As no explicit training set is mentioned in the context of advanced machine learning, the concept of establishing ground truth for a training set in the typical AI/ML sense is not directly applicable. For the development of the immunoassay, the "ground truth" for calibrating the system and establishing the cut-off concentrations would likely have involved controlled samples with known concentrations of opiates and methadone, confirmed by analytical methods like GC/MS during the R&D phase of the test's design. The "Cutoff Concentration" study described within the document is part of the verification of these established cutoffs.
Ask a specific question about this device
(109 days)
Ask a specific question about this device
Page 1 of 1