(39 days)
The STC Opiates Micro-Plate EIA is intended for use in the qualitative determination of opiates in oral fluid collected with the OraSure® Oral Collection Device. For In Vitro Diagnostic Use.
The STC Opiates EIA is a competitive micro-plate immunoassay for the detection of opiates in oral fluid collected with the OraSure® Oral Specimen Collection Device. Specimen or standard is added to an EIA well in combination with an enzyme-labeled hapten derivative. In an EIA well containing an OraSure® specimen positive for opiates, there is a competition between the drug and the enzyme-labeled hapten to bind the antibody fixed onto the EIA well. EIA wells are then washed, substrate is added, and color is produced. The absorbance measured for each well at 450 nm is inversely proportional to the amount of opiates present in the specimen or calibrator/control.
Here's an analysis of the acceptance criteria and study detailed in the provided document for the STC Opiates Micro-Plate EIA (OraSure® Application):
1. Table of Acceptance Criteria and Reported Device Performance
The document does not explicitly state pre-defined acceptance criteria in a quantitative manner for specific performance metrics like sensitivity, specificity, positive predictive value (PPV), or negative predictive value (NPV) that the device must meet. However, it presents the results of its performance metrics in comparison to the predicate device and GC/MS.
Since explicit acceptance criteria are not provided for the STC Opiates Micro-Plate EIA (OraSure® Application) itself, I will infer the performance reported from the studies presented.
| Performance Metric | Acceptance Criteria (Inferred from Predicate/Standard) | Reported Device Performance (STC Opiates Micro-Plate EIA - OraSure® Application) |
|---|---|---|
| Analytical Sensitivity / Limit of Detection (LOD) | Lower LOD (0.2 ng/mL) compared to urine kit or a value that allows detection at clinical cutoff. | 0.2 ng/mL |
| Intra-Assay Precision (% CV) | Comparable to predicate urine kit (5.5%) or within an acceptable range for diagnostic tests. | 8-17% |
| Inter-Assay Precision (% CV) | Comparable to predicate urine kit (6.3%) or within an acceptable range for diagnostic tests. | 9-13% |
| Clinical Sensitivity (Study 1 - compared to GC/MS) | Ideally high (e.g., above 90%) for detection of opiates in oral fluid. The predicate urine kit had 87.5%. | 100.0% (for 10 presumed positive samples) |
| Clinical Specificity (Study 1 - compared to GC/MS) | Ideally high (e.g., above 90%) for detection of opiates in oral fluid. The predicate urine kit had 100.0%. | 87.5% (for 10 presumed positive samples) |
| Ability to detect at clinical doses (Study 2) | Detect opiates at clinically relevant doses, especially those requiring medical intervention. | Detects opiates at intravenous doses as low as 12mg heroin HCl. |
| Cross-Reactivity | Minimal cross-reactivity with non-opiate substances (e.g., 100%, Meperidine 0.1%) | |
| Interference (Common Materials) | Non-interference from common oral fluid adulterants or, if interfered, the effects are known. | Interference from Cranberry Juice, Orange Juice, Cola, Cough Syrup. |
2. Sample Sizes Used for the Test Set and Data Provenance
- Analytical Sensitivity/Limit of Detection (LOD):
- Test set sample size: 12 blank OraSure® devices.
- Data Provenance: Not explicitly stated, but implied to be from laboratory testing as part of device development.
- Precision:
- Test set sample size: For intra-assay, 16 replicates per level for 4 runs (total 64 measurements per level). For inter-assay, 2 samples per level, twice per day for 20 days (total 80 measurements per level). Levels tested were 0, 5, 10, 20 ng/mL morphine in OraSure® Control Matrix.
- Data Provenance: Laboratory testing.
- Analytical Specificity/Cross-Reactivity:
- Test set sample size: Not explicitly stated as a single number, but involves testing various compounds at specified concentrations. Some compounds were tested at a target concentration of 10,000 ng/mL in OraSure® Control Matrix.
- Data Provenance: Laboratory testing.
- Clinical Data - Accuracy (Study 1 - Chronic Opiates Users):
- Test set sample size: 60 samples (10 from self-reported opiate users, 50 presumed negative). Matched urine and OraSure® specimens were collected.
- Data Provenance: Clinical samples from self-reported opiate users. The country of origin is not specified, but given the submission is to the FDA, it is likely the US or similar regulatory environment. The data collection appears to be prospective for the purpose of the study.
- Clinical Data - Controlled Dose Study (Study 2):
- Test set sample size: 5 volunteers. Saliva and urine samples were collected periodically for up to 74 hours after dosing. This generated multiple samples per volunteer.
- Data Provenance: Prospective, controlled clinical study. The country of origin is not specified.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts
- For Study 1 (Chronic Opiates Users): The ground truth was established by Gas Chromatography/Mass Spectrometry (GC/MS). GC/MS is considered a "gold standard" analytical method and itself requires skilled operators and validation rather than "experts" in the clinical sense (e.g., radiologists). No specific number or qualifications of experts involved in interpreting the GC/MS results or clinical assessment for "self-reported opiates users" are mentioned.
- For Study 2 (Controlled Dose Study): The ground truth was based on controlled administration of heroin and subsequent analysis by GC/MS. Similar to Study 1, GC/MS acts as the reference method. No specific number or qualifications of clinical experts involved in determining the "ground truth" are stated.
4. Adjudication Method for the Test Set
- The document describes the comparison of the STC Opiates Micro-Plate EIA with GC/MS (Gas Chromatography/Mass Spectrometry) as the reference method (ground truth). This is a direct comparison rather than an adjudication process involving multiple human readers interpreting the same data. Therefore, no adjudication method (like 2+1 or 3+1) was used for the primary comparison.
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
- No, an MRMC comparative effectiveness study was not done. This device is an in-vitro diagnostic (IVD) assay, not an imaging device or AI algorithm intended to assist human readers in interpretation. The studies focus on the analytical and clinical performance of the assay itself.
6. Standalone (Algorithm Only) Performance Study
- Yes, the performance studies described are inherently "standalone" in the context of this device. The STC Opiates Micro-Plate EIA is a laboratory-based immunoassay, and its performance (sensitivity, specificity, precision, LOD, cross-reactivity) is measured directly against a reference method (GC/MS) or established analytical standards. There is no human-in-the-loop component for the interpretation of the assay's results; the assay provides a quantitative/qualitative result directly.
7. Type of Ground Truth Used
- Analytical studies (LOD, Precision, Cross-Reactivity): The ground truth was established through preparation of known concentrations of substances (e.g., morphine, other compounds) in control matrices, or by using a highly accurate analytical method like GC/MS to confirm concentrations.
- Clinical Studies (Study 1 & 2): The primary ground truth for clinical accuracy (sensitivity and specificity) was Gas Chromatography/Mass Spectrometry (GC/MS), which is considered the gold standard for drug confirmation. In Study 1, "self-reported opiates users" served as an initial categorization, but GC/MS was used for confirmation. In Study 2, controlled drug administration provided the definitive ground truth for exposure, with GC/MS used to confirm presence and concentration.
8. Sample Size for the Training Set
- The document does not explicitly mention a "training set" in the context of machine learning or AI algorithms, as this is an immunoassay device. The term "training set" is not generally applicable here. If one were to interpret "training" as the development and optimization of the assay itself, that process is not detailed with specific sample sizes. The document presents the validation studies (analytical and clinical) of the developed assay.
9. How the Ground Truth for the Training Set Was Established
- As mentioned in point 8, the concept of a "training set" and its associated ground truth establishment is not directly applicable to this immunoassay device. The assay's parameters (e.g., antibody characteristics, enzyme conjugation, substrate reaction) would have been developed and optimized based on standard immunoassay development practices, likely involving numerous experiments with known concentrations of analytes, but these are part of the development phase, not typically referred to as a "training set" in this context.
§ 862.3650 Opiate test system.
(a)
Identification. An opiate test system is a device intended to measure any of the addictive narcotic pain-relieving opiate drugs in blood, serum, urine, gastric contents, and saliva. An opiate is any natural or synthetic drug that has morphine-like pharmocological actions. The opiates include drugs such as morphine, morphine glucoronide, heroin, codeine, nalorphine, and meperedine. Measurements obtained by this device are used in the diagnosis and treatment of opiate use or overdose and in monitoring the levels of opiate administration to ensure appropriate therapy.(b)
Classification. Class II (special controls). An opiate test system is not exempt if it is intended for any use other than employment or insurance testing or is intended for Federal drug testing programs. The device is exempt from the premarket notification procedures in subpart E of part 807 of this chapter subject to the limitations in § 862.9, provided the test system is intended for employment and insurance testing and includes a statement in the labeling that the device is intended solely for use in employment and insurance testing, and does not include devices intended for Federal drug testing programs (e.g., programs run by the Substance Abuse and Mental Health Services Administration (SAMHSA), the Department of Transportation (DOT), and the U.S. military).