(41 days)
The Roche ONLINE Theophylline assay contains an in vitro diagnostic reagent system indicated for the quantitative determination of theophylline, a broncodilator, widely used to treat patients with asthma, apnea (temporary asphyxia), and other obstructive lung diseases. Measurements obtained by this device are used in the diagnosis and treatment of theophylline overdose or in monitoring levels of theophylline to ensure appropriate therapy.
The Roche ONLINE Theophylline assay contains an in vitro diagnostic reagent system indicated for the quantitative determination of theophylline, a bronchodilator, widely used to treat patients with asthma, apnea (temporary asphyxia), and other obstructive lung diseases, in human serum or plasma on automated clinical chemistry analyzers. Measurements obtained by the device are used in the diagnosis and treatment of theophylline overdose and in monitoring levels of theophylline to ensure appropriate therapy. The proposed labeling indicates that the Roche/Hitachi 911, 912, 917, and Modular P analyzers can be used with the Roche ONLINE Theophylline reagent kits.
Here's an analysis of the provided text regarding the "Roche ONLINE Theophylline Assay" and its acceptance criteria and study:
1. Table of Acceptance Criteria and Reported Device Performance
The document doesn't explicitly state quantitative acceptance criteria in terms of pre-defined thresholds. Instead, it states that "All of the evaluation studies gave acceptable results compared to the predicate device." This implies that the acceptance criteria were met if the performance of the new device was "substantially equivalent" to the predicate device.
To infer the acceptance criteria, we can look at the performance of the predicate device and the new device. A common approach for method comparison studies for medical diagnostics is to demonstrate a strong correlation (R-value close to 1) and a slope close to 1 with an intercept close to 0, indicating minimal proportional and constant bias. For precision, the CV% should be comparable or better.
| Performance Metric | Acceptance Criteria (Inferred from Predicate) | Roche ONLINE Theophylline (New Device) Performance | Roche COBAS INTEGRA Theophylline (Predicate Device) Performance |
|---|---|---|---|
| Method Comparison | |||
| n | Sufficient sample size for statistical significance (e.g., n > 100) | 103 | 138 |
| Slope (y vs x) | Close to 1 (e.g., 0.95 - 1.05) | 0.976 | 0.963 |
| Intercept (y vs x) | Close to 0 (e.g., within +/- 0.1) | 0.011 | 0.065 |
| Correlation Coefficient (R) | Close to 1 (e.g., > 0.98 for strong correlation in diagnostics) | 0.996 | 0.998 |
| Range | Clinically relevant range and covering normal/therapeutic/toxic levels | 0.62 to 39.6 µg/mL | 0.16 to 36.3 µg/mL |
| Precision | |||
| CV% (within run) | Low, consistent across levels. Comparable to or better than predicate (Level 1, 2, 3) | L1: 1.0, L2: 0.5, L3: 0.9 | L1: 1.6, L2: 1.9, L3: 1.8 |
| CV% (total) | Low, consistent across levels. Comparable to or better than predicate (Level 1, 2, 3) | L1: 1.9, L2: 1.7, L3: 2.1 | L1: 2.6, L2: 2.6, L3: 2.8 |
Study That Proves the Device Meets Acceptance Criteria:
The study conducted was a performance characteristic evaluation comparing the "Roche ONLINE Theophylline Assay" against the "Roche COBAS INTEGRA Theophylline Assay" (predicate device). The study assessed precision, lower detection limit, method comparison, specificity, and interfering substances. The provided data focuses on precision and method comparison.
2. Sample Size Used for the Test Set and Data Provenance
- Sample Size for Test Set (Method Comparison):
- Roche ONLINE Theophylline (new device): n = 103
- Roche COBAS INTEGRA Theophylline (predicate device): n = 138 (This value is for the predicate's comparison against another assay, not directly against the new device for this study. The n=103 is the relevant sample size for comparing the new device to the predicate).
- Data Provenance: Not explicitly stated. However, given that this is an in-vitro diagnostic device manufactured by Roche Diagnostics Corporation in Indianapolis, IN, it is highly likely the data was collected from clinical samples, possibly from multiple sites, but the geographical origin (e.g., country) is not specified. The study is retrospective, as it compares a new assay's results to an existing, legally marketed assay.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of those Experts
This information is not provided in the document. For chemical assays like this, "ground truth" often refers to a reference method or a highly accurate, often more laborious, laboratory method. It doesn't typically involve human expert consensus in the same way as medical imaging or clinical diagnoses. The comparison here is against a legally marketed predicate device, not necessarily a "ground truth" reference method established by experts.
4. Adjudication Method for the Test Set
This information is not applicable and therefore not provided. Adjudication methods (like 2+1, 3+1) are common in studies where human interpretation of data (e.g., medical images) can be subjective and multiple readers are involved. For an automated chemical assay comparing numerical results, adjudication by experts is not a standard practice. The comparison is objective, based on direct numerical results between the new device and the predicate device.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done, If So, What Was the Effect Size of How Much Human Readers Improve with AI vs Without AI Assistance
This information is not applicable. This is an in-vitro diagnostic device (a laboratory blood test), not an AI-powered diagnostic tool for human interpretation of complex data (like radiology images). Therefore, MRMC studies and the concept of human readers improving with AI assistance are not relevant to this device.
6. If a Standalone (i.e. algorithm only without human-in-the loop performance) Was Done
Yes, this study is inherently a standalone performance study of the device (assay and associated analyzers). The "Roche ONLINE Theophylline Assay" itself is an automated chemical analysis system. The performance metrics presented (method comparison, precision) are reflective of the assay's performance without direct human intervention in the result generation process beyond operating the analyzer and collecting samples.
7. The Type of Ground Truth Used
The "ground truth" in this context is the performance of the legally marketed predicate device (Roche COBAS INTEGRA Theophylline Assay). The study seeks to establish substantial equivalence to this predicate, meaning the new device performs sufficiently similarly. It's not using an independent, gold-standard "ground truth" like pathology or long-term clinical outcomes data, but rather a comparative standard.
8. The Sample Size for the Training Set
This information is not provided and is likely not applicable in the traditional sense for this type of device. The "Roche ONLINE Theophylline Assay" is a chemical reagent system. While there would have been extensive R&D and validation during its development (which might involve testing many samples), these are not typically referred to as a "training set" for an algorithm in the way that machine learning models use them. Chemical assays are developed based on chemical principles and optimized through experimentation, not "trained" on data.
9. How the Ground Truth for the Training Set Was Established
As explained above, the concept of a "training set" and its "ground truth" in the context of machine learning is generally not applicable to this chemical assay. The development process would have involved establishing the accuracy and reliability of the chemical reactions and measurement principles through controlled experiments and analytical studies, rather than using a labelled dataset for algorithmic training.
§ 862.3880 Theophylline test system.
(a)
Identification. A theophylline test system is a device intended to measure theophylline (a drug used for stimulation of the muscles in the cardiovascular, respiratory, and central nervous systems) in serum and plasma. Measurements obtained by this device are used in the diagnosis and treatment of theophylline overdose or in monitoring levels of theophylline to ensure appropriate therapy.(b)
Classification. Class II.