(45 days)
The EXTC CAL is an in vitro diagnostic product for the calibration of the Ecstasy (EXTC) method on the Dimension Vista™ System.
EXTC CAL is a liquid, human urine-based product containing weighed-in quantities of methylenedioxymethamphetamine. The kit consists of six vials, three vials of Calibrator A and three vials of Calibrator B. EXTC CAL is ready for use (no preparation is required). The volume per vial is 2.3 mL. Intermediate levels are automatically prepared and corresponding values calculated by the Dimension Vista™ System.
Here's a breakdown of the acceptance criteria and the study that proves the device meets them, based on the provided text:
1. Table of Acceptance Criteria and Reported Device Performance:
| Acceptance Criteria Category | Specific Acceptance Criterion | Reported Device Performance |
|---|---|---|
| Stability (Shelf Life) | Target shelf life for the Dimension Vista™ System Ecstasy Calibrator is 12 months. Calibrator shelf life is determined by comparing results of the product stored at 4℃ with control stored at -20℃. Recovery versus time is monitored and percent change over time is determined where the allowable shelf life percent change is ≤ 10 %. | "Shelf-life stability (expiration) dating assignment at commercialization reflects the real-time data on file at Dade Behring, Inc." (This implies that the 12-month target with ≤ 10% change was met, as the product received clearance for marketing with an assigned expiration date. While the explicit numerical performance over the 12 months isn't stated, the affirmation of "real-time data on file" supports meeting the criterion.) |
| Stability (On-board) | Vial punctured by the instrument and stored on board is stable for 24 hours. | Performance not explicitly stated in the provided text. The text only says "A vial punctured by the instrument and stored on board is stable for 24 hours." This is given as the criterion itself, not a result of a study. |
| Stability (Open Vial) | Open vial not on instrument, but recapped and stored in a refrigerator is stable for 31 days. | Performance not explicitly stated in the provided text. The text only says "An open vial not on instrument, but recapped and stored in a refrigerator is stable for 31 days." This is given as the criterion itself, not a result of a study. |
| Traceability | The assigned values of the EXTC CAL are traceable to Methylenedioxymethamphetamine (99% purity) (Cerilliant, Inc.). | "The assigned values of the ECTX CAL is traceable to Methylenedioxymethamphetamine (99% purity) (Cerilliant, Inc., 811 Paloma Drive, suite A, Roundrock, TX 78664." (This directly states the criterion is met.) |
| Value Assignment | New calibrator Master Pool values are verified against a previously approved Master Pool lot and Gas Chromatography / Mass Spectrometry (GC/MS) testing. Stock solution concentration is verified by comparing them to Master Pool assigned bottle values. Commercial lot concentration of each level is verified to be within acceptable range using an instrument calibrated with Master Pools and by GC/MS testing. | The text describes the process used for value assignment which inherently acts as the performance demonstration that the values are assigned correctly and verified: "The Master Pool values are verified against the values of a previously approved Master Pool lot and Gas Chromatography / Mass Spectrometry (GC/MS) testing." and "The stock solution concentration is verified by comparing them to the Master Pool assigned bottle values." and "The concentration of each level of the commercial lot is verified to be within acceptable range by using an instrument calibrated with Master Pools and by GC/MS testing." (This indicates the procedure was followed successfully, implying the criterion was met.) |
2. Sample Size Used for the Test Set and Data Provenance:
- Test Set Sample Size: The document does not explicitly state a specific "test set" sample size in terms of number of samples or analytes tested. For stability testing, samples (vials) were subjected to different storage conditions (4°C, -20°C, on-board, open/re-capped). The testing involved "multiple calibrations."
- Data Provenance: The data comes from prospective stability studies conducted by Dade Behring Inc. The text describes how "Calibrator shelf life is determined by comparing results of the product stored at 4℃ with control stored at -20℃" and that "Shelf-life stability (expiration) dating assignment at commercialization reflects the real-time data on file at Dade Behring, Inc." The studies were conducted at Dade Behring, Inc., Newark, DE.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts:
- This device is a calibrator for a clinical toxicology assay, not an imaging or diagnostic device that typically involves expert interpretation for ground truth.
- For Value Assignment: The "ground truth" (or reference standard) is established by using Methylenedioxymethamphetamine (99% purity) from Cerilliant, Inc. as a primary reference material, and then verified through internal Master Pool lots and Gas Chromatography / Mass Spectrometry (GC/MS) testing. This suggests a rigorous chemical and analytical method for establishing the true concentration, rather than human expert consensus.
- Qualifications: The document does not specify qualifications of individuals performing the GC/MS or Master Pool verification, but it can be assumed that these would be trained analytical chemists or laboratory personnel.
4. Adjudication Method for the Test Set:
- No adjudication method is described, as the performance evaluation relies on objective analytical measurements (recovery percentages, GC/MS results) compared to predefined thresholds, rather than subjective interpretation by multiple human observers.
5. If a Multi Reader Multi Case (MRMC) Comparative Effectiveness Study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:
- No, an MRMC comparative effectiveness study was not done. This device is a calibrator, which provides a known concentration standard for an assay, and does not involve human interpretation or AI assistance in the way a diagnostic imaging device would.
6. If a Standalone (i.e. algorithm only without human-in-the-loop performance) was done:
- This question is not applicable to a calibrator device. The "device" (calibrator) itself doesn't perform an algorithm or provide a diagnostic output independently. It's a reagent used by an instrument (Dimension Vista™ System) to perform an assay. Its performance is evaluated through its stability and accuracy in providing known concentrations.
7. The Type of Ground Truth Used:
- The ground truth for the calibrator's value assignment is based on gravimetric addition of highly pure reference material (Methylenedioxymethamphetamine, 99% purity from Cerilliant) and verified by Gas Chromatography / Mass Spectrometry (GC/MS) testing and comparison to previously approved Master Pool lots. This is a highly accurate chemical/analytical ground truth.
8. The Sample Size for the Training Set:
- The concept of a "training set" is not directly applicable to a chemical calibrator in the same way it would be for a machine learning algorithm.
- The document describes the preparation of "Master Pool" lots (five levels) which are then used to verify new commercial calibrator lots. While these Master Pools serve a foundational role in establishing acceptable concentration ranges, they are not a "training set" in the AI sense. The number of samples for forming these Master Pools or for verifying them is not specified beyond "Five levels of Master Pool are prepared."
9. How the Ground Truth for the Training Set Was Established:
- As noted above, a traditional "training set" for an algorithm is not present. However, the foundational "ground truth" for the calibrator's concentration, against which all subsequent production lots are validated, is established through:
- Gravimetric addition of a precisely weighed, high-purity reference material (Methylenedioxymethamphetamine, 99% pure) to drug-free human urine to create the Master Pools. This is the primary method of establishing the "true" concentration.
- Verification against previously approved Master Pool lots.
- Verification using Gas Chromatography / Mass Spectrometry (GC/MS) testing. GC/MS is a highly accurate analytical technique for identifying and quantifying substances.
§ 862.3200 Clinical toxicology calibrator.
(a)
Identification. A clinical toxicology calibrator is a device intended for medical purposes for use in a test system to establish points of reference that are used in the determination of values in the measurement of substances in human specimens. A clinical toxicology calibrator can be a mixture of drugs or a specific material for a particular drug (e.g., ethanol, lidocaine, etc.). (See also § 862.2 in this part.)(b)
Classification. Class II (special controls). The device is exempt from the premarket notification procedures in subpart E of part 807 of this chapter subject to the limitations in § 862.9.