(50 days)
The VALIDATE Chem 7 Calibration Test Set is intended for in vitro diagnostic use in quantitatively verifying calibration, validating reportable ranges, and determining linearity in automated, semi-automated and manual chemistry systems for the following analytes: iron, creatinine, ammonia, and ethanol.
VALIDATE Chem 7 Calibration Verification Test Set is an aqueous based calibration verification test set containing multiple levels used to establish the relationship between theoretical operation and actual performance of the included analytes. Each set contains one bottle each of six (6) levels, including zero. Each bottle contains 5 milliliters.
The provided text describes the VALIDATE Chem 7 Calibration Verification Test Set and its comparison to predicate devices, but it does not explicitly state acceptance criteria in numerical thresholds or specific performance metrics other than correlation coefficients.
The study aims to demonstrate substantial equivalence to existing predicate devices, rather than meeting specific predefined performance targets set by the manufacturer. The core of the "acceptance criteria" here is that the device's performance is comparable to that of the previously cleared predicate devices.
Here's the information requested, based on the provided text, with acknowledgments of what is not present:
1. Table of Acceptance Criteria and Reported Device Performance
As mentioned, explicit numerical acceptance criteria for correlation coefficient, beyond demonstrating "functional equivalence" to predicate devices, are not provided. The acceptance criterion is implicit: the device's performance (correlation coefficient and regression equation) should be similar to or better than that of the predicate devices.
| Analyte | Characteristic/Metric | Acceptance Criteria (Implied: Substantial Equivalence to Predicate) | VALIDATE Chem 7 Reported Performance | Predicate Device Reported Performance (Example: DOCUMENT Iron, Magnesium, Triglyceride CAL•VER) |
|---|---|---|---|---|
| Iron (FE) | Correlation Coefficient (r) | "Functionally equivalent" to predicate device's r | 0.99997 | 0.99979 |
| Regression Equation | "Functionally equivalent" to predicate device's result | Y = 0.327 + 1.001X | Y = -5.539 + 1.007X | |
| Creatinine (CRE) | Correlation Coefficient (r) | "Functionally equivalent" to predicate device's r | 0.99952 | 0.99987 (from DOCUMENT Multi-Analyte CAL•VER) |
| Regression Equation | "Functionally equivalent" to predicate device's result | Y = 0.091 + 0.983X | Y = -0.099 + 1.007X (from DOCUMENT Multi-Analyte CAL•VER) | |
| Ammonia (NH3) | Correlation Coefficient (r) | "Functionally equivalent" to predicate device's r | 0.99994 | 0.99809 (from DOCUMENT Ammonia/Ethanol CAL•VER) |
| Regression Equation | "Functionally equivalent" to predicate device's result | Y = -0.363 + 1.019X | Y = -11.616 + 1.074X (from DOCUMENT Ammonia/Ethanol CAL•VER) | |
| Ethanol (ETOH) | Correlation Coefficient (r) | "Functionally equivalent" to predicate device's r | 0.99998 | 0.99872 (from DOCUMENT Ammonia/Ethanol CAL•VER) |
| Regression Equation | "Functionally equivalent" to predicate device's result | Y = -0.159 + 1.009X | Y = -1.261 + 1.04X (from DOCUMENT Ammonia/Ethanol CAL•VER) |
2. Sample size used for the test set and the data provenance
- Sample Size (Test Set): The document states that the analytes were "tested in triplicate" for the linear regression analysis. It also mentions "six (6) levels, including zero" for the device. Therefore, the test set likely involved 6 levels * 3 replicates = 18 measurements for each analyte to generate the linear regression data. The predicate devices also use 5 levels.
- Data Provenance: The document does not explicitly state the country of origin. It indicates that the study used "pre-production lots of VALIDATE Chem 7 Calibration Verification Test Set" on a "Beckman CX instrument system". The study appears to be prospective in nature, as it involved testing pre-production lots.
3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts
Not applicable. This device is a calibration verification test set, not a diagnostic device requiring expert interpretation of results to establish ground truth. The "ground truth" for linearity and calibration verification in this context is established by the known concentrations/relationships of the levels within the test sets themselves and the analytical methods used to measure them on the instrument.
4. Adjudication method for the test set
Not applicable. As noted above, this study does not involve expert adjudication as it is a performance study for a calibration verification material on an instrument, not a diagnostic interpretation study.
5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance
Not applicable. This is not a diagnostic imaging or reader study. It is a performance study for an in vitro diagnostic (IVD) calibrator/control material. The concept of "human readers" or "AI assistance" is irrelevant to the function of this device.
6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done
Yes, in essence, the study is a "standalone" or "algorithm only" performance evaluation of the VALIDATE Chem 7 Calibration Verification Test Set, where the "algorithm" is the chemistry analyzer (Beckman CX instrument system) measuring the prepared samples. There is no human interpretation involved in the measurement process itself, only in the setup and analysis of the data.
7. The type of ground truth used
The ground truth is based on the known, manufactured concentrations/relationships specified for the calibration verification material and the analytical measurement principles of the Beckman CX instrument system. The study compares the new device's response curve to that of already accepted predicate devices.
8. The sample size for the training set
Not applicable. This device is a chemical calibrator, not a machine learning model. Therefore, there is no "training set" in the context of AI/ML. The manufacturing process of the calibration verification material and its characterization would involve its own internal validation, but that is distinct from a "training set" for an algorithm.
9. How the ground truth for the training set was established
Not applicable, as there is no training set for an AI/ML algorithm. The "ground truth" for the calibrator material itself would be established through precise manufacturing processes, metrological traceability (if applicable), and analytical methods to ensure the stated concentrations/relationships of the levels are accurate.
§ 862.1660 Quality control material (assayed and unassayed).
(a)
Identification. A quality control material (assayed and unassayed) for clinical chemistry is a device intended for medical purposes for use in a test system to estimate test precision and to detect systematic analytical deviations that may arise from reagent or analytical instrument variation. A quality control material (assayed and unassayed) may be used for proficiency testing in interlaboratory surveys. This generic type of device includes controls (assayed and unassayed) for blood gases, electrolytes, enzymes, multianalytes (all kinds), single (specified) analytes, or urinalysis controls.(b)
Classification. Class I (general controls). Except when intended for use in donor screening tests, quality control materials (assayed and unassayed) are exempt from the premarket notification procedures in subpart E of part 807 of this chapter subject to the limitations in § 862.9.