(45 days)
The VALIDATE Chem 1 Calibration Verification Test Set is intended for in vitro diagnostic use for quantitatively verifying calibration, validating reportable ranges, and determining linearity in automated, semi-automated and manual chemistry systems for the following analytes: sodium, potassium, chloride, calcium, phosphorus, lactate, glucose, urea nitrogen, creatinine, triglyceride, ammonia, ethanol, magnesium, and lithium.
VALIDATE Chem 1 Calibration Verification Test Set contains purified chemicals in an aqueous base. Multiple levels are provided to establish the relationship between theoretical operation and actual performance of each of the included analytes. Each set contains one bottle each of six (6) levels, including zero. Each bottle contains 5 milliliters.
The provided 510(k) summary describes the VALIDATE Chem 1 Calibration Verification Test Set, a device intended for in vitro diagnostic use to verify calibration, validate reportable ranges, and determine linearity for various analytes in chemistry systems. The study aims to demonstrate substantial equivalence to predicate devices.
Here's an analysis of the acceptance criteria and the study that proves the device meets them:
1. Table of Acceptance Criteria and Reported Device Performance
The core acceptance criterion for this device, as implied by the substantial equivalence claim, is that its performance in verifying calibration and linearity for various analytes should be comparable to the legally marketed predicate devices. This is assessed through linear regression analysis, specifically by comparing correlation coefficients (r) and regression equations (Y = intercept + slope(X)). While explicit numerical thresholds for acceptance (e.g., minimum 'r' value, acceptable range for slope/intercept) are not directly stated in the provided text, the successful outcome of the 510(k) indicates that the FDA deemed the observed correlations and regression results to be acceptable for demonstrating substantial equivalence.
Given the context of calibration verification, ideal performance would be a correlation coefficient (r) close to 1, a slope close to 1, and an intercept close to 0, indicating a strong linear relationship and minimal bias compared to the reference.
Implicit Acceptance Criteria (based on the presented data and outcome):
- High Correlation Coefficient (r): Values close to 1 (e.g., >0.99) for each analyte when compared to predicate devices.
- Regression Equation indicating close agreement: Slopes close to 1 and intercepts close to 0 when comparing the candidate device's performance to the predicate a specific analyte.
Reported Device Performance (from Table 2 and continued Table 2):
| Analyte | VALIDATE Chem 1 Calibration Verification Test Set (r) | VALIDATE Chem 1 Calibration Verification Test Set (Y=intercept+slope(X)) | Predicate Device (r) (e.g., DOCUMENT Multi-Analyte CAL·VER) | Predicate Device Regression (Y=intercept+slope(X)) |
|---|---|---|---|---|
| NA | 0.99993 | -2.455 + 1.017 | 0.99976 (DOCUMENT Direct ISE CAL·VER) | 3.02 + 0.966 |
| K | 0.99964 | -0.014 + 1.018 | 0.99983 (DOCUMENT Direct ISE CAL·VER) | -0.014 + 0.98 |
| CL | 0.99998 | -0.175 + 1.002 | 0.99972 (DOCUMENT Direct ISE CAL·VER) | -3.751 + 1.017 |
| CA | 0.99992 | -0.017 + 1.007 | 0.99995 (DOCUMENT Multi-Analyte CAL·VER) | -0.609 + 1.124 |
| PO4 | 0.99999 | -0.048 + 1.011 | 0.99985 (DOCUMENT Multi-Analyte CAL·VER) | -0.167 + 1.049 |
| GLU | 0.99998 | -0.486 + 0.998 | 0.99994 (DOCUMENT Multi-Analyte CAL·VER) | 2.529 + 1.02 |
| BUN | 0.99998 | -0.729 + 1.016 | 0.99972 (DOCUMENT Multi-Analyte CAL·VER) | 1.836 + 0.924 |
| CRE | 0.99999 | -0.066 + 1.005 | 0.99997 (DOCUMENT Multi-Analyte CAL·VER) | -0.32 + 1.063 |
| TRIG | 0.99999 | -3.002 + 1.004 | 0.99997 (DOCUMENT Iron, Magnesium, Triglyceride CAL·VER) | 1.58 + 1.016 |
| MG | 0.99984 | 0.022 + 0.992 | 0.99913 (DOCUMENT Iron, Magnesium, Triglyceride CAL·VER) | 0.138 + 1.009 |
| LI | 0.99978 | 0.074 + 0.982 | 0.99959 (DOCUMENT Direct ISE CAL·VER) | -0.052 + 0.986 |
| LAC | 0.99995 | 0.046 + 0.979 | 0.99990 (Roche Diagnostics Precitrol Ethyl Alcohol/Ammonia/Lactate Controls) | -0.049 + 1.00 |
| NH3 | 0.99498 | 29.859 + 0.905 | 0.99945 (DOCUMENT Ammonia/Ethanol CAL·VER) | 10.778 + 0.915 |
| ETOH | 0.99948 | 2.281 + 0.965 | 0.99992 (DOCUMENT Ammonia/Ethanol CAL·VER) | 1.609 + 0.971 |
Interpretation: The reported correlation coefficients are all very high (ranging from 0.99498 to 0.99999), indicating a strong linear relationship between the new device and the predicate devices for all analyzed analytes. The regression equations generally show slopes close to '1' and intercepts relatively close to '0', further supporting the claim of substantial equivalence. The FDA concurred with this assessment, indicating that these results were acceptable.
2. Sample size used for the test set and the data provenance
-
Sample Size for the Test Set:
- The test set consists of the "VALIDATE Chem 1 Calibration Verification Test Set" and several predicate devices.
- The VALIDATE Chem 1 consists of 6 levels (including zero) for each of the 14 analytes.
- Each analyte was tested in triplicate.
- This means for each analyte, there were 6 levels * 3 replicates = 18 data points for the VALIDATE Chem 1.
- For the predicate devices, they mostly consisted of 5 levels (some 1 low and 1 high for the Roche control). Assuming they were also run in triplicate for comparison, this would contribute similar numbers of data points per analyte.
- The "test set" in this context refers to the collection of measurements taken using both the new device and the predicate devices for statistical comparison.
- It is mentioned that "pre-production lots" of the VALIDATE Chem 1 were used.
-
Data Provenance: The study was conducted by Maine Standards Company. There is no explicit mention of the country of origin of the data, but the submitter is based in Windham, ME, USA. The study appears to be prospective in nature, as it involved actively running pre-production lots of the new device and comparing them to existing predicate devices on a specified instrument (Roche Diagnostics Hitachi 911).
3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts
Not applicable. This is not a study involving human interpretation or subjective 'ground truth' establishment. The ground truth for calibration verification materials typically relies on the precisely defined concentrations of analytes within the reference materials themselves, which are then measured by the instrument. The 'ground truth' here is the expected response of the instrument to known concentrations, and the comparison is against predicate devices which also have established (or claimed) linearity and calibration verification properties.
4. Adjudication method for the test set
Not applicable. There was no human adjudication process involved in this study. The comparison was purely quantitative, using linear regression analysis between device measurements and predicate device measurements.
5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance
Not applicable. This is not a study involving human readers or AI assistance. It's a laboratory device performance study.
6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done
Yes, in a sense, a "standalone" performance was evaluated. The VALIDATE Chem 1 Calibration Verification Test Set itself is a standalone product (a set of solutions). Its performance was assessed independently by running it on an instrument (Roche Diagnostics Hitachi 911) and then statistically comparing its results to those obtained from predicate devices. There's no human 'in the loop' acting as an interpreter of the primary output of the calibration verification material, rather they are users of the material within a chemistry system.
7. The type of ground truth used
The "ground truth" for this study is essentially the expected or established performance/linearity of the predicate devices. The study aims to show that the new device performs similarly. The predicate devices themselves are accepted calibration verification materials, and their performance serves as the benchmark against which the new device is compared. The underlying "truth" for any calibration verification material lies in the precisely manufactured and quantified concentrations of the analytes within the solutions.
8. The sample size for the training set
Not applicable. This device is not an AI/ML model that requires a "training set." It is a chemical reagent product used for calibration verification.
9. How the ground truth for the training set was established
Not applicable, as there is no training set for this type of device.
§ 862.1660 Quality control material (assayed and unassayed).
(a)
Identification. A quality control material (assayed and unassayed) for clinical chemistry is a device intended for medical purposes for use in a test system to estimate test precision and to detect systematic analytical deviations that may arise from reagent or analytical instrument variation. A quality control material (assayed and unassayed) may be used for proficiency testing in interlaboratory surveys. This generic type of device includes controls (assayed and unassayed) for blood gases, electrolytes, enzymes, multianalytes (all kinds), single (specified) analytes, or urinalysis controls.(b)
Classification. Class I (general controls). Except when intended for use in donor screening tests, quality control materials (assayed and unassayed) are exempt from the premarket notification procedures in subpart E of part 807 of this chapter subject to the limitations in § 862.9.