(45 days)
The VALIDATE Chem 4 Calibration Verification Test Set is used by trained laboratory professionals for quantitatively verifying calibration, validating reportable ranges, and determining linearity of automated, semi-automated and manual chemistry systems for the following analytes: alkaline phosphatase, alanine aminotransferase, amylase, aspartate aminotransferase, creatine kinase, gamma-glutamyl transferase, lactate dehydrogenase, lipase, total bilirubin and direct bilirubin.
VALIDATE Chem 4 Calibration Verification Test Set contains purified enzymes in a solution of bovine albumin stabilized with ethylene glycol. Multiple levels are provided to establish the relationship between theoretical operation and actual performance of each of the included analytes. Each set contains six (6) levels, including zero. Each bottle contains 5 milliliters.
This document describes the premarket notification (510(k)) for the VALIDATE Chem 4 Calibration Verification Test Set. The primary goal of the study was to demonstrate substantial equivalence to predicate devices, not to establish specific acceptance criteria for standalone performance and then prove the device meets them. Instead, the "acceptance criteria" are implied by the demonstration of strong correlation and similar regression characteristics compared to already-marketed equivalent devices.
Here's an analysis based on the provided text:
1. Table of Acceptance Criteria and Reported Device Performance
The provided document doesn't explicitly state quantitative acceptance criteria in the typical sense (e.g., "sensitivity must be >X%"). Instead, the performance is demonstrated through comparison with predicate devices, using linear regression analysis and correlation coefficients. The implied "acceptance criteria" for substantial equivalence are high correlation coefficients (close to 1) and regression equations that indicate a similar relationship between the test set and the predicate.
Implied Acceptance Criteria for Substantial Equivalence and Reported Device Performance:
| Analyte | Implied Acceptance Criterion (Correlation Coefficient r near 1) | VALIDATE Chem 4 vs Predicate 1 (DOCUMENT Enzyme Linearity Test Set) | VALIDATE Chem 4 vs Predicate 2 (DOCUMENT Total/Direct Bilirubin Test Set) | VALIDATE Chem 4 vs Predicate 3 (Sigma Diagnostics Lipase LIN-TROL) |
|---|---|---|---|---|
| ALT | Ideally r > 0.99 (high correlation) | r = 0.999945 | N/A | N/A |
| AST | Ideally r > 0.99 (high correlation) | r = 0.99964 | N/A | N/A |
| GGT | Ideally r > 0.99 (high correlation) | r = 0.99995 | N/A | N/A |
| CK | Ideally r > 0.99 (high correlation) | r = 0.99113 | N/A | N/A |
| LD | Ideally r > 0.99 (high correlation) | r = 0.99971 | N/A | N/A |
| ALP | Ideally r > 0.99 (high correlation) | r = 0.99999 | N/A | N/A |
| AMY | Ideally r > 0.99 (high correlation) | r = 0.99998 | N/A | N/A |
| LIP | Ideally r > 0.99 (high correlation) | N/A (compared to specific Lipase predicate) | N/A | r = 0.99879 |
| Total Bilirubin | Ideally r > 0.99 (high correlation) | N/A (compared to specific Bilirubin predicate) | r = 0.99956 | N/A |
| Direct Bilirubin | Ideally r > 0.99 (high correlation) | N/A (compared to specific Bilirubin predicate) | r = 0.99627 | N/A |
Note: The regression equations (Y = intercept + slope(X)) are also part of the comparison, aiming for slopes close to 1 and intercepts close to 0, which further indicates equivalence. The table shows these for each comparison, and they generally align with this goal.
2. Sample Size Used for the Test Set and Data Provenance
- Sample Size: The document does not explicitly state the numerical sample size (number of test instances, runs, or measurements) for the test set. It mentions "pre-production lots of VALIDATE Chem 4 Calibration Verification Test Set" were used. For calibration verification devices, the "sample size" typically refers to the number of levels tested (which is 6, including zero, for the VALIDATE Chem 4 Test Set and its predicates) and the number of replicates performed at each level, but this detail is not provided.
- Data Provenance: The data comes from internal testing by Maine Standards Company, LLC, comparing their device against predicate devices on a "Roche Diagnostics Hitachi 911 instrument system." The country of origin of the data is implicitly the United States, given the submitter's address. The study is a prospective comparison of a new device against existing predicate devices.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts
This type of information (number of experts, their qualifications for establishing ground truth) is not applicable to this submission. This is a technical performance study for an in vitro diagnostic (IVD) calibration verification device, not a diagnostic imaging or clinical decision support device requiring human expert consensus for "ground truth." The "ground truth" for this device is the known concentrations or values of the analytes in the predicate devices, which are themselves established through standardized analytical methods.
4. Adjudication Method for the Test Set
Not applicable. As explained above, this is a technical performance study comparing analytical values, not a study requiring human adjudication for diagnostic discrepancies.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done
No, a MRMC comparative effectiveness study was not done. This type of study is typically performed for diagnostic devices where human readers interpret results (e.g., radiologists reading images) and the AI's impact on reader performance is being evaluated. This submission is for an in vitro diagnostic (IVD) calibration verification test set, which is an analytical tool, not a diagnostic interpretation tool requiring human reader involvement in the same way.
6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) Was Done
Yes, a standalone performance assessment was conducted. The entire study is essentially a standalone performance evaluation of the VALIDATE Chem 4 Calibration Verification Test Set. It evaluates the device's ability to correlate linearly with established predicate devices when run on a specific instrument system (Roche Diagnostics Hitachi 911), without human interpretation influencing the measurement outcomes. The "performance" here refers to its analytical characteristics (correlation, linearity) relative to the predicate devices.
7. The Type of Ground Truth Used
The "ground truth" for this study is not clinical outcomes or pathology. It is the known analytical values and linearity performance of the predicate devices. The predicate devices themselves are accepted standards for calibration verification and linearity assessment. Therefore, the "ground truth" is established by the analytical performance of the legally marketed predicate devices.
8. The Sample Size for the Training Set
Not applicable. This submission is for an IVD device that does not use machine learning or AI algorithms requiring a "training set." The device is a chemical reagent set, and its "performance" is determined by its inherent analytical properties when measured, not by being "trained" on data.
9. How the Ground Truth for the Training Set Was Established
Not applicable. As stated in point 8, there is no training set for this device.
§ 862.1660 Quality control material (assayed and unassayed).
(a)
Identification. A quality control material (assayed and unassayed) for clinical chemistry is a device intended for medical purposes for use in a test system to estimate test precision and to detect systematic analytical deviations that may arise from reagent or analytical instrument variation. A quality control material (assayed and unassayed) may be used for proficiency testing in interlaboratory surveys. This generic type of device includes controls (assayed and unassayed) for blood gases, electrolytes, enzymes, multianalytes (all kinds), single (specified) analytes, or urinalysis controls.(b)
Classification. Class I (general controls). Except when intended for use in donor screening tests, quality control materials (assayed and unassayed) are exempt from the premarket notification procedures in subpart E of part 807 of this chapter subject to the limitations in § 862.9.