(257 days)
The BUN Reagent is to be used for the quantitative determination of urea nitrogen in serum, plasma and urine on the Beckman SYNCHRON CX3® System to aid in the diagnosis of renal function and pre renal disease states, such as cardiac decompensation and others.
The Device is a reagent containing sufficient Urease, surfactants and other ingredients necessary for optimum system operation on the Beckman Synchron CX3® Analyzer.
Acceptance Criteria and Device Performance Study for GenChem, Inc. BUN Reagent
This document outlines the acceptance criteria and the results of a study demonstrating the performance of the GenChem, Inc. Electrode, Ion Specific, Urea Nitrogen (BUN) Reagent (K040973).
1. Table of Acceptance Criteria and Reported Device Performance
| Performance Characteristic | Acceptance Criteria (Implicit) | Reported Device Performance |
|---|---|---|
| Precision/Reproducibility | Based on NCCLS EP3-T guidelines; generally low %CV for various BUN levels. | Serum 1 (7.1 mg/dL): Within Run %CV = 9.1%, Total %CV = 9.4% |
| Serum 2 (35.4 mg/dL): Within Run %CV = 1.8%, Total %CV = 1.9% | ||
| Serum 3 (63.8 mg/dL): Within Run %CV = 0.8%, Total %CV = 1.3% | ||
| Urine 1 (21.7 mg/dL): Within Run %CV = 4.1%, Total %CV = 3.8% | ||
| Urine 2 (112.2 mg/dL): Within Run %CV = 0.7%, Total %CV = 1.1% | ||
| Linearity/Reportable Range | Demonstrates linearity across the stated analytical range with a high correlation coefficient and low intercept/slope deviation from ideal. | Slope = 0.995, Intercept = -0.12, Standard Error of Estimate = 0.49, r² = 1.00 |
| Analytical Range: 2 - 150 mgN/dL (Normal); 150 - 300 mgN/dL (ORDAC*) | ||
| Sensitivity | Observed sensitivity limit to be ≤ claimed limit. | Claimed Limit = 2 mg/dL |
| Observed Sensitivity Limit (3 SD) = 1.43 mg/dL | ||
| Analytical Specificity | No adverse effect from common interferents at specified concentrations. | Hemoglobin (up to 500 mg/dL), Bilirubin (up to 20 mg/dL), Lipemia (up to 1800 mg/dL) showed no adverse effect on 15 mg/dL BUN sample. |
| Method Comparison (Correlates with Predicate Device) | Strong correlation with the predicate device across the analytical range, indicated by high R² value, slope close to 1, and intercept close to 0. | Serum (N=80): Intercept = -0.3, Slope = 0.995, R² = 0.999 |
| Plasma (N=80): Intercept = -0.2, Slope = 0.989, R² = 0.998 | ||
| Urine (N=79): Intercept = 0.9, Slope = 0.979, R² = 1.000 |
*ORDAC: Off-Range Dilution and Concentration
2. Sample Size and Data Provenance (Test Set)
-
Precision/Reproducibility:
- Sample Size: 60 replicates for each of 5 samples (3 serum, 2 urine) (N=60 for each sample type). Assayed twice per day in triplicate on 10 different days over a 30-day period.
- Data Provenance: Not explicitly stated, but likely from laboratory controlled samples (control sera, diluted urine pools). This is typically a retrospective analysis of controlled lab materials.
-
Linearity/Assay Reportable Range:
- Sample Size: Analyzed 7 commercially available linearity standards ranging from 0 to 158 mg/dL in triplicate.
- Data Provenance: Commercially available linearity standards, typically retrospective analysis in a controlled lab setting.
-
Sensitivity:
- Sample Size: 21 replicates of a diluted serum control.
- Data Provenance: Diluted serum control, typically retrospective analysis in a controlled lab setting.
-
Analytical Specificity:
- Sample Size: Not explicitly stated as a numerical sample size but involved testing stock samples with varying concentrations of interferents (Hemoglobin, Bilirubin, Lipemia) against a base pool.
- Data Provenance: Laboratory-prepared stock solutions and base pool, typically retrospective analysis in a controlled lab setting.
-
Patient Comparison (Method Comparison):
- Sample Size: 80 serum specimens, 80 plasma specimens, and 79 urine specimens.
- Data Provenance: "Collected from adult patients." This indicates prospective or retrospective patient sample collection, but the context generally implies real patient samples that were de-identified and used for the study. The country of origin is not specified.
3. Number of Experts Used to Establish Ground Truth for the Test Set and Their Qualifications
The studies described in the provided text are for an in vitro diagnostic reagent and system. For such devices, "ground truth" is typically established by comparative analysis with a well-characterized reference method or a legally marketed predicate device, rather than expert interpretation of images or clinical data.
- For Precision, Linearity, Sensitivity, and Analytical Specificity: Ground truth is established by the inherent properties of the reference materials used (e.g., certified control sera, linearity standards with known concentrations) or by accepted laboratory analytical methods. No external "experts" (like radiologists) are involved in establishing this type of ground truth.
- For Patient Comparison: The "ground truth" for the test set (GenChem BUN reagent) is the measurement obtained from the predicate device (Beckman BUN reagent for the CX3). The "experts" in this context would be the skilled laboratory technicians operating the instruments, but their role is to accurately perform the assays, not to interpret an uncertain "ground truth."
4. Adjudication Method for the Test Set
Adjudication methods like 2+1 or 3+1 are typically used in studies where subjective interpretation (e.g., imaging reads) is involved and discrepancies between readers need to be resolved. For an IVD reagent, data is quantitative and is compared statistically. Therefore, no adjudication method as typically understood in image-based studies was used. The comparison was a direct statistical correlation between the test device and the predicate device's quantitative results.
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
No, a multi-reader multi-case (MRMC) comparative effectiveness study was not performed. This type of study is relevant for diagnostic devices that involve human interpretation of results (e.g., radiologists reading images and improved performance with AI assistance). The GenChem BUN Reagent is an automated in vitro diagnostic assay where the output is a quantitative value, not subject to human interpretation in the same way. Therefore, the concept of "human readers improve with AI vs without AI assistance" does not apply here.
6. Standalone (Algorithm Only Without Human-in-the-Loop Performance)
Yes, all performance studies described are essentially standalone performance studies for the reagent on the Beckman CX3® Synchron Analyzer. The analytical performance characteristics (Precision, Linearity, Sensitivity, Analytical Specificity, and Method Comparison) measure the reagent's performance as an "algorithm only" (in the sense of an automated test system yielding a quantitative result without direct human interpretation influencing the result itself). The human "in the loop" would be operating the instrument and performing quality control, but not providing an interpretation that the algorithm augments or replaces.
7. Type of Ground Truth Used
The primary ground truth used for the method comparison study (patient comparison) was the predicate device's measurement (Beckman BUN reagent for the CX3). For other performance characteristics like precision, linearity, and sensitivity, the ground truth was established by reference materials, certified standards, or accepted laboratory methods (e.g., known concentrations of linearity standards, the mean value for precision calculations). This is synonymous with a form of expert consensus if the reference methods are considered the established gold standard.
8. Sample Size for the Training Set
The provided documentation describes performance studies for regulatory submission (510(k)). These are analytical validation studies, not machine learning model development. Therefore, there is no explicit "training set" in the context of an AI/ML algorithm. The "training" in this context refers to the development and optimization of the reagent and its methods, which would typically be done by the manufacturer through various iterations using proprietary internal samples and methodologies, but these are not disclosed as a "training set" in a regulatory submission for an IVD reagent.
9. How the Ground Truth for the Training Set Was Established
As there is no "training set" in the context of an AI/ML algorithm, this question is not applicable. The development of chemical reagents and analytical methods for IVD devices involves extensive R&D and optimization, where "ground truth" is established against known chemical properties, reference methods, and clinical needs during the development phase.
§ 862.1770 Urea nitrogen test system.
(a)
Identification. A urea nitrogen test system is a device intended to measure urea nitrogen (an end-product of nitrogen metabolism) in whole blood, serum, plasma, and urine. Measurements obtained by this device are used in the diagnosis and treatment of certain renal and metabolic diseases.(b)
Classification. Class II.