(42 days)
The Calcium2 assay is used for the quantitation of calcium in human serum, plasma, or urine on the ARCHITECT c System.
Calcium measurements are used in the diagnosis and treatment of parathyroid disease, a variety of bone diseases, chronic renal disease and tetany (intermittent muscular contractions or spasms).
The Calcium2 assay is an automated clinical chemistry assay. Arsenazo III dye reacts with calcium in an acid solution to form a blue-purplex. The color developed is measured at 660 nm and is proportional to the calcium concentration in the sample.
Methodology: Arsenazo III
The provided text describes the performance validation of the Calcium2 assay, specifically for its use in quantitating calcium in human serum, plasma, or urine on the ARCHITECT c System. This is a Class II medical device (Calcium test system), regulated under 21 CFR 862.1145, product code CJY.
Here's an analysis based on the request:
Device: Calcium2 assay
Intended Use: Quantitation of calcium in human serum, plasma, or urine on the ARCHITECT c System. Used in the diagnosis and treatment of parathyroid disease, various bone diseases, chronic renal disease, and tetany.
1. Table of Acceptance Criteria and Reported Device Performance
The document doesn't explicitly outline a pre-defined table of "acceptance criteria" against which each test result is measured, as would be common for AI/ML performance studies. Instead, it presents various performance characteristics with their measured values, often referencing CLSI guidelines for the methodology. The "acceptance" can be inferred from the reported values and the conclusion of substantial equivalence. For quantitative assays like this, performance is typically assessed against established industry standards for precision, accuracy, linearity, and interference.
However, we can infer some "acceptance criteria" implicitly from the context of "acceptable performance" and the ranges presented.
| Performance Characteristic | Acceptance Criteria (Implicit/Standard) | Reported Device Performance |
|---|---|---|
| Reportable Interval (Serum/Plasma) | ||
| Analytical Measuring Interval (AMI) | Defined as the range demonstrating acceptable linearity, imprecision, and bias. | 2.0 – 24.0 mg/dL |
| Reportable Interval | Extends from LoD to upper AMI. | 0.2 – 24.0 mg/dL |
| Reportable Interval (Urine) | ||
| Analytical Measuring Interval (AMI) | Defined as the range demonstrating acceptable linearity, imprecision, and bias. | 2.0 – 24.0 mg/dL |
| Extended Measuring Interval (EMI) | Requires acceptable dilution performance (recovery, imprecision). | 24.0 – 96.0 mg/dL (Dilution recovery within 100% ± 10%, imprecision ≤ 5 %CV for automated dilution, ≤ 6 %CV for manual dilution) |
| Reportable Interval | Extends from LoD to upper EMI. | 0.2 – 96.0 mg/dL |
| Precision (Within-Laboratory - Serum, 20-Day) | ||
| %CV (Control Level 1) | Implied to be acceptable based on CLSI EP05-A3 guidance. | Within-Run: 0.8%; Within-Laboratory: 0.8% (Range 0.8-0.9%) |
| %CV (Control Level 2) | Implied to be acceptable based on CLSI EP05-A3 guidance. | Within-Run: 0.7%; Within-Laboratory: 0.7% (Range 0.6-0.9%) |
| %CV (Panel C, highest conc.) | Implied to be acceptable based on CLSI EP05-A3 guidance. | Within-Run: 0.6%; Within-Laboratory: 0.9% (Range 0.9-0.9%) |
| Precision (System Reproducibility - Serum) | ||
| %CV (Control Level 1) | Implied to be acceptable based on CLSI EP05-A3 guidance. | Repeatability: 0.9%; Within-Laboratory: 1.1%; Reproducibility: 1.4% |
| %CV (Panel A, lowest conc.) | Implied to be acceptable based on CLSI EP05-A3 guidance. | Repeatability: 1.5%; Within-Laboratory: 1.5%; Reproducibility: 2.6% |
| Accuracy (Bias) | ||
| Bias against NIST SRM 956d | Implied to be acceptable. | Ranged from -3.7% to -0.6% |
| Lower Limits of Measurement (Serum) | ||
| LoB | 95th percentile from zero-analyte samples. | 0.1 mg/dL |
| LoD | Lowest concentration detected with 95% probability. | 0.2 mg/dL |
| LoQ | Lowest concentration at which max 20%CV met. | 0.4 mg/dL |
| Interference (Serum, Endogenous & Exogenous) | ||
| No significant interference | Interference within ± 5%. | Most substances showed no significant interference (e.g., Bilirubin up to 40 mg/dL, Hemoglobin up to 1000 mg/dL, Acetaminophen up to 160 mg/L). One exception: Ca-dobesilate at 60 mg/L showed 6% interference (95% CI: 5%, 6%). |
| Method Comparison (Serum & Urine) | ||
| Correlation Coefficient | Implied to be close to 1.00 for equivalence. | Serum: 1.00; Urine: 1.00 |
| Slope | Implied to be close to 1.00 for equivalence. | Serum: 1.02; Urine: 0.96 |
| Intercept | Implied to be close to 0.00 for equivalence. | Serum: -0.1; Urine: 0.1 |
| Dilution Verification (Urine) | ||
| %Recovery | Within 100% ± 10%. | Acceptable performance demonstrated. |
| Imprecision (Automated Dilution) | ≤ 5 %CV. | ≤ 5 %CV. |
| Imprecision (Manual Dilution) | ≤ 6 %CV. | ≤ 6 %CV. |
2. Sample Size Used for the Test Set and the Data Provenance
The provided information is for an in-vitro diagnostic (IVD) test, not an AI/ML model for image analysis. Therefore, the "test set" and "training set" terminology from an AI/ML perspective doesn't directly apply in the same way. Instead, the studies involved various types of samples (controls, panels, human serum/plasma/urine samples) tested under specific experimental designs.
- Precision Studies (Serum & Urine, 20-Day):
- Sample Size: 80 replicates per sample type (2 controls, 3 human panels) for each of the 3 reagent lot/calibrator lot/instrument combinations in the 20-day study. (e.g., 5 samples * 80 replicates = 400 total data points per combination shown in the table).
- Data Provenance: Not explicitly stated (e.g., country of origin). The studies are "within-laboratory" meaning they were conducted in a controlled lab setting, likely at the manufacturer's site or a designated testing facility. The nature of these samples (human serum/urine panels) suggests they are likely representative, but the specific collection method or origin (retrospective/prospective collection from patients) is not detailed.
- System Reproducibility Studies (Serum & Urine):
- Sample Size: 90 replicates per sample type (2 controls, 3 human panels).
- Data Provenance: Same as above, not explicitly stated.
- Accuracy Study:
- Sample Size: Not explicitly stated for replicates, but involved 2 concentrations of NIST SRM 956d standard.
- Data Provenance: NIST standard reference material, a highly controlled and traceable source.
- Lower Limits of Measurement Studies (Serum & Urine):
- Sample Size: n ≥ 60 replicates for zero-analyte and low-analyte level samples for LoB, LoD, and LoQ determination.
- Data Provenance: Not explicitly stated.
- Linearity Studies (Serum & Urine):
- Sample Size: Not explicitly stated how many points were measured across the range, but demonstrated linearity from 2.0 to 24.0 mg/dL.
- Data Provenance: Not explicitly stated.
- Interference Studies (Serum & Urine):
- Sample Size: Not explicitly stated how many replicates, but each substance was tested at 2 levels of the analyte.
- Data Provenance: Not explicitly stated.
- Method Comparison Studies (Serum & Urine):
- Sample Size: 120 serum samples, 112 urine samples.
- Data Provenance: Patient samples. Not explicitly stated if retrospective or prospective collection.
- Tube Type Equivalence Study (Serum):
- Sample Size: Samples from 78 donors.
- Data Provenance: Donor samples. Not explicitly stated if retrospective or prospective.
- Dilution Verification (Urine):
- Sample Size: 5 samples prepared with calcium stock. Replicates for dilution recovery and imprecision were performed but the exact number isn't specified beyond "demonstrated acceptable performance."
- Data Provenance: Prepared samples.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and the Qualifications of Those Experts
This type of IVD device (quantitative chemical assay) relies on objective measurement against a known reference (e.g., NIST standard) or internal validated methods, rather than subjective expert consensus. Therefore, "ground truth" is established through highly controlled laboratory procedures and certified reference materials, not typically through multiple human experts adjudicating results. The "experts" involved are likely laboratory scientists and metrologists operating under strict quality systems (CLSI guidelines, ISO standards).
4. Adjudication Method (e.g. 2+1, 3+1, none) for the Test Set
Not applicable for a quantitative chemical assay. Results are objectively measured by the instrument and verified through calibration and quality control. There is no subjective interpretation requiring adjudication.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance
Not applicable. This is an in-vitro diagnostic test, not an AI/ML-driven imaging or diagnostic tool that assists human readers.
6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done
This is a standalone quantitative assay. The Calcium2 device automatically measures calcium concentration. Human involvement is in sample preparation, loading, and interpreting the numerical output. Its performance is evaluated entirely as an automated system.
7. The Type of Ground Truth Used (expert consensus, pathology, outcomes data, etc.)
The ground truth for this device's performance validation is established primarily through:
- Reference Materials: For accuracy (e.g., NIST SRM 956d for bias estimation).
- Known Concentrations: For linearity, lower limits of measurement, and dilution verification, samples are prepared with known concentrations of calcium or spiked with calcium.
- Comparative Methods: For method comparisons, the "ground truth" is implicitly the performance of the legally marketed predicate device (Abbott Clinical Chemistry Architect/Aeroset Calcium, K062855), which serves as the comparator.
- Statistical Models/Guidelines: CLSI guidelines (EP05-A3, EP09c, EP17-A2, EP06, EP07, EP34) provide the statistical framework and methodology for validating performance characteristics like precision, linearity, and interference.
8. The Sample Size for the Training Set
Not applicable in the typical AI/ML sense. This is a chemistry assay, not a machine learning model that undergoes a "training" phase with a large dataset. The "training" for such a device involves rigorous engineering, chemical formulation, and calibration development, rather than data-driven model training.
9. How the Ground Truth for the Training Set Was Established
As above, "training set" is not a concept for this type of device. The accuracy and reliability of the assay are built into its chemical design, reagent formulation, and instrument calibration process. These processes rely on highly accurate reference standards (like NIST) and established analytical chemistry principles to ensure the device measures calcium correctly across its analytical range.
§ 862.1145 Calcium test system.
(a)
Identification. A calcium test system is a device intended to measure the total calcium level in serum. Calcium measurements are used in the diagnosis and treatment of parathyroid disease, a variety of bone diseases, chronic renal disease and tetany (intermittent muscular contractions or spasms).(b)
Classification. Class II.