(110 days)
The CAPI 3 Hb A1c kit is intended for separation and quantification of the HbA1c glycated fraction of hemoglobin (in IFCC unit (mmol/mol) and NGSP unit (%)) in venous whole human blood, by capillary electrophoresis in alkaline buffer with the CAPILLARYS 3 TERA instrument of hemoglobin A1c is used as an aid in diagnosis of diabetes, as an aid to identify patients who may be at risk for developing diabetes mellitus, and for the monitoring of long-term blood glucose control in individuals with diabetes mellitus. The CAPI 3 Hb A1c kit is intended for in vitro Diagnostic Use Only.
The CAPILLARYS 3 TERA instrument uses the principle of capillary electrophoresis in free solution. With this technique, charged molecules are separated by their electrophoretic mobility in an alkaline buffer with a specific pH. Separation occurs according to the electrolyte pH and electroosmotic flow. The CAPILLARYS 3 TERA instrument has silica capillaries functioning in parallel allowing 12 simultaneous analyses of HbA1c quantification in a whole blood sample. A sample dilution with hemolysing solution is prepared and injected by aspiration at the anodic end of the capillary. A high voltage protein separation is then performed and direct detection of the hemoglobins is made at the cathodic end of the capillary at 415 nm, which is the absorbance wave length specific to hemoglobins. Before each run, the capillaries are washed with a wash solution and prepared for the next analysis with buffer. Direct detection provides accurate relative quantification of individual hemoglobin A1c fraction. In addition, the high resolution of CAPI 3 Hb A1c procedure allows the quantification of HbA1c even in the presence of labile HbA1c. carbamylated and acetylated hemoglobins, and major hemoqlobin variants. By using an alkaline pH buffer, normal and abnormal (or variant) hemoglobins are detected in the following order, from cathode to anode: A2/C, E, S/D, F, A0, other Hb (including minor Hb A1) and then A1c. The HbA1c concentrations are standardized and indicated in %HbA1c (DCCT/NGSP) and in mmol/mol (IFCC) units.
The provided text describes the 510(k) premarket notification for the CAPI 3 Hb A1c kit, a device used for measuring hemoglobin A1c levels. The document outlines the device's indications for use, technological characteristics, and performance data to demonstrate substantial equivalence to a legally marketed predicate device.
Here's an analysis of the acceptance criteria and the study that proves the device meets them, based on the provided text:
1. Table of Acceptance Criteria and Reported Device Performance
The "Special Controls for Diabetes Diagnosis Claim" section (Section 6) explicitly lists the acceptance criteria set by the FDA for devices of this type, along with how the device's performance data addresses these. While the document doesn't present a direct "acceptance criteria vs. reported performance" table, I can synthesize one from Section 6 and the preceding "Performance Data" (Section 5).
Acceptance Criteria and Reported Device Performance for CAPI 3 Hb A1c
| Acceptance Criteria (from Section 6) | Reported Device Performance (from Section 5) |
|---|---|
| 1. Standardization verification: Device must have initial and annual standardization verification by certifying glycohemoglobin standardization organization deemed acceptable by FDA. | c. Traceability, Stability (controls, calibrators, or methods): The CAPI 3 Hb A1c test standardization is traceable to the International Federation of Clinical Chemistry (IFCC) reference calibrators. The CAPI 3 Hb A1c assay is NGSP certified. The NGSP certification expires in one year. See the NGSP website for current certification at http://www.ngsp.org. |
| 2. Precision testing: Must use blood samples with concentrations near 5.0%, 6.5%, 8.0%, and 12% hemoglobin A1c. Testing must evaluate precision over a minimum of 20 days using at least 3 lots of the device and instruments, as applicable. | a. Precision / Reproducibility: The precision was evaluated based on CLSI guidelines EP05-A3. Four whole blood samples at targeted HbA1c concentrations of ~5%, ~6.5%, ~8%, and ~12% were used. The study included two quality control materials and two calibrators. Samples analyzed in duplicate on two capillaries per run on 3 instruments. The study used three lots of kits over 24 days, yielding a total of 1728 results over 72 days. The detailed tables (pages 8-12) show SD and CV values per sample (~5.2%, ~6.5%, ~8.1%, ~11.9% NGSP units) across within-capillary, between-capillary, between-run, between-day, between-lot, and between-instrument variability, demonstrating low CVs for all measured components, with total reproducibilities ranging from 1.0% to 2.0% CV (NGSP units). |
| 3. Accuracy testing: Must include a minimum of 120 blood samples that span the measuring interval of the new device and compare results of the new device to results of the standardized method. Results must demonstrate little or no bias versus the standardized method. | e. Comparison Studies: A method comparison study of 152 variant-free whole blood samples covering the measuring range (3.9% - 16.5% HbA1c) was evaluated. Results were compared to testing at an NGSP reference laboratory using the cleared HPLC HbA1c method (Automated Glycohemoglobin Analyzer HLC-712G8). The samples spanned extensively around decision points (e.g., 23% of samples 6.0%-6.5%, 24% from 6.5%-7.0%). Correlation coefficient (r) was 0.999. Slope was 1.014 (95% CI: 1.007 to 1.021). Y-intercept was -0.142 (95% CI: -0.197 to -0.087). Average bias (all samples) was -0.04 (-0.06 to -0.02)%. Bias at 6.5% was -0.05 (-0.07 to -0.03)%. These indicate very low bias and strong correlation. |
| 4. Total error (TE) evaluation: Must be evaluated using single measurements by the new device compared to the results of the standardized test method, and this evaluation must demonstrate a total error of less than or equal to 6%. | f. Total Error Calculations: Total error (TE) was calculated for four concentrations (5.2%, 6.5%, 8.1%, and 11.9% NGSP units) using the formula %TE= |
| 5. Interference testing: Must demonstrate that there is little to no interference from common hemoglobin variants, including Hemoglobin C, Hemoglobin D, Hemoglobin E, Hemoglobin A2, and Hemoglobin S. | g. Interferences: Hemoglobin Variant Study was performed using specific samples known to contain hemoglobin variants S, C, E, D, A2, and F. The samples were analyzed with a reference method (NGSP laboratory) and the CAPI 3 Hb A1c. The results show low relative % bias from the reference method for these variants across different HbA1c concentrations (e.g., Hb S: 1.6% at ~6.5%, 2.9% at ~9%; Hb C: -1.8% at ~6.5%, 3.9% at ~9%; Hb D: 1.0% at ~6.5%, 0.8% at ~9%; Hb E: 1.5% at ~6.5%, 1.2% at ~9%; Hb A2: 0.7% at ~6.5%, 0.4% at ~9%). "No interference has been observed with HbA1c fraction quantification due to the presence of major abnormal hemoglobins Hb S (≤ 40.8 %), Hb C (≤ 37.6 %), Hb D (≤ 41.3 %) and Hb E (≤ 26.8 %)." |
| 6. Warning statement for HbF or other low-frequency variants (if applicable): When assay interference from Hemoglobin F or interference with other hemoglobin variants with low frequency in the population is observed, a warning statement must be placed in a black box and must appear in all the labeling material for these devices describing the interference and any affected population. | g. Interferences: "A significant negative interference has been observed with fetal hemoglobin (HbF) concentrations > 23%. HbA1c results are invalid for patients with high amounts of HbF (>23%) including those with known Hereditary Persistence of Fetal Hemoglobin." (Implies this will be addressed in labeling, as per acceptance criteria). |
2. Sample Size Used for the Test Set and Data Provenance
-
Precision/Reproducibility Study (Section 5a):
- Sample Size: Four different whole blood samples at specific targeted HbA1c concentrations (~5%, ~6.5%, ~8%, ~12%) were used. These were analyzed in duplicate on two capillaries per run on 3 instruments. The study used three lots of kits over 24 days, resulting in a total of 1728 individual results.
- Data Provenance: The document does not explicitly state the country of origin for these samples. It implies they are clinical samples used in a laboratory setting for reproducibility testing within the manufacturer's or contracted facility. The study design (CLSI guidelines EP05-A3) suggests a prospective, experimentally controlled setup.
-
Comparison Studies (Accuracy) (Section 5e):
- Sample Size: 152 variant-free whole blood samples.
- Data Provenance: The samples covered the measuring range and spanned decision points for diabetes diagnosis. The comparison was against results from an NGSP reference laboratory using a cleared HPLC HbA1c method. The specific origin (e.g., country) or whether these were retrospectively or prospectively collected is not stated, but the nature of a comparison study with a reference method implies a real-world clinical sample set.
-
Interference Studies (Section 5g):
- Sample Size: Two different whole blood samples (one near cut-off, one with elevated HbA1c) were used for endogenous and drug interference testing. For hemoglobin variant interference, a number of specific samples were used (e.g., 20 Hb A2 samples, 19 Hb F samples, 24 each for Hb S, C, D, and E samples).
- Data Provenance: Not explicitly stated regarding country or retrospective/prospective. The description suggests these were specific, characterized samples obtained for interference testing, likely in a controlled laboratory environment.
3. Number of Experts Used to Establish Ground Truth for the Test Set and Qualifications
- For this in-vitro diagnostic device (HbA1c measurement), "ground truth" is typically established by reference methods or standardized laboratory procedures, not by expert consensus in the way imaging AI models are.
- Comparison Study (Section 5e): The ground truth for the 152 samples was established by "testing performed at a NGSP reference laboratory using the cleared HPLC HbA1c method (Automated Glycohemoglobin Analyzer HLC-712G8)." This is the gold standard for HbA1c measurement and implies the highest level of analytical accuracy for the measured values.
- The document does not mention human experts establishing ground truth or their qualifications; it relies on a technical reference standard.
4. Adjudication Method for the Test Set
- Adjudication methods (like 2+1, 3+1) are typically used in subjective interpretation tasks (e.g., radiology image reading) where multiple human readers contribute to a consensus ground truth.
- For quantitative lab tests like HbA1c, adjudication is not applicable in the same sense. The "ground truth" is the result obtained from a single, highly accurate, standardized reference method, which is considered definitive. Any comparison is statistical, typically regressing one device's results against the reference method's results.
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
- No, an MRMC comparative effectiveness study was not done.
- MRMC studies are relevant for medical imaging devices where human readers interpret images, and the AI system is an "assistant" in that interpretive task.
- This device is an automated in-vitro diagnostic assay, meaning it directly measures a biomarker. It's not an AI system designed to assist human interpretation of complex visual data. Therefore, the concept of "human readers improving with AI vs. without AI assistance" does not apply here.
6. Standalone (Algorithm Only Without Human-in-the-Loop Performance)
- Yes, the primary performance shown is "standalone" performance.
- The CAPI 3 Hb A1c kit, when run on the CAPILLARYS 3 TERA instrument, provides a direct quantitative measurement. The performance data presented (precision, linearity, comparison, total error, interferences) are all reflective of the device's analytical performance on its own, without human real-time intervention for result calculation or interpretation beyond standard laboratory procedures for operating the instrument and reporting results.
- The output is a numerical value (%HbA1c or mmol/mol), not an image or diagnosis requiring human interpretation in the loop with an AI algorithm.
7. Type of Ground Truth Used
- The ground truth for the comparison studies (accuracy) was established by a standardized reference method (HPLC HbA1c) run at an NGSP reference laboratory.
- This is a form of analytical reference standard, considered the most accurate and reliable method for determining true HbA1c concentrations in the samples.
- For precision and interference studies, the "ground truth" is inherent to the specific sample concentrations being tested, often prepared or characterized against
these same reference methodologies.
8. Sample Size for the Training Set
- This information is not provided because this is not an AI/ML (Artificial Intelligence/Machine Learning) device requiring a "training set."
- The CAPI 3 Hb A1c kit is an in-vitro diagnostic device based on the principle of capillary electrophoresis, a well-established analytical chemistry technique. Its "performance" is governed by its chemical reagents, instrument design, and physical principles, not by a learned algorithm from data.
- Therefore, there is no "training set" in the context of machine learning. The development and validation of such a device involve optimization of reagents, hardware, and software parameters, followed by rigorous analytical performance validation studies (as detailed in Section 5).
9. How the Ground Truth for the Training Set was Established
- Not applicable, as there is no "training set" for this type of device (see point 8 above).
§ 862.1373 Hemoglobin A1c test system.
(a)
Identification. A hemoglobin A1c test system is a device used to measure the percentage concentration of hemoglobin A1c in blood. Measurement of hemoglobin A1c is used as an aid in the diagnosis of diabetes mellitus and as an aid in the identification of patients at risk for developing diabetes mellitus.(b)
Classification. Class II (special controls). The special controls for this device are:(1) The device must have initial and annual standardization verification by a certifying glycohemoglobin standardization organization deemed acceptable by FDA.
(2) The premarket notification submission must include performance testing to evaluate precision, accuracy, linearity, and interference, including the following:
(i) Performance testing of device precision must, at a minimum, use blood samples with concentrations near 5.0 percent, 6.5 percent, 8.0 percent, and 12 percent hemoglobin A1c. This testing must evaluate precision over a minimum of 20 days using at least three lots of the device and three instruments, as applicable.
(ii) Performance testing of device accuracy must include a minimum of 120 blood samples that span the measuring interval of the device and compare results of the new device to results of a standardized test method. Results must demonstrate little or no bias versus the standardized method.
(iii) Total error of the new device must be evaluated using single measurements by the new device compared to results of the standardized test method, and this evaluation must demonstrate a total error less than or equal to 6 percent.
(iv) Performance testing must demonstrate that there is little to no interference from common hemoglobin variants, including Hemoglobin C, Hemoglobin D, Hemoglobin E, Hemoglobin A2, and Hemoglobin S.
(3) When assay interference from Hemoglobin F or interference with other hemoglobin variants with low frequency in the population is observed, a warning statement must be placed in a black box and must appear in all labeling material for these devices describing the interference and any affected populations.