(230 days)
The CAPILLARYS Hb A1c kit is intended for separation and quantification of the HbA1c glycated fraction of hemoglobin (in IFCC unit (mmol/mol) and NGSP unit (%)) in venous whole human blood, by capillary electrophoresis in alkaline buffer with the CAPILLARYS 2 FLEX-PIERCING instrument of hemoglobin A1c is used as an aid in diagnosis of diabetes, as an aid to identify patients who may be at risk for developing diabetes mellitus, and for the monitoring of long-term blood glucose control in individuals with diabetes mellitus. The CAPILLARYS Hb A 1c kit is intended for in vitro Diagnostic Use Only.
The CAPILLARYS 2 FLEX-PIERCING instrument uses the principle of capillary electrophoresis in free solution. With this technique, charged molecules are separated by their electrophoretic mobility in an alkaline buffer with a specific pH. Separation occurs according to the electrolyte pH and electroosmotic flow.
The CAPILLARYS 2 FLEX-PIERCING instrument has silica capillaries functioning in parallel allowing 8 simultaneous analyses of HbA1c quantification in a whole blood sample. A sample dilution with hemolysing solution is prepared and injected by aspiration at the anodic end of the capillary. A high voltage protein separation is then performed and direct detection of the hemoglobins is made at the cathodic end of the capillary at 415 nm, which is the absorbance wave length specific to hemoglobins. Before each run, the capillaries are washed with a wash solution and prepared for the next analysis with buffer.
Direct detection provides accurate relative quantification of individual hemoglobin A1c fraction. In addition, the high resolution of CAPILLLARYS Hb A1c procedure allows the quantification of HbA1c even in the presence of labile HbA1c, carbamylated and acetylated hemoglobins, and major hemoglobin variants.
By using an alkaline pH buffer, normal and abnormal (or variant) hemoglobins are detected in the following order, from cathode to anode: A2/C, E, S/D, F, A0, other Hb (including minor Hb A1) and then A1c.
The HbA1c concentrations are standardized and indicated in %HbA1c (DCCT/NGSP) and in mmol/mol (IFCC) units.
The provided text describes the performance data for the CAPILLARYS Hb A1c device, which is an in vitro diagnostic test for measuring HbA1c. The acceptance criteria and the study proving adherence to these criteria are detailed, primarily focusing on precision, linearity, and interference.
Here's the breakdown of the requested information:
1. Table of Acceptance Criteria and Reported Device Performance
| Acceptance Criteria (Stated Requirements) | Reported Device Performance |
|---|---|
| Device must have initial and annual standardization verification by certifying glycohemoglobin standardization organization deemed acceptable by FDA. | The CAPILLARYS Hb A1c test standardization is traceable to the International Federation of Clinical Chemistry (IFCC) reference calibrators. The CAPILLARYS Hb A1c assay is NGSP certified. The NGSP certification expires in one year. (http://www.ngsp.org) |
| Performance testing of device precision must, at a minimum, use blood samples with concentrations near 5.0%, 6.5%, 8.0%, and 12% hemoglobin A1c. Testing must evaluate precision over a minimum of 20 days using at least 3 lots of the device and instruments, as applicable. | Precision was evaluated using the CLSI EP5A3 guideline. Four whole blood samples with targeted HbA1c concentrations of ~5%, ~6.5%, ~8%, and ~12% were used. Eight different samples (4 human blood, 2 controls, 2 calibrators) were run in duplicate on two capillaries per run, two runs per day over 20 days per lot, using three lots (total 1440 results per sample over 60 days). The full precision results are provided in extensive tables for both mmol/mol and %HbA1c. The total reproducibility CVs for %HbA1c ranged from 1.2% to 2.0% across the different samples and instruments. |
| Performance testing of accuracy must include a minimum of 120 blood samples that span the measuring interval of the new device and compare results of the new device to results of the standardized method. Results must demonstrate little or no bias versus the standardized method. | A method comparison study included 150 variant-free whole blood samples covering the measuring range (4.4% to 16.6% HbA1c). Results were compared to an NGSP reference laboratory using the cleared HPLC HbA1c method (Tosoh Automated Glycohemoglobin Analyzer HLC-712G8). The correlation coefficient (r) was 0.999. Average bias for all samples was -0.07% (-0.08 to -0.05). Bias at the normal range (6.5%) was -0.09% (-0.11 to -0.07). These values indicate very low bias. |
| Total error of the new device must be evaluated using single measurements by the new device compared to the results of the standardized test method, and this evaluation must demonstrate a total error of less than or equal to 6%. | Total Error (TE) was calculated for 4 concentrations (5.1%, 6.4%, 8.2%, and 12.2% HbA1c) using %Bias and %CV from the precision study. The reported TE% values were 5.9%, 4.1%, 2.8%, and 3.1% respectively, all of which are ≤ 6.0%. |
| Performance testing must demonstrate that there is little to no interference from common hemoglobin variants, including Hemoglobin C, Hemoglobin D, Hemoglobin E, Hemoglobin A2, and Hemoglobin S. | No interference was observed for HbA1c quantification due to the presence of major abnormal hemoglobins: Hb S (≤ 40.8%), Hb C (≤ 37.2%), Hb D (≤ 41.3%), and Hb E (≤ 37.0%). Levels of Hb A2 up to 7.7% did not interfere. Mean relative % Bias for these variants was very low. |
| When assay interference from Hemoglobin F or interference with other hemoglobin variants with low frequency in the population is observed, a warning statement must be placed in a black box and must appear in all the labeling material for these devices describing the interference and any affected population. | Levels of Hb F up to 23% in the blood sample do not interfere. If Hb F levels are higher than 23%, a warning message "Atypical profile – Possible quantitative interference if Hb F or variant > 23 %" is displayed, recommending further analysis. (This criterion is marked "NA" in the table, implying it might be covered by a general warning, rather than a specific black box for this device for this interference). |
2. Sample sizes used for the test set and the data provenance
- Precision Study:
- Sample Size: Four human blood samples, two controls, and two calibrators were used. Each of these 8 distinct samples was analyzed in duplicate on two capillaries per run, two runs per day over 20 days per lot, using three lots. This results in 1440 results per sample (8 samples * 2 replicates * 2 capillaries * 2 runs * 20 days * 3 lots, though the 1440 figure suggests a different multiplication, it's stated as 1440 results per sample meaning 8 * 1440 total measurements).
- Data Provenance: Not explicitly stated regarding country of origin. The study was based on CLSI (USA) guidelines. It's a prospective study as specified by the testing methodology (over 20 days, 3 lots, etc.).
- Accuracy (Comparison) Study:
- Sample Size: 150 variant-free whole blood samples.
- Data Provenance: Not explicitly stated regarding country of origin. This dataset was collected for the purpose of the comparison study, implying a prospective design for comparing to a reference method. The samples spanned the measuring range and were distributed around clinical decision points.
- Interference Study:
- Sample Size: For endogenous factors and drugs, two different whole blood samples were used (one near cut-off, one elevated HbA1c). Ten replicates were analyzed. For hemoglobin variants, specific numbers of samples were used for each variant: 20 for Hb A2, 20 for Hb F, 20 for Hb S, 20 for Hb C, 21 for Hb D, and 22 for Hb E.
- Data Provenance: Not explicitly stated. Assumed to be prospective.
3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts
The ground truth for this device (a quantitative diagnostic test) is established by reference methods and certified reference materials, not expert consensus.
- Precision: No "ground truth" experts in the conventional sense. The "ground truth" for the mean values (target concentrations) of the blood samples, controls, and calibrators would be established through highly standardized and traceable processes by the manufacturer or a reference lab according to CLSI guidelines.
- Accuracy (Comparison): The comparison was against a "cleared HPLC HbA1c method (Automated Glycohemoglobin Analyzer HLC-712G8) performed at a NGSP reference laboratory." An NGSP (National Glycohemoglobin Standardization Program) reference laboratory adheres to stringent guidelines for HbA1c measurement and provides traceability to the DCCT (Diabetes Control and Complications Trial) reference method and IFCC (International Federation of Clinical Chemistry and Laboratory Medicine). The "ground truth" here is the result from this highly standardized and certified reference method, not an individual expert's reading.
- Interference: "Ground truth" for interference studies is typically based on the known concentration of the interfering substance and the measured value using a validated method. For hemoglobin variants, it was also compared against an NGSP laboratory's reference method.
Therefore, the concept of "experts" in the traditional sense of medical image interpretation (like radiologists) doesn't directly apply here. The "expertise" is embedded in the scientific rigor of the reference methods and standardization programs.
4. Adjudication method (e.g. 2+1, 3+1, none) for the test set
Not applicable. This is a quantitative chemical assay, not a qualitative assessment like image interpretation that requires adjudication among human readers. The "ground truth" from the reference method is considered definitive.
5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance
Not applicable. This is a medical device for an automated quantitative laboratory test and does not involve human readers, AI assistance, or MRMC studies.
6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done
Yes, the CAPILLARYS Hb A1c device is an automated in vitro diagnostic instrument. The performance data presented (precision, linearity, accuracy, interference) are all standalone performance characteristics of the algorithm and instrument system without human intervention influencing the measurement result itself once the sample is loaded.
7. The type of ground truth used (expert consensus, pathology, outcomes data, etc)
The ground truth used for performance evaluation was:
- Reference methods from NGSP certified laboratories: For accuracy (comparison) and hemoglobin variant interference studies, the device's results were compared to those obtained from an NGSP reference laboratory using a cleared HPLC HbA1c method. NGSP certification ensures traceability to the primary international reference methods (DCCT/IFCC).
- Gravimetric/Volumetric preparation and known concentrations: For linearity and interference studies with endogenous substances and drugs, samples were likely prepared to specific, known concentrations, serving as the ground truth.
- Certified calibrators and controls: For precision studies, the 'true' values of the controls and calibrators are established through rigorous certification processes, making them the ground truth for evaluating reproducibility.
8. The sample size for the training set
This document describes premarket notification for a traditional IVD device, not an AI/ML-based device that typically undergoes "training" and "testing" phases in the same way. The instrument and its reagents are designed based on established chemical and electrophoretic principles. Therefore, there isn't a "training set" in the machine learning sense. The development of the assay and the robust validation typically leverage extensive historical data and scientific understanding, but it's not "training data" for a machine learning algorithm.
9. How the ground truth for the training set was established
As there isn't a "training set" in the AI/ML context, this question is not applicable. The performance is validated against rigorous clinical laboratory standards and reference methods as described above.
§ 862.1373 Hemoglobin A1c test system.
(a)
Identification. A hemoglobin A1c test system is a device used to measure the percentage concentration of hemoglobin A1c in blood. Measurement of hemoglobin A1c is used as an aid in the diagnosis of diabetes mellitus and as an aid in the identification of patients at risk for developing diabetes mellitus.(b)
Classification. Class II (special controls). The special controls for this device are:(1) The device must have initial and annual standardization verification by a certifying glycohemoglobin standardization organization deemed acceptable by FDA.
(2) The premarket notification submission must include performance testing to evaluate precision, accuracy, linearity, and interference, including the following:
(i) Performance testing of device precision must, at a minimum, use blood samples with concentrations near 5.0 percent, 6.5 percent, 8.0 percent, and 12 percent hemoglobin A1c. This testing must evaluate precision over a minimum of 20 days using at least three lots of the device and three instruments, as applicable.
(ii) Performance testing of device accuracy must include a minimum of 120 blood samples that span the measuring interval of the device and compare results of the new device to results of a standardized test method. Results must demonstrate little or no bias versus the standardized method.
(iii) Total error of the new device must be evaluated using single measurements by the new device compared to results of the standardized test method, and this evaluation must demonstrate a total error less than or equal to 6 percent.
(iv) Performance testing must demonstrate that there is little to no interference from common hemoglobin variants, including Hemoglobin C, Hemoglobin D, Hemoglobin E, Hemoglobin A2, and Hemoglobin S.
(3) When assay interference from Hemoglobin F or interference with other hemoglobin variants with low frequency in the population is observed, a warning statement must be placed in a black box and must appear in all labeling material for these devices describing the interference and any affected populations.