(891 days)
Ortho-Count Calibration Kit for ORTHO CYTORONABSOLUTE Laser Flow Cytometry System is intended to be used to calibrate and verify calibration of the absolute lymphocyte counting function of the ORTHO CYTORONABSOLUTE.
Ortho-Count Calibration Kit for ORTHO CYTORONABSOLUTE Laser Flow Cytometry System, contains a Calibrator Bead Suspension for absolute lymphocyte count calibration and Verifier Bead Suspensions (I, II and III) to verify accuracy of absolute count calibration. The Calibrator Bead Suspension is manufactured as known "events"/unit volume (i.e. the Calibration Number) in the form of microparticle beads suspended in a proprietary buffer. The size of the microparticles has been selected to emulate the range of size of leukocytes detected by the ORTHO CYTORONABSOLUTE under specified gating conditions. The Verifier Bead Suspensions (I, II and III), are composed of materials identical to the Calibrator Bead Suspension and collectively provide a range of particle bead concentrations. The target values chosen approximate values expected from patients with low (Verifier I), normal (Verifier II) and high (Verifier III) white blood cell counts, The Verifiers are used to verify accuracy of the absolute respectively. lymphocyte count calibration immediately following calibration and are also used as daily quality control verification that the ORTHO CYTORONABSOLUTE is in calibration. A properly calibrated instrument will provide absolute counts within + 10% of the labeled events/uL for each Verifier supsension. In the context of immune status monitoring, absolute cell count is defined as the number of cells per unit volume. The ORTHO CYTORONABSOLUTE uses precision stepper motors to deliver diluted sample to the instrument's flow cell at a constant rate. Since constant flow rate means movement of a fixed volume per unit time, analysis time determines the volume of sample analyzed. Calibration of the ORTHO CYTORONABSOLUTE consists of setting analysis time such that a volume corresponding to 1 uL of undiluted sample is examined. Ortho-Count Calibrator Bead Suspension, representing a known number of particles per unit volume (i.e. events/uL) at the same dilution as used for testing samples, is counted until its "Calibration Number" is reached; the instrument is set to this analysis time. In subsequent analyses events are to passage of the equivalent of 1 uL of undiluted sample the instrument flowcell. As a result of this calibration, sample results can be read in absolute count, i.e. "events"/uL.
Here's a breakdown of the acceptance criteria and the study that proves the device meets them, based on the provided text:
Acceptance Criteria and Device Performance
The core acceptance criterion for the Ortho-Count Calibration Kit is that a properly calibrated ORTHO CYTORONABSOLUTE instrument will provide absolute counts within ±10% of the labeled events/uL for each Verifier suspension.
| Acceptance Criteria | Reported Device Performance |
|---|---|
| Absolute counts within ±10% of labeled events/uL for each Verifier suspension. | Verifier I: Mean across sites = 1693, Std. Dev. = 55, % CV = 3.2% (Compared to a target value, which is not explicitly stated but implied to be within 10% of the mean) |
| Verifier II: Mean across sites = 7748, Std. Dev. = 193, % CV = 2.5% | |
| Verifier III: Mean across sites = 15140, Std. Dev. = 409, % CV = 2.7% | |
| No significant differences in absolute lymphocyte counts attributable to instrument or calibration method when compared to predicate device/method. | Table 1: Ratios of Mean Method 1/JT2, Method 2/JT2, and Method 2/Method 1 generally hover around 1.0, with mean standard deviations around 0.04-0.05, indicating no significant differences. For example, the mean ratio of Meth2/Meth1 is 0.99 with a standard deviation of 0.05. |
| Inter-laboratory reproducibility. | Table 2: Overall Inter-Lab % CV values for Verifiers I, II, and III are 3.2%, 2.5%, and 2.7% respectively, which are comparable to within-laboratory precision, indicating good inter-laboratory reproducibility. |
| Strong correlation between direct ORTHO CYTORONABSOLUTE counts and those calculated from combined hematology and other flow cytometer/reagent results. | Figure 2: Regression analysis yields a slope of 0.89, a y-intercept of -31 ± 13, and a correlation coefficient (r) = 0.96. This indicates a strong positive correlation. |
| Utility as quality control indicators over time (60 days). | Figure 3: Shows "tight agreement" in measurements made using each Verifier Bead Suspension over 60 days, confirming suitability as an accurate quality control measure. |
Study Details
-
Sample size used for the test set and the data provenance:
- Table 1 (Comparison to Predicate Device): 10 donors. Data provenance not explicitly stated (e.g., country of origin), and it's not clear if it was retrospective or prospective.
- Table 2 (Inter-laboratory Reproducibility): Verifier Bead Suspensions (I, II, and III) were used across four different ORTHO CYTORONABSOLUTE instruments located in different regions of the United States. The frequency or total number of measurements for establishing "Mean" values for each verifier is not specified, but it's likely prospective for this component.
- Figure 1 (Consistency of Calibration/Cross-Calibration): 41 samples (whole blood panel). Provenance not explicitly stated beyond "different regions of the United States" for the instruments. The samples were handled "overnight at room temperature or shipped for next day delivery to each site," suggesting prospective collection for this study, though the source of the donors/patients is not specified.
- Figure 2 (Direct vs. Calculated Absolute Counts): Data points for CD19, CD3+CD8+, CD3+CD4+, and Total CD3+. The total number of individual samples is not specified, but the regression analysis implies a substantial number of data points for these subsets. Provenance not explicitly stated.
- Figure 3 (Quality Control Indicators): Counts/volume were measured on the three verifier suspensions twice per week over a period of 60 days for four ORTHO CYTORONABSOLUTE instruments.
-
Number of experts used to establish the ground truth for the test set and the qualifications of those experts:
- The document does not explicitly state the number or qualifications of experts used to establish ground truth for the test sets. Instead, the ground truth is established through:
- Comparison to a predicate device (Coulter JT2 calibrated with S-CAL Kit) (for Table 1).
- Labeled 'events'/uL for the Verifier Bead Suspensions themselves, which are manufactured values (for Table 2 and Figure 3).
- "Calculated from combined hematology and other flow cytometer/reagent results" (for Figure 2's comparative ground truth).
- The document does not explicitly state the number or qualifications of experts used to establish ground truth for the test sets. Instead, the ground truth is established through:
-
Adjudication method (e.g. 2+1, 3+1, none) for the test set:
- No adjudication method involving experts is described. The comparisons are quantitative measurements against established methods or known values.
-
If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:
- No MRMC comparative effectiveness study involving human readers (as in interpretation of images or complex data) was conducted. This device is a calibration kit for an automated instrument, not an AI diagnostic tool requiring human interpretation. Therefore, the concept of "human readers improve with AI" is not applicable here.
-
If a standalone (i.e. algorithm only without human-in-the-loop performance) was done:
- The study primarily focuses on the standalone performance of the calibration kit in ensuring the accuracy and reproducibility of the ORTHO CYTORONABSOLUTE instrument's absolute counting function. The ORTHO CYTORONABSOLUTE itself is an automated device, so its performance (when calibrated) is inherently "algorithm only" in terms of cell counting. The "human-in-the-loop" would be the technician operating the flow cytometer and applying the calibration kit.
-
The type of ground truth used (expert consensus, pathology, outcomes data, etc.):
- Known Reference Values: For the Verifier Bead Suspensions, the ground truth is the "labeled events/uL," which are manufactured and accurately determined "events"/unit volume.
- Predicate Device/Method: For the comparison of absolute lymphocyte counts (Table 1), the ground truth is established by a "Coulter JT2 calibrated using S-CAL Kit" which is the predicate device/method.
- Combined Method: For Figure 2, the ground truth is established by "values calculated from separate flow cytometers and automated hematology analyzers."
-
The sample size for the training set:
- The document does not describe a "training set" in the context of machine learning or AI. This is a calibration kit, and its performance is evaluated against established methods and internal consistency, not through an iterative training process for an algorithm.
-
How the ground truth for the training set was established:
- As there is no described training set for an algorithm, this question is not applicable. The kit itself is manufactured with known characteristics (e.g., "known 'events'/unit volume" for the Calibrator Bead Suspension).
§ 864.5220 Automated differential cell counter.
(a)
Identification. An automated differential cell counter is a device used to identify one or more of the formed elements of the blood. The device may also have the capability to flag, count, or classify immature or abnormal hematopoietic cells of the blood, bone marrow, or other body fluids. These devices may combine an electronic particle counting method, optical method, or a flow cytometric method utilizing monoclonal CD (cluster designation) markers. The device includes accessory CD markers.(b)
Classification. Class II (special controls). The special control for this device is the FDA document entitled “Class II Special Controls Guidance Document: Premarket Notifications for Automated Differential Cell Counters for Immature or Abnormal Blood Cells; Final Guidance for Industry and FDA.”