(71 days)
MultiSET is an accessory software to in vitro diagnostic reagents employing CD45 gating for identification and enumeration of lymphocyte subsets in human peripheral blood.
Becton Dickinson MultiSET software is a three-color, direct immunofluorescence method for identifying and enumerating percentages of lymphocyte subsets in erythrocyte-lysed whole blood. The software is an accessory to reagent products which use CD45 gating for immunophenotyping by flow cytometry.
Both the manual CELLQuest software and the automated MultiSET software rely on a lymphocyte gate drawn for the CD45+ leucocytes with low side scatter. Lymphocyte subsets as a percent of total lymphocytes are then identified as a proportion of events included in the lymphocyte gate.
Here's an analysis of the provided text regarding the acceptance criteria and study for the MultiSET software:
1. Table of Acceptance Criteria and Reported Device Performance
The provided text does not explicitly state specific numerical acceptance criteria for the MultiSET software. Instead, the primary acceptance criterion appears to be "equivalent performance" to the predicate device (CELLQuest software).
| Acceptance Criteria Category | Acceptance Criteria (Implicit) | Reported Device Performance |
|---|---|---|
| Accuracy | Equivalent results to manual CELLQuest software. | "Accuracy data demonstrated that MultiSET software provides equivalent results to the manual CELLQuest software." |
| Safety and Effectiveness | As safe and effective as the predicate device. | "The results of the evaluation studies demonstrate that the device is as safe and effective as the predicate device." |
2. Sample Size Used for the Test Set and Data Provenance
- Sample Size: 41 donors
- 17 normal
- 24 abnormal
- Data Provenance:
- Country of Origin: Not explicitly stated, but the testing was performed at "Becton Dickinson Immunocytometry Systems laboratories in San Jose, California." This suggests the data was likely collected within the United States.
- Retrospective or Prospective: Not explicitly stated. However, given that the samples are described as "lysed whole blood samples from 41 donors" that "were analyzed by the automated and manual softwares," it suggests a retrospective analysis of previously collected samples, or at least a concurrent analysis of "fresh" samples. It doesn't imply a long-term prospective study.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and the Qualifications of Those Experts
The text does not mention the use of experts to establish ground truth in the traditional sense (e.g., for image interpretation). Instead, the "ground truth" for evaluating MultiSET was the results obtained from the predicate manual software, CELLQuest. As such, the "experts" were the users operating the CELLQuest software. No specific number or qualifications for these CELLQuest operators are provided.
4. Adjudication Method for the Test Set
No adjudication method is described. The comparison was directly between the automated MultiSET results and the manual CELLQuest results. There's no indication of multiple readers, consensus, or a third party resolving discrepancies.
5. If a Multi Reader Multi Case (MRMC) Comparative Effectiveness Study Was Done, If So, What Was the Effect Size of How Much Human Readers Improve with AI vs Without AI Assistance
No, a Multi Reader Multi Case (MRMC) comparative effectiveness study was not done. The study compared the automated software's performance to a manual software's performance, not human readers with and without AI assistance.
6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) Was Done
Yes, this was a standalone algorithm-only performance study. The MultiSET software (the algorithm) was compared directly against the results of the manual CELLQuest software (which involves human operation, but the comparison itself is between the output of the automated system and the output of the manual system). The "human-in-the-loop" for the MultiSET is minimal once the software is running, as it "automates the steps used to determine analysis gates."
7. The Type of Ground Truth Used
The "ground truth" used was the results generated by the predicate manual software, CELLQuest. This is a form of comparative ground truth against an existing, accepted method, rather than a direct biological or pathological truth.
8. The Sample Size for the Training Set
The text does not mention a separate training set or its sample size. The focus is solely on the "testing" or "performance" dataset of 41 donors. It's possible the software was developed and calibrated internally without a formally described training set in this summary, or that the concept of a distinct training set (as understood in modern AI development) was not as explicitly documented for this type of software development in 1996.
9. How the Ground Truth for the Training Set Was Established
Since a training set is not explicitly mentioned, the method for establishing its ground truth is also not described.
§ 864.5220 Automated differential cell counter.
(a)
Identification. An automated differential cell counter is a device used to identify one or more of the formed elements of the blood. The device may also have the capability to flag, count, or classify immature or abnormal hematopoietic cells of the blood, bone marrow, or other body fluids. These devices may combine an electronic particle counting method, optical method, or a flow cytometric method utilizing monoclonal CD (cluster designation) markers. The device includes accessory CD markers.(b)
Classification. Class II (special controls). The special control for this device is the FDA document entitled “Class II Special Controls Guidance Document: Premarket Notifications for Automated Differential Cell Counters for Immature or Abnormal Blood Cells; Final Guidance for Industry and FDA.”