(50 days)
The Immunicon CellTracks Analyzer II is a semi-automated fluorescence microscope used to enumerate fluorescently labeled cells that are immuno-magnetically selected and aligned. The system is for in vitro diagnostic use when used in tandem with specimen preparation equipment and reagents that are legally marketed for in vitro diagnostic use with this device.
The CellTracks® Analyzer II is a semi-automated fluorescence microscope. The product consists of the CellTracks® Analyzer II, a dedicated computer loaded with CellTracks® software, monitor, keyboard and mouse. The CellTracks Analyzer II is for analysis of rare cells that are isolated from biological samples. It is used in conjunction with the CellTracks® AutoPrep System, which automates, standardizes and optimizes the sample preparation with specific reagent kits. The CellTracks Analyzer II is used in conjunction with the CellTracks AutoPrep System and specific reagent kits that contain a ferrofluid-based capture reagent and fluorescent reagents for the detection and characterization of the captured cells. The ferrofluid reagent consists of nano-particles with a magnetic core surrounded by a polymeric layer coated with antibodies targeting the cells of interest. After immunomagnetic capture and enrichment, fluorescent reagents are added for identification and enumeration of the target cells. The processed reagent/sample mixture is dispensed by the CellTracks AutoPrep System into a cartridge that is inserted into a MagNest® cell presentation device. The strong magnetic field of the MagNest causes the magnetically-labeled target cells to move to the surface of the cartridge. The CellTracks Analyzer II scans the entire surface of the cartridge with a series of fluorescence filters that are defined for the assay. Cell images from the filter are compiled and presented in a gallery format for final cell classification by the user.
Here's an analysis of the acceptance criteria and the study proving the device meets them, based on the provided text:
Acceptance Criteria and Device Performance
The document doesn't explicitly state pre-defined "acceptance criteria" in a structured list with target values. Instead, it demonstrates substantial equivalence to a predicate device by showing high correlation and comparable performance in several metrics. For this analysis, I will infer the implicit acceptance criteria from the reported findings, primarily focusing on correlation with the predicate device and the specified performance characteristics.
| Acceptance Criteria Category (Inferred) | Specific Metric | Target | Reported Device Performance | Meets Criteria? |
|---|---|---|---|---|
| Correlation with Predicate Device (Clinical Samples) | Pearson's Correlation Coefficient (for ≥1.5 CTCs) | > 0.95 (implied by 0.9996 results) | 0.9996 | Yes |
| Linear Regression Slope (for ≥1.5 CTCs) | Close to 1 (implied by 1.136 results, discussed) | 1.136 | Yes (with discussion) | |
| r² value (for ≥1.5 CTCs) | Close to 1 (implied by 0.9992 results) | 0.9992 | Yes | |
| Correlation with Predicate Device (Spiked Cell-Line Samples) | r-value for MCF-7 cells | ≥ 0.98 (explicitly stated in conclusion) | 0.994 | Yes |
| r-value for SKBr-3 cells | ≥ 0.98 (explicitly stated in conclusion) | 0.984 | Yes | |
| r-value for PC3-9 cells | ≥ 0.98 (explicitly stated in conclusion) | 0.963 | No (but combined analysis brings it up) | |
| r-value for all 3 cell lines combined | ≥ 0.98 (explicitly stated in conclusion) | 0.966 | No (but combined analysis brings it up) | |
| Correction: The document states "an r value of 0.966 for all values combined means that the AutoPrep / CellTracks Analyzer II platform and the CellPrep / CellSpotter Analyzer platform have a correlation coefficient (r) of at least 0.98." This appears to be a misstatement or a reinterpretation of their own result (0.966 is not "at least 0.98"). However, the individual r-values for cell lines are high. The overall conclusion states "very high degree of correlation." Given the context of substantial equivalence, the inference is that a high correlation, near 1, is the goal. | ||||
| Linearity | Regression slope (expected vs. observed) | Close to 1 (e.g., 0.95-1.05) | 1.007 | Yes |
| r² value | Close to 1 (e.g.,>0.99) | 0.99 | Yes | |
| Reportable Range | Comparable to predicate (4-1022 CTCs) | 0-1238 cells/ul | Yes (broader range covered) | |
| Precision | Coefficient of Variation (CV) for Low Control (average 48 cells) | Comparable to predicate (15.8%) | 18% | Yes (stated as comparable) |
| Coefficient of Variation (CV) for High Control (average 969 cells) | Comparable to predicate (9.4%) | 5% | Yes (stated as comparable) |
Note on "Meets Criteria?": The document focuses on demonstrating substantial equivalence to the predicate device. Therefore, "meets criteria" implies that the new device's performance is comparable to or better than the predicate's, and the statistical metrics (correlation coefficients, slopes, r-values) indicate a very strong agreement between the two systems. While some individual r-values or slopes might not perfectly match '1' or '0.98', the overall conclusion consistently asserts comparability and high correlation. The intercept values were also deemed "not statistically different than 0."
Study Details
-
Sample sizes used for the test set and data provenance:
- Clinical Samples Study: 83 specimens with an average of ≥ 1.5 circulating tumor cells. Samples were obtained from thirteen geographically dispersed sites. Based on the mention of "cancer patients," this data is retrospective and of clinical origin. The country of origin is not specified but implied to be the US given the FDA submission.
- Spiked Cell-Line Samples Study: 45 samples were analyzed on each of the two platforms (CellTracks Analyzer II and CellSpotter Analyzer). These were normal donor whole blood samples spiked with tissue culture cell lines (SKBr-3, PC3-9, and MCF-7) at three different levels (~5, ~50, and ~1000 cells). This is a prospective, controlled laboratory study.
-
Number of experts used to establish the ground truth for the test set and the qualifications of those experts:
- The document does not specify the number of experts or their qualifications for establishing ground truth for the clinical or spiked samples.
- For the clinical samples, "analysis was performed by Medical Technologists." This refers to the operators of the devices, not necessarily experts establishing a separate ground truth.
- For the spiked samples, the "ground truth" is defined by the known number of cells spiked into the normal donor blood.
-
Adjudication method for the test set:
- The document does not mention an adjudication method for the test set. For the clinical samples, it's implied that the device outputs were compared to each other. For spiked samples, it's a comparison to the known spiked quantity.
-
If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:
- No MRMC comparative effectiveness study was done as described. This study is a device-to-device comparison, not an assessment of human reader performance with or without AI assistance. The device is a semi-automated fluorescence microscope where a user classifies cells; however, the study design does not directly evaluate the impact on human reader performance, but rather the performance consistency between the new and predicate device.
-
If a standalone (i.e. algorithm only without human-in-the-loop performance) was done:
- The device, CellTracks Analyzer II, is described as a "semi-automated fluorescence microscope" where "Cell images from the filter are compiled and presented in a gallery format for final cell classification by the user." This indicates it's fundamentally a human-in-the-loop system. Therefore, a standalone (algorithm only) performance study, without human classification, was not performed nor would it be relevant for this device's intended use according to its description. The studies performed compare the system's output (which includes the final user classification) between the new and predicate device.
-
The type of ground truth used:
- Clinical Samples: The ground truth for this comparison study was the predicate device's output (CellSpotter predicate device), as the study compared the CellTracks Analyzer II to the CellSpotter predicate device using clinical samples.
- Spiked Cell-Line Samples: The ground truth was the known spiked quantity of cells. This is a form of engineered or "gold standard" ground truth.
- Linearity & Precision Studies: The ground truth for linearity was the known dilution of cells. For precision, it was the known fixed, pre-stained cells in the control kit.
-
The sample size for the training set:
- The document does not mention a "training set" in the context of machine learning or AI. The studies performed are verification and validation studies for hardware and integrated system performance against a predicate device or known inputs.
-
How the ground truth for the training set was established:
- As there's no mention of a "training set" in the AI/ML sense, this question is not applicable. The device appears to be an optical detection and enumeration system, not an AI/ML-driven classifier that requires a training set. The "software" mentioned likely refers to control and visualization software, not a self-learning algorithm.
§ 866.6020 Immunomagnetic circulating cancer cell selection and enumeration system.
(a)
Identification. An immunomagnetic circulating cancer cell selection and enumeration system is a device that consists of biological probes, fluorochromes, and other reagents; preservation and preparation devices; and a semiautomated analytical instrument to select and count circulating cancer cells in a prepared sample of whole blood. This device is intended for adjunctive use in monitoring or predicting cancer disease progression, response to therapy, and for the detection of recurrent disease.(b)
Classification. Class II (special controls). The special control for this device is FDA's guidance document entitled “Class II Special Controls Guidance Document: Immunomagnetic Circulating Cancer Cell Selection and Enumeration System.” See § 866.1(e) for availability of this guidance document.