(325 days)
Not Found
Not Found
Here's a breakdown of the acceptance criteria and study details based on the provided text:
Acceptance Criteria and Device Performance
The acceptance criteria for the Technicon H+3 RTC/RTX reticulocyte method are implied by the reported performance characteristics and the stated goals of the evaluation. While explicit threshold values are not given, the device is considered to meet acceptance criteria if its performance is comparable to established methods (NCCLS for reticulocytes) and demonstrates high correlation and low bias for RBC indices.
Table of Acceptance Criteria (Implied) and Reported Device Performance
| Parameter | Implied Acceptance Criteria (e.g., strong correlation, low bias) | Reported Device Performance | Study Type |
|---|---|---|---|
| Reticulocyte Count | Accuracy (vs. Manual Counts) | ||
| % retic (r) | High correlation (e.g., >0.90) | 0.97 | Comparison to NCCLS method |
| % retic (Slope) | Close to 1 (e.g., 0.9-1.1) | 0.93 | Comparison to NCCLS method |
| % retic (Intercept) | Close to 0 | 0.0 | Comparison to NCCLS method |
| % retic (Syx) | Low error | 0.56 | Comparison to NCCLS method |
| % retic (Reference Mean) | Matches H•3 Mean closely | 2.0 | Comparison to NCCLS method |
| % retic (H•3 Mean) | Matches Reference Mean closely | 1.9 | Comparison to NCCLS method |
| abs retic (10^9^/L) (r) | High correlation (e.g., >0.90) | 0.93 | Comparison to NCCLS method |
| abs retic (Slope) | Close to 1 (e.g., 0.9-1.1) | 0.86 | Comparison to NCCLS method |
| abs retic (Intercept) | Close to 0 | 6.4 | Comparison to NCCLS method |
| abs retic (Syx) | Low error | 22.6 | Comparison to NCCLS method |
| abs retic (Reference Mean) | Matches H•3 Mean closely | 84.9 | Comparison to NCCLS method |
| abs retic (H•3 Mean) | Matches Reference Mean closely | 79.1 | Comparison to NCCLS method |
| RBC Indices | Accuracy (vs. H+3 CBC/Diff Mode) | ||
| MCV (r) | High correlation (e.g., >0.95) | 0.974 | Comparison to H+3 CBC/Diff |
| MCV (Slope) | Close to 1 (e.g., 0.9-1.1) | 0.88 | Comparison to H+3 CBC/Diff |
| MCV (Intercept) | Close to 0 | 10.0 | Comparison to H+3 CBC/Diff |
| MCV (Syx) | Low error | 1.0 | Comparison to H+3 CBC/Diff |
| MCV (Bias) | Close to 0 | -0.1 | Comparison to H+3 CBC/Diff |
| CHCM (r) | High correlation (e.g., >0.95) | 0.988 | Comparison to H+3 CBC/Diff |
| CHCM (Slope) | Close to 1 (e.g., 0.9-1.1) | 1.06 | Comparison to H+3 CBC/Diff |
| CHCM (Intercept) | Close to 0 | -1.6 | Comparison to H+3 CBC/Diff |
| CHCM (Syx) | Low error | 0.7 | Comparison to H+3 CBC/Diff |
| CHCM (Bias) | Close to 0 | 0.4 | Comparison to H+3 CBC/Diff |
| CH (r) | High correlation (e.g., >0.95) | 0.980 | Comparison to H+3 CBC/Diff |
| CH (Slope) | Close to 1 (e.g., 0.9-1.1) | 0.95 | Comparison to H+3 CBC/Diff |
| CH (Intercept) | Close to 0 | 2.0 | Comparison to H+3 CBC/Diff |
| CH (Syx) | Low error | 0.6 | Comparison to H+3 CBC/Diff |
| CH (Bias) | Close to 0 | 0.5 | Comparison to H+3 CBC/Diff |
| RDW (r) | High correlation (e.g., >0.90) | 0.937 | Comparison to H+3 CBC/Diff |
| RDW (Slope) | Close to 1 (e.g., 0.9-1.1) | 1.06 | Comparison to H+3 CBC/Diff |
| RDW (Intercept) | Close to 0 | -0.8 | Comparison to H+3 CBC/Diff |
| RDW (Syx) | Low error | 0.5 | Comparison to H+3 CBC/Diff |
| RDW (Bias) | Close to 0 | 0.1 | Comparison to H+3 CBC/Diff |
| HDW (r) | High correlation (e.g., >0.95) | 0.990 | Comparison to H+3 CBC/Diff |
| HDW (Slope) | Close to 1 (e.9., 0.9-1.1) | 1.13 | Comparison to H+3 CBC/Diff |
| HDW (Intercept) | Close to 0 | -0.25 | Comparison to H+3 CBC/Diff |
| HDW (Syx) | Low error | 0.14 | Comparison to H+3 CBC/Diff |
| HDW (Bias) | Close to 0 | 0.11 | Comparison to H+3 CBC/Diff |
| Precision | Low SD and CV | Within Run Precision | |
| Retic (%) (CV) | Low CV (e.g., <15%) | 12.7 | 25 replicates per sample |
| Abs Retic (CV) | Low CV (e.g., <15%) | 12.1 | 25 replicates per sample |
| Other parameters (CV) | Generally low CVs (e.g., <10%, <5%) | 0.3% - 8.1% | 25 replicates per sample |
| Linearity | Linear response over clinically relevant range | Linear from 0% to 26% reticulocytes | Serial dilutions |
| Carryover | No detectable carryover | No detectable carryover | High level human pools |
| Sample Stability | Within acceptable recovery limits | Within ±0.5% recovery at 30 min (15-90 min range) | Assay at intervals |
| RBC Indices Stability | Acceptable stability (e.g., 15-20 min for most, 15-90 min for CH/CHr) | 15-20 min incubation for MCV, CHCM, RDW, HDW; 15-90 min for CH, CHr | Assay at intervals |
Study Information
Accuracy, precision, linearity, carryover, and sample stability of the Technicon H+3 RTC/RTX reticulocyte method were evaluated.
1. Sample Sized used for the test set and the data provenance:
-
Reticulocyte Accuracy: 50 samples from apparently healthy donors and 48 samples from hospital patients (a total of 98 unique samples). Each sample was assayed in duplicate on the Technicon H+3 and independently counted by two technologists using the NCCLS method. This resulted in a total of n=196 (98 samples x 2 duplicates) for the Technicon H+3 analysis and 98 samples for manual counting.
-
RBC Indices Accuracy: The same n=196 (98 samples x 2 duplicates) from the reticulocyte accuracy study were used.
-
Precision: 5 samples. Each sample underwent 25 replicate assays, resulting in 125 assays (5 samples x 25 replicates).
-
Linearity: Not explicitly stated, but involved a "human pool prepared to obtain a high reticulocyte count."
-
Carryover: "High level human pools" were used. Specific sample size not detailed.
-
Prepared Sample Stability: 15 prepared samples from apparently healthy donors.
-
Data Provenance: The origin of the samples (e.g., country) is not specified. The study appears to be prospective in nature, as samples were obtained, prepared, and then analyzed according to the study protocol.
2. Number of experts used to establish the ground truth for the test set and the qualifications of those experts:
- Reticulocyte Accuracy: Two technologists independently performed the manual reticulocyte count using the NCCLS method. Their specific qualifications (e.g., years of experience) are not provided, but it's implied they are qualified to perform the NCCLS proposed standard.
- RBC Indices Accuracy: The "ground truth" for RBC indices was established by comparison to the Technicon H+3 RBC/platelet and hemoglobin methods (CBC/Diff mode), which are presumably established and validated methods for these parameters. No human experts were explicitly involved in establishing this ground truth for the indices in the same way as for reticulocytes.
- Other studies (Precision, Linearity, Carryover, Stability): The ground truth was established by repeated measurements or by reference to established laboratory protocols and expected biological responses rather than by human expert interpretation.
3. Adjudication method for the test set:
- Reticulocyte Accuracy: The text states, "Each sample was counted independently by two technologists using the NCCLS method." It does not explicitly mention an adjudication method (like 2+1 or 3+1) if their counts differed significantly. However, it does note that "laboratories performing similar accuracy evaluations may obtain incorrect least squares regression coefficients when the range of samples is small. In this event, the coefficients can be corrected by averaging manual results from 4 or more technologists, or by calculating the regression coefficients using a method derived by Deming." This suggests an awareness of potential variability in manual counts and a recommended approach (averaging more technologists or a statistical method) to handle it, but not a specific adjudication protocol for the two technologists used in this particular study. Assuming industry standard best practices, close agreement would be expected, or discrepancies would be resolved, but the exact method isn't detailed.
4. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:
- No, an MRMC comparative effectiveness study was not done. This study evaluates the performance of an automated device (Technicon H+3 RTC/RTX) compared to a manual method (for reticulocytes) or another automated method (for RBC indices). It does not involve human readers using or not using AI assistance. The device is the "AI" or automated component here, replacing a manual process for reticulocytes rather than assisting a human reader.
5. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done:
- Yes, a standalone performance evaluation was done. The entire study is a standalone evaluation of the Technicon H+3 RTC/RTX device. The reported performance characteristics (accuracy, precision, linearity, carryover, stability) are for the device operating independently. For reticulocyte counts, the device's performance is compared against a manual method, but the device itself operates in a standalone fashion.
6. The type of ground truth used:
- Reticulocyte Count: Expert consensus (implied agreement between two technologists) using a gold standard method (NCCLS proposed standard for reticulocyte counting using new methylene blue).
- Absolute Reticulocyte Count: Derived from the two technologists' % reticulocyte results and an RBC count from a Technicon H+2 system.
- RBC Indices: Comparison to established measurement from an existing device (Technicon H+3 CBC/Diff mode).
- Other studies (Precision, Linearity, Carryover, Stability): Ground truth was established by meticulous experimental design, serial dilutions, high-level pools, or timed assays, relying on the inherent characteristics of the samples and the ability of reference methods/measurements to define the expected outcomes.
7. The sample size for the training set:
- Not applicable. This study describes the performance evaluation of a medical device (Technicon H+3 RTC/RTX), which is a laboratory analyzer, not a machine learning algorithm that requires a separate "training set" for model development. The device would have been developed and internally validated by the manufacturer, but the provided text only details the performance characteristics for regulatory submission.
8. How the ground truth for the training set was established:
- Not applicable. As this is a medical device performance study, not a machine learning algorithm study with a defined "training set," this question does not apply.
{0}------------------------------------------------
AUG 23 1996 SUMMARY OF SAFETY AND EFFEC
METHOD PERFORMANCE CHARACTERISTICS
The following is a summary of performance characteristics obtained from evaluations of the Technicon H+3 RTC/RTX reticulocyte method. The values shown in this section are point estimates of performance which can be expected from the method. Similar evaluations may produce slightly different results depending on the characteristics of the donor population.
ACCURACY
Reticulocyte Count
Accuracy of the Technicon H+3 RTC/RTX reticulocyte method was evaluated by comparison to the NCCLS proposed standard for reticulocyte counting using new methylene blue. Fifty samples were obtained from apparently healthy donors and forty-eight samples were obtained from hospital patients including patients with known red cell abnormalities. Each sample was counted independently by two technologists using the NCCLS method and was assayed in duplicate on a Technicon H43 for a total sample size of n=196. Reference absolute counts were obtained from the technologist % reticulocyte result and an RBC count from a Technicon H+2 system. The test results appear in the table below.
| Parameter | r | Slope | Intercept | Syx | ReferenceMean | H•3Mean | Range of Samples |
|---|---|---|---|---|---|---|---|
| % retic | 0.97 | 0.93 | 0.0 | 0.56 | 2.0 | 1.9 | 0.2 to 26.2 |
| abs retic (10 9/L) | 0.93 | 0.86 | 6.4 | 22.6 | 84.9 | 79.1 | 7.4 to 552.8 |
Table 4-4 ACCURACY VERSUS MANUAL COUNTS
Due to the substantial error associated with the manual counting technique , laboratories performing similar accuracy evaluations may obtain incorrect least squares regression coefficients when the range of samples is small'. In this event, the coefficients can be corrected by averaging manual results from 4 or more technologists, or by calculating the regression coefficients using a method derived by Deming .
RBC Indices
Accuracy of the Technicon H•3 RTC/RTX reticulocyte method RBC indices was evaluated by comparison to the RBC indices obtained from the H+3 RBC/platelet and hemoglobin methods (CBC/Diff mode) in the experiment described above (n=196). Accuracy of MCV, CHCM, RDW, and HDW was evaluated by comparison to the identical parameter obtained from the RBC/platelet method. The accuracy of CH was evaluated by comparison to MCH which is calculated from the RBC count and hemoglobin measurement.
| Parameter | r | Slope | Intercept | Syx | Mean inCBC/DiffMode | Mean inReticulocyteMode | Bias |
|---|---|---|---|---|---|---|---|
| MCV | 0.974 | 0.88 | 10.0 | 1.0 | 84.8 | 84.7 | -0.1 |
| CHCM | 0.988 | 1.06 | -1.6 | 0.7 | 33.1 | 33.5 | 0.4 |
| CH | 0.980 | 0.95 | 2.0 | 0.6 | 27.5 | 28.0 | 0.5 |
| RDW | 0.937 | 1.06 | -0.8 | 0.5 | 15.7 | 15.8 | 0.1 |
| HDW | 0.990 | 1.13 | -0.25 | 0.14 | 2.79 | 2.90 | 0.11 |
Table 4-5 ACCURACY OF RBC INDICES
{1}------------------------------------------------
SUMMARY OF SAFETY AND EFFECTIVENESS
WITHIN RUN PRECISION CHARACTERISTICS
Within run precision of the reticulocyte method was tested by 25 replicate assays performed for each of 5 samples obtained from apparently healthy donors. The average result, which represents typical performance, appears in the table below.
| Parameter | Mean | SD | CV (%) |
|---|---|---|---|
| Retic (%) | 1.1 | 0.14 | 12.7 |
| Abs Retic (109 cells/L) | 52.2 | 6.3 | 12.1 |
| MCV (fL) | 90.5 | 0.44 | 0.5 |
| MCVr (fL) | 110.8 | 2.1 | 1.9 |
| CHCM (g/dL) | 32.3 | 0.20 | 0.6 |
| CHCMr (g/dL) | 27.7 | 0.5 | 1.5 |
| CH (pg) | 28.6 | 0.08 | 0.3 |
| CHr (pg) | 29.9 | 0.5 | 1.3 |
| RDW (%) | 13.2 | 0.09 | 0.7 |
| RDWr (%) | 14.8 | 1.2 | 8.1 |
| HDW (g/dL) | 2.59 | 0.05 | 1.9 |
| HDWr (g/dL) | 3.04 | 0.22 | 7.2 |
| HHDW (pg) | 3.40 | 0.02 | 0.6 |
| HHDWr (pg) | 3.84 | 0.28 | 7.3 |
Table 4-6 BEPLICATE PRECISION
LINEARITY
The linearity of the % reticulocyte count was tested by making serial dilutions of a human pool prepared to obtain a high reticulocyte count. The results from this study indicate that the response is linear from 0% to 26% reticulocytes.
CARRYOVER
Carryover of the reticulocyte method was measured using high level human pools. No detectable carryover was observed for reticulocyte counts.
PREPARED SAMPLE STABILITY
Reticulocyte Count
Prepared sample stability over the claimed range of 15 to 90 minutes was tested by assaying 15 prepared samples obtained from apparently healthy donors from 15 to 90 minutes at 15 minute intervals. The test results indicate that all samples were within ±0.5% of the % reticulocyte recovery obtained at 30 minutes.
RBC Indices
With the exception of CH and CHr, the RBC indices are not stable over the entire incubation period. When reporting results for RBC indices other than CH and CHr, the samples should incubate for at least 15 minutes, but not longer than 20 minutes. CH and CHr are stable over the entire 15 to 90 minute incubation period.
§ 864.5220 Automated differential cell counter.
(a)
Identification. An automated differential cell counter is a device used to identify one or more of the formed elements of the blood. The device may also have the capability to flag, count, or classify immature or abnormal hematopoietic cells of the blood, bone marrow, or other body fluids. These devices may combine an electronic particle counting method, optical method, or a flow cytometric method utilizing monoclonal CD (cluster designation) markers. The device includes accessory CD markers.(b)
Classification. Class II (special controls). The special control for this device is the FDA document entitled “Class II Special Controls Guidance Document: Premarket Notifications for Automated Differential Cell Counters for Immature or Abnormal Blood Cells; Final Guidance for Industry and FDA.”