(195 days)
Behring Coagulation Timer is a photometric coagulation analyzer intended for clinical use to perform all coagulometric, chromogenic and immunologic coaqulation tests.
Behring Coagulation Timer (BCT) is a fully automatic photo-optical coagulation instrument. The BCT is used in conjunction with the Behringwerke AG test kits for measurement of coagulation, fibrinolysis factors and inhibitors as well as to measure the function of the homeostatic system as a whole.
This document describes the Behring Coagulation Timer (BCT), a new device, and compares its performance to a previously marketed device, the Behring Fibrintimer A (BFA), and in some cases, the Behring Chromotime System. The primary goal is to demonstrate substantial equivalence through correlation and precision studies.
Here's an analysis of the provided information concerning acceptance criteria and the study:
1. A table of acceptance criteria and the reported device performance
The document does not explicitly state formal "acceptance criteria" with numerical thresholds for correlation coefficients, y-intercepts, slopes, or %CV. Instead, it presents results and then a qualitative statement of "These studies support the substantial equivalence..." indicating that the observed performance was deemed acceptable.
However, we can infer implied acceptance criteria from the reported values and the claim of substantial equivalence. For comparative purposes, let's assume "acceptable" performance would be a high correlation (close to 1), a slope close to 1, and a y-intercept close to 0 for correlation studies, and low %CVs for precision studies.
| Acceptance Criteria (Implied) | Reported Device Performance (BCT) |
|---|---|
| Correlation: | |
| Correlation Coefficient (r) ~ 1 | Thromborel S: r = 0.997 |
| Pathromtin: r = 0.99 | |
| Berichrom Plasminogen: r = 0.96 | |
| Berichrom Protein C: r = 0.99 | |
| Slope (m) ~ 1 | Thromborel S: m = 0.97 |
| Pathromtin: m = 1.0 | |
| Berichrom Plasminogen: m = 0.97 | |
| Berichrom Protein C: m = 1.0 | |
| Y-intercept (b) ~ 0 | Thromborel S: b = 0.72 |
| Pathromtin: b = -0.87 | |
| Berichrom Plasminogen: b = 9.9 | |
| Berichrom Protein C: b = -0.69 | |
| Precision: (%CV should be low) | |
| Thromborel S: | |
| Intra-Assay %CV: 0.50 to 0.81 | |
| Inter-Assay %CV: 2.04 to 6.36 | |
| Pathromtin: | |
| Intra-Assay %CV: 1.55 to 2.95 | |
| Inter-Assay %CV: 1.75 to 2.51 | |
| Berichrom Plasminogen: | |
| Intra-Assay %CV: 2.85 to 3.61 | |
| Inter-Assay %CV: 2.19 to 3.95 | |
| Berichrom Protein C: | |
| Intra-Assay %CV: 3.33 to 3.96 | |
| Inter-Assay %CV: 1.90 to 2.54 |
2. Sample size used for the test set and the data provenance (e.g. country of origin of the data, retrospective or prospective)
-
Correlation Studies:
- Thromborel S & Pathromtin: 83 samples were evaluated for both assays when comparing BCT vs BFA.
- Berichrom Plasminogen: 58 samples were evaluated when comparing BCT vs Behring Chromotime System.
- Berichrom Protein C: 74 samples were evaluated when comparing BCT vs Behring Chromotime System.
-
Precision Studies:
- For each of the four representative assays (Thromborel S, Pathromtin, Berichrom Plasminogen, Berichrom Protein C), 3 levels of control were run.
- Each level of control was run in replicates of 8 for five days, totaling n=40 measurements per level per assay (3 levels * 40 measurements = 120 measurements per assay).
- Intra-assay precision was calculated from either n=4 or n=8 precision values.
-
Data Provenance: The document does not specify the country of origin of the data samples or whether the studies were retrospective or prospective. Given the manufacturer's address in Germany, it's plausible the studies were conducted there. The nature of these lab-based tests (correlation and precision of an instrument) leans towards a prospective testing methodology where samples are run on both systems for comparison.
3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts (e.g. radiologist with 10 years of experience)
This study does not involve expert adjudication or interpretation for its ground truth. The "ground truth" for the correlation studies is the measurement obtained from the legally marketed equivalent device (BFA or Behring Chromotime System), which is also a lab instrument. For precision studies, the ground truth is the expected value of the control material, and the study assesses the device's ability to consistently reproduce measurements around that value. Therefore, no human experts are involved in establishing the "ground truth" in the way they would be in image interpretation or diagnostic performance studies.
4. Adjudication method (e.g. 2+1, 3+1, none) for the test set
Not applicable. As explained in point 3, there is no need for expert adjudication in this type of instrument performance study. The comparison is between instrument readings.
5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance
Not applicable. This is a study evaluating the performance of a lab instrument (coagulation timer), not an AI-assisted diagnostic tool that involves human readers interpreting cases.
6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done
Yes, in essence, the entire study is a standalone performance evaluation of the BCT, compared to other standalone instruments (BFA, Behring Chromotime System). There is no "human-in-the-loop" component being evaluated in the context of diagnostic interpretation. The BCT itself is an automated instrument.
7. The type of ground truth used (expert consensus, pathology, outcomes data, etc.)
The ground truth for the correlation studies is the measurements obtained from the predicate devices (Behring Fibrintimer A and Behring Chromotime System). These devices are themselves calibrated lab instruments. For precision, the ground truth refers to the expected values of control materials used, and the goal is to demonstrate the BCT's ability to consistently measure these values. This is effectively a "reference standard" or "comparative device" ground truth.
8. The sample size for the training set
This document describes performance validation studies for a device, not a machine learning model. Therefore, there is no "training set" in the context of AI/ML. The device (BCT) operates based on pre-programmed algorithms and photometric principles, not through a learned model.
9. How the ground truth for the training set was established
Not applicable, as there is no training set for this type of device.
§ 864.5425 Multipurpose system for in vitro coagulation studies.
(a)
Identification. A multipurpose system for in vitro coagulation studies is a device consisting of one automated or semiautomated instrument and its associated reagents and controls. The system is used to perform a series of coagulation studies and coagulation factor assays.(b)
Classification. Class II (special controls). A control intended for use with a multipurpose system for in vitro coagulation studies is exempt from the premarket notification procedures in subpart E of part 807 of this chapter subject to the limitations in § 864.9.