(369 days)
PT-Multi Calibrator is intended as a calibrator set for local PT/INR calibration and/or local verification of the INR system for plasma based procedures with designated Siemens thromboplastins Dade® Innovin® or Thromborel® S on the BCS® / BCS® XP Systems.
PT-Multi Calibrator is a set of certified plasmas for local PT/INR calibration and/or local verification of the INR system for plasma based procedures using Siemens Dade® Innovin® or Thromborel® S reagents on Siemens BCS® Coaguiation Systems.
The calibrator levels are manufactured using a combination of normal and depleted human plasma.
Here's a summary of the acceptance criteria and the study details for the PT-Multi Calibrator, based on the provided document:
This device is an in-vitro diagnostic (IVD) calibrator, not an AI/ML powered device, therefore some of the requested information (e.g., number of experts, adjudication method, MRMC studies, standalone performance, training set details) are not applicable. I will provide information relevant to IVD calibrator performance evaluation.
1. Table of Acceptance Criteria and Reported Device Performance
The document does not explicitly state "acceptance criteria" in a quantitative manner as typically seen for novel medical devices. Instead, it presents performance data from "Method Comparison studies" and "Precision studies," implying that the suitability of the device is demonstrated by these results, generally expected to align with established clinical performance for such calibrators. For the purpose of this response, I will interpret the presented study results as demonstrating the device meets the implicit performance standards for a calibrator.
Implicit Performance Criteria (Demonstrated by Study Results)
| Performance Metric | Description | Reported Device Performance (Summary) |
|---|---|---|
| Method Comparison (INR Calibration Accuracy) | Demonstrated through linear regression analysis comparing the PT-Multi Calibrator to conventional local ISI/MNPT calibration methods. The regression equations (y = mx + b) are typically expected to have a slope (m) close to 1 and an intercept (b) close to 0, indicating good agreement between methods. | Thromborel S: |
- Individual Sites: Slopes (y=mx+b) ranged from 0.92 to 1.03; Intercepts ranged from -0.02 to +0.05.
- All Sites Combined: y=0.93x + 0.03 (without extrapolation), y=0.94x + 0.03 (with extrapolation).
Innovin: - Individual Sites: Slopes ranged from 0.92 to 1.10; Intercepts ranged from -0.20 to +0.06.
- All Sites Combined: y=0.97x - 0.04. These results suggest good linearity and agreement with existing methods. |
| Precision (Reproducibility & Repeatability) | Evaluated by coefficients of variation (CV%) for assays (e.g., Within-day CV%, Between-run CV%, Between-day CV%, Within-device CV%) across different calibrator levels (CPP, C2, NPP, PPP) and reagent lots. Lower CV% values indicate higher precision. | General Trend Across Sites, Reagents, and Calibrator Lots: - Repeatability %: Generally ranged from ~1% to ~2.5% for most calibrator levels, with some instances slightly higher (e.g., up to 4.7%).
- Within-day-CV %: Typically ranged from ~1% to ~3%, with some values up to ~6%.
- Between-run-CV %: Often around or below 1%, with some values up to ~4%.
- Between-day-CV %: Generally ranged from ~1% to ~3%, with some values up to ~5%.
- Within-device-CV %: Generally ranged from ~1% to ~3%, with some values up to ~5%. These values are typical for high-quality IVD calibrators, indicating good precision. |
2. Sample Sizes Used for the Test Set and Data Provenance
-
Method Comparison Studies (Test Set):
- Sample Size (n):
- For Thromborel S (without extrapolation): Individual sites had 'n' values ranging from 118 to 138.
- For Thromborel S (with extrapolation): Individual sites had 'n' values ranging from 119 to 139.
- For Innovin: Individual sites had 'n' values ranging from 102 to 136.
- Total for Thromborel S: 738 (without extrapolation) and 768 (with extrapolation) samples.
- Total for Innovin: 693 samples.
- Data Provenance: The studies were conducted at three different sites: Denver (presumably USA), Munich (Germany), and Marburg (Germany). The data is prospective as these are controlled validation studies for regulatory submission.
- Sample Size (n):
-
Precision Studies (Test Set):
- Sample Size: The tables indicate 20-day ANOVA precision studies. While specific 'n' values per measurement are not explicitly stated for individual data points, the nature of ANOVA precision studies typically involves multiple measurements over multiple days, runs, and replicates for each calibrator level. The tables report mean INR values and various CV% metrics, derived from this systematic measurement process. The number of independent measurements is substantial given the 20-day duration and multiple types of CVs.
- Data Provenance: The studies were conducted at the same three sites: Denver, Munich, and Marburg. The data is prospective as these are controlled validation studies for regulatory submission.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications
Not Applicable (N/A) for this device.
This device is an in vitro diagnostic calibrator. Its "ground truth" is established by its intrinsic chemical properties and its ability to accurately calibrate an instrument to a known standard (e.g., International Normalized Ratio - INR). The ground truth for calibrators is typically established through:
- Traceability: Ensuring the calibrator's assigned values are traceable to recognized international reference materials or methods (e.g., WHO international standards for PT/INR).
- Manufacturing Control: Strict quality control during manufacturing to ensure consistency and accuracy of the calibrator's analyte concentrations.
- Method Comparison: Comparison against established, conventional calibration methods (as presented in the documentation).
Therefore, "experts" in the sense of clinical decision-makers (like radiologists) are not used to establish the ground truth for this type of calibrator.
4. Adjudication Method for the Test Set
Not Applicable (N/A) for this device.
Adjudication methods (like 2+1, 3+1) are typically used in studies involving human interpretation of data where consensus is needed to define a ground truth (e.g., image interpretation). For an IVD calibrator, the results are quantitative measurements, and the accuracy and precision are determined by statistical analysis of these measurements against a known reference or by internal consistency, not by human adjudicators.
5. If a Multi Reader Multi Case (MRMC) Comparative Effectiveness Study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance
Not Applicable (N/A) for this device.
This device is an in vitro diagnostic calibrator, not an AI-powered diagnostic tool used by human readers. Therefore, MRMC studies and "improvement with AI vs without AI assistance" are not relevant to its evaluation.
6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) was done
Not Applicable (N/A) for this device.
This device is a "standalone" product in the sense that it is a physical calibrator kit. Its performance is evaluated intrinsically (precision) and in comparison to other calibration methods (accuracy) when used with an instrument, not as an algorithm separate from human interaction. The concept of "human-in-the-loop performance" or an "algorithm only" study does not apply to a physical calibrator.
7. The type of Ground Truth Used
For an in vitro diagnostic calibrator, the "ground truth" is intrinsically linked to:
- Reference Methods/Materials: The calibrator's values are ultimately traceable to established reference methods and/or internationally recognized reference materials for the specific analyte (in this case, Prothrombin Time/INR).
- Conventional Local Calibration Methods: In the method comparison studies, the PT-Multi Calibrator's performance is compared against "conventional local ISI/MNPT" (International Sensitivity Index / Mean Normal Prothrombin Time). This conventional method serves as the established "truth" or reference for evaluating the new calibrator's accuracy.
8. The Sample Size for the Training Set
Not Applicable (N/A) for this device in the context of machine learning.
The concept of a "training set" typically applies to machine learning algorithms. As a physical IVD calibrator, the device itself does not undergo machine learning training. Its development involves chemical and manufacturing processes, and its specifications are derived from extensive R&D and validation studies. The method comparison and precision studies described above represent the validation/test sets for demonstrating the device's performance.
9. How the Ground Truth for the Training Set was Established
Not Applicable (N/A) for this device.
As explained in point 8, there is no "training set" in the machine learning sense for this product. The ground truth for calibrators is established through traceability to international standards, rigorous manufacturing controls, and analytical validation against existing, validated methods.
§ 864.5425 Multipurpose system for in vitro coagulation studies.
(a)
Identification. A multipurpose system for in vitro coagulation studies is a device consisting of one automated or semiautomated instrument and its associated reagents and controls. The system is used to perform a series of coagulation studies and coagulation factor assays.(b)
Classification. Class II (special controls). A control intended for use with a multipurpose system for in vitro coagulation studies is exempt from the premarket notification procedures in subpart E of part 807 of this chapter subject to the limitations in § 864.9.