(207 days)
The SmartCheck INRTM System is intended for quantitative testing of Prothrombin Time in capillary blood. Results are given in International Normalized Ratio (INR) units.
The SmartCheck INR System is indicated for use by trained medical professionals in a point of care setting for monitoring the INR of patients on oral anticoagulant therapy.
The SmartCheck INRTM system consists of a hand held battery- powered meter with ROM calibration chip, disposable, dual- chambered, test strips and high and low level external liquid controls. When blood is applied to the strip it migrates by capillary action to the reaction chambers of the strip. The reaction chamber contains a metallic disc and rabbit brain thromboplastin reagent. When blood enters the reaction chamber a magnetic field is applied to the strip to activate disc movement. Upon clot formation, the disc becomes immobilized and clotting is optically detected. Clotting time is calculated and displayed as an INR value.
The provided text describes the SmartCheck INR™ System, a device for quantitative testing of Prothrombin Time in capillary blood. The submission focuses on demonstrating its substantial equivalence to a predicate device through non-clinical and clinical performance evaluations.
Here's an analysis of the acceptance criteria and the study used to prove the device meets them:
1. A table of acceptance criteria and the reported device performance
The submission does not explicitly state pre-defined acceptance criteria in terms of specific numeric thresholds for accuracy and precision. Instead, it presents regression analysis results and aims to show "correlated favourably" and "substantially equivalent" to a venous plasma based PT test and the predicate device.
However, we can infer performance metrics that are implicitly considered for acceptance based on the reported data:
| Metric | Acceptance Criteria (Implied) | Reported Device Performance |
|---|---|---|
| Accuracy (Correlation Coefficient) | High correlation to reference method (venous plasma INR) | 0.9235 (recTP), 0.9009 (PTHS+), 0.9036 (PT/Fib) |
| Accuracy (Slope) | Close to 1.0 (indicating proportional agreement) | 1.17 (recTP), 1.48 (PTHS+), 1.31 (PT/Fib) |
| Accuracy (Intercept) | Close to 0 (indicating no systematic bias) | -0.49 (recTP), -1.19 (PTHS+), -0.90 (PT/Fib) |
| Accuracy (Mean Bias) | Close to 0, or within acceptable clinical limits | -0.045 or -1.7% (recTP), 0.036 or 1.4% (PTHS+), -0.062 or -2.3% (PT/Fib) |
| Precision (Standard Deviation) | Low standard deviation | 0.227 INR |
| Precision (Coefficient of Variation - %CV) | Low %CV | 8.70% |
Study Proving Acceptance Criteria:
The study involved two main components:
- Accuracy Study: Comparing SmartCheck INR capillary blood results against venous plasma INR measured by an ACL 10000 laboratory analyzer using three different thromboplastin reagents (recTP, PTHS+, and PT/Fib).
- Precision Study: Measuring SmartCheck INR capillary blood from duplicate finger-sticks to assess within-device variability.
2. Sample sized used for the test set and the data provenance
- Accuracy Test Set Sample Size:
- 315 for recTP reagent
- 310 for PTHS+ reagent
- 314 for PT/Fib reagent
- Precision Test Set Sample Size: 280
- Data Provenance: The document does not explicitly state the country of origin. It indicates the submitter, Unipath Ltd, is based in the UK. The studies are referred to as "non-clinical and clinical performance," implying prospective data collection for the clinical part.
3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts
This section is not applicable to this type of device. The ground truth for INR measurements in this context is established by a reference laboratory method (venous plasma INR on an ACL 10000 analyzer) rather than expert consensus/adjudication of images or clinical assessments.
4. Adjudication method for the test set
This is not applicable (see point 3). The comparison is between the device's measurement and a laboratory reference measurement, not a subjective interpretation requiring adjudication.
5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance
This is not applicable. The SmartCheck INR System is a diagnostic device that measures a biomarker (INR); it is not an AI-assisted diagnostic tool for human readers/interpreters, nor does it involve "readers" in the traditional sense of interpreting images or other complex data.
6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done
Yes, the accuracy and precision studies described represent the standalone performance of the SmartCheck INR system. The device itself provides the INR value, and its performance is evaluated directly against a reference method (accuracy) and internally (precision). There is no "human-in-the-loop" interpretation of the device's primary output. The human interaction is limited to operating the device and applying the blood sample.
7. The type of ground truth used
The ground truth used for the accuracy study was the venous plasma INR results obtained from an ACL 10000 laboratory analyzer using recombinant thromboplastin (recTP), PTHS+ reagent, and PT/Fibrinogen reagent (PT/Fib). This is a well-established, objective laboratory reference method for INR measurement.
8. The sample size for the training set
The document does not report a separate training set sample size. This is typical for a device like the SmartCheck INR, which uses a pre-defined chemical and mechanical detection method rather than a machine learning or AI algorithm that requires a distinct training phase on clinical data. The calibration of the device would likely be done internally and validated through the clinical studies presented here.
9. How the ground truth for the training set was established
As there's no explicitly mentioned training set in the context of machine learning, this question is not directly applicable. For devices that are calibrated, the "ground truth" for calibration would be established by standardized reference materials and methods during manufacturing and internal testing, ensuring accuracy across the device's operating range before clinical evaluation.
§ 864.5425 Multipurpose system for in vitro coagulation studies.
(a)
Identification. A multipurpose system for in vitro coagulation studies is a device consisting of one automated or semiautomated instrument and its associated reagents and controls. The system is used to perform a series of coagulation studies and coagulation factor assays.(b)
Classification. Class II (special controls). A control intended for use with a multipurpose system for in vitro coagulation studies is exempt from the premarket notification procedures in subpart E of part 807 of this chapter subject to the limitations in § 864.9.