(198 days)
VALIDATE® Anemia Calibration Verification/Linearity Test Kit solutions are intended for in vitro diagnostic use in the quantitative determination of linearity, calibration and verification of reportable range for the following analytes: Ferritin Set: ferritin (FERR), Vitamin B12(VITB) and folate (FOL), on automated instrument systems. The product is intended for use with quantitative assays on the indicated analyzers specified in the labeling.
Each VALIDATE® Anemia Calibration Verification / Linearity Test Kit contains two analyte sets of purified chemicals in a human serum base. The kit includes a Ferritin set containing five liguid levels, 3.0mL each, and a Vitamin B12/Folate set containing five liquid levels, 4.0 mL each. The sets are provided to establish the relationship between theoretical and actual performance of the included analytes: Ferritin, Vitamin B12 and Folate. Material of human origin used in the manufacture of this test kit was tested at the donor level using FDA approved methods and was found to be non-reactive for HBsAG and to antibodies to HCV and HIV-1/2.
The provided text describes the performance data for the VALIDATE® Anemia Calibration Verification/Linearity Test Kit. This product is a quality control material, not a device that directly measures a patient's condition or relies on AI. Therefore, the common acceptance criteria and study designs typically associated with AI-powered medical devices (such as sensitivity, specificity, MRMC studies, and ground truth established by experts for a test set) are not applicable here.
Instead, the acceptance criteria and study detailed focus on the technical performance of the calibration verification/linearity kit itself, to ensure it provides accurate and linear results when used with automated instrument systems.
Here's an analysis based on the information provided, reinterpreting some categories to fit the context of a calibration verification kit:
1. Table of Acceptance Criteria and Reported Device Performance:
| Feature/Test | Acceptance Criteria | Reported Device Performance |
|---|---|---|
| Value Assignment | Levels 1-5 must meet specified Ferritin, Vitamin B12, and Folate targets. (Ranges determined by upper/lower detection limits; intermediate levels calculated based on equal dilutions following EP6-A guidelines). | All stated recovery values met internal pre-determined acceptance criteria. |
| Recovery Targets (Low/High Levels) | 90 to 110% of DOM value for product levels 2-5 during stability testing. | All product levels tested within the 90 to 110% of control acceptance criteria limits. |
| Freeze/Thaw Stability | Within 90 to 110% of control acceptance criteria limits. | Tested within the 90 to 110% of control acceptance criteria limits after 6 freeze/thaw events. |
| Shelf Life | Maintain stability for claim duration. | Set at 12 months based on real-time open vial studies (worst-case scenario). |
| Product Linearity | Demonstrated linearity on specified automated systems. | Product linearity performance was demonstrated for both automated systems (Roche® Cobas 6000 Chemistry Analyzer and Beckman-Coulter® Access II Immunochemistry Analyzer). |
2. Sample Size Used for the Test Set and Data Provenance:
- Sample Size: The number of individual test kit units or measurements used for "value assignment" and "stability" testing is not explicitly stated as a numerical sample size. It refers to testing on "Levels 1 through 5" of the kit. For stability, testing was done at "date of manufacture (DOM), followed by testing at specific intervals post manufacture" and "one month post-expiration." Freeze/thaw stability was assessed for "four (4) freeze/thaw events claim" and tested after "6 freeze/thaw events."
- Data Provenance: The studies were conducted internally by Maine Standards Company LLC. The text does not specify the country of origin of the raw data, but the company is based in the US. The studies appear to be prospective to validate the manufacturing and stability of the product.
3. Number of Experts Used to Establish Ground Truth for the Test Set and Their Qualifications:
This question is not directly applicable in the typical sense of expert review for medical imaging or diagnostic algorithms. For this device, the "ground truth" or reference values are established by the manufacturing process itself:
- "Levels 1 and 5 are prepared independently by the addition of Ferritin, Vitamin B12 and Folate to a human serum base."
- "Intermediate 2, 3, and 4 are subsequently prepared from Levels 1 and 5 by equal part dilutions following EP6-A guidelines."
- The actual "ground truth" for the performance evaluation is the predetermined theoretical concentrations that the calibration verification kit levels should represent based on their formulation. The device's performance is then measured by how well instruments recover these known concentrations and demonstrate linearity.
There's no mention of external experts adjudicating results for the test set.
4. Adjudication Method for the Test Set:
Not applicable in the conventional sense. The test results are compared against internal, pre-determined acceptance criteria based on the known composition and expected behavior of the calibration verification material, and adherence to CLSI EP6-A guidelines.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:
Not applicable. This is a quality control material intended for instrument calibration and linearity verification, not a diagnostic device that assists human readers.
6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) was done:
Not applicable. This device is used by automated analytical instruments (Roche Cobas, Beckman Access) to verify their performance. Its "standalone" performance means its inherent accuracy in representing known concentrations, which is what the "Value Assignment" and "Linearity" sections address. The analytical instruments are the "algorithm" in this context, and this device checks their accuracy.
7. The Type of Ground Truth Used:
The ground truth is based on:
- Known concentrations: The product is manufactured with specific, known concentrations of analytes (Ferritin, B12, Folate) at Levels 1 and 5.
- Dilution principles: Intermediate levels (2, 3, 4) are prepared by precise dilutions from Levels 1 and 5, following CLSI EP6-A guidelines, establishing their theoretical concentrations.
- Reference Standards: The product is described as "traceable to a reference standard based on the automated instrument platform it is used on." This implies a higher-level standard (e.g., NIST traceable or other internationally recognized standards) against which the instrument's calibrators are established, and by extension, this product helps verify that performance.
8. The Sample Size for the Training Set:
Not applicable. This is not a machine learning or AI device that requires a "training set." The product is a manufactured reagent with a defined chemical composition.
9. How the Ground Truth for the Training Set was Established:
Not applicable, as there is no "training set" for this type of device.
§ 862.1660 Quality control material (assayed and unassayed).
(a)
Identification. A quality control material (assayed and unassayed) for clinical chemistry is a device intended for medical purposes for use in a test system to estimate test precision and to detect systematic analytical deviations that may arise from reagent or analytical instrument variation. A quality control material (assayed and unassayed) may be used for proficiency testing in interlaboratory surveys. This generic type of device includes controls (assayed and unassayed) for blood gases, electrolytes, enzymes, multianalytes (all kinds), single (specified) analytes, or urinalysis controls.(b)
Classification. Class I (general controls). Except when intended for use in donor screening tests, quality control materials (assayed and unassayed) are exempt from the premarket notification procedures in subpart E of part 807 of this chapter subject to the limitations in § 862.9.