(22 days)
The UCFP CAL is an in vitro diagnostic product for the calibration of Urinary/Cerebrospinal Fluid Protein (UCFP) method on the Dimension Vista" System.
UCFP CAL is an aqueous product containing dilute human serum. The kit consists of three vials of Calibrator A. The volume per vial is 1.5 mL. UCFP CAL is ready for use, where no preparation is required. System water is used as the UCFP zero calibrator (Level 1) for the Dimension VistaTM System.
The provided text describes the acceptance criteria and performance of the Dimension Vista™ System Urinary/Cerebrospinal Fluid Protein Calibrator (UCFP CAL - KC260).
Here's a breakdown of the requested information:
1. Table of Acceptance Criteria and Reported Device Performance
| Acceptance Criteria Category | Specific Criteria | Reported Device Performance |
|---|---|---|
| Shelf-life Stability | Shelf-life stability (expiration) dating assignment at commercialization reflects the real-time data on file at Dade Behring, Inc. | Target: 12 months. |
| Testing Method: Comparing results of product stored at 4°C with control stored at -20°C. The 4°C material values are recovered versus the calibration, and percent change over time is monitored. | ||
| Acceptance Threshold: Percent change should be less than or equal to 3%. | ||
| Result: The document states that "Shelf-life stability (expiration) dating assignment at commercialization reflects the real-time data on file at Dade Behring, Inc." This implies the 12-month target was met based on real-time data that exhibited the percent change within the 3% criteria. | ||
| On-board Stability (Instrument Punctured Vial) | Stability of a vial punctured by the instrument and stored on board. | Result: Stable for seven days. |
| Open Vial Stability (Recapped, Refrigerated) | Stability of an open vial not stored on board of the instrument, but recapped and stored in a refrigerator. | Result: Stable for 31 days. |
| Testing Method: Vials are opened/punctured on day zero. A quantity sufficient for multiple calibrations is removed, and the vials are recapped and stored at 2 - 8 °C. Opened/punctured vials are tested on days 8, 15, 22, and 32 versus freshly opened vials. (The acceptance criteria for this testing, likely related to recovery or percent change, is implied to be met for the reported stability duration, but the specific numerical criteria are not explicitly stated in the provided text beyond the 3% for shelf-life, which likely applies here too). | ||
| Traceability | The assigned values of the Urinary/Cerebrospinal Fluid Protein Calibrator must be traceable to a recognized standard. | Result: Traceable to the National Institute of Standards and Technology - Standard Reference Material 927 (NIST SRM 927). |
| Value Assignment | Verification of assigned values for the commercial lot calibrator. | Testing Method: The final bottle assignment for the test calibrator level of the commercial lot is tested with N = 90 replicates, using multiple reagent lots on multiple instruments. |
| Result/Process: Purified human IgG and human serum albumin are weighed into an aqueous solution to form a stock solution, assayed, and further diluted. Master Pool values are verified using an instrument calibrated with a previously approved Master Lot as a control. The control Master Pool values were assigned with an instrument calibrated with the standard reference material. Purified human IgG and human serum albumin are gravimetrically added to stock solution at target concentrations and verified using an instrument calibrated with Master Pool assigned values. Calculated quantities of human serum stock solution are added to a base matrix to form the calibrator level, which is then verified using an instrument calibrated with the Master Pool assigned values. The 90 replicates with multiple reagent lots on multiple instruments confirm the robustness and consistency of the value assignment process. (Explicit acceptance criteria for this specific N=90 test, such as CV% or bias, are not provided but are implied to be met by the described verification process). |
2. Sample Size Used for the Test Set and the Data Provenance
- Shelf-life Stability: The text mentions "real-time data on file at Dade Behring, Inc." This indicates the data is prospective (generated over time as the product ages). The specific number of samples or vials tested for the 12-month shelf-life is not explicitly stated.
- Open Vial Stability (On-board & Recapped): "Vials are opened /punctured on day zero. A quantity sufficient for multiple calibrations is removed and the vials are recapped and stored at 2 - 8 °C. Opened/punctured vials are tested on days 8, 15, 22, and 32 versus freshly opened vials." The exact number of "vials" is not given, but it implies a cohort of vials was tracked prospectively over time.
- Value Assignment: N = 90 replicates are used for final bottle assignment of the commercial lot. The data provenance is internal to Dade Behring, Inc., as it's part of their manufacturing and quality control process. It is prospective for each commercial lot manufactured.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and the Qualifications of Those Experts
This type of information is not applicable to this device. This device is a calibrator, which inherently has a "ground truth" derived from its chemical composition and traceability to a recognized standard (NIST SRM 927), rather than requiring expert interpretation of medical images or clinical data. Expertise in metrology and analytical chemistry is implicit in calibrator development and value assignment, but not in the format of "experts used to establish ground truth" as would be seen in diagnostic imaging studies.
4. Adjudication Method for the Test Set
This is not applicable to this device. Adjudication methods (like 2+1, 3+1) are typically used when subjective expert opinion is involved in establishing ground truth for diagnostic decisions. For a calibrator, the "truth" is based on objective measurements and traceability to a primary standard.
5. If a Multi Reader Multi Case (MRMC) Comparative Effectiveness Study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance
This is not applicable to this device. This device is an in vitro diagnostic calibrator, not an AI-powered diagnostic tool. Therefore, MRMC studies involving human readers and AI assistance are irrelevant to its evaluation.
6. If a Standalone (i.e. algorithm only without human-in-the-loop performance) was done
This is not applicable to this device. The device is a calibrator, a physical reagent used to set the measurement scale of an instrument. It does not involve an algorithm or "standalone" performance in the sense of an AI system. Its performance is measured by its chemical stability and its ability to correctly assign values based on its composition and traceability.
7. The Type of Ground Truth Used
The ground truth for the calibrator's values and performance is based on:
- Chemical Composition/Gravimetric Addition: Purified human IgG and human serum albumin are "weighed into an aqueous solution" and "added gravimetrically to stock solution at target concentrations." This established a primary, known concentration.
- Traceability to NIST SRM 927: The assigned values are explicitly stated to be "traceable to the National Institute of Standards and Technology- Standard Reference Material 927." This is the ultimate ground truth for the concentration of the analytes.
- Reference Instrument/Master Pool: The Master Pool values are verified using an instrument calibrated with a "previously approved Master Lot" and ultimately with "the standard reference material." This hierarchical traceability ensures the accuracy of assigned values.
8. The Sample Size for the Training Set
This is not applicable in the conventional sense of an AI model's training set. Calibrators are developed and manufactured based on established chemical and analytical principles. Their "training" is in the form of R&D, formulation, and manufacturing process controls, rather than data fed to an algorithm. The "90 replicates" mentioned are for the final verification (test set) of each commercial lot, not a training set.
9. How the Ground Truth for the Training Set Was Established
As above, this is not applicable in the context of an AI model's training set. The "ground truth" for developing and manufacturing the calibrator is the known, precisely measured chemical concentrations of its components, directly traceable to recognized
standards like NIST SRM 927. The manufacturing process aims to produce a product that consistently meets these known, objective chemical specifications.
§ 862.1150 Calibrator.
(a)
Identification. A calibrator is a device intended for medical purposes for use in a test system to establish points of reference that are used in the determination of values in the measurement of substances in human specimens. (See also § 862.2 in this part.)(b)
Classification. Class II (special controls). The device is exempt from the premarket notification procedures in subpart E of part 807 of this chapter subject to the limitations in § 862.9.