(25 days)
The IMMULITE® Total T3 Calibration Verification Material (CVM) is intended for in vitro diagnostic use in the verification of calibration of the IMMULITE Total T3 assay on the IMMULITE 2000 systems.
The IMMULITE® Total T4 Calibration Verification Material (CVM) is for in vitro diagnostic use in the verification of calibration of the IMMULITE Total T4 assay on the IMMULITE 2000 systems.
The IMMULITE® TBG Calibration Verification Material (CVM) is for in vitro diagnostic use in the verification of calibration of the IMMULITE TBG assay on the IMMULITE 2000 systems.
The IMMULITE® FSH Calibration Verification Material (CVM) is for in vitro diagnostic use in the verification of calibration of the IMMULITE FSH assay on the IMMULITE 2000 systems.
The IMMULITE® Estradiol Calibration Verification Material (CVM) is for in vitro diagnostic use in the verification of calibration of the IMMULITE Estradiol assay on the IMMULITE 2000 systems.
IMMULITE® 2000 Total T3 Calibration Verification Material (CVM) contains one set of four vials, 3 mL each. LT3CVM1 contains T3-free human serum matrix with preservatives. LT3CVM2, LT3CVM3 and LT3CVM4 contain low, intermediate and high levels of T3 respectively, in human serum matrix with preservatives. CVMs are supplied frozen in lyophilized form.
IMMULITE® 2000 Total T4 Calibration Verification. Material (CVM) contains one set of four vials, 3 mL each. LT4CVM1 contains T4-free human serum matrix with preservatives. LT4CVM2, LT4CVM3 and LT4CVM4 contain low, intermediate and high levels of T4 respectively, in human serum matrix with preservatives. CVMs are supplied frozen in lyophilized form.
IMMULITE® 2000 TBG Calibration Verification Material (CVM) contains one set of four vials, 2.0 mL each. LTBCVM1 contains a bovine-based matrix with preservatives. LTBCVM2, LTBCVM3 and LTBCVM4 contain low, intermediate and high levels of TBG respectively, in human-based matrix with preservatives. CVMs are supplied frozen in lyophilized form.
IMMULITE® 2000 FSH Calibration Verification Material (CVM) contains one set of four vials, 3 mL each. LFSCVM1 contains a bovine serum with preservatives. LFSCVM2. LFSCVM3 and LFSCVM4, 2 mL each, contain low, intermediate and high levels of human source FSH respectively, in bovine serum matrix with preservatives. CVMs are supplied frozen in lyophilized form.
IMMULITE® 2000 Estradiol Calibration Verification Material (CVM) contains one set of four vials. 2 mL each. LE2CVM1 contains human serum with preservatives. LE2CVM2. LE2CVM3 and LE2CVM4 contain low, intermediate and high levels of human source Estradiol respectively, in human serum matrix with preservatives. CVMs are supplied frozen in lyophilized form.
The provided text describes the acceptance criteria and performance studies for several In Vitro Diagnostic (IVD) Calibration Verification Materials (CVMs) manufactured by Siemens Healthcare Diagnostics Inc. These CVMs are intended for use with IMMULITE 2000 systems to verify the calibration of specific assays (Total T3, Total T4, TBG, FSH, and Estradiol).
Since the information is provided for multiple CVMs within a single 510(k) summary (K133124), I will choose one representative CVM to fully describe the acceptance criteria and study that proves the device meets them. I will select the IMMULITE® 2000 Total T3 Calibration Verification Material for this detailed description.
It's important to note that this document is for a Calibration Verification Material (CVM), which is a type of quality control material used in laboratory diagnostics, not an AI-powered diagnostic device for image interpretation. Therefore, many of the requested items related to AI/human reader studies, expert ground truth, and MRMC studies are not applicable to this type of device.
Here's the description for the IMMULITE® 2000 Total T3 Calibration Verification Material:
1. A table of acceptance criteria and the reported device performance
The device performance is demonstrated through stability studies and value assignment testing. The primary acceptance criteria for stability are based on the dose value of the CVM falling within a specified percentage difference of the assigned dose.
Table: Acceptance Criteria for Stability of IMMULITE® 2000 Total T3 CVM
| CVM Level | Assigned Dose (ng/dL) | Guideline Criteria % difference to assigned dose | Acceptable Dose range (ng/dL) | Review Limits (if guideline criteria not met) |
|---|---|---|---|---|
| LT3CVM1 | 0.00 | not applicable | ≤40.00 | Controls are within 2SD of target from stability calibrator curve |
| LT3CVM2 | 101 | ±12% | 88.88 - 113.12 | Controls are within 2SD of target from stability calibrator curve |
| LT3CVM3 | 215 | ±8% | 197.8 - 232.2 | Controls are within 2SD of target from stability calibrator curve |
| LT3CVM4 | 610 | ±15% | 518.5 - 701.5 | Controls are within 2SD of target from stability calibrator curve |
The document states that the stability study was conducted to validate shelf life claim and "ensure that it maintains optimal product performance on IMMULITE 2000 platforms throughout the established shelf life of the CVM." The CVMs are stable up to 5 years when stored frozen at -20°C prior to opening. The protocol involved running CVMs and reference CVMs in duplicate (as a minimum) and determining the dose value from the reference calibrator curve.
For actual reported performance, the summary states: "Performance testing has been carried out to demonstrate that this device meets the performance specifications for its intended use." And "The substantial equivalence of the device is supported by the non-clinical testing and was found to be comparable and supports the claims of substantial equivalence, product safety and effectiveness." While the summary outlines the acceptance criteria, it does not provide specific numerical results from the stability studies that "prove" the device met these criteria, beyond the general statement of compliance. This is typical for a 510(k) summary, which focuses on summarizing the regulatory submission rather than detailing all raw data.
2. Sample size used for the test set and the data provenance
-
Test Set (Stability Study):
- The stability study involved testing CVM levels (LT3CVM1, LT3CVM2, LT3CVM3, LT3CVM4) at specific time-points (1, 1092, 1456, 1820 days).
- The CVMs and reference CVMs were run in "duplicate (as a minimum)".
- Data Provenance: Not explicitly stated, but assumed to be internal laboratory testing by Siemens Healthcare Diagnostics Inc. (manufacturer), likely in the US given the submission to the FDA. The study is prospective in nature, as it's designed to validate shelf life (i.e., forward-looking stability over time).
-
Test Set (Value Assignment and Expected Values):
- To establish Expected Values/Target Values/Reference Range (Table 4), each CVM level was tested for a total of 27 replicates. This was comprised of 9 runs and 3 replicates per run, across 6 different reagent kit lots and 8 IMMULITE 2000 systems.
- Data Provenance: Not explicitly stated, but assumed to be internal laboratory testing by Siemens Healthcare Diagnostics Inc. (manufacturer), likely in the US. This data collection would be prospective for the purpose of establishing these values.
- Additionally, 25 patient samples (5 normal patients samples and 20 patient panel samples) were used to validate CVM value assignments, and "Two tri-level commercially available controls" were also used. The provenance of these patient samples is not specified (e.g., country of origin, retrospective/prospective collection methods).
3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts
This is not applicable. This is a Calibration Verification Material, which is a chemical reference material/quality control, not an AI or image analysis device that requires expert human interpretation for its output. The "ground truth" or reference values for the CVMs are established through a process called "Value Assignment" which relies on assigned reference calibrators, gravimetric preparation, and internal quality control procedures, rather than human expert consensus on medical images or clinical outcomes.
4. Adjudication method (e.g. 2+1, 3+1, none) for the test set
This is not applicable for the same reasons as #3. Adjudication methods are typically for subjective assessments (e.g., image interpretation) where multiple readers disagree. For this IVD device, performance is measured against quantitative analytical criteria.
5. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance
This is not applicable. This is an IVD device (Calibration Verification Material), not an AI-powered diagnostic system that assists human readers.
6. If a standalone (i.e. algorithm only without human-in-the loop performance) was done
This is not applicable. This is an IVD device, not an algorithm. Its "performance" is its ability to maintain its assigned values and function within specified analytical limits.
7. The type of ground truth used (expert consensus, pathology, outcomes data, etc)
The "ground truth" for these Calibration Verification Materials (CVMs) is established through a process called Value Assignment. Key elements of this include:
- Traceability to internal assigned reference calibrators: These calibrators are prepared using Total T3 antigen stock solution.
- Traceability to internal material which is gravimetrically prepared. This implies a gravimetric (mass-based) method for the primary reference, which is a highly accurate quantitative method.
- The CVM values are calculated based on recovered values from runs using these assigned reference calibrators and then averaged across all systems.
- Validation of CVM value assignments is performed by calculating the recovery of patient samples (5 normal, 20 panel) and commercially available controls, with controls required to fall within their target ranges.
In essence, the ground truth is an analytically established reference value system linked to gravimetric preparations and verified against well-characterized patient samples and commercial controls.
8. The sample size for the training set
- Training Set (for "Value Assignment" and "Expected Values"): For the establishment of expected values, each CVM level was tested on a total of 27 replicates (9 runs, 3 replicates per run) across 6 reagent kit lots and 8 IMMULITE 2000 systems.
- Additionally, two tri-level commercially available controls and 25 patient samples (5 normal patients, 20 patient panel samples) were used to validate CVM value assignments.
It's important to clarify that for a CVM, there isn't a "training set" in the machine learning sense. These samples and runs are part of the thorough characterization and validation process to assign accurate values and ranges to the CVMs.
9. How the ground truth for the training set was established
As explained in #7, the "ground truth" (assigned/target values for the CVM levels) is established through a rigorous internal value assignment process:
- The CVMs are a subset of 6-level Total T3 calibrators, which are not commercialized but used internally for manufacturing and release testing.
- These calibrators and CVMs are value assigned using assigned reference calibrators.
- The assigned reference calibrators are prepared using Total T3 antigen stock and are traceable to internal material that has been gravimetrically prepared.
- CVMs are manufactured using qualified materials and measurement procedures.
- CVM dose values are generated using a curve derived from the assigned reference calibrators.
- CVM values are calculated from recovered values for each run on each instrument independently, then averaged across all systems.
- Quality control for this process involves verifying the recovery of patient samples (normal, panel) and commercially available controls, ensuring they fall within their expected target ranges.
This process ensures that the assigned values for the CVMs are accurate and traceable to fundamental analytical methods and established references.
§ 862.1660 Quality control material (assayed and unassayed).
(a)
Identification. A quality control material (assayed and unassayed) for clinical chemistry is a device intended for medical purposes for use in a test system to estimate test precision and to detect systematic analytical deviations that may arise from reagent or analytical instrument variation. A quality control material (assayed and unassayed) may be used for proficiency testing in interlaboratory surveys. This generic type of device includes controls (assayed and unassayed) for blood gases, electrolytes, enzymes, multianalytes (all kinds), single (specified) analytes, or urinalysis controls.(b)
Classification. Class I (general controls). Except when intended for use in donor screening tests, quality control materials (assayed and unassayed) are exempt from the premarket notification procedures in subpart E of part 807 of this chapter subject to the limitations in § 862.9.