(90 days)
The Trinidad CH Vancomycin (Vanc) assay is for in vitro diagnostic use in the quantitative measurement of vancomycin in human serum or plasma on the Trinidad CH System. Vanc test results may be used in the diagnosis and treatment of vancomycin overdose and in monitoring levels of vancomycin to ensure appropriate therapy.
The Trinidad CH Drug 3 Calibrator (DRUG3 CAL) is intended for in vitro diagnostic use in the calibration of Vancomycin (Vanc) on the Trinidad CH System.
The Trinidad CH Vancomycin (Vanc) assay is based on a homogeneous particle enhanced turbidimetric inhibition immunoassay (PETINIA) technique which uses a synthetic particle-vancomycin conjugate (PR) and monoclonal vancomycin specific antibody (Ab). Vancomycin present in the sample competes with vancomycin on the particles for available antibody, thereby decreasing the rate of aggregation. Hence, the rate of aggregation is inversely proportional to the concentration of vancomycin in the sample. The rate of aggregation is measured using bichromatic turbidimetric readings at 545 nm and 694 nm.
The Trinidad CH Drug 3 Calibrator (DRUG3 CAL) is a 5 level calibrator product prepared from bovine serum base product.
The Siemens Healthcare Diagnostics Inc. Trinidad CH Vancomycin (Vanc) assay and Trinidad CH Drug 3 Calibrator (DRUG3 CAL) underwent several studies to establish their performance and substantial equivalence to predicate devices. Here's a breakdown based on the provided document:
1. Table of Acceptance Criteria and Reported Device Performance
The document does not explicitly present a formal "acceptance criteria" table with pass/fail thresholds for all metrics. However, it presents the results of various performance studies which implicitly serve as evidence for meeting regulatory requirements for substantial equivalence. The "Acceptance Criteria" column below is inferred from the document's reporting of satisfactory results against established clinical laboratory guidelines (e.g., CLSI documents) or from comparisons to the predicate device.
| Performance Metric | Acceptance Criteria (Inferred) | Reported Device Performance (Trinidad CH Vancomycin / DRUG3 CAL) |
|---|---|---|
| Detection Limit (LoB) | Demonstrated low limit of blank according to CLSI EP17-A2. | 0.1 µg/mL |
| Detection Limit (LoD) | Demonstrated low limit of detection according to CLSI EP17-A2. | 0.2 µg/mL |
| Limit of Quantitation (LoQ) | LoQ should support the claimed measuring interval (3.0 µg/mL) with a total error ≤ 20%. | 2.8 µg/mL (measured), supporting a claim of 3.0 µg/mL with a total error ≤ 20%. |
| Linearity | Nonlinear coefficients are nonsignificant (all p values ≥ 0.05). If significant, allowable bias ≤ 0.5 µg/mL or 10%, whichever is greater. | Passed for all 10 levels across a range of 0.9 to 51.1 µg/mL. Absolute Bias ranged from -0.3 to 0.4 µg/mL, and % Bias ranged from -2.3% to 0.9%, all well within allowable bias limits. |
| Precision (Repeatability - SD) | Within acceptable variation for clinical assays (e.g., as per CLSI EP05-A3 guidelines). | Serum QC: 0.16 µg/mL (2.6% CV). Serum (11.2 µg/mL): 0.20 µg/mL (1.8% CV). Serum QC (17.5 µg/mL): 0.28 µg/mL (1.6% CV). Serum (32.7 µg/mL): 0.46 µg/mL (1.4% CV). Plasma (45.8 µg/mL): 0.78 µg/mL (1.7% CV). |
| Precision (Within-Lab - SD) | Within acceptable variation for clinical assays (e.g., as per CLSI EP05-A3 guidelines). | Serum QC: 0.18 µg/mL (2.7% CV). Serum (11.2 µg/mL): 0.23 µg/mL (2.0% CV). Serum QC (17.5 µg/mL): 0.35 µg/mL (2.0% CV). Serum (32.7 µg/mL): 0.67 µg/mL (2.0% CV). Plasma (45.8 µg/mL): 0.83 µg/mL (1.8% CV). |
| Interferences | Bias exceeding 10% is considered interference. No significant interference expected from common interfering substances at specified concentrations. | No interference detected at the following concentrations: Hemoglobin 600 mg/dL, Conjugated Bilirubin 20 mg/dL, Unconjugated Bilirubin 20 mg/dL, Lipemia (Intralipid) 1000 mg/dL. Note: Vancomycin crystalline degradation product (CDP-1) at 20 µg/mL showed cross-reactivity (21.0% at 0 µg/mL Vanc, 19.1% at 10 µg/mL Vanc). |
| Method Comparison (Serum) | Good agreement with predicate device (Dimension RxL VANC) when tested with patient samples (e.g., high correlation coefficient, low bias). | N=100, r = 0.997, Regression Equation: $y = 1.04x - 1.04 \frac{\text{µg}}{\text{mL}}$ for a sample interval of 4.4-48.1 µg/mL. (Demonstrated good agreement). |
| Matrix Equivalency (Plasma vs Serum) | Good agreement between plasma and serum samples across the assay's measuring interval (e.g., high correlation coefficient, low bias). | N=60, r = 0.990, Regression Equation: $y = 1.00x + 0.49 \mu g/mL$ for a sample interval of 4.0–43.3 µg/mL. (Demonstrated equivalency). |
2. Sample Size Used for the Test Set and the Data Provenance
- Detection Limit (LoB/LoD):
- LoB: 4 samples with no analyte tested (N=5) for 3 days, one run per day, 3 reagent lots. Total reps = 60 to determine rank for non-parametric approach.
- LoD: 4 low serum samples tested (N=5) for 3 days, one run per day, 3 reagent lots.
- Limit of Quantitation (LoQ): 4 low serum samples processed on three reagent lots for three days, on one instrument. Total of 60 measurements per reagent lot, leading to a total of 180 determinations.
- Linearity Study: 10 samples (prepared by mixing high and low concentration samples). Six replicates were measured for each sample.
- Precision Studies: 80 replicates (n=2 replicates, two times a day for at least 20 days) for controls, serum, and plasma pools.
- Interferences: Conducted using a "paired difference worst case scenario" approach with fresh sample pools containing low or high levels of vancomycin. Specific N not provided for individual interferents.
- Method Comparison: 100 "remnant de-identified samples" (native, spiked, and diluted).
- Matrix Equivalency: 60 matched samples (serum and lithium heparin plasma). Some samples were diluted or spiked.
Data Provenance:
- All studies were "conducted internally by Siemens Healthcare Diagnostic Inc. R&D organization personnel."
- For the Method Comparison, "Remnant de-identified samples were tested. No patient history information was obtained on these samples." This suggests a retrospective data collection from a clinical laboratory or sample bank, likely within the US, given Siemens Healthcare Diagnostics is a US-based entity and the FDA submission is US-centric.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and the Qualifications of Those Experts
The document does not explicitly mention the use of experts to establish a "ground truth" in the sense of clinical interpretation or diagnosis for the test sets.
For quantitative assays like this:
- The "ground truth" for linearity, precision, and detection limits is based on the known concentrations of prepared standards or spiked samples.
- For method comparison, the "ground truth" comes from the measurements generated by the predicate device.
- The personnel conducting the study were "laboratory technicians with training similar to personnel who would conduct the tests in a hospital laboratory setting," and they were "trained on the operation of both the device and the predicate device." These individuals are not referred to as "experts" in the clinical sense of radiologists or pathologists, but rather as trained laboratory professionals.
4. Adjudication Method for the Test Set
Not applicable in the context described. Adjudication methods (like 2+1 or 3+1) are typically used for establishing ground truth in diagnostic studies involving subjective interpretation (e.g., image analysis by multiple radiologists). This submission describes performance studies for a quantitative measurement assay where the "ground truth" is either inherent to the sample preparation (e.g., spiked concentrations) or established by a reference method (the predicate device). No human interpretation requiring adjudication is mentioned.
5. If a Multi Reader Multi Case (MRMC) Comparative Effectiveness Study Was Done, If So, What Was the Effect Size of How Much Human Readers Improve with AI vs Without AI Assistance
No, a Multi-Reader Multi-Case (MRMC) comparative effectiveness study was not conducted, nor is it applicable. MRMC studies evaluate the performance of human readers, often with or without AI assistance, on a set of cases, typically for diagnostic tasks like image interpretation. This submission is for an in vitro diagnostic assay that quantitatively measures vancomycin levels, not for an AI-powered diagnostic imaging device or a clinical decision support system that directly involves human "readers" interpreting cases.
6. If a Standalone (i.e. algorithm only without human-in-the loop performance) Was Done
This device is a standalone algorithm (an in vitro diagnostic assay) in the sense that its performance is measured directly and quantitatively on biological samples to determine vancomycin concentration. The "algorithm" here is the chemical reaction and measurement process (PETINIA technique) implemented on the Trinidad CH System, which then outputs a numerical value. There is no human "in-the-loop" interpretation component for the direct measurement itself; humans operate the instrument and interpret the final quantitative result. The performance data presented (linearity, precision, detection limits, method comparison) are effectively "standalone" performance metrics of the assay system.
7. The Type of Ground Truth Used (expert consensus, pathology, outcomes data, etc.)
The ground truth used for the studies varied by the specific test:
- Detection Limits, LoQ, Linearity: Ground truth was established by known, prepared concentrations of vancomycin in control matrices (e.g., bovine serum base, spiked native serum).
- Method Comparison: Ground truth was established by the measurements from the legally marketed predicate device (Dimension RxL VANC assay), which is considered an accepted method for vancomycin quantification.
- Matrix Equivalency: Ground truth for comparison was the measurements from the new device itself on serum samples to compare against its performance on plasma from the same patient.
8. The Sample Size for the Training Set
The document does not explicitly mention a "training set" in the context of machine learning or AI algorithms. As this is a traditional in vitro diagnostic assay based on an immunoassay technique, it is developed and validated through laboratory experiments, not typically "trained" on a dataset in the way an AI algorithm would be. The experiments described (e.g., development of the assay, optimization of reagents, calibration) are analogous to the development phase, but not termed a "training set."
9. How the Ground Truth for the Training Set Was Established
As explained above, there isn't a "training set" in the AI sense for this type of IVD device. The development of such an assay involves:
- Careful formulation of reagents (synthetic particle-vancomycin conjugate, monoclonal vancomycin specific antibody).
- Optimization of reaction conditions and measurement parameters.
- Calibration using highly characterized standards (like the Trinidad CH Drug 3 Calibrator, which is prepared from bovine serum base). The calibrators themselves would be assigned values based on precise preparation and analytical verification against reference methods or materials.
The "ground truth" during assay development would stem from known quantities of vancomycin standards and reference materials, used to ensure the assay accurately measures concentrations across its intended range.
§ 862.3950 Vancomycin test system.
(a)
Identification. A vancomycin test system is a device intended to measure vancomycin, an antibiotic drug, in serum. Measurements obtained by this device are used in the diagnosis and treatment of vancomycin overdose and in monitoring the level of vancomycin to ensure appropriate therapy.(b)
Classification. Class II.