(89 days)
Immunoassay for the in vitro quantitative determination of PCT (procalcitonin) in human serum and plasma (K2 and K3 EDTA, Li-Heparin).
The Elecsys BRAHMS PCT assay is intended for use to determine the change of PCT over time as an aid in assessing the cumulative 28-day risk of all-cause mortality for patients diagnosed with severe sepsis or septic shock in the Intensive Care Unit (ICU) or when obtained in the emergency department or other medical wards prior to ICU admission. The electrochemiluminescence immunoassay "ECLIA" is intended for use on Elecsys and cobas e immunoassay analyzers.
Procalcitonin (PCT) is a biomarker associated with the inflammatory response to bacterial infection that aids in the risk assessment of critically ill patients on their first day of Intensive Care Unit (ICU) admission for progression to severe sepsis and septic shock. The percent change in PCT level over time aids in the prediction of cumulative 28-day mortality in patients with severe sepsis and septic shock.
A PCT level that declines ≤ 80 % from the day that severe sepsis or septic shock was clinically diagnosed (Day 0) to four days after clinical diagnosis (Day 4) is associated with higher cumulative 28-day risk of all-cause mortality than a decline > 80 %.
The combination of the first PCT level (≤ 2.0 ng/mL or > 2.0 ng/mL) at initial diagnosis of severe sepsis or septic shock with the patient's clinical course and the change in PCT level over time until Day 4 provides important additional information about the mortality risk.
The PCT level on Day 1 (the day after severe septic shock is first clinically diagnosed) can be used to calculate the percent change in PCT level at Day 4 if the Day 0 measurement is unavailable.
The Elecsys BRAHMS PCT assay is a two-step sandwich immunoassay with streptavidin microparticles and an electrochemiluminescence detection system. PCT in the sample reacts with these labeled antibodies to form a sandwich complex. This complex binds to streptavidin coated magnetic microparticles, which are magnetically captured onto an electrode. Application of voltage to the electrode induces chemiluminescence which is measured by a photomultiplier tube. Results are determined via a calibration curve which is instrument-specifically generated by 2-point calibration and a master curve provided via the reagent barcode. An optional Procalcitonin CalCheck product is also available.
The provided text describes the Elecsys BRAHMS PCT assay, an immunoassay for the quantitative determination of procalcitonin (PCT) in human serum and plasma, intended to aid in assessing the 28-day risk of all-cause mortality for patients with severe sepsis or septic shock. The document outlines the performance characteristics and a clinical study conducted to support its indications for use.
Here's an analysis of the acceptance criteria and the study that proves the device meets them, based on the provided text:
1. Table of Acceptance Criteria and Reported Device Performance
The document does not explicitly present a table of acceptance criteria with numerical targets for clinical performance endpoints (e.g., specific sensitivity, specificity, or predictive values that the device needed to meet to be considered "accepted"). Instead, it presents the results of the clinical study and infers substantial equivalence to a predicate.
However, the non-clinical performance evaluation sections (analytical sensitivity, precision, linearity, etc.) implicitly have acceptance criteria, as the reported performance values are deemed satisfactory to demonstrate substantial equivalence. The document states: "The information provided in this 510(k) Premarket Notification will support a determination of substantial equivalence for the Elecsys BRAHMS PCT test system. The data supports a safe, effective device which performs as well as or better than the predicate device." This strongly suggests that the reported values met internal or regulatory expectations for substantial equivalence.
Here's a table summarizing the reported device performance from the provided text, derived from the non-clinical and clinical performance evaluations:
| Performance Metric | Acceptance Criteria (Inferred from Substantial Equivalence and Study Design) | Reported Device Performance (Elecsys BRAHMS PCT) |
|---|---|---|
| Non-Clinical Performance | ||
| Precision (cobas e 411 Analyzer) | Expected to be within acceptable variability for a diagnostic immunoassay. | Repeatability: |
- HS1: 5.03% CV @ 0.079 ng/mL
- HS2: 1.71% CV @ 0.442 ng/mL
- HS3: 2.35% CV @ 1.49 ng/mL
- HS4: 1.38% CV @ 27.5 ng/mL
- HS5: 2.23% CV @ 71.4 ng/mL
- HS6: 1.35% CV @ 88.1 ng/mL
- PC1: 1.91% CV @ 0.456 ng/mL
- PC2: 1.83% CV @ 9.07 ng/mL
Intermediate Precision: - HS1: 6.44% CV @ 0.079 ng/mL
- HS2: 3.14% CV @ 0.442 ng/mL
- HS3: 3.40% CV @ 1.49 ng/mL
- HS4: 3.24% CV @ 27.5 ng/mL
- HS5: 3.66% CV @ 71.4 ng/mL
- HS6: 3.18% CV @ 88.1 ng/mL
- PC1: 3.21% CV @ 0.456 ng/mL
- PC2: 3.35% CV @ 9.07 ng/mL |
| Limit of Blank (LoB) | Determined as the 95th percentile of blank-sample measurements. | 0.015 ng/mL |
| Limit of Detection (LoD) | Smallest amount of analyte detectable with 95% probability. | 0.02 ng/mL |
| Limit of Quantitation (LoQ) | Lowest concentration quantifiable with intermediate precision ≤ 20%. | 0.06 ng/mL |
| Measuring Range | Needs to cover clinically relevant range. | 0.02 – 100 ng/mL |
| Hook Effect | No hook effect within tested concentration. | No hook effect up to 1000 ng/mL |
| Interferences | Recovery within ± 15% of initial value for endogenous and exogenous substances. | Unaffected by icterus ( 80% or ≤ 80%) significantly associated with 28-day cumulative mortality (Two-sided Fisher's Exact Test p-value = 0.002). Adjusted for ICU vs. non-ICU: p-value = 0.020. |
| Prognostic Accuracy (Day 0-4 interval) | Sensitivity and Specificity values demonstrating utility in risk assessment. | ICU Patients: - Sensitivity: 73.4% (62.9-83.8)
- Specificity: 35.0% (28.2-41.8)
Non-ICU Patients: - Sensitivity: 72.3% (55.9-88.6)
- Specificity: 44.4% (38.4-50.3) |
| Hazard Ratio for 28-day all-cause mortality (ΔPCT) | Hazard Ratio (HR) significantly > 1.0 for positive ΔPCT (≤ 80%). | Univariate Model (Per-Protocol): - ΔPCT4.0 (≤ 80% vs. > 80%): HR = 1.80 (1.15-2.82); p=0.011
- ΔPCT4.1 (≤ 80% vs. > 80%): HR = 1.61 (1.04-2.49); p=0.034 |
2. Sample Size Used for the Test Set and Data Provenance
-
Test Set Sample Size:
- Method Comparison: 2617 samples were used to compare the Elecsys BRAHMS PCT assay with the predicate device (BRAHMS PCT sensitive KRYPTOR®).
- Clinical Performance Evaluation: 858 adult patients were enrolled. The "per protocol analysis population" was 598 subjects.
- Reference Range: 282 self-reported healthy individuals.
-
Data Provenance:
- Clinical Performance Evaluation: The clinical trial was a prospective study conducted across 13 investigational sites in the US. This indicates prospective data from multiple US sites.
- Reference Range: Implied to be from healthy individuals, likely domestic (US) given the submission context.
- Non-Clinical Studies: The text does not explicitly state the country of origin for all non-clinical samples or specific data provenance for each analytical study, but it implies internal testing by Roche Diagnostics, which has sites in Germany and the United States (as per Establishment Registration).
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts
The document describes a clinical study for an in vitro diagnostic (IVD) device that measures a biomarker (PCT) to aid in risk assessment for mortality. For IVD devices, "ground truth" for clinical performance is typically defined by the clinical outcome itself, not by expert consensus on interpreting images or other subjective data.
-
For the clinical performance study: The ground truth was cumulative 28-day all-cause mortality. This is an objective clinical outcome. It does not involve expert interpretation or adjudication in the way that, for example, a radiology AI model might. The diagnosis of "severe sepsis or septic shock" would have been established by clinicians based on recognized medical criteria at each investigational site. The "experts" in this context would be the attending clinicians who managed the patients and determined their clinical status and outcomes, but their role was in providing the clinical data, not in acting as a ground truth panel for the PCT assay results themselves.
-
For the reference range study: The ground truth was "self-reported healthy individuals," which implies a general population considered free from disease, not requiring expert adjudication of individual cases.
4. Adjudication Method for the Test Set
As mentioned above, the primary clinical endpoint was objective (28-day all-cause mortality). There is no indication of an "adjudication method" in the sense of a panel of experts reviewing the outcome or interpreting the device's results. Outcomes like mortality are typically recorded facts. The diagnosis of severe sepsis or septic shock would have been made by the local clinicians at each site.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was done
No, an MRMC comparative effectiveness study was not done. This type of study is typically associated with imaging devices where multiple human readers interpret cases and their performance is compared with and without AI assistance. The Elecsys BRAHMS PCT assay is an in vitro diagnostic (blood test) device quantifying a biomarker, not an imaging device.
6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) was done
Yes, the very nature of an in vitro diagnostic (IVD) test like the Elecsys BRAHMS PCT assay is that it operates as a standalone algorithm/device. It quantitatively measures PCT levels from a human specimen. The "algorithm" in this context is the immunochemical reaction and the analyzer's detection system, which then calculates a numerical PCT value. The clinical performance study evaluates this standalone measurement in relation to a clinical outcome (mortality risk) and how its results can be integrated into clinical decision-making by human clinicians, but the device itself functions independently to produce the PCT value.
7. The Type of Ground Truth Used
- Clinical Performance Evaluation: The ground truth was outcomes data, specifically cumulative 28-day all-cause mortality for patients diagnosed with severe sepsis or septic shock.
- Reference Range: The ground truth was based on a population of self-reported healthy individuals.
- Non-Clinical Studies (e.g., Linearity, Dilution): Ground truth would be the known concentration of spiked PCT or reference materials.
8. The Sample Size for the Training Set
The document does not explicitly mention a "training set" in the context of machine learning or AI models. For an IVD assay like this, "training" usually refers to the development and optimization process (e.g., optimizing reagent concentrations, reaction times), rather than a distinct dataset used for machine learning. The clinical performance data presented serves as a validation dataset to demonstrate the assay's utility when used according to its indications.
If "training set" refers to the data used to initially establish the assay's performance characteristics or calibrate it, then:
- Calibrator Value Assignment: The calibrators are run in duplicate on at least six cobas e 411 analyzers and least three cobas e 601/cobas e 602/MODULAR ANALYTICS E170 analyzers with "all Elecsys BRAHMS PCT reagent lots available."
- PreciControl Value Assignment: Controls are run in duplicate on at least six analyzers of the master analyzer platform and compared to a master calibration curve.
However, these are more akin to standardization and quality control procedures rather than a machine learning training set for an "algorithm."
9. How the Ground Truth for the Training Set Was Established
Since an explicit "training set" for an AI/ML algorithm is not described, the concept of establishing its ground truth isn't directly applicable in that sense.
For the assay's development and analytical performance:
- Calibrators: Recombinant PCT in a human serum matrix (PCT CalSet). Their assigned values are determined through multiple runs on multiple analyzers using a Master Calibration Curve and must meet pre-defined acceptance criteria for associated controls (PreciControl PCT).
- Controls: Recombinant PCT in human plasma (PreciControl PCT). Their assigned values are read from the master calibration curve and must meet pre-defined acceptance criteria.
- Analytical Specificity/Interference: Ground truth is established by spiking known concentrations of interferents into known PCT samples.
In essence, the "ground truth" for the analytical performance characteristics is established by using controlled, manufactured materials with known properties or by challenging the assay with known substances at defined concentrations.
§ 866.3215 Device to detect and measure non-microbial analyte(s) in human clinical specimens to aid in assessment of patients with suspected sepsis.
(a)
Identification. A device to detect and measure non-microbial analyte(s) in human clinical specimens to aid in assessment of patients with suspected sepsis is identified as an in vitro device intended for the detection and qualitative and/or quantitative measurement of one or more non-microbial analytes in human clinical specimens to aid in the assessment of patients with suspected sepsis when used in conjunction with clinical signs and symptoms and other clinical and laboratory findings.(b)
Classification. Class II (special controls). The special controls for this device are:(1) Premarket notification submissions must include the device's detailed Indications for Use statement describing what the device detects and measures, the results provided to the user, whether the measure is qualitative and/or quantitative, the clinical indications for which the test is to be used, and the specific population(s) for which the device use is intended.
(2) Premarket notification submissions must include detailed documentation of the device description, including (as applicable), all device components, software, ancillary reagents required but not provided, explanation of the device principle and methodology, and for molecular devices include detailed documentation of the primer/probe sequence, design, and rationale for sequence selection.
(3) Premarket notification submissions must include detailed documentation of applicable analytical studies, such as, analytical sensitivity (Limit of Detection, Limit of Blank, and Limit of Quantitation), precision, reproducibility, analytical measuring range, interference, cross-reactivity, and specimen stability.
(4) Premarket notification submissions must include detailed documentation of a prospective clinical study or, if appropriate, results from an equivalent sample set. This detailed documentation must include the following information:
(i) Results must demonstrate adequate device performance relative to a well-accepted comparator.
(ii) Clinical sample results must demonstrate consistency of device output throughout the device measuring range likely to be encountered in the Intended Use population.
(iii) Clinical study documentation must include the original study protocol (including predefined statistical analysis plan), study report documenting support for the Indications for Use(s), and results of all statistical analyses.
(5) Premarket notification submissions must include evaluation of the level of the non-microbial analyte in asymptomatic patients with demographic characteristics (
e.g., age, racial, ethnic, and gender distribution) similar to the Intended Use population.(6) As part of the risk management activities performed under 21 CFR 820.30 design controls, you must document an appropriate end user device training program that will be offered as part of your efforts to mitigate the risk of failure to correctly operate the instrument.
(7) A detailed explanation of the interpretation of results and acceptance criteria must be included in the device's 21 CFR 809.10(b)(9) compliant labeling, and a detailed explanation of the interpretation of the limitations of the samples (
e.g., collected on day of diagnosis) must be included in the device's 21 CFR 809.10(b)(10) compliant labeling.