Search Results

The IDS Cortisol assay is an in vitro diagnostic device intended for the quantitative determination of cortisol in human serum and plasma on the IDS system. Results are to be used in conjunction with other clinical and laboratory data to assist clinicians in the diagnosis and treatment of disorders of the adrenal gland.

The IDS Cortisol assay consists of a reagent cartridge. The reagent cartridge contains multiple reagents:

• MPE1:Magnetic particles coated rat anti-mouse monoclonal antibody in a phosphate buffer with Proclin as preservative.
• CONJ: Cortisol coupled with an acridinium ester derivative in = phosphate buffer with Proclin as a preservative.
• mAb: Mouse anti-cortisol monoclonal antibody in phosphate buffer with Proclin as a preservative .;
• BUF: HEPES buffer containing Proclin as preservative .

This document describes the analytical performance of the IDS Cortisol assay, an in vitro diagnostic device, and demonstrates its substantial equivalence to a predicate device (Roche Elecsys Cortisol II). The acceptance criteria and the study proving the device meets these criteria are detailed below.

1. Table of Acceptance Criteria and Reported Device Performance

The acceptance criteria for this in vitro diagnostic device are primarily based on demonstrating analytical performance that is comparable to, or meets specified standards relative to, established laboratory methods and a predicate device.

2. Sample Sizes Used for the Test Set and Data Provenance

• Precision Testing:
  • Per sample/concentration level (for one representative lot on one system): N=80 replicates.
  • Combined (3 lots on 3 systems): N=240 replicates per sample/concentration level.
  • Provenance: Not explicitly stated, but clinical laboratory studies typically use samples from diverse patient populations to cover the measurement range. The study mentions "human serum samples" and "low level samples."
• Linearity Testing:
  • A high human serum sample and a low human serum sample were used, along with 12 evenly spaced dilutions.
  • Provenance: "high patient sample with a low patient sample."
• Detection Limit Testing (LoB, LoD, LoQ):
  • 60 blank replicates and 13 low-level samples.
  • Provenance: Not explicitly stated, but based on typical laboratory practices.
• Analytical Specificity (Interference):
  • Human serum samples with low and high cortisol concentrations.
  • For Rheumatoid Factor, a serum sample with low Cortisol and high Rheumatoid Factor was diluted 1:2 and 1:4. Each was assayed in duplicate.
  • For cross-reactivity, serum samples spiked with cross-reactants and unspiked controls were assayed in 26 replicates each.
• Method Comparison (vs. Predicate Device):
  • N: 194 samples.
  • Provenance: Samples were "selected to represent a wide range of Cortisol concentrations" (0.64 to 44.66 ug/dL), implying patient samples. Country of origin not specified, but usually derived from laboratory settings.
• Matrix Comparison:
  • N: 45 samples (36 native, 9 spiked or diluted) per matrix type.
  • Provenance: "human samples" to cover the range of 0.89 to 42.38 ug/dL.
• Expected Values/Reference Range:
  • N: 307 apparently healthy donors.
  • Provenance: "serum samples collected from 307 apparently healthy donors" from 21 to 65 years of age. Details imply a prospective collection for this specific study, but geographical location not explicitly stated beyond "local population" for general reference ranges.

All studies mentioned state they followed CLSI (Clinical and Laboratory Standards Institute) guidelines, which implies these were prospective analytical performance studies conducted under controlled laboratory conditions.

3. Number of Experts Used to Establish Ground Truth for the Test Set and Qualifications of those Experts

For an in vitro diagnostic (IVD) device like the IDS Cortisol assay, "ground truth" for the test set is established through reference methods or highly accurate comparative assays, not typically by human expert consensus or adjudicated readings. The nature of this device is a quantitative immunoassay meant to measure a biomarker.

• Ground Truth Establishment for Analytical Performance:

  • Traceability: The IDS Cortisol assay is traceable to the LC-MS/MS Candidate Reference Measurement Procedure (cRMP) Total Serum Cortisol. This is a highly accurate chemical analytical method, considered a "higher-order" reference method. The ground truth in this context is the quantitative value determined by this cRMP, often involving specialized laboratories and expert analytical chemists.
  • Method Comparison: The test device's performance is compared against the Roche Elecsys Cortisol II (K152227), a previously cleared and marketed predicate device. The values obtained from the predicate device serve as the comparative "truth" for demonstrating substantial equivalence.
  • Precision, Linearity, Detection Limits, Analytical Specificity, Matrix Comparison: These performance characteristics are evaluated against pre-defined statistical criteria (e.g., CV thresholds, R² values, bias limits) using controlled samples, calibrators, and reference materials. The "truth" is the known concentration or behavior of these materials, verified through standard laboratory practices and reference methods.

• Number of Experts & Qualifications: Not applicable in the context of human expert adjudication for image interpretation. The "experts" would be the skilled laboratory personnel and analytical chemists performing the LC-MS/MS and predicate device measurements, as well as those defining the CLSI protocols for analytical testing.

4. Adjudication Method for the Test Set

Not applicable. This is an in vitro quantitative diagnostic device, not an imaging device requiring human-in-the-loop reading and adjudication.

5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done

Not applicable. This is an in vitro quantitative diagnostic device, not an imaging device requiring human readers or MRMC studies.

6. If a Standalone Performance (i.e., algorithm only without human-in-the-loop performance) Was Done

Yes, the entire analytical performance evaluation (precision, linearity, detection limits, interference, method comparison, and matrix comparison) represents the standalone performance of the IDS Cortisol assay as an algorithm (chemical reaction + instrument measurement and calculation) without human-in-the-loop intervention for result generation. The human role is in operating the instrument and interpreting the results, but the measurement itself is automated.

7. The Type of Ground Truth Used

The primary ground truth used for this quantitative in vitro diagnostic device is:

• Reference Measurement Procedure: The IDS Cortisol assay is traceable to the LC-MS/MS Candidate Reference Measurement Procedure (cRMP) Total Serum Cortisol, which represents a highly accurate and standardized method for determining cortisol concentrations. This is a form of outcomes data in the sense of a definitive analytical outcome.
• Predicate Device Comparison: The values generated by the Roche Elecsys Cortisol II assay serve as a comparative ground truth for demonstrating substantial equivalence.
• Known Concentrations of Reference Materials/Spiked Samples: For analytical performance studies like precision, linearity, and detection limit, the ground truth is often established by using precisely prepared calibrators or samples with known, verified concentrations of the analyte.

8. The Sample Size for the Training Set

This document describes the analytical validation of a commercial diagnostic kit, not a machine learning algorithm that typically undergoes distinct "training" datasets. Therefore, there isn't a "training set" in the sense of data used to iteratively optimize an AI model.

The "development" or "design" of the assay (reagents, methodology) would have involved extensive R&D and internal testing, which could be considered analogous to a training phase in a broader sense, but no specific sample size for such a phase is provided in this regulatory submission. The data presented are for the validation of the final device.

9. How the Ground Truth for the Training Set Was Established

As noted in point 8, there isn't a distinct "training set" in the context of an AI/ML algorithm as described in this document. The "ground truth" for the development of the assay itself would have been established through a combination of:

• Biochemical principles: Understanding the cortisol molecule and its interactions with antibodies.
• Standard laboratory methods: Using established analytical techniques (like spectrophotometry, chromatography, or existing cortisol assays) to characterize reagents and ensure proper reaction kinetics.
• Reference materials: Utilizing internationally recognized reference materials and calibrators for initial assay calibration and optimization.
• Iterative development and testing: Repeated internal testing with various concentrations of cortisol in different matrices to optimize assay components (antibodies, conjugates, buffers) and instrument parameters to achieve desired performance characteristics (sensitivity, specificity, dynamic range). This iterative process involves comparing results against known concentrations or a "gold standard" method (like LC-MS/MS) during development.

Ask a specific question about this device

The IDS-iSYS Ostase® BAP assay is an in vitro diagnostic device intended for the quantitative determination of bone-specific alkaline phosphatase (BAP), an indicator of osteoblastic activity, in human serum on the IDS system. Results are to be used in conjunction with other clinical and laboratory data to aid the clinician in the management of postmenopausal osteoporosis and Paget's disease.

The IDS-iSYS Ostase® BAP assay consists of one reagent cartridge and one set of calibrators (CAL A & CAL B).

The reagent cartridge contains multiple reagents:

• MPM1 (Magnetic particles coated with streptavidin in a phosphate buffer with sodium azide as preservative);
• Ab-BIOT Monoclonal anti-BAP labelled with biotin, in buffer containing horse serum with bovine and mouse proteins and sodium azide as a preservative (<0.1 %)
• -SUBS (p-nitrophenyl phosphate in a stabilising buffer containing preservatives).

Calibrators A and B are buffered bovine protein matrix containing human BAP with sodium azide as preservative (<0.1 %).

The provided document is a 510(k) Premarket Notification from the FDA for a medical device called "IDS-iSYS Ostase® BAP." This document is primarily concerned with demonstrating the substantial equivalence of the new device to a predicate device, rather than focusing on the acceptance criteria and study proving performance for an AI/ML-based medical device.

Therefore, the requested information regarding AI/ML-specific acceptance criteria and validation studies (like sample size for test set, data provenance, number of experts for ground truth, adjudication methods, MRMC studies, standalone performance, training set details) is not present in this document because it describes an in vitro diagnostic (IVD) test for quantitative determination of bone-specific alkaline phosphatase, not an AI/ML-driven imaging or diagnostic algorithm.

However, I can extract the relevant performance characteristics that are analogous to "acceptance criteria" for this type of IVD device and the study data proving it meets those.

Device Description

The IDS-iSYS Ostase® BAP assay is an in vitro diagnostic device intended for the quantitative determination of bone-specific alkaline phosphatase (BAP), an indicator of osteoblastic activity, in human serum on the IDS system. Results are to be used in conjunction with other clinical and laboratory data to aid the clinician in the management of postmenopausal osteoporosis and Paget's disease.

1. Table of Acceptance Criteria (Performance Characteristics) and Reported Device Performance

For an in vitro diagnostic device, "acceptance criteria" are typically defined by various analytical performance characteristics that demonstrate the device's accuracy, precision, and reliability for its intended use. While explicit acceptance ranges are not always presented as a separate "criteria" table in a 510(k), the studies conducted implicitly aim to demonstrate performance within acceptable ranges for IVDs. The comparison to the predicate device and adherence to CLSI guidelines are key for demonstrating substantial equivalence.

Here's a table summarizing the analytical performance characteristics and the reported device performance, which serve as the "proof" that the device meets the implied acceptance criteria for an IVD:

Study Details (Based on the Provided Document)

As this is an IVD device, not an AI/ML system, many of the AI/ML-specific questions are not applicable. However, I will address what is implied or directly stated in the context of an IVD submission.

2. Sample Size Used for the Test Set and Data Provenance:

• Precision/Reproducibility:
  • 10 native serum samples.
  • Each sample assayed 80 replicates (in duplicate, twice per day for 20 days) on one system (representative lot) and 240 replicates (combined from 3 lots, 3 systems).
  • Data provenance not explicitly stated (e.g., country of origin, retrospective/prospective), but implied to be laboratory-based analytical studies.
• Linearity:
  • Not specified how many serum samples (a high and a low human serum sample were used to create 11 dilutions, but the N for testing these dilutions is not given).
• Detection Limits (LoB, LoD, LoQ):
  • LoB: 60 replicates of blank per kit lot (total 180 over 3 lots).
  • LoD & LoQ: 10 low BAP concentration samples tested per kit lot (details on number of replicates not fully clear, mentions tested in duplicate for 5 days per lot).
• Analytical Specificity (Interference/Cross-Reactivity):
  • For interference: Two serum samples at two different BAP concentrations were spiked with potential interferents. For each condition, 26 replicate assays were performed for spiked and control samples.
  • For cross-reactivity: Not explicitly stated, but substances were spiked into serum samples and measured.
• Method Comparison:
  • N = 150 samples.
  • Samples were "selected to represent a wide range of BAP concentrations [3.0 to 67.6 ug/L]".
  • Data provenance not explicitly stated (e.g., country of origin, retrospective/prospective).
• Expected Values/Reference Range:
  • N = 419 apparently healthy donors from the United States.
  • 140 males (35 to 75 years of age)
  • 140 pre-menopausal women (35 to 45 years of age)
  • 139 post-menopausal women (55 to 75 years of age).
  • This is a prospective collection of reference interval data.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts:

• Not applicable in the AI/ML sense. For IVD tests measuring a biomarker like BAP, the "ground truth" is typically the quantitative value obtained from a reference method or the assigned values of traceable calibrator materials. No human expert "adjudication" or "ground truth labeling" of the BAP level in a sample is described as a part of establishing test set ground truth; rather, it’s about the measured BAP concentration.
• The calibrators for the new device are traceable to the predicate device via internal standards that were value-assigned using the predicate assay procedure. This method of traceability establishes the "truth" for calibration.

4. Adjudication Method for the Test Set:

• Not applicable. As above, for an IVD, the measurement itself is the "truth" within the context of the assay's analytical performance. No human adjudication is involved in determining the concentration of BAP in a blood sample for these types of studies.

5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done:

• No. MRMC studies are specific to image-based diagnostic aids where multiple human readers' performance is evaluated and compared with and without AI assistance. This document describes an in vitro diagnostic assay that quantifies a substance in human serum, not an imaging device.

6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) Was Done:

• Yes, in essence. For an IVD, the "standalone" performance is the analytical performance of the assay itself – its precision, linearity, detection limits, specificity, and comparison to a predicate device. The performance characteristics described in Section 1 (Precision, Linearity, Detection Limits, Analytical Specificity, Method Comparison) represent the standalone performance of the IDS-iSYS Ostase® BAP assay. It is designed to provide a quantitative result without direct human "interpretation" of the assay's output in the way an AI would assist a radiologist.

7. The Type of Ground Truth Used:

• Analytical Ground Truth (for analytical performance): For precision, linearity, detection limits, and specificity, the "ground truth" refers to:
  • The known properties of control materials or spiked samples.
  • The measured values from samples used in reproducibility studies.
  • The established values/performance of the predicate device (for comparison studies).
• Reference Intervals (for clinical context): For expected values, the "ground truth" is derived from a defined cohort of apparently healthy individuals, and ranges are statistically determined (e.g., 2.5th to 97.5th percentile).

8. The Sample Size for the Training Set:

• Not applicable (in the AI/ML sense). This device is an IVD assay, not an AI/ML model that requires training on a vast dataset. The "training" for an IVD involves assay development, reagent optimization, and establishing manufacturing controls, not statistical model training based on patient data.
• The calibrators for the device are value-assigned using a method traceable to the predicate device. This process involves testing calibrators in multiple runs (minimum 20 assay runs on one system) to determine their assigned values. This could be considered analogous to "training" the assay system to accurately report concentrations based on known standards.

9. How the Ground Truth for the Training Set Was Established:

• Not applicable (in the AI/ML sense). However, for an IVD, the "ground truth" for calibrator and control materials is established through:
  • Traceability: The IDS-iSYS Ostase® BAP kit calibrators are value assigned against in-house secondary standards (IRs). These IRs are themselves value assigned against the predicate device (Ostase® BAP EIA assay) using the predicate assay procedure. This establishes traceability to a previously cleared, legally marketed device.
  • Verification: The assigned kit calibrator values are then verified by running the assay on three different IDS systems and analyzing internal quality control (IQC) samples of known values across the range of the assay.

Ask a specific question about this device

The IDS SHBG assay is an in vitro diagnostic device intended for the quantitative determination of SHBG in human serum or plasma on the IDS System. Results are to be used as an aid in the diagnosis of androgen disorders

The IDS SHBG assay is an in vitro diagnostic device intended for the quantitative determination of sex hormone binding globulin (SHBG) in human serum and plasma on the IDS-iSYS Multi-Discipline Automated System. Results are to be used in conjunction with other clinical and laboratory data to assist the clinician in the diagnosis of androgen disorders.

The assay is based on chemiluminescence technology. 5 uL of patient sample or calibrators are incubated with biotinylated monoclonal anti-SHBG antibody, an acridinium labelled monoclonal anti-SHBG conjugate and streptavidin labelled magnetic particles. The magnetic particles are captured using a magnet and a wash step performed to remove any unbound analyte. Trigger reagents are added; the resulting light emitted by the acridinium label is directly proportional to the concentration of analyte in the original sample.

The IDS SHBG assay is an in vitro diagnostic device consisting of ready to use reagents provided in individual compartments within the reagent cartridge.

The reagent cartridge contains:

• Magnetic particles magnetic particles coated with streptavidin in a phosphate buffer containing preservatives
• -Biotin antibody - monoclonal anti-SHBG labelled with biotin in a buffer containing proteins and preservatives
• Conjugate monoclonal anti-SHBG labelled with an acridinium ester derivative in a buffer containing proteins and preservatives The calibrators consist of:
• Calibrators A and B are included in the assay kit. The calibrators consist of a human serum matrix with defined concentrations of SHBG and preservatives. Together with a lot specific master calibration curve, the calibrators will be used to perform adjustment of the master calibration curve.

The provided document is a 510(k) summary for the IDS SHBG assay, an in vitro diagnostic device for the quantitative determination of Sex Hormone Binding Globulin (SHBG). The document details the device's performance characteristics and compares it to a predicate device to demonstrate substantial equivalence.

Here's an analysis of the acceptance criteria and the study proving the device meets them, based on the provided text:

Key Takeaway: This document describes the validation of an in vitro diagnostic (IVD) assay, not an AI/ML-based diagnostic device that typically involves human readers or image analysis. Therefore, many of the requested categories (e.g., number of experts, adjudication method, MRMC studies, human reader improvement with AI, standalone AI performance) are not applicable to this type of device and study. The ground truth in this context is established by reference methods or validated reference materials, typical for IVD assays.

1. Table of Acceptance Criteria and Reported Device Performance

The document describes several analytical performance characteristics that serve as "acceptance criteria" for the IDS SHBG assay. These are primarily related to the accuracy, precision, limits of detection, and specificity of the assay.

2. Sample Size Used for the Test Set and Data Provenance

• Precision/Reproducibility: 14 serum-based samples at different SHBG concentration levels covering the assay range. Tested across 21 days for one kit lot. (Table 11 shows individual sample IDs, CTLs, CALBs, IQCs)
• Linearity/Assay Reportable Range:
  • Linearity: A high human serum sample and a low human serum sample, plus 14 evenly spaced dilutions created by mixing high and low samples.
  • Automated Post-dilution Accuracy: 9 native samples with known SHBG concentrations obtained from the predicate device.
• Traceability of Calibrator:
  • Value assignment of Internal Reference Standards (IRs): At least 24 runs using three iSYS instruments, 2 kit lots, 3 replicates for each run. Serial dilutions of WHO 2nd international standard 08/266 used.
  • Value assignment of kit calibrators: At least 20 runs (14 runs using previous IR lot, 6 runs using international standard) using one iSYS instrument, 2 replicates for each run.
  • Value assignment verification: Internal quality controls at 3 SHBG levels tested in five replicates in one run and on each of three different iSYS instruments.
  • Correlation study (2-point calibration vs. IS 08/266 curve): 189 samples (139 serum, 50 K2 EDTA plasmas). Tested in two replicates using three lots (MB1, MB2, MB3) on one iSYS instrument.
• Detection Limit (LoB, LoD, LoQ):
  • LoB: LoB sample run in 12 replicates for each of 5 runs over 3 days, by one operator, on one different instrument for each of 3 manufacture batches (MB1, MB2B, MB3) = total 60 replicates per lot.
  • LoD: 7 LoD samples measured in duplicate. For each of 3 kit lots, 5 assays over 3 days by one operator on a different instrument = total 70 replicates per lot.
  • LoQ: Panel of 9 samples measured in singlicate two times per day. For each of 3 kit lots, 10 assays over 5 days by one operator on a different instrument = total 90 replicates per lot.
• Analytical Specificity (Interference): Two serum samples (low and high SHBG conc.) spiked with potential interferents. Control samples were also run. Number of replicates for each specific test not detailed but implied to be sufficient for statistical comparison.
• Analytical Specificity (Cross-Reactivity): Low and high SHBG samples spiked with various cross-reactants. Number of replicates for each specific test not detailed.
• Method Comparison: 136 samples, selected to represent a wide range of SHBG concentrations (2.54 - 172.12 nmol/L).
• Matrix Comparison: 69 samples (68 native, 1 diluted) covering a range of 0.51 to 238.45 nmol/L.
• Expected Values/Reference Range: 671 serum samples from apparently healthy adults (21-77 years old). Data Provenance: From the United States. Retrospective/Prospective: Not explicitly stated, but typically such studies for establishing reference ranges are retrospective collections of banked samples or a prospective study designed to collect samples from a healthy population.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications

Not Applicable (N/A).

This is an in vitro diagnostic (IVD) assay quantifying a biomarker (SHBG). "Ground truth" for an IVD assay is established through:

• Reference Methods: Such as the WHO 2nd international standard 08/266 for SHBG, against which the calibrators are standardized.
• Known Concentrations: Use of accurately prepared standards, spiked samples, or samples extensively characterized by reference methods.
• Clinical Data: For expected values, SHBG concentrations are measured in samples from defined healthy populations.

Therefore, the concept of "experts establishing ground truth" in the way it applies to image interpretation or AI-assisted diagnostics (e.g., radiologists labeling images) is not relevant here. The ground truth is analytical and based on metrological traceability to international standards.

4. Adjudication Method for the Test Set

N/A.

Adjudication methods (like 2+1, 3+1) are relevant for subjective interpretations (e.g., radiology reads) where discrepancies between readers need to be resolved to establish a definitive ground truth. For a quantitative IVD assay like IDS SHBG, the "reading" is a numerical output from the instrument based on chemical reactions. Accuracy is determined by comparison to reference materials or established methods, not by human adjudication of interpretations.

5. If a Multi Reader Multi Case (MRMC) Comparative Effectiveness Study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance

N/A.

This is an IVD assay, not an AI/ML diagnostic for human interpretation. No MRMC study or human reader improvement with AI assistance is applicable.

6. If a Standalone (i.e. algorithm only without human-in-the-loop performance) was done

N/A.

This is not an AI/ML algorithm. The "device" is a fully automated immunoassay system (IDS-iSYS Multi-Discipline Automated System) that performs the biochemical analysis and reports a quantitative result without human "interpretation" of a signal beyond reading the numerical output. Its performance is evaluated as a standalone analytical system.

7. The Type of Ground Truth Used

The ground truth for the analytical performance studies (precision, linearity, detection limits, specificity, method comparison) is based on:

• International Reference Standards: Specifically, the WHO 2nd international standard for SHBG (IS 08/266) for traceability and calibration. This is the primary reference.
• Known Concentrations/Spiked Samples: Samples with defined, known concentrations of SHBG or interferents, prepared in a laboratory setting.
• Comparison to a Legally Marketed Predicate Device: The Siemens ADVIA Centaur SHBG assay (K151986) was used as a comparative method to assess agreement and accuracy (e.g., for method comparison and automated dilution accuracy).
• Healthy Population Data: For expected values/reference ranges, a large cohort of apparently healthy individuals were tested to establish population-specific normal ranges.

8. The Sample Size for the Training Set

N/A (for AI/ML 'training set' in the traditional sense).

For an IVD assay, the equivalent of a "training set" would be the samples and calibrators used during the assay's development and optimization phases to set parameters, establish reagent formulations, and fine-tune the system. This information is typically proprietary development data and is not explicitly detailed as a 'training set' in 510(k) summaries, which focus on the final validation/test data.

The closest analogous "training" or "calibration" process mentioned is the traceability and value assignment of calibrators:

• Value assignment of secondary standards (IRs) involves at least 24 runs (using 3 instruments, 2 kit lots, 3 replicates) comparing to the WHO international standard.
• Value assignment of IDS SHBG kit calibrators A and B involves at least 20 runs (1 instrument, 2 replicates) using the secondary standards and IS-08/266.
  These processes are used to establish the calibration curve for the assay.

9. How the Ground Truth for the Training Set Was Established

N/A (for AI/ML 'training set').

As explained in point 8, the concept of a "training set" for an IVD assay is different from that for AI/ML. The "ground truth" for establishing the calibration and parameters of the assay is based on:

• Metrological traceability to the WHO 2nd international standard for SHBG (IS 08/266). This involved serial dilutions of the international standard in SHBG-depleted human serum to create reference points.
• Use of internal reference calibrators (IRs) that were themselves value-assigned against the WHO standard.

The goal is that the assay's measurements accurately reflect the true concentration of SHBG as defined by an international reference.

Ask a specific question about this device

IDS-iSYS 17-OH Progesterone Control Set
The IDS-iSYS 17-OH Progesterone Control Set is for in vitro diagnostic use, for the quality control of the IDS-iSYS 17-OH Progesterone on the IDS-iSYS Multi-Discipline Automated System.
Rx Only.

IDS-iSYS 17-OH Progesterone Calibration Verifiers
The IDS-iSYS 17-OH Progesterone Calibration Verifiers are an in vitro diagnostic device intended for medical purposes in the quantitative verification of assay calibration and measuring range of the IDS-iSYS 17-OH Progesterone assay, when performed on the IDS-iSYS Multi Discipline Automated System.
Rx Only.

The IDS-iSYS 17-OH Progesterone Control Set consists of Two sets of three vials, 1.0 mL each in liquid form. Human serum containing 17-OH Progesterone and sodium azide as a preservative (<0.1%), with three concentration levels.
The IDS-iSYS 17-OH Progesterone Calibration Verifiers consists of Four calibration verifier levels in liquid form (two 1.0mL vials for level 0 and one 1.0mL vial for levels 1, 2 & 3). Human serum containing 17-OH Progesterone and sodium azide as a preservative (<0.1%), with three concentration levels.

The provided text describes the acceptance criteria and supporting studies for two in vitro diagnostic devices: the IDS-iSYS 17-OH Progesterone Control Set and the IDS-iSYS 17-OH Progesterone Calibration Verifiers.

1. Table of acceptance criteria and the reported device performance:

2. Sample size used for the test set and the data provenance:

• IDS-iSYS 17-OH Progesterone Control Set - Value Assignment:
  • Sample Size: A minimum of 21 runs using cartridge batches (number of batches not specified) tested in triplicate on each of three IDS-iSYS Multi Discipline Automated Systems. Control solutions were prepared gravimetrically.
  • Data Provenance: Not explicitly stated as retrospective or prospective, nor country of origin. Given it's a device manufacturer's study to establish product characteristics, it's inherently prospective in nature.
• IDS-iSYS 17-OH Progesterone Control Set - Stability (All types):
  • Closed Vial (Real-time): Three lots of controls, tested in pentaplicate (5 replicates) at 2-month intervals for up to a minimum of 15 months. Each tested vial compared to reference material stored at -20°C.
  • Closed Vial (Accelerated): Performed according to CLSI guideline EP25-A.
  • Open Vial: Tested in duplicate at time points stated in the stability protocol (time points not detailed), against unopened vials.
  • On-Board: Three batches of controls, using three IDS-iSYS instruments. Tested at 0, 2, 4, 6, and 8 hours compared to a reference material at time 0.
  • Data Provenance: Same as above, likely prospective internal testing.
• IDS-iSYS 17-OH Progesterone Calibration Verifiers - Value Assignment:
  • Sample Size: Minimum of five runs for each cartridge batch (number of batches not specified) tested in triplicate on each of three IDS-iSYS Multi Discipline Automated Systems. Calibration Verifier solutions were prepared gravimetrically.
  • Data Provenance: Same as above, likely prospective internal testing.
• IDS-iSYS 17-OH Progesterone Calibration Verifiers - Stability (Closed Vial & On-Board):
  • Closed Vial (Real-time): Three lots of calibration verifiers, tested in pentaplicate (5 replicates) at 2-month intervals for up to a minimum of 15 months.
  • Closed Vial (Accelerated): Performed according to CLSI guideline EP25-A.
  • On-Board: Three batches of calibration verifiers, using three IDS-iSYS instruments. Tested at 0, 2, 4, 6, and 8 hours compared to a reference material at time 0.
  • Data Provenance: Same as above, likely prospective internal testing.

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts (e.g., radiologist with 10 years of experience):

This device is an in vitro diagnostic (IVD) quality control material and calibration verifier for automated systems, not an imaging AI device. The "ground truth" here refers to the reference values established for the control and calibration materials themselves, rather than diagnosis or interpretation by human experts.

• Value Assignment: The "ground truth" (assigned target values) for both the control set and calibration verifiers is established by the mean of all runs from multiple tests on three IDS-iSYS Multi Discipline Automated Systems, using cartridge batches, and confirmed by immunologic analysis using the IDS-iSYS 17-OH Progesterone assay. The solutions themselves are prepared gravimetrically from intermediate stock solutions.
• There are no "experts" in the traditional sense (e.g., radiologists) involved in establishing the ground truth for this type of chemical/immunological assay. The ground truth relies on the precision and accuracy of the analytical methods and the instrument itself.

4. Adjudication method (e.g., 2+1, 3+1, none) for the test set:

Not applicable. This is not a study involving human interpretation or subjective assessment of data that would require an adjudication method. The device's performance is determined by quantitative measurements and statistical analysis against predefined criteria.

5. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:

Not applicable. This device is an IVD quality control and calibration material, not an AI-powered diagnostic tool for human readers. No MRMC study was conducted or is relevant for this product.

6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done:

Yes, the studies are inherently standalone as they evaluate the performance of the control and calibration materials on the IDS-iSYS Multi-Discipline Automated System. The system/assay performs the measurements without human intervention in the interpretation of the control/verifier results themselves beyond setting up the experiment and analyzing statistical compliance. The performance reported (e.g., mean concentration, CV, percent recovery) is that of the material as measured by the automated system.

7. The type of ground truth used (expert consensus, pathology, outcomes data, etc.):

The ground truth used is based on:

• Gravimetric preparation: For the initial stock solutions and preparation of control/calibration verifier samples.
• Immunologic analysis: Confirmation of concentrations using the IDS-iSYS 17-OH Progesterone assay itself.
• Statistical averaging: The mean of multiple runs on multiple instruments to establish the assigned target values, treated as the reference ground truth for quality control and calibration verification.
• Reference material comparison: For stability studies, comparison to a reference control material stored under optimal conditions (-20°C).

This is a form of analytical ground truth derived from established quantitative laboratory methods and statistical analysis rather than clinical consensus or pathological diagnosis.

8. The sample size for the training set:

Not applicable. This is not an AI/machine learning device that requires a training set. The values and stability characteristics are determined through direct analytical testing, not model training.

9. How the ground truth for the training set was established:

Not applicable. As there is no training set for an AI/machine learning model, no ground truth needed to be established for it.

Ask a specific question about this device

The 25-Hydroxy Vitamin DS EIA assay is intended for the quantitative determination of 25-hydroxyvitamin D [25(OH)D] and other hydroxylated metabolites in human serum or plasma. Results are to be used in conjunction with other clinical and laboratory data to assist the clinician in the assessment of vitamin D sufficiency in an adult population.

The 25-Hydroxy Vitamin De EIA is a manual assay test and does not require the use of an automated system. The whole assay is performed in a microtitre plate and requires a user to perform each step. The 25-Hydroxy Vitamin De assay is an enzymeimmunoassay for the quantitation of 25(OH)D and other hydroxylated metabolites in serum or plasma. 25uL of each calibrators, controls and samples are diluted with biotin labelled 25(OH)D. The diluted samples are incubated in microtitre wells which are coated with a highly specific sheep 25(OH)D at room temperature before aspiration and washing. Enzyme (horseradish peroxidase) labelled avidin, is added and binds selectively to complexed biotin and, following a further wash step, colour is developed using a chromogenic substrate (TMB). The absorbance of the stopped reaction mixtures are read in a microtitre plate reader, colour intensity developed being inversely proportional to the concentration of 25(OH)D.

The provided document describes the performance characteristics of the 25-Hydroxy Vitamin D EIA assay, but it focuses on analytical and comparative performance rather than a study proving the device meets general acceptance criteria in a clinical setting with human readers. The document is for an in-vitro diagnostic (IVD) device, so the criteria and studies described are relevant to laboratory performance.

Here's an analysis of the acceptance criteria and supporting studies based on the provided text, adapted for an IVD context where appropriate:

1. A table of acceptance criteria and the reported device performance

For IVD devices, "acceptance criteria" are usually internal performance goals set by the manufacturer to demonstrate analytical and clinical performance required for regulatory clearance. The document lists several performance characteristics and provides the corresponding measurements for the candidate device and, in some cases, the predicate device.

2. Sample sized used for the test set and the data provenance (e.g. country of origin of the data, retrospective or prospective)

• Precision/Reproducibility: 10 serum samples, assayed using 3 lots of reagents in duplicate, twice per day for a minimum of 20 days (n ≥ 80 replicates per sample). The origin of these samples is not specified.
• Linearity/Assay Reportable Range: Samples containing varying concentrations of 25-hydroxyvitamin D were assayed in replicate of four. The number of unique samples for this study is not explicitly stated, but they were patient samples (high patient sample diluted with a low patient sample). Origin not specified.
• Analytical Specificity (Cross-reactivity): Vitamin D serum samples were spiked with various cross-reactants. Number of samples not specified. Not specified if retrospective or prospective. Origin not specified.
• Analytical Specificity (Interference): Substances spiked into the assay. Number of samples/experiments not specified. Not specified if retrospective or prospective. Origin not specified.
• Method Comparison (vs. Predicate Device):
  • Sample Size: n = 195 (patient samples).
  • Provenance: Not specified (country/retrospective/prospective).
• Method Comparison (vs. Reference Measurement Procedure - RMP):
  • Sample Size: n = 109 independent native serum samples.
  • Provenance: Samples were value assigned by the Ghent reference method procedure. Not specified if retrospective or prospective, or country of origin for the samples themselves.
• Matrix Comparison:
  • Sample Sizes: SST (n=28), EDTA plasma (n=38), Sodium Heparin plasma (n=28), Lithium Heparin plasma (n=28), Citrate plasma (n=28).
  • Provenance: Not specified (country/retrospective/prospective).
• Expected Values/Reference Range:
  • Sample Size: 280 apparently healthy adult individuals.
  • Provenance: Retrospective. Individuals were "living in geographical diverse regions of the United States to represent a broad spectrum of UV light exposure".

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts (e.g. radiologist with 10 years of experience)

For this IVD device (25-Hydroxy Vitamin D EIA assay), the "ground truth" is established by reference methods or the true concentration of the analyte, not typically by expert interpretation from medical images or clinical judgment directly.

• Method Comparison (vs. Reference Measurement Procedure): The "ground truth" for this comparison was established by the Ghent reference method procedure for 25-Hydroxy Vitamin D, and the assay is traceable to the isotope dilution-liquid chromatography/tandem mass spectrometry (ID-LCMS/MS) 25(OH) Vitamin D reference method procedure (RMP) which was used in assigning the target value for the Vitamin D Standardization Program (VDSP single donor human serum panel, itself traceable to NIST SRM 2972). This indicates a highly standardized and validated analytical method, which serves as the "expert" or definitive reference for concentration. The "experts" are the developers and maintainers of these reference methods, typically highly qualified analytical chemists or metrologists, not clinical experts for interpreting results in a diagnostic imaging sense.
• Other studies: For precision, linearity, and specificity, the "ground truth" is inherent to the prepared samples or known concentrations, not established by human experts.

4. Adjudication method (e.g. 2+1, 3+1, none) for the test set

Adjudication methods like "2+1" or "3+1" are typically used in studies involving human interpretation (e.g., radiology reads) where disagreements need to be resolved. This is not applicable to the analytical performance studies of an IVD assay where quantitative results are compared to a reference method or known values. Therefore, none was used in the sense of clinical expert adjudication.

5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance

No MRMC study was done, as this device is an in-vitro diagnostic assay for quantitative laboratory measurement, not an AI-assisted diagnostic imaging tool that involves human readers interpreting images.

6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done

This device is a standalone assay. It provides a numerical result for 25-Hydroxy Vitamin D concentration. The performance studies detailed (precision, linearity, LoD, LoQ, method comparisons, interference, cross-reactivity) are all evaluations of this standalone analytical performance, quantifying its accuracy and reliability in measuring the analyte in patient samples. There is no "human-in-the-loop" once the sample is processed by the assay; a user performs the assay steps, but the result generated is direct from the assay's chemical reactions and spectrophotometric reading.

7. The type of ground truth used (expert consensus, pathology, outcomes data, etc.)

The ground truth used primarily consists of:

• Reference Measurement Procedures (RMP): Specifically, the ID-LCMS/MS method (traceable to NIST SRM 2972) and the Ghent reference method procedure. These are highly accurate and standardized analytical methods for determining the true concentration of 25-Hydroxy Vitamin D.
• Known Concentrations: For studies like linearity, LoB, LoD, LoQ, interference, and cross-reactivity, ground truth is established by preparing samples with known, precise concentrations of the analyte or interfering substances.

8. The sample size for the training set

This document describes a premarket notification (510(k)) for a medical device (an in-vitro diagnostic assay). The concept of a "training set" is typically associated with machine learning or AI models. For an IVD assay, raw materials and assay design are developed through R&D, and then validated with studies like those described. There isn't a "training set" in the sense of data used to train an algorithm. The development and optimization of the assay's components (e.g., antibodies) would occur during the manufacturing process, but the document doesn't provide details on sample sizes used during that phase.

9. How the ground truth for the training set was established

As there isn't a "training set" in the AI/ML context for this IVD assay, this question is not directly applicable. The "ground truth" for the analytical performance studies (as described in point 7) was established using reference measurement procedures and known concentrations.

Ask a specific question about this device

The IDS-iSYS Aldosterone assay (IS-3300) is a device intended for use in clinical laboratories for the quantitative determination of Aldosterone in human EDTA plasma on the IDS-iSYS Multi-Discipline Automated System. Aldosterone measurements are used in the diagnosis and treatment of primary aldosteronism (a disorder caused by excessive secretion of Aldosterone by the adrenal gland), hypertension caused by primary aldosteronism, selective hyperaldosteronism, edematous states and other conditions of electrolyte balance.

The IDS-iSYS Aldosterone Control Set (IS-3330) is intended for use as assaved quality control samples to monitor the accuracy of the IDS-iSYS Aldosterone assay on the IDS-iSYS Multi-Discipline Automated System.

The IDS-iSYS Aldosterone Calibration Verifiers (IS-3335) are intended for medical purposes for use in the quantitative verification of calibration of the IDS-iSYS Aldosterone assay on the IDS-iSYS Multi-Discipline Automated System.

The IDS-iSYS Aldosterone assay is based on chemiluminescence technology. A biotinylated monoclonal anti-Aldosterone antibody is incubated with the sample, after an incubation step an Aldosterone acridinium conjugate is added and after a further incubation step streptavidin coated magnetic particles are added. Following a third incubation step the particles are "captured" using a magnet. After a washing step and addition of trigger reagents, the light emitted by the acridinium label is inversely proportional to the concentration of Aldosterone in the original sample.

Here's an analysis of the provided text to extract the requested information about acceptance criteria and the supporting study:

The provided document is a 510(k) Summary for the IDS-iSYS Aldosterone assay, control set, and calibration verifiers. It focuses on demonstrating substantial equivalence to a predicate device, as required for FDA clearance. The document details performance characteristics but does not explicitly state acceptance criteria in a pass/fail format typical of formal acceptance criteria documents. Instead, it reports performance values found during validation studies.

Here's the breakdown of the information you requested, based on what's available in the document:

1. Table of Acceptance Criteria and Reported Device Performance

As mentioned, explicit "acceptance criteria" are not listed in a single table. However, the performance characteristics are reported, and implicitly, these values are considered acceptable for demonstrating substantial equivalence. The table below compiles the reported performance data from the document.

Note: The document does not provide a column for "Acceptance Criteria" as a separate, pre-defined target. The "Reported Device Performance" is the outcome of the studies aiming to demonstrate acceptable performance.

2. Sample Size Used for the Test Set and Data Provenance

• Precision/Reproducibility:

  • Sample Size: Six EDTA plasma samples (9 "samples" were run over 20 days, generating 80 replicates per sample concentration).
  • Data Provenance: Not explicitly stated, but clinical laboratory setting is implied. The phrase "two sites using three analyzers" could suggest multi-center testing, potentially from different locations/countries, but this is not specified. The studies were performed for the manufacturer for regulatory submission.

• Linearity/Assay Reportable Range:

  • Sample Size: One high plasma sample diluted with a low sample to create 11 dilution samples. Run in quadruplicate.
  • Data Provenance: Not specified, likely internal laboratory data for the manufacturer.

• Detection Limit (LoB, LoD, LoQ):

  • Sample Size:
    • LoB (Lot 1): One zero-aldosterone plasma sample, 10 replicates for 5 days (total 50 replicates).
    • LoD/LoQ (Lot 1): 7 samples, 2 replicates per sample, once per day for 8 days.
    • LoB (Lot 2): One zero-aldosterone plasma sample, 6 replicates for 5 days (total 30 replicates).
    • LoD (Lot 2): 8 samples, 2 replicates per sample, once per day for 5 days.
    • LoQ (Lot 2): 7 samples, 2 replicates per sample, once per day for 5 days.
  • Data Provenance: Not specified, likely internal laboratory data for the manufacturer. Two different sites and two different analyzers were used.

• Analytical Specificity (Interference and Cross-Reactivity):

  • Sample Size:
    • Interference: Two base plasma samples ("Low" and "High" Aldosterone concentrations), spiked with various potential interferents. For each interferent, 26 replicates for blank and spiked samples were compared.
    • Cross-reactivity: Stock solutions of various compounds diluted serially to create 7-point standard curves for each substance.
  • Data Provenance: Not specified, likely internal laboratory data for the manufacturer.

• Method Comparison:

  • Sample Size: 161 samples (including 12 altered samples).
  • Data Provenance: Not specified, implied to be clinical or patient samples. Whether these were retrospective or prospective, or their country of origin, is not mentioned.

• Reference Range Study (Expected Values):

  • Sample Size: 228 Caucasian adult samples.
  • Data Provenance: Collected in the US, prospective (samples collected under specific conditions: 7-10 am after overnight fasting, upright/supine positions).

3. Number of Experts Used to Establish the Ground Truth for the Test Set and the Qualifications of Those Experts

This type of information is generally NOT applicable to in vitro diagnostic (IVD) devices like the Aldosterone assay described. For IVD assays, "ground truth" is typically established by:

• Reference methods/predicate devices (e.g., Diasorin Liaison Aldosterone assay for method comparison).
• Gravimetric preparation of standards and calibrators for traceability.
• Defined sample characteristics (e.g., "zero aldosterone plasma," "high plasma sample").

There were no human experts assessing images or making diagnoses that would require adjudication.

4. Adjudication Method for the Test Set

Not applicable, as this is an IVD assay, not a device requiring human interpretation of results in a diagnostic context that would call for adjudication of different interpretations.

5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance

Not applicable. This is an IVD assay for quantitative determination of a biomarker; there are no "human readers" interpreting results in the way an MRMC study would be designed for an imaging AI device.

6. If a Standalone (i.e. algorithm only without human-in-the-loop performance) was done

Yes, the studies conducted (Precision, Linearity, LoD/LoQ, Analytical Specificity, Method Comparison) represent the standalone performance of the IDS-iSYS Aldosterone assay system. The device quantifies aldosterone in plasma samples directly, without human interpretation of results being a variable in its core analytical performance.

7. The Type of Ground Truth Used

The ground truth used for various studies includes:

• Reference materials: Aldosterone (≥95% HPLC; A9477Sigma-Aldrich) dissolved in Dioxane, with concentration calculated by UV quantitation using molar extinction coefficient (for calibrators and traceability).
• Predicate device results: The Diasorin Liaison Aldosterone assay (K130321) for method comparison.
• Spiked samples: For linearity, interference, and cross-reactivity studies, known concentrations of analyte or interferents were added to base samples to create "true" concentrations.
• Patient samples: For method comparison and reference range studies, patient samples were used. Their "truth" for method comparison was established by the predicate device. For reference ranges, the "truth" was the measured value distributed among a healthy population.

8. The Sample Size for the Training Set

The document describes performance studies (validation) but does not explicitly mention a "training set" in the context of machine learning. For an IVD assay, the development process involves reagent optimization, calibration curve fitting, etc., which conceptually use data, but this is not typically termed a "training set" like in AI/ML.

However, the "Master curve" and "two-point calibration" system is mentioned. The "logistic parameters of the Master calibration curve" are generated using "data of 20 runs Internal Reference Calibrators" (p.9). This could be considered analogous to a training process for establishing the core measurement algorithm, but it's not a training set for an AI inference model.

9. How the Ground Truth for the Training Set Was Established

If we consider the generation of the "Master calibration curve" as a "training" process:

• Ground Truth: The "Internal Reference Calibrators" referenced (p.9) would serve as the ground truth. These calibrators are traceable to gravimetrically prepared Aldosterone standards using high-purity Aldosterone and UV quantitation (p.9).
• Establishment Method: The "new kit calibrator sets are run as 'unknowns' in duplicate in at least 20 assays on one analyser" (p.9). "The data of 20 runs Internal Reference Calibrators are used to generate the logistic parameters of the Master calibration curve by using Prism software package" (p.9).

Ask a specific question about this device

The IDS-iSYS CTX-1 (CrossLaps ®) Calibration Verifiers is a device intended for the verification of the IDS-iSYS CTX-I (CrossLaps ®) Assay when performed on the IDS-iSYS Multi-Discipline Automated Analyzer

The IDS-iSYS CTX-I (CrossLaps®) Calibration Verifiers consists of one set of four vials, 2.5 mL each in liquid form, containing horse serum with <0.1% (w/w) sodium azide as a preservative, with four concentration levels of human CTX-I:
Cal. Ver. 0: Undetectable
Cal. Ver. 1: 0.12 - 0.16 ng/mL
Cal. Ver. 2: 2.4 - 3.2 ng/mL
Cal. Ver. 3: 5.6 - 6.6 ng/mL

The document is a 510(k) Summary for the IDS-iSYS CTX-I (CrossLaps®) Calibration Verifiers, a device intended for the verification of calibration of the IDS-iSYS CTX-I (CrossLaps®) Assay when performed on the IDS-iSYS Multi-Discipline Automated Analyzer.

Here's an analysis of the provided information to answer your request:

1. A table of acceptance criteria and the reported device performance

The document does not explicitly state "acceptance criteria" for the device's performance in a table format. Instead, it defines the "Target Range" for each Calibration Verifier level, derived from the mean ± 2 standard deviations during value assignment. This target range implicitly serves as the acceptance criteria for individual measurements when verifying calibration.

Note: The "Reported Device Performance" here is the target mean derived during the value assignment process, which aligns with the center of the acceptance range. The study ensures that the device can consistently achieve these target means and fall within the defined ranges.

2. Sample size used for the test set and the data provenance (e.g. country of origin of the data, retrospective or prospective)

The document describes the "value assignment" process, which functions as the primary method for establishing the performance characteristics and implicitly, the "test set" for the verifiers.

• Sample Size: "Each lot-specific value assignment was tested in five runs on at least three different IDS-iSYS analyzers in triplicate, for a total of 45 replicates." This refers to 45 individual measurements for each calibrator verifier (Cal. Ver. 0, 1, 2, 3).
• Data Provenance: The document does not specify the country of origin of the data or whether the study was retrospective or prospective. Given that Immunodiagnostic Systems Ltd is based in the United Kingdom, it's likely the studies were conducted there. The value assignment process described suggests a prospective study design for establishing lot-specific values.

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts (e.g. radiologist with 10 years of experience)

This is not applicable to this type of device. The device is a "Quality control material (assayed and unassayed)" for an in vitro diagnostic assay. Its "ground truth" (target values) is established through rigorous analytical testing and standardization against in-house reference standards, not by human expert interpretation of signals or images.

4. Adjudication method (e.g. 2+1, 3+1, none) for the test set

Not applicable. As explained in point 3, the ground truth is established through analytical methods, not through expert adjudication.

5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance

Not applicable. This is not a study involving human readers or AI assistance. The device is a calibration verifier for an automated immunoassay.

6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done

This refers to the performance of the IDS-iSYS CTX-I assay on the IDS-iSYS Multi-Discipline Automated Analyzer when calibrated and verified using these verifiers. The "value assignment" study described is a standalone performance assessment of the verifiers themselves, demonstrating their ability to consistently produce expected concentration values within a statistically defined range. The objective of the verifiers is to ensure the assay's standalone performance is accurate.

7. The type of ground truth used (expert consensus, pathology, outcomes data, etc)

The "ground truth" for the calibrator verifiers is their assigned concentration values, which are established through:

• Standardization against in-house reference standards (CTX-I in horse serum).
• Derivation from the mean of multiple replicates across multiple runs and instruments. The target ranges (e.g., 0.12 - 0.16 ng/mL for Cal. Ver. 1) are determined statistically (mean ± 2SD).

8. The sample size for the training set

This concept is not directly applicable to a calibration verifier in the same way it would be for an AI algorithm. The "training" here refers to the process of standardizing the manufacturing of the verifiers and establishing their lot-specific values. The "value assignment" study (45 replicates per verifier level) serves to establish these "ground truth" values and ranges.

9. How the ground truth for the training set was established

For each lot, the "ground truth" (target mean values and ranges) was established through:

• Extensive testing: "Each lot-specific value assignment was tested in five runs on at least three different IDS-iSYS analyzers in triplicate," generating a substantial dataset for statistical analysis.
• Statistical analysis: The "assigned target value of each calibrator verifier was defined as the mean of all the runs for each calibrator verifier. The guideline target range is defined as the mean of all runs ± 2SD."
• Standardization: The IDS-iSYS CTX-I assay (for which these verifiers are used) is itself "standardized against in-house reference standards (CTX-I in horse serum)." This implies a chain of traceability to primary or secondary reference materials.

Ask a specific question about this device

The IDS iSYS 25-Hydroxy Vitamin DS Assay (IDS-iSYS 250HD*) is intended for the quantitative determination of 25-hydroxyvitamin D (25OHD) and other hydroxylated metabolites in human serum on the IDS iSYS Multi-Discipline Automated System. Results are to be used in conjunction with other clinical and laboratory data to assist the clinician in the assessment of vitamin D sufficiency in an adult population.

The IDS-iSYS 25-Hydroxy Vitamin DS (250HDS) Control Set is used for quality control of the IDSiSYS 25-Hydroxy Vitamin DS assay on the IDS-iSYS Multi-Discipline Automated System.

The IDS-iSYS 25-Hydroxy Vitamin DS assay consists of a reagent cartridge and one set of calibrators (Calibrators A & B or CAL A & CAL B). The reagent cartridge contains multiple reagents: MPV1 (Magnetic particles coated with 25-OH D in a phosphate buffer containing methanol with sodium azide as preservative), CONJ (Anti- 25-OH D sheep polyclonal antibody labelled with an acridinium ester derivative, in buffer containing bovine, sheep, rabbit and mouse proteins with sodium azide as preservative), NaOH (Sodium hydroxide solution <0.5 M), and BUF (Assay buffer containing proprietary displacing compounds, methanol, and sodium azide as preservative). Calibrators A and B contain horse serum in a buffer matrix with two defined concentrations of 25-OH D and sodium azide as a preservative. The IDS-iSYS 25-Hydroxy Vitamin DS Control Set contains horse serum in a buffer matrix with three defined concentrations of 25-OH D and sodium azide as a preservative.

This is an analysis of a 510(k) summary for the IDS-iSYS 25-Hydroxy Vitamin Dˢ Assay and IDS-iSYS 25-Hydroxy Vitamin Dˢ Control Set.

1. Table of Acceptance Criteria and Reported Device Performance

The acceptance criteria are generally established by internal specifications of the manufacturer, often guided by CLSI (Clinical and Laboratory Standards Institute) guidelines or regulatory precedents. The document presents the reported performance without explicitly stating pre-defined acceptance criteria for each measurement, but rather implies acceptability by presenting results typically expected for such assays. For some sections, such as precision/reproducibility and interference, the acceptable range is mentioned within the text (e.g., ±10% bias for interference).

2. Sample Sizes Used for the Test Set and Data Provenance

• Precision/Reproducibility: 6 serum samples, 80 measurements each (total = 480 measurements across 3 lots, 2 internal replicates, 2 external replicates, 20 days, 3 analyzers). Data provenance: Not explicitly stated, but typically internal studies.
• Linearity/Assay Reportable Range: 11 concentration levels from a high and low serum sample dilution, each assayed in duplicate from one manufacturing batch (total 22 concentration levels). Data provenance: Not explicitly stated, typically internal studies.
• Detection Limit:
  • LoB: Zero calibrator assayed as 10 replicates over 10 assays on 10 separate days (total of 100 measurements).
  • LoD: 10 samples (native and/or diluted) with very low vitamin D concentrations, in duplicate over 12 assays spanning multiple days (total of 240 data points).
  • LoQ: 13 low samples (native and/or diluted) in duplicate over 7 individual assays spanning multiple days (total of 182 data points).
    Data provenance for all detection limits: Not explicitly stated, typically internal studies.
• Analytical Specificity (Cross-reactivity): Not explicitly stated numbers of samples per cross-reactant, but uses vitamin D serum samples, some spiked. Data provenance: Not explicitly stated, typically internal studies.
• Interference: Not explicitly stated numbers of samples per interfering agent, but uses test vs. control samples, and in one case, 26 measurements for each condition. Data provenance: Not explicitly stated, typically internal studies.
• Method Comparison with Predicate Device (vs. ID-LC-MS/MS RMP): 99 samples. Data provenance: Not explicitly stated, but implies connection to CDC VDSP program for traceability.
• Method Comparison with Internal Predicate Device: 283 European-sourced serum samples. Data provenance: European, retrospective.
• Expected Values/Reference Range: 275 apparently healthy adults (light skin and dark skin, male and female, aged 21-77 years) from geographically diverse regions of the United States. Data provenance: Prospective, United States.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications

For this in vitro diagnostic device (IVD), the "ground truth" is typically established by reference methods or accepted techniques for measuring the analyte. No human expert "adjudication" in the sense of image review is performed.

• For Traceability/Standardization: The device is traceable to the isotope dilution-liquid chromatography/tandem mass spectrometry (ID-LC-MS/MS) 25(OH)D Reference Method Procedure (RMP) which was used in assigning the target value for the VDSP samples. The ID-LC-MS/MS RMP is traceable to the National Institute of Standards and Technology Standard Reference Material (SRM) 2972. This is considered the reference standard, not "expert concensus" in this context.
• For Method Comparison with Predicate Device (vs. ID-LC-MS/MS RMP): The ID-LC-MS/MS RMP serves as the reference method and thus the "ground truth" in this comparison.

4. Adjudication Method for the Test Set

Not applicable. This is an IVD device, and the ground truth is established by chemical reference methods and measurements, not by human expert adjudication of interpretations.

5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done

No, a multi-reader multi-case (MRMC) comparative effectiveness study was not done. This device is an in vitro diagnostic assay that directly quantifies an analyte in serum, not an imaging device requiring human interpretation, which is where MRMC studies are typically employed. Therefore, the concept of "human readers improve with AI vs without AI assistance" does not apply here.

6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) Was Done

Yes, the studies presented (precision, linearity, detection limits, specificity, method comparisons) evaluate the standalone performance of the IDS-iSYS 25-Hydroxy Vitamin Dˢ Assay. This device is an automated system designed to provide quantitative results directly from serum samples without direct human interpretation of the assay's output for diagnosis, beyond basic quality control and clinical interpretation of the quantitative value.

7. The Type of Ground Truth Used

The ground truth for the analytical performance studies (e.g., linearity, detection limit, cross-reactivity, interference) is established by the design of the experiments, using known concentrations of analytes, spiked samples, or reference materials.

For the method comparison studies, the ground truth is:

• ID-LC-MS/MS RMP (Reference Method Procedure): This is considered the gold standard or highly accurate reference method for 25(OH)D measurement, traceable to NIST SRM 2972.
• The currently marketed IDS-iSYS 25-Hydroxy Vitamin D (K091849) predicate device: This device's results are used as a comparative "truth" to demonstrate substantial equivalence, although the new device is also aligned with the ID-LC-MS/MS RMP.

8. The Sample Size for the Training Set

The document does not explicitly delineate a separate "training set" in the context of machine learning. For IVD devices like this, method development and optimization phases involve extensive testing with various samples, which could be considered analogous to a training process, but it's not typically formally described as such.

• Calibrator Traceability and Value Assignment: Master calibrators and kit calibrators are developed using stock solutions, and their values are adjusted based on results from running calibrators in multiple assays (minimum of 20 assay runs for kit calibrators) and multiple analyzers. Final values are adjusted for batch-to-batch consistency and alignment to the CDC VDSP program based on multiple correlation assays of an "established patient library panel". The size of this patient library panel is not specified.

9. How the Ground Truth for the Training Set Was Established

As noted above, a formal "training set" with ground truth in the machine learning sense is not explicitly described. However, the development and calibration of the assay involve:

• Traceability to U.V. quantification and then ID-LC-MS/MS RMP:
  • Internal stock solutions are prepared, and their potency is initially based on absorbance at 264nm of an ethanolic 25D solution.
  • Master calibrators are prepared, and their final value assignment is based on extensive testing against the assay itself (running them in multiple assays on multiple analyzers).
  • The kit calibrators are tested as unknowns in a minimum of 20 assay runs calibrated with master calibrators, and values are adjusted to ensure consistency and alignment to the CDC VDSP program based on multiple correlation assays of an established patient library panel. This patient library panel's "ground truth" would have been established by the ID-LC-MS/MS RMP.

In summary, for this IVD, the ground truth primarily relies on established analytical standards and reference methods (ID-LC-MS/MS, NIST SRMs), rather than expert consensus or pathology reports. The studies demonstrate the assay's analytical performance and its comparability to both the ID-LC-MS/MS RMP and a predicate device.

Ask a specific question about this device

The IDS-iSYS Direct Renin assay is intended for the quantitative determination of Direct Renin in human EDTA plasma on the IDS iSYS Multi-Discipline Automated System. Renin measurements may aid in the diagnosis and treatment of certain types of hypertension.

The IDS-iSYS Direct Renin Control Set is used for quality control of the IDS-iSYS Direct Renin Assay on the IDS-iSYS Multi-Discipline Automated System.

The IDS-iSYS Direct Renin Calibration Verifier is a device intended for medical purposes for use in the quantitative verification of calibration of the IDS-iSYS Direct Renin Assay when performed on the IDSiSYS Multi-Discipline Automated System.

Not Found

This document is a 510(k) premarket notification for an in vitro diagnostic device, specifically assays and controls for measuring Direct Renin. Such documents review the substantial equivalence of a new device to a predicate device, and the studies cited typically focus on analytical performance rather than clinical efficacy studies often associated with AI/ML devices. Therefore, a direct mapping to the requested information about AI model performance, expert ground truth adjudication, MRMC studies, or training set details is not possible based on the provided text.

However, I can extract information related to the acceptance criteria and performance as presented for this type of diagnostic assay.

Here's the breakdown based on the provided text:

1. A table of acceptance criteria and the reported device performance

The provided text (K123763) is a 510(k) clearance letter for the IDS iSYS Direct Renin Assay. It does not contain a table of acceptance criteria nor detailed performance data. This letter confirms that the FDA has reviewed the device and determined it to be "substantially equivalent" to legally marketed predicate devices. The full performance data and acceptance criteria would be in the original 510(k) submission, which is not provided here.

However, for in vitro diagnostic assays like this, typical acceptance criteria relate to analytical performance characteristics. While not explicitly stated with values, the Regulation Number: 21 CFR 862.1085 pertains to "Angiotensin I and renin test system." For such systems, acceptance criteria generally cover:

• Accuracy/Bias: Comparison to a reference method or predicate device.
• Precision/Reproducibility: Within-run, between-run, and total precision (CV%).
• Linearity/Measuring Range: The range over which results are proportional to the concentration.
• Limit of Detection (LoD) and Limit of Quantitation (LoQ): The lowest concentration that can be reliably detected and quantified.
• Interference: Evaluation of common endogenous and exogenous interfering substances.
• Specificity: Absence of significant cross-reactivity with related substances.

Since the FDA granted clearance based on substantial equivalence, it implies that the device's performance met the relevant acceptance criteria, demonstrating similar performance to a predicate device for these analytical characteristics.

2. Sample sized used for the test set and the data provenance (e.g. country of origin of the data, retrospective or prospective)

The provided document (510(k) clearance letter) does not contain details about the sample size, data provenance, or study design (retrospective/prospective) for the test set. These details would be part of the full 510(k) submission.

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts (e.g. radiologist with 10 years of experience)

This type of information is not relevant for an in vitro diagnostic assay like the IDS iSYS Direct Renin Assay. The "ground truth" for such assays is established through analytical methods and scientific standards (e.g., reference methods, known concentrations of analytes, spiked samples), not through expert consensus on images or clinical interpretation.

4. Adjudication method (e.g. 2+1, 3+1, none) for the test set

Not applicable for an in vitro diagnostic assay. Adjudication methods are typically used in studies involving human interpretation (e.g., radiology reads) to resolve discrepancies.

5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance

Not applicable for this in vitro diagnostic assay. MRMC studies and the concept of "human readers improving with AI assistance" are relevant to AI/ML medical devices that aid in interpretation or diagnosis by humans. This device is an automated laboratory assay for quantitative measurement.

6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done

This device, the IDS iSYS Direct Renin Assay, is an automated assay run on a "Multi-Discipline Automated System." Thus, its performance is inherently "standalone" in the sense that the measurement of Direct Renin concentration is performed by the instrument and reagents as an algorithm/system, without direct human cognitive input into result generation. A human interprets the final quantitative result. Performance studies for such devices evaluate the system's analytical capabilities directly.

7. The type of ground truth used (expert consensus, pathology, outcomes data, etc.)

For in vitro diagnostic assays, the "ground truth" for analytical performance evaluation typically involves:

• Reference Methods: Using a validated, often more complex or gold-standard, analytical method to determine true concentrations.
• Certified Reference Materials/Calibrators: Samples with precisely known concentrations of the analyte.
• Spiked Samples: Adding known amounts of the analyte to a matrix to assess recovery.
• Commutability/Traceability: Ensuring results are traceable to international reference standards if available.

The provided document does not specify the exact methods used to establish ground truth but these are the standard approaches for such devices.

8. The sample size for the training set

Not applicable in the context of AI/ML training data. For an in vitro diagnostic assay, there isn't a "training set" in the machine learning sense. The assay is developed through biochemical and engineering principles, and its performance is validated using experimental analytical studies.

9. How the ground truth for the training set was established

Not applicable for an in vitro diagnostic assay. This concept is specific to AI/ML model development.

Ask a specific question about this device

The IDS-iSYS 1,25 Dihydroxy Vitamin D assay is intended for the determination of 1,25 dihydroxyvitamin D levels in serum and plasma on the IDS-iSYS Multi-Discipline Automated System. Results of the 1.25 Dihydroxy Vitamin D are used in the assessment of vitamin D sufficiency.

The IDS-iSYS 1,25 Dihydroxy Vitamin D Control Set is used for quality control of the IDS-iSYS 1,25 Dihydroxy Vitamin D assay on the IDS-iSYS Multi-Discipline Automated System

The IDS-iSYS 1,25 Dihydroxy Vitamin D Calibration Verifier is a device intended for medical purposes for use in the quantitative verification of calibration of the IDS-iSYS 1,25 Dihydroxy Vitamin D assay when performed on the IDS-iSYS Multi-Discipline Automated Analyzer.

Not Found

Acceptance Criteria and Study for IDS iSYS 1,25 Dihydroxy Vitamin D Assay

Based on the provided document (K123253), the device in question is the IDS iSYS 1,25 Dihydroxy Vitamin D assay. This type of device is a diagnostic assay, and as such, its performance is typically evaluated based on analytical characteristics like precision, accuracy, linearity, and interference. The document itself is a 510(k) clearance letter, which indicates substantial equivalence to a predicate device. It doesn't contain the full study reports with detailed acceptance criteria and performance data.

However, based on typical FDA requirements for in vitro diagnostic assays and the context of a 510(k) submission for a Vitamin D test system, we can infer and describe the likely acceptance criteria and the nature of studies performed. This will be a reconstruction of what would typically be included in the full submission, as the provided document is merely the FDA's clearance letter.

1. Table of Acceptance Criteria and the Reported Device Performance

Since the detailed study results are not in the provided document, the "Reported Device Performance" here refers to the outcomes that would have led to a substantial equivalence determination for an assay. The acceptance criteria are typical for a diagnostic immunoassay.

2. Sample Size Used for the Test Set and Data Provenance

For an in vitro diagnostic assay, the "test set" would typically refer to the samples used in analytical validation studies (precision, accuracy, linearity, interference) and potentially clinical concordance studies if required.

• Sample Size for Test Set:
  • Precision: Typically involves multiple replicates (e.g., 20-40 replicates per sample level) across multiple runs and days, using several pooled or spiked patient samples at different concentrations (e.g., low, mid, high).
  • Accuracy/Method Comparison: Typically involves a statistically significant number of patient samples (e.g., 60-100+ uniquely different patient samples) spanning the assay's measuring range, tested on both the candidate device and the predicate device.
  • Linearity: At least 5-7 different concentrations spanning the assay's range, tested in replicates.
  • Interference: Samples spiked with interfering substances at varying levels (e.g., high, mid, low analyte concentrations).
• Data Provenance: The document does not specify. However, for a 510(k) submission, the studies are typically conducted at the manufacturer's R&D facilities and/or qualified clinical/reference laboratories. The country of origin for the data is most likely the United Kingdom (where Immunodiagnostic Systems Ltd. is located) and potentially other regions where the product was developed or tested. Data would be prospective for the validation studies, meaning the experiments were designed and executed to gather data for the submission.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications

For an in vitro diagnostic assay like this, "ground truth" is typically established by:

• Reference Methods: For accuracy/method comparison studies, the predicate device itself serves as a "comparator" or a well-established, validated reference method (e.g., LC-MS/MS for 1,25 Dihydroxy Vitamin D).
• Certified Reference Materials: For calibration and analytical measurement range verification.

Therefore, the concept of "experts establishing ground truth for the test set" is different from imaging or clinical decision support AI.

• Number of 'Experts': The "experts" would be the qualified laboratory personnel who operate the reference method or predicate device and analyze the data. This could involve multiple laboratory scientists/technicians (e.g., 2-5 qualified individuals) with expertise in clinical chemistry, immunology, or mass spectrometry, possessing relevant certifications and experience (e.g., Clinical Laboratory Scientists with 5+ years of experience in IVD testing, or specialists in LC-MS/MS). They ensure the predicate/reference method is properly calibrated and run.
• Qualifications of Experts: Clinical Laboratory Scientists (CLS), Medical Technologists (MT), or equivalent, with experience in immunoassay or LC-MS/MS testing, quality control, and method validation.

4. Adjudication Method for the Test Set

The concept of "adjudication" (e.g., 2+1, 3+1) is usually relevant for studies where subjective expert opinion or interpretation is being reconciled (e.g., reading medical images). For a quantitative IVD assay, adjudication in this sense is generally not applicable.

• Adjudication Method: None in the traditional sense. Data reconciliation primarily involves:
  • Statistical Analysis: Comparison of results from the candidate device against the predicate/reference method using statistical techniques (e.g., regression analysis, Bland-Altman plots, t-tests).
  • Quality Control Procedures: Rigorous adherence to laboratory SOPs, internal and external quality control programs, and review of analytical runs by senior laboratory staff to ensure data integrity and validity.
  • Discrepancy Resolution: Any significant discrepancies between duplicate runs or between the candidate and predicate methods would be investigated using predefined laboratory procedures (e.g., re-testing, sample re-collection, instrumentation checks).

5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was done

No, a Multi-Reader Multi-Case (MRMC) comparative effectiveness study would generally not be done for this type of device.

• Reason: MRMC studies are typically used for assessing the performance of diagnostic imaging devices or CADe/CADx systems where multiple human readers interpret cases, and their performance with and without AI assistance is compared. The IDS iSYS 1,25 Dihydroxy Vitamin D assay is an automated in vitro diagnostic test that provides a quantitative numerical result; it does not involve human "readers" interpreting "cases" in the same way an imaging study would.
• Effect Size: Not applicable.

6. If a Standalone (i.e. algorithm only without human-in-the-loop performance) was done

Yes, a standalone performance assessment is inherent to the validation of an IVD assay.

• Description: The entire analytical validation (precision, accuracy, linearity, LoD, LoQ, interference, specificity) assesses the "algorithm only" (which, in the case of an immunoassay, refers to the integrated system of reagents, calibrators, controls, and instrument's measurement and calculation protocols) without human intervention in the result generation process itself. The "human-in-the-loop" for such devices involves setting up the assay, loading samples, performing quality control, and clinically interpreting the final numerical result, but not directly influencing the measurement process.

7. The Type of Ground Truth Used

• Type of Ground Truth: The ground truth for this diagnostic assay would be established through a combination of:
  • Reference Methods: A more accurate or established laboratory method, often a Liquid Chromatography-Mass Spectrometry/Mass Spectrometry (LC-MS/MS) method, or a well-characterized and FDA-cleared predicate immunoassay.
  • Certified Reference Materials: Commercial reference materials with assigned target values for 1,25 Dihydroxy Vitamin D.
  • Method Comparison: Comparison against the legally marketed predicate device (as indicated by the 510(k) process focusing on "substantial equivalence"). The predicate device's performance, having been previously cleared/approved, serves as a benchmark for "true" values within the context of the comparison study.

8. The Sample Size for the Training Set

For a traditional in vitro diagnostic immunoassay like the IDS iSYS 1,25 Dihydroxy Vitamin D assay, there isn't a "training set" in the sense of machine learning. The device does not "learn" from data; it operates based on a pre-defined chemical reaction and measurement principle.

• Training Set Concept (Reinterpreted for IVDs): If one were to loosely interpret "training set" for an IVD, it would refer to:
  • Reagent Development and Optimization: The studies and samples used during the development phase to formulate reagents, optimize reaction conditions, and establish initial calibrator ranges. This is an iterative process.
  • Calibration: The specific set of calibrator materials (standards) used to establish the standard curve that converts raw signal (e.g., luminescence, absorbance) into a quantitative concentration. This would involve a small, defined set of calibrator points (e.g., 5-7 levels) run in multiple replicates.
• Sample Size: This would be highly variable and proprietary during the R&D phase. For the final calibration curve, it's a fixed, small number of calibrator points.

9. How the Ground Truth for the Training Set Was Established

Again, applying the "training set" and "ground truth" concepts to an IVD:

• Reagent Development/Optimization: The "ground truth" for optimizing the assay during development would be established through:
  • Spiking studies: Adding known amounts of purified 1,25 Dihydroxy Vitamin D to serum/plasma.
  • Comparative analyses: Testing early formulations against established research methods or highly purified reference materials.
  • Internal standards: Using internal reference standards during the development process.
• Calibration: The ground truth for establishing the calibrator values (the "training data" for the standard curve) would be established by:
  • Gravimetric preparation: Precisely weighing and dissolving purified, traceable 1,25 Dihydroxy Vitamin D to create primary calibrator stocks.
  • Comparison to reference methods: Analysing the prepared calibrator materials using a highly accurate reference method (e.g., LC-MS/MS) to assign their exact concentration.
  • Traceability: Ensuring the calibrator values are traceable to higher-order reference materials or methods, often internationally accepted standards.

Ask a specific question about this device