(189 days)
This kit is intended for measuring human Immunoglobulin D (IgD) in serum as an aid in the diagnosis of abnormal protein metabolism and the body's lack of ability to resist infectious agents.
Not Found
This appears to be a 510(k) clearance letter from the FDA for a medical device. This type of document typically approves a device based on its substantial equivalence to a predicate device, rather than detailing a specific study and its acceptance criteria for a novel device or AI algorithm.
Therefore, the requested information regarding acceptance criteria, specific studies, sample sizes, expert involvement, and ground truth establishment, as it would apply to a typical pre-market approval (PMA) or a new AI/software as a medical device (SaMD) submission, is not present in this document.
Here's why and what we can infer:
- 510(k) Clearance: This is a premarket submission made to FDA to demonstrate that the device to be marketed is at least as safe and effective, that is, substantially equivalent, to a legally marketed predicate device. The primary focus is on demonstrating equivalence, not necessarily on proving performance against predefined acceptance criteria for a novel technology through extensive new clinical trials.
- "Human IgD Liquid Reagent Kit for use on the Behring BNII Analyzer": This is an in-vitro diagnostic (IVD) kit. For IVDs, substantial equivalence often involves demonstrating comparable analytical and clinical performance to an already cleared predicate device. This would typically involve studies on:
- Analytical Performance: Precision, accuracy, linearity, limit of detection, limit of quantitation, analytical specificity (interference, cross-reactivity).
- Clinical Performance: Correlation to a predicate method using patient samples or clinical utility in diagnosis, but often not in the same rigorous blinded, multi-reader, prospective fashion as a novel diagnostic imaging AI.
Despite the lack of specific details, I can address your request based on the typical practices for such a device and what might have been included in the full 510(k) application (which is not provided here).
Hypothetical Description of Acceptance Criteria and Study (Based on Typical IVD 510(k) Requirements)
Given that this is an In-Vitro Diagnostic (IVD) device (a reagent kit), the acceptance criteria would typically focus on analytical performance metrics and concordance with a predicate device or an established gold standard.
1. Table of Acceptance Criteria and Reported Device Performance
| Performance Metric | Acceptance Criteria (Hypothetical) | Reported Device Performance (Hypothetical) |
|---|---|---|
| Analytical Performance | ||
| Precision (Intra-assay CV) | LOQ | Achieved LOQ |
| Limit of Detection (LoD) | ≤ 0.5 mg/L | 0.2 mg/L (95% probability of detection) |
| Limit of Quantitation (LoQ) | ≤ 1.0 mg/L (CV 90% with predicate device (e.g., ELISA) | Overall agreement of 96% (n=200) compared to predicate device in patient samples |
| Diagnostic Sensitivity | Appropriate for intended use population | (May not be a primary endpoint for a 510(k) if based on equivalence, but might be reported if available) |
| Diagnostic Specificity | Appropriate for intended use population | (Same as above) |
2. Sample size used for the test set and the data provenance
- Test Set (for performance validation): Typically, clinical validation for an IVD 510(k) might involve anywhere from 100 to 500 patient samples to compare against a predicate or reference method. Additional samples would be used for analytical studies (e.g., dilution series, spiked samples, interference panels).
- Data Provenance: The data would almost certainly be retrospective for comparison studies, possibly with a smaller prospective component (e.g., for method comparison on fresh samples). Given The Binding Site is a UK-based company (though the letter is to a US representative), it's plausible the data could have originated from multiple countries, including the UK and the US, or from sites that supplied samples to the manufacturer for testing.
3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts
For an IgD assay, the "ground truth" isn't typically established by a panel of human experts in the same way an image interpretation might be.
- Ground Truth for IVDs: The ground truth for quantitative assays like this is usually established by:
- Reference Methods: Highly accurate, often more laborious or less accessible, laboratory methods (e.g., nephelometry using a specific calibrator, or potentially a standardized ELISA method).
- Predicate Device Results: For a 510(k), the predicate device's results on the same samples would serve as the primary "ground truth" for demonstrating substantial equivalence.
- Clinical Diagnosis: For evaluating clinical utility, patient samples would be selected based on confirmed clinical diagnoses (e.g., from patient charts adjudicated by a pathologist or clinical immunologist), and then the IgD levels would be correlated with these diagnoses. The number of such "experts" would be the clinicians/pathologists involved in the original patient diagnosis, not necessarily a panel specifically created for the device study.
4. Adjudication method for the test set
- Not applicable in the traditional sense. Since the device measures a quantitative analyte, "adjudication" wouldn't be done by multiple readers reviewing the device's output. Instead, it would involve:
- Statistical comparison: Statistical methods (e.g., Bland-Altman analysis, regression analysis, percent agreement for categorical results) would be used to compare the device's results to the reference method or predicate device.
- QC interpretation: Laboratory professionals would review quality control data to ensure runs are valid.
5. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance
- Not applicable. This device is an in-vitro diagnostic reagent kit for measuring a specific protein (Immunoglobulin D). It is not an AI-powered diagnostic imaging tool or a system designed to assist human readers (e.g., radiologists) in interpreting complex data. Therefore, an MRMC study and the concept of "human readers improving with AI" are not relevant to this type of device.
6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done
- Yes, by nature. An IVD kit on an automated analyzer like the Behring BNII is designed to provide a quantitative result (an IgD concentration) based on reagents and instrumentation, without human interpretation of the "performance" of an algorithm in the sense of a diagnostic decision. The result is generated by the analytical process, and then a human interprets that numerical result in a clinical context.
7. The type of ground truth used
- Reference Method / Predicate Device Results. For this type of IVD, the ground truth for the measured IgD concentration would be established by:
- Testing the same samples on a legally marketed predicate device (e.g., another FDA-cleared IgD assay).
- Using a highly characterized reference laboratory method or a calibrated reference material for accuracy studies.
- Clinical Outcomes Data/Pathology: While these define the relevance of IgD levels (e.g., high IgD associated with certain conditions), they are generally not the direct "ground truth" for the measurement of IgD itself. The device measures the analyte; clinicians use that measurement in conjunction with pathology and other outcomes to form a diagnosis.
8. The sample size for the training set
- Not explicitly applicable in the AI sense. This is a reagent kit, not an AI algorithm that undergoes "training." The development would involve analytical characterization using various buffers, spiked samples, and potentially a diverse set of real patient samples to optimize the reagent formulation and assay parameters. The sample sizes for these development/optimization phases are part of the manufacturer's internal R&D process and wouldn't be termed "training sets" in the machine learning context.
9. How the ground truth for the training set was established
- Not applicable as an "AI training set." If we were to interpret "training set" as the samples used during assay development and optimization, the "ground truth" for these samples would be established through:
- Known concentrations: For spiked samples or linearity studies.
- Reference method results: For native patient samples to ensure the new assay correlates well with established methods during its development.
- Characterized patient samples: Samples from patients with known IgD-related conditions or from healthy individuals to ensure the assay performs as expected across the relevant clinical range.
§ 866.5510 Immunoglobulins A, G, M, D, and E immunological test system.
(a)
Identification. An immunoglobulins A, G, M, D, and E immunological test system is a device that consists of the reagents used to measure by immunochemical techniques the immunoglobulins A, G, M, D, an E (serum antibodies) in serum. Measurement of these immunoglobulins aids in the diagnosis of abnormal protein metabolism and the body's lack of ability to resist infectious agents.(b)
Classification. Class II (performance standards).