(363 days)
The EUROIMMUN Anti-M2-3E ELISA (IgG) test kit is intended for the qualitative or semi-quantitative determination of IgG class autoantibodies against the mitochondrial antigens M2 in human serum and plasma. It is used as an aid in the diagnosis of primary billary cirrhosis (PBC), in conjunction with other laboratory and clinical findings.
The EUROIMMUN Anti-M2-3E ELISA (IgG) consists of a microwell ELISA plate coated with M2-3E antigen, 3 calibrators, positive and negative control, peroxidase-labelled anti-human IgG conjugate, sample buffer, wash buffer concentrate, TMB chromogen/substrate solution and stop solution.
The provided text describes the performance characteristics of the EUROIMMUN Anti-M2-3E ELISA (IgG) test kit and its comparison to a predicate device and clinical studies. However, it does not explicitly define "acceptance criteria" in a separate section with specific numerical thresholds for each performance metric before presenting the study results. Instead, the "Performance Characteristics" section M presents the study results for analytical and clinical performance.
Based on the information provided, I will infer what constitutes "acceptance criteria" from the reported performance, especially for the "Method comparison with predicate device" and "Clinical studies" sections, where the device needs to show substantial equivalence or good diagnostic accuracy.
1. Table of Acceptance Criteria and Reported Device Performance
Inferred Acceptance Criteria:
- Precision/Reproducibility: Intra-assay and Inter-assay CVs 90%
- Clinical Sensitivity (for PBC): High sensitivity (e.g., >90%).
- Clinical Specificity: High specificity (e.g., >90%) across various control groups.
Reported Device Performance:
| Performance Metric | Inferred Acceptance Criteria | Reported Device Performance |
|---|---|---|
| Precision/Reproducibility | Intra-assay CVs 90%. | Overall agreement: 91.9% (95% C.I.: 85.6% - 96.0%) for 127 samples. Positive agreement: 98.7% (95% C.I.: 93.1% - 100.0%). Negative agreement: 79.5% (95% C.I.: 64.7% - 90.2%). |
| Matrix Comparison (Serum vs. Plasma) | 95% C.I. of slope contains 1.0; 95% C.I. of intercept contains 0. | EDTA plasma: slope 0.98-1.04 (contains 1.0), intercept -1.30-2.75 (contains 0). Heparin plasma: slope 0.96-1.01 (contains 1.0), intercept -1.32-1.11 (contains 0). Citrate plasma: slope 0.98-1.06 (contains 1.0), intercept -1.02-3.18 (contains 0). All conditions satisfied. |
| Clinical Sensitivity (for PBC) | High sensitivity (e.g., >90%) for PBC. | 92.8% (for PBC patients). For PBC/AIH overlap syndrome patients, 100.0%. |
| Clinical Specificity | High specificity (e.g., >90%) across various control groups. | Overall specificity: 97.4% (95% C.I.: 96.2% - 98.3%) for 929 control samples. (Ranges from 92.0% for SLE to 100.0% for Viral Hepatitis, PSC, and Asymptomatic Blood Donors). |
| Expected Values (Healthy Donors) | All healthy blood donors negative at cutoff. | 200 apparently healthy blood donors were all negative (0 positives) with the 20 RU/ml cutoff. |
2. Sample sizes used for the test set and the data provenance
- Precision/Reproducibility:
- Intra-assay: 12 unique samples, each tested 20 times.
- Inter-assay: 12 unique samples, each tested 4 times over 6 days (total 24 determinations per sample).
- Lot to Lot: 8 unique samples, each tested across 3 different lots and 2 runs (total 6 determinations per sample).
- Linearity: 5 patient sera, each serially diluted up to 1/32, with single determinations.
- Analytical Specificity (Cross-reactivity): 29 sera (10 PCA, 10 GBM, 9 LKM positive).
- Interference: 5 different specimens at varying Anti-M2-3E concentrations, spiked with potential interfering substances.
- Method Comparison with Predicate Device: 127 clinically characterized samples (39 AIH, 49 PBC, 32 AIH/PBC overlap, and 7 created "borderline" samples by mixing) collected at EUROIMMUN UK Ltd. This data appears to be retrospective as samples were already "clinically characterized."
- Matrix Comparison: 21 sample pairs of serum and corresponding plasma.
- Clinical Studies: 1180 clinically characterized samples (251 from PBC patients and 929 from control groups). The text mentions "performed in cooperation with several university hospitals," suggesting retrospective collection of already diagnosed patient samples.
3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts
The document does not specify the number of experts used to establish the ground truth, nor their qualifications for any of the studies (method comparison or clinical studies). It states samples were "clinically characterized" for the method comparison and "clinically characterized" for the clinical study, but does not detail the process or who performed it.
4. Adjudication method for the test set
The document does not describe any adjudication method for establishing the ground truth for either the method comparison or the clinical studies. Ground truth appears to be based on "clinical characterization" which typically implies diagnosis based on multiple clinical and laboratory findings, but the specific process for confirming these diagnoses is not elaborated.
5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance
This is not applicable to the provided document. The device is an in vitro diagnostic ELISA test kit, not an AI-assisted diagnostic tool that would involve human readers interpreting images or data with and without AI assistance. The performance is based on the assay's ability to detect autoantibodies.
6. If a standalone (i.e. algorithm only without human-in-the loop performance) was done
Yes, the studies presented are all standalone performance evaluations of the EUROIMMUN Anti-M2-3E ELISA (IgG) test kit. The performance characteristics (analytical and clinical) measure the device's ability to detect specific autoantibodies directly, without human interpretation of the assay's output other than reading the optical density and interpreting the result against the established cutoff.
7. The type of ground truth used (expert consensus, pathology, outcomes data, etc.)
The ground truth for the clinical studies appears to be based on clinical diagnosis of Primary Biliary Cirrhosis (PBC) and various control conditions. The phrase "clinically characterized samples" implies that these diagnoses were established through a combination of clinical findings, patient history, other laboratory tests, and possibly biopsy/pathology results, as part of routine medical practice at the "several university hospitals." However, specific details like "expert consensus" or "pathology" are not explicitly mentioned as the sole or primary ground truth source.
8. The sample size for the training set
The document does not mention a distinct "training set" in the context of machine learning or algorithm development. The ELISA test kit is a biochemical assay. The "calibrators" are used to establish a calibration curve for quantitative or semi-quantitative results, and "controls" are used for quality assurance, but these are not a "training set" in the AI/ML sense. The product development would involve internal validation and optimization, but specific "training set sizes" are not provided.
9. How the ground truth for the training set was established
As there is no "training set" in the context of an algorithm, this question is not applicable. For the calibrators and controls included in the kit, their "ground truth" (i.e., their known concentration or reactivity level) is established during the manufacturing and quality control processes of the diagnostic kit.
§ 866.5090 Antimitochondrial antibody immunological test system.
(a)
Identification. An antimitochondrial antibody immunological test system is a device that consists of the reagents used to measure by immunochemical techniques the antimitochondrial antibodies in human serum. The measurements aid in the diagnosis of diseases that produce a spectrum of autoantibodies (antibodies produced against the body's own tissue), such as primary biliary cirrhosis (degeneration of liver tissue) and chronic active hepatitis (inflammation of the liver).(b)
Classification. Class II (performance standards).