(146 days)
This in vitro diagnostic method is intended to qualitatively measure IgM antibodies to Toxoplasma gondii in human serum using the Bayer Immuno 1™ system. Measurements of anti-Toxoplasma IgM antibodies are presumptive for the diagnosis of recent Toxoplasma infection. Anti-Toxoplasma gondii IgM antibody measurements must be performed in conjunction with an anti-Toxoplasma gondii IgG antibody assay.
This method has not been cleared or approved by the FDA for blood or plasma donor screening.
The use of the Bayer Immuno 1 Toxoplasma IgM method has not been validated for testing of cord blood or neonatal serum.
This method is intended for in vitro diagnostic use only on the Bayer Immuno 1 system.
The Bayer IMMUNO 1 Toxoplasma IgM assay is a qualitative, heterogeneous IgM antibody capture magnetic separation sandwich immunoassay. A 5 uL patient serum sample (calibrator or control) and 20 uL of mIMP (monoclonal ImmunoMagnetic Particle) Reagent are mixed in the reaction cuvette and incubated with 150 µL of Reagent 1 (R1) containing fluorescein labeled monoclonal anti-human IgM antibody. After 15 minutes, the particles are washed and 100 uL of a binary reagent (R2), consisting of purified Toxoplasma antigen and monoclonal F(ab), anti-Toxoplasma alkaline phosphatase conjugate, is added. The binary reagent reacts with the bound Toxoplasma specific IgM from the sample. After a 12 minute incubation, the particles are washed to remove unbound conjugate and the substrate (pNPP) is added. The rate of hydrolysis of the substrate is measured at 405 nm and is proportional to the concentration of Toxoplasma specific IgM in the sample. The patient results are reported as index values relative to the cut-off (index) calibrator. Time to first result is 38 minutes.
Bayer Corporation Business Group Diagnostics sought to add the qualitative measurement of IgM antibodies to Toxoplasma gondii in human serum to the Bayer Immuno 1 analyzer system. The performance of the Bayer Immuno 1 Toxoplasma IgM assay was established by comparison to a predicate device, the Abbott IMx Toxoplasma IgM assay.
Here's a breakdown of the acceptance criteria and study details:
1. Table of Acceptance Criteria and Reported Device Performance
The acceptance criteria were benchmarked against the predicate device, the Abbott IMx Toxoplasma IgM assay. The reported performance is presented as relative to this predicate unless otherwise specified as "resolved" with a third-party EIA method.
| Metric | Acceptance Criteria (Implicit - Relative to Abbott IMx) | Reported Device Performance (Bayer Immuno 1 Toxoplasma IgM assay) | Notes |
|---|---|---|---|
| Combined Clinical Data | |||
| Relative Sensitivity | High agreement with predicate | 89.4% (93/104) | This refers to agreement with the Abbott IMx assay before discrepancy resolution. |
| Relative Specificity | High agreement with predicate | 98.3% (975/992) | This refers to agreement with the Abbott IMx assay before discrepancy resolution. |
| % Agreement | High agreement with predicate | 97.4% (1068/1096) | Overall agreement with the Abbott IMx assay. |
| Resolved Sensitivity | Improved agreement after resolution | 94.7% (108/114) | After discordant samples were re-tested with a second EIA method (Sanofi Platelia or Denmark in-house method) and the ground truth was updated. |
| Resolved Specificity | Improved agreement after resolution | 99.1% (980/988) | After discordant samples were re-tested with a second EIA method (Sanofi Platelia or Denmark in-house method) and the ground truth was updated. |
| Prevalence of Toxoplasma IgM (Normal Donors) | High correlation with predicate | 99.65% (285/286) | Agreement with Abbott Toxo M IMx assay for normal donor samples. The prevalence of Toxoplasma IgM antibodies in normal donors was 0.35% for the Immuno 1 reagents. |
| Interference Studies | No adverse effect on performance | Demonstrated no adverse effect | Tested with various potentially problematic sera (acute EBV/CMV, RF, ANA, HAMA, multiple myeloma, hemolyzed, lipemic, elevated bilirubin). |
| Specificity of IgM class antibody | Confirmed IgM class specificity | All positive samples reduced after DTT treatment and IgM activity located in IgM fraction post-fractionation. | Indicated the Immuno 1 Toxoplasma IgM assay is specific for IgM class antibody, addressing potential interference from IgG. |
| Precision | Acceptable within-run and total precision | Within-run %CV: 3.1-15.1%; Total %CV: 4.5-16.2% | Across various control and patient samples, indicating good reproducibility. Calibration every 30 days was found acceptable. |
| Reproducibility | No significant difference across sites | No significant difference in recovery across sites | Demonstrated by testing a 20-member panel of samples at three different clinical sites. |
| ROC Curve Analysis (vs. Abbott IMx) | High Area Under the Curve (AUC) | San Francisco: 0.942; University of Texas: 0.967 | Indicates high accuracy relative to the Abbott IMx method, assuming IMx is the reference. 95% CI: 0.915-0.963 (SF) and 0.945-0.982 (UT). |
| Reagent Stability | Maintain performance over time | 12-month shelf-life (2-8°C), 30-day on-system stability, shipping under refrigeration. | Confirmed for unopened product, opened and on-system, and shipping conditions. |
| Controls & Calibrator Stability | Maintain performance over time | 12-month shelf-life (2-8°C), 90-day open vial stability (2-8°C), shipping under refrigeration. | Confirmed for unopened product, opened vial, and shipping conditions. |
2. Sample Sizes Used for the Test Set and Data Provenance
- Prevalence of Toxoplasma IgM (Normal Donors):
- Test Set Size: 287 random donor samples.
- Data Provenance: Not explicitly stated, but implies general population samples, likely from the US given the study locations (Tarrytown, NY). Retrospective.
- Clinical Sensitivity and Specificity:
- Test Set Size: 1121 samples.
- Data Provenance:
- 988 prenatal, hospital, and blood donor samples from the US (University of Texas Medical Branch, Galveston, Texas; San Francisco General Hospital Medical Center, San Francisco, California; Bioclinical Partners, Franklin, MA). Prospective/Retrospective (not explicitly differentiated for all).
- 133 samples from toxoplasmosis patients from Denmark (State Serum Laboratory), France (Rangueil University Hospital Center, Laboratory of Parasitology-Mycology), and Poland (National Institute of Hygiene, Parasitology Department). Likely retrospective.
- Validation of Cut-off:
- Test Set Size: 400 serum samples (287 normal donors, 15 sequential samples from a Toxoplasma infected patient, and samples with potentially interfering substances).
- Data Provenance: Normal donor samples likely from the US. Interference samples were from various patient types; provenance not specified.
- Precision:
- Test Set Size: The table aggregates data (e.g., Negative Control (1420), Positive Control (914), Index Calibrator (931), Positive 1 (478), Positive 2 (324)). These numbers refer to the total number of measurements/runs, not unique patient samples.
- Data Provenance: Pooled across the three clinical trial sites (San Francisco, University of Texas, Tarrytown).
- Reproducibility:
- Test Set Size: 20-member panel of negative and positive samples.
- Data Provenance: Tested across the three clinical trial sites (San Francisco, University of Texas, Tarrytown).
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications
The document does not explicitly state the number or qualifications of "experts" used to establish the initial ground truth for the test set.
- For the initial evaluation of clinical sensitivity and specificity, the Abbott IMx Toxoplasma IgM assay was used as the reference method. This implies the "ground truth" was derived from the results of this predicate device.
- For discordant samples, a second Toxoplasma IgM EIA method was used for resolution (Denmark in-house method or Sanofi Platelia method). The results of these methods were used to re-evaluate the "ground truth" for those specific samples. This suggests that these established diagnostic methods were used as the standard, rather than a panel of individual clinical experts reviewing each case independently.
4. Adjudication Method for the Test Set
A form of discrepancy resolution was used:
- Method: "Resolution - A second Toxoplasma IgM EIA method was used to reconcile the discordant samples. The other EIA method was the Denmark in-house method (for samples provided by the State Serum Institute, Copenhagen, Denmark) or the Sanofi Platelia method (for all other samples). If the IMx value and the second EIA value disagreed, then the second EIA value was substituted. Otherwise, the IMx value was retained. IMMUNO 1 values were not changed."
- This is not a traditional 2+1 or 3+1 expert adjudication but rather an adjudication by a third, independent, and established laboratory method when the initial comparison (Immuno 1 vs. Abbott IMx) was discordant.
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
No, a multi-reader multi-case (MRMC) comparative effectiveness study as typically understood for human reader performance with and without AI assistance was not done. This study is for an in-vitro diagnostic device (an automated analyzer), not an AI system assisting human interpretation of images or other medical data. The comparison is between two automated diagnostic assays.
6. Standalone (Algorithm Only Without Human-in-the-Loop Performance) Study
Yes, the entire study focuses on the standalone performance of the Bayer Immuno 1 Toxoplasma IgM assay. It is an automated in-vitro diagnostic method, meaning its performance is evaluated directly without human interpretation as part of the primary diagnostic output. Human involvement would be in operating the instrument and interpreting the final quantitative/qualitative result provided by the assay, but not in deriving the initial measurement.
7. Type of Ground Truth Used
The primary ground truth for clinical performance evaluation was established by a predicate device (Abbott IMx Toxoplasma IgM assay). For discordant results, a second, independent commercial or in-house EIA method (Sanofi Platelia or Denmark in-house method) served as the adjudicating "ground truth." This is functionally a "reference method" ground truth.
8. Sample Size for the Training Set
The document does not explicitly mention a separate "training set" in the context of device development as might be seen for machine learning algorithms. This device is an immunoassay system, and its development would typically involve optimizing reagents and protocols. The "Validation of Cut-off" section describes setting the cut-off using 400 serum samples, which might represent a form of internal parameter setting or optimization rather than a separate "training set" in the modern AI sense.
9. How the Ground Truth for the Training Set Was Established
Given that this is an immunoassay system and not a machine learning model, the concept of a "training set" and its "ground truth" as typically defined for AI development is not directly applicable.
For the validation of the cut-off, which is a crucial parameter setting similar to training in some contexts:
- The cut-off was established by testing a population of 400 serum samples (287 normal donors, 15 sequential samples from a Toxoplasma infected patient, and samples containing potentially interfering substances).
- The "ground truth" for these samples would have been their established status (negative, positive, or negative but with interferents) based on clinical diagnosis, serological history, or known spiked interferents.
- The cut-off was set based on the statistical distribution of results from the negative population: "The cut-off was set at greater than 7 standard deviations (SD) above the mean of the negative population; asymptomatic donor samples and patient samples containing potentially interfering substances." This statistical approach defines the classification threshold rather than relying on an external ground truth for each sample in a training set.
§ 866.3780
Toxoplasma gondii serological reagents.(a)
Identification. Toxoplasma gondii serological reagents are devices that consist of antigens and antisera used in serological tests to identify antibodies toToxoplasma gondii in serum. Additionally, some of these reagents consist of antisera conjugated with a fluorescent dye (immunofluorescent reagents) used to identifyToxoplasma gondii from clinical specimens. The identification aids in the diagnosis of toxoplasmosis caused by the parasitic protozoanToxoplasma gondii and provides epidemiological information on this disease. Congenital toxoplasmosis is characterized by lesions of the central nervous system, which if undetected and untreated may lead to brain defects, blindness, and death of an unborn fetus. The disease is characterized in children by inflammation of the brain and spinal cord.(b)
Classification. Class II (performance standards).