(133 days)
The Gold Standard Diagnostics Borrelia burgdorferi IgG ELISA Test Kit is intended as a qualitative test for the detection of IgG antibodies to B. burgdorferi sensu stricto in human serum from symptomatic patients or people suspected of infection. When used as the first-tier screening test, positive and equivocal results must be supplemented through additional testing by one of the following methods:
· Standard two-tier test methodology (STTT) using an IgG blot test following current interpretation guidelines, OR
• Modified two-tier test methodology (MTT) using the Gold Standard Diagnostics Borrelia burgdorferi VlsE-OspC IgG/ IgM ELISA Test.
The assay can also be used as a second-tier confirmation test using the MTTT methodology when used with the Gold Standard Diagnostics Borrelia burgdorferi VlsE-OspC IgG/IgM ELISA Test as the first-tier screening test.
Positive test results by either the STTT or MTTT methodology are supportive evidence for the presence of antibodies and exposure to Borrelia burgdorferi, the cause of Lyme disease. A diagnosis of Lyme disease should be made based on the presence of Borrelia burgdorferi antibodies, history, symptoms, and other laboratory findings.
The kit includes 12 x 8 well Antigen Coated strips. Conjugate. Substrate, Stop Solution, Wash Buffer, Diluent, Negative Control, Positive Control, and Cutoff Control. The controls are provided to determine if the assay is functioning properly and to determine the antibody level. The reagents are sufficient for 96 determinations.
During the test procedure, antibodies to B. burgdorferi (sensu stricto) if present in the human serum sample will bind to the antigens coated onto the wells forming antigen-antibody complexes. Excess antibodies are removed by washing. A conjugate of goat anti-human IgG antibodies conjugated with horseradish peroxidase is then added, which binds to the antigenantibody complexes. Excess conjugate is removed by washing. This is followed by the addition of a chromogenic substrate, tetramethylbenzidine (TMB). If specific antibodies to the antigen are present in the patients' serum, a blue color will develop. The enzymatic reaction is then stopped with a stopping solution causing the contents of the well to turn yellow. The wells are read photometrically with a microplate reader at 450nm.
The antigens used in the Gold Standard Diagnostics Borrelia burgdorferi IgG ELISA Test kit is a combination of B. burgdorferi sensu stricto strain B31 lysate, B. burgdorferi sensu stricto strain 2591 lysate, and a recombinant VlsE from B. burgdorferi sensu stricto strain B31. The lysates use spirochetes growing in BSK-H complete medium until mid-exponential phase. The recombinant VlsE protein is produced in E. coli SURE2 cells and purified by affinity chromatography. The purity of each antigen is assayed by SDS-PAGE followed by Coomassie staining and/or Western blotting.
This document describes the validation of the Gold Standard Diagnostics Borrelia burgdorferi IgG ELISA Test Kit, including a new proposed Modified Two-tier Testing (MTTT) methodology. The provided text primarily focuses on comparative studies against predicate devices and existing standard methods, rather than defining explicit acceptance criteria in a quantitative table format. However, implied acceptance criteria can be derived from the performance percentages presented.
Based on the provided information, here's a structured response addressing the requested points:
1. Table of Acceptance Criteria (Implied) and Reported Device Performance
The acceptance criteria are not explicitly stated as numerical cutoffs in the document. However, they can be inferred from the reported "Percent Agreement" values in the comparison studies. The studies aim to demonstrate substantial equivalence to established predicate devices and methodologies.
| Study Type / Performance Metric | Implied Acceptance Criteria (Goal: High Agreement/Sensitivity/Specificity) | Reported Device Performance (Gold Standard Diagnostics Borrelia burgdorferi IgG ELISA Test Kit) |
|---|---|---|
| Comparison with Predicate Device (STTT) | ||
| Positive Percent Agreement (PPA) with Predicate IgG ELISA | High (> 90% typically desired for equivalence) | 90.2% (55/61) [95% CI: 79.8% - 96.3%] |
| Negative Percent Agreement (NPA) with Predicate IgG ELISA | High (> 95% typically desired for equivalence) | 99.6% (460/462) [95% CI: 98.5% - 99.9%] |
| Second-Tier Testing (STTT) - Agreement with FDA Cleared IgG Blot | ||
| 2nd Tier PPA (Tier 1 Pos/Equiv samples) | 100.0% (as reported) | 100.0% (37/37) [95% CI: 92.2% - 100.0%] |
| Clinical Sensitivity (STTT) | ||
| Early Lyme | Improvement over predicate | 46.6% (27/58) vs. Predicate: 27.6% (16/58) |
| Disseminated Lyme | Improvement over predicate | 82.4% (14/17) vs. Predicate: 52.9% (9/17) |
| Late Lyme | High agreement with predicate | 97.4% (38/39) vs. Predicate: 97.4% (38/39) |
| CDC Panel (STTT) | ||
| Healthy | High (> 99% agreement, implying high specificity) | 99.0% (1/100 positive/equivocal) |
| Early Lyme | Improvement over predicate | 68.3% (41/60 positive/equivocal) vs. Predicate: 35.0% (21/60) |
| Cardiac Lyme | Comparable/Improvement | 66.7% (2/3 positive/equivocal) vs. Predicate: 100.0% (3/3) |
| Neurological Lyme | Improvement over predicate | 85.7% (6/7 positive/equivocal) vs. Predicate: 42.9% (3/7) |
| Late Lyme | High agreement with predicate | 100.0% (20/20 positive/equivocal) vs. Predicate: 100.0% (20/20) |
| Look-alike Disease | High (> 85% agreement, implying high specificity) | 87.8% (11/90 positive/equivocal) vs. Predicate: 88.9% (10/90) |
| Method Comparison MTTT – IgG (vs. STTT) | ||
| PPA (second-tier test of MTTT vs. STTT) | 100.0% (as reported) | 100.0% (23/23) [95% CI: 85.2% - 100.0%] |
| NPA (second-tier test of MTTT vs. STTT) | High (implied > 95%) | 0.0% (0/15) [95% CI: 0.0% - 21.8%] - Note: This NPA is for a subset of samples that were negative by IgG Immunoblot, not overall. The overall NPA is 96.7% |
| PPA (Overall MTTT vs. STTT) | 100.0% (as reported) | 100.0% (23/23) [95% CI: 85.2% - 100.0%] |
| NPA (Overall MTTT vs. STTT) | High (implied > 95%) | 96.7% (443/458) [95% CI: 94.7% - 98.2%] |
| Clinical Sensitivity (MTTT) | ||
| Early Lyme | Improvement over predicate STTT | 48.4% (30/62) vs. Predicate STTT: 8.1% (5/62) |
| Disseminated Lyme | Improvement over predicate STTT | 81.8% (18/22) vs. Predicate STTT: 22.7% (5/22) |
| Late Lyme | Comparable/Improvement | 97.6% (40/41) vs. Predicate STTT: 95.1% (39/41) |
| CDC Reference Panel (MTTT) | ||
| Healthy | 100.0% (as reported) | 100.0% (0/100 positive/equivocal) |
| Early Lyme | Improvement over predicate STTT | 60.0% (36/60 positive/equivocal) vs. Predicate STTT: 33.3% (12/60) |
| Cardiac Lyme | Improvement over predicate STTT | 66.7% (2/3 positive/equivocal) vs. Predicate STTT: 33.3% (1/3) |
| Neurological Lyme | Improvement over predicate STTT | 85.7% (6/7 positive/equivocal) vs. Predicate STTT: 14.3% (1/7) |
| Late Lyme | 100.0% (as reported) | 100.0% (20/20 positive/equivocal) vs. Predicate STTT: 100.0% (20/20) |
| Look-alike Disease | 100.0% (as reported) | 100.0% (0/90 positive/equivocal) vs. Predicate STTT: 100.0% (0/90) |
2. Sample Size Used for the Test Set and Data Provenance
-
Determination of Assay Cutoff (Nonclinical):
- Sample Size: 210 normal sera (105 endemic, 105 non-endemic) + 194 samples (114 Lyme disease stages, 8 healthy negative, 72 negative Lyme disease with other conditions). Total = 404 samples.
- Data Provenance: Not explicitly stated, but "endemic region of Lyme disease" and "non-endemic region of Lyme disease" suggest geographical diversity. Retrospective, as these are "tested" samples for cutoff determination.
-
Precision (Nonclinical):
- Sample Size: 48 runs for each of 4 panel members (negative, high negative, low positive, moderate positive) + kit controls. This is a repetitive testing design, not individual patient samples.
-
Reproducibility (Nonclinical):
- Sample Size: 90 runs (3 sites x 2 runs/day x 5 days x 3 replicates) for each of 4 panel members (negative, high negative, low positive, moderate positive) + kit controls (30 runs for Pos/Neg control, 60 runs for Cutoff control).
-
Analytical Specificity:
- Sample Size: 208 asymptomatic individuals (103 from endemic regions, 105 from non-endemic regions).
- Data Provenance: Endemic and non-endemic regions. Retrospective (asymptomatic individuals' samples).
-
Cross Reactivity:
- Sample Size: 377 samples.
- Data Provenance: Samples obtained from "serum vendors who confirmed their positivity for each respective marker," implying retrospective, controlled samples.
-
Comparison/Prospective Study (Clinical):
- STTT Comparison: 523 serum samples.
- MTTT Comparison: 481 serum samples for the initial screening. 38 positive/equivocal samples were carried forward for second-tier testing comparison.
- Data Provenance: "Prospective samples submitted for Lyme serology testing" from "three sites (one internal and two external reference laboratories)." This indicates a prospective study design with samples from potentially diverse geographical locations (clinical labs).
-
Sensitivity Study (Clinical):
- STTT: 114 clinically characterized samples.
- MTTT: 125 clinically characterized samples.
- Data Provenance: "Clinically characterized samples" from unspecified sources; likely retrospective collection of known patient cohorts.
-
CDC Panel (Clinical):
- Sample Size: 280 positive and negative specimens.
- Data Provenance: From the "Center of Disease Control (CDC)." These are reference panels, typically collected and characterized over time, so retrospective.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and the Qualifications of Those Experts
The document does not explicitly state the number or qualifications of experts used to establish the ground truth for the test sets.
- For "clinically characterized samples" and "CDC panel" samples, the implication is that these samples have a pre-defined and reliable "clinical diagnosis" or "reference characterization" based on established medical criteria. This often involves consensus from clinical experts (e.g., infectious disease specialists) and/or a combination of clinical presentation, symptoms, and other laboratory findings, but the specifics are not detailed here.
- For the "Second-Tier Testing" comparison, the "FDA cleared IgG blot assay" and "predicate B. burgdorferi IgG blot test" serve as the ground truth. These are approved diagnostic methods, and their interpretation would follow established guidelines.
4. Adjudication Method for the Test Set
The document does not describe any specific adjudication method (e.g., 2+1, 3+1) for resolving discrepancies in the test results or for establishing the initial ground truth diagnoses. The comparisons are based on the results of the different assays and against established clinical characterizations or CDC panels.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance
No MRMC study was performed or described. This device is an ELISA test kit, a laboratory diagnostic assay for detecting antibodies, not an AI or imaging diagnostic tool that involves human readers interpreting images. Therefore, the concept of "human readers improving with AI assistance" is not applicable.
6. If a Standalone (i.e. algorithm only without human-in-the-loop performance) was done
Yes, the primary performance evaluation is standalone. The Gold Standard Diagnostics Borrelia burgdorferi IgG ELISA Test Kit operates as a standalone diagnostic assay. Its performance (sensitivity, specificity, agreement) is measured as the output of the kit itself (optical density readings converted to units and qualitative results) without direct human interpretation of complex visual patterns or AI assistance. The results are quantitative and then interpreted qualitatively (positive, equivocal, negative) based on predefined cutoffs.
7. The Type of Ground Truth Used (expert consensus, pathology, outcomes data, etc.)
The ground truth used varies by study:
- Predicate Devices/Methodologies: For the STTT and MTTT comparison studies, the ground truth for the device's performance is a comparison against the results from legally marketed predicate devices (e.g., Trinity Biotech MarDx Borrelia burgdorferi EIA IgG Test Kit) and/or a FDA cleared IgG blot assay. This is a form of comparative ground truth against established methods.
- Clinical Diagnosis/Characterization: For the Sensitivity Study and CDC Panel, the ground truth is "clinically characterized samples" and "positive and negative specimens from the Centers of Disease Control (CDC)." This implies a clinical diagnosis or consensus diagnosis based on comprehensive patient data (history, symptoms, other laboratory findings) or a reference panel with established disease status.
- Analytical Ground Truth: For non-clinical studies like analytical specificity and cross-reactivity, the ground truth is often the confirmed absence or presence of specific conditions in the tested samples, usually based on prior laboratory testing or sample vendor claims.
8. The Sample Size for the Training Set
The document does not explicitly mention a "training set" in the context of machine learning. The term "training set" is usually associated with the development of AI algorithms. For an ELISA kit, development involves:
- Antigen Selection and Optimization: Which antigens to use (e.g., B31 lysate, 2591 lysate, recombinant VlsE).
- Reagent Formulation: Optimizing conjugate, substrate, buffers, etc.
- Cutoff Determination: "The cutoff was determined by testing a total of 210 normal sera..." and "An additional 194 samples..." This process of determining the cutoff could be considered analogous to a 'calibration' or 'training' phase for a traditional diagnostic test, where a dataset is used to define the diagnostic thresholds. In this sense, a total of 404 samples (210 normal + 194 other known samples) were used for cutoff determination.
9. How the Ground Truth for the Training Set Was Established
As discussed above, for establishing the cutoff (analogous to training/calibration):
- Normal Sera: The 210 normal sera were likely verified as "normal" through standard clinical practice or donor screening, indicating the absence of target antibodies/disease in a healthy population.
- Known Positive/Negative/Other Disease Samples: The additional 194 samples included "different phases of Lyme disease," "negative healthy samples," and "negative Lyme disease samples but do have other diseases that may cause serologic cross-reactivity." The "ground truth" for these samples would be their known clinical status or disease classification, typically established through comprehensive clinical evaluation, follow-up, and/or panels from disease banks. The ROC analysis was used to confirm the chosen cutoff provides the best compromise between sensitivity and specificity, indicating an analytical optimization process based on these known samples.
§ 866.3830
Treponema pallidum treponemal test reagents.(a)
Identification. Treponema pallidum treponemal test reagents are devices that consist of the antigens, antisera and all control reagents (standardized reagents with which test results are compared) which are derived from treponemal sources and that are used in the fluorescent treponemal antibody absorption test (FTA-ABS), theTreponema pallidum immobilization test (T.P.I.), and other treponemal tests used to identify antibodies toTreponema pallidum directly from infecting treponemal organisms in serum. The identification aids in the diagnosis of syphilis caused by bacteria belonging to the genusTreponema and provides epidemiological information on syphilis.(b)
Classification. Class II (performance standards).