(42 days)
The ACCESS® Estradiol assay is a paramagnetic particle chemiluminescent immunoassay for the quantitative determination of Estradiol levels in human serum using the ACCESS® Immunoassay System.
The ACCESS® Estradiol reagents and the ACCESS® Immunoassay Analyzer comprise the ACCESS® Immunoassay System for the quantitative determination of Estradiol levels in human serum.
The provided document is a 510(k) summary for the ACCESS® Estradiol assay, which is an in-vitro diagnostic device (IVDD). The questions appear to be tailored for AI/Machine Learning (ML) medical devices, which typically require different types of studies and acceptance criteria than IVDDs.
Given the nature of the device (an immunoassay for quantifying Estradiol levels) and the document (510(k) submission from 1997), many of the requested fields are not applicable or the information is not provided in the same format as for AI/ML device studies.
However, I will extract and describe the available information as best as possible, indicating when a question is not applicable to this type of device or when the information is not present.
1. Table of Acceptance Criteria and Reported Device Performance
For an IVDD product like the ACCESS® Estradiol assay, "acceptance criteria" are typically defined by performance specifications and comparisons to a predicate device, rather than explicit thresholds of diagnostic accuracy (e.g., AUC, sensitivity, specificity) against a ground truth in a clinical setting. The "performance" refers to the analytical performance of the assay itself.
| Acceptance Criteria (Implied from Study Goals) | Reported Device Performance (ACCESS® Estradiol assay) |
|---|---|
| Precision: Demonstrate acceptable reproducibility of results. | Within-run CV: 11.6% (at 117 pg/ml) to 2.7% (at 3041 pg/ml) |
| Total Imprecision CV: 21.1% (at 72 pg/ml) to 5.2% (at 1456 pg/ml) | |
| Accuracy (Spiking Recovery): Recover spiked analyte within an acceptable range. | Recovery: 92% to 106% (mean recovery of 105% ± 7% SD for dilution recovery) |
| Accuracy (Dilution Recovery): Recover diluted analyte within an acceptable range. | Mean recovery: 105% with a standard deviation of 7% |
| Correlation with Predicate Device: Strong linear correlation with the legally marketed predicate device. | Correlation (r): 0.958 |
| Linear Regression: y = -39 + 0.983x (compared to Abbott IMx® Estradiol Kit) | |
| Analytical Sensitivity (LOD): Lowest detectable level distinguishable from zero. | Lowest detectable level: 23 pg/ml (with 95% confidence) |
| Analytical Specificity: Minimal cross-reactivity with structurally similar compounds. | Estrone sulfate: Not detectable at 3600 pg/ml |
| Estriol sulfate: Apparent cross-reactivity of 0.04% at 10,000,000 pg/ml |
2. Sample Size Used for the Test Set and Data Provenance
- Test Set Sample Size:
- Correlation Study: 214 samples were used for comparison between the ACCESS® Estradiol assay and the Abbott IMx® Estradiol Kit.
- Spiking Recovery: 10 serum samples.
- Dilution Recovery: 18 patient samples.
- Analytical Sensitivity: Implied use of Calibrator S0 (zero calibrator) multiple times to determine the limit of detection, but a specific number of samples/replicates is not given.
- Analytical Specificity: Not specified, but involved spiking into S0 calibrator.
- Precision Studies: Number of samples not explicitly stated, but typically involves multiple replicates of control materials or pooled patient samples run over several days/runs.
- Data Provenance: The document does not explicitly state the country of origin for the samples. Given the applicant's address in Chaska, MN, USA, it is highly probable the samples were from the USA.
- Retrospective or Prospective: Unspecified, but for analytical performance studies of IVDDs, these are typically laboratory-based studies using samples collected specifically for method validation or archived samples, rather than prospective clinical trials in the sense of AI/ML devices.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts
- Not Applicable in the context of diagnostic accuracy for an AI/ML device. For an IVDD, the "ground truth" for analytical studies is typically established by:
- The known concentration of an analyte in a spiked sample.
- The calculated expected concentration in a diluted sample.
- The results from a well-established, legally marketed predicate device (as in the correlation study).
- Known pure reference materials for specificity.
- There are no "experts" in the sense of human readers/interpreters establishing a ground truth for imaging or clinical diagnosis here.
4. Adjudication Method for the Test Set
- Not Applicable. Adjudication methods (e.g., 2+1, 3+1) are common in AI/ML studies where multiple human readers interpret data to reach a consensus ground truth. For an IVDD's analytical performance studies, this concept does not apply.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was Done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance
- No. An MRMC study is not relevant for an immunoassay kit. This device directly measures a biomarker concentration and does not involve human readers interpreting complex data that could be augmented by AI.
6. If a Standalone (i.e. algorithm only without human-in-the loop performance) was done
- Yes, in spirit. The performance described (precision, accuracy, correlation, sensitivity, specificity) is the standalone performance of the assay system (reagents + analyzer). There is no "human-in-the-loop" component in the direct measurement process of this IVDD; the analyzer outputs a quantitative result. The human then interprets that numerical result, which is distinct from human readers interpreting images or clinical data where AI might assist.
7. The Type of Ground Truth Used
- Reference Method Comparisons / Known Concentrations:
- Correlation Study: The results from the Abbott IMx® Estradiol Kit, a legally marketed predicate device, served as the comparative reference (analogous to a ground truth for comparison).
- Spiking Recovery: Known amounts of estradiol spiked into serum.
- Dilution Recovery: Expected concentrations based on dilution of patient samples.
- Analytical Sensitivity/Specificity: Defined concentrations of the analyte or interferent.
8. The Sample Size for the Training Set
- Not Applicable / Not Provided for this type of device. Immunoassays are not "trained" like AI/ML algorithms. The assay's parameters (e.g., antibody concentrations, incubation times, calibrator values) are developed and optimized during product development. The concept of a "training set" for an AI algorithm does not apply here. Calibrators are used to establish a standard curve for each run, which is different from a "training set" for an AI model.
9. How the Ground Truth for the Training Set was Established
- Not Applicable. As explained in point 8, there is no "training set" in the AI/ML sense. For the calibrators included with the assay, their "ground truth" (i.e., their assigned estradiol concentration) would have been established through a rigorous value assignment process, often involving reference methods or gravimetric preparation of highly pure standards. This is part of the manufacturing process for IVDDs.
§ 862.1260 Estradiol test system.
(a)
Identification. An estradiol test system is a device intended to measure estradiol, an estrogenic steroid, in plasma. Estradiol measurements are used in the diagnosis and treatment of various hormonal sexual disorders and in assessing placental function in complicated pregnancy.(b)
Classification. Class I (general controls). The device is exempt from the premarket notification procedures in subpart E of part 807 of this chapter subject to § 862.9.