(62 days)
ImmunoCyt is a qualitative direct immunocytofluorescence assay intended for use in conjunction with cytology to increase the overall sensitivity for the detection of tumor cells exfoliated in the urine of patients previously diagnosed with bladder cancer. ImmunoCyt is indicated for use as an aid in the management of bladder cancer in conjunction with urinary cytology and cystoscopy.
ImmunoCyt is an in vitro diagnostic device that contains a solution of three monoclonal antibodies. Two antibodies are reactive to epitopes selectively detected on a mucin found in bladder cancer cells, and one antibody reacts with a bladder cancer-associated glycosylated form of the carcinoembryonic antigen. The antibodies are coupled with fluorescent markers. This solution is used to detect tumor cells exfoliated in the urine of bladder cancer patients. ImmunoCyt also contains a sample fixative and a blocking solution, as well as positive and negative control cells.
Here's a breakdown of the acceptance criteria and study information for the ImmunoCyt device, based on the provided 510(k) summary:
Acceptance Criteria and Device Performance
The 510(k) summary primarily focuses on demonstrating substantial equivalence to predicate devices, rather than explicit pre-defined acceptance criteria for ImmunoCyt's performance. However, the reported sensitivity and specificity values against the predicate devices imply a comparative acceptance.
| Acceptance Criteria (Implied) | Reported ImmunoCyt Performance | Predicate Device 1 (Bard BTA stat Test) Performance | Predicate Device 2 (AuraTek FDP) Performance |
|---|---|---|---|
| Sensitivity | 94% | 66% | 67% |
| Specificity | 50% | 80% | 70% |
Note on "Acceptance Criteria": It's crucial to understand that for a 510(k) submission, the primary "acceptance criterion" is often substantial equivalence to a legally marketed predicate device. This means demonstrating that the new device is as safe and effective as the predicate. While ImmunoCyt's stated sensitivity is higher than the predicates, its specificity is lower. The FDA ultimately found it substantially equivalent due to other factors and its intended use in conjunction with cytology.
Study Details
2. Sample sizes and data provenance:
- Test Set for Sensitivity: 87 urine samples from patients with bladder tumor recurrences confirmed by histology.
- Test Set for Specificity 1 (monitored patients): 154 urine samples from patients with negative cystoscopy (monitored for bladder tumor recurrence).
- Test Set for Specificity 2 (normal individuals): 170 urine samples from normal individuals (without genitourinary symptoms).
- Test Set for Specificity 3 (other GU disorders): 100 urine samples from patients with various genitourinary disorders other than bladder cancer, as established by cystoscopy.
- Data Provenance: Samples were collected retrospectively from 14 hospital centers in Canada.
3. Number of experts used to establish the ground truth for the test set and their qualifications:
- The document does not explicitly state the number of experts or their qualifications for establishing the ground truth (histology or cystoscopy results). It only states "confirmed by histology" or "established by cystoscopy." Usually, these are performed and interpreted by pathologists and urologists, respectively, but specific numbers and experience are not provided.
4. Adjudication method for the test set:
- The document does not explicitly describe an adjudication method (e.g., 2+1, 3+1). The ground truth appears to be based on clinical and pathological findings (histology, cystoscopy) as standard clinical practice rather than a consensus interpretation of the ImmunoCyt results themselves for ground truth establishment.
5. If a multi-reader multi-case (MRMC) comparative effectiveness study was done:
-
A formal MRMC comparative effectiveness study in the sense of human readers with and without AI assistance was not conducted.
-
However, a reproducibility study was performed that involved multiple readers and multiple laboratories to assess within-reader, between-reader, and between-laboratory variability of the ImmunoCyt device's interpretation. This is different from a comparative effectiveness study of the device aiding human readers.
- Within-reader variability: Three readers from three different laboratories, each performing triplicate and blinded readings of one panel of three patient slides and one panel of three mock samples.
- Between-reader variability: Nine readers from three different laboratories, each performing single and blinded reading of a panel of nine mock samples slides.
- Between-laboratory variability: Three readers performing single and blinded reading of four patient slides for each level of positivity.
-
Effect size improvement with AI vs without AI assistance: Not applicable, as this type of MRMC study was not performed.
6. If a standalone (i.e., algorithm only without human-in-the-loop performance) was done:
- The ImmunoCyt device itself is an in vitro diagnostic immunocytofluorescence assay, which implies a laboratory technician/microscopist is reading the stained cells. It's not an "algorithm" in the typical standalone AI sense. The performance reported (sensitivity and specificity) is the standalone performance of the assay as interpreted by human observers, separate from the ground truth.
- Its intended use is “in conjunction with cytology” which means it's an additive test to a human-interpreted cytology result.
7. The type of ground truth used:
- Clinical Ground Truth:
- For bladder cancer recurrence confirmation: Histology.
- For absence of bladder cancer in monitored patients: Negative cystoscopy.
- For normal individuals and other genitourinary disorders: Cystoscopy (to establish presence/absence of cancer or other conditions).
8. The sample size for the training set:
- The document does not explicitly mention a training set size. As ImmunoCyt is an antibody-based immunocytochemistry assay, rather than a machine learning algorithm, the concept of a "training set" for model development isn't directly applicable in the same way it would be for an AI device. The manufacturing and QC processes would be more analogous to its "training."
9. How the ground truth for the training set was established:
- Not applicable as it's not an AI/ML device with a distinct "training set" in the conventional sense. The "ground truth" for developing and validating the antibodies and assay components would have been established through standard biological and chemical methods during its development.
§ 864.1860 Immunohistochemistry reagents and kits.
(a)
Identification. Immunohistochemistry test systems (IHC's) are in vitro diagnostic devices consisting of polyclonal or monoclonal antibodies labeled with directions for use and performance claims, which may be packaged with ancillary reagents in kits. Their intended use is to identify, by immunological techniques, antigens in tissues or cytologic specimens. Similar devices intended for use with flow cytometry devices are not considered IHC's.(b)
Classification of immunohistochemistry devices. (1) Class I (general controls). Except as described in paragraphs (b)(2) and (b)(3) of this section, these devices are exempt from the premarket notification requirements in part 807, subpart E of this chapter. This exemption applies to IHC's that provide the pathologist with adjunctive diagnostic information that may be incorporated into the pathologist's report, but that is not ordinarily reported to the clinician as an independent finding. These IHC's are used after the primary diagnosis of tumor (neoplasm) has been made by conventional histopathology using nonimmunologic histochemical stains, such as hematoxylin and eosin. Examples of class I IHC's are differentiation markers that are used as adjunctive tests to subclassify tumors, such as keratin.(2) Class II (special control, guidance document: “FDA Guidance for Submission of Immunohistochemistry Applications to the FDA,” Center for Devices and Radiologic Health, 1998). These IHC's are intended for the detection and/or measurement of certain target analytes in order to provide prognostic or predictive data that are not directly confirmed by routine histopathologic internal and external control specimens. These IHC's provide the pathologist with information that is ordinarily reported as independent diagnostic information to the ordering clinician, and the claims associated with these data are widely accepted and supported by valid scientific evidence. Examples of class II IHC's are those intended for semiquantitative measurement of an analyte, such as hormone receptors in breast cancer.
(3) Class III (premarket approval). IHC's intended for any use not described in paragraphs (b)(1) or (b)(2) of this section.
(c)
Date of PMA or notice of completion of a PDP is required. As of May 28, 1976, an approval under section 515 of the Federal Food, Drug, and Cosmetic Act is required for any device described in paragraph (b)(3) of this section before this device may be commercially distributed. See § 864.3.