Search Results

The SMART Leish is a qualitative diagnostic real-time PCR test for the rapid detection of Leishmania species and the identification of L. major in skin lesion scrapings and punch biopsies from individuals suspected of having cutaneous leishmaniasis. The test utilizes real-time polymerase chain reaction assay on the Cepheid SmartCycler® II Dx to detect Leishmania species and L. major. The SMART Leish is indicated for use in patients with clinical presentations and travel history.

The SMART Leish consists of an assay reagent kit and assay definition files for the polymerase chain reaction (PCR) instrument platform. The kit contains sufficient reagents, in lyophilized bead form, to qualitatively assay 50 clinical samples for both Leishmania genus and L. major targets. Additional required accessories that are specified include the PCR instrument platform, a deoxyribonucleic acid (DNA) purification kit, and a positive extraction control. The device components and required accessories are listed as follows.

For the Leishmania assays, a tissue specimen (skin scraping or punch biopsy) from an individual suspected of being infected with Leishmania species or L. major is collected in 70% or 100% ethanol, and the DNA is extracted from the specimen using the Qiagen QlAamp DNA Mini Kit. An aliquot of this DNA is tested using the Leishmania genus assay, which will amplify a portion of DNA encoding the Leishmania species 16S ribosomal ribonucleic acid (rRNA) gene if present. Amplified targets are detected using a TaqMan® hybridization probe with 6-carboxyfluorescein.(FAM) reporter dye (517 nm) and a Black Hole Quencher® (BHQ). This assay also contains a positive internal control consisting of a nonsensical, non-naturally occurring DNA sequence, with Texas Red reporter dye (615 nm) and BHQ, used to detect evidence of PCR inhibition and confirm the integrity of assay reagents in negative specimens. If the sample tests positive for Leishmania species, another aliquot of DNA may be tested using the L. major assay, which will amplify a portion of DNA encoding the GPI gene that is specific to L. major. Amplified targets are detected using a TaqMan hybridization probe with FAM reporter dye and a BHQ. The DNA amplification, detection of fluorescence, and interpretation of signals are done automatically by the SmartCycler instrument for each assay. The thermocycling protocols take approximately 45-55 minutes.

Here's a breakdown of the acceptance criteria and the study details for the SMART Leish device, based on the provided text:

Acceptance Criteria and Reported Device Performance

The document does not explicitly state pre-defined "acceptance criteria" in terms of numerical thresholds for sensitivity, specificity, or agreement. However, the reported performance suggests what the submitter considered acceptable for demonstrating substantial equivalence. The provided data focuses on the clinical performance and various analytical performance metrics.

Study Details

2. Sample size used for the test set and the data provenance

• Leishmania Genus Assay: 312 prospective specimens (225 positive, 87 negative)
• Leishmania major Assay: 187 prospective specimens (96 positive, 91 negative)
• Data Provenance: Multi-center, retrospective clinical study conducted at two military U.S. hospital sites. The patient population consisted entirely of military personnel who were at the time, or had previously been, deployed to Southwest Asia—primarily Afghanistan and Iraq.

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts

The document does not specify the number or qualifications of experts used to establish the ground truth.

4. Adjudication method (e.g. 2+1, 3+1, none) for the test set

The document does not specify an adjudication method. The ground truth was established by "culture and/or microscopy." In cases of discrepancy (false positives by SMART Leish), sequencing methods were used for further investigation, but this is a reconciliation rather than a primary adjudication method for establishing initial ground truth.

5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance

• No, an MRMC comparative effectiveness study was not done.
• This device is a diagnostic assay (PCR test), not an AI-assisted imaging or interpretation tool for human readers. Therefore, the concept of "human readers improve with AI vs without AI assistance" does not apply here.

6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done

• Yes, a standalone performance study was done. The SMART Leish is a fully automated diagnostic assay where "DNA amplification, detection of fluorescence, and interpretation of signals are done automatically by the SmartCycler instrument for each assay." The clinical performance data presented (Tables 2 and 3) reflects this standalone performance against the clinical truth.

7. The type of ground truth used (expert consensus, pathology, outcomes data, etc)

The ground truth for the clinical performance study (test set) was established using culture and/or microscopy. For discordant results (false positives by SMART Leish compared to initial ground truth), sequencing methods were used for further analysis.

8. The sample size for the training set

The document does not specify a "training set" or its size. As a PCR diagnostic assay, it would typically undergo a development and optimization phase, but the document focuses on validation/testing. The "analytical reactivity" and "analytical specificity" studies used various Leishmania strains and non-target organisms, which could be considered part of the development/validation process.

• Analytical Reactivity Study: 36 Leishmania strains.
• Analytical Specificity Study: 121 DNA samples (11 L. major, 25 non-L. major, 85 non-target organisms from various categories).

9. How the ground truth for the training set was established

Since no explicit "training set" is mentioned in the context of machine learning, the establishment of ground truth would refer to the characteristics of the strains used in analytical studies. These would be:

• For Leishmania strains: Identity established through standard microbiological and molecular characterization methods (e.g., taxonomic classification, phenotypic and genotypic analysis, likely confirmed by sequencing).
• For non-target organisms: Identity established through standard microbiological and molecular characterization methods.

Ask a specific question about this device