Search Results

The ATTEST Drug Screen Cup and the ATTEST Drug Screen Dip Card are rapid lateral flow immunoassays for the qualitative detection of 6-Acetylmorphine, d-Amphetamine, Benzoylecgonine, EDDP, d/1-Methadone, d-Methamphetamine, d/Methylenedioxymethamphetamine, Nortriptyline, Oxazepam, Oxycodone, Phencyclidine, d-Propoxyphene, Secobarbital and THC-COOH in human urine. The test cut-off concentrations and the compounds the tests are calibrated to are as follows:

The single or multi-test panels can consist of the above listed analytes in any combination, up to a maximum of 16 analytes, with and without on-board adulteration/specimen validity tests (SVT) in the cup format. The drug screen tests are intended for prescription use only.

The tests provide only a preliminary result. To obtained a confirmed analytical result, a more specific alternative chemical method should be used. Gas Chromatography / Mass Spectrometry (GC/MS), Liquid Chromatography / Mass Spectrometry (LC/MS) and their tandem mass-spectrometer versions are the preferred confirmatory methods. Careful consideration and judgment should be applied to any drugs of abuse screen test result, particularly when evaluating preliminary positive results.

For prescription use, the devices consist of:

• a. 10 or 25 test cups or dip cards with or without adulteration/specimen validity tests
• b. Package Insert
• c. Procedure Card

Here's a breakdown of the acceptance criteria and study details for the ATTEST Drug Screen Cup and Dip Card, based on the provided document:

1. Table of Acceptance Criteria and Reported Device Performance

The document doesn't explicitly state "acceptance criteria" as a separate, quantitative target for each metric before presenting the results. Instead, it presents the performance data and implies that these results demonstrate the device's suitability. The primary performance metric presented is agreement with gold-standard methods (GC/MS, LC/MS or equivalent), and for most categories, the reported agreement is 100% or very close to it.

For the purpose of this analysis, I will infer the "acceptance criteria" from the reported performance, assuming that the FDA's acceptance is based on these high agreement rates, especially at or further from the cutoff concentrations.

2. Sample Size Used for the Test Set and Data Provenance

• Sample Size for Test Set: The method comparison/accuracy study (which serves as the "test set" for performance evaluation) uses the following sample sizes per drug test/cutoff combination:

  • Drug-free: 40 samples
  • -50% C/O to +25% C/O to +50% C/O: Varies (e.g., 4 for 6-AM/10, 5 for THC/20)
  • >+50% C/O: Varies (e.g., 35 for 6-AM/10, 46 for THC/20)
  • Total samples per drug test/cutoff category appears to be around 80-100 samples.

• Data Provenance: The document states that the studies used "clinical urine specimens previously quantitated for the target drugs of abuse by gold-standard methods (GC/MS, LC/MS or equivalent)." There is no explicit mention of the country of origin for these clinical specimens or if they were retrospective or prospective. Given the context of a 510(k) submission, these are typically retrospective studies using archived, de-identified clinical samples.

3. Number of Experts Used to Establish the Ground Truth and Qualifications

• The document does not mention the use of human experts to establish the ground truth for the test set.
• Ground truth for the method comparison/accuracy study was established by "gold-standard methods (GC/MS, LC/MS or equivalent)", which are analytical laboratory techniques, not expert human interpretation.

4. Adjudication Method for the Test Set

• Since the ground truth was established by instrumental analytical methods (GC/MS, LC/MS), there was no human adjudication (e.g., 2+1, 3+1) involved in determining the "true" presence or absence of drugs for the test set. The results from the gold-standard methods served as the definitive determination.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

• No MRMC comparative effectiveness study was done. The ATTEST Drug Screen Cup and Dip Card are qualitative lateral flow immunoassays and function as standalone diagnostic devices. Their performance is evaluated against chemical reference methods, not against human readers. Therefore, the concept of "effect size of how much human readers improve with AI vs without AI assistance" is not applicable.

6. Standalone (Algorithm Only Without Human-in-the-Loop Performance)

• Yes, a standalone performance evaluation was done. The ATTEST Drug Screen Cup and Dip Card are designed to be used as standalone qualitative tests. The performance data presented (precision, analytical specificity, and method comparison/accuracy tables) directly reflect the device's diagnostic capability without human in-the-loop interaction for result interpretation, beyond simply reading a positive or negative line visually as designed by a qualitative immunoassay. The results are based on the device's output (presence or absence of control/test lines).

7. Type of Ground Truth Used

• The ground truth used was analytical gold-standard methods: Gas Chromatography / Mass Spectrometry (GC/MS), Liquid Chromatography / Mass Spectrometry (LC/MS), and their tandem mass-spectrometer versions. The document explicitly states these are the "preferred confirmatory methods."

8. Sample Size for the Training Set

• The document does not specify the sample size used for the training set. As this device is a rapid lateral flow immunoassay (a chemical test, not an AI/machine learning algorithm), the concept of a "training set" in the context of predictive modeling is not directly applicable in the same way. The device's components and antibodies are developed and optimized through laboratory work and validation, rather than "training" on a large dataset of results in the way an AI algorithm would be.

9. How the Ground Truth for the Training Set Was Established

• Given that this is a chemical immunoassay, the concept of a "training set" for ground truth establishment, as it pertains to AI/machine learning, is not directly relevant. The "ground truth" during the development and optimization of such assays would involve:

  • Known concentrations of target analytes and potential cross-reactants: Used to optimize antibody binding, cutoff concentrations, and minimize false positives/negatives.
  • Control samples: Including negative urine samples (drug-free) and positive urine samples spiked with known drug concentrations.
  • Reference materials: Calibrators and controls with established values, often verified by GC/MS or LC/MS.

  The "precision/reproducibility" and "analytical specificity" studies described in the document reflect parts of the characterization and validation process that would inform the final design and performance claims of the device.

Ask a specific question about this device

The ATTEST Drug Screen Cup and the ATTEST Drug Screen Dip Card are rapid lateral flow immunoassays for the qualitative detection of 6-Acetylmorphine, d-Ampletamine, Benzoylecgonine, EDDP, d/1-Methadone, d-Methamphetamine, d/Methylenedioxymethamphetamine, Nortriptyline, Oxazepam, Oxycodone, Phencyclidine, d-Propoxyphene, Secobarbital and THC-COOH in human urine. The test cut-off concentrations and the compounds the tests are calibrated to are as follows:

The single or multi-test panels can consist of the above listed analytes in any combination, up to a maximum of 16 analytes, with and without on-board adulteration/specimen validity tests (SVT) in the cup format. The drug screen tests are intended for prescription use only.

The tests provide only a preliminary result. A more specific alternative chemical method should be used in order to obtain a confirmed presumptive positive result if the donor doesn't admit use or anytime required by testing procedures. Gas Chromatography / Mass Spectrometry (GC/MS), Liquid Chromatography / Mass Spectrometry (LC/MS) and their tandem mass-spectrometer versions are the preferred confirmatory methods. Careful consideration and judgment should be applied to any drugs of abuse screen test result, particularly when evaluating preliminary positive results.

For professional use, the devices consist of:

• a. 10 or 25 test cups or dip cards with or without adulteration/specimen validity tests
• b. Package insert
• c. Procedure Card

The provided text describes the performance characteristics of the ATTEST Drug Screen Cup and ATTEST Drug Screen Dip Card, which are rapid lateral flow immunoassays for the qualitative detection of various drugs in human urine.

Here's an analysis of the acceptance criteria and the study proving the device meets them:

1. Table of Acceptance Criteria and Reported Device Performance

The acceptance criteria for qualitative drug screening devices are typically based on agreement with a gold-standard confirmatory method (like GC/MS, LC/MS) at and around the cut-off concentrations. The concept of "agreement" translates to sensitivity and specificity for qualitative tests. While explicit numeric acceptance criteria (e.g., "must achieve >95% positive agreement") aren't directly stated as "acceptance criteria" in a separate table, they are implied by the performance statistics presented in the "Method Comparison/Accuracy Studies" section.

The reported device performance is presented in the two large tables under "Method Comparison/Accuracy Studies" (Pages 14-16), showing the agreement of the ATTEST devices (Cup and Dip Card) with GC/MS or LC/MS results.

Let's interpret the tables to deduce the implicit acceptance criteria and the reported performance. The "Agreement" column in these tables represents the percentage of agreement between the device's qualitative result (Negative or Positive) and the quantitative result from the gold-standard method, particularly around the cutoff and at significant concentrations above and below it.

For each drug assay (e.g., 6-AM/10, AMP/500, etc.), the data shows the number of samples that fell into different concentration ranges relative to the cutoff (e.g., "Drug-free", "-50% C/O to +50% C/O"). The "Agreement" percentage is then provided for both Negative and Positive results.

Implicit Acceptance Criteria (Deduced from Reported Performance):

Note: The performance percentages across the two formats (Cup and Dip Card) are presented identically in the provided combined tables for all assays, suggesting the performance data is pooled or very similar for both. The discordant results tables (pages 15-17) provide a more granular view for specific instances where the device result differed from the GC/MS or LC/MS result, primarily for samples near the cutoff. For example, for AMP/500, a 97.7% negative agreement means 2.3% of negative samples were discordant (likely false negative or near-cutoff negative that read positive by device). For Amp/500, 3 "Neg" results were in the "-50% C/O to +25% C/O to +50% C/O, >+50% C/O).
* The "Summary of Discordant Results" tables (pages 15-17) list specific instances of samples where the device's qualitative result did not match the quantitative GC/MS/LC/MS result (e.g., "Positive" device result for a sample with a drug concentration slightly below the cutoff, or "Negative" for a sample slightly above). These are qualitative examples, not the full dataset.

• Data Provenance: The studies used clinical urine specimens "previously quantitated for the target drugs of abuse by gold-standard methods (GC/MS, LC/MS or equivalent)." The text does not specify the country of origin of the data or whether the samples were retrospective or prospective collections. However, given the nature of a 510(k) submission for a device to be marketed in the US, it is generally assumed that the clinical studies are relevant to the US population.

3. Number of Experts Used to Establish Ground Truth for the Test Set and Qualifications

This type of device (lateral flow immunoassay for drug screening) does not rely on human experts for establishing ground truth for the test set. The ground truth is established by analytical methods (Gas Chromatography / Mass Spectrometry (GC/MS), Liquid Chromatography / Mass Spectrometry (LC/MS) and their tandem mass-spectrometer versions), which are considered the preferred confirmatory methods for drug testing. Therefore, there were no human "experts" in the sense of radiologists interpreting images to establish medical ground truth.

4. Adjudication Method for the Test Set

Not applicable. Since the ground truth is established by objective analytical methods (GC/MS, LC/MS), there's no need for an adjudication process involving human readers or experts. The assay results are compared directly against the quantitative chemical analysis.

5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was Done

No, an MRMC comparative effectiveness study was not done. The device is a rapid diagnostic test for drug detection, not an AI-powered image analysis tool for human interpretation. The study is a comparison of the device's performance against a gold-standard chemical analysis method, not a study of human readers. Therefore, there is no effect size related to human readers improving with AI assistance.

6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) was done

Yes, the study described is essentially a standalone (algorithm/device-only) performance evaluation. The device's qualitative results are directly compared to the quantitative results from GC/MS/LC/MS, without human interpretation as an intermediate step. The "intended professional users" mentioned in Section M.a.iv (Detection Limit/sensitivity) are those who read the test result (presence/absence of a line), not those who interpret complex data as an AI model would.

7. The Type of Ground Truth Used

The ground truth used for the test set was analytical confirmation by gold-standard chemical methods: Gas Chromatography / Mass Spectrometry (GC/MS), Liquid Chromatography / Mass Spectrometry (LC/MS), or equivalent. This is explicitly stated in "Method Comparison/Accuracy Studies" (Page 14).

8. The Sample Size for the Training Set

The document describes a 510(k) submission for a diagnostic device. Unlike AI/ML models, these traditional diagnostic devices do not typically have a "training set" in the sense of a data set used to train an algorithm. The device's formulation and antibodies are developed through a research and development process, and its performance is then validated (tested) using various samples.

The document mentions "The precision, reproducibility and sensitivity of the ATTEST Drug Screen Cup and the ATTEST Drug Screen Dip Card were evaluated at Advin Biotech and at three (3) external testing sites. Data obtained at all testing sites across the intended use populations indicate >99% correlation at +/-50% of each assay cutoff." (Page 8-9). This suggests internal development and testing, but not explicitly a "training set" as understood in machine learning. The stability and specificity studies also involve testing with numerous samples.

9. How the Ground Truth for the Training Set was Established

As there is no "training set" for this type of device in the AI sense, this question is not directly applicable. The device's underlying chemistry and design are based on established immunochromatographic principles. The process involves:

• Standardization: The assays are harmonized with DHHS/SAMHSA screening cutoffs (Page 8-9).
• Analytical Specificity: This is established by testing the device with contrived solutions of chemically related/structurally similar compounds and other potential interferents (Pages 9-13). This involves verifying that the device reacts appropriately to the target analyte and does not cross-react significantly with other substances.
• Precision/Reproducibility: Evaluated through repeated testing at various concentrations and across different sites (Page 8).

These evaluations contribute to the overall confidence in the device's performance and design, akin to "establishing ground truth" for its analytical properties, but it's fundamentally different from training an AI model on a labeled dataset.

Ask a specific question about this device