Search Results

The ATTEST Drug Screen Cup and the ATTEST Drug Screen Dip Card are rapid lateral flow immunoassays for the qualitative detection of 6-Acetylmorphine, d-Amphetamine, Benzoylecgonine, EDDP, d/1-Methadone, d-Methamphetamine, d/Methylenedioxymethamphetamine, Nortriptyline, Oxazepam, Oxycodone, Phencyclidine, d-Propoxyphene, Secobarbital and THC-COOH in human urine. The test cut-off concentrations and the compounds the tests are calibrated to are as follows:

The single or multi-test panels can consist of the above listed analytes in any combination, up to a maximum of 16 analytes, with and without on-board adulteration/specimen validity tests (SVT) in the cup format. The drug screen tests are intended for prescription use only.

The tests provide only a preliminary result. To obtained a confirmed analytical result, a more specific alternative chemical method should be used. Gas Chromatography / Mass Spectrometry (GC/MS), Liquid Chromatography / Mass Spectrometry (LC/MS) and their tandem mass-spectrometer versions are the preferred confirmatory methods. Careful consideration and judgment should be applied to any drugs of abuse screen test result, particularly when evaluating preliminary positive results.

For prescription use, the devices consist of:

• a. 10 or 25 test cups or dip cards with or without adulteration/specimen validity tests
• b. Package Insert
• c. Procedure Card

Here's a breakdown of the acceptance criteria and study details for the ATTEST Drug Screen Cup and Dip Card, based on the provided document:

1. Table of Acceptance Criteria and Reported Device Performance

The document doesn't explicitly state "acceptance criteria" as a separate, quantitative target for each metric before presenting the results. Instead, it presents the performance data and implies that these results demonstrate the device's suitability. The primary performance metric presented is agreement with gold-standard methods (GC/MS, LC/MS or equivalent), and for most categories, the reported agreement is 100% or very close to it.

For the purpose of this analysis, I will infer the "acceptance criteria" from the reported performance, assuming that the FDA's acceptance is based on these high agreement rates, especially at or further from the cutoff concentrations.

2. Sample Size Used for the Test Set and Data Provenance

• Sample Size for Test Set: The method comparison/accuracy study (which serves as the "test set" for performance evaluation) uses the following sample sizes per drug test/cutoff combination:

  • Drug-free: 40 samples
  • -50% C/O to <-25% C/O: Varies (e.g., 4 for 6-AM/10, 22 for THC/20)
  • -25% C/O to C/O: Varies (e.g., 1 for 6-AM/10, 6 for THC/20)
  • C/O to +25% C/O: Varies (e.g., 1 for 6-AM/10, 2 for THC/20)
  • >+25% C/O to +50% C/O: Varies (e.g., 4 for 6-AM/10, 5 for THC/20)
  • >+50% C/O: Varies (e.g., 35 for 6-AM/10, 46 for THC/20)
  • Total samples per drug test/cutoff category appears to be around 80-100 samples.

• Data Provenance: The document states that the studies used "clinical urine specimens previously quantitated for the target drugs of abuse by gold-standard methods (GC/MS, LC/MS or equivalent)." There is no explicit mention of the country of origin for these clinical specimens or if they were retrospective or prospective. Given the context of a 510(k) submission, these are typically retrospective studies using archived, de-identified clinical samples.

3. Number of Experts Used to Establish the Ground Truth and Qualifications

• The document does not mention the use of human experts to establish the ground truth for the test set.
• Ground truth for the method comparison/accuracy study was established by "gold-standard methods (GC/MS, LC/MS or equivalent)", which are analytical laboratory techniques, not expert human interpretation.

4. Adjudication Method for the Test Set

• Since the ground truth was established by instrumental analytical methods (GC/MS, LC/MS), there was no human adjudication (e.g., 2+1, 3+1) involved in determining the "true" presence or absence of drugs for the test set. The results from the gold-standard methods served as the definitive determination.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

• No MRMC comparative effectiveness study was done. The ATTEST Drug Screen Cup and Dip Card are qualitative lateral flow immunoassays and function as standalone diagnostic devices. Their performance is evaluated against chemical reference methods, not against human readers. Therefore, the concept of "effect size of how much human readers improve with AI vs without AI assistance" is not applicable.

6. Standalone (Algorithm Only Without Human-in-the-Loop Performance)

• Yes, a standalone performance evaluation was done. The ATTEST Drug Screen Cup and Dip Card are designed to be used as standalone qualitative tests. The performance data presented (precision, analytical specificity, and method comparison/accuracy tables) directly reflect the device's diagnostic capability without human in-the-loop interaction for result interpretation, beyond simply reading a positive or negative line visually as designed by a qualitative immunoassay. The results are based on the device's output (presence or absence of control/test lines).

7. Type of Ground Truth Used

• The ground truth used was analytical gold-standard methods: Gas Chromatography / Mass Spectrometry (GC/MS), Liquid Chromatography / Mass Spectrometry (LC/MS), and their tandem mass-spectrometer versions. The document explicitly states these are the "preferred confirmatory methods."

8. Sample Size for the Training Set

• The document does not specify the sample size used for the training set. As this device is a rapid lateral flow immunoassay (a chemical test, not an AI/machine learning algorithm), the concept of a "training set" in the context of predictive modeling is not directly applicable in the same way. The device's components and antibodies are developed and optimized through laboratory work and validation, rather than "training" on a large dataset of results in the way an AI algorithm would be.

9. How the Ground Truth for the Training Set Was Established

• Given that this is a chemical immunoassay, the concept of a "training set" for ground truth establishment, as it pertains to AI/machine learning, is not directly relevant. The "ground truth" during the development and optimization of such assays would involve:

  • Known concentrations of target analytes and potential cross-reactants: Used to optimize antibody binding, cutoff concentrations, and minimize false positives/negatives.
  • Control samples: Including negative urine samples (drug-free) and positive urine samples spiked with known drug concentrations.
  • Reference materials: Calibrators and controls with established values, often verified by GC/MS or LC/MS.

  The "precision/reproducibility" and "analytical specificity" studies described in the document reflect parts of the characterization and validation process that would inform the final design and performance claims of the device.

Ask a specific question about this device

The OralTox® Oral Fluid Drug Test is a competitive binding, lateral flow immunochromatographic assay for the qualitative and simultaneous detection of Amphetamine, Cocaine, Marijuana (THC), Methamphetamine, Opiates, Phencyclidine, Oxycodone and Methadone in human oral fluid at the cutoff concentrations listed below and their metabolites:

The test provides only preliminary test results. A more specific alternative chemical method must be used in order to obtain a confirmed analytical result. Liquid Chromatography/Mass Spectrometry/ Mass Spectrometry (LC-MS/MS) is the preferred confirmatory method. It is not intended to detect intermittent dosing of Oxycodone. Clinical consideration and professional judgment should be exercised with any drug of abuse test result. particularly when the preliminary result is positive.

The OralTox Oral fluid Drug Test is an immunochromatographic assay that uses a lateral flow system for the qualitative detection of Amphetamine, Cocaine, Cannabinoids, Methamphetamine, Morphine, Phencyclidine. Oxycodone and Methadone (target analytes) in human oral fluid. The products are single-use in vitro diagnostic devices. Each test kit contains a test cup, a package insert and a sample collection sponge. Each test device is sealed with a desiccant in an aluminum pouch.

The provided document describes the OralTox® Oral fluid Drug Test, a competitive binding, lateral flow immunochromatographic assay for the qualitative and simultaneous detection of several drugs in human oral fluid. This submission is a 510(k) premarket notification, indicating the device is intended to be substantially equivalent to previously marketed devices.

Here's an analysis of the acceptance criteria and the study proving the device meets them, based on the provided text:

Acceptance Criteria and Reported Device Performance

The core acceptance criteria for this device are related to its analytical performance in detecting specific drugs at defined cutoff concentrations. The studies performed focus on demonstrating the accuracy and reliability of the device in comparison to a confirmed analytical method (LC/MS/MS).

Table of Acceptance Criteria and Reported Device Performance (Methadone and Oxycodone - as these were the focus of new data in this submission)

Study Details

The provided document describes analytical performance studies and method comparison studies. There is no mention of clinical studies involving patients or a human-in-the-loop (MRMC) study. The device is an in-vitro diagnostic (IVD) device, and the focus is on its analytical accuracy against a gold standard lab method.

1. A table of acceptance criteria and the reported device performance:

   • See table above. The acceptance criteria are largely implied from the nature of the tests performed and the typical expectations for an IVD device. For example, for precision/reproducibility, the expectation is that samples significantly below the cutoff are consistently negative, and samples significantly above are consistently positive, with an acceptable range of uncertainty around the cutoff itself. For method comparison, a high percentage of agreement with the confirmatory method is expected.

2. Sample sizes used for the test set and the data provenance:

   • Precision-Reproducibility-Cut-Off: Samples were prepared at 8 concentrations (-100%, -75%, -50%, -25%, +25%, +50%, +75%, +100% of cutoff). For each concentration, tests were performed "two runs per day for 10 days per device lot." With 3 lots tested (for Oxycodone and Methadone), this means: 8 concentrations x 2 runs/day x 10 days x 3 lots = 480 samples per drug (e.g., 60 samples per concentration/lot).
   • Method Comparison Studies: "Total 932 samples" were tested across all drugs. For Oxycodone, the breakdown is: 152 drug-free, 47 less than half cutoff, 32 near cutoff negative, 29 near cutoff positive, 174 high positive = 434 samples. For Methadone, the breakdown is: 277 drug-free, 13 less than half cutoff, 20 near cutoff negative, 15 near cutoff positive, 173 high positive = 498 samples. (434 + 498 = 932 total, aligning with the stated total).
   • Data Provenance: The document does not explicitly state the country of origin. It indicates that the samples were "prepared by spiking drug in negative oral fluid samples" for precision studies. For method comparison, "Method comparison studies for the Oral fluid Drug Test were performed at eight testing sites with three operators at each site." This suggests retrospective sample collection and external lab testing for the LC/MS/MS ground truth, though the exact nature (e.g., banked samples, newly collected for the study) is not specified. The samples are human oral fluid.

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts:

   • The ground truth was established by Liquid Chromatography/Mass Spectrometry/Mass Spectrometry (LC-MS/MS), which is described as the "preferred confirmatory method." LC-MS/MS is an analytical chemistry technique, not typically performed by "experts" in the clinical sense (like radiologists). The "ground truth" is derived from the analytical measurement itself, likely performed by trained laboratory technicians or biochemists following established protocols. Therefore, the concept of "number of experts" or "qualifications" beyond standard lab accreditation is not applicable in the way it would be for image interpretation.

4. Adjudication method (e.g., 2+1, 3+1, none) for the test set:

   • No adjudication method described. The test is a qualitative assay (positive/negative based on visual line presence/absence), and the ground truth is a quantitative analytical measurement (LC-MS/MS). Discrepancies between the device result and the LC-MS/MS result are reported as discordant results, not subject to further "adjudication" by human experts in this context.

5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:

   • No, an MRMC comparative effectiveness study was not done. This is an in-vitro diagnostic (IVD) device, not an AI-powered image interpretation device. It is a rapid immunoassay for drug detection. The "readers" are the individuals visually interpreting the test result (presence or absence of a line), not medical professionals interpreting complex images. The study focuses on the analytical performance of the device itself against a gold standard lab method.

6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done:

   • This is a standalone diagnostic device. The performance data presented (precision, linearity, stability, interference, specificity, pH effect, drug recovery, and method comparison studies) characterize the standalone analytical performance of the device itself. The "human-in-the-loop" component is limited to the visual interpretation of the presence/absence of a line, which is a straightforward qualitative assessment, not a complex diagnostic decision.

7. The type of ground truth used (expert consensus, pathology, outcomes data, etc):

   • The ground truth used was quantitative analytical measurement via Liquid Chromatography/Mass Spectrometry/Mass Spectrometry (LC-MS/MS). For drug detection, this is typically considered the gold standard.

8. The sample size for the training set:

   • This document describes a premarket notification for an IVD device. The methods described here relate to validation/test set studies. The document does not mention a "training set" in the context of machine learning. Lateral flow immunoassays are developed and optimized through chemical and biological engineering, not by training a machine learning algorithm on a dataset.

9. How the ground truth for the training set was established:

   • As there is no mention of a "training set" or machine learning model in this document, this question is not applicable. The device's mechanism is based on immunochromatography (antigen-antibody reactions), not data-driven learning.

Ask a specific question about this device

First Sign® Drug of Abuse Cup Test Marijuana is a qualitative lateral flow immunoassay intended for the detection of Marijuana in human urine at cut-off concentration of 20 ng/mL. The tests provide only preliminary test results. A more specific alternative method must be used to obtain a confirmed analytical result. Gas Chromatography/Mass Spectrometry is the preferred confirmatory method. Clinical consideration and professional judgment should be used when you get any drug of abuse test result. It should be used particularly when the preliminary result is positive. For in vitro diagnostic use only. The test is intended for prescription use.

First Sign® Drug of Abuse Dip Card Test Marijuana is a qualitative lateral flow immunoassay intended for the detection of Marijuana in human urine at cut-off concentration of 20 ng/mL. The tests provide only preliminary test results. A more specific alternative method must be used to obtain a confirmed analytical result. Gas Chromatography/Mass Spectrometry is the preferred confirmatory method. Clinical consideration and professional judgment should be used when you get any drug of abuse test result. It should be used particularly when the preliminary result is positive. For in vitro diagnostic use only. The test is intended for prescription use.

First Sign Multi-Drug Cup Test is a qualitative lateral flow immunoassay intended for the detection of Amphetamine, Cocaine, and Methamphetamine in human urine at cut-off concentrations of 500 ng/mL, and 500 ng/mL, respectively. The tests provide only preliminary test results. A more specific alternative method must be used to obtain a confirmed analytical result. Gas Chromatography/Mass Spectrometry is the preferred confirmatory method. Clinical consideration and professional judgment should be used when you get any drug of abuse test result. It should be used particularly when the preliminary result is positive. For in vitro diagnostic use only. The tests are intended for over-the-counter use.

First Sign® Multi-Drug Dip Card Test is a qualitative lateral flow immunoassay intended for the detection of Amphetamine, Cocaine, and Methamphetamine in human urine at cut-off concentrations of 500 ng/mL, and 500 ng/mL, respectively. The tests provide only preliminary test results. A more specific alternative method must be used to obtain a confirmed analytical result. Gas Chromatography/Mass Spectrometry is the preferred confirmatory method. Clinical consideration and professional judgment should be used when you get any drug of abuse test result. It should be used particularly when the preliminary result is positive. For in vitro diagnostic use only. The tests are intended for over-the-counter use.

First Sign Multi-Drug Cup Test, First Sign Multi-Drug Dip Card Test, First Sign Drug of Abuse Cup Test Marijuana, and First Sign Drug of Abuse Dip Card Marijuana are lateral flow, immunochromatographic assays. The First Sign Multi-Drug Cup Test and the First Sign Multi-Drug Dip Card Test are for the qualitative detection of Amphetamine, Cocaine, and Methamphetamine in human urine. First Sign Drug of Abuse Cup Test Marijuana, and First Sign Drug of Abuse Dip Card Marijuana are for the qualitative detection of Marijuana in human urine. The products are single-use in vitro diagnostic devices, which come in the formats of Dip Cards or Cups. Each test kit contains a Test Device (in one of the two formats), a package insert and a urine cup for sample collection. Each test device is sealed with a desiccant in an aluminum pouch.

The provided document describes the analytical and user performance of various First Sign® drug tests (Multi-Drug Cup Test, Multi-Drug Dip Card Test, Drug of Abuse Cup Test Marijuana, and Drug of Abuse Dip Card Test Marijuana). These are qualitative lateral flow immunoassays intended for the detection of Amphetamine, Cocaine, Methamphetamine, and Marijuana in human urine. The document does not describe an AI medical device, but rather an in-vitro diagnostic device. Therefore, many of the requested points related to AI/MRMC studies are not applicable.

Here's an attempt to extract relevant information and apply it to the closest possible concepts for an AI device's acceptance criteria and study, based on the provided content:

Device Description:
The devices are lateral flow, immunochromatographic assays for the qualitative detection of Amphetamine, Cocaine, and Methamphetamine (Multi-Drug tests) and Marijuana (Drug of Abuse tests) in human urine. They are single-use in vitro diagnostic devices available in Dip Card or Cup formats.

Acceptance Criteria and Device Performance (Interpreted for a diagnostic device):

The acceptance criteria for these devices would typically be established based on their ability to correctly identify positive and negative samples around the defined cut-off concentrations for each drug. The "Precision" and "Lay-user study" data provide the closest insight into this, demonstrating the device's ability to consistently provide correct results across various concentrations relative to the cut-off.

Since this is a qualitative test (positive/negative), the performance is measured by the percentage of correct results at different concentrations, especially near the cut-off.

Table of Acceptance Criteria (Inferred) and Reported Device Performance:

Breakdown of Requested Information (Applying to this non-AI device where possible):

1. A table of acceptance criteria and the reported device performance:

   • Acceptance Criteria (Inferred): For a qualitative drug test, the acceptance criteria are generally that samples below the cut-off should test negative, and samples at or above the cut-off should test positive, with high accuracy at concentrations away from the cut-off, and acceptable accuracy around the cut-off. Specifically, the data shows an expectation of 100% correct results for samples significantly above or below the cut-off (e.g., +/- 50% to +/- 100%) and a tolerance for some misclassification directly at or very close to the cut-off.
   • Reported Device Performance: See the table above and the detailed "Precision" and "Lay-user study" results within the document (pages 8-20). For instance, in the precision study, samples at the cut-off typically showed a mix of positive and negative results (e.g., 3-/47+ or 2-/48+ out of 50 tests), indicating the expected performance around the threshold. In the lay-user study, performance at +25% cutoff for COC DipCard was 90% correct, and for AMP Cup and MET Cup at +25% cutoff, it was 95% correct, with near-perfect accuracy further from the cutoff.

2. Sample sizes used for the test set and the data provenance:

   • Precision Study:
     • Sample Size: For each drug and each format (e.g., AMP Cup, COC Dip Card), there were 9 concentration levels tested (from -100% to +100% of cut-off). For each concentration, there were 2 runs per day for 25 days, totalling 50 tests per concentration per lot. Since three lots were tested, this amounts to 150 tests per concentration level per drug per format.
     • Data Provenance: Samples were "prepared by spiking drug in negative urine samples." This suggests laboratory-controlled, prospectively prepared samples. The origin of the negative urine samples (e.g., country) is not specified.
   • Method Comparison Studies (Clinical Samples):
     • Sample Size: "80 (40 negative and 40 positive) unaltered clinical samples for each target drug" were used. This means 80 clinical samples per drug per format. (e.g., 80 for AMP Dip Card, 80 for AMP Cup, etc.).
     • Data Provenance: "clinical samples." This implies retrospective (or collected for study), human biological samples. Country of origin is not specified but implied to be in the testing region (likely USA given the FDA submission).
   • Lay-user Study:
     • Sample Size: "320 lay persons testing the devices." Urine samples were prepared at 8 concentration levels.
     • Data Provenance: "Urine samples were prepared... by spiking drugs into drug free-pooled urine specimens." This indicates laboratory-prepared, prospective samples.

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts:

   • Ground Truth Establishment:
     • For the Precision Study and Lay-user study, the ground truth was established by the precise concentrations of spiked drugs confirmed by GC/MS (Gas Chromatography/Mass Spectrometry). GC/MS is considered the "preferred confirmatory method" and the gold standard for drug detection and quantification in urine.
     • For the Method Comparison Studies, the ground truth for clinical samples was established by GC/MS results.
   • Experts and Qualifications: The document does not mention human experts establishing the ground truth for the test sets. The ground truth relies on the analytical accuracy of GC/MS. The "three different operators" (or "laboratory assistants") mentioned in the Precision and Method Comparison studies were performing the device tests, not establishing the ground truth. No specific qualifications for these operators are provided beyond their role.

4. Adjudication method (e.g., 2+1, 3+1, none) for the test set:

   • For the analytical and method comparison studies, the results of the device were compared directly to the GC/MS ground truth. There was no human adjudication process described for discrepancies in the test results themselves, as the GC/MS is considered definitive. The "discordant results" tables show instances where the viewer disagreed with GC/MS, but there's no mention of a re-adjudication of the GC/MS reference result itself.

5. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:

   • N/A. This document describes an in-vitro diagnostic device (drug test kit), not an AI medical device. Therefore, no MRMC study involving AI assistance for human readers was conducted or is applicable.

6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done:

   • N/A. This is not an AI algorithm. The device itself is a standalone test kit that provides a visual qualitative result (lines appearing on the dip card/cup). The "lay-user study" demonstrates the performance of the device when interpreted by its intended users (without "human-in-the-loop performance" in the AI sense, but rather human-as-user interpretation). The "Method Comparison Studies" also represent standalone performance, where "viewers" (operators) interpret the device results against a gold standard.

7. The type of ground truth used (expert consensus, pathology, outcomes data, etc.):

   • The primary ground truth used for all performance studies was Gas Chromatography/Mass Spectrometry (GC/MS) results, which is an analytical gold standard for quantifying drug concentrations in urine.

8. The sample size for the training set:

   • N/A. This is not a machine learning/AI device, so there is no "training set" in the computational sense. The device's performance is based on its chemical and biological components, not trained data.

9. How the ground truth for the training set was established:

   • N/A. As there is no training set for an AI model, this question is not applicable.

Ask a specific question about this device

Healgen Amphetamine Test is an immunochromatographic assay for the qualitative determination of Amphetamine in human urine at a Cut-Off concentration of 500 ng/mL. The test is available in a Strip format, a Dip Card format and a Cup format.

The test provides only preliminary test results. A more specific alternative chemical method must be used in order to obtain a confirmed analytical result. GC/MS is the preferred confirmatory method. Clinical consideration and professional judgment should be exercised with any drug of abuse test result, particularly when the preliminary result is positive. For in vitro diagnostic use only. It is intended for prescription and for over-the-counter use.

Healgen Cocaine Test is an immunochromatographic assay for the qualitative determination of Benzoylecgonine in human urine at a Cut-Off concentration of 150 ng/mL. The test is available in a Strip format, a Dip Card format and a Cup format.

The test provides only preliminary test results. A more specific alternative chemical method must be used in order to obtain a confirmed analytical result. GC/MS is the preferred confirmatory method. Clinical consideration and professional judgment should be exercised with any drug of abuse test result, particularly when the preliminary result is positive. For in vitro diagnostic use only. It is intended for prescription and for over-the-counter use.

Healgen Methamphetamine Test is an immunochromatographic assay for the qualitative determination of Methamphetamine in human urine at a Cut-Off concentration of 500 ng/mL. The test is available in a Strip format, a Cassette format, a Dip Card format and a Cup format.

The test provides only preliminary test results. A more specific alternative chemical method must be used in order to obtain a confirmed analytical result. GC/MS is the preferred confirmatory method. Clinical consideration and professional judgment should be exercised with any drug of abuse test result, particularly when the preliminary result is positive. For in vitro diagnostic use only. It is intended for prescription and for over-the-counter use.

Healgen Amphetamine Test, Healgen Cocaine Test and Healgen Methamphetamine Test are immunochromatographic assays for Amphetamine, Cocaine and Methamphetamine. Each assay test is a lateral flow system for the qualitative detection of Amphetamine, Benzoylecgonine and Methamphetamine (target analyte) in human urine. The products are in vitro diagnostic devices, which come in the form of: Strips, Cassettes, DipCards, or Cups. Each product contains a Test Device (in one of the four formats), and a package insert. Each test device is sealed with a desiccant in an aluminum pouch.

The document describes several tests conducted to prove the performance of the Healgen Amphetamine Test, Healgen Cocaine Test, and Healgen Methamphetamine Test devices.

Here's a breakdown of the requested information:

1. A table of acceptance criteria and the reported device performance

The document does not explicitly state formal "acceptance criteria" through a table or specific thresholds that define success. Instead, it presents performance characteristic studies (precision, cut-off verification, interference, specificity, and comparison to GC/MS) and a lay-user study.

Based on the presented data, the implicit acceptance criteria appear to be:

• Precision: Consistent results across different lots, operators, and concentration levels, especially for samples near the cut-off.
• Cut-off Verification: Accurate classification of samples at or near the specified cut-off concentrations for each drug.
• Interference: No false positive or false negative results due to common interfering substances in urine.
• Specificity: Low or no cross-reactivity with structurally similar but non-target compounds at expected physiological concentrations.
• Method Comparison (with GC/MS): High concordance with GC/MS results, especially for clearly negative and clearly positive samples, with reasonable performance around the cut-off.
• Lay-user usability: High percentage of correct results by lay users and clear instructions for use.

Here's a summary table of the reported device performance for key aspects:

2. Sample size used for the test set and the data provenance

• Precision Study (Test Set):
  • Sample Size: For each drug (Amphetamine, Cocaine, Methamphetamine) and each device format (Strip, Cassette, Cup, Dip Card), there were 8 concentration levels tested. For each concentration, tests were performed 2 runs per day for 25 days by 3 different operators, for 3 different lots of devices. This means for one drug and one format: 8 concentrations * 50 results (2 runs/day * 25 days) * 3 lots = 1200 individual tests.
  • Data Provenance: Samples were "negative samples" spiked with drugs to specific concentrations. The drug concentrations were confirmed by GC/MS. The samples were "blind labeled and randomized" by a person not involved in testing. This indicates retrospective preparation of controlled samples, likely performed in a lab setting, not from actual patient samples from a specific country.
• Cut-off Verification Study (Test Set):
  • Sample Size: A total of 150 samples (equally distributed at -50%, -25%, cut-off, +25%, +50% cut-off) were tested. Given 5 concentration levels, this suggests 30 samples per concentration. These were tested using 3 different lots of each device by 3 different operators. So, 150 samples * 3 lots * 3 operators = 1350 individual tests per drug and format.
  • Data Provenance: Samples were prepared by spiking drug in "negative samples." Concentrations confirmed by GC/MS. This indicates retrospective preparation of controlled samples.
• Interference Study (Test Set):
  • Sample Size: Not explicitly stated as a total number of samples, but "Potential interfering substances" were tested by adding them to drug-free urine and target drug urine (+25% cut-off samples). These were tested using three batches of each device for all formats.
  • Data Provenance: Controlled laboratory-prepared urine samples with specified interfering substances and drug concentrations.
• Specificity Study (Test Set):
  • Sample Size: Not explicitly stated as a total number of samples, but drug metabolites and other components were tested using three batches of each device for all formats.
  • Data Provenance: Controlled laboratory-prepared urine samples with specific drug metabolites or related compounds at various concentrations.
• Method Comparison Study (Test Set):
  • Sample Size: For each drug (Amphetamine, Cocaine, Methamphetamine) and each device format, 80 unaltered clinical samples were used (40 negative and 40 positive). These were tested independently by 3 different laboratory assistants (viewers). For one drug and one format: 80 samples * 3 viewers = 240 individual tests.
  • Data Provenance: "unaltered clinical samples." The origin (e.g., country) is not specified. It is reasonable to assume these were retrospective clinical samples as they were "blind labeled."
• Lay-user Study (Test Set):
  • Sample Size: 560 lay persons tested each of the Amphetamine, Cocaine, and Methamphetamine devices, resulting in a total of 1680 individuals who performed the study. Each participant received 1 blind labeled sample and a device.
  • Data Provenance: Urine samples were prepared by spiking drugs into "drug free-pooled urine specimens" to specific concentrations. Concentrations were confirmed by GC/MS. These were retrospectively prepared controlled samples, not actual patient samples. The study participants were "diverse educational and professional backgrounds and ranged in age from 21 to > 50 years."

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts

• Precision, Cut-off, Interference, Specificity, and Lay-user Studies: The ground truth for these studies was established by GC/MS (Gas Chromatography/Mass Spectrometry), which is a gold standard analytical method for drug concentration determination. The individuals performing the GC/MS analysis are typically trained analytical chemists or laboratory technicians, but their specific qualifications are not detailed in the document.
• Method Comparison Study: The ground truth for the 80 unaltered clinical samples was established by GC/MS. As above, specific qualifications of the GC/MS operators are not provided. The device results were then compared to these GC/MS ground truth results.

4. Adjudication method for the test set

• Precision, Cut-off, Interference, Specificity, and Lay-user Studies: No explicit adjudication method is mentioned for the ground truth. The ground truth was set by scientific preparation and GC/MS confirmation of samples. For the device results, multiple operators conducted the tests (3 operators for precision, cut-off; 3 batches for interference/specificity), but it's not stated that their results were adjudicated against each other; they are presented individually or as aggregate performance.
• Method Comparison Study: The document lists results for Viewer A, Viewer B, and Viewer C separately. There is no explicit mention of an adjudication method for comparing interpretations between these viewers or for resolving discordant results by the device itself. The device results from each viewer are directly compared to the GC/MS ground truth.

5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance

• No, an MRMC comparative effectiveness study was NOT done. This document describes the performance of in-vitro diagnostic test kits (dipstick, cassette, cup formats) for qualitative drug detection by visual interpretation. These are standalone devices, not AI-assisted reading systems.
• Therefore, there is no effect size reported for human readers improving with AI vs. without AI assistance.

6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done

• Yes, in essence, the "Analytical Performance" section (Precision, Cut-off, Interference, Specificity) and elements of the "Method Comparison Studies" (comparison to GC/MS ground truth) represent standalone performance of the device itself (the immunochromatographic assay reaction and visual line formation), without explicit human 'interpretation' as the primary variable. The visual reading done by operators/viewers in the precision and method comparison studies still involves human interpretation of the color lines.
• However, the Lay-user study specifically evaluated human-in-the-loop performance (lay users interpreting the device according to instructions). Other sections primarily focus on the inherent analytical performance of the test strips/cassettes/cups.

7. The type of ground truth used (expert consensus, pathology, outcomes data, etc.)

• The primary ground truth used throughout these studies is GC/MS (Gas Chromatography/Mass Spectrometry), which is an objective chemical analytical method considered the gold standard for confirming drug concentrations in toxicology.
• For the lay user study, the ground truth was also established by GC/MS for the prepared spiked samples.

8. The sample size for the training set

• The document describes premarket notification studies for an in-vitro diagnostic device. It does not mention a "training set" in the context of machine learning or AI models.
• The device is a laboratory assay; it does not involve algorithms that require training data. The "training" in this context would refer to the development and optimization of the chemical reagents and manufacturing process, which are not detailed as a "training set" in a data science sense.

9. How the ground truth for the training set was established

• N/A, as there is no "training set" in the context of data for an AI/ML algorithm.

Ask a specific question about this device