Search Results

First Sign™ Drug of Abuse Tests are immunochromatographic assays for the qualitative determination of Methadone, Phencyclidine, and Oxycodone in human urine at cut-off concentrations of 300 ng/mL, 25 ng/mL, and 100 ng/mL, respectively. The tests are available in a Cup format and a Dip Card format.

The tests may yield preliminary positive results even when prescription drugs Methadone and Oxycodone are ingested, at prescribed doses; it is not intended to distinguish between prescription use or abuse of this drug. There are no uniformly recognized cutoff concentration levels for Methadone and Oxycodone in urine. The tests provide only preliminary test results. A more specific alternative chemical method must be used in order to obtain a confirmed analytical result. Gas Chromatography/Mass Spectrometry is the preferred confirmatory method. Clinical consideration and professional judgment should be exercised with any drug of abuse test result, particularly when the preliminary result is positive.

For in vitro diagnostic use only. The tests are intended for over-the-counter and for prescription use.

First Sign™ Drug of Abuse Tests are immunochromatographic assays. Each assay test is a lateral flow system for the qualitative detection of Methadone, Phencyclidine, and Oxycodone in human urine. The products are single-use in vitro diagnostic devices, which come in the formats of DipCards or Cups. Each test kit contains a Test Device (in one of the two formats), a package insert and a urine cup for sample collection. Each test device is sealed with a desiccant in an aluminum pouch.

The provided text describes the performance characteristics and studies for the "First Sign® Drug of Abuse Cup Test" and "First Sign® Drug of Abuse Dip Card Test" for Methadone, Phencyclidine, and Oxycodone.

The acceptance criteria for each drug are implicitly defined by the reported performance, specifically in the precision, cut-off verification, and comparison studies. For instance, in the precision study, at -100% to -25% of the cut-off, all results were expected to be negative, and at +25% to +100% of the cut-off, all results were expected to be positive, with some allowance for variation at the exact cut-off concentration.

Here's a breakdown of the requested information:

1. A table of acceptance criteria and the reported device performance

The document does not explicitly state acceptance criteria in a separate table. However, the performance tables demonstrate the device's adherence to the expected behavior around the cut-off concentrations. The "Precision" section's implied criteria are 100% correct negatives for concentrations below -25% of the cut-off, 100% correct positives for concentrations above +25% of the cut-off, and a high percentage of correct results at +/-25% and at the cut-off values. The "Cut-off" section verifies the stated cut-off values.

Table: Acceptance Criteria (Implied from Precision and Cut-off Studies) and Reported Device Performance

2. Sample sized used for the test set and the data provenance (e.g. country of origin of the data, retrospective or prospective)

• Precision Study:
  • Sample Size: For each drug and each format (Dip Card/Cup), 8 concentrations were tested across 3 lots, with samples run 2 times per day for 25 days by 3 different operators. This is not a simple count of unique clinical samples. It uses spiked samples. Each individual concentration point (e.g., -100% cut-off) had 50 measurements (2 tests/day * 25 days by each of 3 lots, but the values are given as 50-/0+ or 50+/0- for each lot, suggesting 50 replicates per lot at each concentration).
  • Data Provenance: The samples were "prepared by spiking drug in negative samples." The document does not specify the country of origin, but the testing was conducted "in-house." These are prospective, laboratory-prepared samples.
• Cut-off Verification Study:
  • Sample Size: 150 samples ("equally distributed at concentrations of -50% cut-off; cut-off; +25% cut-off; +50% cut-off"). These were tested using three different lots of each device by three different operators.
  • Data Provenance: Similar to precision study, these were laboratory-prepared samples ("spiked drug in negative samples"). No country of origin is specified.
• Comparison Studies (Clinical Samples):
  • Sample Size: 80 "unaltered clinical samples" for each drug (40 negative and 40 positive). This means 80 samples for Methadone, 80 for Phencyclidine, and 80 for Oxycodone, for both Dip Card and Cup formats (though the tables suggest the same samples were likely used for both formats for a given viewer).
  • Data Provenance: "in-house" studies, using "unaltered clinical samples." The origin of these clinical samples is not specified (e.g., country, retrospective/prospective).
• Lay-user Study:
  • Sample Size: 280 lay persons for Methadone, 280 for Phencyclidine, and 280 for Oxycodone devices. Each participant received 1 blind-labeled sample. The samples themselves were prepared at 7 different concentrations, with 20 samples per concentration. This means a total of 140 samples for each drug (7 concentrations * 20 samples/concentration) were tested by the lay-users.
  • Data Provenance: "Urine samples were prepared... by spiking drugs into drug free-pooled urine specimens." The study was performed "at three intended user sites." No country of origin is specified. These are prospective, laboratory-prepared samples.

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts (e.g. radiologist with 10 years of experience)

• Precision, Cut-off, and Lay-user Studies (Spiked Samples): The ground truth was established by the precise spiking of drugs into negative urine samples at known concentrations, confirmed by GC/MS (Gas Chromatography/Mass Spectrometry). This is an analytical chemistry method, and implicitly, the expertise lies in the laboratory personnel conducting these precise preparations and GC/MS confirmations. No further expert qualifications are provided.
• Comparison Studies (Clinical Samples): The ground truth was established by GC/MS results for each of the 80 unaltered clinical samples per drug. No details on the number or qualifications of the GC/MS experts are provided, but GC/MS is the "preferred confirmatory method" for drug of abuse tests.

4. Adjudication method (e.g. 2+1, 3+1, none) for the test set

• Precision Study: "All sample aliquots were blind-labeled and randomized by the person who prepared samples and did not take part in the sample testing." Tests were performed by "three different operators." The results are aggregated, but no specific adjudication method among operators is explicitly described. It appears each operator's results were simply recorded.
• Cut-off Verification Study: Tested by "three different operators." Similar to precision study, results are aggregated, no specific adjudication is mentioned.
• Comparison Studies: "three different laboratory assistants" ("Viewer A, B, C") tested the samples. The tables show individual viewer results, followed by a list of "Discordant Results." This implies that there wasn't a formal adjudication method (like 2+1 or 3+1) in place to yield a single "device" result, but rather a comparison of each viewer's interpretation against the GC/MS ground truth.
• Lay-user Study: Each "participant was provided with the package insert, 1 blind labeled sample and a device." The results appear to be individual lay person interpretations compared to the GC/MS ground truth, with no inter-lay user adjudication.

5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance

• No, an MRMC comparative effectiveness study involving AI assistance was not done.
• The studies involved human readers (laboratory assistants, lay-users) interpreting the device results, and these devices are immunochromatographic assays (lateral flow tests), not AI software. Therefore, there is no "AI vs without AI assistance" component to these studies.

6. If a standalone (i.e. algorithm only without human-in-the loop performance) was done

• No, a standalone algorithm-only performance study was not done. The devices are physical tests interpreted by humans (either trained laboratory personnel or lay-users). They do not involve an algorithm separate from human interpretation.

7. The type of ground truth used (expert concensus, pathology, outcomes data, etc)

• For all studies (Precision, Cut-off, Comparison, Lay-user), the ground truth was established by Gas Chromatography/Mass Spectrometry (GC/MS) of the spiked or clinical urine samples. GC/MS is a highly accurate analytical method considered the gold standard for confirming drug concentrations.

8. The sample size for the training set

• The document implies that these studies (precision, cut-off, comparison, lay-user) represent the validation of the device. It does not mention a separate "training set" in the context of machine learning or algorithm development, as this medical device is an immunoassay, not an AI/ML product. The development process for such a test would involve internal R&D and optimization, but the specific term "training set" as used in AI/ML is not applicable here.

9. How the ground truth for the training set was established

• As a traditional immunoassay device, there is no "training set" in the AI/ML sense. The ground truth for the analytical and clinical performance studies was established using GC/MS of urine samples with known (spiked) or confirmed (clinical) drug concentrations.

Ask a specific question about this device

First Sign™ Drug of Abuse Tests are immunochromatographic assays for the qualitative determination of Oxazepam , Methamphetamine, and Morphine in human urine at cut-off concentrations of 300 ng/mL, and 2000 ng/mL, respectively. The tests are available in a Cup format and a Dip Card format.

The tests may yield preliminary positive results even when prescription drug Oxazepam is ingested, at prescribed doses; it is not intended to distinguish between prescription use or abuse of this drug. There is no uniformly recognized cutoff concentration level for oxazepam in urine. The tests provide only preliminary test results. A more specific alternative chemical method must be used in order to obtain a confirmed analytical result. Gas Chromatography/Mass Spectrometry is the preferred confirmatory method. Clinical consideration and professional judgment should be exercised with any drug of abuse test result, particularly when the preliminary result is positive.

For in vitro diagnostic use only. The tests are intended for over-the-counter and for prescription use.

First Sign™ Drug of Abuse Tests are immunochromatographic assays. Each assay test is a lateral flow system for the qualitative detection of Oxazepam , Methamphetamine , and Morphine in human urine. The products are single-use in vitro diagnostic devices, which come in the formats of DipCards or Cups. Each test kit contains a Test Device (in one of the two formats), a package insert and a urine cup for sample collection. Each test device is sealed with a desiccant in an aluminum pouch.

Here's an analysis of the acceptance criteria and study detailed in the provided text:

1. Table of Acceptance Criteria and Reported Device Performance

The document does not explicitly state formal "acceptance criteria" in terms of specific quantitative benchmarks (e.g., "sensitivity must be >90%"). Instead, it describes performance characteristics and then presents the results of studies to demonstrate that the device performs acceptably. The implied acceptance criterion for these qualitative drug tests is that they generally agree with GC/MS results, especially at and significantly above/below cut-off values.

Based on the provided performance characteristics, here's a summary:

2. Sample Size and Data Provenance (Test Set)

• Sample Size (Trained Professionals, Method Comparison):
  • For each of the three drugs (Oxazepam, Methamphetamine, Morphine) and each format (Dip Card, Cup), 80 clinical samples were used.
  • Total samples for method comparison: 3 drugs * 2 formats * 80 samples/drug/format = 480 samples.
  • Breakdown of 80 samples: 10 Negative, 10 Low Negative, 20 Near Cutoff Negative, 15 Near Cutoff Positive, 25 High Positive.
• Sample Size (Lay User Study):
  • 280 lay persons for Oxazepam devices.
  • 280 lay persons for Methamphetamine devices.
  • 280 lay persons for Morphine devices.
  • Each lay person tested 1 blind labeled sample and a device.
  • Total samples tested by lay users: Roughly 280 samples/drug * 3 drugs = 840 samples.
• Data Provenance (Method Comparison): "unaltered clinical samples" - implies retrospective collection, origin unknown based on the text.
• Data Provenance (Lay User Study): "Urine samples were prepared... by spiking drugs into drug free-pooled urine specimens." This indicates the samples were synthesized or spiked rather than naturally occurring clinical samples, and then blind-labeled.

3. Number of Experts and Qualifications for Ground Truth (Test Set)

• Number of Experts:
  • Method Comparison (Professional User): "three different laboratory assistants for each format of the device." Total of 6 unique readers (3 for dip card, 3 for cup) if they were distinct, or 3 if the same 3 read both formats (the wording "Different set of operators tested each format" in the precision study suggests distinct operators, but here it states "for each format", which could mean the same set of 3 rotated). The data is presented as Viewer A, B, C for each.
  • Lay User Study: 280 lay persons per drug. Not "experts" in the traditional sense, but the intended users.
• Qualifications of Experts: The "three different laboratory assistants" are not further qualified (e.g., radiologist with 10 years of experience).

4. Adjudication Method (Test Set)

• None specified for the professional user readings. The raw results from each "Viewer" (laboratory assistant) are presented individually, and then compared against the GC/MS result. They are not pooled or adjudicated to form a single "device" result.
• For the Lay User Study: There is no "adjudication" between lay users. Each lay user's individual result is recorded and compared to the GC/MS confirmed concentration.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

• No MRMC comparative effectiveness study was done to measure human reader improvement with AI vs. without AI assistance. This device is a standalone qualitative diagnostic test (immunochromatographic assay), not an AI-assisted interpretation tool for human readers.

6. Standalone Performance (Algorithm Only)

• Yes, a standalone performance was done, but it's not an "algorithm" in the typical AI sense. The device itself (the immunochromatographic assay) is designed to provide a result (positive/negative) based on a chemical reaction, which can then be visually interpreted. The precision studies, cut-off studies, interference, specificity, and specific gravity/pH studies all demonstrate the standalone performance of the device without explicit human interpretation variability considered (though a human reads the test line).
• The "Method Comparison" and "Lay-user study" then introduce the human element (laboratory assistants and lay users, respectively) reading these standalone device results.

7. Type of Ground Truth Used (Test Set)

• Gas Chromatography/Mass Spectrometry (GC/MS) was explicitly used as the preferred confirmatory method for establishing ground truth for both the professional method comparison study and for confirming the concentrations of the spiked samples in the lay user study and precision/cut-off studies.

8. Sample Size for the Training Set

• The document describes performance characteristics and equivalence to a predicate device, but does not specify a separate "training set" for the development of the device itself or for any AI/algorithmic component (as there isn't one). The studies described are performance validation studies.

9. How the Ground Truth for the Training Set was Established

• Not applicable as no "training set" is mentioned in the context of device development. The ground truth for all performance evaluation studies (precision, cut-off, method comparison, lay user) was established using GC/MS.

Ask a specific question about this device