Search Results

First Sign™ Drug of Abuse Tests are immunochromatographic assays for the qualitative determination of Methadone, Phencyclidine, and Oxycodone in human urine at cut-off concentrations of 300 ng/mL, 25 ng/mL, and 100 ng/mL, respectively. The tests are available in a Cup format and a Dip Card format.

The tests may yield preliminary positive results even when prescription drugs Methadone and Oxycodone are ingested, at prescribed doses; it is not intended to distinguish between prescription use or abuse of this drug. There are no uniformly recognized cutoff concentration levels for Methadone and Oxycodone in urine. The tests provide only preliminary test results. A more specific alternative chemical method must be used in order to obtain a confirmed analytical result. Gas Chromatography/Mass Spectrometry is the preferred confirmatory method. Clinical consideration and professional judgment should be exercised with any drug of abuse test result, particularly when the preliminary result is positive.

For in vitro diagnostic use only. The tests are intended for over-the-counter and for prescription use.

First Sign™ Drug of Abuse Tests are immunochromatographic assays. Each assay test is a lateral flow system for the qualitative detection of Methadone, Phencyclidine, and Oxycodone in human urine. The products are single-use in vitro diagnostic devices, which come in the formats of DipCards or Cups. Each test kit contains a Test Device (in one of the two formats), a package insert and a urine cup for sample collection. Each test device is sealed with a desiccant in an aluminum pouch.

The provided text describes the performance characteristics and studies for the "First Sign® Drug of Abuse Cup Test" and "First Sign® Drug of Abuse Dip Card Test" for Methadone, Phencyclidine, and Oxycodone.

The acceptance criteria for each drug are implicitly defined by the reported performance, specifically in the precision, cut-off verification, and comparison studies. For instance, in the precision study, at -100% to -25% of the cut-off, all results were expected to be negative, and at +25% to +100% of the cut-off, all results were expected to be positive, with some allowance for variation at the exact cut-off concentration.

Here's a breakdown of the requested information:

1. A table of acceptance criteria and the reported device performance

The document does not explicitly state acceptance criteria in a separate table. However, the performance tables demonstrate the device's adherence to the expected behavior around the cut-off concentrations. The "Precision" section's implied criteria are 100% correct negatives for concentrations below -25% of the cut-off, 100% correct positives for concentrations above +25% of the cut-off, and a high percentage of correct results at +/-25% and at the cut-off values. The "Cut-off" section verifies the stated cut-off values.

Table: Acceptance Criteria (Implied from Precision and Cut-off Studies) and Reported Device Performance

2. Sample sized used for the test set and the data provenance (e.g. country of origin of the data, retrospective or prospective)

• Precision Study:
  • Sample Size: For each drug and each format (Dip Card/Cup), 8 concentrations were tested across 3 lots, with samples run 2 times per day for 25 days by 3 different operators. This is not a simple count of unique clinical samples. It uses spiked samples. Each individual concentration point (e.g., -100% cut-off) had 50 measurements (2 tests/day * 25 days by each of 3 lots, but the values are given as 50-/0+ or 50+/0- for each lot, suggesting 50 replicates per lot at each concentration).
  • Data Provenance: The samples were "prepared by spiking drug in negative samples." The document does not specify the country of origin, but the testing was conducted "in-house." These are prospective, laboratory-prepared samples.
• Cut-off Verification Study:
  • Sample Size: 150 samples ("equally distributed at concentrations of -50% cut-off; cut-off; +25% cut-off; +50% cut-off"). These were tested using three different lots of each device by three different operators.
  • Data Provenance: Similar to precision study, these were laboratory-prepared samples ("spiked drug in negative samples"). No country of origin is specified.
• Comparison Studies (Clinical Samples):
  • Sample Size: 80 "unaltered clinical samples" for each drug (40 negative and 40 positive). This means 80 samples for Methadone, 80 for Phencyclidine, and 80 for Oxycodone, for both Dip Card and Cup formats (though the tables suggest the same samples were likely used for both formats for a given viewer).
  • Data Provenance: "in-house" studies, using "unaltered clinical samples." The origin of these clinical samples is not specified (e.g., country, retrospective/prospective).
• Lay-user Study:
  • Sample Size: 280 lay persons for Methadone, 280 for Phencyclidine, and 280 for Oxycodone devices. Each participant received 1 blind-labeled sample. The samples themselves were prepared at 7 different concentrations, with 20 samples per concentration. This means a total of 140 samples for each drug (7 concentrations * 20 samples/concentration) were tested by the lay-users.
  • Data Provenance: "Urine samples were prepared... by spiking drugs into drug free-pooled urine specimens." The study was performed "at three intended user sites." No country of origin is specified. These are prospective, laboratory-prepared samples.

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts (e.g. radiologist with 10 years of experience)

• Precision, Cut-off, and Lay-user Studies (Spiked Samples): The ground truth was established by the precise spiking of drugs into negative urine samples at known concentrations, confirmed by GC/MS (Gas Chromatography/Mass Spectrometry). This is an analytical chemistry method, and implicitly, the expertise lies in the laboratory personnel conducting these precise preparations and GC/MS confirmations. No further expert qualifications are provided.
• Comparison Studies (Clinical Samples): The ground truth was established by GC/MS results for each of the 80 unaltered clinical samples per drug. No details on the number or qualifications of the GC/MS experts are provided, but GC/MS is the "preferred confirmatory method" for drug of abuse tests.

4. Adjudication method (e.g. 2+1, 3+1, none) for the test set

• Precision Study: "All sample aliquots were blind-labeled and randomized by the person who prepared samples and did not take part in the sample testing." Tests were performed by "three different operators." The results are aggregated, but no specific adjudication method among operators is explicitly described. It appears each operator's results were simply recorded.
• Cut-off Verification Study: Tested by "three different operators." Similar to precision study, results are aggregated, no specific adjudication is mentioned.
• Comparison Studies: "three different laboratory assistants" ("Viewer A, B, C") tested the samples. The tables show individual viewer results, followed by a list of "Discordant Results." This implies that there wasn't a formal adjudication method (like 2+1 or 3+1) in place to yield a single "device" result, but rather a comparison of each viewer's interpretation against the GC/MS ground truth.
• Lay-user Study: Each "participant was provided with the package insert, 1 blind labeled sample and a device." The results appear to be individual lay person interpretations compared to the GC/MS ground truth, with no inter-lay user adjudication.

5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance

• No, an MRMC comparative effectiveness study involving AI assistance was not done.
• The studies involved human readers (laboratory assistants, lay-users) interpreting the device results, and these devices are immunochromatographic assays (lateral flow tests), not AI software. Therefore, there is no "AI vs without AI assistance" component to these studies.

6. If a standalone (i.e. algorithm only without human-in-the loop performance) was done

• No, a standalone algorithm-only performance study was not done. The devices are physical tests interpreted by humans (either trained laboratory personnel or lay-users). They do not involve an algorithm separate from human interpretation.

7. The type of ground truth used (expert concensus, pathology, outcomes data, etc)

• For all studies (Precision, Cut-off, Comparison, Lay-user), the ground truth was established by Gas Chromatography/Mass Spectrometry (GC/MS) of the spiked or clinical urine samples. GC/MS is a highly accurate analytical method considered the gold standard for confirming drug concentrations.

8. The sample size for the training set

• The document implies that these studies (precision, cut-off, comparison, lay-user) represent the validation of the device. It does not mention a separate "training set" in the context of machine learning or algorithm development, as this medical device is an immunoassay, not an AI/ML product. The development process for such a test would involve internal R&D and optimization, but the specific term "training set" as used in AI/ML is not applicable here.

9. How the ground truth for the training set was established

• As a traditional immunoassay device, there is no "training set" in the AI/ML sense. The ground truth for the analytical and clinical performance studies was established using GC/MS of urine samples with known (spiked) or confirmed (clinical) drug concentrations.

Ask a specific question about this device

First Sign™ Drug of Abuse Tests are immunochromatographic assays for the qualitative determination of Oxazepam , Methamphetamine, and Morphine in human urine at cut-off concentrations of 300 ng/mL, and 2000 ng/mL, respectively. The tests are available in a Cup format and a Dip Card format.

The tests may yield preliminary positive results even when prescription drug Oxazepam is ingested, at prescribed doses; it is not intended to distinguish between prescription use or abuse of this drug. There is no uniformly recognized cutoff concentration level for oxazepam in urine. The tests provide only preliminary test results. A more specific alternative chemical method must be used in order to obtain a confirmed analytical result. Gas Chromatography/Mass Spectrometry is the preferred confirmatory method. Clinical consideration and professional judgment should be exercised with any drug of abuse test result, particularly when the preliminary result is positive.

For in vitro diagnostic use only. The tests are intended for over-the-counter and for prescription use.

First Sign™ Drug of Abuse Tests are immunochromatographic assays. Each assay test is a lateral flow system for the qualitative detection of Oxazepam , Methamphetamine , and Morphine in human urine. The products are single-use in vitro diagnostic devices, which come in the formats of DipCards or Cups. Each test kit contains a Test Device (in one of the two formats), a package insert and a urine cup for sample collection. Each test device is sealed with a desiccant in an aluminum pouch.

Here's an analysis of the acceptance criteria and study detailed in the provided text:

1. Table of Acceptance Criteria and Reported Device Performance

The document does not explicitly state formal "acceptance criteria" in terms of specific quantitative benchmarks (e.g., "sensitivity must be >90%"). Instead, it describes performance characteristics and then presents the results of studies to demonstrate that the device performs acceptably. The implied acceptance criterion for these qualitative drug tests is that they generally agree with GC/MS results, especially at and significantly above/below cut-off values.

Based on the provided performance characteristics, here's a summary:

2. Sample Size and Data Provenance (Test Set)

• Sample Size (Trained Professionals, Method Comparison):
  • For each of the three drugs (Oxazepam, Methamphetamine, Morphine) and each format (Dip Card, Cup), 80 clinical samples were used.
  • Total samples for method comparison: 3 drugs * 2 formats * 80 samples/drug/format = 480 samples.
  • Breakdown of 80 samples: 10 Negative, 10 Low Negative, 20 Near Cutoff Negative, 15 Near Cutoff Positive, 25 High Positive.
• Sample Size (Lay User Study):
  • 280 lay persons for Oxazepam devices.
  • 280 lay persons for Methamphetamine devices.
  • 280 lay persons for Morphine devices.
  • Each lay person tested 1 blind labeled sample and a device.
  • Total samples tested by lay users: Roughly 280 samples/drug * 3 drugs = 840 samples.
• Data Provenance (Method Comparison): "unaltered clinical samples" - implies retrospective collection, origin unknown based on the text.
• Data Provenance (Lay User Study): "Urine samples were prepared... by spiking drugs into drug free-pooled urine specimens." This indicates the samples were synthesized or spiked rather than naturally occurring clinical samples, and then blind-labeled.

3. Number of Experts and Qualifications for Ground Truth (Test Set)

• Number of Experts:
  • Method Comparison (Professional User): "three different laboratory assistants for each format of the device." Total of 6 unique readers (3 for dip card, 3 for cup) if they were distinct, or 3 if the same 3 read both formats (the wording "Different set of operators tested each format" in the precision study suggests distinct operators, but here it states "for each format", which could mean the same set of 3 rotated). The data is presented as Viewer A, B, C for each.
  • Lay User Study: 280 lay persons per drug. Not "experts" in the traditional sense, but the intended users.
• Qualifications of Experts: The "three different laboratory assistants" are not further qualified (e.g., radiologist with 10 years of experience).

4. Adjudication Method (Test Set)

• None specified for the professional user readings. The raw results from each "Viewer" (laboratory assistant) are presented individually, and then compared against the GC/MS result. They are not pooled or adjudicated to form a single "device" result.
• For the Lay User Study: There is no "adjudication" between lay users. Each lay user's individual result is recorded and compared to the GC/MS confirmed concentration.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

• No MRMC comparative effectiveness study was done to measure human reader improvement with AI vs. without AI assistance. This device is a standalone qualitative diagnostic test (immunochromatographic assay), not an AI-assisted interpretation tool for human readers.

6. Standalone Performance (Algorithm Only)

• Yes, a standalone performance was done, but it's not an "algorithm" in the typical AI sense. The device itself (the immunochromatographic assay) is designed to provide a result (positive/negative) based on a chemical reaction, which can then be visually interpreted. The precision studies, cut-off studies, interference, specificity, and specific gravity/pH studies all demonstrate the standalone performance of the device without explicit human interpretation variability considered (though a human reads the test line).
• The "Method Comparison" and "Lay-user study" then introduce the human element (laboratory assistants and lay users, respectively) reading these standalone device results.

7. Type of Ground Truth Used (Test Set)

• Gas Chromatography/Mass Spectrometry (GC/MS) was explicitly used as the preferred confirmatory method for establishing ground truth for both the professional method comparison study and for confirming the concentrations of the spiked samples in the lay user study and precision/cut-off studies.

8. Sample Size for the Training Set

• The document describes performance characteristics and equivalence to a predicate device, but does not specify a separate "training set" for the development of the device itself or for any AI/algorithmic component (as there isn't one). The studies described are performance validation studies.

9. How the Ground Truth for the Training Set was Established

• Not applicable as no "training set" is mentioned in the context of device development. The ground truth for all performance evaluation studies (precision, cut-off, method comparison, lay user) was established using GC/MS.

Ask a specific question about this device

First Sign™ Drug of Abuse Tests are immunochromatographic assays for the qualitative determination of Amphetamine (d-amphetamine), Cocaine (Benzoylecgonine), and Marijuana (11-nor-A9-THC-9-COOH ) in human urine at cut-off concentrations of 1000 ng/mL, and 50 ng/mL, respectively. The tests are available in a Cup format and a Dip Card format.

The tests provide only preliminary test results. A more specific alternative chemical method must be used in order to obtain a confirmed analytical result. GC/MS is the preferred confirmatory method. Clinical consideration and professional iudgment should be exercised with any drug of abuse test result, particularly when the preliminary result is possive. For in vitro diagnostic use only. The tests are intended for over-the-counter and for prescription use.

First Sign™ Drug of Abuse Tests are immunochromatographic assays. Each assay test is a lateral flow system for the qualitative detection of cocaine, amphetamine, and marijuana in human urine. The products are single-use in vitro diagnostic devices, which come in the formats of DipCards or Cups. Each test kit contains a Test Device (in one of the two formats), a package insert and a urine cup for sample collection. Each test device is sealed with a desiccant in an aluminum pouch.

This document describes the performance characteristics of the First Sign™ Drug of Abuse Dip Card Test and First Sign™ Drug of Abuse Cup Test for Amphetamine, Cocaine, and Marijuana.

Here's an analysis based on your request:

1. Table of Acceptance Criteria and Reported Device Performance

The document does not explicitly state "acceptance criteria" as a set of predefined thresholds. However, the performance is demonstrated through various studies. The primary measure of performance is the ability of the device to correctly identify positive and negative samples relative to a defined cut-off, as confirmed by GC/MS.

For the precision studies, the acceptance criterion implicitly appears to be 100% agreement for samples significantly below (-50%, -75%, -100% cut-off) and significantly above (+50%, +75%, +100% cut-off) the cut-off. For samples near the cut-off, some discordance is expected.

For the comparison studies (against GC/MS), the performance is presented through the number of correct classifications. For the lay-user study, the performance is given as the "percentage of correct results (%)".

Below is a summary of the reported device performance for samples at the cut-off and at +25% and -25% of the cut-off, as these are the most critical regions for accuracy.

Amphetamine (AMP) - Cut-off: 1000 ng/mL (all values are percentage of samples correctly classified)

Cocaine (COC) - Cut-off: 300 ng/mL

Marijuana (THC) - Cut-off: 50 ng/mL

Note: For the precision and lay-user studies, performance at the exact cut-off received mixed results (some negative, some positive), indicating the inherent variability near the detection limit. For the comparison studies, samples were categorized into "Near Cutoff Negative" (between -50% and cut-off) and "Near Cutoff Positive" (between cut-off and +50%), not specifically at the cut-off. Performance percentages for comparison studies are not directly provided in the text and would require calculation from the tables.

2. Sample Size and Data Provenance for Test Set

• Precision Studies:

  • For each drug (AMP, COC, THC) and each format (Dip Card, Cup), 9 different concentrations were tested: -100%, -75%, -50%, -25%, Cut-off, +25%, +50%, +75%, +100% of the cut-off.
  • Each concentration was tested two runs per day for 25 days, using three different lots of the device. This means for each drug/format/concentration, there were 50 tests per lot, totaling 150 tests per concentration. (e.g., 50-/0+ means 50 negative results, 0 positive results).
  • Total tests for precision: 3 drugs * 2 formats * 9 concentrations * 150 tests/concentration = 8100 tests.
  • Data Provenance: "Samples were prepared by spiking drug in negative samples." "Each drug concentration was confirmed by GC/MS." This suggests prospective, laboratory-controlled data using spiked urine samples. The country of origin is not specified but implied to be the location of the manufacturing company (W.H.P.M., Inc., Irwindale, CA).

• Cut-off Verification Study:

  • A total of 150 samples equally distributed at -50% cut-off; -25% cut-off; cut-off; +25% cut-off; +50% cut-off were tested.
  • This implies 5 concentrations * 30 samples/concentration = 150 samples.
  • Data Provenance: Not explicitly stated but likely lab-prepared spiked samples similar to precision studies.

• Comparison Studies (Method Comparison):

  • 80 clinical samples (40 negative and 40 positive) were used for each drug (AMP, COC, THC) and each format (Cup, Dip Card).
  • Total samples for comparison studies: 3 drugs * 2 formats * 80 samples/drug/format = 480 clinical samples.
  • Data Provenance: "Un-altered clinical samples." This suggests retrospective or prospectively collected clinical samples, but without further detail on how they were collected. The country of origin is not specified but implied to be the location of the study (in-house).

• Lay-user Study:

  • 280 lay persons tested the Amphetamine devices.
  • 280 lay persons tested the Cocaine devices.
  • 280 lay persons tested the Marijuana devices.
  • Each participant was given 1 blind labeled sample. These samples were prepared at 7 different concentrations: negative, +/-25%, +/-50%, +/-75%, +/-100% of the cut-off. This suggests that for each drug, 7 concentrations * 20 samples/concentration = 140 samples were prepared. Given 280 lay persons, it implies each person tested one sample, and samples were duplicated for the number of participants.
  • Data Provenance: "Urine samples were prepared at the following concentrations; negative, +/-75%, +/-25% of the cutoff by spiking drugs into drug free-pooled urine specimens. The concentrations of the samples were confirmed by GC/MS." This is prospective, laboratory-prepared spiked urine samples, presented as blind-labeled to the lay users. Country of origin not specified, but study performed "at three intended user sites".

3. Number of Experts and Qualifications for Ground Truth

• Precision, Cut-off, Interference, Specificity, and Effect of Urine Specific Gravity and pH Studies: The ground truth was established by GC/MS (Gas Chromatography/Mass Spectrometry). GC/MS is a laboratory analytical method and does not involve human experts in establishing the "ground truth" concentration of the drugs. These studies were performed by laboratory personnel.
• Comparison Studies (Method Comparison): The ground truth for the 480 clinical samples was also established by GC/MS. These were "compared to GC/MS results," meaning GC/MS served as the reference method (ground truth). The tests themselves were run by "three different laboratory assistants." Their qualifications beyond being "laboratory assistants" are not specified.
• Lay-user Study: The ground truth for the spiked samples was established by GC/MS.

4. Adjudication Method for the Test Set

• Precision, Cut-off, Interference, Specificity, and Effect of Urine Specific Gravity and pH Studies: The data presented are direct results from the devices (negative/positive) for specific concentrations as confirmed by GC/MS. There is no mention of an adjudication method as the results are quantitative and compared against a defined cut-off.
• Comparison Studies: The tests were run by "three different laboratory assistants" (Viewers A, B, C). Each viewer independently provided results (Positive/Negative). The tables present results for each viewer separately. There is no explicit adjudication method (like rule-based consensus, e.g., 2+1, 3+1). Discordant results are noted for individual viewers without a resolution process described for a single, unified "device result."
• Lay-user Study: Each lay person provided individual results. The data aggregates these individual results. There is no adjudication method described for resolving differences between lay users.

5. Multi Reader Multi Case (MRMC) Comparative Effectiveness Study

No. The document presents neither an MRMC study design nor a comparative effectiveness study evaluating human readers with and without AI assistance. The studies involve (1) laboratory personnel interpreting the device, and (2) lay users interpreting the device. There is no AI component mentioned.

6. Standalone (Algorithm Only) Performance

Yes, in essence, the "Analytical Performance" section (Precision, Interference, Specificity, Cut-off, Stability, Effect of Urine Specific Gravity and pH) and the "Comparison Studies" with laboratory assistants can be considered standalone performance of the device without explicit human-in-the-loop assistance influencing the device's detection capabilities. The human element here is about reading and interpreting the visual line on the test, not assisting an underlying algorithm. The device itself is an immunochromatographic assay, which is a chemical/biological-based test, not an AI algorithm.

7. Type of Ground Truth Used

The type of ground truth used across all reported studies (Precision, Cut-off, Comparison, Lay-user) is GC/MS (Gas Chromatography/Mass Spectrometry). This is a laboratory-based, highly accurate chemical method considered the gold standard for drug confirmation in urine. All samples (spiked or clinical) had their drug concentrations confirmed using GC/MS.

8. Sample Size for the Training Set

This document describes a medical device which is an immunochromatographic assay, not an AI/ML-based device. Therefore, there is no "training set" in the context of machine learning algorithms. The performance is based on the chemical reactions and design of the lateral flow assay.

9. How the Ground Truth for the Training Set Was Established

As there is no "training set" for an AI/ML algorithm mentioned, this question is not applicable. The device operates based on antigen-antibody immunochemistry, not learned patterns from data.

Ask a specific question about this device