Search Results

This device is intended for the quantitation of specific organophosphate metabolites by LC/MS/MS. The device system includes organophosphate metabolite calibrators to calibrate the system and organophosphate metabolite controls for quality control monitoring of the system. This device is intended for use in a single laboratory to detect and measure the concentration of specific organophosphate metabolites in human urine from individuals who have signs and symptoms consistent with cholinesterase poisoning. The data obtained by this device will be used following an exposure or suspected exposure event to confirm exposure, identify the causative agent, and distinguish exposed from the unexposed patients.

Device Description

The device comprises internal standards which serve as the basis for the calibrators and quality control materials. A solid phase extraction cartridge is initially used to extract the metabolites for further separation and analysis on a tandem mass spectrometer and HPLC system.

AI/ML Overview

This document describes CDC's Quantitation of Organophosphate Metabolites in Urine by LC/MS/MS device.

Acceptance Criteria and Device Performance Study

The study evaluates the analytical performance of the device, focusing on precision, linearity/reportable range, traceability, stability, detection limits, and analytical specificity.

1. Table of Acceptance Criteria and Reported Device Performance

The acceptance criteria are implied by the stated performance characteristics deemed sufficient by the FDA for classification. The reported device performance is directly from the provided tables.

Performance Characteristic	Acceptance Criteria (Implied)	Reported Device Performance and Fulfillment
Precision	Adequate precision across the measuring range. Specific criteria for %CV are not explicitly stated as acceptance criteria but are implied by the provided data being deemed "sufficient" by the FDA.	Within-day (n=4, 1 operator): OP-1: 5.4% CV (15 ng/mL), 2.9% CV (75 ng/mL) OP-2: 2.4% CV (15 ng/mL), 2.4% CV (75 ng/mL) OP-3: 2.7% CV (15 ng/mL), 4.1% CV (75 ng/mL) OP-4: 5.8% CV (15 ng/mL), 1.9% CV (75 ng/mL) OP-5: 3.1% CV (15 ng/mL), 4.0% CV (75 ng/mL) Between-day (n=20, 14 days, 3 operators): OP-1: 4.2% CV (15 ng/mL), 3.0% CV (75 ng/mL) OP-2: 3.4% CV (15 ng/mL), 2.8% CV (75 ng/mL) OP-3: 4.1% CV (15 ng/mL), 2.7% CV (75 ng/mL) OP-4: 4.0% CV (15 ng/mL), 3.1% CV (75 ng/mL) OP-5: 4.2% CV (15 ng/mL), 2.9% CV (75 ng/mL) (All values reported for two concentrations (15 and 75 ng/mL) for five organophosphate metabolites (OP-1 to OP-5). The data "met the acceptance criteria" according to the section, implying these values are acceptable.)
Linearity/Reportable Range	Adequate accuracy across the measuring range. The claimed measuring range is 2 – 200 ng/mL. Recovery percentages are expected to be within an acceptable range, although specific percentage limits are not explicitly stated.	Claimed Measuring Range: 2 – 200 ng/mL Recovery Studies: Spiked pooled urine samples analyzed in triplicate. OP-1: % Recovery from 82.6% to 101.6% across reported concentrations (2-200 ng/mL). OP-2: % Recovery from 92.6% to 99.9% across reported concentrations (2-200 ng/mL). OP-3: % Recovery from 86.5% to 105.2% across reported concentrations (2-200 ng/mL). OP-4: % Recovery from 81.1% to 96.0% across reported concentrations (2-200 ng/mL). OP-5: % Recovery from 92.2% to 112.9% across reported concentrations (2-200 ng/mL). (The full range of recovery data for various concentrations supports the claimed measuring range, implying accuracy within this range is acceptable.)
Traceability	Calibrator and quality control material values should be traceable.	Calibrators are prepared by a vendor using a dilution scheme and verified for precision, accuracy, and linear response. Values assigned to calibrators and quality control materials are traceable to internal standards synthesized by a vendor. No certified reference materials are available. (This fulfills the traceability requirement.)
Stability	Adequate stability for calibrators and quality control materials to support proposed shelf life.	Calibrator and quality control material storage stability was evaluated. All data met acceptance criteria and support a shelf life claim of 2 years at -70°C. (This fulfills the stability requirement.)
Detection Limits (LoQ)	Limit of Quantitation (LoQ) should be consistent with the claimed measuring range (2-200 ng/mL).	LoQ for all 5 metabolites (OP-1 to OP-5) is 2 ng/mL. LoB: Ranged from 0.029 to 0.406 ng/mL. LoD: Ranged from 0.13 to 0.69 ng/mL. (The determined LoQ of 2 ng/mL aligns with the lower end of the claimed measuring range.)
Analytical Specificity	Demonstrate ability to detect specific organophosphate metabolites and absence of interference from other substances (e.g., related pesticides, naturally occurring compounds) to avoid false positives.	200 individual unexposed urine samples: No peaks above LoQ were detected for all five metabolites. This indicates no false positives from naturally occurring compounds in unexposed urine. Six non-specific pesticide metabolites (dialkylphosphates) spiked into samples: No interfering peaks associated with the organophosphates were detected. (This demonstrates specificity for organophosphate metabolites over similar compounds and in the absence of exposure.)
Clinical Cut-off	A defined clinical cut-off for determining exposure.	All values ≥ 2 ng/mL indicate exposure. Values < 2 ng/mL reported as "non-detectable". (This is a clear clinical cut-off.)
Expected Values/Reference Range	Clear understanding of expected values in exposed vs. unexposed subjects.	200 individual unexposed urine samples showed no positive peaks. Since organophosphate metabolites are not naturally occurring, 0 ng/mL is expected for an unexposed subject. (This confirms the expected values in a non-exposed population.)

2. Sample sizes used for the test set and the data provenance

Precision/Reproducibility:
- Within-day: n=4 per analyte (for each of 2 concentrations)
- Between-day: n=20 per analyte (for each of 2 concentrations) over 14 days.
- Data Provenance: The study was conducted using quality control materials in synthetic urine and spiked pooled urine samples. It is an analytical study, not involving human subjects or geographical data, therefore the provenance is the laboratory where the testing was performed (likely within the CDC). This is a prospective analytical study.
Linearity/Assay Reportable Range: Spiked pooled urine samples were prepared and analyzed in triplicate at various concentrations. The exact total number of individual urine samples used to form the "pooled urine" is not specified for the linearity study specifically, but the "unexposed urine samples" used for analytical specificity and expected values provides context.
Detection Limit: Spiked synthetic urine samples at 1, 2, and 5 ng/mL, as well as a blank synthetic urine sample. The exact number of replicates or distinct samples at each concentration is not specified beyond "measuring spiked synthetic urine samples".
Analytical Specificity: 200 individual unexposed urine samples.
Expected Values/Reference Range: 200 individual unexposed urine samples.

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts

Analytical Study: This is an analytical performance study for a quantitative LC/MS/MS assay establishing measurements of chemical analytes, not a diagnostic study requiring clinical ground truth established by experts interpreting images or patient data. The "ground truth" for the test set (e.g., known concentrations in spiked samples, absence of analytes in blank/unexposed samples) is established by the preparation methods of the samples themselves.
No clinical experts or their quantification were used to establish ground truth for this analytical test set.

4. Adjudication method (e.g., 2+1, 3+1, none) for the test set

None. This is an analytical study determining the performance of a quantitative chemical assay. Adjudication methods like 2+1 or 3+1 are typically used in clinical studies where human readers are making subjective interpretations (e.g., radiology reads) and a consensus among experts is needed to establish ground truth.

5. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance

Not applicable. This study is for a quantitative chemical assay (LC/MS/MS) and does not involve human readers interpreting cases or AI assistance in those interpretations. Therefore, an MRMC study or evaluation of human reader improvement with AI is not relevant.

6. If a standalone (i.e., algorithm only without human-in-the-loop performance) was done

Yes, this is a standalone analytical performance study. The device itself (LC/MS/MS system and associated method) is an "algorithm only" device in the sense that it performs a quantitative measurement on a biological sample to produce a numerical result. Its performance is evaluated independently of human interpretation of the result in a clinical context, though human operators are involved in running the assay. The study demonstrated the device's ability to accurately and precisely quantify organophosphate metabolites.

7. The type of ground truth used (expert consensus, pathology, outcomes data, etc.)

The ground truth used is primarily analytical ground truth based on:
- Known concentrations of organophosphate metabolites in spiked synthetic urine and pooled urine samples for linearity and detection limit studies.
- Absence of target analytes in synthetic urine blanks and individual unexposed human urine samples for detection limits and analytical specificity.
- Pre-defined concentrations in quality control materials for precision studies.

8. The sample size for the training set

Not applicable. This is an analytical device for quantitative measurement, not a machine learning or AI model that requires a "training set" in the conventional sense. The development of the LC/MS/MS method itself would involve optimization and calibration, but this is distinct from the statistical training of an AI algorithm.

9. How the ground truth for the training set was established

Not applicable for the same reason as above. The "ground truth" or reference for establishing the method's parameters would have been based on established analytical chemistry principles, known chemical standards, and optimized laboratory procedures.

Ask a Question

Ask a specific question about this device

Page 1 of 1