(38 days)
The BreathScan® Alcohol Detector is an in vitro medical device to qualitatively detect the presence of alcohol in the human breath. It is a disposable screening device for one-time use. The detector is available at several detection cut-offs; 0.02, 0.04, 0.05, and 0.08 percent breath alcohol. The device is used only as a screening device and is only an indication of the possible presence of alcohol in the blood of the test subject. Correlation between breath alcohol_content and blood alcohol content depends on many variables. BreathScan is not intended to legally determine blood alcohol presence, level, or inference of intoxication.
The BreathScan® Alcohol Detector (Tester) is a visual qualitative test for alcohol in human breath. The Tester contains chemicals that change color in the presence of alcohol vapors) utilizing the patented technology (4,740,475). The Tester is made up of a 2-part glass vessel inside a plastic tube. One part contains light yellow crystals that change color when exposed to alcohol vapors. The other part is an opening to blow into while running the test. If alcohol is present, the crystals will change from yellow to light green/blue. How many crystals turn color will depend on the cut-off of the Tester and how much alcohol is in the breath. The yellow crystals in the Tester are coated with potassium dichromate (K)Cr30-) and sulfuric acid (H2SO4). The amount of these indicator chemicals is adjusted according to the selected cutoff of the Tester. A color change is produced when alcohol vapors are oxidized to acetic acid and the indicator chemicals change to chromium sulfate [Cr2(SO4)3]. The majority of crystals change from vellow to light aqua (greenish-bluish-green) when alcohol vapors are present at a level equal to or exceeding the cutoff of the Tester. The BreathScan® Tester is available in several cut-offs (.02%, .04%, .05%, and .08%). The cut-off is printed on the Tester label and is expressed as a specific percentage of breath alcohol.
Here's a breakdown of the acceptance criteria and the study information for the BreathScan® Alcohol Detector, based on the provided text:
1. Acceptance Criteria and Reported Device Performance
The provided text indicates that the BreathScan® Alcohol Detector was evaluated against the guidelines of the DOT/NHTSA Model Specifications for Alcohol Screening Devices. While specific numerical acceptance criteria from the DOT/NHTSA Model Specifications are not explicitly stated in the document, the device's performance is compared to a predicate device (ALCO SENSOR IV) and is assessed for user readability and usability.
Implicit Acceptance Criteria (derived from the text):
- Accuracy against a standard: Performance should be comparable to or meet the guidelines of validated evidentiary breath tests (like ALCO SENSOR IV) and DOT/NHTSA Model Specifications.
- User Usability: Users should be able to read and understand directions and properly use the device.
- Low False Positives: The device should have a low rate of false positive readings when no alcohol is present.
Here's a table summarizing the observed performance for the 0.04% BreathScan® Tester:
| BreathScan® 0.04% Tester Result | Predicate Results (ALCO SENSOR IV) Groupings | Reported Device Performance | Comments |
|---|---|---|---|
| Positive | Less than 60% below Cutoff (0.040 to 0.064%) | 15 | The device reported 15 positives in this range. The table seems to indicate these are "true positives" relative to the predicate's categorization for results above the cutoff. |
| Positive | High Positive (>0.064%) | 10 | The device reported 10 positives in this high range. These are "true positives" relative to the predicate. |
| Negative | Less than 60% below Cutoff (0.040 to 0.064%) | 5 | This indicates 5 false negatives. The predicate device showed these as positive, but the BreathScan® reported them as negative. |
| Negative | High Positive (>0.064%) | 1 | This indicates 1 false negative. The predicate device showed this as positive, but the BreathScan® reported it as negative. |
Additional Performance Metric:
- False Positives (without alcohol): 1 false positive reading out of 1200 readings (for the 0.04% tester) that did not contain alcohol. This indicates a very low rate of false positives when no alcohol is present.
2. Sample Size and Data Provenance
- Test Set Sample Size:
- For the comparison study with the Alco-Sensor IV: The numbers in Table 1 sum to 93 test cases (0+0+15+10+37+25+5+1 = 93) for the 0.04% tester's comparison to the predicate device.
- For the false positive evaluation: 1200 readings of the 0.04% tester did not contain alcohol.
- Data Provenance: Not explicitly stated (e.g., country of origin). The data appears to be prospective as it involves "user studies" and "bench testing" performed to establish device performance.
3. Number of Experts and Qualifications (Ground Truth for Test Set)
- Number of Experts: Not specified.
- Qualifications of Experts: The ground truth comes from an "evidentiary breath test, the ALCO SENSOR IV manufactured by Intoximeters, Inc. which is a DOT/NHTSA approved device." It also mentions it was "administered by a trained technician." While not "experts" in the sense of medical specialists, the ground truth is established by a validated, approved device and trained personnel.
4. Adjudication Method for the Test Set
The adjudication method wasn't explicitly described in terms of "2+1" or "3+1." The comparison relies on a single predicate device (ALCO SENSOR IV) as the reference standard. The results from the BreathScan® are then compared directly to the categorization provided by the ALCO SENSOR IV.
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
No, a multi-reader multi-case (MRMC) comparative effectiveness study was not explicitly mentioned or conducted as described (i.e., comparing human readers with and without AI assistance). The "user studies" mentioned focus on the user's ability to read directions and use the device, not on improving human reader performance with AI. The device itself is a standalone qualitative test.
6. Standalone (Algorithm Only) Performance
Yes, a standalone performance assessment was done. The BreathScan® Alcohol Detector is a standalone device that provides a qualitative visual result (color change) without human interpretation beyond observing the color. The "bench and user testing" results, particularly the comparison to the Alco-Sensor IV and the false positive rate evaluation, represent a standalone performance assessment of the device itself.
7. Type of Ground Truth Used (for Test Set)
The primary ground truth used for the test set was:
- Comparison to an Evidentiary Breath Test: The ALCO SENSOR IV, a DOT/NHTSA approved device, served as the reference standard for actual breath alcohol concentrations.
- Known Alcohol Content: For the false positive evaluation, the "1200 readings... that did not contain alcohol" indicates controlled samples with known negative alcohol content.
8. Sample Size for the Training Set
The document does not explicitly mention a separate "training set" or its sample size. This type of device (chemical reaction-based) typically does not involve machine learning or an explicit "training phase" in the way an AI algorithm would. Its development likely involved calibration and validation against known alcohol concentrations in a lab setting during its design and manufacturing, rather than a data-driven training set for an algorithm.
9. How the Ground Truth for the Training Set Was Established
Since a "training set" for an AI algorithm is not applicable, the concept of establishing ground truth for it is also not applicable here. The device's calibration and performance were likely established through engineering and chemical validation against known concentrations of alcohol and performance testing against established standards (e.g., DOT/NHTSA).
§ 862.3050 Breath-alcohol test system.
(a)
Identification. A breath-alcohol test system is a device intened to measure alcohol in the human breath. Measurements obtained by this device are used in the diagnosis of alcohol intoxication.(b)
Classification. Class I (general controls). The device is exempt from the premarket notification procedures in subpart E of part 807 of this chapter subject to the limitations in § 862.9.