(274 days)
This method is for the semi-quantitative determination of biotinidase, EC 3.5.1.12, activity in dried whole blood spots using a spectrophotometer. Measurement of biotinidase activity is primarily for the diagnosis and treatment of biotinidase deficiency in newborns. This method is intended for in vitro diagnostic use in screening for decreased levels of biotinidase activity and not for monitoring purposes.
Biotinidase activity is determined by measuring the color that develops from p-Aminobenzoic Acid (PABA) after PABA is released from Biotinyl-p-Aminobenzoate (Biotin-PAB). Samples with biotinidase activity develop a purple color. Samples without biotinidase activity remain straw-colored. Patient samples of whole blood collected on standardized filter paper are eluted in a standard 96 well microplate. The plate is incubated with Biotin-PAB in a buffer at 37℃ for 240 minutes on a combination incubator/shaker. Following incubation, TCA is added to the sample mixture and the resulting precipitate is removed through filtration. The PABA in the filtrate is subsequently diazotized and coupled to a napthol derivative to form an azo dye by the successive addition of sodium nitrite, acidic ammonium sulfamate and finally, N-1-naphthylethylenediamine dihydrochloride (NED). The azo dye is measured colorimetrically at 550 nm on a commercial microplate absorbance reader with a reference measurement at 690 mm.
Here's a breakdown of the acceptance criteria and the study that proves the device meets them, based on the provided text:
Acceptance Criteria and Reported Device Performance
The document does not explicitly present a formal "acceptance criteria" table with pre-defined thresholds for performance that the new device must meet. Instead, it compares the performance of the new device (SPOTCHECK Biotinidase Microplate Reagent Kit) against a legally marketed predicate device (K010844 Biotinidase Kit, 50-Hour) to demonstrate substantial equivalence, implying that performance comparable to the predicate device is the "acceptance criteria."
However, we can infer some criteria and report the device's performance based on the provided data:
| Acceptance Criterion (Inferred from Predicate Performance or Standards) | SPOTCHECK Biotinidase Microplate Reagent Kit Reported Performance | Notes / Comparisons to Predicate |
|---|---|---|
| Linearity (Correlation Coefficient) | ≥ 0.999 (typical) | Achieved with PABA standards ranging from 0 to 200 MRU. Also deemed linear over 5 to 213 MRU, adhering to CLSI EP6-A. Predicate device also uses calibration. |
| Sensitivity (Limit of Blank - LoB) | 2 MRU | Determined using CLSI EP17-A protocol. |
| Sensitivity (Limit of Detection - LoD) | 3 MRU | Determined using CLSI EP17-A protocol. "Correlates well with the sensitivity of the predicate device." LoQ = LoD. |
| Clinical Cutoff for Profound Deficiency | 10% mean activity of population | "Same for both devices." Each lab must determine its own range. |
| Clinical Cutoff for Partial Activity | 37% mean activity of population | "Same for both devices." Each lab must determine its own range. All samples below this require follow-up. |
| Classification of Deficient Samples | 11 out of 11 correctly classified as "Deficient" | Compared against predicate device results for 2 clinically-confirmed patients and 10 CDC deficient controls. |
| Classification of Partial Activity Samples | 4 out of 5* correctly classified as "Partial Activity" | Compared against predicate device. One partial* sample and 11 deficient samples represent the 2 clinically-confirmed patients and 10 CDC deficient controls. |
| Classification of Normal Samples | 559 out of 560 correctly classified as "Normal" | Compared against predicate device for 564 unclassified patient samples. One discrepant result (Normal by microplate kit, Partial Activity by predicate) was not confirmed by follow-up. One discrepant result (Normal by microplate kit, Deficient by predicate) was not confirmed by follow-up. |
| Within-Run Precision (Low Activity) | CV = 6.3% (Avg 18.3 MRU) | Predicate device CV = 17% (Avg 0.54 ERU). New device shows significant improvement. |
| Within-Run Precision (Moderate Activity) | CV = 4.4% (Avg 50.7 MRU) | Predicate device CV = 3.2% (Avg 14.6 ERU). |
| Within-Run Precision (Normal Activity) | CV = 3.8% (Avg 124.9 MRU) | Predicate device CV = 4.7% (Avg 79.6 ERU). New device shows improvement. |
| Total Precision (Low Activity) | CV = 9.4% (Avg 18.3 MRU) | Predicate device CV = 56% (Avg 0.54 ERU). New device shows significant improvement. |
| Total Precision (Moderate Activity) | CV = 7.3% (Avg 50.7 MRU) | Predicate device CV = 6.4% (Avg 14.6 ERU). |
| Total Precision (Normal Activity) | CV = 7.1% (Avg 124.9 MRU) | Predicate device CV = 5.8% (Avg 79.6 ERU). |
| Interference: Sulfamethoxazole | 1.58 mmol/L causes clinically significant effect. | Predicate: "No limit established, but a known interference is cited." Suggests patients treated with sulfonamides need alternate screening. |
| Interference: Albumin | Above normal concentrations can show 1.6 MRU positive interference per 1 g/dL. | Predicate: 25 g/L albumin no interference; > 25 g/L significant interference. |
| Interference: Hemoglobin | 2 g/L hemoglobin no clinically significant interference. | Predicate: 1 g/L hemoglobin no clinically significant interference. |
| Interference: Lipids | 37 mmol/L caused a decrease in response (may cause false positives). | Predicate: 2.5 g/L lipids no clinically significant interference. "No risk to deficient patients" noted for new device. |
| Interference: Bilirubin | 342 µmol/L (direct/indirect) no clinically significant interference. | Predicate: 0.25 g/L bilirubin no interference. |
| Interference: Trimethoprim | 138 µmol/L caused a decrease in response (may cause false positives). | Predicate: "No limit established." "No risk to deficient patients" noted for new device. |
| Interference: Gamma globulin | 60 g/L no clinically significant interference. | Predicate: "No limit established." |
Punctuation for specific values in the table above can be found in the excerpt itself.
2. Sample size used for the test set and the data provenance:
- Test Set Sample Size:
- Classification Study: 564 unclassified patient samples, 2 biotinidase deficient patient samples, and 10 deficient controls. Total of 576 samples.
- Precision Study:
- Within-Run & Total Precision (main): n = 80 for each activity level (Low, Moderate, Normal) for the Microplate Kit. n = 32 (Low), n = 44 (Moderate), n = 44 (Normal) for the Predicate Device.
- Additional 5-day Low Activity Precision: n = 80 for the Microplate Kit.
- Data Provenance:
- Classification Study: The 2 deficient patient samples were "clinically-confirmed." The 10 deficient controls were "provided by the Centers for Disease Control (CDC)." The 564 unclassified patient samples are not explicitly stated in terms of specific country of origin, but generally, patient samples for such studies imply a clinical setting.
- Prospective/Retrospective: The text implies a retrospective analysis as samples "were treated according to the procedures detailed under SPEDMEN COLLECTION AND PREPARATION FOR ANALYSIS in the product insert," suggesting these were existing samples processed with the new kit.
3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts (e.g. radiologist with 10 years of experience):
- The document implies that the ground truth for the two "clinically-confirmed" deficient patient samples was established by clinical diagnosis, as they are referred to as "persons clinically-confirmed as such."
- For the 10 deficient controls, the ground truth was "provided by the Centers for Disease Control (CDC)," which suggests these are established and certified deficient samples.
- The "experts" themselves and their specific qualifications are not detailed in the provided text. The predicate device's classification served as a comparative ground truth for other samples and for discrepancies.
4. Adjudication method (e.g. 2+1, 3+1, none) for the test set:
- The document mentions "Discrepant results ** were not confirmed by follow-up testing." This indicates that for samples where the new Microplate Kit and the Predicate Device yielded different classifications (e.g., Microplate Kit: Normal, Predicate Device: Partial Activity), no explicit expert adjudication or further follow-up gold standard testing was performed to resolve the discrepancy for the purpose of this study report. The discrepancy is simply noted. Therefore, an explicit adjudication method (like 2+1 or 3+1) was not described or applied to resolve discrepancies for the reported results. The predicate device's classification was primarily used as a comparative benchmark, not an adjudicated ground truth in cases of disagreement.
5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:
- No, an MRMC comparative effectiveness study was not done. This device is an in-vitro diagnostic (IVD) reagent kit that measures biotinidase activity, not an AI-powered diagnostic imaging tool or a system involving human readers interpreting outputs. It's a chemical assay. The comparison is between the new kit and a predicate kit (both laboratory assays), not between humans with and without AI assistance.
6. If a standalone (i.e. algorithm only without human-in-the loop performance) was done:
- Yes, the performance presented is standalone for the device (algorithm/assay only). The kit measures biotinidase activity using a spectrophotometer, which is an automated reading of a chemical reaction. The results described (linearity, sensitivity, precision, classification) are direct outputs of the assay kit itself without human interpretation influencing the measurable outcome, although trained laboratory personnel are required for manual steps (e.g., pipetting, filtration). The "human-in-the-loop" would be the technician performing the assay and reviewing the numerical results, but the performance metrics reported are for the assay's output itself.
7. The type of ground truth used (expert consensus, pathology, outcomes data, etc):
- Clinical Confirmation/Certified Controls and Predicate Device Comparison:
- For deficient patient samples, "clinical confirmation" of biotinidase deficiency was the ground truth.
- For deficient controls, the "Centers for Disease Control (CDC)" provided samples with established deficiency.
- For the bulk of the samples (564 unclassified), the predicate device's classification served as the comparative benchmark. While not a "true" independent ground truth for every sample, it's used as the standard against which the new device's classifications are measured for substantial equivalence.
8. The sample size for the training set:
- The document does not explicitly mention a separate "training set" in the context of machine learning or AI. As a reagent kit for a chemical assay, it does not typically involve explicit "training data" in the AI sense. The development of such kits often involves extensive R&D and optimization, but this is usually not referred to as a "training set" in the same way an AI model would have one.
9. How the ground truth for the training set was established:
- As a "training set" in the AI sense is not specified, the method for establishing its "ground truth" is also not applicable or described within the document. The development of the assay would rely on established biochemical principles and validation using known concentrations and clinical samples.
§ 862.1118 Biotinidase test system.
(a)
Identification. The biotinidase test system is an in vitro diagnostic device intended to measure the activity of the enzyme biotinidase in blood. Measurements of biotinidase are used in the treatment and diagnosis of biotinidase deficiency, an inborn error of metabolism in infants, characterized by the inability to utilize dietary protein bound vitamin or to recycle endogenous biotin. The deficiency may result in irreversible neurological impairment.(b)
Classification. Class II (special controls). The special control is sale, distribution, and use in accordance with the prescription device requirements in § 801.109 of this chapter.