(316 days)
The INDX" Dip-S-Ticks scrub typhus test is an enzyme immunoassay for use as an aid in the diagnosis of scrub typhus, and detects total IgG and IgM antibodies to Orientia tsursugamushi. The Dip-S-Ticks test is qualitative when used to test a single specimen; when using paired specimens to detect sero-conversion the test may be semi-quantitative. The test is intended for use in serum as well as heparinized plasma, whole blood and finger-stick capillary blood and is intended to be performed by trained medical personnel only.
The INDX® DIP-S-TICKS® Scrub Typhus Test For the Detection of IgG and IgM Antibodies to Orientia tsutsugamushi (scrub typhus) utilizes an enzyme-linked immunoassay (ELISA) dotblot technique for the detection of IgM as well as total (IgM and IgG) dengue antibodies. The antigen is dispensed as discrete dots onto a solid membrane. After adding specimen to a reaction vessel, an assay strip is inserted, allowing patient antibodies reactive with the test antigen to bind to the strip's solid support membrane. In the second stage, the reaction is enhanced by removal of non-specifically bound materials. During the third stage, alkaline phosphatase-conjugated anti-human IgG/IgM antibodies are allowed to react with bound patient antibodies. Finally, the strip is transferred to enzyme substrate reagent, which reacts with bound alkaline phosphatase to produce an easily seen, distinct spot.
Here's a breakdown of the acceptance criteria and the study that proves the device meets them, based on the provided text:
Device: INDX® DIP-S-TICKS® Scrub Typhus Test For the Detection of IgG and IgM Antibodies to Orientia tsutsugamushi (scrub typhus)
1. Table of Acceptance Criteria and Reported Device Performance
The acceptance criteria are not explicitly defined as pass/fail thresholds in the document. Instead, the studies aim to demonstrate substantial equivalence to predicate devices (IFA/IIP methods). The "Summary of Comparisons" table found on page 2 provides average performance characteristics across all comparison studies. The individual study results also serve as evidence of performance.
| Performance Metric | Acceptance Criteria (Implied by Predicate Equivalence) | Reported Device Performance (Average across 4 studies) |
|---|---|---|
| Sensitivity | Comparable to predicate device (IFA/IIP) | 91.4% (ranging from 86.7% to 100.0%) |
| Specificity | Comparable to predicate device (IFA/IIP) | 89.1% (ranging from 66.7% to 100.0%) |
| Agreement (with predicate) | Comparable to predicate device (IFA/IIP) | 89.5% (ranging from 80.2% to 96.7%) |
2. Sample Size Used for the Test Set and Data Provenance
The test set consisted of samples from 262 febrile patient sera across four independent studies.
- Study 1: 91 presumptive positive samples (retrospective)
- Study 2: 60 sera from febrile patients (prospective)
- Study 3: 83 sera from febrile patients (prospective)
- Study 4: 28 paired acute and convalescent sera (extension of Study 1, retrospective)
Data Provenance:
- Study 1 & 4: NorthCentral Malaysia (from a U.S. medical reference laboratory for rickettsial diseases)
- Study 2: Northern Thailand (Chiangrai Hospital)
- Study 3: North-East Thailand (Mahara Hospital in Nakhon Ratchasima province)
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications
The document does not explicitly state the number of experts or their specific qualifications (e.g., radiologist with 10 years experience). However, the ground truth was established by comparison to legally marketed predicate devices:
- Indirect Immunofluorescence (IFA) Slide Test: Used in Study 1 and Study 4.
- Indirect Immunoperoxidase (IIP) method: Used in Study 2 and Study 3.
The predicate methods are established serological tests for rickettsial species. It is implied that the interpretation of these predicate tests would be performed by trained medical technologists, as mentioned in the "Intended Use" section for the INDX device itself, and as such, these results form the "expert consensus" or established diagnostic standard against which the new device is compared.
4. Adjudication Method for the Test Set
The document does not describe an explicit adjudication method for the test set in terms of multiple expert reviews and conflict resolution. The ground truth was established by the results of the predicate IFA or IIP methods.
5. If a Multi Reader Multi Case (MRMC) Comparative Effectiveness Study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance
No, a multi-reader multi-case (MRMC) comparative effectiveness study was not performed. This device is a semi-quantitative Enzyme Immunoassay (EIA) dot-blot test, not an AI-assisted diagnostic tool.
6. If a Standalone (i.e. algorithm only without human-in-the-loop performance) was done
Yes, this study is inherently a standalone performance evaluation of the device. The INDX DIP-S-TICKS test is an in-vitro diagnostic device that is read directly by trained medical technologists. The performance metrics (Sensitivity, Specificity, Agreement) reported are for the device itself against the predicate methods. The "human-in-the-loop" aspect here refers to the medical technologist performing and interpreting the test, but the study design is evaluating the diagnostic accuracy of the device's output.
7. The Type of Ground Truth Used (expert consensus, pathology, outcomes data, etc)
The ground truth was established based on the results of predicate serological tests (IFA or IIP methods), which are considered the diagnostic standard for scrub typhus. For Study 4, a 4-fold rise in antibody titer in the predicate IFA test was used to indicate active disease.
8. The Sample Size for the Training Set
The document does not specify a separate "training set" in the context of machine learning or algorithm development. This refers to a traditional in-vitro diagnostic device evaluation. The studies presented are primarily for performance evaluation and comparison to predicate devices.
Non-Clinical Tests did include some evaluations that could be considered internal validation or characterization:
- Normal Population: 10 asymptomatic normal donors.
- Negative Patient Control Sera: 51 sera with confirmed non-rickettsial diseases.
- Precision Study: Repeated tests on specific human immune sera (S0023, S0024, S0025, HAS).
9. How the Ground Truth for the Training Set Was Established
As there isn't a "training set" in the context of an algorithm seeking ground truth to learn from, this question isn't directly applicable. However, for the internal evaluations mentioned in point 8, the ground truth was established as follows:
- Normal Population: Negative IFA result for scrub typhus.
- Negative Patient Control Sera: Confirmed not to have scrub typhus through various disease diagnoses (Rheumatoid Factor, Antinuclear Antibody, Malaria, Bartonellosis, Leptospirosis, Typhoid, Cholera).
- Precision Study: These were internal characterization studies, not directly reliant on an external "ground truth" but rather testing the consistency of the device's own measurements with known positive or negative control sera. The "gold standard" for these sera would also likely have been established by predicate methods or clinical diagnosis.
§ 866.3500 Rickettsia serological reagents.
(a)
Identification. Rickettsia serological reagents are devices that consist of antigens and antisera used in serological tests to identify antibodies to rickettsia in serum. Additionally, some of these reagents consist of rickettsial antisera conjugated with a fluorescent dye (immunofluorescent reagents) used to identify rickettsia directly from clinical specimens. The identification aids in the diagnosis of diseases caused by virus-like bacteria belonging to the genusRickettsiae and provides epidemiological information on these diseases. Rickettsia are generally transmitted by arthropods (e.g., ticks and mosquitoes) and produce infections in humans characterized by rash and fever (e.g., typhus fever, spotted fever, Q fever, and trench fever).(b)
Classification. Class I (general controls). The device is exempt from the premarket notification procedures in subpart E of part 807 of this chapter subject to § 866.9.