Search Filters

Search Results

Found 3 results

510(k) Data Aggregation

    K Number
    DEN200070
    Device Name
    Simple 2 Test
    Date Cleared
    2023-11-15

    (1094 days)

    Product Code
    Regulation Number
    866.3385
    Type
    Direct
    Reference & Predicate Devices
    Why did this record match?
    Reference Devices :

    K111409, K132251, K180681, K200866

    AI/MLSaMDIVD (In Vitro Diagnostic)TherapeuticDiagnosticis PCCP AuthorizedThirdpartyExpeditedreview
    Intended Use

    The Simple 2 Test is intended for in vitro detection and identification of Chlamydia trachomatis (CT) and/or Neisseria gonorrhoeae (GC) in home-collected specimens which are shipped to a clinical laboratory for testing using the Aptima Combo 2 Assay on the Panther System. This product is available over-the-counter (OTC) to consumers 18 years of age and older.

    The Simple 2 Test contains all the necessary components to collect urine from male patients (Simple 2 Urine Home Collection Kit (Penile)) or vaginal swabs from female patients (Simple 2 Swab Home Collection Kit (Vaginal)) in their home, or in similar environments, without supervision from a healthcare provider.

    The Simple 2 Test Collection Kits may also be used to self-collect specimens in a clinic.

    The testing is performed, as determined to be appropriate, based on the results of LetsGetChecked Suitability Questionnaire.

    This test system is not a substitute for visits to a healthcare provider. The information provided by this product should not be used to start, stop, or change any course of treatment unless advised by your healthcare provider.

    Testing is limited to the manufacturer, Priva Path laboratories (d.b.a. LetsGetChecked. Inc.).

    Device Description

    The LetsGetChecked Simple 2 Test is composed of the Simple 2 Urine Home Collection Kit (Penile) and the Simple 2 Swab Home Collection Kit (Vaginal) and the samples are shipped to LetsGetChecked's Clinical Laboratory Improvement Amendments (CLIA)-certified laboratory for processing using the Aptima Combo 2 Assay on the Panther System. The Aptima Combo 2 Assay was previously cleared for the qualitative in vitro detection of Chlamydia trachomatis (CT) and/or Neisseria gonorrhoeae (GC) when used with male urine and self-collected vaginal swabs collected in clinical settings.

    The home collection kits are intended to be distributed for purchase online without the need for prescription from a physician or healthcare professional (over-the-counter use). After purchase, the user activates the kit on LetsGetChecked's website and is requested to fill out the Suitability Questionnaire to determine whether this test is appropriate for this individual. The user follows detailed directions for self-collecting a urogenital specimen (male urine or female vaginal swab, as applicable), packaging and shipping it to the LetsGetChecked CLIA certified laboratory for testing (PrivaPath lab). The test results are delivered to the individual via LetsGetChecked online portal. Positive and invalid/inconclusive results are followed up with a call from the LetsGetChecked Care Team. If the patient received a positive chlamydia result, treatment options are discussed. Patients receiving a positive gonorrhea test result, will be advised to contact their healthcare provider to receive appropriate treatment. The LetsGetChecked Care Team will also contact all patients who test negative for chlamydia and gonorrhea, but do report symptoms, or concerns, or where further investigation is indicated. These patients are internally flagged for follow-up.

    AI/ML Overview

    The provided text describes the evaluation of the Simple 2 Test, a device for the in vitro detection and identification of Chlamydia trachomatis (CT) and/or Neisseria gonorrhoeae (GC) in home-collected specimens. The device is intended for over-the-counter use by consumers 18 years of age and older.

    The acceptance criteria for the device are focused on usability and comprehension by lay users and the analytical performance of the collection kits under various challenging conditions (flex studies, interfering substances, shipping stability), ensuring that the home collection process does not compromise the accuracy of the downstream testing performed by the Aptima Combo 2 Assay on the Panther System.

    Here's a breakdown of the requested information based on the provided document:

    1. Table of Acceptance Criteria and Reported Device Performance

    The document does not explicitly present a single table outlining predefined "acceptance criteria" alongside specific performance metrics the device must meet. Instead, it describes various studies and their outcomes, implicitly demonstrating the device's acceptable performance. The "Success Rate" shown in Tables 3, 4, 7, and 8 for usability/comprehension, and the "Agreement with expected result" in Tables 9, 10, 11, 12, 13, 14, 16, 18, 19, and 20 for analytical performance, serve as the evidence of meeting the unstated acceptance goals.

    Based on the summaries of the studies, the implicit acceptance criterion for usability and comprehension appears to be a high success rate (generally aiming for 100% in critical steps or demonstrating that modifications resolve issues) for proper specimen collection, packaging, and comprehension of instructions and results. For analytical performance, the criterion is largely 100% agreement with expected results under various challenging conditions, or demonstration that the test robustly performs within acceptable limits despite certain challenging conditions, with any identified limitations clearly called out in labeling.

    Implicit Acceptance Criteria and Reported Performance (Selected examples from the document):

    CategoryStudy/ParameterImplicit Acceptance Criteria (Goal)Reported Device Performance (Key Findings)
    Usability & Comprehension (Male Urine Kit)Ability to transfer correct urine volume (after modification)High success rate"The results from this study showed that 9% (8/89) of lay users had difficulty transferring correct volume of urine... Therefore, modifications to the home collection kit and instructions were necessary..." (Initial study). Modifications made: graduated transfer pipette and revised instructions. The second study's "Success Rate" for various steps indicates high user compliance, though exact percentage for volume transfer is redacted in Table 3. The text states: "replacing generic transfer pipette with the graduated transfer pipette allows the users to transfer an appropriate amount of urine..."
    Comprehension of IFU, warnings, and general infoHigh success rateData redacted in Table 3, but the text states: "...results from the questionnaire support the conclusion that lay users demonstrated comprehension of the critical elements in the labeling..."
    Usability & Comprehension (Female Vaginal Swab Kit)Ease of use of collection kitHigh success rateInitial study led to modifications for closer alignment with Hologic's instructions and FAQs. Success rates for various steps are redacted in Table 7, but the text states: "...results from the questionnaire support the conclusion that lay users demonstrated comprehension of the critical elements in the labeling..."
    Flex Studies (Urine Kit - Underfilled Tubes)CT/GC detection at LoD with varying urine volumes100% agreement with expected results (or robust performance within acceptable limits)CT: 100% agreement when at least [b-4] of urine transferred; 80% at lower volumes. GC: 100% agreement when at least [b-4] of urine transferred; 60% at lower volumes. The conclusion states: "...underfilling the Transport Tube with urine will generate accurate results if at least [b-4] of the specified volume) of urine is transferred."
    Flex Studies (Urine Kit - Overfilled Tubes)CT/GC detection at LoD with varying urine volumes100% agreement with expected resultsCT/GC: 100% agreement across all tested overfilled volumes (Table 10). The conclusion states: "The results also showed that the test is not sensitive to overfilling the Transport Tube."
    Flex Studies (Urine Kit - Delay of Urine Transfer)CT/GC detection at LoD with delay in transfer100% agreement until specified delay limitCT: 100% agreement up to 6 days, then drops to 60% at Week 3. GC: 100% agreement up to 1 day, then drops to 0% at 2 days. The conclusion states: "False negative results for CT were obtained when the urine was transferred... three weeks... False negative results were obtained for GC when the transfer of urine... was delayed more than one day." Mitigation: IFU directs immediate transfer and same-day shipping.
    Interfering Substances (Urine Kit)CT/GC detection at LoD in presence of contaminants100% agreement with expected resultsCT: 100% agreement with all substances. GC: False negatives obtained with certain brands of hand soap (0% and 67%) and hand sanitizer (50%). Mitigation: Labeling cautions and instructions to wash/dry hands.
    Interfering Substances (Vaginal Swab Kit)CT/GC detection at LoD in presence of contaminants100% agreement with expected resultsCT/GC: 100% agreement with all substances (Table 14).
    Shipping Stability (Urine Kit - Summer/Winter)CT/GC detection at LoD after exposure to extreme temperaturesMaintain sample integrity allowing for expected results (>95% positivity at LoD)Summer: 100% positive for both CT/GC. Winter: [b-4] positive for CT, [b-4] positive for GC. The conclusion states: "The data demonstrate that the urine sample integrity is maintained even when exposed to extremes of temperature during shipping." Acceptable as >95% positive at LoD when a certain (redacted) number of samples were tested.
    Shipping Stability (Vaginal Swab Kit - Summer/Winter)CT/GC detection at LoD after exposure to extreme temperaturesMaintain sample integrity allowing for expected resultsSummer/Winter: 100% positive for both CT/GC. The conclusion states: "The study results demonstrate that the vaginal swab sample integrity is maintained even when exposed to extremes of temperature during shipping."

    2. Sample Size Used for the Test Set and Data Provenance

    The test sets are primarily derived from usability and analytical performance studies.

    • Usability Studies (Test Set):

      • Simple 2 Urine Home Collection Kit (Penile):
        • Initial study: 89 male participants.
        • Final usability study: 32 male participants.
      • Simple 2 Swab Home Collection Kit (Vaginal):
        • Initial study: 85 female participants.
        • Final usability study: 34 participants (33 female, 1 transgender male).
      • Data Provenance: The studies were conducted remotely from lay users' homes via online video conferencing. No specific country of origin is mentioned beyond being a US FDA submission, implying the studies were conducted in the US or for the US market. The studies appear to be prospective in nature, as they involve participants actively performing tasks following instructions and providing feedback/data.
    • Analytical Performance Studies (Flex, Interfering Substances, Shipping Stability - Test Set):

      • These studies involved specific numbers of replicates for each condition (e.g., [b-4] positive and [b-4] negative replicates for flex studies on urine volume). The exact total sample sizes for these analytical tests are scattered across tables with redacted information (e.g., "[b-4] positives/[b-4] tested").
      • Data Provenance: These are laboratory-based analytical studies, not human clinical trials. The data provenance is internal to the manufacturer's testing or a clinical laboratory. No specific country of origin is stated, but given the FDA submission, the data is expected to be relevant to US regulatory standards. These are experimental evaluations, likely prospective in nature within a controlled lab setting, designed to mimic real-world conditions.

    3. Number of Experts Used to Establish Ground Truth for the Test Set and Qualifications

    • Usability Studies: Ground truth for these studies was generally based on the objective observation of successful completion of tasks (e.g., transferring correct urine volume, proper packaging) and comprehension of instructions/information as assessed by study staff questions. The document does not specify the number or qualifications of experts involved in observing or assessing these lay user interactions directly. However, the design of the modifications and the evaluation criteria would have been informed by regulatory requirements and potentially human factors experts.
    • Analytical Performance Studies: For these studies, the ground truth is the expected presence or absence of CT/GC in the spiked or unspiked samples. This is a scientific, laboratory-defined ground truth, established by the precise spiking of organisms at known concentrations into negative matrices or using truly negative samples. The "experts" in this context are the laboratory scientists and technicians conducting the assays and preparing the samples, presumed to be qualified in molecular diagnostics. No specific number or external qualifications are mentioned for establishing this analytical ground truth, as it's inherent to the experimental design (e.g., "spiked with CT or GC organisms at the target concentration of LoD").

    4. Adjudication Method for the Test Set

    • Usability Studies: The success/failure of user actions and comprehension was likely determined by the study staff observing and questioning participants. The document doesn't detail a formal "adjudication" process akin to expert reader consensus, but rather a direct assessment of task completion and questionnaire responses. Issues identified (e.g., difficulty transferring urine) led to design modifications, implying an iterative evaluation process rather than a strict adjudication of initial failures.
    • Analytical Performance Studies: The results (e.g., #positives/#tested, agreement with expected result) are quantitative outcomes of the biochemical assay. There is no mention of an "adjudication method" in the sense of multiple human experts reviewing results. The expectation is that the laboratory results directly represent the device's performance under the tested conditions.

    5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

    There is no mention (and no indication) of a Multi-Reader Multi-Case (MRMC) comparative effectiveness study involving human readers with and without AI assistance. This device is not an AI-assisted diagnostic. It is a home collection kit for a laboratory test, wherein the "device" is the collection system and the final analysis is done by an established IVD system (Aptima Combo 2 Assay on the Panther System). The studies focus on the usability and analytical integrity of the home-collected samples, not on AI-assisted interpretation of diagnostic images or data.

    6. Standalone (Algorithm Only Without Human-in-the-Loop Performance) Study

    This question is Not Applicable in the context of this device. The Simple 2 Test is a collection kit, not a standalone algorithm. The "algorithm" (nucleic acid amplification assay) is the Aptima Combo 2 Assay on the Panther System, which is a pre-existing cleared/approved device. The performance data presented relates to the integrity and suitability of the home-collected samples for use with that established assay, not a novel standalone algorithm.

    7. Type of Ground Truth Used

    • Usability Studies: The ground truth for these studies was the observed proper execution of instructed tasks and comprehension of information by lay users, assessed against predefined criteria for successful use of the collection kit and understanding of the accompanying materials.
    • Analytical Performance Studies: The ground truth was analytically established through precise spiking of known concentrations of CT and GC organisms into negative matrices, or the use of confirmed negative samples. This constitutes a laboratory-defined ground truth (e.g., "spiked with GC or CT at 3x LoD").

    8. Sample Size for the Training Set

    The document describes two phases of usability/comprehension studies (initial and final) for both the urine and vaginal swab kits. The initial studies for both kits served a similar function to a training set or preliminary evaluation, leading to modifications in the kit and instructions.

    • Simple 2 Urine Home Collection Kit (Penile): 89 male participants in the initial study.
    • Simple 2 Swab Home Collection Kit (Vaginal): 85 female participants in the initial study.
      These initial studies acted as "training" or optimization phases by identifying deficiencies that informed product improvements.

    For the analytical "flex studies, interfering substances, and shipping stability" tests, the concept of a "training set" is not applicable in the typical machine learning sense. These are experimental validations. The initial LoD (Limit of Detection) establishment, mentioned briefly, might involve some preliminary testing that could be seen as "training" for setting analytical parameters, but no specific sample size for such an activity is provided.

    9. How the Ground Truth for the Training Set Was Established

    For the usability studies, the ground truth for the initial "training" phase was based on direct observation of user performance and comprehension assessments, identifying areas where users struggled or misunderstood. For example, the 9% of users having difficulty transferring urine volume in the initial penile kit usability study highlighted a deficiency that needed to be addressed. This identified "ground truth" (i.e., areas of user error) then informed the modifications.

    For the analytical studies, as noted in point 7, ground truth is laboratory-defined by precise manipulation of samples (e.g., spiking with known concentrations, ensuring negative samples are truly negative). This analytical ground truth would be established through standard laboratory and molecular biology techniques.

    Ask a Question

    Ask a specific question about this device

    K Number
    K162673
    Manufacturer
    Date Cleared
    2017-06-15

    (262 days)

    Product Code
    Regulation Number
    866.3305
    Why did this record match?
    AI/MLSaMDIVD (In Vitro Diagnostic)TherapeuticDiagnosticis PCCP AuthorizedThirdpartyExpeditedreview
    Intended Use

    The Aptima Herpes Simplex Viruses 1 & 2 assay (Aptima HSV 1 & 2 assay) is an in vitro diagnostic nucleic acid amplification test (NAAT), using real time transcription-mediated amplification (TMA), for the qualitative detection and differentiation of herpes simplex virus type 1 (HSV-1) and type 2 (HSV-2) messenger RNA (mRNA) in clinician-collected swab specimens from anogenital skin lesions. The assay is intended for use with swab specimens placed in Aptima specimen transport medium (STM) or in viral transport media (VTM) that is immediately diluted into STM.

    The Aptima HSV 1 & 2 assay is intended for use as an aid in the diagnosis of HSV-1 and/or HSV-2 infections in symptomatic male and female patients. The Aptima HSV 1 & 2 assay is indicated for use on the Panther® system.

    Device Description

    The Aptima Herpes Simplex Virus 1 & 2 assay (Aptima HSV assay) is a nucleic acid amplification test (NAAT) developed for use on the fully automated Panther system that utilizes target capture, transcription mediated amplification (TMA), and real-time detection of HSV-1, HSV-2, and an internal control (IC). The Aptima HSV assay amplifies and detects mRNAs for HSV-1 and HSV-2. These RNAs are expressed from the viral genome during the infection cycle, and are packaged inside HSV-1 and HSV-2 viral particles prior to virus release from infected cells. The Aptima HSV assay therefore detects virus-infected cells and the mature virus particles themselves.

    The Aptima HSV assay involves three main steps, which all take place in a single tube on the Panther® system: target capture, target amplification by TMA, and detection of the amplification products (amplicon) by the fluorescent labeled probes (torches). The assay incorporates an IC in every test to monitor targeted nucleic acid capture, amplification and detection.

    When the Aptima HSV assay is performed, the targeted viral mRNA and IC are isolated using magnetic microparticles and target-specific capture oligomers, in a process called target capture. The capture oligomers contain sequences complementary to specific regions of the targeted RNA (HSV mRNA or IC) as well as a string of deoxyadenosine residues. During the hybridization step, the sequence-specific regions of the capture oligomers bind to specific regions of the RNA target molecules. The microparticles, including the captured RNA target molecules bound to them, are pulled to the side of the reaction tube using magnets and the supernatant is aspirated. The particles are washed to remove residual specimen matrix that may contain amplification inhibitors. After target capture steps are completed, the specimens are ready for amplification.

    Target amplification occurs via TMA, which is a transcription-based nucleic acid amplification method that utilizes two enzymes, MMLV (Moloney murine leukemia virus) reverse transcriptase and T7 RNA polymerase. The reverse transcriptase is used to generate a DNA copy (containing a promoter sequence for T7 RNA polymerase) of the target sequence. T7 RNA polymerase produces multiple copies of RNA amplicon from the DNA copy template. Detection is achieved using single-stranded nucleic acid torches that are present during the amplification of the target and hybridize specifically to the amplicon in real time. Each torch has a fluorophore and a quencher. The quencher suppresses the fluorescence of the fluorophore as it is designed to be in close proximity when not hybridized to the amplicon. When the torch binds to the amplicon, the quencher is moved farther away from the fluorophore and it will emit a signal at a specific wavelength when excited by a light source. More torch hybridizes when more amplicon is present. The increase in fluorescent signal from progressive amplification is detected by fluorometers within the Panther system. The Panther system can detect and discriminate between the three fluorescent signals corresponding to HSV-1, HSV-2 and IC amplification products. The fluorescence (measured in relative fluorescence units [RFU]) is monitored over time to produce a real-time fluorescence emergence curve for each reporter dye. The Panther system software compares the fluorescence emergence curves to fixed cut off times to report results (TTime) for HSV-1, HSV-2 and IC.

    AI/ML Overview

    Here's an analysis of the acceptance criteria and the study proving the device meets them, based on the provided text:

    Acceptance Criteria and Device Performance for Aptima Herpes Simplex Viruses 1 & 2 Assay

    1. Table of Acceptance Criteria and Reported Device Performance

    The acceptance criteria are implied by the clinical performance targets presented in the study. While explicit pre-defined acceptance thresholds (e.g., "Sensitivity must be >90%") are not directly stated as pass/fail criteria, the reported performance metrics demonstrate the device's capability. For this analysis, I will use the clinical performance study results as the reported device performance against generally expected high standards for diagnostic accuracy.

    Note: The document does not explicitly state the pre-defined "acceptance criteria" numerical targets. The reported performance below represents the observed results of the clinical study, which presumably met the internal performance requirements for the manufacturer and FDA review.

    Table 1: Acceptance Criteria (Implied) and Reported Device Performance

    MetricTarget (Implied Acceptance)Reported Device Performance (Combined, VTM)Reported Device Performance (Combined, STM)
    HSV-1 SensitivityHigh sensitivity, typically >90% for diagnostic assays.93.4% (95% CI: 85.5-97.2)94.7% (95% CI: 87.1-97.9)
    HSV-1 SpecificityHigh specificity, typically >95-98% for diagnostic assays.99.8% (95% CI: 98.8 - >99.9)99.6% (95% CI: 98.4-99.9)
    HSV-2 SensitivityHigh sensitivity, typically >90% for diagnostic assays.96.9% (95% CI: 94.0-98.4)98.4% (95% CI: 96.1-99.4)
    HSV-2 SpecificityHigh specificity, typically >95-98% for diagnostic assays.97.5% (95% CI: 94.9-98.8)92.8% (95% CI: 89.1-95.3)
    ReproducibilityConsistent results across sites, operators, and reagent lots, especially at low concentrations.Agreement with expected results generally high, with some variability at concentrations near or below LoD (e.g., 46.3% - 100%).Agreement with expected results generally high, with some variability at concentrations near or below LoD (e.g., 46.3% - 100%).
    Limit of Detection (LoD)Low detection limit to ensure detection of low viral loads.HSV-1: 60.6-186.9 TCID50/mL (depending on strain/media)HSV-2: 18.2-128.8 TCID50/mL (depending on strain/media)
    Interfering SubstancesNo significant impact on assay sensitivity or specificity.No effect observed for tested substances at specified concentrations.No effect observed for tested substances at specified concentrations.
    Cross-ReactivityNo cross-reactivity with non-target microorganisms.No evidence of cross-reactivity or microbial interference (except for Streptococcus pneumoniae at 1x10^6 CFU/mL where cross-reactivity was observed).No evidence of cross-reactivity or microbial interference (except for Streptococcus pneumoniae at 1x10^6 CFU/mL where cross-reactivity was observed).
    Co-Infection DetectionAbility to detect both HSV-1 and HSV-2 when present.100% detection for both HSV-1 and HSV-2 in co-infected panels.100% detection for both HSV-1 and HSV-2 in co-infected panels.

    2. Sample size used for the test set and the data provenance

    • Clinical Test Set Sample Size:
      • Total Subjects: 544 evaluable subjects (195 males and 349 females).
      • Evaluated for HSV-1: 528 VTM specimens and 531 STM specimens.
      • Evaluated for HSV-2: 533 VTM specimens and 535 STM specimens.
    • Data Provenance:
      • Country of Origin: United States. The study was conducted at 17 US clinical sites.
      • Retrospective or Prospective: Prospective. The study is described as a "prospective, multicenter clinical study."

    3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts

    The document does not specify the number of experts used to establish the ground truth or their specific qualifications (e.g., "radiologist with 10 years of experience").

    However, it describes the methods used for the composite reference method:

    • ELVIS HSV ID and D3 Typing Test system viral culture
    • A validated bidirectional PCR/sequencing procedure
    • A third FDA-cleared assay for HSV-1 and HSV-2 was used for final composite reference interpretation when the initial methods disagreed or when PCR/sequencing detected both types.

    This suggests that the ground truth was established through a combination of highly reliable laboratory tests, implying a rigorous approach to defining true positive/negative cases, rather than relying solely on individual expert interpretation without further clarification.

    4. Adjudication method (e.g., 2+1, 3+1, none) for the test set

    The document describes an adjudication method for the ground truth:

    • "A third FDA-cleared assay for HSV-1 and HSV-2, was used to determine the final composite reference interpretation when the ELVIS D3 culture and PCR/sequencing results did not agree on the type of HSV detected or when PCR/sequencing detected both HSV-1 and HSV-2."

    This indicates a hierarchical or tie-breaking system rather than a simple 2+1 or 3+1 consensus among human readers. It relies on a "composite reference method" combining results from multiple validated laboratory tests.

    5. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance

    This is a diagnostic assay for detecting viral RNA, not an imaging device that involves human readers interpreting images with or without AI assistance. Therefore, no MRMC comparative effectiveness study involving human readers with AI assistance was performed or reported in this submission. The device (Aptima HSV 1 & 2 Assay) is designed to provide a direct qualitative result (positive/negative for HSV-1 and/or HSV-2) from a processed specimen.

    6. If a standalone (i.e. algorithm only without human-in-the loop performance) was done

    Yes, a standalone performance study was done in the sense that the Aptima HSV 1 & 2 Assay (the "algorithm" in this context) directly processes specimens and generates results without requiring human interpretation for its output. The clinical performance study directly evaluated the accuracy of the assay's results against a composite reference standard. The "human-in-the-loop" would be the clinician collecting the sample and laboratory technicians running the test and reporting the results, but the analytical output itself is determined by the assay system.

    7. The type of ground truth used (expert consensus, pathology, outcomes data, etc.)

    The ground truth used was a composite reference method combining:

    • ELVIS HSV ID and D3 Typing Test system viral culture
    • A validated bidirectional PCR/sequencing procedure
    • A third FDA-cleared assay for HSV-1 and HSV-2 (used for tie-breaking/discrepancy resolution)

    This is a robust form of ground truth based on multiple established laboratory diagnostic methods.

    8. The sample size for the training set

    The document does not report a sample size for a training set. This is expected for a diagnostic assay of this type, as it's not a machine learning or AI algorithm that requires a separate "training set" in the conventional sense. The assay's design and optimization (e.g., probe sequences, amplification conditions) would have been developed iteratively, but a distinct "training set" for performance evaluation is not applicable here. The analytical studies (LoD, cross-reactivity, etc.) and the clinical performance study represent the validation of the finalized assay.

    9. How the ground truth for the training set was established

    As there is no explicit "training set" in the context of machine learning, this question is not directly applicable. The assay's components and parameters would have been optimized using internal development processes and validated through analytical studies. For these analytical studies (e.g., LoD, cross-reactivity), the "ground truth" (i.e., known-positive or known-negative samples, specific viral strains/concentrations) would have been established through well-characterized laboratory standards, spiked samples, and reference materials.

    Ask a Question

    Ask a specific question about this device

    K Number
    K122062
    Manufacturer
    Date Cleared
    2013-01-09

    (180 days)

    Product Code
    Regulation Number
    866.3860
    Reference & Predicate Devices
    Why did this record match?
    Reference Devices :

    K111409

    AI/MLSaMDIVD (In Vitro Diagnostic)TherapeuticDiagnosticis PCCP AuthorizedThirdpartyExpeditedreview
    Intended Use

    The APTIMA Trichomonas vaginalis Assay is an in vitro qualitative nucleic acid amplification test (NAAT) for the detection of ribosomal RNA (rRNA) from Trichomonas vaginalis to aid in the diagnosis of trichomoniasis using the PANTHER System.

    The assay may be used to test the following specimens from symptomatic or asymptomatic women: clinician-collected endocervical swabs, clinician-collected vaginal swabs, and specimens collected in PreservCyt Solution.

    Device Description

    The ATV Assay with the modified TCR, referred to as ATV Assay (Version 2) in this submission is the subject of this premarket notification. The ATV Assay (Version 2) is similar to the ATV Assay originally cleared (ref: K102911), except for the formulation of the TCR. The TCR is a HEPES-buffered solution containing lithium salts and derivatized magnetic beads. A second target capture oligo was added to the TCR formulation in order to accommodate future specimen types.

    The TCR modification did not result in the change of assay technology. The ATV Assay (Version 2) uses Target Capture (TC), Transcription Mediated Amplification (TMA), and Hybridization Protection Assay (HPA) technologies to qualitatively detect ribosomal RNA (rRNA) from Trichomonas vaginalis. The overall assay design as well as the assay procedural steps remain unchanged from that previously described in the original 510(k) for the ATV Assay (K102911).

    The ATV Assay (Version 2) kit is comprised of 3 boxes:

    1. Refrigerated Box Contains the Amplification Reagent, Enzyme Reagent, Probe Reagent and Target Capture Reagent-B
    2. Room Temperature Box Contains Amplification Reconstitution Solution, Enzyme Reconstitution Solution, Probe Reconstitution Solution, Selection Reagent and Target Capture Reagent
    3. Controls Box Contains the Negative and Positive Controls

    The ATV Assay (Version 2) on PANTHER would utilize three specimen collection kits. These collection kits were cleared for use with the originally cleared ATV Assay and other commercialized APTIMA Assays.

    1. APTIMA Unisex Swab Specimen Collection Kit for Endocervical and Male Urethral Swab Specimens
    2. APTIMA Vaginal Swab Specimen Collection Kit
    3. APTIMA Specimen Transfer Kit

    Instrumentation
    The ATV Assay (Version 2) was validated using the PANTHER System, which was previously cleared in May 2012 (Ref: K111409).

    AI/ML Overview

    Acceptance Criteria and Device Performance Study for APTIMA® Trichomonas vaginalis Assay (PANTHER® System)

    This section provides a summary of the acceptance criteria and the study conducted to demonstrate the APTIMA® Trichomonas vaginalis Assay on the PANTHER® System meets these criteria.

    1. Table of Acceptance Criteria and Reported Device Performance

    The primary acceptance criteria for this diagnostic device are its clinical sensitivity and specificity across different specimen types and symptom statuses.

    Performance MetricSpecimen TypeSymptom StatusAcceptance Criteria (Implicit from prior clearance/predicate, demonstrated as 95% Confidence Interval)Reported Device Performance (95% CI)
    SensitivityClinician-collected Vaginal Swab (CVS)AsymptomaticExpected to be high (e.g., >80% or 90%)100% (75.8-100)
    SymptomaticExpected to be high100% (93.7-100)
    OverallExpected to be high100% (94.7-100)
    Endocervical Swab (ES)AsymptomaticExpected to be high100% (80.6-100)
    SymptomaticExpected to be high100% (93.0-100)
    OverallExpected to be high100% (94.6-100)
    PreservCyt Solution liquid Pap (PCyt)AsymptomaticExpected to be high100% (83.2-100)
    SymptomaticExpected to be high100% (94.3-100)
    OverallExpected to be high100% (95.6-100)
    SpecificityClinician-collected Vaginal Swab (CVS)AsymptomaticExpected to be high (e.g., >95%)97.3% (94.6-98.7)
    SymptomaticExpected to be high98.8% (97.0-99.5)
    OverallExpected to be high98.2% (96.7-99.0)
    Endocervical Swab (ES)AsymptomaticExpected to be high98.3% (96.1-99.3)
    SymptomaticExpected to be high97.9% (95.8-99.0)
    OverallExpected to be high98.1% (96.7-98.9)
    PreservCyt Solution liquid Pap (PCyt)AsymptomaticExpected to be high99.4% (97.7-99.8)
    SymptomaticExpected to be high97.9% (95.9-98.9)
    OverallExpected to be high98.6% (97.4-99.2)

    Note: The document does not explicitly state numerical acceptance criteria in a structured table. However, the reported performance characteristics (Sensitivity, Specificity, PPV, NPV) with narrow 95% Confidence Intervals consistently demonstrate high agreement with the "patient infected status algorithm," indicating that the device performs as expected for a diagnostic test of this nature, meeting an implicit acceptance threshold for high diagnostic accuracy. The agreement studies with the predicate device further support this.

    2. Sample Sizes Used for the Test Set and Data Provenance

    The clinical performance study used the following sample sizes for the test set:

    • Vaginal Swabs: 667 (after exclusions for invalid results which were 11 out of 689 initial samples)
    • Endocervical Swabs: 700 (after exclusions for invalid results which were 24 out of 737 initial samples)
    • PreservCyt Solution liquid Pap specimens: 774 (after exclusions for invalid results which were 1 out of 791 initial samples)

    Data Provenance: The study utilized retrospective, leftover specimens collected from consenting subjects during a previous, prospective, multicenter clinical study of the ATV Assay on the TIGRIS DTS System.

    • Country of Origin: 9 US clinical sites (obstetrics and gynecology, family planning, and STD clinics).

    For the agreement study with the TIGRIS DTS System for asymptomatic subjects:

    • Vaginal Swabs: 227
    • Endocervical Swabs: 227
    • PreservCyt Solution liquid Pap specimens: 226
    • Data Provenance: Prospectively collected specimens from asymptomatic subjects enrolled from 6 US clinical sites.

    3. Number of Experts Used to Establish Ground Truth and Qualifications

    The document does not specify the number of experts or their specific qualifications (e.g., years of experience for radiologists) for establishing the ground truth for the clinical study.

    4. Adjudication Method for the Test Set

    The ground truth for the clinical performance study was established by a patient infected status algorithm based on the results from two reference tests performed on vaginal swab specimens:

    1. Commercially available culture system
    2. Wet mount microscopic examination

    Adjudication Rule:

    • Infected Patient Status: At least one of the reference test results was required to be positive.
    • Non-infected Patient Status: Both reference tests were required to be negative.

    5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

    No Multi-Reader Multi-Case (MRMC) comparative effectiveness study was done. This is a diagnostic assay (Nucleic Acid Amplification Test) and its performance is determined by its analytical and clinical characteristics against a defined ground truth, not by human reader interpretation. No human readers are involved in the direct interpretation of the assay results, which are automatically interpreted by the PANTHER System software.

    6. Standalone Performance Study (Algorithm Only)

    Yes, a standalone performance study (algorithm only, without human-in-the-loop performance) was done. The entire evaluation of the APTIMA® Trichomonas vaginalis Assay on the PANTHER® System, including its analytical and clinical performance, is based on the automated interpretation of the test results by the PANTHER System's software. The assay results (RLU values) are automatically interpreted as negative, positive, or invalid by the system.

    7. Type of Ground Truth Used

    The type of ground truth used for the clinical studies was an expert consensus-based algorithm derived from established diagnostic methods:

    • Culture for Trichomonas vaginalis
    • Wet mount microscopic examination

    This algorithm defined the "patient infected status" against which the device's performance was measured.

    8. Sample Size for the Training Set

    The document does not explicitly state the sample size for a "training set" in the context of developing the algorithm itself. The information provided focuses on the validation data set used for clinical performance evaluation. Medical devices, especially diagnostic assays, often undergo development and internal validation on various sample sets, but these are typically not reported as explicitly as training sets in the context of machine learning model development. The focus here is on the analytical and clinical validation of the final assay.

    9. How the Ground Truth for the Training Set was Established

    As noted above, an explicit "training set" for the algorithm's development is not detailed. The ground truth for validating the assay's performance (the clinical test set) was established using a patient infected status algorithm based on a combination of culture and wet mount microscopic examination results. This is a common practice for validating new diagnostic tests against existing gold standards or established diagnostic pathways.

    Ask a Question

    Ask a specific question about this device

    Page 1 of 1