Search Results

Ask a specific question about this device

The Visby Medical Respiratory Health Test is a single-use (disposable), fully integrated, automated Reverse Transcription Polymerase Chain Reaction (RT-PCR) in vitro diagnostic test intended for the simultaneous qualitative detection and differentiation of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), influenza A, and influenza B RNA in nasopharyngeal swab and anterior nasal swab specimens from individuals with signs and symptoms of respiratory tract infection. Clinical signs and symptoms of respiratory tract infection due to SARS-CoV-2, influenza A, and influenza B can be similar.

The Visby Medical Respiratory Health Test is intended for use as an aid in the differential diagnosis of SARS-CoV-2, influenza A, and influenza B infection if used in conjunction with other clinical and epidemiological information, and laboratory findings. SARS-CoV-2, influenza A, and influenza B viral RNA are generally detectable in nasopharyngeal swab and anterior nasal swab specimens during the acute phase of infection. This test is not intended to detect influenza C virus infections.

Positive results are indicative of the identified virus, but do not rule out bacterial infection or co-infection with other organisms not detected by the test. The agent(s) detected by the Visby Medical Respiratory Health Test may not be the definitive cause of disease. Negative results do not preclude SARS-CoV-2, influenza A, or influenza B infection. The results of this test should not be used as the sole basis for diagnosis, treatment, or other patient management decisions.

The Visby Medical Respiratory Health Test is a single-use (disposable), fully integrated, compact device containing a reverse transcription polymerase chain reaction (RT-PCR) based assay for qualitative detection of influenza B, and/or SARS-CoV-2 viral RNA in upper respiratory tract specimens. The device automatically performs all steps required to complete lysis, reverse transcription (RT), PCR amplification, and detection.

Specimen collected using nasopharyngeal (NP) or anterior nasal (AN) swabs (without transport media) are placed in the Visby Medical Respiratory Health Buffer and then transferred into the sample port of the device using the provided fixed volume pipette. The sample enters a lysis module and rehydrates the RT enzyme and RT primers. The mixture then moves through a sample preparation module where viruses and human cells are simultaneously lysed, and RNA is reverse transcribed. The resulting fluid (containing cDNA) is then mixed with lyophilized PCR reagents containing the DNA polymerase enzyme and PCR primers. The PCR mixture (containing cDNA template and reagents) is then thermal cycled to amplify the targets, including human beta-2 microglobulin (B2M) RNA, which serves as a process control. After PCR, the biotinylated product is hybridized to covalently bound capture probes at specific locations along a flow channel. The flow channel is configured to facilitate an enzymatic reaction that uses streptavidin bound horseradish peroxidase (HRP) and a colorimetric substrate that forms a purple precipitate. The operator observes a color change at the specific locations indicating the presence of an amplified target. Test results can be expected in approximately 30 minutes: illumination of a "DONE" status light on the front of the device and a purple color in the "RESULTS VALID" spot, indicate a successful test. A purple spot adjacent to "Flu A", "Flu B", and/or "COVID-19" signifies the presence of, influenza A, influenza B, and/or SARS-CoV-2 viral RNA.

Here's an analysis of the acceptance criteria and study detailed in the provided text:

Acceptance Criteria and Reported Device Performance

The document doesn't explicitly state "acceptance criteria" in a separate table. However, based on the Summary of Performance Data, the inferred acceptance criteria are the achieved Positive Percentage Agreement (PPA) and Negative Percentage Agreement (NPA) values in the clinical evaluation. The study aims to demonstrate substantial equivalence to the predicate device, implying that the performance needs to be comparable or better.

Here's a table summarizing the reported device performance, which implicitly represents the met acceptance criteria:

Study Details:

• Sample size used for the test set and the data provenance:

  • Sample Size: A total of 1,501 subjects were included in the performance analysis after exclusions. 1,575 Visby tests were initially performed.
  • Data Provenance: Prospectively collected fresh specimens from subjects presenting with signs and symptoms of a viral respiratory infection at five CLIA Waived study sites in the US (urgent care and family care clinics). Specimens were collected and tested between May 2022 and Feb 2024.

• Number of experts used to establish the ground truth for the test set and the qualifications of those experts:

  • The document does not specify the number or qualifications of experts used to establish ground truth for the test set. Instead, it states that the comparator assays define the ground truth.

• Adjudication method (e.g., 2+1, 3+1, none) for the test set:

  • The document describes comparing the Visby Medical Respiratory Health Test results against an "FDA-cleared influenza molecular test and an FDA-EUA authorized SARS-CoV-2 RT-PCR test as a comparator." It also mentions "alternate molecular assay" for discordant results (footnotes a, b, e, f, g, h, i, j in Table 2). This indicates that the ground truth was established by these comparator assays, potentially with some form of reference-standard-based comparison and possibly resolution of discrepancies with alternate assays, rather than a multi-expert adjudication method on the test set itself.

• If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:

  • No, an MRMC comparative effectiveness study was not done. This device is an in vitro diagnostic (IVD) test for molecular detection of viruses, not an AI-assisted diagnostic imaging or human-read interpretation system. The "operators" in the reproducibility study were "non-laboratorians representing healthcare professionals," but their performance was evaluated against expected results for spiked samples, not in comparison to their own performance with and without an AI assistant on clinical cases.

• If a standalone (i.e., algorithm only without human-in-the-loop performance) was done:

  • Yes, a standalone performance evaluation was primarily done for the device. The clinical study evaluated the device's ability to detect viral RNA in specimens. While "typical CLIA Waived operators" performed the test, their role was to execute the device's protocol, and the device's detection accuracy was then compared against the comparator assays. The "device automatically performs all steps required to complete lysis, reverse transcription (RT), PCR amplification, and detection," implying it functions as a standalone diagnostic unit once the sample is loaded.

• The type of ground truth used (expert consensus, pathology, outcomes data, etc.):

  • The ground truth for the clinical evaluation was established by comparator molecular assays: an "FDA-cleared influenza molecular test" and an "FDA-EUA authorized SARS-CoV-2 RT-PCR test." Discordant results were sometimes further investigated with an "alternate molecular assay."

• The sample size for the training set:

  • The document does not specify a separate training set or its sample size. For IVD devices like this RT-PCR test, the "training" typically refers to the assay development and optimization process in the lab, rather than a distinct training set in the way AI/ML models are trained. The clinical performance study evaluates the final, optimized device.

• How the ground truth for the training set was established:

  • As no distinct "training set" in the context of an AI/ML model is described, this information is not applicable. The development and optimization of such a diagnostic test would involve analytical studies (e.g., LoD, inclusivity, cross-reactivity) where the "ground truth" for those specific experiments (e.g., known concentrations of viral targets) is established by careful spiking and molecular characterization in a laboratory setting.

Ask a specific question about this device

The Visby Medical Sexual Health Test is a single-use (disposable), fully-integrated, automated Polymerase Chain Reaction (PCR) in vitro diagnostic test intended for use in point-of-care or clinical laboratory settings for the rapid detection and differentiation of DNA from Chlamydia trachomatis, Neisseria gonorrhoeae, and Trichomonas vaginalis in self-collected female vaginal swab specimens using the Visby Medical Sexual Health Vaginal Specimen Collection Kit in a health care setting. The test results are to aid in the diagnosis of symptomatic infections with Chlamydia trachomatis, Neisseria gonorrhoeae, and Trichomonas vaginalis.

The Visby Medical Sexual Health Test (Visby Test) is a single-use (disposable), fully integrated, rapid, compact device containing a PCR-based assay for direct qualitative detection and differentiation of DNA from CT, NG, and TV. The test system includes the Visby Medical Sexual Health device, the Visby Medical power supply, the Visby Medical Vaginal Specimen Collection kit, and fixed-volume transfer pipettes. The device processes self-collected vaginal swab samples by automatically performing all steps required to complete lysis, polymerase chain reaction, and amplicon detection.

The patient uses the Visby Medical Vaginal Specimen Collection Kit to self-collect a vaginal specimen with the provided flocked swab, and then the patient elutes the specimen into the Visby Medical Collection Media. The test operator transfers the collection media containing the patient specimen into the sample port of the device using the provided fixed-volume pipette where it rehydrates a lyophilized internal process control. The sample enters a lysis module, where the DNA of the sample and the internal process control are extracted using a combination of chemical lysis and high temperature. The extracted DNA enters a mixing chamber where it rehydrates lyophilized PCR reagents, followed by thermocycling to amplify target DNA. If present, the amplified pathogen target (CT, NG, and/or TV) and internal process control hybridize to specific probes located on a flow channel. Detection of the target-specific PCR product is accomplished via an enzyme-linked colorimetric assay using streptavidin bound horseradish peroxidase (HRP) and a colorimetric substrate that forms a purple precipitate. Test results can be expected in approximately 30 minutes: a green check mark will appear, and a purple color will appear in the "Results Valid" spot, indicating a valid test. A purple spot adjacent to "Chlamydia," "Gonorrhea," and/or "Trichomoniasis" signifies the presence of amplified CT, NG, and/or TV DNA in the sample.

Vaginal self-collection kits containing a tube of collection media compatible with the Sexual Health test and a collection swab are also packaged separately. External controls recommended in the product labeling are commercially available from a different manufacturer.

Here's a breakdown of the acceptance criteria and study information based on the provided document:

1. Table of Acceptance Criteria and Reported Device Performance

The document does not explicitly state formal acceptance criteria in terms of specific PPA/NPA thresholds for substantial equivalence to the predicate device. Instead, the studies presented aim to demonstrate "comparability" or "equivalence" to the predicate device (Visby Medical Sexual Health Click Test, K200748/CW200003). Therefore, the reported performance is compared directly against the predicate where applicable.

Based on the "Clinical Performance of the Visby Test compared to the Click test performed by untrained operators" (Table 3) and "Performance of Visby Test Compared to Click Test - Clinical Specimens" (Table 4), the reported device performance is:

Note: The acceptance criteria for the LoD comparison was explicitly stated: "The LoD values were determined to be equivalent if the lowest concentrations of organism with at least 19/20 detection rate from the two devices were within 3-fold of each other." As seen in Table 1, this criterion was met for all organisms.

2. Sample Size Used for the Test Set and Data Provenance

• LoD Comparison: Not explicitly stated as a "test set" in the traditional sense for PPA/NPA. For each LoD multiple, 20 Visby devices and 20 Click devices were tested. The data provenance appears to be laboratory-generated (spiked samples in negative pooled clinical vaginal sample).
• Reproducibility Study: Seven (7) panel members (negative, low positive, moderate positive for CT, NG, TV) were used. Each panel member was tested by 6 operators (2 per site) three times each day over 6 non-consecutive days, across 3 reagent lots. This results in $7 \times 6 \times 3 \times 6 = 756$ individual test results reported (though the table summarizes 108 tests per panel member for overall agreement, implying 108 replicates per panel across sites/operators/days). Data provenance is likely laboratory-contrived samples (cultured organisms in negative pooled clinical vaginal swab matrix).
• Multicenter Study with Untrained Operators:
  • Sample Size: 102 samples initially selected; 1 was excluded, so 101 samples were included in the final analysis (30 CT positive, 20 NG positive, 30 TV positive, 33 negative based on comparator results, though the sum of these is 113, implying some overlap or specific positivity counts within the 101 total).
  • Data Provenance: De-identified, frozen, self-collected vaginal swab specimens that are a subset of patient samples previously characterized in the clinical study for the Click Test. This indicates retrospective human clinical data from the U.S. (implied by FDA submission context, though not explicitly stated for this particular subset).
• Single Center Study with Trained Operators:
  • Sample Size: 359 specimens initially; 1 excluded due to insufficient volume, 7 excluded due to retest issues. Final analysis included 351 specimens.
  • Data Provenance: De-identified, frozen, archived self-collected vaginal swab specimens in Visby Medical Collection Media. These included specimens from the original clinical study of the Click Test and from two other previous Visby Medical studies. This indicates retrospective human clinical data, likely from the U.S.
• Contrived Specimens Spiked at Low Organism Concentrations:
  • Sample Size: 80 contrived samples (20 CT positive, 20 NG positive, 20 TV positive, and 20 negative).
  • Data Provenance: Laboratory-generated, spiked samples using individual clinical matrices.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications

The document mentions that for the Multicenter Study and Single Center Study:

• The samples were "previously characterized in the clinical study for the Click Test" and "based on comparator results from the clinic study for Click Test."
• "TV PIS (based on the original clinical study for Click Test) for all seven specimens were negative."

This strongly implies that the ground truth for these clinical test sets was established during the clinical studies for the predicate device (Click Test). The report for the current device (Visby Test) does not detail the specific number or qualifications of experts for establishing the ground truth for these retrospective samples, as that would have been documented in the original predicate device's submission. It refers to "comparator results," which typically means a highly sensitive and specific reference method (e.g., PCR with bidirectional sequencing, culture followed by definitive identification) performed and interpreted by qualified laboratory personnel, rather than a panel of clinical experts adjudicating cases for complex imaging or clinical diagnoses.

For the LoD and Contrived samples studies, the "ground truth" is known by design, as the samples are deliberately spiked with known concentrations of organisms.

4. Adjudication Method for the Test Set

The document does not describe a formal adjudication method (like 2+1 or 3+1) for establishing the ground truth in the context of clinical samples.

• For the multicenter and single-center clinical studies, the reference standard (ground truth) was based on "comparator results" from the original clinical studies of the predicate device. This typically implies a highly accurate lab-based method.
• For the reproducibility, LoD, and contrived sample studies, the ground truth was known by design (known presence/absence and concentration of spiked organisms).

5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance

This device is an in vitro diagnostic (IVD) test that directly detects nucleic acids via PCR and provides a visually interpreted result. It is not an AI-based imaging analysis tool or a system that "assists human readers" in the way an MRMC study would typically evaluate for AI in radiology or pathology. The "operators" are interpreting the device's output (presence of a purple spot), not interpreting complex data that AI might enhance. Therefore, an MRMC comparative effectiveness study for human readers with and without AI assistance is not applicable and was not done.

The studies performed evaluate the performance of the device itself when operated by different user groups (trained vs. untrained) and compared to a predicate device.

6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) was done

The Visby Medical Sexual Health Test is a fully integrated, automated PCR diagnostic test that provides a visual result (purple spot). The results are then visually interpreted by an operator ("Operator visually interprets test results" - pg 6). While the internal PCR process is automated, the final "reading" is done by a human looking at the colorimetric output. Therefore, it's not a purely "algorithm-only" device without human intervention for result interpretation. However, the performance data presented (PPA/NPA) reflects the device's accuracy in producing the correct visual output, as subsequently interpreted by a human operator. The studies demonstrate the performance of the device system including human interpretation in different use environments.

7. The Type of Ground Truth Used

• Clinical Comparison Studies (Multicenter and Single Center): The ground truth was established by "comparator results" from the original clinical studies of the predicate device (Visby Medical Sexual Health Click Test). This typically refers to a highly accurate laboratory-based reference method, such as a laboratory-developed test (LDT) using PCR with sequencing confirmation, or culture followed by definitive identification, considered the gold standard for detecting these pathogens.
• LoD, Reproducibility, and Contrived Samples Studies: The ground truth was established by design, using cultured organisms spiked at known concentrations into negative clinical matrices.

8. The Sample Size for the Training Set

The document describes performance data for the Visby Medical Sexual Health Test, which is an "updated version of the Visby Medical Sexual Health Click test (Click Test, the predicate)." It explicitly states that "The Visby Test has no changes to the PCR primer and probe sequences, reagent formulations, detection method, or result interpretations."

Since this is an updated version of an existing device with no changes to the core diagnostic method (PCR primers, probes, detection), it's highly unlikely that a separate "training set" in the context of machine learning model development was used for this specific device. The document focuses on demonstrating equivalence to the predicate device. If the predicate device (Click Test) involved any form of algorithm development requiring a training set, that would have been part of its original 510(k) submission (K200748/CW200003). For the current submission, the studies are primarily verification and validation, showing that the new manufacturing process and simplified user interface do not negatively impact performance compared to the predicate.

Therefore, a sample size for a training set for the Visby Medical Sexual Health Test is not applicable / not provided in this document.

9. How the Ground Truth for the Training Set Was Established

As mentioned in point 8, a "training set" for an algorithm is not described or applicable to the type of update this device represents. If the predicate device (Click Test) had a training set, the ground truth for that would have been established according to its original clearance. This document focuses on demonstrating equivalence of the updated physical device.

Ask a specific question about this device

The Visby Medical Sexual Health Click Test is a single-use (disposable), fully-integrated, automated Polymerase Chain Reaction (PCR) in vitro diagnostic test intended for use in point of-care or clinical laboratory settings for the rapid detection and differentiation of DNA from Chlamydia trachomatis, Neisseria gonorrhoeae, and Trichomonas vaginalis in seff-collected female vaginal swab specimens using the Visby Medical Sexual Health Vaginal Specimen Collection Kit in a health care setting. The test results are to aid in the diagnosis of symptomatic infections with Chlamydia trachomatis, Neisseria gonorrhoeae, and Trichomonas vaginalis.

The test system includes the Visby Medical Sexual Health Click device, the Visby Medical power supply, the Visby Medical Vaginal Collection kit, and fixed-volume transfer pipettes. The device processes a vaginal swab sample by automatically performing all steps required to complete lysis, polymerase chain reaction, and amplicon detection.

The patient uses the Visby Medical Vaginal Collection Kit to self-collect a vaginal specimen with the provided flocked swab, and then the patient elutes the specimen into the Visby Medical Collection Media. The test operator transfers the collection media containing the patient specimen into the sample port of the device using the provided fixed-volume pipette where it rehydrates a lyophilized internal process control. The sample enters a lysis module, where the DNA of the sample and the internal process control are extracted using a combination of chemical lysis and high temperature. The extracted DNA enters a mixing chamber where it rehydrated lyophilized PCR reagents, followed by thermocycling to amplify target DNA. If present, the amplified pathogen target (CT, NG, and/or TV) and internal process control hybridize to specific probes located on a flow channel. Detection of the target-specific PCR product is accomplished via an enzyme-linked colorimetric assay using streptavidin bound horseradish peroxidase (HRP) and a colorimetric substrate that forms a purple precipitate. Test results can be expected in approximately 30 minutes: a green check mark will appear, and a purple color will appear in the "Control" spot, indicating a valid test. A purple spot adjacent to "Chlamydia," "Gonorrhoeae," and/or "Trichomonas" signifies the presence of amplified CT, NG, and/or TV DNA in the sample. Tests with invalid results due to an error (indicated by a red X status light) or failure to develop a purple spot in the "Control" spot, are retested with a new Visby device.

The Visby Medical Sexual Health Click Test is an automated Polymerase Chain Reaction (PCR) in vitro diagnostic test for the rapid detection and differentiation of DNA from Chlamydia trachomatis (CT), Neisseria gonorrhoeae (NG), and Trichomonas vaginalis (TV) in self-collected female vaginal swab specimens.

Here's an analysis of its acceptance criteria and the study that proves its performance:

1. Table of Acceptance Criteria and Reported Device Performance

The acceptance criteria are implicitly defined by the clinical performance observed in the study, aiming for high sensitivity and specificity. The reported device performance is based on the achieved Positive Percent Agreement (PPA) and Negative Percent Agreement (NPA) or Sensitivity and Specificity against a Composite Comparator Result (CCR) or Patient Infected Status (PIS).

2. Sample Size Used for the Test Set and Data Provenance

• Test Set Sample Size:
  • CT: 1774 subjects
  • NG: 1786 subjects
  • TV: 1765 subjects
  • Initially, 1899 subjects were enrolled, 1881 were eligible, and 1789 were included in the data analysis.
• Data Provenance: The study was conducted at 14 clinical sites geographically distributed across the United States. The study subjects were prospectively enrolled females.

3. Number of Experts Used to Establish Ground Truth and Their Qualifications

• The document does not specify the number of experts explicitly used to establish the ground truth.
• The ground truth was established by a Composite Comparator Result (CCR) for CT and NG, and a Patient Infected Status (PIS) algorithm for TV, which were comprised of three FDA-cleared NAAT assays.
• The qualifications of the individuals interpreting the results of these FDA-cleared NAAT assays are not explicitly stated, but it can be inferred that they would be trained laboratory personnel given the nature of NAAT testing.

4. Adjudication Method for the Test Set

• Adjudication Method: For all three organisms (CT, NG, TV), the ground truth (CCR/PIS) was determined as follows:
  • A study participant was considered infected if a positive result was reported from two of the three FDA-cleared NAAT tests.
  • If the test results of the first two NAATs were discordant (one positive, one negative), the CCR/PIS was determined by the third NAAT. This is often referred to as a "2 out of 3" or "tie-breaker" adjudication method.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

• No MRMC comparative effectiveness study was done in the context of human readers improving with or without AI assistance. This device is a fully-automated in vitro diagnostic test, and the clinical study evaluated the device's performance directly against a laboratory-based ground truth, not human reader performance.
• However, a reproducibility study was conducted with "untrained users" (non-laboratorians) in CLIA waived settings, to assess the device's performance when used by typical healthcare professionals in such environments. This indirectly assesses the device's usability and reliability in a real-world setting where "human readers" (the operators) process samples and interpret simple visual results (green check mark for valid, purple spots for positive). The study demonstrated robust reproducibility and that untrained users could perform and interpret results accurately, but it wasn't a comparative effectiveness study of human reader improvement.

6. Standalone Performance Study (Algorithm Only Without Human-in-the-Loop Performance)

• Yes, the clinical performance described in section H is a standalone performance study (device only without human-in-the-loop performance influencing the result generation). The device processes a vaginal swab sample fully-integrated and automatically, performing lysis, PCR, and amplicon detection, and then provides a visual result (a green check mark for valid and purple spots for positive). The study evaluated how this automated device's results compared to the established ground truth.
• While human operators handled the samples and initiated the test, the test interpretation is designed to be straightforward and automatic (visual detection of colored spots), making the clinical performance metrics largely reflective of the algorithm's standalone capability to detect the presence of pathogens. The reproducibility study explicitly confirmed that "untrained users can perform the test and interpret the results accurately."

7. Type of Ground Truth Used

• The ground truth used was a Composite Comparator Result (CCR) for CT and NG, and a Patient Infected Status (PIS) algorithm for TV.
• This ground truth was derived from the results of three FDA-cleared NAAT (Nucleic Acid Amplification Test) assays. This is a common and robust method for establishing ground truth in molecular diagnostics, as NAATs are highly sensitive and specific.

8. Sample Size for the Training Set

• The document focuses on the validation of the device through non-clinical and clinical studies. It does not specify the sample size for a training set.
• As an in vitro diagnostic (IVD) device, its development likely involves extensive internal optimization and testing (which could be considered analogous to "training" in the context of AI/ML, though this device is PCR-based with colorimetric detection, not explicitly a machine learning algorithm) using characterized samples and analytical performance studies (Limit of Detection, Inclusivity, Cross-Reactivity, etc.). These analytical studies serve as a rigorous "training" and development phase, but specific "training set sizes" as might be reported for a machine learning model are not applicable or provided here.

9. How the Ground Truth for the Training Set Was Established

• Since a specific "training set" with established ground truth as in AI/ML model development is not explicitly mentioned or applicable to this type of PCR-based diagnostic, this question cannot be directly answered from the provided text.
• However, the analytical studies (LoD, Inclusivity, Cross-Reactivity, Competitive Interference, Interfering Substances, Reproducibility) described in section G involve using well-characterized organisms (known strains/serovars with specified concentrations) spiked into negative clinical vaginal sample matrix. The "ground truth" for these analytical samples is the known presence/absence and concentration of the spiked organisms. This rigorous analytical characterization serves to ensure the robust performance of the assay.

Ask a specific question about this device