Search Results

CHLOE BLAST is indicated to provide adjunctive information on events occurring during embryo development that may predict further development to the blastocyst stage on Day 5 of development. This adjunctive information aids in the selection of embryo(s) for transfer on Day 3, when, following morphological assessment, there are multiple embryos deemed suitable for transfer or freezing.

CHLOE BLAST is to be used only for the analysis of images captured by the EmbryoScope version D incubator system.

CHLOE BLAST is a decision support tool designed to automatically analyze time lapse videos of developing embryos, retrieved from EmbryoScope (version D) Time Lapse Incubators (TLI) system. It is intended to provide adjunctive information on developmental events up to Day 3 that may predict progression to the blastocyst stage by Day 5.

CHLOE BLAST is a cloud-based software as a medical device (SaMD) that uses a convolutional neural network (CNN) to analyze TLI videos from insemination to Day 3. The output is the "CHLOE Score", which is a blastocyst development prediction value associated with the likelihood of the embryo reaching blastocyst stage at Day 5.

This information aids in the selection of embryo(s) for transfer on Day 3, when, following morphological assessment, there are multiple normally fertilized embryos deemed suitable for transfer or freezing. In a clinical setting, the CHLOE score is intended to be used by the embryologist as adjunctive information, to be used only after the embryologists complete their independent morphological assessments based on the lab's standard of care (e.g., Istanbul Consensus Grading).

The main user interaction is via the graphic user interface (GUI) available via Chrome browsers. It includes screens for treatments overview, manual embryo assessment, and score presentation, and integrates with the day-to-day normal operation in IVF clinics using TLI.

Here's a breakdown of the acceptance criteria and the study proving the device meets those criteria, based on the provided FDA clearance letter:

Acceptance Criteria and Device Performance

1. Table of Acceptance Criteria and Reported Device Performance

Note: The document presents acceptance criteria primarily as "AUC lower bound >0.8" for various performance metrics. It also establishes an Odds Ratio (OR) greater than 1 as the primary endpoint for clinical utility.

Study Details Proving Device Meets Acceptance Criteria

2. Sample Sizes and Data Provenance

• Non-Clinical Performance (Algorithm Validation):
  • Morphokinetic Events Detection: 1,094 embryos from 143 slides. Collected from two sites: one in the US and one in Norway. The data provenance is retrospective, as it's a "test dataset... entirely independent from the dataset utilized in the CHLOE BLAST clinical study."
  • Blast Prediction: 1,726 embryos from 233 slides. Collected from two sites: one in the US and one in Norway. The data provenance is retrospective.
• Clinical Performance (CHLOE BLAST Clinical Study):
  • 703 embryos from 59 mothers.
  • Data collected from three different sites located in the United States.
  • Data provenance: Prospective collection for the purpose of this study (described as a "pivotal, multicenter, single arm, observational, prospective assessment study").

3. Number of Experts and Qualifications for Ground Truth

• Non-Clinical Performance (Algorithm Validation):
  • Morphokinetic Stages and Blast Annotations: Three independent embryologists.
  • Qualifications: "The annotators were not involved in the training or tuning of the model and were blinded to each other's labels." No explicit years of experience are stated for these annotators.
• Clinical Performance (CHLOE BLAST Clinical Study):
  • Morphology Grading (Assessors): Three embryologists.
  • Qualifications: "blinded to CHLOE information," and performed grading according to SART standards. No explicit years of experience are stated.
  • Clinical Assessment (Panelists): Five independent embryologists.
  • Qualifications: All "in practice during the study period and from a range of geographical areas within the United States." 3 were senior embryologists with over 10 years of clinical embryology experience each, and the other 2 were junior embryologists with less than 3 years of clinical embryology experience.

4. Adjudication Method for the Test Set

• Non-Clinical Performance (Algorithm Validation):
  • Ground Truth: "Each embryo video was viewed by three independent embryologists who provided their morphokinetic stages and Blast annotations based on the time-lapse videos. The annotators were not involved in the training or tuning of the model and were blinded to each other's labels." It implies a consensus-based approach, but directly states, "The TLI videos were annotated at a frame level with the ground truth of one of the morphokinetic stages and at a video level with blastulation results."
• Clinical Performance (CHLOE BLAST Clinical Study):
  • Morphology Grading (Assessors): "Then, the following parameters were categorized by majority agreement (at least 2 of 3 Assessors): Severe asymmetry (yes/no), Fragmentation > 25% (yes/no), Number of cells (1 through 8, 9≤)." This is a clear 2 out of 3 (2+1) consensus method for specific parameters.
  • Clinical Assessment (Panelists): No explicit adjudication method is stated for the Panelists' predictions. Each Panelist performed their own independent predictions, and the study analyzed the collective performance as well as individual improvements.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

• Yes, an MRMC comparative effectiveness study was done as part of the clinical performance study.
• Effect Size of Human Readers Improvement with AI vs. without AI assistance:
  • The primary endpoint focused on the Odds Ratio (OR) for predicting blastocyst formation in Good/Fair embryos.
  • Without AI assistance (Morphology Only): OR = 3.77 (95% CI: 2.97, 4.79)
  • With AI assistance (Morphology + CHLOE Score): OR = 5.67 (95% CI: 4.6, 6.99)
  • This represents an improvement in the Odds Ratio from 3.77 to 5.67 for the primary endpoint.
  • For all embryos, the OR improved from 6.93 (without CHLOE) to 8.51 (with CHLOE).
  • For subject-level sensitivity, it improved from 80.36%-83.93% (traditional morphology) to 87.50%-92.86% (with CHLOE).
  • For Top 2 Embryo analysis, the OR improved from 3 (without CHLOE) to 10.73 (with CHLOE).

6. Standalone (Algorithm Only without Human-in-the-Loop) Performance

• Yes, a standalone performance assessment was done as part of the "Non-Clinical Performance – Algorithm Validation" section.
• The algorithm's performance in predicting blastocyst formation was assessed independently, yielding an AUC of 0.88 (95% CI: 0.86, 0.90). This demonstrates the algorithm's capability on its own.

7. Type of Ground Truth Used

• For Non-Clinical Performance (Algorithm Validation):
  • Expert Consensus: Morphokinetic stages and blast annotations were established by three independent embryologists.
  • Outcomes Data: The "blastulation results" (blastocyst Yes/No) are actual outcomes.
• For Clinical Performance (CHLOE BLAST Clinical Study):
  • Expert Consensus: Morphology grading by three "Assessors" with majority agreement (2 out of 3).
  • Outcomes Data: The "actual blastocyst outcome" (Yes/No) which the algorithm and human readers are predicting.

8. Sample Size for the Training Set

• The document states: "The study dataset included data collected specifically for the purpose of this study according to the predefined inclusion and exclusion criteria and was segregated from algorithm training and verification datasets."
• "The dataset used for the performance test was entirely independent from the dataset utilized in the CHLOE BLAST clinical study described in section 9, and the clinics that provided data for the performance dataset were not used to collect data for the clinical study."
• The specific sample size for the training set is NOT PROVIDED in this document. It only clearly states that the various test sets were independent from the training data.

9. How Ground Truth for the Training Set Was Established

• The document implies that the training data exists and was used to develop the CNN, but it does NOT specify how the ground truth for the training set was established. It only focuses on how ground truth was established for the independent testing and clinical validation sets.

Ask a specific question about this device

The KIDScore D3 tool provides decision support for prediction of embryos developing to the blastocyst stage by scoring them according to their statistical viability.

Adjunctive information provided by KIDScore D3 aids in the selection of embryo(s) for either transfer on Day 3, freezing or continued embryo development when, following morphological assessment on Day 3, there are multiple embryos deemed suitable for transfer or freezing.

The KIDScore D3 tool is only to be used with the EmbryoScope timelapse incubator systems.

The KIDScore D3 decision support tool is an adjunctive algorithm that is designed to support embryologists in their decision about which embryos are suitable for transfer. The tool is an optional accessory to the EmbryoViewer software. It is used in the "Compare & Select" function. The "D3" in the name refers to the use of the algorithm on Day 3 for aiding the embryologist in preparing for transfer of the embryo to the female patient.

KIDScore D3 utilizes the following manually annotated parameters to aid in identifying embryos that are suitable for transfer:

• Pronuclei (number of pronuclei): ●
• tPNf (time from insemination until pronuclei is fading) ●
• t2 (time from insemination to complete division to two cells) ●
• t3 (time from insemination to complete division to three cells) ●
• t4 (time for insemination to complete division to four cells) .
• t5 (time from insemination to complete division to five cells) ●
• t8 (time from insemination to complete division to eight cells) ●

The KIDScore D3 assigns scores by comparing the parameters above in embryos to the model criteria, one criterion at a time until the process stops either because the embryo did not pass one of the criteria in the sequence or because the last criterion in the model was reached. From the information available at day three of incubation, the KIDScore D3 divides embryos into five score groups (1-5, as described below):

• 0 = The embryo is not 2PN
• 1 = Initial development was too fast or the embryo displayed a direct cleavage from one to three cells
• 2 = The embryo was slow to develop
• 3 = Embryo development was irregular and the development pace increased from day two to day three
• 4 = Embryo development was irregular and the development pace slowed from day two to day three

and/or

The number of cells annotated at 66 hours was not as expected

5 = The embryo passed all of the avoidance criteria included in the model.

One or more computers running the EmbryoViewer software may be connected to the ES Server. KIDScore D3 is stored on the computer running the ES Server software. Calculations related to the model in KIDScore D3 are performed on the computer running the ES Server software.

This document describes the KIDScore D3 device, an adjunctive algorithm designed to support embryologists in selecting suitable embryos for transfer by predicting their likelihood of developing to the blastocyst stage.

Here's an analysis of the acceptance criteria and the study proving the device meets them:

1. Table of Acceptance Criteria and Reported Device Performance

The acceptance criteria are implicitly defined by the primary endpoint of the clinical study, which required the blastocyst Odds Ratio (OR) for the adjunct prediction to be statistically significantly greater than 1 for Good/Fair embryos (graded A, B, or C using Day 3 morphology).

Ask a specific question about this device

The Eeva System is indicated to provide adjunctive information on events occurring during the first two days of development that may predict further development to the blastocyst stage on Day 5 of development. This adjunctive information aids in the selection of embryo(s) for transfer on Day 3 when, following morphological assessment on Day 3, there are multiple embryos deemed suitable for transfer or freezing. The device may also be used to collect additional time-lapse images until Day 5 of development for embryos not selected for transfer, to allow monitoring of continued embryo development.

The Eeva™ System is an Assisted Reproduction Embryo Image Assessment System (21 CFR 884.6195), installed in an IVF lab and used by embryologists and other IVF professionals. None of the System components have an individual, prior 510(k) clearance. Eeva System, Model EVS210 requires the use of the 12-microwell configuration of the Eeva™ Dish (K141663, also referred to as the "dish"), which is placed on the Eeva Scope (an assisted reproductive microscope). The Eeva Scopes are placed in commercially-available standard-sized incubators. The microscope employs high resolution time-lapse imaging to record an embryo's development during its first two days of incubation. Automated measurements of cell division timing parameters and the Eeva Test results are provided to the user after approximately 42 hours predicting the likelihood of whether an embryo will develop to the blastocyst stage. In Eeva System, Model EVS2210, image recording may continue through Day 5 of embryo development.

The Eeva™ System (EVS2210) provides adjunctive information on early embryo development (first two days) to predict progression to the blastocyst stage by Day 5. This information assists in selecting embryos for transfer on Day 3, especially when multiple suitable embryos are identified through morphological assessment.

1. Table of Acceptance Criteria and Reported Device Performance

The provided document does not explicitly list acceptance criteria for specific performance metrics (like sensitivity, specificity, PPV, NPV) with predefined thresholds. However, it states that "simulated clinical testing (mechanical analysis) demonstrates that the Eeva System Model EVS2210 is informative, and the average specificity, sensitivity, positive predictive value, and negative predictive value performance are substantially equivalent in the adjunctive use of the subject and predicate devices." This implies that the performance of the Eeva System (EVS2210) closely matched that of its predicate device, Eeva System (EVS2000).

The "Algorithm Software Validation" and "Simulated Clinical Use" tests aimed to evaluate the ability of the Eeva System software to predict blastocyst formation and its clinical performance, including determination of sensitivity, specificity, positive predictive value, negative predictive value, and odds ratio.

Since the document asserts substantial equivalence, the implied acceptance criteria are that the EVS2210's performance metrics (sensitivity, specificity, PPV, NPV, odds ratio) should be comparable to or not worse than those of the predicate device (EVS2000).

2. Sample size used for the test set and the data provenance

The document mentions "Simulated Clinical Use" for evaluation but does not explicitly state the sample size used for the test set or the data provenance (e.g., country of origin, retrospective/prospective). It refers to "clinical data submitted for the predicate device" as being "representative of expected safety and effectiveness of the Eeva System Model EVS2210," but this doesn't specify if new data was used for the EVS2210's simulated clinical use or if it entirely leveraged the predicate's data.

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts

The document does not specify the number or qualifications of experts used to establish the ground truth for the "Simulated Clinical Use" test set.

4. Adjudication method for the test set

The document does not describe any adjudication method used for the test set.

5. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance

The document does not mention a multi-reader multi-case (MRMC) comparative effectiveness study, nor does it describe an effect size for human reader improvement with or without AI assistance. The device provides "adjunctive information" to aid embryologists, suggesting it's intended for human-in-the-loop use, but a formal MRMC study is not detailed.

6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done

The "Simulated Clinical Use" is described as evaluating "clinical performance of the Eeva System software including determination of sensitivity, specificity, positive predictive value, negative predictive value, and odds ratio." This suggests a standalone evaluation of the algorithm's performance in predicting blastocyst formation. The device provides "adjunctive information," implying its output is then used by an embryologist. Therefore, a standalone evaluation of the software's predictive capability appears to have been performed.

7. The type of ground truth used (expert consensus, pathology, outcomes data, etc.)

The ground truth used for the "Algorithm Software Validation" and "Simulated Clinical Use" tests was the "blastocyst formation." This is an objective biological outcome (whether an embryo develops to the blastocyst stage by Day 5).

8. The sample size for the training set

The document does not provide information regarding the sample size for the training set used for the Eeva System's algorithm.

9. How the ground truth for the training set was established

The document does not provide information on how the ground truth for the training set was established. It only mentions that the device evaluates cell division timing parameters to predict blastocyst formation.

Ask a specific question about this device

The Eeva System is indicated to provide adjunctive information on events occurring during the first two days of development that may predict further development to the blastocyst stage on Day 5 of development. This adjunctive information aids in the selection of embryo(s) for transfer on Day 3 when, following morphological assessment on Day 3, there are multiple embryos deemed suitable for transfer or freezing.

The Eeva System provides image recording and automated analysis of cell division from high resolution time-lapse images collected until day 3 (72 hours) of development. Results of cell division timing parameters (time from first to second mitosis: and time from second to third mitosis) are provided to the user in addition to a prediction of the likelihood that an embryo will develop to the blastocyst stage. These timing parameters are based on those published in a study by Wong, et. al. (2010).

The Eeva System incorporates: (1) a set of up to four time-lapse image microscopes that automatically take darkfield microscopy images of embryos at regular intervals (every 5 minutes) while the embryos remain in the incubator environment, (2) Eeva Computer and other components (Control Box, Station, Scope Screen and Printer), (3) system software for image capture and recording, user interface, and patient database and (4) image analysis software that automatically identifies embryo development events, compares their times to specified timing parameters and makes a prediction of embryo development to the blastocyst stage. The system is installed in an In Vitro Fertilization (IVF) laboratory, and is to be used as an adjunct to the traditional morphological method to identify the embryos that are more likely to develop into blastocysts.

The Eeva™ System is indicated as an adjunct to traditional morphology evaluation to aid in the selection of embryo(s) for transfer on Day 3 when multiple embryos are deemed suitable for transfer or freezing. It helps predict the likelihood of an embryo developing to the blastocyst stage on Day 5/6.

1. Table of Acceptance Criteria & Reported Device Performance:

The document primarily focuses on clinical performance characteristics rather than specific hard-coded acceptance thresholds for every metric. However, for "Software Validation," a clear acceptance criterion is defined and met. For the primary and secondary endpoints in the Pivotal Adjunct Use Study, the outcome of statistical significance against stated objectives serves as the "acceptance."

Ask a specific question about this device