The KIDScore D3 tool provides decision support for prediction of embryos developing to the blastocyst stage by scoring them according to their statistical viability.

Adjunctive information provided by KIDScore D3 aids in the selection of embryo(s) for either transfer on Day 3, freezing or continued embryo development when, following morphological assessment on Day 3, there are multiple embryos deemed suitable for transfer or freezing.

The KIDScore D3 tool is only to be used with the EmbryoScope timelapse incubator systems.

Device Description

The KIDScore D3 decision support tool is an adjunctive algorithm that is designed to support embryologists in their decision about which embryos are suitable for transfer. The tool is an optional accessory to the EmbryoViewer software. It is used in the "Compare & Select" function. The "D3" in the name refers to the use of the algorithm on Day 3 for aiding the embryologist in preparing for transfer of the embryo to the female patient.

KIDScore D3 utilizes the following manually annotated parameters to aid in identifying embryos that are suitable for transfer:

Pronuclei (number of pronuclei): ●
tPNf (time from insemination until pronuclei is fading) ●
t2 (time from insemination to complete division to two cells) ●
t3 (time from insemination to complete division to three cells) ●
t4 (time for insemination to complete division to four cells) .
t5 (time from insemination to complete division to five cells) ●
t8 (time from insemination to complete division to eight cells) ●

The KIDScore D3 assigns scores by comparing the parameters above in embryos to the model criteria, one criterion at a time until the process stops either because the embryo did not pass one of the criteria in the sequence or because the last criterion in the model was reached. From the information available at day three of incubation, the KIDScore D3 divides embryos into five score groups (1-5, as described below):

0 = The embryo is not 2PN
1 = Initial development was too fast or the embryo displayed a direct cleavage from one to three cells
2 = The embryo was slow to develop
3 = Embryo development was irregular and the development pace increased from day two to day three
4 = Embryo development was irregular and the development pace slowed from day two to day three

and/or

The number of cells annotated at 66 hours was not as expected

5 = The embryo passed all of the avoidance criteria included in the model.

One or more computers running the EmbryoViewer software may be connected to the ES Server. KIDScore D3 is stored on the computer running the ES Server software. Calculations related to the model in KIDScore D3 are performed on the computer running the ES Server software.

AI/ML Overview

This document describes the KIDScore D3 device, an adjunctive algorithm designed to support embryologists in selecting suitable embryos for transfer by predicting their likelihood of developing to the blastocyst stage.

Here's an analysis of the acceptance criteria and the study proving the device meets them:

1. Table of Acceptance Criteria and Reported Device Performance

The acceptance criteria are implicitly defined by the primary endpoint of the clinical study, which required the blastocyst Odds Ratio (OR) for the adjunct prediction to be statistically significantly greater than 1 for Good/Fair embryos (graded A, B, or C using Day 3 morphology).

Acceptance Criteria	Reported Device Performance (KIDScore D3)	Reported Device Performance (Eeva System - Predicate)
Blastocyst Odds Ratio (OR) for adjunct prediction (for Good/Fair embryos) statistically significantly greater than 1	4.13	2.57
95% Confidence Interval for OR	3.48 - 4.9	1.88 - 3.51
P-value for OR	<0.0001	<0.0001

Conclusion: The KIDScore D3 device’s reported performance (OR of 4.13 with a p-value <0.0001) significantly exceeds the acceptance criterion, demonstrating its ability to predict blastocyst outcome when used as an adjunct to morphology.

2. Sample Size Used for the Test Set and Data Provenance

Sample Size: 81 patients, including their embryos (number of embryos not explicitly stated for the test set, but mentioned as "embryos from 81 patients").
Data Provenance: Retrospective. The data was a subset taken from a total collection of 4152 embryos from 1338 treatments carried out in European IVF clinics between 2009 and 2014.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts

The document does not explicitly state the number of experts used to establish the ground truth for the test set nor their specific qualifications (e.g., years of experience). It mentions:

"Embryologists were masked to imaging data and evaluation was only based on morphology and KIDScore D3 scores." This implies embryologists were involved in the morphological grading, which is part of the overall assessment.
The "adjunct prediction of blastocyst outcome and the actual blastocyst outcome" were assessed. This "actual blastocyst outcome" serves as the ground truth.

4. Adjudication Method for the Test Set

The document does not explicitly describe an adjudication method (like 2+1 or 3+1) for resolving discrepancies in ground truth establishment for the test set. It states that "Embryologists were masked to imaging data and evaluation was only based on morphology and KIDScore D3 scores," which suggests individual assessments were made.

5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was Done, If So, What Was the Effect Size of How Much Human Readers Improve with AI vs Without AI Assistance

No, a multi-reader multi-case (MRMC) comparative effectiveness study evaluating human readers' improvement with and without AI assistance was not explicitly conducted or reported. The study focused on the device's ability to predict blastocyst outcome as an adjunct to morphology, and its performance was compared to the predicate device, not necessarily against human readers in an assisted vs. unassisted paradigm for their performance improvement. The study evaluated "the utilization of established morphology methods with adjunct outcome of an algorithm (KIDScore D3)" and concluded that "the adjunct use of KIDScore D3 improved the selection of embryos for transfer compared with morphology alone." However, it does not quantify the effect size of how much human readers themselves improved.

6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) was Done

Yes, a standalone performance of the algorithm was implicitly evaluated. The primary endpoint, "the association between the adjunct prediction of blastocyst outcome and the actual blastocyst outcome," assesses the KIDScore D3's predictive capability in conjunction with morphology. While it's adjunctive information, the study's results (Odds Ratio for KIDScore D3) clearly reflect the algorithm's standalone predictive power within the context of the study design. It was evaluated for its ability to predict which embryos are most likely to develop to blastocyst stage. The phrase "Embryologists were masked to imaging data and evaluation was only based on morphology and KIDScore D3 scores" means the KIDScore D3 scores were provided directly to the embryologists as part of their evaluation, indicating the algorithm's output was central to the assessment.

7. The Type of Ground Truth Used

The ground truth used was actual blastocyst outcome (i.e., whether an embryo developed to the blastocyst stage). This can be considered a form of outcome data directly observed during the IVF process.

8. The Sample Size for the Training Set

The document does not explicitly state the sample size for the training set. It mentions the "data included embryos from 81 patients" for the clinical study (test set), and that this "data are a subset taken from a total collection of 4152 embryos from 1338 treatments where all sibling embryos have been annotated for the morphokinetic events required by KIDScore D3." This larger collection of 4152 embryos is likely the source from which the algorithm was developed and presumably trained, but the exact training set size is not segregated.

9. How the Ground Truth for the Training Set Was Established

The document does not explicitly state how the ground truth for the training set (if it was a separate, formally defined set) was established. However, given the nature of the device, it's highly probable that similar to the test set, the ground truth for the training data (from the "total collection of 4152 embryos") would have been established by observing whether the embryos developed to the blastocyst stage, potentially combined with expert morphological assessment done during the standard clinical practice at the time the data was collected (2009-2014). The manual annotation of parameters like pronuclei, tPNf, t2, t3, t4, t5, and t8 would have been linked to these observed blastocyst outcomes to develop the KIDScore D3 model.

Summary

{0}------------------------------------------------

Image /page/0/Picture/0 description: The image shows the logos of the Department of Health & Human Services and the Food and Drug Administration (FDA). The Department of Health & Human Services logo is on the left, and the FDA logo is on the right. The FDA logo is a blue square with the letters "FDA" in white, followed by the words "U.S. FOOD & DRUG ADMINISTRATION" in blue.

July 19, 2019

Vitrolife A/S Belinda Dueholm Regulatory Affairs Specialist Jens Juuls Vej 20 Viby J, 8260 DENMARK

Re: K182798

Trade/Device Name: KIDScore D3 Regulation Number: 21 CFR 884.6195 Regulation Name: Assisted reproduction embryo image assessment system Regulatory Class: Class II Product Code: PBH Dated: June 17, 2019 Received: June 20, 2019

Dear Belinda Dueholm:

We have reviewed your Section 510(k) premarket notification of intent to market the device referenced above and have determined the device is substantially equivalent (for the indications for use stated in the enclosure) to legally marketed predicate devices marketed in interstate commerce prior to May 28, 1976, the enactment date of the Medical Device Amendments, or to devices that have been reclassified in accordance with the provisions of the Federal Food, Drug, and Cosmetic Act (Act) that do not require approval of a premarket approval application (PMA). You may, therefore, market the device, subject to the general controls provisions of the Act. Although this letter refers to your product as a device, please be aware that some cleared products may instead be combination products. The 510(k) Premarket Notification Database located at https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpmn/pmn.cfm identifies combination product submissions. The general controls provisions of the Act include requirements for annual registration, listing of devices, good manufacturing practice, labeling, and prohibitions against misbranding and adulteration. Please note: CDRH does not evaluate information related to contract liability warranties. We remind you, however, that device labeling must be truthful and not misleading.

If your device is classified (see above) into either class II (Special Controls) or class III (PMA), it may be subject to additional controls. Existing major regulations affecting your device can be found in the Code of Federal Regulations, Title 21, Parts 800 to 898. In addition, FDA may publish further announcements concerning your device in the Federal Register.

{1}------------------------------------------------

Please be advised that FDA's issuance of a substantial equivalence determination does not mean that FDA has made a determination that your device complies with other requirements of the Act or any Federal statutes and regulations administered by other Federal agencies. You must comply with all the Act's requirements, including, but not limited to: registration and listing (21 CFR Part 807); labeling (21 CFR Part 801); medical device reporting of medical device-related adverse events) (21 CFR 803) for devices or postmarketing safety reporting (21 CFR 4, Subpart B) for combination products (see https://www.fda.gov/combination-products/guidance-regulatory-information/postmarketing-safety-reportingcombination-products); good manufacturing practice requirements as set forth in the quality systems (QS) regulation (21 CFR Part 820) for devices or current good manufacturing practices (21 CFR 4, Subpart A) for combination products; and, if applicable, the electronic product radiation control provisions (Sections 531-542 of the Act); 21 CFR 1000-1050.

Also, please note the regulation entitled, "Misbranding by reference to premarket notification" (21 CFR Part 807.97). For questions regarding the reporting of adverse events under the MDR regulation (21 CFR Part 803), please go to https://www.fda.gov/medical-device-safety/medical-device-reportingmdr-how-report-medical-device-problems.

For comprehensive regulatory information about medical devices and radiation-emitting products, including information about labeling regulations, please see Device Advice (https://www.fda.gov/medicaldevices/device-advice-comprehensive-regulatory-assistance) and CDRH Learn (https://www.fda.gov/training-and-continuing-education/cdrh-learn). Additionally, you may contact the Division of Industry and Consumer Education (DICE) to ask a question about a specific regulatory topic. See the DICE website (https://www.fda.gov/medical-device-advice-comprehensive-regulatoryassistance/contact-us-division-industry-and-consumer-education-dice) for more information or contact DICE by email (DICE@fda.hhs.gov) or phone (1-800-638-2041 or 301-796-7100).

Sincerely,

For Sharon M. Andrews Assistant Division Director DHT3B: Division of Reproductive. Gynecology and Urology Devices OHT3: Office of Gastrorenal, ObGyn, General Hospital and Urology Devices Office of Product Evaluation and Quality Center for Devices and Radiological Health

Enclosure

{2}------------------------------------------------

Indications for Use

510(k) Number (if known) K182798

Device Name KIDScore D3

Indications for Use (Describe)

The KIDScore D3 tool provides decision support for prediction of embryos developing to the blastocyst stage by scoring them according to their statistical viability.

The KIDScore D3 tool is only to be used with the EmbryoScope timelapse incubator systems.

Type of Use (Select one or both, as applicable):

Prescription Use (Part 21 CFR 801 Subpart D)
Over-The-Counter Use (21 CFR 801 Subpart C)

CONTINUE ON A SEPARATE PAGE IF NEEDED.

This section applies only to requirements of the Paperwork Reduction Act of 1995.

DO NOT SEND YOUR COMPLETED FORM TO THE PRA STAFF EMAIL ADDRESS BELOW.

The burden time for this collection of information is estimated to average 79 hours per response, including the time to review instructions, search existing data sources, gather and maintain the data needed and complete and review the collection of information. Send comments regarding this burden estimate or any other aspect of this information collection, including suggestions for reducing this burden, to:

Department of Health and Human Services Food and Drug Administration Office of Chief Information Officer Paperwork Reduction Act (PRA) Staff PRAStaff@fda.hhs.gov

"An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB number."

{3}------------------------------------------------

510(k) Summary - KIDScore D3 K182798

1. Submitter Information

Submitter	Vitrolife A/SJens Juuls Vej 208260 Viby JDenmark
Contact Person:Telephone:	Ms. Belinda Dueholm+45 7221 7900 (main)+45 2076 3707 (direct)

1. Date Prepared: July 19th, 2019

3. Device Information

Proprietary Name:	KIDScore D3
Common Name:	Assisted Reproduction Embryo Image Assessment System
Regulation Number:	21 CFR 884.6195
Regulation Name:	Assisted Reproduction Embryo Image Assessment System
Product Code:	PBH (Embryo Image Assessment System, Assisted Reproduction
Regulatory Class	II

4. Predicate Device

Eeva(TM) System, K142147, Auxogyn Inc.

The predicate device has not been subject to any design related recalls.

5. Device Description

KIDScore D3 utilizes the following manually annotated parameters to aid in identifying embryos that are suitable for transfer:

Pronuclei (number of pronuclei): ●
tPNf (time from insemination until pronuclei is fading) ●
t2 (time from insemination to complete division to two cells) ●

{4}------------------------------------------------

t3 (time from insemination to complete division to three cells) ●
t4 (time for insemination to complete division to four cells) .
t5 (time from insemination to complete division to five cells) ●
t8 (time from insemination to complete division to eight cells) ●

0 = The embryo is not 2PN
1 = Initial development was too fast or the embryo displayed a direct cleavage from one to three cells
2 = The embryo was slow to develop
3 = Embryo development was irregular and the development pace increased from day two to day three
4 = Embryo development was irregular and the development pace slowed from day two to day three

and/or

The number of cells annotated at 66 hours was not as expected

5 = The embryo passed all of the avoidance criteria included in the model.

6. Indications for Use

The KIDScore D3 tool provides decision support for prediction of likelihood of embryos developing to the blastocyst stage by scoring them according to their statistical viability.

The KIDScore D3 tool is only to be used with the EmbryoScope and EmbryoScope+ timelapse incubator systems.

{5}------------------------------------------------

7. Comparison of Intended Use and Technological Characteristics of the Subject and Predicate Device

Intended Use

Devices	Subject device (K182798) – KIDScore D3	Predicate device (K142147) – Eeva System
Indications for Use	The KIDScore D3 tool provides decision support for prediction of likelihood of embryos developing to the blastocyst stage by scoring them according to their statistical viability.Adjunctive information provided by KIDScore D3 aids in the selection of embryo(s) for either transfer on Day 3, freezing or continued embryo development when, following morphological assessment on Day 3, there are multiple embryos deemed suitable for transfer or freezing.The KIDScore D3 tool is only to be used with the EmbryoScope and EmbryoScope+ time-lapse incubator systems.	The Eeva System is indicated to provide adjunctive information on events occurring during the first two days of development that may predict further development to the blastocyst stage on Day 5 of development. This adjunctive information aids in the selection of embryo(s) for transfer on Day 3 when, following morphological assessment on Day 3, there are multiple embryos deemed suitable for transfer or freezing. The device may also be used to collect additional time-lapse images until Day 5 of development for embryos not selected for transfer, to allow monitoring of continued embryo development.

The intended use of the subject device is the same as the predicate device – both are intended to provide adjunctive information (decision support) to the embryologist/clinician at Day 3 of culture to aid the user in selecting embryos for transfer/freezing during assisted reproduction procedures.

Technological Characteristics

	Subject device(KIDScoreD3,K182798)	Predicate device(Eeva System,K142147)	Comments
Device Design	Integrates theKIDScore D3 decisionsupport tool on theCompare & Selectpage of theEmbryoviewer softwarepresent on theEmbryoscope andEmbryoscope +devices.	Time-lapse systemused in conventionalincubator with analysissoftware thatautomatically identifiesembryo developmentevents for use in ablastocyst predictionmodel	Different: The subjectdevice is a softwareaccessory to be usedwith the Embryoscopeimaging incubator toprovide adjunctiveinformation regardingblastocyst quality. Thepredicate device is atime lapse imagingsystem to be used in aconventional incubatorthat utilizes automatedimage analysis toprovide adjunctiveblastocyst qualityinformation. These

{6}------------------------------------------------

			differences do not raisedifferent questions ofsafety andeffectiveness.
Algorithm	SoftwareUser annotations ofembryo developmentparameters used asalgorithm inputsScore based on criteriawhich are either relatedto embryo morphologyor development stage	SoftwareCell tracking and eventinference used asalgorithm inputs	Different. The subjectdevice utilizes anelimination model withuser annotation as theinput, whereas thepredicate deviceutilizes a predictivealgorithm that usesautomated cell trackingand image analysisinputs. Thesedifferences do not raisedifferent questions ofsafety andeffectiveness.
Annotation method	Manual input ofembryo developmentevents and timing	Image analysissoftware automaticallyidentifies embryodevelopment eventsand timing	Different. The subjectdevice utilizes inputfrom manualannotations, whereasthe predicate deviceutilizes an automatedimage analysis system.These differences donot raise differentquestions of safety andeffectiveness.

The technological characteristics of the subject and predicate device are different - the subject device has different predictive algorithms for blastocyst quality, and different software and annotation functions. However, different types of safety and effectiveness questions are not raised by these differences in technological characteristics.

8. Summary of Non-Clinical Performance Testing

Devices classified under 21 CFR 884.6195 (Assisted Reproduction Embryo Image Assessment System) and product code PBH must address several non-clinical special controls, including software validation, verification, and hazard analysis, an assessment of light exposure and output, simulated use, cleaning and disinfection, package integrity and transit testing, electrical safety and electromagnetic compatibility testing, and prediction algorithm reproducibility.

As the subject device is an optional accessory to the EmbryoViewer software used in the EmbryoScope+ incubator (cleared under K173264), some of the special controls do not apply to this submission. The previously cleared version of the Embryoscope+ has met the special controls associated with the majority of the non-clinical performance testing listed above.

In the current submission, the sponsor has addressed the special controls associated with software and algorithm reproducibility. The software verification testing provided in this

{7}------------------------------------------------

submission met the requirements of the FDA Guidance Document, "Guidance for the Content of Premarket Submissions for Software Contained in Medical Devices" issued on May 11, 2005. The reproducibility of the prediction algorithm is addressed in the summary of clinical testing described below.

9. Summary of Clinical Testing

Devices classified under 21 CFR 884.6195 (Assisted Reproduction Embryo Image Assessment System) and product code PBH must address a clinical special control to demonstrate reasonable assurance of safety and effectiveness of the device to predict embryo development. The clinical study is described below.

The safety and effectiveness of the KIDScore D3 was investigated in a prospective, single arm, multi-center clinical study conducted at six sites in the United States. This was a non-interventional clinical study where the KIDScore D3 was not used during patient treatment. The purpose of the study was to collect data to evaluate the safety and effectiveness of the KIDScore D3's ability to predict which embryos are most likely to develop to blastocyst stage. This was evaluated by using the KIDScore D3 as adjunct information to traditional morphological grading. Imaging data was collected on embryos cultured to day 5. Embryologists were masked to imaging data and evaluation was only based on morphology and KIDScore D3 scores.

Objective

The study was aimed at analyzing the utilization of established morphology methods with adjunct outcome of an algorithm (KIDScore D3) that provides a score (1 - 5) from timings of morphokinetic events.

Study Design

A double-blinded, multi-center study, designed to evaluate the odds ratios and other measures for outcomes of methodologies used for embryo assessment: day 3 morphology alone and day 3 morphology with KIDScore D3 results as adjunct information.

Selection of Study Data

The data included embryos from 81 patients who underwent in vitro fertilization (IVF) treatment using their own eggs or donor eggs. These data are a subset taken from a total collection of 4152 embryos from 1338 treatments where all sibling embryos have been annotated for the morphokinetic events required by KIDScore D3. These data originate from treatments that were carried out in European IVF clinics, in the period 2009 - 2014.

Primary endpoint

The primary endpoint of the study was to assess the association between the adjunct prediction of blastocyst outcome and the actual blastocyst outcome. The purpose was to determine if KIDScore D3 was informative for embryos graded as an A, B, or C using Day 3 morphology category assignment as compared to the predicate device (Eeva system). For those Good/Fair embryos, the blastocyst Odds Ratio (OR) for the adjunct prediction is required to be statistically significantly greater than 1.

{8}------------------------------------------------

Secondary endpoints

The following calculations were assessed as secondary endpoints:

Embryo level diagnostic performance measures .
- o specificity
- o sensitivity
- negative predictive value o
- positive predictive value o
- negative likelihood ratio o
- positive likelihood ratio o
. Top 2 Embryo analysis
Treatment level analysis

Results

The study demonstrated that the adjunct use of KIDScore D3 improved the selection of embryos for transfer compared with morphology alone. In the primary endpoint, it was shown that the adjunct use of KIDScore D3 was informative for blastocyst outcome.

The study showed that both the KIDScore D3 and the Eeva system were predictive of blastocyst outcome, with odds ratios greater than one for both systems, as described in the figure below:

	KIDScore D3	Eeva
Odds ratio	4.13	2.57
95% CI	3.48 - 4.9	1.88 - 3.51
P value	<0.0001	<0.0001

10. Conclusion

The subject and predicate devices have the same intended use and fundamental technological characteristics. The differences in technological characteristics between subject and predicate devices do not raise different questions of safety and effectiveness. The performance data and clinical testing demonstrate that the subject device is substantially equivalent to the predicate device.

Regulation Number and Section

§ 884.6195 Assisted Reproduction Embryo Image Assessment System.

(a)
Identification. An Assisted Reproduction Embryo Image Assessment System is a prescription device that is designed to obtain and analyze light microscopy images of developing embryos. This device provides information to aid in the selection of embryo(s) for transfer when there are multiple embryos deemed suitable for transfer or freezing.(b)
Classification. Class II (special controls). The special control(s) for this device are:(1) Clinical performance testing must demonstrate a reasonable assurance of safety and effectiveness of the device to predict embryo development. Classification performance (sensitivity and specificity) and predictive accuracy (Positive Predictive Value and Negative Predictive Value) must be assessed at the subject and embryo levels.
(2) Software validation, verification, and hazard analysis must be provided.
(3) Non-clinical performance testing data must demonstrate the performance characteristics of the device. Testing must include the following:
(i) Total light exposure and output testing;
(ii) A safety analysis must be performed based on maximum (worst-case) light exposure to embryos, which also includes the safety of the light wavelength(s) emitted by the device;
(iii) Simulated-use testing;
(iv) Mouse Embryo Assay testing to assess whether device operation impacts growth and development of mouse embryos to the blastocyst stage;
(v) Cleaning and disinfection validation of reusable components;
(vi) Package integrity and transit testing;
(vii) Hardware fail-safe validation;
(viii) Electrical equipment safety and electromagnetic compatibility testing; and
(ix) Prediction algorithm reproducibility.
(4) Labeling must include the following:
(i) A detailed summary of clinical performance testing, including any adverse events;
(ii) Specific instructions, warnings, precautions, and training needed for safe use of the device
(iii) Appropriate electromagnetic compatibility information;
(iv) Validated methods and instructions for cleaning and disinfection of reusable components; and
(v) Information identifying compatible cultureware and explain how they are used with the device.