Fetal EchoScan is a machine learning-based computer-assisted diagnosis (CADx) software device indicated as an adjunct to fetal heart ultrasound examination in pregnant women aged 18 or older undergoing second-trimester anatomic ultrasound exams.

When utilized by an interpreting physician, Fetal EchoScan provides information regarding the presence of any of the following suspicious radiographic findings:

overriding artery
septal defect at the cardiac crux
abnormal relationship of the outflow tracts
enlarged cardiothoracic ratio
right ventricular to left ventricular size discrepancy
tricuspid valve to mitral valve annular size discrepancy
pulmonary valve to aortic valve annular size discrepancy
cardiac axis deviation

Fetal EchoScan is to be used with cardiac fetal ultrasound video clips containing interpretable 4-chamber, left ventricular outflow tract, right ventricular outflow tract standard views.

Fetal EchoScan is intended for use as a concurrent reading aid for interpreting physicians (OB-GYN, MFM). It does not replace the role of the physician or of other diagnostic testing in the standard of care. When utilized by an interpreting physician, this device provides information that may be useful in rendering an accurate diagnosis regarding the potential presence of morphological abnormalities that might be suggestive of fetal congenital heart defects that may be useful in determining the need for additional exams.

Fetal EchoScan is not intended for use in multiple pregnancies, cases of heterotaxy, and postnatal ultrasound exams.

Device Description

Fetal EchoScan is a cloud-based software-only device which uses neural networks to detect suspicious cardiac radiographic findings for further review by trained and qualified physicians. Fetal EchoScan is intended to be used as an adjunct to the interpretation of the second-trimester fetal anatomic ultrasound exam performed between 18 and 24 weeks of gestation, for pregnant women aged 18 or more.

AI/ML Overview

Here's a breakdown of the acceptance criteria and study details for the Fetal EchoScan device, based on the provided document:

1. Table of Acceptance Criteria and Reported Device Performance

The document does not explicitly state "acceptance criteria" as a set of predefined thresholds. Instead, it presents the performance of the device in various metrics and then concludes that these results demonstrate substantial equivalence. For the purpose of this request, I will infer the implied acceptance criteria from the reported performance and the conclusion of substantial equivalence.

Inferred Acceptance Criteria & Reported Device Performance

Metric / Finding	Inferred Acceptance Criteria (Implicit from conclusion of substantial equivalence)	Fetal EchoScan Performance (Worst-Case Sensitivity / Best-Case Specificity)	Fetal EchoScan Performance (Best-Case Sensitivity / Worst-Case Specificity)	Aided Reader Performance (ROC AUC)
Standalone Performance
Any suspicious findings	High Sensitivity & High Specificity	Sensitivity: 0.977 (0.954-0.989) Specificity: 0.977 (0.961-0.987)	Sensitivity: 0.987 (0.967-0.995) Specificity: 0.963 (0.944-0.976)	N/A
Overriding artery	High Sensitivity & High Specificity	Sensitivity: 0.894 (0.820-0.940) Specificity: 0.989 (0.977-0.995)	Sensitivity: 0.942 (0.880-0.973) Specificity: 0.979 (0.963-0.988)	0.953 (0.916-0.990)
Cardiac crux septal defect	High Sensitivity & High Specificity	Sensitivity: 0.905 (0.823-0.951) Specificity: 0.995 (0.985-0.998)	Sensitivity: 0.917 (0.838-0.959) Specificity: 0.989 (0.977-0.995)	0.971 (0.943-0.999)
Abnormal OT relationship	High Sensitivity & High Specificity	Sensitivity: 0.869 (0.781-0.925) Specificity: 0.991 (0.979-0.996)	Sensitivity: 0.952 (0.884-0.981) Specificity: 0.989 (0.977-0.995)	0.972 (0.953-0.992)
Enlarged CTR	High Sensitivity & High Specificity	Sensitivity: 0.955 (0.876-0.985) Specificity: 1.000 (0.993-1.000)	Sensitivity: 0.955 (0.876-0.985) Specificity: 1.000 (0.993-1.000)	0.960 (0.930-0.989)
Cardiac axis deviation	High Sensitivity & High Specificity	Sensitivity: 0.945 (0.851-0.981) Specificity: 1.000 (0.993-1.000)	Sensitivity: 0.945 (0.851-0.981) Specificity: 1.000 (0.993-1.000)	0.967 (0.932-1.000)
PV/AV size discrepancy	High Sensitivity & High Specificity	Sensitivity: 0.954 (0.914-0.975) Specificity: 0.989 (0.977-0.995)	Sensitivity: 0.954 (0.914-0.975) Specificity: 0.989 (0.977-0.995)	0.979 (0.962-0.997)
RV/LV size discrepancy	High Sensitivity & High Specificity	Sensitivity: 0.950 (0.900-0.975) Specificity: 1.000 (0.993-1.000)	Sensitivity: 0.950 (0.900-0.975) Specificity: 1.000 (0.993-1.000)	0.991 (0.983-0.999)
TV/MV size discrepancy	High Sensitivity & High Specificity	Sensitivity: 0.943 (0.896-0.970) Specificity: 1.000 (0.993-1.000)	Sensitivity: 0.943 (0.896-0.970) Specificity: 1.000 (0.993-1.000)	0.964 (0.938-0.990)
MRMC Study Performance
ROC AUC (any suspicious finding)	Significantly higher with aid than unaided	N/A	N/A	Aided: 0.974 (0.957-0.990) Unaided: 0.825 (0.741-0.908)
Mean Sensitivity (Any finding)	Increased with aid	N/A	N/A	Aided: 0.935 (0.892-0.978) Unaided: 0.782 (0.686-0.878)
Mean Specificity (Any finding)	Increased with aid	N/A	N/A	Aided: 0.970 (0.949-0.991) Unaided: 0.759 (0.630-0.887)
Conclusive output rate	High	98.8% (95% CL, 97.8-99.3)	N/A	N/A

2. Sample Size and Data Provenance for the Test Set

Sample Size for Standalone Test Set: 877 clinically acquired fetal ultrasound exams.
Sample Size for MRMC Test Set: 200 exams.
Data Provenance: The data was collected from 11 centers in the U.S.A. and France. It was retrospectively collected as it refers to "clinically acquired fetal ultrasound exams".

3. Number of Experts and Qualifications for Ground Truth

Number of Experts: Three (3) pediatric cardiologists.
Qualifications of Experts: The document specifies "pediatric cardiologists" but does not provide details on their years of experience or other specific qualifications beyond their specialty.

4. Adjudication Method for the Test Set

Adjudication Method: Majority voting. This means that if at least two out of the three pediatric cardiologists agreed on the presence or absence of a finding, that was established as the ground truth.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

Was an MRMC study done? Yes.
Effect size of human readers improvement with AI vs. without AI assistance:
- ROC AUC for any suspicious finding: +14.9% increase (from 0.825 unaided to 0.974 aided, p=0.002).
- Mean Sensitivity for any suspicious finding: +15.3% increase (from 0.782 unaided to 0.935 aided).
- Mean Specificity for any suspicious finding: +21.1% increase (from 0.759 unaided to 0.970 aided).

6. Standalone (Algorithm Only) Performance Study

Was a standalone study done? Yes.
The results are presented in Table 1, showing sensitivity and specificity for "Any suspicious findings" and each individual finding, calculated under two scenarios for inconclusive outputs.

7. Type of Ground Truth Used

Type of Ground Truth: Expert consensus. Specifically, it was derived from a truthing process by three pediatric cardiologists using majority voting.

8. Sample Size for the Training Set

The document states that "The ultrasound examinations used for training and validation are entirely distinct from the examinations used in standalone testing," but it does not explicitly provide the sample size for the training set.

9. How the Ground Truth for the Training Set Was Established

The document states that the "ultrasound examinations used for training and validation are entirely distinct from the examinations used in standalone testing." However, similar to the training set sample size, it does not explicitly describe how the ground truth for the training set was established. It only details the ground truth establishment for the test sets (standalone and MRMC).

Summary

{0}------------------------------------------------

Image /page/0/Picture/0 description: The image contains the logo of the U.S. Food & Drug Administration (FDA). On the left is the Department of Health & Human Services logo. To the right of that is the FDA logo, which is a blue square with the letters "FDA" in white. To the right of the blue square is the text "U.S. FOOD & DRUG ADMINISTRATION" in blue.

November 14, 2024

BrightHeart Christophe Gardella CTO 7-11 boulevard Haussmann Paris, 75009, France

Re: K242342

Trade/Device Name: Fetal EchoScan Regulation Number: 21 CFR 892.2060 Regulation Name: Radiological computer-assisted diagnostic software for lesions suspicious of cancer Regulatory Class: Class II Product Code: POK Dated: October 24, 2024 Received: October 24, 2024

Dear Christophe Gardella:

We have reviewed your section 510(k) premarket notification of intent to market the device referenced above and have determined the device is substantially equivalent (for the indications for use stated in the enclosure) to legally marketed predicate devices marketed in interstate commerce prior to May 28, 1976, the enactment date of the Medical Device Amendments, or to devices that have been reclassified in accordance with the provisions of the Federal Food, Drug, and Cosmetic Act (the Act) that do not require approval of a premarket approval application (PMA). You may, therefore, market the device, subject to the general controls provisions of the Act. Although this letter refers to your product as a device, please be aware that some cleared products may instead be combination products. The 510(k) Premarket Notification Database available at https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpm/pm.cfm identifies combination product submissions. The general controls provisions of the Act include requirements for annual registration, listing of devices, good manufacturing practice, labeling, and prohibitions against misbranding and adulteration. Please note: CDRH does not evaluate information related to contract liability warranties. We remind you, however, that device labeling must be truthful and not misleading.

If your device is classified (see above) into either class II (Special Controls) or class III (PMA), it may be subject to additional controls. Existing major regulations affecting your device can be found in the Code of Federal Regulations, Title 21, Parts 800 to 898. In addition, FDA may publish further announcements concerning your device in the Federal Register.

Additional information about changes that may require a new premarket notification are provided in the FDA guidance documents entitled "Deciding When to Submit a 510(k) for a Change to an Existing Device" (https://www.fda.gov/media/99812/download) and "Deciding When to Submit a 510(k) for a Software Change to an Existing Device" (https://www.fda.gov/media/99785/download).

Your device is also subject to, among other requirements, the Quality System (QS) regulation (21 CFR Part 820), which includes, but is not limited to. 21 CFR 820.30. Design controls; 21 CFR 820.90. Nonconforming product; and 21 CFR 820.100, Corrective and preventive action. Please note that regardless of whether a change requires premarket review, the QS regulation requires device manufacturers to review and approve changes to device design and production (21 CFR 820.30 and 21 CFR 820.70) and document changes and approvals in the device master record (21 CFR 820.181).

{1}------------------------------------------------

Please be advised that FDA's issuance of a substantial equivalence determination does not mean that FDA has made a determination that your device complies with other requirements of the Act or any Federal statutes and regulations administered by other Federal agencies. You must comply with all the Act's requirements, including, but not limited to: registration and listing (21 CFR Part 807); labeling (21 CFR Part 801); medical device reporting of medical device-related adverse events) (21 CFR Part 803) for devices or postmarketing safety reporting (21 CFR Part 4, Subpart B) for combination products (see https://www.fda.gov/combination-products/guidance-regulatoryinformation/postmarketing-safety-reporting-combination-products); good manufacturing practice requirements as set forth in the quality systems (QS) regulation (21 CFR Part 820) for devices or current good manufacturing practices (21 CFR Part 4, Subpart A) for combination products; and, if applicable, the electronic product radiation control provisions (Sections 531-542 of the Act); 21 CFR Parts 1000-1050.

All medical devices, including Class I and unclassified devices and combination product device constituent parts are required to be in compliance with the final Unique Device Identification System rule ("UDI Rule"). The UDI Rule requires, among other things, that a device bear a unique device identifier (UDD) on its label and package (21 CFR 801.20(a)) unless an exception or alternative applies (21 CFR 801.20(b)) and that the dates on the device label be formatted in accordance with 21 CFR 801.18. The UDI Rule (21 CFR 830.300(a) and 830.320(b)) also requires that certain information be submitted to the Global Unique Device Identification Database (GUDID) (21 CFR Part 830 Subpart E). For additional information on these requirements, please see the UDI System webpage at https://www.fda.gov/medical-device-advice-comprehensive-regulatory-assistance/unique-deviceidentification-system-udi-system.

Also, please note the regulation entitled, "Misbranding by reference to premarket notification" (21 CFR 807.97). For questions regarding the reporting of adverse events under the MDR regulation (21 CFR Part 803), please go to https://www.fda.gov/medical-device-safety/medical-device-reporting-mdr-how-report-medicaldevice-problems.

For comprehensive regulatory information about mediation-emitting products, including information about labeling regulations, please see Device Advice (https://www.fda.gov/medical-devices/deviceadvice-comprehensive-regulatory-assistance) and CDRH Learn (https://www.fda.gov/training-and-continuingeducation/cdrh-learn). Additionally, you may contact the Division of Industry and Consumer Education (DICE) to ask a question about a specific regulatory topic. See the DICE website (https://www.fda.gov/medical-devices/deviceadvice-comprehensive-regulatory-assistance/contact-us-division-industry-and-consumer-education-dice) for more information or contact DICE by email (DICE@fda.hhs.gov) or phone (1-800-638-2041 or 301-796-7100).

Sincerely.

Jessica Lamb

Jessica Lamb, Ph.D. Assistant Director, Imaging Software Team DHT8B: Division of Radiological Imaging Devices and Electronic Products OHT8: Office of Radiological Health Office of Product Evaluation and Quality Center for Devices and Radiological Health

{2}------------------------------------------------

Indications for Use

510(k) Number (if known) K242342

Device Name Fetal EchoScan

Indications for Use (Describe)

When utilized by an interpreting physician, Fetal EchoScan provides information regarding the presence of any of the following suspicious radiographic findings:

overriding artery
septal defect at the cardiac crux
abnormal relationship of the outflow tracts
enlarged cardiothoracic ratio
right ventricular to left ventricular size discrepancy
tricuspid valve to mitral valve annular size discrepancy
pulmonary valve to aortic valve annular size discrepancy
cardiac axis deviation

Fetal EchoScan is to be used with cardiac fetal ultrasound video clips containing interpretable 4-chamber, left ventricular outflow tract, right ventricular outflow tract standard views.

Fetal EchoScan is not intended for use in multiple pregnancies, cases of heterotaxy, and postnatal ultrasound exams.

Type of Use (Select one or both, as applicable)

X Prescription Use (Part 21 CFR 801 Subpart D)

Over-The-Counter Use (21 CFR 801 Subpart C)

CONTINUE ON A SEPARATE PAGE IF NEEDED.

{3}------------------------------------------------

This section applies only to requirements of the Paperwork Reduction Act of 1995.

DO NOT SEND YOUR COMPLETED FORM TO THE PRA STAFF EMAIL ADDRESS BELOW.

The burden time for this collection of information is estimated to average 79 hours per response, including the time to review instructions, search existing data sources, gather and maintain the data needed and complete and review the collection of information. Send comments regarding this burden estimate or any other aspect of this information collection, including suggestions for reducing this burden, to:

Department of Health and Human Services Food and Drug Administration Office of Chief Information Officer Paperwork Reduction Act (PRA) Staff PRAStaff@fda.hhs.gov

"An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB number."

{4}------------------------------------------------

1. SUBMITTER

Applicant:	BrightHeart7-11 boulevard HaussmannParis 75009, France
Contact:	Christophe GardellaChief Technical OfficerTel. +003669650566Email. christophe@brightheart.fr
Submission Correspondent:	Christophe Gardella
Date Prepared:	October 21, 2024

2. DEVICE

Device Trade Name:	Fetal EchoScan
Device Common Name:	Medical image analyzer
Classification Name	Radiological computer-assisted diagnostic software for lesions suspicious for cancer 21 CFR 892.2060
Regulatory Class:	Class II
Product Code:	POK

3. PREDICATE DEVICE

Predicate Device: Ultromics Ltd., EchoGo Pro [K201555]

The predicate device has not been subject to a design-related recall.

4. DEVICE DESCRIPTION

{5}------------------------------------------------

5. INTENDED USE/INDICATIONS FOR USE

When utilized by an interpreting physician, Fetal EchoScan provides information regarding the presence of any of the following suspicious radiographic findings:

overriding artery
septal defect at the cardiac crux
. abnormal relationshin of the outflow tracts
enlarged cardiothoracic ratio
right ventricular to left ventricular size discrepancy
tricuspid valve to mitral valve annular size discrepancy
pulmonary valve to aortic valve annular size discrepancy
. cardiac axis deviation

Fetal EchoScan is to be used with cardiac fetal ultrasound video clips containing interpretable 4-chamber, left ventricular outflow tract, right ventricular outflow tract standard views.

Fetal EchoScan is not intended for use in multiple pregnancies, cases of heterotaxy and postnatal ultrasound exams.

6. SUBSTANTIAL EQUIVALENCE

Technological Comparisons

The table below compares the key technological features of the subject devices to the predicate device (Ultromics Ltd., EchoGo Pro [K201555]).

Table 1. Technological Comparison

	Subject DeviceFetal EchoScan	Predicate DeviceEchoGo Pro v1.0.2
510(k) Number	TBD	K201555
	Subject Device	Predicate Device
	Fetal EchoScan	EchoGo Pro v1.0.2
Applicant	BrightHeart	Ultromics Ltd.
Classification Regulation	892.2060	892.2060
Product Code	POK	POK
Device Type	SaMD	SaMD
Software algorithm	Machine Learning Model	Machine Learning Model
Imaging Modality	Fetal Ultrasound	Adult Stress Echocardiography
Model Inputs	Fetal ultrasound study containingthe following views: 4 chamber, leftventricular outflow tract, rightventricular outflow tract	Electrocardiogram with apical 2chamber, 4 chamber and parasternalshort axis (SAX) views.
Model method	Suspicious radiographic findingscategorized into 2 groups:"classification" features arebased on the identification ofmorphological features withinthe video clip. "measurement" features arebased on the detection andsegmentation of key anatomicpoints	The software automatically registersimages, and segments and analysesselected regions of interest (ROI).Geometric parameters are calculatedfrom the approved contours and arefed into a fixed classification modelthat has been previously trained ondatasets with known outcomes. Theoutput of the model generates areport which contains a categoricalassessment as to whether the dataare consistent with significant CADor not.
Model trained to identify	Identifiable suspicious radiographicfindings of the fetal heart overriding artery septal defect at the cardiac crux abnormal relationship of theoutflow tracts enlarged cardiothoracic ratio right ventricular to leftventricular size discrepancy tricuspid valve to mitral valveannular size discrepancy pulmonary valve to aortic valveannular size discrepancy cardiac axis deviation	Coronary artery disease
	Subject DeviceFetal EchoScan	Predicate DeviceEchoGo Pro v1.0.2
Model Output	For each frame the softwareevaluates whether the findings are:present, absent, or inconclusive.A “record summary table” displaysa summary of results for each videoclip. An “exam summary table”displays a summary of the results forthe overall study.	Categorical assessment as towhether the data are suggestive of ahigher or lower possibility ofsignificant coronary artery diseaseor not

{6}------------------------------------------------

{7}------------------------------------------------

The Fetal EchoScan device and EchoGo Pro differ in that the Fetal EchoScan uses as input fetal ultrasound video clips, whereas the EchoGo Pro device uses as input stress echocardiograms.

As such, the Fetal EchoScan device identifies suspicious radiographic findings in the fetal heart, while EchoGo Pro identifies findings that are suspicious for the presence of coronary arterv disease in adults. This difference in technological characteristics reflects underlying imaging technologies and patient populations, but does not raise different questions of safety and effectiveness. Both devices are Machine Learning models intended to assist the physician in making a diagnosis of cardiovascular disease, and questions regarding the ability of the device to correctly perform this function are the same.

In summary, the subject and predicate devices share the same fundamental technological characteristics, namely the use of software to assist physicians who are interpreting radiological images in making a diagnosis. The differences in implementations described above do not raise different questions of safety and effectiveness, so EchoGo Pro device can be used as a predicate device for the Fetal EchoScan device.

7. PERFORMANCE DATA

Software Verification and Validation Testing

Software verification and validation testing was conducted and documentation was provided as recommended by FDA's Guidance for Industry and FDA Staff, "Guidance for the Content of Premarket Submissions for Software Contained in Medical Devices." and in accordance with IEC 62304:2016, Medical device software - Software life cycle processes.

Cybersecurity documentation was provided as recommended by FDA's Guidance for Industry and FDA Staff, "Content of Premarket Submissions for Management of Cybersecurity in Medical Devices".

Bench Testing

The following bench testing was performed to demonstrate substantial equivalence:

{8}------------------------------------------------

Standalone testing

The device performance for identification of suspicious radiographic findings was validated with a dataset of 877 clinically acquired fetal ultrasound exams during the 2nd trimester of pregnancy (18 to 24 weeks of gestational age), from 11 centers. Each exam consisted of all images and video clips recorded during the examination.

Demographic distribution:

gender: female
age: 18 vears or older ●
ethnicity/country: U.S.A. and France.

Information about how the reference standard was derived from the dataset:

The reference standard was derived from the dataset through a truthing process in which three ● pediatric cardiologists assessed the presence or absence of each of the eight findings, and majority voting was used.
The truthing process was conducted independently of the Fetal EchoScan device. ●

Description of how the independence of test data from training data was ensured:

The ultrasound examinations used for training and validation are entirely distinct from the examinations used in standalone testing.
The subject device was evaluated based on 2 different scenarios:
. One where all inconclusive device outputs are counted as negative. This corresponds to a "worst-case scenario" for sensitivity (all Inconclusive exams which are Positive for the ground truth are counted as False Negative), and a "best-case scenario" for specificity (all Inconclusive exams which are Negative for the ground truth are counted as True Negative).
. One where all inconclusive device outputs are counted as positive. This corresponds to a "worst-case scenario" for specificity (all Inconclusive exams which are Negative for the ground truth are counted as False Positive), and a "best-case scenario" for sensitivity (all Inconclusive exams which are Positive for the ground truth are counted as True Positive).

The AI system had a conclusive output regarding the presence of any finding for 98.8% (95% CL, 97.8 ; 99.3) of exams. The standalone testing demonstrated that Fetal EchoScan detects suspicious findings with high sensitivity and high specificity, as shown in Table 1.

Table 1. Sensitivity and Specificity (with 95% CI) of Fetal EchoScan for the detection of any suspicious radiographic finding and of each suspicious radiographic finding.

Note: Specificity for each suspicious radiographic finding is computed on exams negative to all findings according to the ground truth (i.e., excluding exams negative to the analyzed finding but positive to other findings).

{9}------------------------------------------------

	Inconclusive ExamsCounted as Negative		Inconclusive ExamsCounted as Positive
	Sensitivity(Worst-Case)	Specificity(Best-Case)	Sensitivity(Best-Case)	Specificity(Worst-Case)
Any suspicious findings	0.977 (0.954;0.989)	0.977 (0.961;0.987)	0.987 (0.967;0.995)	0.963 (0.944;0.976)
Overriding artery	0.894 (0.820;0.940)	0.989 (0.977;0.995)	0.942 (0.880;0.973)	0.979 (0.963;0.988)
Cardiac crux septaldefect	0.905 (0.823;0.951)	0.995 (0.985;0.998)	0.917 (0.838;0.959)	0.989 (0.977;0.995)
Abn. OT relationship	0.869 (0.781;0.925)	0.991 (0.979;0.996)	0.952 (0.884;0.981)	0.989 (0.977;0.995)
Enlarged CTR	0.955 (0.876;0.985)	1.000 (0.993;1.000)	0.955 (0.876;0.985)	1.000 (0.993;1.000)
Cardiac axis deviation	0.945 (0.851;0.981)	1.000 (0.993;1.000)	0.945 (0.851;0.981)	1.000 (0.993;1.000)
PV/AV size discrepancy	0.954 (0.914;0.975)	0.989 (0.977;0.995)	0.954 (0.914;0.975)	0.989 (0.977;0.995)
RV/LV size discrepancy	0.950 (0.900;0.975)	1.000 (0.993;1.000)	0.950 (0.900;0.975)	1.000 (0.993;1.000)
TV/MV sizediscrepancy	0.943 (0.896;0.970)	1.000 (0.993;1.000)	0.943 (0.896;0.970)	1.000 (0.993;1.000)

Stratified analysis by gestational age, by mother's age, by ultrasound machine make and model and by image quality indicated that performance was consistent across subgroups.

Reader study

Additionally, clinical performance of Fetal EchoScan was evaluated in a fully-crossed, multiple-reader multiple-case (MRMC) study, in which 14 readers reviewed 200 exams (18 to 24 weeks of gestational age). Readings were done in a randomized order, aided by Fetal EchoScan and unaided, with a 30 days washout period between both readings. Each exam consisted of all images and video clips recorded during the examination.

Demographic distribution:

gender: female
age: 18 years or older
ethnicity/country: U.S.A. and France. ●

Information about how the reference standard was derived from the dataset:

The reference standard was derived from the dataset through a truthing process in which three pediatric cardiologists assessed the presence or absence of each of the eight findings, and majority voting was used.
The truthing process was conducted independently of the Fetal EchoScan device. ●

{10}------------------------------------------------

Description of how the independence of test data from training data was ensured:

The ultrasound examinations used for training and validation are entirely distinct from the ● examinations used in standalone testing.
Table 2 and Figure 1 present reader performance when unaided and when aided by the device for identification of any suspicious radiographic finding and for each suspicious radiographic finding. The study results indicate that reviews by interpreting physicians were more accurate when aided by the Fetal EchoScan device compared to when unaided:
. The ROC AUC for detection of any suspicious radiographic finding was significantly higher in the aided compared to the unaided reading condition (see Figure 1): 0.974 (95% CI 0.957-0.990) vs 0.825 (0.741-0.908), p=0.002 (using the Dorfman-Berbaum-Metz and Obuchowski-Rockette method): +14.9% increase in AUC.
The mean sensitivity for identification of any claimed suspicious finding was 0.935 (0.892-0.978) in the aided reading condition vs 0.782 (0.686-0.878) in the unaided reading condition: +15.3% increase in sensitivity.
The mean specificity for identification of any claimed suspicious finding was 0.970 (0.949-0.991) in the aided reading condition vs 0.759 (0.630-0.887) in the unaided reading condition: +21.1% increase in specificity.

Table 2. Empirical ROC AUC Analysis by Suspicious Radiographic Finding

Note: Per-finding AUC is based on specificity computed excluding exams negative to the analyzed finding but possibly positive to other findings.

	Aided	Unaided	Aided minus Unaided
	Model Estimate AUC(95% CI)	Model Estimate AUC(95% CI)	Model Estimate Difference(95% CI)	DBM-ORp-value
Any suspicious findings	0.974 (0.957 ; 0.990)	0.825 (0.741 ; 0.908)	0.149 (0.066 ; 0.232)	0.002
Overriding artery	0.953 (0.916 ; 0.990)	0.803 (0.719 ; 0.888)	0.150 (0.063 ; 0.237)	0.002
Cardiac crux septal defect	0.971 (0.943 ; 0.999)	0.857 (0.782 ; 0.933)	0.114 (0.042 ; 0.186)	0.004
Abn. OT relationship	0.972 (0.953 ; 0.992)	0.832 (0.738 ; 0.927)	0.140 (0.048 ; 0.232)	0.005
Enlarged CTR	0.960 (0.930 ; 0.989)	0.746 (0.666 ; 0.826)	0.214 (0.131 ; 0.297)	<0.001
Cardiac axis deviation	0.967 (0.932 ; 1.000)	0.786 (0.704 ; 0.867)	0.181 (0.106 ; 0.256)	<0.001
PV/AV size discrepancy	0.979 (0.962 ; 0.997)	0.839 (0.756 ; 0.921)	0.140 (0.060 ; 0.221)	0.002
RV/LV size discrepancy	0.991 (0.983 ; 0.999)	0.868 (0.801 ; 0.936)	0.123 (0.055 ; 0.190)	0.001
TV/MV size discrepancy	0.964 (0.938 ; 0.990)	0.850 (0.779 ; 0.921)	0.114 (0.048 ; 0.179)	0.002

{11}------------------------------------------------

Figure 1. ROC AUC analysis for detection of any suspicious radiographic finding by OB-CYNs and MFMs in aided and unaided conditions.

Image /page/11/Figure/3 description: The image is a plot comparing the sensitivity and specificity of two different methods: AI-aided and unaided. The x-axis represents '1 - Specificity', while the y-axis represents 'Sensitivity'. The AI-aided method, represented by a red line, has an AUC of 97.4%, while the unaided method, represented by a gray line, has an AUC of 82.5%. A dashed line is also present, representing the line of no discrimination.

Stratified analysis by gestational age, by mother's age, BMI and race, by ultrasound machine make and model, image quality, reader specialty, reader country of practice and for each suspicious finding indicated that performance was consistent across subgroups.

Fetal EchoScan was validated only with Fujifilm, GE, Philips, Samsung and Toshiba ultrasound devices and is intended only to be used with these ultrasound vendors.

8. CONCLUSION

The results of the testing described above demonstrate that the Fetal EchoScan is as safe and effective as the predicate device and supports a determination of substantial equivalence.

Regulation Number and Section

§ 892.2060 Radiological computer-assisted diagnostic software for lesions suspicious of cancer.

(a)
Identification. A radiological computer-assisted diagnostic software for lesions suspicious of cancer is an image processing prescription device intended to aid in the characterization of lesions as suspicious for cancer identified on acquired medical images such as magnetic resonance, mammography, radiography, or computed tomography. The device characterizes lesions based on features or information extracted from the images and provides information about the lesion(s) to the user. Diagnostic and patient management decisions are made by the clinical user.(b)
Classification. Class II (special controls). The special controls for this device are:(1) Design verification and validation must include:
(i) A detailed description of the image analysis algorithms including, but not limited to, a detailed description of the algorithm inputs and outputs, each major component or block, and algorithm limitations.
(ii) A detailed description of pre-specified performance testing protocols and dataset(s) used to assess whether the device will improve reader performance as intended.
(iii) Results from performance testing protocols that demonstrate that the device improves reader performance in the intended use population when used in accordance with the instructions for use. The performance assessment must be based on appropriate diagnostic accuracy measures (
e.g., receiver operator characteristic plot, sensitivity, specificity, predictive value, and diagnostic likelihood ratio). The test dataset must contain sufficient numbers of cases from important cohorts (e.g., subsets defined by clinically relevant confounders, effect modifiers, concomitant diseases, and subsets defined by image acquisition characteristics) such that the performance estimates and confidence intervals of the device for these individual subsets can be characterized for the intended use population and imaging equipment.(iv) Standalone performance testing protocols and results of the device.
(v) Appropriate software documentation (
e.g., device hazard analysis; software requirements specification document; software design specification document; traceability analysis; and description of verification and validation activities including system level test protocol, pass/fail criteria, results, and cybersecurity).(2) Labeling must include:
(i) A detailed description of the patient population for which the device is indicated for use.
(ii) A detailed description of the intended reading protocol.
(iii) A detailed description of the intended user and recommended user training.
(iv) A detailed description of the device inputs and outputs.
(v) A detailed description of compatible imaging hardware and imaging protocols.
(vi) Warnings, precautions, and limitations, including situations in which the device may fail or may not operate at its expected performance level (
e.g., poor image quality or for certain subpopulations), as applicable.(vii) Detailed instructions for use.
(viii) A detailed summary of the performance testing, including: Test methods, dataset characteristics, results, and a summary of sub-analyses on case distributions stratified by relevant confounders (
e.g., lesion and organ characteristics, disease stages, and imaging equipment).