(99 days)
Not Found
Yes
The document explicitly states that the device is "machine learning-based" and uses "neural networks". It also mentions "The AI system" in the context of its performance.
No
The device is a diagnostic aid, providing information to assist physicians in identifying potential fetal heart abnormalities; it does not directly treat or prevent a disease or condition.
Yes
The device is described as a "computer-assisted diagnosis (CADx) software device" and is intended to provide "information that may be useful in rendering an accurate diagnosis." It also identifies "suspicious radiographic findings" that might be "suggestive of fetal congenital heart defects."
Yes
The device description explicitly states that Fetal EchoScan is a "cloud-based software-only device".
Based on the provided information, this device is not an In Vitro Diagnostic (IVD).
Here's why:
- IVDs analyze samples taken from the human body. The definition of an IVD involves examining specimens such as blood, urine, tissue, etc., in vitro (outside the body).
- Fetal EchoScan analyzes images of the human body. The input to Fetal EchoScan is fetal ultrasound video clips, which are images generated from within the body. The software processes these images to identify potential findings.
Therefore, Fetal EchoScan falls under the category of a medical image analysis software or computer-assisted diagnosis (CADx) device, not an IVD.
No
The clearance letter does not explicitly state that the FDA has reviewed and approved or cleared a PCCP for this specific device, nor does it reference any PCCP in the "Control Plan Authorized (PCCP) and relevant text" section.
Intended Use / Indications for Use
Fetal EchoScan is a machine learning-based computer-assisted diagnosis (CADx) software device indicated as an adjunct to fetal heart ultrasound examination in pregnant women aged 18 or older undergoing second-trimester anatomic ultrasound exams.
When utilized by an interpreting physician, Fetal EchoScan provides information regarding the presence of any of the following suspicious radiographic findings:
- overriding artery
- septal defect at the cardiac crux
- abnormal relationship of the outflow tracts
- enlarged cardiothoracic ratio
- right ventricular to left ventricular size discrepancy
- tricusrum valve to mitral valve annular size discrepancy
- pulmonary valve to aortic valve annular size discrepancy
- cardiac axis deviation
Fetal EchoScan is to be used with cardiac fetal ultrasound video clips containing interpretable 4-chamber, left ventricular outflow tract, right ventricular outflow tract standard views.
Fetal EchoScan is intended for use as a concurrent reading aid for interpreting physicians (OB-GYN, MFM). It does not replace the role of the physician or of other diagnostic testing in the standard of care. When utilized by an interpreting physician, this device provides information that may be useful in rendering an accurate diagnosis regarding the potential presence of morphological abnormalities that might be suggestive of fetal congenital heart defects that may be useful in determining the need for additional exams.
Fetal EchoScan is not intended for use in multiple pregnancies, cases of heterotaxy, and postnatal ultrasound exams.
Product codes
POK
Device Description
Fetal EchoScan is a cloud-based software-only device which uses neural networks to detect suspicious cardiac radiographic findings for further review by trained and qualified physicians. Fetal EchoScan is intended to be used as an adjunct to the interpretation of the second-trimester fetal anatomic ultrasound exam performed between 18 and 24 weeks of gestation, for pregnant women aged 18 or more.
Mentions image processing
Not Found
Mentions AI, DNN, or ML
Fetal EchoScan is a machine learning-based computer-assisted diagnosis (CADx) software device
Fetal EchoScan is a cloud-based software-only device which uses neural networks to detect suspicious cardiac radiographic findings
Software algorithm: Machine Learning Model
Both devices are Machine Learning models
Input Imaging Modality
Fetal Ultrasound
Fetal ultrasound study containing the following views: 4 chamber, left ventricular outflow tract, right ventricular outflow tract
cardiac fetal ultrasound video clips containing interpretable 4-chamber, left ventricular outflow tract, right ventricular outflow tract standard views.
Anatomical Site
Fetal heart
Indicated Patient Age Range
18 or older
Intended User / Care Setting
interpreting physicians (OB-GYN, MFM)
Description of the training set, sample size, data source, and annotation protocol
The ultrasound examinations used for training and validation are entirely distinct from the examinations used in standalone testing.
The ultrasound examinations used for training and validation are entirely distinct from the examinations used in standalone testing.
Description of the test set, sample size, data source, and annotation protocol
Standalone testing:
Dataset: 877 clinically acquired fetal ultrasound exams
Data Source: 11 centers
Annotation Protocol: The reference standard was derived from the dataset through a truthing process in which three pediatric cardiologists assessed the presence or absence of each of the eight findings, and majority voting was used. The truthing process was conducted independently of the Fetal EchoScan device.
Reader Study:
Sample Size: 200 exams
Annotation Protocol: The reference standard was derived from the dataset through a truthing process in which three pediatric cardiologists assessed the presence or absence of each of the eight findings, and majority voting was used. The truthing process was conducted independently of the Fetal EchoScan device.
Summary of Performance Studies (study type, sample size, AUC, MRMC, standalone performance, key results)
Standalone Testing:
Study Type: Bench Testing / Standalone testing
Sample Size: 877 fetal ultrasound exams
Standalone Performance: The AI system had a conclusive output regarding the presence of any finding for 98.8% (95% CL, 97.8 ; 99.3) of exams. The standalone testing demonstrated that Fetal EchoScan detects suspicious findings with high sensitivity and high specificity, as shown in Table 1.
Key Results:
Sensitivity (Worst-Case) for Any suspicious findings: 0.977 (0.954; 0.989)
Specificity (Best-Case) for Any suspicious findings: 0.977 (0.961; 0.987)
Sensitivity (Best-Case) for Any suspicious findings: 0.987 (0.967; 0.995)
Specificity (Worst-Case) for Any suspicious findings: 0.963 (0.944; 0.976)
Performance was consistent across subgroups for gestational age, mother's age, ultrasound machine make and model, and image quality.
Reader Study:
Study Type: Multiple-Reader Multiple-Case (MRMC) study
Sample Size: 14 readers reviewed 200 exams
Key Results:
The ROC AUC for detection of any suspicious radiographic finding was significantly higher in the aided compared to the unaided reading condition: 0.974 (95% CI 0.957-0.990) vs 0.825 (0.741-0.908), p=0.002 (using the Dorfman-Berbaum-Metz and Obuchowski-Rockette method): +14.9% increase in AUC.
The mean sensitivity for identification of any claimed suspicious finding was 0.935 (0.892-0.978) in the aided reading condition vs 0.782 (0.686-0.878) in the unaided reading condition: +15.3% increase in sensitivity.
The mean specificity for identification of any claimed suspicious finding was 0.970 (0.949-0.991) in the aided reading condition vs 0.759 (0.630-0.887) in the unaided reading condition: +21.1% increase in specificity.
Performance was consistent across subgroups for gestational age, mother's age, BMI and race, ultrasound machine make and model, image quality, reader specialty, reader country of practice and for each suspicious finding.
Key Metrics (Sensitivity, Specificity, PPV, NPV, etc.)
Standalone Testing:
Sensitivity (Worst-Case): 0.977 (0.954; 0.989) for Any suspicious findings
Specificity (Best-Case): 0.977 (0.961; 0.987) for Any suspicious findings
Sensitivity (Best-Case): 0.987 (0.967; 0.995) for Any suspicious findings
Specificity (Worst-Case): 0.963 (0.944; 0.976) for Any suspicious findings
Specific values for each finding are provided in Table 1, including for overriding artery, cardiac crux septal defect, Abn. OT relationship, Enlarged CTR, Cardiac axis deviation, PV/AV size discrepancy, RV/LV size discrepancy, TV/MV size discrepancy.
Reader Study:
ROC AUC for Any suspicious findings: Aided: 0.974 (0.957 ; 0.990), Unaided: 0.825 (0.741 ; 0.908)
Mean sensitivity for Any suspicious finding: Aided: 0.935 (0.892-0.978), Unaided: 0.782 (0.686-0.878)
Mean specificity for Any suspicious finding: Aided: 0.970 (0.949-0.991), Unaided: 0.759 (0.630-0.887)
AUC values for each suspicious radiographic finding are provided in Table 2, including for Overriding artery, Cardiac crux septal defect, Abn. OT relationship, Enlarged CTR, Cardiac axis deviation, PV/AV size discrepancy, RV/LV size discrepancy, TV/MV size discrepancy.
Predicate Device(s):
Ultromics Ltd., EchoGo Pro [K201555]
Reference Device(s):
Not Found
Predetermined Change Control Plan (PCCP) - All Relevant Information
Not Found
§ 892.2060 Radiological computer-assisted diagnostic software for lesions suspicious of cancer.
(a)
Identification. A radiological computer-assisted diagnostic software for lesions suspicious of cancer is an image processing prescription device intended to aid in the characterization of lesions as suspicious for cancer identified on acquired medical images such as magnetic resonance, mammography, radiography, or computed tomography. The device characterizes lesions based on features or information extracted from the images and provides information about the lesion(s) to the user. Diagnostic and patient management decisions are made by the clinical user.(b)
Classification. Class II (special controls). The special controls for this device are:(1) Design verification and validation must include:
(i) A detailed description of the image analysis algorithms including, but not limited to, a detailed description of the algorithm inputs and outputs, each major component or block, and algorithm limitations.
(ii) A detailed description of pre-specified performance testing protocols and dataset(s) used to assess whether the device will improve reader performance as intended.
(iii) Results from performance testing protocols that demonstrate that the device improves reader performance in the intended use population when used in accordance with the instructions for use. The performance assessment must be based on appropriate diagnostic accuracy measures (
e.g., receiver operator characteristic plot, sensitivity, specificity, predictive value, and diagnostic likelihood ratio). The test dataset must contain sufficient numbers of cases from important cohorts (e.g., subsets defined by clinically relevant confounders, effect modifiers, concomitant diseases, and subsets defined by image acquisition characteristics) such that the performance estimates and confidence intervals of the device for these individual subsets can be characterized for the intended use population and imaging equipment.(iv) Standalone performance testing protocols and results of the device.
(v) Appropriate software documentation (
e.g., device hazard analysis; software requirements specification document; software design specification document; traceability analysis; and description of verification and validation activities including system level test protocol, pass/fail criteria, results, and cybersecurity).(2) Labeling must include:
(i) A detailed description of the patient population for which the device is indicated for use.
(ii) A detailed description of the intended reading protocol.
(iii) A detailed description of the intended user and recommended user training.
(iv) A detailed description of the device inputs and outputs.
(v) A detailed description of compatible imaging hardware and imaging protocols.
(vi) Warnings, precautions, and limitations, including situations in which the device may fail or may not operate at its expected performance level (
e.g., poor image quality or for certain subpopulations), as applicable.(vii) Detailed instructions for use.
(viii) A detailed summary of the performance testing, including: Test methods, dataset characteristics, results, and a summary of sub-analyses on case distributions stratified by relevant confounders (
e.g., lesion and organ characteristics, disease stages, and imaging equipment).
0
Image /page/0/Picture/0 description: The image contains the logo of the U.S. Food & Drug Administration (FDA). On the left is the Department of Health & Human Services logo. To the right of that is the FDA logo, which is a blue square with the letters "FDA" in white. To the right of the blue square is the text "U.S. FOOD & DRUG ADMINISTRATION" in blue.
November 14, 2024
BrightHeart Christophe Gardella CTO 7-11 boulevard Haussmann Paris, 75009, France
Re: K242342
Trade/Device Name: Fetal EchoScan Regulation Number: 21 CFR 892.2060 Regulation Name: Radiological computer-assisted diagnostic software for lesions suspicious of cancer Regulatory Class: Class II Product Code: POK Dated: October 24, 2024 Received: October 24, 2024
Dear Christophe Gardella:
We have reviewed your section 510(k) premarket notification of intent to market the device referenced above and have determined the device is substantially equivalent (for the indications for use stated in the enclosure) to legally marketed predicate devices marketed in interstate commerce prior to May 28, 1976, the enactment date of the Medical Device Amendments, or to devices that have been reclassified in accordance with the provisions of the Federal Food, Drug, and Cosmetic Act (the Act) that do not require approval of a premarket approval application (PMA). You may, therefore, market the device, subject to the general controls provisions of the Act. Although this letter refers to your product as a device, please be aware that some cleared products may instead be combination products. The 510(k) Premarket Notification Database available at https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpm/pm.cfm identifies combination product submissions. The general controls provisions of the Act include requirements for annual registration, listing of devices, good manufacturing practice, labeling, and prohibitions against misbranding and adulteration. Please note: CDRH does not evaluate information related to contract liability warranties. We remind you, however, that device labeling must be truthful and not misleading.
If your device is classified (see above) into either class II (Special Controls) or class III (PMA), it may be subject to additional controls. Existing major regulations affecting your device can be found in the Code of Federal Regulations, Title 21, Parts 800 to 898. In addition, FDA may publish further announcements concerning your device in the Federal Register.
Additional information about changes that may require a new premarket notification are provided in the FDA guidance documents entitled "Deciding When to Submit a 510(k) for a Change to an Existing Device" (https://www.fda.gov/media/99812/download) and "Deciding When to Submit a 510(k) for a Software Change to an Existing Device" (https://www.fda.gov/media/99785/download).
Your device is also subject to, among other requirements, the Quality System (QS) regulation (21 CFR Part 820), which includes, but is not limited to. 21 CFR 820.30. Design controls; 21 CFR 820.90. Nonconforming product; and 21 CFR 820.100, Corrective and preventive action. Please note that regardless of whether a change requires premarket review, the QS regulation requires device manufacturers to review and approve changes to device design and production (21 CFR 820.30 and 21 CFR 820.70) and document changes and approvals in the device master record (21 CFR 820.181).
1
Please be advised that FDA's issuance of a substantial equivalence determination does not mean that FDA has made a determination that your device complies with other requirements of the Act or any Federal statutes and regulations administered by other Federal agencies. You must comply with all the Act's requirements, including, but not limited to: registration and listing (21 CFR Part 807); labeling (21 CFR Part 801); medical device reporting of medical device-related adverse events) (21 CFR Part 803) for devices or postmarketing safety reporting (21 CFR Part 4, Subpart B) for combination products (see https://www.fda.gov/combination-products/guidance-regulatoryinformation/postmarketing-safety-reporting-combination-products); good manufacturing practice requirements as set forth in the quality systems (QS) regulation (21 CFR Part 820) for devices or current good manufacturing practices (21 CFR Part 4, Subpart A) for combination products; and, if applicable, the electronic product radiation control provisions (Sections 531-542 of the Act); 21 CFR Parts 1000-1050.
All medical devices, including Class I and unclassified devices and combination product device constituent parts are required to be in compliance with the final Unique Device Identification System rule ("UDI Rule"). The UDI Rule requires, among other things, that a device bear a unique device identifier (UDD) on its label and package (21 CFR 801.20(a)) unless an exception or alternative applies (21 CFR 801.20(b)) and that the dates on the device label be formatted in accordance with 21 CFR 801.18. The UDI Rule (21 CFR 830.300(a) and 830.320(b)) also requires that certain information be submitted to the Global Unique Device Identification Database (GUDID) (21 CFR Part 830 Subpart E). For additional information on these requirements, please see the UDI System webpage at https://www.fda.gov/medical-device-advice-comprehensive-regulatory-assistance/unique-deviceidentification-system-udi-system.
Also, please note the regulation entitled, "Misbranding by reference to premarket notification" (21 CFR 807.97). For questions regarding the reporting of adverse events under the MDR regulation (21 CFR Part 803), please go to https://www.fda.gov/medical-device-safety/medical-device-reporting-mdr-how-report-medicaldevice-problems.
For comprehensive regulatory information about mediation-emitting products, including information about labeling regulations, please see Device Advice (https://www.fda.gov/medical-devices/deviceadvice-comprehensive-regulatory-assistance) and CDRH Learn (https://www.fda.gov/training-and-continuingeducation/cdrh-learn). Additionally, you may contact the Division of Industry and Consumer Education (DICE) to ask a question about a specific regulatory topic. See the DICE website (https://www.fda.gov/medical-devices/deviceadvice-comprehensive-regulatory-assistance/contact-us-division-industry-and-consumer-education-dice) for more information or contact DICE by email (DICE@fda.hhs.gov) or phone (1-800-638-2041 or 301-796-7100).
Sincerely.
Jessica Lamb
Jessica Lamb, Ph.D. Assistant Director, Imaging Software Team DHT8B: Division of Radiological Imaging Devices and Electronic Products OHT8: Office of Radiological Health Office of Product Evaluation and Quality Center for Devices and Radiological Health
2
Indications for Use
510(k) Number (if known) K242342
Device Name Fetal EchoScan
Indications for Use (Describe)
Fetal EchoScan is a machine learning-based computer-assisted diagnosis (CADx) software device indicated as an adjunct to fetal heart ultrasound examination in pregnant women aged 18 or older undergoing second-trimester anatomic ultrasound exams.
When utilized by an interpreting physician, Fetal EchoScan provides information regarding the presence of any of the following suspicious radiographic findings:
- overriding artery
- septal defect at the cardiac crux
- abnormal relationship of the outflow tracts
- enlarged cardiothoracic ratio
- right ventricular to left ventricular size discrepancy
- tricuspid valve to mitral valve annular size discrepancy
- pulmonary valve to aortic valve annular size discrepancy
- cardiac axis deviation
Fetal EchoScan is to be used with cardiac fetal ultrasound video clips containing interpretable 4-chamber, left ventricular outflow tract, right ventricular outflow tract standard views.
Fetal EchoScan is intended for use as a concurrent reading aid for interpreting physicians (OB-GYN, MFM). It does not replace the role of the physician or of other diagnostic testing in the standard of care. When utilized by an interpreting physician, this device provides information that may be useful in rendering an accurate diagnosis regarding the potential presence of morphological abnormalities that might be suggestive of fetal congenital heart defects that may be useful in determining the need for additional exams.
Fetal EchoScan is not intended for use in multiple pregnancies, cases of heterotaxy, and postnatal ultrasound exams.
Type of Use (Select one or both, as applicable)
X Prescription Use (Part 21 CFR 801 Subpart D)
Over-The-Counter Use (21 CFR 801 Subpart C)
CONTINUE ON A SEPARATE PAGE IF NEEDED.
3
This section applies only to requirements of the Paperwork Reduction Act of 1995.
DO NOT SEND YOUR COMPLETED FORM TO THE PRA STAFF EMAIL ADDRESS BELOW.
The burden time for this collection of information is estimated to average 79 hours per response, including the time to review instructions, search existing data sources, gather and maintain the data needed and complete and review the collection of information. Send comments regarding this burden estimate or any other aspect of this information collection, including suggestions for reducing this burden, to:
Department of Health and Human Services Food and Drug Administration Office of Chief Information Officer Paperwork Reduction Act (PRA) Staff PRAStaff@fda.hhs.gov
"An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB number."
4
1. SUBMITTER
| Applicant: | BrightHeart
7-11 boulevard Haussmann
Paris 75009, France |
|---------------------------|----------------------------------------------------------------------------------------------------------|
| Contact: | Christophe Gardella
Chief Technical Officer
Tel. +003669650566
Email. christophe@brightheart.fr |
| Submission Correspondent: | Christophe Gardella |
| Date Prepared: | October 21, 2024 |
2. DEVICE
| Device Trade Name: | Fetal EchoScan |
|---|---|
| Device Common Name: | Medical image analyzer |
| Classification Name | Radiological computer-assisted diagnostic software for lesions suspicious for cancer 21 CFR 892.2060 |
| Regulatory Class: | Class II |
| Product Code: | POK |
3. PREDICATE DEVICE
Predicate Device: Ultromics Ltd., EchoGo Pro [K201555]
The predicate device has not been subject to a design-related recall.
4. DEVICE DESCRIPTION
Fetal EchoScan is a cloud-based software-only device which uses neural networks to detect suspicious cardiac radiographic findings for further review by trained and qualified physicians. Fetal EchoScan is intended to be used as an adjunct to the interpretation of the second-trimester fetal anatomic ultrasound exam performed between 18 and 24 weeks of gestation, for pregnant women aged 18 or more.
5
5. INTENDED USE/INDICATIONS FOR USE
Fetal EchoScan is a machine learning-based computer-assisted diagnosis (CADx) software device indicated as an adjunct to fetal heart ultrasound examination in pregnant women aged 18 or older undergoing second-trimester anatomic ultrasound exams.
When utilized by an interpreting physician, Fetal EchoScan provides information regarding the presence of any of the following suspicious radiographic findings:
- overriding artery
- septal defect at the cardiac crux
- . abnormal relationshin of the outflow tracts
- enlarged cardiothoracic ratio
- right ventricular to left ventricular size discrepancy
- tricuspid valve to mitral valve annular size discrepancy
- pulmonary valve to aortic valve annular size discrepancy
- . cardiac axis deviation
Fetal EchoScan is to be used with cardiac fetal ultrasound video clips containing interpretable 4-chamber, left ventricular outflow tract, right ventricular outflow tract standard views.
Fetal EchoScan is intended for use as a concurrent reading aid for interpreting physicians (OB-GYN, MFM). It does not replace the role of the physician or of other diagnostic testing in the standard of care. When utilized by an interpreting physician, this device provides information that may be useful in rendering an accurate diagnosis regarding the potential presence of morphological abnormalities that might be suggestive of fetal congenital heart defects that may be useful in determining the need for additional exams.
Fetal EchoScan is not intended for use in multiple pregnancies, cases of heterotaxy and postnatal ultrasound exams.
6. SUBSTANTIAL EQUIVALENCE
Technological Comparisons
The table below compares the key technological features of the subject devices to the predicate device (Ultromics Ltd., EchoGo Pro [K201555]).
Table 1. Technological Comparison
| | Subject Device
Fetal EchoScan | Predicate Device
EchoGo Pro v1.0.2 |
|---------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| 510(k) Number | TBD | K201555 |
| | Subject Device | Predicate Device |
| | Fetal EchoScan | EchoGo Pro v1.0.2 |
| Applicant | BrightHeart | Ultromics Ltd. |
| Classification Regulation | 892.2060 | 892.2060 |
| Product Code | POK | POK |
| Device Type | SaMD | SaMD |
| Software algorithm | Machine Learning Model | Machine Learning Model |
| Imaging Modality | Fetal Ultrasound | Adult Stress Echocardiography |
| Model Inputs | Fetal ultrasound study containing
the following views: 4 chamber, left
ventricular outflow tract, right
ventricular outflow tract | Electrocardiogram with apical 2
chamber, 4 chamber and parasternal
short axis (SAX) views. |
| Model method | Suspicious radiographic findings
categorized into 2 groups:
"classification" features are
based on the identification of
morphological features within
the video clip. "measurement" features are
based on the detection and
segmentation of key anatomic
points | The software automatically registers
images, and segments and analyses
selected regions of interest (ROI).
Geometric parameters are calculated
from the approved contours and are
fed into a fixed classification model
that has been previously trained on
datasets with known outcomes. The
output of the model generates a
report which contains a categorical
assessment as to whether the data
are consistent with significant CAD
or not. |
| Model trained to identify | Identifiable suspicious radiographic
findings of the fetal heart overriding artery septal defect at the cardiac crux abnormal relationship of the
outflow tracts enlarged cardiothoracic ratio right ventricular to left
ventricular size discrepancy tricuspid valve to mitral valve
annular size discrepancy pulmonary valve to aortic valve
annular size discrepancy cardiac axis deviation | Coronary artery disease |
| | Subject Device
Fetal EchoScan | Predicate Device
EchoGo Pro v1.0.2 |
| Model Output | For each frame the software
evaluates whether the findings are:
present, absent, or inconclusive.
A “record summary table” displays
a summary of results for each video
clip. An “exam summary table”
displays a summary of the results for
the overall study. | Categorical assessment as to
whether the data are suggestive of a
higher or lower possibility of
significant coronary artery disease
or not |
6
7
The Fetal EchoScan device and EchoGo Pro differ in that the Fetal EchoScan uses as input fetal ultrasound video clips, whereas the EchoGo Pro device uses as input stress echocardiograms.
As such, the Fetal EchoScan device identifies suspicious radiographic findings in the fetal heart, while EchoGo Pro identifies findings that are suspicious for the presence of coronary arterv disease in adults. This difference in technological characteristics reflects underlying imaging technologies and patient populations, but does not raise different questions of safety and effectiveness. Both devices are Machine Learning models intended to assist the physician in making a diagnosis of cardiovascular disease, and questions regarding the ability of the device to correctly perform this function are the same.
In summary, the subject and predicate devices share the same fundamental technological characteristics, namely the use of software to assist physicians who are interpreting radiological images in making a diagnosis. The differences in implementations described above do not raise different questions of safety and effectiveness, so EchoGo Pro device can be used as a predicate device for the Fetal EchoScan device.
7. PERFORMANCE DATA
Software Verification and Validation Testing
Software verification and validation testing was conducted and documentation was provided as recommended by FDA's Guidance for Industry and FDA Staff, "Guidance for the Content of Premarket Submissions for Software Contained in Medical Devices." and in accordance with IEC 62304:2016, Medical device software - Software life cycle processes.
Cybersecurity documentation was provided as recommended by FDA's Guidance for Industry and FDA Staff, "Content of Premarket Submissions for Management of Cybersecurity in Medical Devices".
Bench Testing
The following bench testing was performed to demonstrate substantial equivalence:
8
Standalone testing
The device performance for identification of suspicious radiographic findings was validated with a dataset of 877 clinically acquired fetal ultrasound exams during the 2nd trimester of pregnancy (18 to 24 weeks of gestational age), from 11 centers. Each exam consisted of all images and video clips recorded during the examination.
Demographic distribution:
- gender: female
- age: 18 vears or older ●
- ethnicity/country: U.S.A. and France.
Information about how the reference standard was derived from the dataset:
- The reference standard was derived from the dataset through a truthing process in which three ● pediatric cardiologists assessed the presence or absence of each of the eight findings, and majority voting was used.
- The truthing process was conducted independently of the Fetal EchoScan device. ●
Description of how the independence of test data from training data was ensured:
-
The ultrasound examinations used for training and validation are entirely distinct from the examinations used in standalone testing.
The subject device was evaluated based on 2 different scenarios: -
. One where all inconclusive device outputs are counted as negative. This corresponds to a "worst-case scenario" for sensitivity (all Inconclusive exams which are Positive for the ground truth are counted as False Negative), and a "best-case scenario" for specificity (all Inconclusive exams which are Negative for the ground truth are counted as True Negative).
-
. One where all inconclusive device outputs are counted as positive. This corresponds to a "worst-case scenario" for specificity (all Inconclusive exams which are Negative for the ground truth are counted as False Positive), and a "best-case scenario" for sensitivity (all Inconclusive exams which are Positive for the ground truth are counted as True Positive).
The AI system had a conclusive output regarding the presence of any finding for 98.8% (95% CL, 97.8 ; 99.3) of exams. The standalone testing demonstrated that Fetal EchoScan detects suspicious findings with high sensitivity and high specificity, as shown in Table 1.
Table 1. Sensitivity and Specificity (with 95% CI) of Fetal EchoScan for the detection of any suspicious radiographic finding and of each suspicious radiographic finding.
Note: Specificity for each suspicious radiographic finding is computed on exams negative to all findings according to the ground truth (i.e., excluding exams negative to the analyzed finding but positive to other findings).
9
| | Inconclusive Exams
Counted as Negative | | Inconclusive Exams
Counted as Positive | |
|-------------------------------|-------------------------------------------|----------------------------|-------------------------------------------|-----------------------------|
| | Sensitivity
(Worst-Case) | Specificity
(Best-Case) | Sensitivity
(Best-Case) | Specificity
(Worst-Case) |
| Any suspicious findings | 0.977 (0.954;
0.989) | 0.977 (0.961;
0.987) | 0.987 (0.967;
0.995) | 0.963 (0.944;
0.976) |
| Overriding artery | 0.894 (0.820;
0.940) | 0.989 (0.977;
0.995) | 0.942 (0.880;
0.973) | 0.979 (0.963;
0.988) |
| Cardiac crux septal
defect | 0.905 (0.823;
0.951) | 0.995 (0.985;
0.998) | 0.917 (0.838;
0.959) | 0.989 (0.977;
0.995) |
| Abn. OT relationship | 0.869 (0.781;
0.925) | 0.991 (0.979;
0.996) | 0.952 (0.884;
0.981) | 0.989 (0.977;
0.995) |
| Enlarged CTR | 0.955 (0.876;
0.985) | 1.000 (0.993;
1.000) | 0.955 (0.876;
0.985) | 1.000 (0.993;
1.000) |
| Cardiac axis deviation | 0.945 (0.851;
0.981) | 1.000 (0.993;
1.000) | 0.945 (0.851;
0.981) | 1.000 (0.993;
1.000) |
| PV/AV size discrepancy | 0.954 (0.914;
0.975) | 0.989 (0.977;
0.995) | 0.954 (0.914;
0.975) | 0.989 (0.977;
0.995) |
| RV/LV size discrepancy | 0.950 (0.900;
0.975) | 1.000 (0.993;
1.000) | 0.950 (0.900;
0.975) | 1.000 (0.993;
1.000) |
| TV/MV size
discrepancy | 0.943 (0.896;
0.970) | 1.000 (0.993;
1.000) | 0.943 (0.896;
0.970) | 1.000 (0.993;
1.000) |
Stratified analysis by gestational age, by mother's age, by ultrasound machine make and model and by image quality indicated that performance was consistent across subgroups.
Reader study
Additionally, clinical performance of Fetal EchoScan was evaluated in a fully-crossed, multiple-reader multiple-case (MRMC) study, in which 14 readers reviewed 200 exams (18 to 24 weeks of gestational age). Readings were done in a randomized order, aided by Fetal EchoScan and unaided, with a 30 days washout period between both readings. Each exam consisted of all images and video clips recorded during the examination.
Demographic distribution:
- gender: female
- age: 18 years or older
- ethnicity/country: U.S.A. and France. ●
Information about how the reference standard was derived from the dataset:
- The reference standard was derived from the dataset through a truthing process in which three pediatric cardiologists assessed the presence or absence of each of the eight findings, and majority voting was used.
- The truthing process was conducted independently of the Fetal EchoScan device. ●
10
Description of how the independence of test data from training data was ensured:
-
The ultrasound examinations used for training and validation are entirely distinct from the ● examinations used in standalone testing.
Table 2 and Figure 1 present reader performance when unaided and when aided by the device for identification of any suspicious radiographic finding and for each suspicious radiographic finding. The study results indicate that reviews by interpreting physicians were more accurate when aided by the Fetal EchoScan device compared to when unaided: -
. The ROC AUC for detection of any suspicious radiographic finding was significantly higher in the aided compared to the unaided reading condition (see Figure 1): 0.974 (95% CI 0.957-0.990) vs 0.825 (0.741-0.908), p=0.002 (using the Dorfman-Berbaum-Metz and Obuchowski-Rockette method): +14.9% increase in AUC.
-
The mean sensitivity for identification of any claimed suspicious finding was 0.935 (0.892-0.978) in the aided reading condition vs 0.782 (0.686-0.878) in the unaided reading condition: +15.3% increase in sensitivity.
-
The mean specificity for identification of any claimed suspicious finding was 0.970 (0.949-0.991) in the aided reading condition vs 0.759 (0.630-0.887) in the unaided reading condition: +21.1% increase in specificity.
Table 2. Empirical ROC AUC Analysis by Suspicious Radiographic Finding
Note: Per-finding AUC is based on specificity computed excluding exams negative to the analyzed finding but possibly positive to other findings.
| Aided | Unaided | Aided minus Unaided | ||
|---|---|---|---|---|
| Model Estimate AUC | ||||
| (95% CI) | Model Estimate AUC | |||
| (95% CI) | Model Estimate Difference | |||
| (95% CI) | DBM-OR | |||
| p-value | ||||
| Any suspicious findings | 0.974 (0.957 ; 0.990) | 0.825 (0.741 ; 0.908) | 0.149 (0.066 ; 0.232) | 0.002 |
| Overriding artery | 0.953 (0.916 ; 0.990) | 0.803 (0.719 ; 0.888) | 0.150 (0.063 ; 0.237) | 0.002 |
| Cardiac crux septal defect | 0.971 (0.943 ; 0.999) | 0.857 (0.782 ; 0.933) | 0.114 (0.042 ; 0.186) | 0.004 |
| Abn. OT relationship | 0.972 (0.953 ; 0.992) | 0.832 (0.738 ; 0.927) | 0.140 (0.048 ; 0.232) | 0.005 |
| Enlarged CTR | 0.960 (0.930 ; 0.989) | 0.746 (0.666 ; 0.826) | 0.214 (0.131 ; 0.297) |