EFAI PNXXR is a software workflow tool designed to aid the clinical assessment of adult (22 years of age or older) Posteroanterior (PA) view Chest X-Ray cases with features suggestive of pneumothorax in the medical care environment. EFAI PNXXR analyzes cases using an artificial intelligence algorithm to identify suspected findings. It makes case-level output available to a PACS/workstation for worklist prioritization or triage. EFAI PNXXR is not intended to direct attention to specific portions or anomalies of an image. Its results are not intended to be used on a stand-alone basis for clinical decision-making nor is it intended to rule out pneumothorax or otherwise preclude clinical assessment of X-Ray cases.

Device Description

EFAI ChestSuite XR Pneumothorax Assessment System, herein referred to as EFAI PNXXR, is a radiological computer-assisted triage and notification software system. The software uses deep learning techniques to automatically analyze PA chest x-rays and sends notification messages to the picture archiving and communication system (PACS)/workstation to allow suspicious findings of pneumothorax to be identified.

The device is intended to provide a passive notification through the PACS/workstation to the radiologists indicating the existence of a case that may potentially benefit from the prioritization. It does not mark, highlight, or direct users' attention to a specific location on the original chest X-ray. The device aims to aid in prioritization and triage of radiological medical images only.

The deployment environment is recommended to be in a local network with an existing hospitalgrade IT system in place. EFAI PNXXR should be installed on a specialized server supporting deep learning processing. The configurations are only being operated by the manufacturer:

· Local network setting of input and output destinations;

AI/ML Overview

Here's a breakdown of the acceptance criteria and study details for EFAI ChestSuite XR Pneumothorax Assessment System, based on the provided FDA 510(k) summary:

Acceptance Criteria and Device Performance

1. Acceptance Criteria Table and Reported Device Performance

Metric	Acceptance Criteria	Reported Device Performance (EFAI PNXXR)
Sensitivity	Lower bound of 95% CI should exceed 0.8	0.97 (95% CI=0.94-0.99)
Specificity	Lower bound of 95% CI should exceed 0.8	0.98 (95% CI=0.96-0.99)
AUC	Not explicitly stated as an acceptance criterion threshold, but reported for effectiveness comparison.	0.99 (95% CI=0.98-1.00)
Processing Time	Comparable with the predicate device (red dot™)	23.3 seconds (95% CI=[23.2, 23.4]) compared to predicate's 29.3 seconds

2. Sample Size and Data Provenance for Test Set

Sample Size: 800 anonymized Chest X-ray images.
Data Provenance: Retrospective, multi-center study. Data was collected from 3 institutions in the US and 1 institution outside the US (OUS). The dataset was explicitly stated as not being used for model development or analytical validation.

3. Number of Experts and Qualifications for Ground Truth

Number of Experts: Three.
Qualifications of Experts: US board-certified radiologists.

4. Adjudication Method for Test Set

Adjudication Method: Majority agreement among the three board-certified radiologists. (This generally implies a 2-out-of-3 consensus.)

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

Was an MRMC study done? No, an MRMC comparative effectiveness study was not reported. The study focused on the standalone performance of the algorithm against a defined ground truth and compared its performance metrics (sensitivity, specificity, AUC, processing time) to a predicate device.

6. Standalone Performance (Algorithm Only)

Was a standalone performance study done? Yes, a retrospective, blinded, multicenter study was performed to establish the standalone performance of EFAI PNXXR. The study compared the device's pneumothorax classification performance and processing time against the predicate device (red dot™).

7. Type of Ground Truth Used

Type of Ground Truth: Expert consensus. Specifically, the reference standard (ground truth) was generated by the majority agreement between the three board-certified radiologists.

8. Sample Size for Training Set

The sample size for the training set is not specified in the provided document. The document mentions that the test set was "Neither of the datasets were used as part of the EFAI PNXXR model development or analytical validation testing," implying a separate (and unquantified) training dataset.

9. How Ground Truth for Training Set Was Established

The method for establishing ground truth for the training set is not specified in the provided document. It only states that the test set (from section 2 above) was not part of the model development.

Summary

{0}------------------------------------------------

Image /page/0/Picture/0 description: The image contains the logo of the U.S. Food and Drug Administration (FDA). On the left is the Department of Health & Human Services logo. To the right of that is the FDA logo, which is a blue square with the letters "FDA" in white. To the right of the square is the text "U.S. FOOD & DRUG ADMINISTRATION" in blue.

Ever Fortune AI Co., Ltd. % Ti-Hao Wang Chief Technology Officer RM.D. 8F, NO. 573, SEC 2, Taiwan Blvd, West Dist. Taichung City, 403020 TAIWAN

November 8, 2022

Re: K221552

Trade/Device Name: EFAI ChestSuite XR Pneumothorax Assessment System Regulation Number: 21 CFR 892.2080 Regulation Name: Radiological Computer Aided Triage And Notification Software Regulatory Class: Class II Product Code: QFM Dated: October 3, 2022 Received: October 4, 2022

Dear Ti-Hao Wang:

We have reviewed your Section 510(k) premarket notification of intent to market the device referenced above and have determined the device is substantially equivalent (for the indications for use stated in the enclosure) to legally marketed predicate devices marketed in interstate commerce prior to May 28, 1976, the enactment date of the Medical Device Amendments, or to devices that have been reclassified in accordance with the provisions of the Federal Food, Drug, and Cosmetic Act (Act) that do not require approval of a premarket approval application (PMA). You may, therefore, market the device, subject to the general controls provisions of the Act. Although this letter refers to your product as a device, please be aware that some cleared products may instead be combination products. The 510(k) Premarket Notification Database located at https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpmn/pmn.cfm identifies combination product submissions. The general controls provisions of the Act include requirements for annual registration, listing of devices, good manufacturing practice, labeling, and prohibitions against misbranding and adulteration. Please note: CDRH does not evaluate information related to contract liability warranties. We remind you, however, that device labeling must be truthful and not misleading.

If your device is classified (see above) into either class II (Special Controls) or class III (PMA), it may be subject to additional controls. Existing major regulations affecting your device can be found in the Code of Federal Regulations, Title 21, Parts 800 to 898. In addition, FDA may publish further announcements concerning your device in the Federal Register.

Please be advised that FDA's issuance of a substantial equivalence determination does not mean that FDA has made a determination that your device complies with other requirements of the Act or any Federal statutes and regulations administered by other Federal agencies. You must comply with all the Act's requirements, including, but not limited to: registration and listing (21 CFR Part 807); labeling (21 CFR Part 801); medical device reporting of medical device-related adverse events) (21 CFR 803) for

{1}------------------------------------------------

devices or postmarketing safety reporting (21 CFR 4, Subpart B) for combination products (see https://www.fda.gov/combination-products/guidance-regulatory-information/postmarketing-safety-reportingcombination-products); good manufacturing practice requirements as set forth in the quality systems (QS) regulation (21 CFR Part 820) for devices or current good manufacturing practices (21 CFR 4, Subpart A) for combination products; and, if applicable, the electronic product radiation control provisions (Sections 531-542 of the Act); 21 CFR 1000-1050.

Also, please note the regulation entitled, "Misbranding by reference to premarket notification" (21 CFR Part 807.97). For questions regarding the reporting of adverse events under the MDR regulation (21 CFR Part 803), please go to https://www.fda.gov/medical-device-safety/medical-device-reportingmdr-how-report-medical-device-problems.

For comprehensive regulatory information about mediation-emitting products, including information about labeling regulations, please see Device Advice (https://www.fda.gov/medicaldevices/device-advice-comprehensive-regulatory-assistance) and CDRH Learn (https://www.fda.gov/training-and-continuing-education/cdrh-learn). Additionally, you may contact the Division of Industry and Consumer Education (DICE) to ask a question about a specific regulatory topic. See the DICE website (https://www.fda.gov/medical-device-advice-comprehensive-regulatoryassistance/contact-us-division-industry-and-consumer-education-dice) for more information or contact DICE by email (DICE@fda.hhs.gov) or phone (1-800-638-2041 or 301-796-7100).

Sincerely.

Jessica Lamb

Jessica Lamb, Ph.D. Assistant Director Imaging Software Team DHT8B: Division of Radiological Imaging Devices and Electronic Products OHT8: Office of Radiological Health Office of Product Evaluation and Quality Center for Devices and Radiological Health

Enclosure

{2}------------------------------------------------

Image /page/2/Picture/0 description: The image shows the logo for Ever Fortune AI. The logo consists of a stylized human figure with a green globe on top, followed by the text "EVER" on the top line and "FORTUNE.AI" on the bottom line. The text and the human figure are in a matching teal color.

DEPARTMENT OF HEALTH AND HUMAN SERVICES Food and Drug Administration

Indications for Use

Form Approved: OMB No. 0910-0120 Expiration Date: 06/30/2023 See PRA Statement below.

510(k) Number (if known) K221552

Device Name EFAI Chestsuite XR Pneumothorax Assessment System

Indications for Use (Describe)

Type of Use (Select one or both, as applicable)

	Prescription Use (Part 21 CFR 801 Subpart D)
	Over

ver-The-Counter Use (21 CFR 801 Subpart C)

CONTINUE ON A SEPARATE PAGE IF NEEDED.

This section applies only to requirements of the Paperwork Reduction Act of 1995.

*DO NOT SEND YOUR COMPLETED FORM TO THE PRA STAFF EMAIL ADDRESS BELOW."

The burden time for this collection of information is estimated to average 79 hours per response, including the time to review instructions, search existing data sources, gather and maintain the data needed and complete and review the collection of information. Send comments regarding this burden estimate or any other aspect of this information collection, including suggestions for reducing this burden, to:

Department of Health and Human Services Food and Drug Administration Office of Chief Information Officer Paperwork Reduction Act (PRA) Staff PRAStaff@fda.hhs.gov

"An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB number."

FORM FDA 3881 (6/20)

Page 1 of 1

PSC Publishing Services (301) 443-6740 EF

{3}------------------------------------------------

510(k) Summary

1. General Information

K221552

510(k) Sponsor	Ever Fortune.AI Co., Ltd.
Address	Rm. D, 8F. No. 573, Sec. 2 Taiwan Blvd.West Dist.Taichung City 403020TAIWAN
Applicant	Joseph Chang
Contact Information	886-04-23213838 #216joseph.chang@everfortune.ai
Correspondence Person	Ti-Hao Wang, MD
Contact Information	886-04-23213838 #168tihao.wang@everfortune.ai
Date Prepared	May 31, 2022

2. Proposed Device

Proprietary Name	EFAI ChestSuite XR Pneumothorax Assessment System v1.0
Common Name	EFAI PNXXR v1.0
Classification Name	Radiological Computer-Assisted Prioritization Software For Lesions
Regulation Number	21 CFR 892.2080
Regulation Name	Radiological Computer Aided Triage and Notification Software
Product Code	QFM
Regulatory Class	II

3. Predicate Device

Proprietary Name	red dot™
Premarket Notification	K191556
Classification Name	Radiological Computer-Assisted Prioritization Software For Lesions
Regulation Number	21 CFR 892.2080
Regulation Name	Radiological Computer Aided Triage and Notification Software
Product Code	QFM
Regulatory Class	II

{4}------------------------------------------------

4. Device Description

· Local network setting of input and output destinations;

5. Intended Use / Indications for Use

{5}------------------------------------------------

6. Comparison of Technological Characteristics with Predicate Device

Table below provides a comparison of the intended use and key technological features of EFAI PNXXR with that of the Primary Predicate, red dot™ (K191556).

	Ever Fortune.AI Co., Ltd.(EFAI)	Behold.AI Technologies Limited	Supported Modalities	X-Ray (PA view)	X-Ray (PA or AP view)
Company			Body Part	Chest	Chest
Device Name	EFAI PNXXR	red dot™	Artificial IntelligenceAlgorithm	Yes	Yes
510k Number	K221552	K191556	Limited to analysis ofimaging data	Yes	Yes
Regulation No.	21CFR 892.2080	21CFR 892.2080	Aids promptidentification of caseswith indicatedfindings	Yes	Yes
Classification	II	II	Image Input	DICOM	DICOM
Product Code	QFM	QFM	Identify patientswith a pre-specifiedclinical condition	Yes	Yes
IntendedUse/Indication forUse	EFAI PNXXR is a softwareworkflow tool designed to aidthe clinical assessment of adult(22 years of age or older)Posteroanterior (PA) view ChestX-Ray cases with featuressuggestive of pneumothorax inthe medical care environment.EFAI PNXXR analyzes casesusing an artificial intelligencealgorithm to identify suspectedfindings. It makes case-leveloutput available to aPACS/workstation for worklistprioritization or triage. EFAIPNXXR is not intended to directattention to specific portions oranomalies of an image. Itsresults are not intended to beused on a stand-alone basis forclinical decision-making nor is	The red dot™ software platformis a software workflow tooldesigned to aid the clinicalassessment of adult Chest X-Raycases with features suggestive ofPneumothorax in the medicalcare environment. red dot™analyzes cases using an artificialintelligence algorithm to identifysuspected findings. It makescase-level output available to aPACS/workstation for worklistprioritization or triage. red dot™is not intended to direct attentionto specific portions of an imageor to anomalies other thanPneumothorax. Its results are notintended to be used on a stand-alone basis for clinical decision-making nor is it intended to rule out Pneumothorax or otherwise preclude clinical assessment of X-Ray cases.	Clinical condition	Pneumothorax	Pneumothorax
Intended user	Hospital networks and trainedclinicians	Hospital networks and trainedclinicians	Alert to finding	Passive notification flagged forreview	Passive notification flagged forreview
Independent ofstandard of careworkflow	Yes; No cases are removedfrom worklist	Yes; No cases are removed fromworklist
Where results arereceived	PACS / RIS / EPR/Workstation	PACS / RIS / EPR / Workstation

Table - Comparison with the Predicate Device.

{6}------------------------------------------------

The proposed device, EFAI PNXXR, is substantially equivalent to the claimed predicate, red dot™ (K191556).

{7}------------------------------------------------

7. Performance Data

Performance of the EFAI PNXXR v1.0 has been evaluated and verified in accordance with software specifications and applicable performance standards through software verification and validation testing. Additionally, the software validation activities were performed in accordance with IEC 62304:2006/A1:2016 - Medical device software – Software life cvcle processes, in addition to the FDA Guidance documents, "Guidance for the Content of Premarket Submissions for Software Contained in Medical Devices"(2005), and "Content of Premarket Submission for Management of Cybersecurity in Medical Devices."

To establish the standalone performance of EFAI PNXXR, a retrospective, blinded, multicenter study was performed to compare the pneumothorax classification performance and processing time of EFAI PNXXR against the predicate device, red dot™ (Behold.AI, K191556). The standalone dataset consists of 800 anonymized Chest X-ray images consecutively collected from 3 institutions from the US and 1 institution from the OUS. Neither of the datasets were used as part of the EFAI PNXXR model development or analytical validation testing.

The standalone dataset included pneumothorax positive (n=182) and negative cases (n=618). The cases were acquired from several X-Ray scanner manufacturers, including Samsung Electronics, Shimadzu, Toshiba, Canon Inc., Fujifilm Corporation, GE Healthcare, Konica Minolta, Philips Medical Systems, Swissray, etc. The X-Ray is taken in a standard chest X-ray protocol in PA view. Confounding cases in the dataset include possible confounders as the following: Airspace Disease, Atelectasis, Blebs, Cardiomegaly, Fracture, Infiltrate, Mass, Nodule, Obstructive Airways Disease, Pleural Effusion, Pneumonia, Scoliosis and Image Ouality Issues.

Three US board-certified radiologists determined the presence of pneumothorax in each case independently. The reference standard (ground truth) was generated by the majority agreement between the three board-certified radiologists. The performance acceptance criteria were set such that the lower bounds of 95% confidence intervals of both sensitivity and specificity should exceed 0.8. Summary of results: The dataset included 56.50% males and 43.25% females, and the mean age of cases was 50.2 years. Overall, the EFAI PNXXR was able to demonstrate sensitivity and specificity of 0.97 (95% CI=0.94-0.99) and 0.98 (95% CI=0.96-0.99) respectively, as well as an AUC of 0.99 (95% CI=0.98-1.00), which is substantially equivalent to the predicate device (Behold.ai red dot™ (K191556). The average performance time of the EFAI PNXXR was 23.3 seconds with a 95% CI of [23.2. 23.4] and was comparable with the predicate device, red dot™ (Behold.AI, K191556, 29.3 seconds). The subgroup analysis included gender, age, manufacturer, data source (US and OUS), size and location of pneumothorax, and demonstrated consistent performance for the device across all subgroups.

The table below provides a more detailed description of the performance across different manufacturers.

{8}------------------------------------------------

Manufacturer	EFAI PNXXR
	Sensitivity(95% Wilson CI)	Specificity(95% Wilson CI)
Samsung Electronics	1.00 (0.86, 1.00]	0.99 (0.97, 1.00)
Shimadzu	0.98 (0.91, 1.00)	0.99 (0.95, 1.00)
Toshiba	1.00 (0.90, 1.00]	0.99 (0.95, 1.00)
Others a, b	0.94 (0.85, 0.97)	0.81 (0.66, 0.91)

Table. Sensitivity and specificity of EFAI PNXXR by manufacturer

ª Other X-ray manufacturers include Canon Inc., Fujifilm Corporation, GE Healthcare, Konica Minolta, Philips Medical Systems, Swissray, etc.

b Manufacturers are merged into the category "Others" if the number of cases or PNX Cases or PNX Cases) were less than or equal to five in the study.

We also evaluated cases with image quality issues or radiologic findings other than pneumothorax, to see if these possible confounders systematically affect the software's performance. We found the device performs consistently and reliably under these circumstances. The results demonstrate that the EFAI PNXXR device is as safe and effective as the predicate device red dot™.

8. Conclusion

Based on the information submitted in this premarket notification, and based on the indications for use, technological characteristics, and performance testing, the EFAI PNXXR v1.0 raises no new questions of safety and effectiveness and is substantially equivalent to the predicate device in terms of safety, effectiveness, and performance.

Regulation Number and Section

§ 892.2080 Radiological computer aided triage and notification software.

(a)
Identification. Radiological computer aided triage and notification software is an image processing prescription device intended to aid in prioritization and triage of radiological medical images. The device notifies a designated list of clinicians of the availability of time sensitive radiological medical images for review based on computer aided image analysis of those images performed by the device. The device does not mark, highlight, or direct users' attention to a specific location in the original image. The device does not remove cases from a reading queue. The device operates in parallel with the standard of care, which remains the default option for all cases.(b)
Classification. Class II (special controls). The special controls for this device are:(1) Design verification and validation must include:
(i) A detailed description of the notification and triage algorithms and all underlying image analysis algorithms including, but not limited to, a detailed description of the algorithm inputs and outputs, each major component or block, how the algorithm affects or relates to clinical practice or patient care, and any algorithm limitations.
(ii) A detailed description of pre-specified performance testing protocols and dataset(s) used to assess whether the device will provide effective triage (
e.g., improved time to review of prioritized images for pre-specified clinicians).(iii) Results from performance testing that demonstrate that the device will provide effective triage. The performance assessment must be based on an appropriate measure to estimate the clinical effectiveness. The test dataset must contain sufficient numbers of cases from important cohorts (
e.g., subsets defined by clinically relevant confounders, effect modifiers, associated diseases, and subsets defined by image acquisition characteristics) such that the performance estimates and confidence intervals for these individual subsets can be characterized with the device for the intended use population and imaging equipment.(iv) Stand-alone performance testing protocols and results of the device.
(v) Appropriate software documentation (
e.g., device hazard analysis; software requirements specification document; software design specification document; traceability analysis; description of verification and validation activities including system level test protocol, pass/fail criteria, and results).(2) Labeling must include the following:
(i) A detailed description of the patient population for which the device is indicated for use;
(ii) A detailed description of the intended user and user training that addresses appropriate use protocols for the device;
(iii) Discussion of warnings, precautions, and limitations must include situations in which the device may fail or may not operate at its expected performance level (
e.g., poor image quality for certain subpopulations), as applicable;(iv) A detailed description of compatible imaging hardware, imaging protocols, and requirements for input images;
(v) Device operating instructions; and
(vi) A detailed summary of the performance testing, including: test methods, dataset characteristics, triage effectiveness (
e.g., improved time to review of prioritized images for pre-specified clinicians), diagnostic accuracy of algorithms informing triage decision, and results with associated statistical uncertainty (e.g., confidence intervals), including a summary of subanalyses on case distributions stratified by relevant confounders, such as lesion and organ characteristics, disease stages, and imaging equipment.