(92 days)
The Keeogo is robotic exoskeleton that fits orthotically on the user's waist, thigh, and shin, outside of clothing. The device is intended to help assist ambulatory function in rehabilitation settings under the supervision of a trained healthcare professional for the following population:
Individuals with stroke who have gait deficient hip (MMT Hip >= 3) and knee strength (MMT Knee >= 2) and who are capable of standing and initiating gait movement without assistance.
The trained healthcare professional must successfully complete a training program prior to fitting and tuning the device. The device is not intended for sports.
KeeogoTM Dermoskeleton System is an ambulatory assistive device that is fitted to the lower body, and is powered at the knee. This computer-controlled orthosis provides complementary force to the knee joint to assist with: (1) knee flexion and extension in the swing phase of gait, and (2) eccentric knee control and extension in the weight bearing phase.
KeeogoTM Dermoskeleton System does not move through a pre-determined pattern of movement, but rather integrates seamlessly with movements initiated by the user themselves, and provides assistance based on the detected activity.
Here's an analysis of the acceptance criteria and study detailed in the provided document, addressing each of your requested points:
The provided text describes the B-Temia Inc. Keeogo Dermoskeleton System, a robotic exoskeleton intended to assist ambulatory function in rehabilitation settings for individuals with stroke. The submission focuses on demonstrating substantial equivalence to a predicate device.
1. Table of Acceptance Criteria and Reported Device Performance:
The document doesn't explicitly state "acceptance criteria" in the traditional sense of pre-defined quantitative thresholds for clinical performance that the device must meet to be approved. Instead, it presents the results of a clinical trial designed to show improvement with the device, which implicitly served as evidence for effectiveness.
| Performance Metric | Acceptance Criteria (Implicit) | Reported Device Performance |
|---|---|---|
| Safety | No Serious Adverse Events | No Serious Adverse Events reported for participant or physical therapist/clinician. |
| Effectiveness (Gait) | Statistically significant improvement in Wisconsin Gait Scale (WGS) | Participant group showed a statistically significant improvement in WGS (p 50% of effectiveness assessments for 75% of participants. |
2. Sample Size Used for the Test Set and Data Provenance:
- Sample Size for Effectiveness Population: 48 subjects
- Sample Size for Safety Population: 55 subjects
- Data Provenance: Prospective. The study mentions "Trial Sites" which are:
- The Shirley Ryan AbilityLab (Chicago, Illinois, USA)
- Human Performance and Engineering Research (HPER) (West Orange, New Jersey, USA)
- James J Peters VA Medical Center - Center for the Medical Consequences of Spinal Cord Injury (Bronx, New York, USA)
- Assistive Technology Clinic (ATC) (Toronto, Ontario, Canada)
Therefore, the data provenance includes both USA and Canada.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications:
The document does not specify the number of experts used or their qualifications for establishing the ground truth measurements (e.g., Wisconsin Gait Scale scores, MMT scores). These are typically assessed by trained clinicians or researchers as part of standard clinical practice or research protocols. While "trained healthcare professional" is mentioned for device supervision, it doesn't detail the assessors of the outcome measures.
4. Adjudication Method for the Test Set:
The document does not specify any adjudication method for the test set. Clinical outcome assessments like the Wisconsin Gait Scale are typically performed by a single trained assessor at each site.
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study:
No, a Multi-Reader Multi-Case (MRMC) comparative effectiveness study was not done. This study design is typically used for diagnostic devices where multiple readers interpret images or data. The Keeogo is a therapeutic/assistive device, and the study was focused on its direct impact on patient gait, not on reader interpretation. Therefore, there's no mention of effect size of human readers improving with/without AI assistance.
6. Standalone Performance:
Yes, a standalone performance was done, in the sense that the clinical trial evaluated the performance of the device when used by participants (i.e., with human-in-the-loop) but the "performance" here refers to the patient's functional improvement. The device itself (the "algorithm only") operates in real-time to assist the user's movements based on detected activity, but its "performance" is inherently linked to the human using it. The study assesses the combined human-device system's effectiveness.
7. Type of Ground Truth Used:
The ground truth for effectiveness was established using clinical outcome assessments administered by trained personnel. Specifically, the primary outcome mentioned is the Wisconsin Gait Scale (WGS), which is a scalar measure of gait quality. Other assessments included "30SCT, TST-up, TST-down, ClinRO, PRO." The MMT (Manual Muscle Test) scores for hip and knee strength were used as inclusion criteria. Safety ground truth was based on the reporting of adverse events.
8. Sample Size for the Training Set:
The document does not mention a separate training set or its sample size for the device's algorithm. The "Training Program" described refers to the training for clinicians on how to use the device, not a data set used to train an AI model within the device. For powered exoskeletons, the "training" of the device typically happens through development and testing of its control algorithms rather than a distinct "training set" of patient data in the way a diagnostic AI would have. The mechanism described for the device is that it "does not move through a pre-determined pattern of movement, but rather integrates seamlessly with movements initiated by the user themselves, and provides assistance based on the detected activity." This implies a reactive control system, not a predictive AI model trained on a large dataset of patient movements.
9. How the Ground Truth for the Training Set Was Established:
As there is no explicit "training set" for the device's algorithm mentioned (it appears to be a reactive control system assisting human-initiated movements, rather than a predictive AI), the concept of "ground truth for the training set" as it applies to AI/ML models is not directly applicable here. The device's "training" would be through engineering design and validation of its control logic.
§ 890.3480 Powered lower extremity exoskeleton.
(a)
Identification. A powered lower extremity exoskeleton is a prescription device that is composed of an external, powered, motorized orthosis that is placed over a person's paralyzed or weakened limbs for medical purposes.(b)
Classification. Class II (special controls). The special controls for this device are:(1) Elements of the device materials that may contact the patient must be demonstrated to be biocompatible.
(2) Appropriate analysis/testing must validate electromagnetic compatibility/interference (EMC/EMI), electrical safety, thermal safety, mechanical safety, battery performance and safety, and wireless performance, if applicable.
(3) Appropriate software verification, validation, and hazard analysis must be performed.
(4) Design characteristics must ensure geometry and materials composition are consistent with intended use.
(5) Non-clinical performance testing must demonstrate that the device performs as intended under anticipated conditions of use. Performance testing must include:
(i) Mechanical bench testing (including durability testing) to demonstrate that the device will withstand forces, conditions, and environments encountered during use;
(ii) Simulated use testing (
i.e., cyclic loading testing) to demonstrate performance of device commands and safeguard under worst case conditions and after durability testing;(iii) Verification and validation of manual override controls are necessary, if present;
(iv) The accuracy of device features and safeguards; and
(v) Device functionality in terms of flame retardant materials, liquid/particle ingress prevention, sensor and actuator performance, and motor performance.
(6) Clinical testing must demonstrate a reasonable assurance of safe and effective use and capture any adverse events observed during clinical use when used under the proposed conditions of use, which must include considerations for:
(i) Level of supervision necessary, and
(ii) Environment of use (
e.g., indoors and/or outdoors) including obstacles and terrain representative of the intended use environment.(7) A training program must be included with sufficient educational elements so that upon completion of training program, the clinician, user, and companion can:
(i) Identify the safe environments for device use,
(ii) Use all safety features of device, and
(iii) Operate the device in simulated or actual use environments representative of indicated environments and use.
(8) Labeling for the Physician and User must include the following:
(i) Appropriate instructions, warning, cautions, limitations, and information related to the necessary safeguards of the device, including warning against activities and environments that may put the user at greater risk.
(ii) Specific instructions and the clinical training needed for the safe use of the device, which includes:
(A) Instructions on assembling the device in all available configurations;
(B) Instructions on fitting the patient;
(C) Instructions and explanations of all available programs and how to program the device;
(D) Instructions and explanation of all controls, input, and outputs;
(E) Instructions on all available modes or states of the device;
(F) Instructions on all safety features of the device; and
(G) Instructions for properly maintaining the device.
(iii) Information on the patient population for which the device has been demonstrated to have a reasonable assurance of safety and effectiveness.
(iv) Pertinent non-clinical testing information (
e.g., EMC, battery longevity).(v) A detailed summary of the clinical testing including:
(A) Adverse events encountered under use conditions,
(B) Summary of study outcomes and endpoints, and
(C) Information pertinent to use of the device including the conditions under which the device was studied (
e.g., level of supervision or assistance, and environment of use (e.g., indoors and/or outdoors) including obstacles and terrain).