(249 days)
A software system used with the Microsoft Kinect v2 intended to support the physical rehabilitation of adults in the clinic and at home. The system includes rehabilitation exercises for the lower extremity with audio-visual feedback & graphic movement representations for patients as well as remotely accessible patient performance video. Patient assessment, exercise guidance and approval by a medical professional is required prior to use.
The Vera™ software utilizes three off-the-shelf accessories: a motion sensing camera, a monitor and a computer; to provide physical therapists and physicians with a software system to prescribe customized physical therapy plans for patients. Vera™ software provides medical professionals the tools to record video and track movements in three-dimensional space in order to identify motion and count repetitions during patient participation with a prescribed exercise. Data derived from motion tracking as well as the recorded video can be reviewed by medical professionals to assess exercise movements and adherence to the prescribed exercise; Vera™ is a software tool that helps extend that expertise. The Vera™ software has three separate applications: Patient Application, Clinician Application, and Server Application.
Here's an analysis of the acceptance criteria and study detailed in the provided document for the Vera device:
1. Table of Acceptance Criteria and Reported Device Performance
| Acceptance Criteria | Reported Device Performance (Vera) |
|---|---|
| Correct Description of Exercises | 97.0% (223 out of 230 exercises) |
| Exercises recognized as typical for rehabilitation | 96.7% of therapists agreed |
| Therapists would recommend exercises to appropriate patients | 96.7% of therapists would recommend |
| Safety based on clinician review | Clinicians identified "universal risks" inherent in rehabilitation, not specific to Vera. No risks inherent to the device itself were explicitly identified, only those related to patient assessment and auxiliary equipment. |
| Accuracy of Vera in data processing | Validated by clinical study results "predominantly 100% exceeding the 95% acceptance criterion for all Vera activity movements." |
2. Sample Size Used for the Test Set and Data Provenance
- Sample Size for Test Set: 230 observations (10 Physical Therapists (PTs) x 23 Exercises/PT).
- Data Provenance: The study involved "healthy volunteers performing rehabilitation movements." The location of these volunteers or the study is not explicitly stated in the provided text. It is a prospective study as clinicians reviewed videos of performances.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts
- Number of Experts: 10 Physical Therapists (PTs).
- Qualifications of Experts: The document explicitly states "Clinicians reviewed a series of video recordings." In a later section, it mentions "10 PTs," confirming the experts were Physical Therapists. No further details regarding their years of experience or specific sub-specialties are provided.
4. Adjudication Method for the Test Set
The document does not explicitly state an adjudication method (such as 2+1 or 3+1). It describes that "Clinicians reviewed a series of video recordings... to verify that the movement seen in the video is recognized as physical therapy exercise and determine if they would recommend the movement to their patients." It appears each therapist made individual assessments, and the results were aggregated to report percentages of agreement. There is no mention of a process to resolve disagreements among the 10 therapists if they occurred.
5. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, if so, what was the effect size of how much human readers improve with AI vs without AI assistance?
No, a multi-reader multi-case (MRMC) comparative effectiveness study evaluating human readers' improvement with or without AI assistance was not performed. The study described focuses on clinician recognition and acceptance of exercises delivered by the Vera system, not on improving human reader performance.
6. If a standalone (i.e., algorithm only without human-in-the-loop performance) was done?
The study described evaluates the "clinical relevance of exercises delivered by the Vera System" and whether movements "executed by an individual interacting with the system" are recognized as exercises by clinicians. This is not a standalone algorithm performance study. While the Vera system tracks movements and provides data, the core of this particular study is about clinician perception of the exercises facilitated by the system, rather than the raw algorithmic accuracy in isolation. The document states a "hazard analysis of the Vera™ software also indicates low risk to the end user" and that "Vera™ has been validated for accuracy / performance effectiveness both clinically and through software testing with appropriate change management and design controls," which may allude to some standalone testing, but the details of such are not provided for the clinical study section.
7. The type of ground truth used (expert consensus, pathology, outcomes data, etc.)
The ground truth for this specific study was expert consensus / expert opinion from the 10 Physical Therapists. They evaluated whether the movements performed by healthy volunteers via the Vera system were indeed recognized as physical rehabilitation exercises and if they would recommend them.
8. The sample size for the training set
The document does not provide any information regarding a training set size. This study appears to be a clinical validation of the exercises, not an evaluation of a machine learning model's training data. The Vera device itself is a "software system utilizing optical recording for rehabilitation exercises" and includes "motion sensing camera" technology (Kinect v2). While such a system internally would have trained algorithms for skeletal tracking and movement analysis, the details of that training are not part of the provided clinical performance data summary.
9. How the ground truth for the training set was established
As no training set is discussed in the context of this document's clinical performance section, the method for establishing its ground truth is not provided.
§ 890.5360 Measuring exercise equipment.
(a)
Identification. Measuring exercise equipment consist of manual devices intended for medical purposes, such as to redevelop muscles or restore motion to joints or for use as an adjunct treatment for obesity. These devices also include instrumentation, such as the pulse rate monitor, that provide information used for physical evaluation and physical planning purposes., Examples include a therapeutic exercise bicycle with measuring instrumentation, a manually propelled treadmill with measuring instrumentation, and a rowing machine with measuring instrumentation.(b)
Classification. Class II (special controls). The device, when it is a measuring exerciser or an interactive rehabilitation exercise device for prescription use only, is exempt from the premarket notification procedures in subpart E of part 807 of this chapter subject to the limitations in § 890.9.