Abstracts

ORIGINAL ARTICLE

^IPost Graduation Program in Human Movement Sciences, Universidade do Estado de Santa Catarina (UDESC), Florianópolis, SC, Brazil

^IIAssociation of Assistance to Disabled Children (AACD), São Paulo, SP, Brazil

^IIIPost Graduation Program in Rehabilitation Science, Universidade Federal de Minas Gerais (UFMG), Belo Horizonte, MG, Brazil

Correspondence to

ABSTRACT

BACKGROUND: The Wolf Motor Function Test (WMFT) evaluates the upper limb (UL) performance of adults with hemiparesis by combining time and quality of movement measures in both isolated movements and functional tasks.

OBJECTIVES: To translate and adapt the WMFT form, functional ability scale (FAS) and manual to Brazilian Portuguese and evaluate the intra and inter-rater reliabilities.

METHODS: Fifteen individuals with a mean age of 57.9±11.1 years and a mean time since stroke onset of 68.5±53.5 months participated. The WMFT was administered by one physiotherapist based on information in the manual, and video observations were assessed by two other independent physical therapists. Information regarding compensatory movements was included in the FAS. Intra-class correlation coefficients (ICCs) and Bland-Altman plots were calculated to examine the intra- and inter-rater reliabilities for performance time and FAS, whereas weighted kappa (Kp) was used to examine the agreement strength for FAS.

RESULTS: The inter-rater ICC values for performance time were above 0.75 in 13 of the 15 tasks. For the FAS, they ranged from 0.87-0.99 for all evaluated tasks, with Kp values ranging from 0.63-0.92. For intra-rater reliability, the ICC ranged from 0.99-1.0 and from 0.96-1.0 for time measurement and FAS, respectively. Kp values ranged from 0.79-0.96 for individual and 0.93 for total scores.

CONCLUSION: The Brazilian version of the WMFT showed adequate intra- and inter-rater reliabilities for evaluating the paretic UL of individuals with stroke.

Keywords: translating; stroke; hemiplegia; upper extremity; Wolf Motor Function Test; reproducibility of results.

Introduction

It is estimated that 45 to 75% of adults who have suffered a stroke have difficulty using the hemiparetic upper limb (UL) in activities of daily living (ADLs) in the chronic stage¹. Reliable systematic reviews provide adequate information for clinical decision making about UL rehabilitation and demonstrate the effects of an intervention designed to improve the functionality, coordination and precision of UL movements^2,3.

The Wolf Motor Function Test (WMFT) was initially developed to evaluate the effects of constraint-induced movement therapy in individuals with hemiparesis^1-3. The original version consisted of 21 tasks organized according to the joints involved (from the shoulder to the fingers) and the difficulty level (from gross to fine motor activity), evaluating the UL function by means of one or multiple joint movements and functional tasks. As shown in ^{Appendix 1} Appendix 1 , this test was subsequently modified to a version with 17 sequential tasks⁴ in order to simplify its application. The WMFT evaluates the speed of task performance, quantifies the quality of movement by means of a functional ability scale (FAS) and measures handgrip strength and shoulder flexion strength in two specific tasks². The final result provides the average task completion time, the median FAS score, handgrip strength in kgf and the weight lifted in grams in the shoulder flexion movement. Because they involve different units, the strength tasks are not included in the final performance time or the FAS³. Handgrip strength measurement by dynamometer has already been tested for reliability⁵, and thus the two strength measures were not included in this reliability study.

The WMFT tasks must be videotaped from a standardized position and distance, and scoring is carried out by analyzing the videos. The intraclass correlation coefficient (ICC) of the inter-rater reliability of the total FAS score in the original version of the WMFT was 0.88. The measurement of task performance speed showed high reliability both in direct measurement⁶ (ICC=0.97) and in video scoring (ICC=0.96)⁵. For the results to be reliable, reading the manual and using the specified material are recommended⁶.

The FAS has six levels, with "zero" indicating no attempt to move the UL that is being tested and "five" indicating that the movement seems to be normal. In the original version, subjects are scored according to the presence of abnormal movements such as lack of coordination between the segments, lack of fine coordination, fluidity or precision, slow task performance and difficulty in performing resistance activities. Although other tests evaluating the paretic UL in patients with hemiparesis have already been translated and adapted to Portuguese, the WMFT is the only one that combines measurements of time and quality of movement performance in both isolated movements of specific joints and complex functional tasks; thus, it is an evaluation that can be applied to patients with different levels of impairment. Among widely used tests that have been validated in Brazil, the Test d'Évaluation des Membres Supérieurs de Personnes Âgées (TEMPA)⁷ evaluates only the function component, whereas the Fugl-Meyer scale⁸ evaluates only movement.

The TEMPA⁷ evaluates UL focal function according to time, functional score and observational analysis of task performance. It includes four unilateral tasks and four bilateral tasks that represent ADLs and is recommended when the aim is to evaluate activity limitations. The Fugl-Meyer scale⁸ evaluates the recovery of the post-stroke paretic UL by analyzing movement components according to the presence/absence of abnormal synergies. Fugl-Meyer is recommended for the evaluation of impairments in body function or structure, without considering functionality. However, the WMFT can offer the advantage of evaluating both the function and body structure components⁹. Nevertheless, Massie et al.¹⁰ reported that in the outcome measures used in constraint-induced movement therapy (CIMT), including the WMFT, aspects of movement quality related to compensatory movements and incoordination between the segments, which are characteristic of this population, are not considered¹¹.

Thus, the aims of the present study were to translate and adapt the WMFT form and application manual to Brazilian Portuguese, adding information to the FAS regarding compensatory movements, and to evaluate the test's intra- and inter-rater reliability using video.

Methods

Translation and adaptation of the WMFT

The WMFT was created in 1989² in American English and the application manual was developed in 2000⁴. In the present study, the form, manual and FAS were independently translated into Portuguese by two Brazilian physical therapists and a Brazilian occupational therapist. These professionals, fluent in English, were trained by the group of authors of the latest version of the test at the University of Alabama at Birmingham (UAB), USA. The translated manual is available on request from the authors.

The manual was translated with the consent of the original author. Information was added to the FAS (Table 1A) regarding the original scale (Table 1B) including details about compensatory movement, incoordination between the segments and further explanation added about the movement quality items, which are based on the Quality of Movement Scale used in CIMT¹².

This version was back-translated by a physical therapist fluent in both languages †and then sent to the UAB group for approval. The group considered that the original concepts were preserved in the adapted version. A group of six physical therapists experienced in neurological rehabilitation considered the adapted version sufficiently clear¹³.

Before scoring the videos, the untrained examiner read the manual and discussed problematic issues with the trained examiner. Questions were also discussed with researchers from UAB, who sent updates of the manual to facilitate scoring. Clarifications included: if in doubt between two levels, the lower score should be adopted; in case of a discrepancy between the quality of shoulder movement and hand or elbow movement, the task category must be taken into consideration. For example, pinching is the tested movement in the paper clip lifting task, so hand performance is of primary importance, even if the quality of shoulder movement is lower. Thus, information of this type was added to the manual.

Application of the WMFT

The WMFT was applied by a physical therapist who had no previous experience with the test and relied completely on the information contained in the manual. While the test was being applied, the 15 tasks were videotaped according to the guidelines in the manual. The videos were subsequently evaluated by two other physical therapists. The two strength tasks were not performed (7 and 14), as was the procedure in the original reliability studies^1,3. The physical therapist who applied the test did not participate in the scoring. The examiner directed each task according to the instructions and commands described in the manual. Each task was demonstrated both slowly and quickly to the subject, who was then requested to do it as quickly as possible.

To evaluate performance speed, the video was used to time the tasks from the moment of the verbal command "Go!" until the moment it was completed. The six-level FAS was used to evaluate the quality of movement in each task (Table 1A). The total score for performance time was calculated by the mean performance time of all tasks. When the individual was unable to accomplish a certain task, a score of 121 seconds was given, since 120 seconds is the maximum time allowed for each task¹⁴. In this study, the mean and median of the score were used, since using only the mean can lead to arbitrary representation of the values when incomplete tasks are involved. Functional performance was evaluated by the median of the scores of all tasks¹⁵.

The WMFT was applied using a model that establishes the standard position of each object and its initial and/or final position in the tasks. All tasks were videotaped and FAS was evaluated according to the instructions in the manual.

Participants

The study included a convenience sample of 15 adults (57.9±11.1 years) with chronic hemiparesis (68.5±53.5 months after stroke) who were recruited from the Clinical School of Physical Therapy of the Universidade do Estado de Santa Catarina (UDESC), Florianópolis, SC, Brazil. After signing the consent form approved by the UDESC Research Ethics Committee (Nº 144/2009), the Fugl-Meyer motor recovery assessment⁸ was applied and all individuals who met the following inclusion criteria were included: age >18 years, being at least six months post-stroke and having no cognitive impairment according to the Mini Mental State Examination (MMSE) with the cutoff points recommended by Brucki et al.¹⁶. Individuals with other neurological diseases, those who presented full motor recovery and those with a totally paralyzed limb were excluded.

Intra and inter-rater reliability

Based on the videos, test performance was evaluated by two independent examiners who, without conferring with each other, recorded the task performance time and scored the FAS. To determine intra-rater reliability, the same examiners reevaluated the videos after one month.

Statistical analysis

ICCs with confidence intervals (CI) of 95% were calculated to evaluate intra- and inter-rater reliability for time measurement and FAS¹⁷. Absolute agreement ICC was also used, which determines the correlation of evaluations in terms of absolute scores¹⁸.

Weighted kappa (Kp) was used to evaluate intra- and inter-rater reliability of the FAS since it is recommended for evaluating ordinal scales¹⁹. However, the ICC was also used for comparison with the literature. For a more detailed analysis of scoring differences by both examiners, a Bland-Altman plot was used to compare the total time and FAS scores²⁰. The level of significance was set at 5%.

Results

Sample characterization

The sample consisted of individuals with mild UL (MS>50) impairment (46%), moderate impairment (30<MS<49) (27%) and severe impairment (MS<30) (27%)⁸ (Table 2).

Inter-rater reliability

Table 3 describes the mean and median of time, median FAS, inter-rater ICC, and the FAS Kp value for each task. For time measurement, the inter-rater ICC ranged from 0.87-1 in 13 of 15 tasks²¹, and the tasks "forearm to table" and "hand to box" showed lower than adequate ICC values. When all tasks were considered (total score), the ICC values were 0.94 (0.82-0.98) for the FAS and 0.99 (0.98-0.99) for time; the inter-rater absolute agreement was 0.94 (0.83-0.98) for the FAS and 0.99 (0.98-0.99) for total time score.

The Bland-Altman plot demonstrated inter-rater reliability. For the FAS, the average difference between the two evaluations did not differ significantly from zero and the limits of agreement represented 8.2 and 9.6% of the scale range. Similarly, the inter-rater difference for time was very close to zero.

Intra-rater reliability

The ICC values for time measurement, with the exception of the task "forearm to box", ranged from 0.99-1 and for the FAS from 0.96-1. For the FAS, the Kp value ranged from 0.79 ("lift the basket") to 0.96 ("reach and retrieve"), with Kp=0.93 (CI=0.78 to 1.0) for the total score.

The Bland-Altman plot showed that the average difference between the two evaluations did not differ significantly from zero, the limits of agreement represented 3.4% of the FAS range, and the time difference was very close to zero (Figure 1).

Discussion

This study translated and adapted the WMFT to Brazilian Portuguese and determined its intra- and inter-rater reliability for individuals with hemiparesis whose impairment ranged from mild to severe. In general, the results were similar to previous reliability studies^4,15.

The reliability of inter-rater time measurement was assessed to determine if the two examiners followed the same criteria for gauging the beginning and end of the task as well as failure to complete the task. The inter-rater reliability of time measurement in the translated version of the WMFT was adequate, with an ICC similar to that found by Morris et al.⁴ in the original version.

When individually analyzed, 13 of the 15 tasks had high levels of inter-rater reliability for time measurement. The tasks "forearm to table" and "hand to box" showed unsatisfactory and moderate levels (ICC<0.75), respectively²¹. This was due to the fact that one of the examiners considered the task "forearm to table" incomplete for two subjects and the task "hand to box" incomplete for another subject, which resulted in scores of 121 seconds, while the other examiner considered the task completed and recorded a different total time. However, one of the videos was stopped immediately after task completion, which made it difficult to judge the final position and whether the task was completed or not. This was considered a limitation of the study. Such a discrepancy could have been minimized by continuing to record for a few seconds after task completion. This information was added to the manual in the instructions about videotaping.

The median can characterize the performance of time measurement with greater reliability since it is less influenced by outlier values from unfinished tasks (121 seconds)²². However, for evaluating the effect of an intervention, the mean may be more sensitive both in time and in FAS; for example, if the subject is able to perform one of the tasks that he did not perform before an intervention, this may be best represented by the mean¹⁵.

The absolute agreement between evaluators in the translated and adapted version of the FAS was adequate (ICC=0.94) and slightly higher than values found in the original version⁴ (ICC=0.88). These results suggest that the examiners not only scored functional ability in a similar manner but also tended to give to the same individuals the same absolute scores. Regarding individual tasks in the FAS, all showed adequate inter-rater reliability (ICC>0.87) with good Kp values (Kp>0.61)¹⁷.

The intra-rater ICC value for total score of time measurement and FAS was 100%, despite slight differences observed in the mean, median and ICC values of the individual tasks. The absolute agreement for the FAS was more sensitive to these small changes (Kp=0.93). To evaluate the FAS, Kp is recommended since it evaluates ordinal scales and is more rigid regarding a scoring difference of two or more levels between two examiners, i.e., if one examiner gives a score of "2" and the other gives a "4". All other tasks would show an agreement level between "good" and "very good" in Kp, despite showing lower values than the ICC¹⁷. Although Kp is the most suitable method for ordinal scales such as the FAS, the ICC was also used for comparison with the literature.

Since this study's results depended on the evaluation of a video by the same examiner one month apart, this may have resulted in a higher ICC than that of the original version (0.90 for time and 0.95 for FAS), in which the same examiner administered two tests two weeks apart⁴. The reason for evaluating the same video was to eliminate any variability due to the subject in two test applications so as to better consider the examiner's ability to judge the beginning and end of the same video after a time interval between the two observations. These high reliability coefficients are probably due to the detailed scoring description provided in the manual, the degree to which the test was standardized and the fact that the Brazilian version of the FAS is more detailed. Sanford et al.²³ highlighted the importance of standardizing the manuals of assessment instruments in order to reduce measurement errors.

The WMFT includes a variety of movements that can be useful in both the clinical evaluation of patients with hemiparesis and as a research tool^24,25. It is a scale that evaluates motor deficits by means of quantitative variables, performance time, fine coordination and fluidity, among other clinically relevant characteristics². For example, compensatory movements of the trunk have been reported to limit UL motor recovery after stroke²⁶. The modified version of FAS explicitly demonstrates how to distinguish these movements and, thus, may assist in identifying arm movement recovery as opposed to the use of compensatory strategies which would thus help the clinician decide on a therapeutic approach. Like the TEMPA, the WMFT includes a qualitative evaluation of movement, an advantage not shared by tests such as Jebsen Hand Function Test²⁴, which scores only the speed of task performance. Although more difficult to objectively quantify, these characteristics are important for evaluating movement. For the quality score to be reliable, it is suggested that the videos should be evaluated by examiners who understand the main compensation strategies of patients with different levels of impairment.

In the Brazilian version of the FAS, further details on compensatory movement, abnormal synergies, proximal control and primitive grip patterns were added. This information was based on the Quality of Movement Scale used in the CIMT, which was created by the same group responsible for the WMFT. In fact, the WMFT was designed to evaluate the results of this therapeutic intervention^4,12,25. The authors of the WMFT agreed to allow the original concept of the scale to remain in the Brazilian Portuguese version. This modification aimed to take into account changes in movement that are commonly observed in hemiparetic subjects and that were not clearly covered in the scale. Another reason for the modification was to allow better classification of patients in each of the six levels.

After the time measurement and FAS evaluation, the examiners discussed the points that were difficult to interpret in the manual, which were then rewritten. For example, the initial position in the eighth task was redescribed with greater clarity ("the forearm in neutral position"). In addition to the instructions sent by UAB, a suggestion made by Whitall et al.¹⁵ was added about instructing the patient not to use the less affected arm except in the bimanual task of folding the towel. Other modifications included some camera angles considered difficult to visualize for the functional evaluation, which were changed to a more favorable position (frontal) in the following tasks: lifting a can, lifting a pencil, lifting a paper clip, stacking checkers, flipping cards and folding a towel. It is essential that the videotaping meet the specifications of the manual regarding the position and distance of the camera in order to properly view the compensatory movements. Videotaping the entire body on the paretic side allows visualization of compensation due to anterior trunk displacement^3,26, while videotaping frontally and at close range facilitates the observation of compensatory movements made while gripping objects¹¹.

All subjects were able to perform at least the initial tasks of the test, even the most affected. A patient with a score of eight on the Fugl-Meyer scale was able to perform eight of the 15 tasks. The organization of the test regarding task complexity, i.e., from the easiest to the most complicated, motivated patients with greater difficulties. Other advantages of this test are that it also includes a bimanual task and uses common materials, unlike the Action Research Arm Test^6,27,28, which uses cylinders and wood blocks, materials with low ecological validity because they are not encountered in daily living^29,30. Although specific equipment is required for the administration of the test, most of the items are common, inexpensive and easily obtained. This fact, combined with adequate reliability and internal consistency⁶, makes the WMFT an excellent test for research purposes. One obstacle in clinical practice may be the time it takes to apply the test, which is about 30 minutes. However, in situations where it is important to document changes during treatment, the WMFT can provide valuable information.

The results indicate that the Brazilian version of the WMFT has appropriate intra- and inter-rater reliability for the total test score and that it can be used to evaluate functionality of the paretic UL of adults with hemiparesis. The video observation method, carried out according to the manual's instructions, proved to be a reliable means for evaluating the time and quality of movement, making it easier to apply the WMFT in clinical and research environments.

References

Appendix 1

Publication Dates

History