ARTICLE

A Psychometric Study of the Chinese Version of the Assessment of Communication and Interaction Skills Wei-Ling Hsu, MSc, OTC Ay-Woan Pan, PhD, OTR, OTC Tsyr-Jang Chen, PhD

Wei-Ling Hsu, MS, OTC, is with the Hsu-An Rehabilitation Clinic, Taipei, Taiwan, and is a Research Assistant, School of Occupational Therapy, College of Medicine, National Taiwan University, Room 407, No. 17, Xu-Zhou Rd., Taipei 100, Taiwan. Ay-Woan Pan, PhD, OTR, OTC, is with the Department of Psychiatry, National Taiwan University Hospital and is an Associate Professor, School of Occupational Therapy, College of Medicine, National Taiwan University, Room 407, No. 17, XuZhou Rd., Taipei 100, Taiwan. Tsyr-Jang Chen, PhD, is an Associate Professor, Department of Mechanical Engineering, LungHwa University of Science and Technology. Address correspondence to: Wei-Li Hsu, National Taiwan University, Room 407, No. 17, Xu-Zhou Rd., Taipei 100, Taiwan (E-mail: [email protected]). The study was supported and funded by the National Science Council of Taiwan (NSC-95-2516-S-002-007). We would like to thank Dr. Gary Kielhofner of his kindness to offer us the copyright of the Chinese ACIS and his excellent input on this manuscript. Occupational Therapy in Health Care, Vol. 22(2–3), 2008 Available online at http://www.haworthpress.com/web/OTHC
C 2008 by Informa Healthcare USA, Inc. All rights reserved. doi: 10.1080/07380570801991818

177

178

OCCUPATIONAL THERAPY IN HEALTH CARE

ABSTRACT. This study examined the dependability of the Assessment of Communication and Interaction Skills—Chinese version (ACIS-C) with psychiatric participants in Taiwan. A convenience sample of 101 participants diagnosed with psychiatric illness were recruited from four day-care wards in northern and eastern Taiwan. The results of the Rasch analysis showed that the ACIS-C items coalesced to form a measure of communication/interaction and the 4-point rating scale functioned as intended. The ACIS-C differentiated participants into six levels of communication and interaction skills. The findings support the conclusion that the ACIS-C is a valid and sensitive tool when used with Chinese clients. Moreover the study supports the generalizability of the Model of Human Occupation concept of communication and interaction skills to an Eastern context. KEYWORDS. Social skills assessment, model of human occupation, Validity, Rasch analysis

INTRODUCTION The Assessment of Communication and Interaction Skills (ACIS) is a systematic observational tool (Forsyth, Kielhofner, & Lai, 1999) used to assess client’s capacities to express themselves and to interact with others. Simon found modest interrater reliability with the first version of the ACIS (Simon, 1989). Salamy revised the ACIS but found that the discriminative validity of the ACIS was not adequate (Salamy, 1993). Forsyth, Kielhofner, and Lai (1999) further revised the assessment and examined its validity. The study demonstrated that the instrument worked well to capture communication and interaction skills and showed better discrimination among clients. Forsyth, Haglund, Kjellberg, and Kielhofner (2003) translated the ACIS into Swedish and their study found the reliability, validity, and sensitivity of the ACIS were all acceptable. This study also suggested that the ACIS could be used in a different language and cultural context. This was an important finding because communication and interaction are very influenced by language and culture. Nonetheless, the ACIS has only been studied with Westerners. The purpose of the study was to examine whether the instrument would also work in an Eastern language and cultural context.

Hsu et al.

179

METHODS Aim and Questions This psychometric study aimed to examine the suitability of the Chinese version of the ACIS (ACIS-C) when used with Taiwanese persons who have psychiatric disorders. Specifically, the study asked:

r Do the items of the ACIS coalesce to measure communication and interaction skills?

r Does the 4-point rating scale operate as intended? r Are the items of the ACIS well matched to the subjects of this study? r Is the ACIS able to effectively discriminate between subjects with different levels of communication and interaction skills? Participants One hundred and one participants were recruited from four day-care wards in northern and eastern Taiwan. All participants met DSM-IV diagnostic criteria for schizophrenia and/or affective disorder. Their demographic characteristics are shown in Table 1. Procedure Participants who received a score greater than 24 on the Mini Mental Status Exam (Folstein, Folstein, & McHugh, 1975) were eligible for the study. After informed consent was obtained, each participant was observed in each of the following contexts: (a) a group discussion of ward-relevant issues guided by a leader and (b) a one-to-one interview. The rater (first author) is an occupational therapist who has worked in psychiatric practice for more than 2 years. The rater learned MOHO during undergraduate and graduate studies. She has translated the ACIS into Chinese and has a thorough understanding of the scale. Instrument The ACIS is a 20-item observational assessment (Forsyth, Salamy, Simon, & Kielhofner, 1998), which can be used to observe social skills

180

OCCUPATIONAL THERAPY IN HEALTH CARE TABLE 1. Demographics of the Participants

Diagnosis Schizophrenia Unipolar depression Bipolar disorder Gender Female Male Educational level High school University and above Junior high school Elementary school Marital status Unmarried Married Divorced Employment status Unemployed Employed Age

N = 101

%

78 12 11

77.2 11.9 10.9

56 45

55.4 44.6

44 29 21 7

43.6 28.7 20.8 6.9

71 21 9

70.3 20.8 8.9

98 97.0 3 3.0 Mean = 38.6 years (SD = 11. 5 years)

of persons who are more than 18 years old. The ACIS recognizes the influence of context and culture on social skill performance. Thus, the therapist needs to record the context he (she) observes the client in and observe the performance of the clients across different contexts (Forsyth, Salamy, Simon, & Kielhofner, 1998). The administration of the ACIS requires about 20 to 60 minutes. The ACIS uses a 4-point rating scale, where 1 represents skills deficit and 4 represents good performance of that skill. High scores indicate better social skills (Forsyth, Salamy, Simon, & Kielhofner). A two-stage translation procedure was utilized (Hachey, Jumoorty, & Mercier, 1995).The original ACIS was translated into Chinese. Then it was back-translated into English and compared to the original ACIS. Four items were found to be inconsistent. Thus, the Chinese translation of these items was modified and the items were, once again, back-translated. When the revised items were compared with the original ACIS, they were acceptably similar.

Hsu et al.

181

Data Analysis The Rasch measurement model (Fisher, 1993; Grip, Merbitz, & Morris, 1989; Stone & Wright, 1979) was used to answer the study questions. The Rasch computer program WINSTEPS (Linacre, 2003) was used for the analysis. The acceptable ranges of the infit Mnsq statistics were used to determine whether items coalesced to form a single measure. For this study we considered Mnsq values between 0.6 and 1.4, associated with ZSTD values between +2 and −2, to indicate items had adequate fit. If the Infit Mnsq value is too high, there is unexpected high variability in subjects’ responses to that item. The rating scale was evaluated by examining ordered step measures. The person-item map was examined to determine if items were appropriately targeted to the participants in the study. Finally, the separation index was used to examine the ability of

TABLE 2. ACIS-C Item Fit Statistics and Measure Order Infit Item

MNSQ

ZSTD

Measurea

Error

9. 5. 7. 17. 4. 13. 12. 18. 6. 10. 2. 14. 1. 16. 15. 3. 11. 8. 20. 19.

.82 1.20 1.70b .96 .97 1.00 .97 1.30 .80 1.03 1.01 .85 1.03 .86 .86 .95 1.05 .73 .85 .83

−2.0 2.0 6.3b −.4 −.3 .0 −.3 3.0 −2.4 .3 .1 −1.8 .3 −1.6 −1.6 −.5 .5 −3.2 −1.6 −1.9

−2.12 −1.82 −1.23 −1.21 −.69 −.59 −.57 −.45 −.23 −.17 .14 .43 .45 .72 .75 .87 1.08 1.33 1.56 1.75

.15 .15 .14 .14 .14 .14 .14 .14 .14 .14 .14 .14 .14 .14 .14 .14 .14 .14 .15 .14

Asks Orients Articulates Conforms Maneuvers Shares Modulates Focuses Postures Engages Gazes Speaks Contacts Collaborates Sustains Gestures Expresses Asserts Respects Relates

a Larger, b Mnsq

positive values indicate that the item is more challenging. > 1.4 associated with Zstd > ±2 indicates a misfitting item.

182

OCCUPATIONAL THERAPY IN HEALTH CARE

the instrument to detect different levels of communication and interaction skills. RESULTS As indicated in Table 2, all ACIS-C items met the criteria for fit except for the item, “Articulates.” Table 2 also shows item calibrations and the error associated with them. The larger the calibration, the more challenging the item. Thus, the easiest item is “Asks” and the hardest item is “Relates.” Figure 1 shows that there is no displacement of the 4-point rating scale structure of the ACIS-C. Each rating was used in order as intended; the step from ratings 1 to 2 was −3.89, from 2 to 3 was −.04, and from 3 to 4 was 3.92, prospectively. FIGURE 1. Steps of the 4-point rating scale.

Hsu et al. FIGURE 2. Map of item difficulty and person ability.

183

184

OCCUPATIONAL THERAPY IN HEALTH CARE

Figure 2 is the map of persons and items. The far left column shows the distribution of participants according to their level of communication and interaction skills. The second, third, and fourth columns show the distribution of items according to each of the four ratings on the ACIS. The map demonstrates that items are separated adequately and appropriately targeted to the participants in this study. The person separation index was 4.64 indicating that the ACIS-C can separate the subjects into six significantly different levels of communication and interaction levels (Strata = (4(4.64) + 1)/3= 6.52). DISCUSSION Nineteen out of 20 items of the ACIS-C items worked well to constitute a measure of communication and interaction skills. This finding supports the conclusion that the ACIS-C is a valid measure of communication/interaction in a Chinese population. The item “Articulates” may have misfit because of medication side effects (Hope, Mueser, Penn, Reed, & Spaulding, 1995; e.g., slurred speech) for some subjects. Because “Articulates” is an important dimension of communication and has not been shown to misfit in other studies of the ACIS it should be kept in the scale. Further research should determine whether the item misfits in other Taiwanese populations. The ACIS-C also showed good measurement qualities. The 4-point rating scale worked appropriately. The items were well targeted to the participants and the instrument was sensitive in detecting different levels of communication and interaction skills in the participants. In summary, the results of the study support the conclusion that the ACIS-C is psychometrically sound. Future research should examine interrater and intrarater reliability as well as concurrent and predictive validity. The findings of this study also support the conclusion that the concept of communication and interaction skills in the Model of Human Occupation (Kielhofner, 2008) is relevant to an Eastern population. Importantly, this study provides evidence of the validity of MOHO in a culture distinct from its Western origins. REFERENCES Fisher, A. G. (1993). The assessment of IADL motor skills: an application of manyfaceted Rasch analysis. American Journal of Occupational Therapy, 47(4), 319– 329.

Hsu et al.

185

Folstein, M. F., Folstein, S. E., & McHugh, P. R. (1975). Mini-Mental State: a practical method for grading the cognitive state of patients for the clinician. Journal of Psychiatric Research, 12, 189–198. Forsyth, K., Haglund, L., Kjellberg, A., & Kielhofner, G. (2003). The measurement properties of the Swedish version of the Assessment of Communication and Interaction Skills. Scandinavian Journal of Caring Sciences, 17(3), 271–277. Forsyth, K., Kielhofner, G., & Lai, J. S. (1999). The assessment of communication and interaction skills (ACIS): measurement properties. British Journal of Occupational Therapy, 62(2), 69–74. Forsyth, K., Salamy, M., Simon, M., & Kielhofner, G. (1998). The Assessment of Communication and Interaction Skills (version 4.0). Chicago: Model of Human Occupation Clearinghouse, Department of Occupational Therapy, College of Applied Health Sciences, University of Illinois at Chicago. Grip, J. C., Merbitz, C., & Morris, J. (1989). Ordinal scale and foundations of misinference. Archives of Physical Medicine and Rehabilitation, 70(4), 308–312. Hachey, R., Jumoorty, J., & Mercier, C. (1995). Methodology for validating the translation of test measurements applied to occupational therapy. Occupational Therapy International, 2, 190–203. Hope, D. A., Mueser, K. T., Penn, D. L., Reed, D., & Spaulding, W. (1995). Information processing and social competence in chronic schizophrenia. Schizophrenia Bulletin, 21(2), 269-281. Kielhofner, G. (2008). A Model of Human Occupation: Theory and Application (4th ed.). Baltimore, MD: Lippincott, Williams, & Wilkins. Linacre, J. M. (2003). A User’s Guide to WINSTEP, MINISTEP: Rasch Model Computer Programs. Chicago: MESA. Salamy, M. (1993). Construct validity of the assessment for communication and interaction skills. Unpublished master’s thesis, University of Illinois at Chicago, Chicago, IL. Simon, S. (1989). The development of an assessment for communication and interaction skills. Unpublished master’s thesis, University of Illinois at Chicago, Chicago, IL. Stone, M. H., & Wright, B. D. (1979). Best test design. Chicago, MESA.

Received: 08/09/2007 Revised: 10/26/2007 Accepted: 10/26/2007