Title: Virtual Pandemic Flu Triage Exercise in the Virtual Learning Environment Play2Train: Measurable Improvement in Team Interaction and Preparedness Competencies. Authors: Rameshsharma Ramloll, PhD. Research Assistant Professor at the Institute of Rural Health Idaho State University; Jaishree Beedasy, PhD. Research Assistant Professor at the Institute of Rural Idaho State University The study was supported by a Grant from the U.S. Department of Health and Human Services (DHHS), Office of the Assistant Secretary for Preparedness and Response (ASPR). The contents are the sole responsibility of the authors and do not necessarily represent the official views of the DHHS. Abstract Practicing drills and exercises to improve individual and team performance in real physical hospital settings often requires lock-down of the healthcare center, and effects on real patients are unpredictable. A virtual triage exercise was developed around a pandemic influenza scenario and was carried out in a 3 D interactive virtual environment. Without putting real patients or participants at risk, the environment allowed geographically dispersed first responders and hospital staff to practice their roles in a common setting. Results showed that the exercise produced perceived improvements in preparedness competencies and team interactions.

Introduction The paper describes one approach for developing, implementing, and assessing a virtual triage operation designed to enhance emergency preparedness of first responders and hospital staff in response to a pandemic flu event. Participants through their avatars performed relevant competency related activities in this immersive setting to enhance their skills and knowledge. This exercise was part of a training program for preparing healthcare providers and first responders for disasters and emergencies in the state of Idaho. Background Recent devastating manmade and natural disasters, and the looming threat of a flu pandemics heighten the importance of having a well-prepared workforce to recognize and respond to public health threats in a unified and coordinated manner. Doing a full scale exercise is not always an 1

option. Very few hospitals can afford to orchestrate the chaos and downtime that would result from a pandemic illness or other disasters through full scale live exercises. Such exercises are high stress multi-agency activities involving actual deployment of resources in a coordinated response that is not possible under many circumstances. Live exercises are too costly and most hospitals cannot stage them as often as needed to meet the training requirements for emergency preparedness. During live full scale exercises, the quality of care takes a hit because hospital staff is distracted and operates with strained resources under altered work flow conditions. Practice in real physical settings requires lock-down of a hospital/healthcare center, and the effects on real patients are often unpredictable. Traditionally, tabletop exercises have been favored in the preparedness planning for such disasters. These are mostly limited to face to face discussions centered on a hypothetical scenario, facilitated by emergency preparedness experts. Online 3D Multiuser Virtual learning environments (MUVLEs) provide simulations designed to depict actual or assumed real-life situations. These environments can be used to explore the processes and consequences of decision-making; conduct “what-if” analyses of existing plans; and develop new plans and carry out certain preparedness competency training activities. These virtual exercises do not involve the use of actual resources, virtual interactive objects are used instead. Virtual worlds allow simultaneous participation from different agencies in different geographic locations to practice for uncommon emergency situations in a safe, reproducible and flexible setting. A virtual exercise is not a substitute for a full scale live exercise but it can be done repetitively while keeping costs and disruptions in daily workflows under control. Exercises in virtual environments have rarely been developed from the get-go to replace live exercises. The main functions of a virtual exercise are to increase opportunities for collaborative learning, for diagnosing live exercise plans, and for allowing geographically dispersed stakeholders to participate in a common emergency preparedness effort.

Distributed, contextualized learning and rehearsing in Virtual Environments Virtual environments can support an approach to learning which emphasizes student centeredness and enables learning to be related to context and to practice. The student in collaborative learning theory is seen as an active participant in the learning process who constructs knowledge through interaction with peers and instructors, which has the potential to produce greater learning than a 2

student learning on his own (Harasim 1989).1 Furthermore, the role of the instructor moves away from provider of knowledge to facilitator of learning (Hammond 2002).2 Technology, used selectively and judiciously, can facilitate learning and sustain development of higher-order thinking by delivering content using instructional methods that support knowledge construction not reproduction, reflective practice, context and content dependent knowledge construction, collaborative construction of knowledge through social negotiation, and multiple representations of reality and creative options for engaging learners (Jonassen et al. 1993, DeBourgh 2002).3, 4 Computer-assisted simulation training is understood to be valuable in many mission-critical industries and in the military, where it has been shown that large, multi-user distributed simulations can help people train to work together effectively (Shilling et al. 2002).5 Users also have the ability to replay the experience which is not possible in real life (Small 2004).6 This ability to rehearse also plays an important role in turning simulation into a methodology to improve human performance (Scerbo 2005, Halamek 2006).7, 8 In other domains, evidence has been documented regarding how computer simulations evoke emotions and tensions similar to real life (Huang and Alessi 1999).9 According to Mantovani et al. (2003)10, virtual reality represents a promising area with high potential of enhancing the training of healthcare professionals. Traditional distance education delivery modes can create a cognitive and social distance because of the geographic distance among participants and between participants and instructor. Students are more successful in their learning when online courses develop community among the participants (Pallof et al. 1999).11 Multiplayer Online Games The power of games as educational tools is rapidly gaining recognition (Gee 2005, Bonk and Dennen 2005, Fanton 2007, Federation of American Scientists 2006).12, 13, 14, 15. Massively Multiplayer Online Games (MMOGs) are video games capable of supporting hundreds or thousands of players simultaneously, and are played on the Internet. A user can interact within the environment or virtual world by virtue of an avatar, or a computerized character that represents the user. The avatar manipulates and interacts with objects in the virtual world. Massively multiplayer online games have characteristics that set them apart from other games. They are shared spaces where thousands and even millions of players can experience the same game. They are persistent and evolving online environments. The persistent and evolving nature of MMOGs makes them more like the real world and less like the static, intermittent, stand-alone games. 3

A multidisciplinary team at Idaho State University has developed an interactive virtual training environment Play2Train (Ramloll et al. 2007, Ramloll 2007, Ramloll et al. 2008)16, 17, 18 on the Second Life Platform to deliver emergency preparedness training courses. Users (participants) navigate and interact within this a simulated environment via their avatars (Hewitt et al. 2008, Boulos et al. 2008).19, 20 The virtual exercise was carried out in this Play2Train environment. Purpose of exercise The virtual exercise was designed to enhance preparedness skills of the participants, evaluate a triage operation during a pandemic influenza outbreak and observe healthcare workers and first responders interact according to their assigned roles within HICS. The specific aims of the exercise included a)improving the knowledge and skills associated with recognizing and responding to a highly communicable disease outbreak (i.e, the incident command chain, the functional roles and responsibilities, communication, prevention and containment), b)promoting interactions among participants and to illustrate the need for teamwork and coordination, c)applying lessons learned during the virtual exercise to improve preparedness plan, d)increasing pandemic flu event awareness, and e)assessing the innovative training approach in an Online MultiUser Virtual Environment, Play2Train. Core Competencies and Learning Objectives Core competencies were identified for this Pandemic flu triage and formed the foundation of the exercise. These competencies helped define the exercise objective to improve the competencyrelated knowledge and skills associated with recognizing and responding to a highly communicable disease outbreak i.e., the incident command chain, the functional roles and responsibilities, communication, prevention and containment. The participants were represented by their avatars and based on their organization’s current pandemic influenza plan, in response to the pandemic flu event they should be able to demonstrate (enact), describe or list: their expected roles in their practice setting, the process of providing reliable information to others including administrators, staff, media, public and patients, their ability to distinguish between those symptoms, events, and presentations of common infectious illness and those of serious emerging infectious diseases, and the appropriate steps to limit the spread of an infectious disease, and use of appropriate personal protective equipment.

4

METHODS Designing and Developing the Virtual exercise Our multidisciplinary team consisted of multimedia engineers, emergency preparedness professionals and evaluators. The team developed 1) the scenario, 2) trained the participants, 3) designed the virtual learning environment and the virtual resources, and 4) developed and administered the evaluation tools. Scenario We developed a scenario to study triage under a surge situation on a virtual hospital grounds during a Pandemic Flu event. In specifying the design and content of the exercise, reports and health organization documentations and guides were consulted including the 2006 Rand Report on Tabletop Exercises for Pandemic Influenza Preparedness in Local Public Health Agencies (Dausey 2006)21, Homeland Security Exercise and Evaluation Program (HSEEP), the website flu.gov which is the pandemic One-stop website to the US Government pandemic flu information and the HHS Pandemic Influenza Plan Supplement 3 Healthcare Planning. These documents were particularly helpful for the triage set up at a healthcare center. The virtual exercise was stimulated by the scenario. The latter began with an announcement of a Pandemic flu event giving the following details “The Idaho Department of Health and Welfare in collaboration with the regional emergency support function partners is responding response to an outbreak of a novel strain of influenza A (H5N1) virus. The department of Public Health in cooperation with the CDC has declared a state of emergency as the pandemic emerges. The pandemic begins to pick up momentum and hospitals are facing overwhelming surge. Patients are coming to BMH. Utilizing the Incident Command System structure, BMH is activating its Pandemic Influenza response plan for the performance of triage, medical treatments, medical education, communications and surveillance and facility access.” We developed handouts for participants, the scenario script, and a package of tools to assess the exercise including pre/post tests, participant satisfaction questionnaires, video recordings of participant activities and conducted debrief sessions. The pre/post test included questions related to the participants’ roles and responsibilities in response to the pandemic flu event before and after the exercise; a satisfaction questionnaire and a debrief session at the end of the session gauged the reactions of participants to the exercise, its content and the virtual environment. The observations by the evaluators added to the list of what was done well and what could be done better. The results 5

and lessons learned provided evidence of what actions needed to improve the exercise. The machinima (video) of the session was an important tool for further observations and to revisit the exercise by participants and evaluators. Participants The team recruited and trained the participants to make them functional in the virtual learning environment. Nineteen participants located in different cities in South East Idaho took part in this virtual exercise. Participants included nurses, physicians, administrative staff from Bingham Memorial Hospital (BMH), first responders, fire fighters and community volunteers. The volunteers played sick patient roles with specific symptoms and the other participants took their assigned roles within HICS. While all participants were cognizant of their preparedness plans at their workplace, they were requested to review it, and they were also provided with a list which summarized the various roles and their respective responsibilities to help them with this specific scenario. During the exercise a couple of participants had to leave (for real life patients) thereby impacting the chain of command. This situation provided opportunities for participants to adapt to unexpected evolutions resulting from reduced staff emulating real life situations where individuals become sick or have to sign out for family reasons.

Orientation and Training in Virtual Learning Environment The training and the exercise were implemented on our virtual environment Play2Train. Users navigated and interacted within this a simulated environment via their avatars. The virtual setting was a hospital in a rural area, decontamination tents and hospital resources familiar to the hospital staff, and the first responders. The environment was developed with familiar objects to impart a sense of plausibility within the virtual setting to allow immersion of the participants. This was enhanced by the participants customizing their avatar looks and wearing clothing or uniforms according to their roles. Resources including masks, gurneys, wheelchairs, ambulances, beds, specific uniforms for each role such as nurses, doctors, security, were developed to facilitate the tasks to be performed. The virtual environment was set up to support and record the collaborative interactions between responders during an emergency event. Discussions and meetings were held in the virtual environment Play2Train to discuss and develop the exercise. Orientation and training in virtual environment spanned over a couple of weeks for 4 6

hours per week. The duration of the virtual exercise was 2.5 hours. While most of the participants attended all the sessions, a few could not make it to all the sessions given their busy work schedules. On the other hand a few participants accessed, played and practiced in the environment on their own in addition to the official scheduled sessions. The pictures (Figures A-C) give a visual picture of the actual virtual exercise. A short clip showing highlights of the triage exercise is also available on YouTube: http://www.youtube.com/watch?v=v1doVBFl9fs

Figure A Meetings in Virtual Environment to Plan Exercise

Figure B Patients Arrive in Triaging Area

Figure C Patients are Triaged And Post Triage Participants in the Virtual Conference Room for the Debrief Session.

7

RESULTS PrePost Test Results To assess the identified core competencies, pre and post tests were developed and administered. The self perceived competence of the participants were rated using a five point Likert scale. Self efficacy results can be used to predict work performance. Research reveals that there is a strong relationship between self-efficacy and work-related performance (Stajkovic and Luthans 1998).22 Effectiveness of the intervention was assessed by analyzing the pre-test and post-test results. The participants who had completed the pre-test and post-test were included in the analysis. On a five-point scale of 1 to 5, with 5 being the most positive response, the mean scores for the responses increased from pre test to post test for all the competency-related knowledge and skills (Appendix 1-2). The mean overall pretest score was 4.2 and the post test score was 4.7. We compared the mean test scores before (pre-test) and after (post-test) the participants completed the virtual exercise using a paired t-test. It showed that there was statistically significant difference (increase) from pre to post test mean scores for the following: 

I am aware of my expected role and responsibilities within the pandemic influenza action plan of the hospital



I am knowledgeable about infection control measures and about the appropriate steps to limit the spread of a pandemic influenza



In case of a pandemic event I am knowledgeable of how triage, separation protocols and their implementation will take place at the hospital (knowledgeable about the plan)



Pandemic Flu is a potential threat to my community

The perceived increase of these competencies and increased awareness of a pandemic threat after the virtual exercise is encouraging and address the two primary objectives of the virtual exercise, i.e., to increase pandemic flu event awareness and to improve the knowledge and skills associated with recognizing and responding to a highly communicable disease outbreak i.e., the incident command chain, the functional roles and responsibilities, communication, prevention and containment. Participants’ reactions Participants’ reactions to the virtual exercise and its content were very positive and scored an overall average rating of 4.3 out of 5, with all but one having a mean score of 4.1 or above (Appendix 2). 8

“The VL scenario helped me gain additional insight into team collaboration” made the highest score with 4.7. “Participation in the exercise was appropriate for my role” had the lowest score of 3.8. While other participants had scored it highly, one of the participants had given it a score of 1. She had commented that during this virtual drill there was a real life patient occurrence and she had to leave the virtual exercise to attend to real life patients. Debrief results and Observations The participants remarked that the training in the VLE enhanced the team concept and allowed different organizations from different places to train together in an effective way. Some participants appreciated the details from the bottled water, wheel chairs, gurneys to the special effects of the rain and everything in between which allowed for a suspension of disbelief. They observed that this environment provided the possibility to conduct exercises, drills, without disrupting the running of the hospital and causing unneeded anxiety or creating risks to participant or patients. A couple of participants pointed out that initially it took time to get a feel of what was going to happen in the virtual setting until the mock exercises. A few participants did not have enough practice time, and encountered navigation, and camera control problems. A couple of computers were running slower rendering some of the tasks difficult to complete. Certain tasks such as driving the ambulance were not easy in the environment – better scripts have to be improved to facilitate user interaction. Comments included that it was the team could be expanded in the future by having more cross agency key players join in for a disaster drill and interact. Another important point mentioned was that they learned more about their own emergency plan “by playing their own roles” in the virtual setting and remarked that it is a useful tool to test and to improve their plan.

Discussion; Limitations and Conclusions The major limitation of this study has to do with the extent to which the findings can be generalized beyond this study. The number of participants is too limited for any inferences and generalizations. Another limitation applicable to this study, is that participants were not fully representative of the target population. In this particular case, there were only 19 participants who participated in the complete study, and most of them came from the South East region of Idaho and mostly from one small rural hospital. Further work in different contexts and with larger numbers of participants, however, is needed for generalizable findings. However, this study presents a new approach for

9

training in teams, and researchers, educators and virtual environment specialists can benefit from our findings. Valuable lessons were learned during this virtual exercise by observing situation-response interactions and analyzing the evaluation results. The virtual environment was successful in bringing together remotely located participants. In addition, participants reported a high level of immersion in the exercise; they identified easily with their roles and were able to communicate effectively with peers. The feedback and debrief were very positive, and the pre and post tests results showed that the exercise produced perceived improvements in competency. Observations and participants’ feedback revealed that the team interaction afforded by the environment was remarkable and allowed participants to practice without stress in a safe environment. The virtual exercise highlighted issues which would help enhance the hospital preparedness plan. A couple of participants had to leave (to take care of real life patients) thereby impacting the chain of command and communication and coordination with others. This created more realism and the chain of command had to adjust to this situation. This gave rise to revisiting their preparedness plan. The virtual exercise also provided opportunities for the unraveling of the level of planning and care required to avoid cross contamination occurrences if too many patients came in. It also allowed the participants to apply what they had learned in didactic courses, e.g. how to deal with different types of volunteers. Viewing the machinima (video) of the recorded session allowed the participants to revisit and to review how the process worked and the evolution of their role in the process. They could thus identify and evaluate their own strengths and weaknesses. As expected those who attended all the training sessions were more comfortable and confident during the virtual exercises. Another lesson learned was that adequate and effective orientation and training was required to make navigation, communication and interaction with objects easier for the participants. As the interface becomes more intuitive the environment will allow a better experience. Despite the growing application of online 3-D virtual training, research and evaluation literature regarding virtual preparedness exercises and their appropriate development and relative effectiveness in improving public health preparedness is sparse. Our endeavor is to contribute towards this literature by documenting our experience. In the future, we plan to carry out studies to evaluate whether competency gains achieved in the virtual environment transfer to real world situations and further research the human experience and engagement within the virtual environment. 10

References 1. Harasim L. On-line education: A new domain. In Mason R, Kaye A (eds.): Mindweave: Communication, Computers and Distance Education, Oxford, Pergamon Press, 1989, pp. 50-62. 2. Hammond M. Discourses on collaborative networked learning. Proceedings of the Third International Conference on Networked Learning. Sheffield University, 2002. 3. Jonassen DH, McAleese TMR, Duffy TM. A manifesto for a constructivist approach to technology in higher education. In Duffy TM, Lowyck J, Jonassen DH (eds): The design of constructivistic learning environments: Implications for instructional design and the use of technology. SpringerVerlag, Heidelburg, 1993, pp. 231–247. 4. DeBourgh GA. Simple Elegance: Course Management Systems As Pedagogical Infrastructure to Enhance Science Learning. e-journal The Technology Source. May/June 2002. Available at http://technologysource.org/?view=article&id=277 Accessed March 28 2007 5. Shilling R, Zyda M, Wardynski C. Introducing Emotion into Military Simulation and Videogame Design: America’s Army: Operations and VIRTE. In Proceedings of the Game On Conference, London, United Kingdom. Available at http://gamepipe.usc.edu/~zyda/pubs/ShillingGameon2002.pdf. Accessed March 28 2007. 6. Small SD. Medical Education: Thoughts on Patient Safety Education and the Role of Simulation. American Medical Association Journal of Ethics. 2004: 6(3). 7. Scerbo MW. Human factors in Medical Modeling and Simulation. MMVR Conference. Available at http://www.simcen.org/mmvr2005/slides/med-sim2005-scerbo.pdf. Accessed March 23 2007. 8. Halamek LP. Simulation-Based Training: Opportunities for the Acquisition. Virtual Mentor. American Medical Association Journal of Ethics of Unique Skills. 2006; 8: 84-87. 9. Huang MP, Alessi NE. Presence as an emotional experience. In Westwood JD, Hoffman HM, Robb RA, & Stredney D (eds.): Medicine meets virtual reality: The convergence of physical and informational technologies options for a new era in healthcare. Amsterdam: IOS Press, 1999, pp. 148–153. 10. Mantovani F, Castelnuovo G, Gaggioli A, Riva G. Virtual Reality Training for Health-Care Professionals. CyberPsychology & Behavior. 2003; 6(4): 389-395. 11. Palloff RM, Pratt K. Building Learning Communities in Cyberspace: San Francisco, CA, Jossey-Bass, 1999.

11

12. Gee, JP. Why video games are good for your soul: Pleasure and learning. Book: Print (Paperback), ISBN: 1863355758. Melbourne, Australia, The Learner, 2005. 13. Bonk C, Dennen V. Massive multiplayer online gaming: A research framework for military training and education. Available at http://www.adlnet.gov/SiteCollectionDocuments/archive/GameReport_Bonk_final.pdf Accessed August 15, 2008. 14. Fanton JF. Do Video Games Help Kids Learn? Digital Media and Learning Panel Discussion, Mac Arthur Foundation, Introductory Remarks, Newberry Library, Chicago, IL, 2007. Available at http://www.macfound.org/site/c.lkLXJ8MQKrH/b.1142275/apps/nl/content3.asp?content_id=%7B1CF15D B7-36ED-4899-8EA4-B0D4ECA22F48%7D¬oc=1 Accessed August 15, 2008. 15. Federation of American Scientists. Harnessing the power of video games for learning. Washington, DC, 2006. Available at http://www.fas.org/gamesummit/Resources/Summit on Educational Games.pdf 16. Ramloll R., Beedasy J, Stamm BH, Piland N, Cunningham B. Play2Train: A generic large-scale virtual environment for emergency preparedness training. Telemedicine and e-Health Journal. 2007; 13(2): 200. 17. Ramloll R. The Power of Play, Real-life experts meet in a virtual world to prepare for disaster. Most Wired Magazine. October 17 2007 issue. 18. Ramoll R, Beedasy J, Piland N, Stamm BH, Cunningham B. Collaborative To Enhance Learning. 3-D Virtual Learning Environments. Telemedicine and e-Health. 2008: March 14(s1), 29-30. 19. Hewitt A, Spencer S, Ramloll R et al. Expanding CERC Beyond Public Health: Sharing Best Practices with Healthcare Managers via Virtual Learning. Journal of Health Promotion Practice. 2008; 9(4): 83s-87s. 20. Boulos K, Maged N, Ramloll R et al. Web 3D for Public, Environmental and Occupational Health: Early Examples from Second Life. International Journal of Environmental Research and Public Health. 2008; 5:290-317. DOI: 10.3390/ijerph5040290, ISSN 1660-4601. 21. Dausey DJ, Aledort JE, Lurie N. RAND Report Tabletop Exercises for Pandemic Influenza Preparedness in Local Public Health Agencies. RAND, Santa Monica, CA, 2006. Available at http://www.rand.org/pubs/technical_reports/2006/RAND_TR319.pdf Accessed April 21 2007 22. Stajkovic AD, Luthans F. Self-efficacy and work-related performance: A meta-analysis. Psychological Bulletin. 1998. 124:240-261. doi: 10.1037/0033-2909.124.2.240

12

Appendix 1- Comparison Of Pre To Post Scores Virtual Exercise

Q1. I am aware of my expected role and responsibilities within the pandemic influenza action plan of the hospital

Mean

SD

Mean

SD

Pre

Pre

Post

Post

4.1

0.60

4.8

.44

Q2.I am confident that I can use the chain of command to perform emergency response tasks effectively during a pandemic event at the

P 0.02 0.59

4.4

0.73

4.6

.73

4.3

0.71

4.7

.50

4.2

0.67

4.6

.73

4.2

0.67

4.8

.44

4.1

0.78

4.6

.73

hospital Q3. I can provide reliable information to others including administrators, staff, and patients and external partners Q4. I am aware of how to communicate with the media or to direct them to the appropriate person for information Q5. I am knowledgeable about infection control measures and about the appropriate steps to limit the spread of a pandemic influenza Q6. I am know how and when to use appropriate personal protective equipment in my practice Q7. In case of a pandemic event I am knowledgeable of how triage, separation protocols and their implementation will take place at the

0.08 0.08 0.01 0.10 0.00

3.8

0.83

4.7

.50

4.0

0.50

4.7

.50

hospital (knowledgeable about the plan) Q8.Pandemic Flu is a potential threat to my community

5 4.5 4 3.5 3 2.5 2 1.5 1 0.5 0

0.00

Pre Post

Q1

Q2

Q3

Q4

Q5

Q6

Q7

Q8

Comparison of pre post and tests scores

13

Appendix 2- Participant Satisfaction Scores

The VL scenario helped me gain additional insight into team collaboration The facilitator(s)/controller(s) was knowledgeable about the material and kept the exercise on target The VL scenario could be used in place of a class discussion as an “applied lesson” or exercise The exercise was well organized and structured The level and mix of participants included the right people for this exercise I enjoyed the virtual learning (VL) experience The VL helped me to gain further knowledge about managing emergency preparedness I feel better prepared to respond to a Pandemic flu event at the hospital effectively The briefing and/or exercise narrative hand out helped me understand and become engaged in the scenario The method of delivery used for the training facilitated my learning experience The exercise scenario was realistic The VL scenario helped me to understand the complexity of making real-time decisions in a crisis situation I have improved my understanding of my functional roles and responsibilities in effectively responding to a Pandemic Flu event I have improved my approach for coordination and collaboration necessary to effectively respond to a Pandemic Flu event I have improved my communication skills in responding to a Pandemic Flu event The VL scenario allowed me to better understand my role within the response team than a traditional discussion tabletop exercise around a table. The VL scenario allowed me to better enact my role within the response team than a traditional discussion tabletop exercise around a table. Participation in the exercise was appropriate for my role

Mean

Std. Error of Mean

SD

4.7

0.153

0.483

4.6

0.163

0.516

4.6 4.6

0.221 0.267

0.699 0.843

4.5 4.5

0.167 0.224

0.527 0.707

4.5

0.224

0.707

4.4

0.163

0.516

4.4

0.221

0.699

4.4 4.3

0.267 0.3

0.843 0.949

4.3

0.335

1.059

4.2

0.249

0.789

4.2

0.249

0.789

4.2

0.249

0.789

4.1

0.277

0.876

4.1 3.8

0.458 0.389

1.449 1.229

14

Determining tiie competencies to be addressed and ...

(Scerbo 2005, Halamek 2006).7, 8 In other domains, evidence has been .... A short clip showing highlights of the triage exercise is also available on. YouTube: ...

604KB Sizes 3 Downloads 116 Views

Recommend Documents

A Mad Poem Addressed to My Nephews and NiecesTest Booklet.pdf ...
A Mad Poem Addressed to My Nephews and NiecesTest Booklet.pdf. A Mad Poem Addressed to My Nephews and NiecesTest Booklet.pdf. Open. Extract.

tiie state of'gujarat - TaxGuru
leamed Joint Commissioner of Commercial Tax, Division-l0, Rajkot. (hereinafter referred to as the fnst appellate authority) on 31-03-2015 whereby the appeal ...

letter addressed to Captain Thomas Zwolfer - Ground Zero Center for ...
Aug 10, 2015 - Abolish Nuclear Weapons”—in keeping with our theme for this weekend's ... stop building and deploying these illegal and immoral weapons.

letter addressed to Captain Thomas Zwolfer - Ground Zero Center for ...
Aug 10, 2015 - Kitsap-Bangor to call for an immediate halt to the proposed updating and ... contact us through Ground Zero Center for more information and ...

225 130311 NV CURE - Gibson ltr and Self addressed stamped ...
Page 1 of 4. Page 1 of 4. Page 2 of 4. Page 2 of 4. Page 3 of 4. Page 3 of 4. 225 130311 NV CURE - Gibson ltr and Self addressed stamped envelope.pdf. 225 130311 NV CURE - Gibson ltr and Self addressed stamped envelope.pdf. Open. Extract. Open with.

Population-Based-Nursing-Concepts-And-Competencies-For ...
Retrying... Whoops! There was a problem loading this page. Retrying... Population-Based-Nursing-Concepts-And-Competencies-For-Advanced-Practice.pdf.

Determining Sample Size
Nov 6, 1992 - This document is Fact Sheet PEOD-6, a series of the Program Evaluation and Organizational Development, Florida Cooperative Extension.

Need to consider human factors when determining first-line technique ...
May 21, 2016 - sive care unit, these events are usually managed by the ... not necessarily support the conclusion that cannula techniques ... London, 2011.

iOS Competencies - DeadRatGames
Performance. Acceptance. Regression. Continuous Integration. XCTest. Deployment. Device testing. TestFlight. App Store. Provisioning profiles. Code-signing.

Need to consider human factors when determining first-line technique ...
May 21, 2016 - 2Department of Anaesthesia, Monash Medical Centre, 246 Clayton Road, Clayton, Melbourne, ... E-mail: [email protected].

Determining Sample Size
Nov 6, 1992 - the new practice only affected 2,000 farmers. The sample size that would now be necessary is shown in. Equation 4. As you can see, this adjustment (called the finite population correction) can substantially reduce the necessary sample s

A Reuse-Based Approach to Determining Security ... - Semantic Scholar
declarative statements about the degree of protection required [17]. Another ..... be Internet script kiddies, business competitors or disgruntled employees. ..... administration risk analysis and management method conforming to ISO15408 (the ...

Need to consider human factors when determining first-line technique ...
May 21, 2016 - consider the likelihood of technical success of each of these meth- ods. .... 4th National Audit Project of The Royal College of Anaesthe-.

Be Going to Activities.
Expresamos el futuro con el verbo be en presente,going to y el infinitivo: It is going to rain.(Va a llover ... 's going to make a phone call. 0 He. 1 They. 2 She. 3 He.

Interpersonal-Competencies-ParentGuide.pdf
ResearchCollaboration.org. Page 2 of 2. Interpersonal-Competencies-ParentGuide.pdf. Interpersonal-Competencies-ParentGuide.pdf. Open. Extract. Open with.

Additional Strategies for Determining Importance.pdf
Responses: Three sticky notes, each one coded * to mark three important ideas in. the text. oThe main idea of this strategy is to show student that there is more ...

Procedure for determining soil-bound organic carbon and nitrogen ...
Available from: Kevin Geyer Harrison, Feb 02, 2015 ... We present a method for determining soil-bound organic carbon, which we define as soil ..... the numerical data used to calculate the percentage of carbon and nitrogen present in the ...