Dynamic assessment of fieldwork Che-Ming Chen; Ciou-Lien Chen and Tzu-Yu Lin (National Taiwan Normal University, Taiwan)

Abstract: Teachers need to know about their students' learning progress and difficulties during the learning process because this information is important for teachers to interactively adapt their teaching. However, it can be difficult to do so while actually in the field. The aim of this study is to design a dynamic assessment module (DAM) in a GPS-embedded smart phone which can integrate fieldwork instruction and assessment to meet individual student’s learning needs in the field. The GPS navigation function of the smart phone can guide students to each learning waypoint. The DAM can automatically show multiple-choice questions regarding the geography knowledge of each waypoint to students as pop-ups. The DAM can identify each student’s learning difficulties based on students’ answers and simultaneously provide them the relevant instruction as feedback. The goal of the DAM is not only to evaluate a student's fieldwork performance but, more importantly, to scaffold them to complete fieldwork tasks. Keywords: Dynamic assessment, smart phone, fieldwork

1. Introduction Fieldwork is one of the most important methods for teaching geography because students can learn in the real-world context and develop various hands-on skills. However, it’s difficult to evaluate students’ learning outcomes and how they develop knowledge and skills during the fieldtrip, especially when a large number of students are leaded by only a few teachers. Although teachers can evaluate students’ performance after the fieldtrip in the classroom, this conventional summative assessment can measure students’ static rather than ongoing performance. The results of the summative assessment reflect students’ past achievements and cannot contribute to teaching and learning during the fieldwork. The popularity of mobile phones, GPS, and wireless networks provide new opportunities for improving the learning environments (Huang, et. al., 2008). Not only learners can learn anytime, anywhere by carrying mobile devices, but also teachers can monitor students’ learning statuses and provide relevant assists to them on the fly (Liu, et. al., 2003). It becomes possible to conduct dynamic assessment in the field by means of the integration of these ICTs. In this way, the assessment can be conducted with ongoing learning. Moreover, assessment itself becomes a valuable learning process (Parsons and Fenwick, 1999). This study aims to develop a dynamic assessment module for geographic fieldwork in high school level. The DAM is implemented in a mobile learning environment. The field tests were conducted in the Guizikeng Water and Soil Conservation Outdoor Classroom in Taipei to evaluate the system performance and the effectiveness of the module.

2. Dynamic assessment and mobile learning 2.1 Principles of dynamic assessment Dynamic assessment (DA) challenges the conventional assessment which separates assessment and instruction. DA integrates them according to Vygotsky’s sociocultural theory of mind that individual’s ability is developed by mediated interactions with others and cultural artifacts. DA provides these interactions to activate learners’ development of full range abilities. Therefore,

1

DA is not only an assessment but also an instruction. In other words, DA interactively scaffolds students’ learning (Poehner, 2008). Within the context of sociocultural theory, Vygotsky introduced the notion of the zone of proximal development (ZPD), which was defined as: “the distance between the actual development level as determined by independent problem solving and the level of potential development as determined through problem solving under adult guidance”(Vygotsky, 1978. p. 86). Based on the concept of ZPD, Campione and Brown (1985) developed the procedure of DA as 4 phases: (1) pre-test: diagnose the initial level of learner expertise; (2) learning: acquire new information in the domain; (3) transfer: using that information in novel situations; (4) posttest: evaluate the final level of learner expertise. Therefore, DA can improve learners’ performance during the assessment process. In the context of the social studies, Parsons and Fenwick (1999) recommended teachers to adopt DA because it is based on authentic tasks and focuses on situational learning. They point out that DA is effective to social studies classrooms in three ways: (1) It’s derived from realword contexts; (2) It’s relevant to the content and context of the course delivered; (3) It makes evaluation itself a valuable learning experience. Unlike standardized assessments, DA is process-oriented, student-centered, and relevant to the goals of social studies.

2.2 Dynamic assessment and mobile learning The advance of the information and communication technologies provides potential solutions to overcome the common problems faced in the outdoor courses, such as time limitation, large number of students, safety concern, etc. With handheld devices, students can learn individually at anytime and anywhere (Liu, et. al., 2003). The embedded GPS in the handheld devices can report students’ locations via wireless networks to their teachers. Teachers can also send the instruction to the students based on their learning behaviors or demands. The implementation of DA requires frequent interaction between learners and teachers in an authentic situation. This requirement can be met in mobile learning environment. Current research trends show that DA is an important assessment mode in the e-learning environment. Some e-learning researchers have demonstrated that DA can effectively facilitate synchronous learning. For example, Kalyuga and Sweller (2005) designed a DA from the perspective of cognitive load. The DA can evaluate learners’ expertise based on their content knowledge and the measures of cognitive load. The instruction was dynamically adapted to changing levels of expertise. To evaluate the effectiveness of this DA approach, 30 K-10 students in the algebra course were randomly separated into the learner-adapted experimental group and the non-adapted control group. The instructional packages of elementary algebra were delivered through desktop computers. The result indicated that the experimental group has significantly higher average efficiency gains than the control group. Chen and Chen (2009) focused on the effectiveness of the assessment mechanism in e-learning systems. They used 6 computational intelligence theories of data mining to identify the key formative assessment rules based on the web-based learning portfolios of individual learners. The learning performance of learners was evaluated according to these rules and the proposed learning feedback was then provided to enhance learning. The quasi-experimental method was conducted with 2 K-3 classes of students as the participants in the mathematics course. The results also showed that the learning performance of the experimental group was significantly better than the control group. Huang, et al. (2008) developed a decision tree approach to conducting DA in a u-learning (ubiquitous learning) environment that employs PDA, wireless networks, and RFID (Radio Frequency Identification). The decision tree algorithm implemented in the DA can guide K-4 students to identify the critical features of plants in a butterfly garden. The semi-structured interviews of the 5 participants showed that the DA approach can enhance the motivation of these students. Although these researches

2

demonstrated that DA can facilitate e-learning, most studies preferred using elementary students to highschool students as participants. Besides, most DA systems used PDA as the learning device. PDA’s market is significantly declining as smart phone’s market is rapidly growing. It’s relevant to develop DA in the context of high school geography fieldwork using smart phones.

3. Dynamic assessment module for fieldwork The goal of this study is to develop a DAM for the high school fieldwork. The components of the system include: (1) a system architecture that supports the two-way communication between teachers and learners; and (2) the DAM that includes the field test items and the realtime feedback instructions.

3.1 System architecture In the mobile learning environment, each student has a smart phone embedded with GPS receiver. The GPS coordinates of individual students can be transmitted to the streaming server via the wireless network so that teachers can track their real-time locations. As the student moves approximately (< 15m) to each waypoint of the field tests, the system will activate the test items as pop-ups in the smart phone. The DAM will provide feedback instructions to the students based on their responses to the test items. The learning portfolios of each student can be stored in the database and reviewed by the teachers using Web browser simultaneously (Figure 1).

Figure 1. System architecture

3.2 Development of the dynamic assessment module The reason why DA can facilitate students’ broader learning is that it tries to maximize the transferability of what has been learned based on individual student’s ZPD. This study employed Gagńe’s concept of lateral transfer and vertical transfer to develop the test items. The lateral transfer refers to apply the knowledge and skills have been learned to new situations at the same level of complexity. The vertical transfer means apply the subordinate knowledge and skills to higher-order learning (Gagné, 1970).

3.2.1 Development of the test items In the DAM, there are 3 categories of test items including questions for learning, questions for lateral transfer, and questions for vertical transfer. (1) Questions for learning: students learn concepts or execute fieldwork tasks by answering these questions. If the type of the question is multiple-choice, the DAM will automatically show hints to students when they input a wrong answer and ask students to answer that question again. If students answer correct answer in the first place, the DAM will directly guide the student to the next waypoint without further instruction. The system also supports other types of questions. For example, students are asked to type their responses 3

in a small text box for the short answer questions. For the picture-taking questions, students have to look for the target object in the field and send its digital photo back to the streaming server using the smart phone via wireless network. For these types of questions, the system won’t provide instant instruction to students because there are no standard answers for them. However, if teachers find it’s necessary to provide feedback to students according to their responses shown in the monitor interface, teachers can send instant messages to their smart phones. (2) Questions for lateral transfer: When students complete the questions for learning, they are assumed to have a specific level of ability. The DAM begins to provide them the questions for transfer. The dimensions of the transfer in this study are defined based on Bloom's revised taxonomy of cognitive domain (Anderson and Krathwohl, 2001). Learning and assessment activities can be matched to 6 key cognitive processes. In the sequence of increasing complexity, these processes are remembering, understanding, applying, analyzing, evaluating, and creating. The questions for lateral transfer require students to apply the knowledge or skills obtained from the questions for learning to new situations at the same level of cognitive process. It’s worth a mention that “new situations” in other disciplines such as mathematics or languages may simply refer to replace a variable or verb in the question. However, in the context of geographic fieldwork, “new situations” usually involve moving to another location. The accessibility for each question has to be taken into consideration. (3) Questions for vertical transfer: These questions are used to promote students’ learning performance from the current level of cognitive process to a higher level. The students’ current performance is identified from the questions for learning. The DAM will provide relevant instruction to scaffold students’ learning, if they fail to reply correct answers. Students’ abilities are therefore expanded level by level. A comprehensive example using a set of questions is given below to demonstrate the 3 categories of questions in the DAM. These 3 questions have the same learning objective that students can explain the impacts of local land use on the occurrence of debris flow. The question no. 1 is a question for learning (Table 1). A student will obtain one hint whenever he selects a wrong answer until he punches the correct choice. The DAM will then navigate students to the waypoint for the question no. 2, a question for lateral transfer (Table 2). This question is set for the same level of cognitive process, analyzing, but it requires a different skill, map interpretation. The DAM will provide 2 hints at most for one question. When a student fails for the third time, the system will provide the correct answer and its interpretation. The DAM sequentially guides students to the waypoint for the question no. 3, a question for vertical transfer (Table 3). The level of cognitive process has been promoted from analyzing to evaluating. Because it is a short answer question, teachers rather than the system will provide hints to students. The DAM not only sends hints as the instruction to students, but also stores the process of answering questions in each student’s portfolio in the server. The descriptive statistics of each student’s learning performance are provided to teachers simultaneously. Teachers are able to trace each student’s current location, historical track, and ongoing performance. If they find some students need further help beyond the DAM’s instruction, they can send relevant information as instant text message to students via monitor interface.

3.2.2 Field test The question items of the DAM are still under development. At present, only the system functionality has been tested in the field. The preliminary results showed that all of the software components including GPS navigation, DAM’s auto feedback, monitor interface, and instant text message have successfully passed integration testing in the Guizikeng Water and Soil Conservation Outdoor Classroom (Figure 2 & 3).

4

Table 1. Question for learning Item no. 1 Theme Environmental hazard Objective Explain the impacts of local land use on the occurrence of debris flow Question Cognitive learning analyze category level Vertical Lateral 3 2 transfer no. transfer no. Question：Find the location of this picture. Why the hill is so steep? Options: (A) erosion by rainwater (B) erosion by wind (C) quarrying activities (correct) (D) landscape design System auto responses (hints): (A) It’s possible, but rainwater is not the major force. Think about the human factor. (B) The wind here is not strong at all. Think about the human factor. (C) Correct answer. Move to next waypoint. (D) Unlike a design work, it looks bare and collapsing. Think about the local history of industry.

Table 2. Question for lateral transfer Item no. 2 Theme Environmental hazard Objective Explain the impacts of local land use on the occurrence of debris flow Question Cognitive lateral tranfer analyze category level Learning 1 transfer no. 3 no. Question：Analyze the contour map of this place in 1926. What change had taken place regarding the terrain?

5

Table 2. Question for lateral transfer (cont.) Options: (A) hilltop became flat (B) foothill became flat (correct) (C) the stream in the east disappeared (D) a pond appears in the west System auto responses (hints): Hint 1: You are standing on point A. The square box shows the hill and flat ground in the picture. Hint 2: The terrain of point A became flat now. Hint 3: (correct answer) The place where you are standing at used to be a slope, but now it became a flat because of the quarrying activities in the past. Table 3. Question for vertical transfer Item no. 3 Theme Environmental hazard Objective Explain the impacts of local land use on the occurrence of debris flow Question Cognitive vertical tranfer evaluate category level Learning no. 1 transfer no. 2 Question：If there were no quarrying industry in this area, would it change the possibility of the occurrence of the debris flow in 1977? Teachers can provide the following hints based on students’ answers: 1. The occurrence of a debris flow can be evaluated by 3 criteria. 2. These criteria are a large amount of loose sediments, large amounts of rainfall, and slope. 3. The quarrying industry in the past provided large amount of loose sediments. If there were no quarrying activities here, it would reduce the possibility of the occurrence of the debris flow in 1977.

Learning

Lateral transfer

Vertical transfer

Figure 2. Screenshots of the smart phone

6

Google Map showing students’ locations (red dots) and test spots (purple dots)

switch to statistical tools

student list

instant text messages from students

personal record

Figure 3. A screenshot of the monitor interface

4. Conclusions In this study, a dynamic assessment module was developed based on Vygotsky’s notion of the zone of proximal development in a mobile environment. The module can provide instant feedbacks when students need instruction according to their performance evaluated by the DA. The highly interactive function between a student and the system makes personalized learning possible, which may lower teachers’ loading during the fieldwork. Besides, teachers can review the descriptive statistics of all students’ learning performance during the fieldwork or after the fieldwork. The merits of formative and summative assessments can still be retained. The system functions of the DAM have been verified by field tests. The next step of this study is to prove the effectiveness of the DAM by inviting high school geography teachers and students as participants in the field test.

Acknowledgement This study is supported by the National Science Council of Taiwan under the project number 96-2415-H-003-005-MY2.

References: Anderson, L. W. and Krathwohl, D. R., et al (Eds.) (2001) A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom's Taxonomy of Educational Objectives. Allyn & Bacon. Boston, MA (Pearson Education Group). Campione, J. C. & Brown, A. L. (1985) Dynamic assessment: one approach and some initial data, Technical Report No. 361, University of Illinois at Urbana-Champaign, Center for the Study of Reading. Chen, C. M. & Chen, M. C. (2009) Mobile formative assessment tool based on data mining techniques for supporting web-based learning, Computers & Education, 52(1): 256-273. Gagné, R. M. (1970) The conditions of learning, 2nd ed., London: Holt, Rinehart and Winston. Huang, S. H., Wu, T. T., Chu, H. C. & Hwang, G. J. (2008) A decision tree approach to conducting dynamic assessment in a context-aware ubiquitous learning environment. Fifth IEEE International Conference on Wireless, Mobile, and Ubiquitous Technology in Education, 89-94. 7

Kalyuga, S. & Sweller, J. (2005) Rapid dynamic assessment of expertise to improve the efficiency of adaptive e-learning, Educational Technology Research and Development, 53: 83-93. Liu, T. C., Wang, H. Y., Liang, J. K., Chan, T. W., Ko, H. W. & Yang, J. C. (2003) Wireless and mobile technologies to enhance teaching and learning, Journal of Computer Assisted Learning, 19: 371-382. Parsons, J. & Fenwick, T. J. (1999) Using dynamic assessment in the social studies classroom, Canadian Social Studies, 34(1): 153-155. Poehner, E. M. (2008) Dynamic Assessment: A Vygotskian Approach to Understanding and Promoting L2 Development, Springer US. Vygotsky, L. (1978) The Role of Play in Development, Mind in Society, Cambridge, MA: Harvard University Press.

Authors: Prof. Che-Ming Chen, Department of Geography, National Taiwan Normal University, 162, Sec. 1, Ho-ping East Rd., Taipei City 10610, Taiwan, [email protected] Ms. Ciou-Lien Chen, Department of Geography, National Taiwan Normal University, 162, Sec. 1, Ho-ping East Rd., Taipei City 10610, Taiwan, [email protected] Mr. Tzu-Yu Lin, Department of Geography, National Taiwan Normal University, 162, Sec. 1, Ho-ping East Rd., Taipei City 10610, Taiwan, [email protected]

8