Group work peer-assessment in an online environment Tinoca, L.F.a,b, Oliveira, I.a,b & Pereira, A.a a

Departamento de Ciências da Educação, Universidade Aberta Centro de Investigação em Educação Portugal b

[email protected]; [email protected]; [email protected]

Abstract In online open and distant learning, group work is a commonly used strategy, given its collaborative nature and constructivist framework (Bates, T. & Poole, G, 2003; Dillenbourg, P., 1999; Garrison, D. R., & Anderson, T., 2003; Jonassen, D.H., 2005). However, in the assessment of group projects, the instructor often faces difficulties in waging the contribution of different students towards the final product. On the one hand, the final product resulting from a group project can be objectively assessed; on the other hand, student contribution towards that product is much more subjective and frequently hidden from the instructor’s supervision. This context has led us to create a group project where the students (n=23) where responsible for the assessment, of each of the group element’s, contribution towards the final product. They were given a set of criteria to guide their assessment and each asked to conduct a self-assessment and also to assess each of the group colleagues. Two main research questions were asked: How do students perceive this way of assessing group work? How much do the students believe that the process of group work (assessed by them) should weigh when compared with the group’s final product (assessed by the instructors). To answer these questions, after the Course was finished, each student was asked to answer a questionnaire. Moreover, each of the online discussion forums was also coded and added to this analysis. This type of assessment was welcomed by the students and effectively implemented by all the groups (n=8) but one. Nevertheless, the interpretation made by each group of what should be valued and how it should be operationalized varied significantly. This raised issues of equity and validity of the suggested method. Keywords: Computer supported collaborative learning (CSCL), collaborative learning, online assessment, learning community, group work Introduction Based on the recent developments on open and distance learning (ODL), the Universidade Aberta has assumed a student centered pedagogical model (Pereira et al., In Press) founded on flexibility – where asynchronous technology is preferred and students and instructors can participate on the Courses from wherever they are located – and supported on three types of interaction: student-content; student-instructor; and student-student. Moreover, on the principle of digital inclusion, giving Access to a higher education institution to an adult population that had no previous competences on instructional and communication technologies (ICT). During the last 2 years, the research project MEDEIA on which these authors participate, has been developing work related to creation of learning environments supported by virtual classroom systems (VCS). According to Pereira et al. (2003), the assessment strategies and instruments used in online education are much different from those used in traditional classrooms, or traditional ODL. These authors refer the use of diagnostic, formative and summative assessments, and the most commonly used assessment instruments identified in their research were: tests, final exams, essays, projects, problem solving, case studies, and portfolios; which are all cited as common to the mentioned learning environments. The task of developing online Courses, and more specifically, designing Curricular Unit plans is on of the Professors tasks. In the implementation of this task we argue about the importance of adopting a curricular congruence where the significance and objectives of the

assessment should attain a balanced coverage all the curricular objectives and competences, and the principle of integration assuming assessment as an integral part of the learning process and as formative and constructive outcome. This study explores assessment processes and the significance that the participants attribute to assessment in general, and to self-assessment and peer-assessment in particular. In order to achieve a more valid, fair and responsible assessment practice we should consider not only individual tasks, but also the participants’ engagement in small group projects. In fact, nowadays, more and more often professionals are expected to work collaboratively in small groups, and tasks are designed to be interdependent, requiring the sharing of information and knowledge. In this perspective, the online Course where the participants were enrolled defined as one of its major goals the development of argumentative and debating competencies in collaborative settings, both on online discussion forums and during the development of small group projects. It is, therefore, necessary to further understand the how the processes associated with online collaboration and discussion are assessed, and how their assessment is perceived by and impacts the participants. Only by doing that can we build on the knowledge that we have about the development of Curricular Units, and based on it can the responsible Professors continually and consistently improve the quality of their Courses. Theoretical background Framed by a new collaborative approach where “learning emerges form the work done with our peers, sharing experiences and perspectives based on common goals and negotiated group interactions” (Pereira et al., 2007), the design of learning and teaching activities, as well as the respective assessment practices, takes on a central role on every Curricular Unit. In this context, assessment constitutes a very important part of the learning process, guides the learning results and is essential for the instructional design (Comeaux, 2005). Collaborative Learning The notion that peer interaction stimulates knowledge production and produces cognitive gains (Perret-Clermont et al., 1991; Dillenbourg, 1999) explains many pedagogical decisions. Several activities, such as confronting point of views, explaining a procedure to a peer, or peer-assessment might be designed to promote the interaction of two or more individuals. Virtual worlds and learning environments provide participants with the possibility to appropriate knowledge and develop competencies through exploration, research and experimentation, putting them in contexts, groups and situations that offer diverse learning settings. This way, we argue for the endorsement of collaborative learning contexts that stride for the quality of learning through the understanding of the relationships between participants, tools/artifacts, and social groups. The instructor should act mainly as a facilitator to the learning process, directing his participation towards the orientation of the community/group work in a productive direction while supervising the peripheral participants (Lave & Wenger, 1991), to whom tools may be provided to self-regulate their interactions (Dillenbourg, 1999). This may very well be achieved through the explicit definition of the criteria structuring the communication interactions in this alternative approach. The construction and appropriation of knowledge is highly influenced by the individuals’ social experiences (Vygotsky, 1978). According to this perspective, learning is an activity grounded on sociocultural interactions (Wertsch, 1991), and the construction of knowledge is achieved socially through the participation in communities of practice (Lave & Wenger, 1991). Therefore, it is extremely important to pay careful attention in the creation of virtual learning environments to the social interaction and collaborative learning settings. On any given situation, participants produce and create the contexts of their community of practice, supporting themselves on the cognitive, social, and physical aspects of the environment that they consider relevant. But we must have present that in order for the collaboration to take place there must be a motive and space for negotiation. According to Dillenbourg (1999) collaboration is characterized by participants who share a common goal,

are at a similar level and can perform the same actions while working together. These last two aspects, are concerned with the action symmetry, as each participant has access to the same set of actions – the symmetry of knowledge and the symmetry of status – meaning that the participants have similar knowledge and a similar status in their community of practice. On another hand, the interactions defined as collaborative have the possibility to influence the peers cognitive processes, to be negotiable and may produce misunderstandings, that are a significant part in the “collaborative learning dynamics model” (Dillenbourg, 1999, p.10). Furthermore, the multiplicity of contexts and interactions also contribute to the learning setting. Strijbos and Fischer (2007) emphasize that collaborative learning is a multidisciplinary where several fields have been contributing with research, such as psychology, educational sciences, sociology, anthropology communication science and computer science. In terms of effects, Rovai (2004) considers that collaborative learning is successful when the group agrees on a product that translates the contribution of each member. This way, group work can contribute to the development of a collaborative and participative learning environment. Nevertheless, research (Swan et al., 2000; McConnell, 2006) has shown that this type of work may lead into more polarized decisions and heated discussions. In these situations, the instructor’s role becomes extremely important, and requires him to supervise and regulate the group work. In ODL, usually older, students are characterized by their high motivation to learn, fueled by their realization that learning will help them to better perform in their professional settings. What is more, according to Rovai (2004) their life experiences constitute a very good resource for collaborative learning Both sociocognitive and sociocultural approaches include the defining aspects of collaborative learning and place assessment in the core of the curricular project. This happens through the emphasis that they give to the role of assessment on reflexive and metacognitive practices, assuring the coherence of the decisions made throughout the development of a Course in general, and of a Curricular Unit in particular. Assessment as Communication Assessment always happens in a social setting where the social relationships between all the participants in the process – instructors and students – takes on a relevant role, and can be perceived as an act of communication, that is central to this system of social relationships (Barlow, 1992; Pinto & Santos, 2006). This means that the social environment is an integral part of this network of relationships and influences the ways the participants interact in particular when assessment is concerned Within the interactions between instructor and student, assessment takes a very important role, becoming the negotiation fundamental to clarify and validate the relationship between knowing and assessment. When viewed as a communication process, assessment takes on a specific intent and demands a sharing of the communication codes, particularly in the clarification of the assessment criteria. The participants’ performance on a given task is subject to the manner in which they understand the assessment criteria, or in other words, the instructor’s expectations (Pinto & Santos, 2006). Therefore, it is extremely important to be adamant on making the criteria explicit so that they can be properly understood and appropriated by the participants. In addition, McConnell (2006) refers that when the participants are actively engaged in decisions about what is learned, how it is learned, and also on decisions concerning the assessment criteria and the ways in which their work as well as that of their peers is judged, their relationship towards learning shows a different quality from when this is not done. We have been talking about assessment, and therefore, it is important to distinguish here formative from summative assessment, as they have different functions and goals. Formative assessment allows us to gauge the progress attained as far as objectives or competencies

are concerned, it provides instructors and students alike with information about what needs to be perfected and what is already known. Summative assessment performs a judgment about what a student has acquired and determines if he or she may carry on his work on a certain subject, or its possible placement on the job market. Elwood and Klenowski (2002), distinguish “assessment of learning”, as assessment focused on classifying and scaling, from “assessment for learning” as assessment meant for the students, through feedback, to understand their own learning processes and the goals that they intend to achieve. Still, both cases, formative and summative assessment, influence learning, meaning that the process of assessment provides the students with information about the type of learning that they are expected to show, as well as knowledge about what type of strategies to use to maximize their success. The students participation in their own assessment constitutes a very important part of their preparation for the real world (McConnell, 2006), and consequentially, Curricular Units should be designed in order to enable that involvement. We believe that self-assessment and peer-assessment make very important contribution towards that involvement. McConnell (2006) lists a series of advantages of this assessment processes when applied to e-groups and communities, for both the students and the instructor. As far as the instructor is concerned, he highlights the formative contribution from this type of assessment, the use of more student centered processes, the faster availability of feedback, and the facilitation of curricular change. For the student he points out, the reduced power of the instructor through the removal of certain barriers, providing an improved sense of authorship to the students of the assessment process, and their reduced dependence from the instructor. Besides, he refers that these assessment processes give the students an opportunity to assess their own work in a very similar way to what they will need to do in their futures professions and as a result helps them to develop enterprising competences. McConnell (2006) holistic approach to online Course design gives particular attention to some aspects that include: “-collaborative self-peer-tutor assessment processes; - collaborative evaluation of the experience of learning and teaching; - redesign of the Course based on the evaluation of student and tutor experiences” (p. 33). We support this way of framing online Course design since, as we have already presented, thinking online pedagogy implies considering an adequate process of data collection, but also a theoretical framework about learning that frames the interpretation of such data. This way we provide the regulatory function of learning with the settings and resources it needs to be successfully implemented (Pinto & Santos, 2006). There is a clear will in this process to accept and implement new ways of conceptualizing the instructional and learning processes. The objective is to design a Course where instructors and students are embedded in significant practices through collaborative and cooperative learning that assure the development of knowledge in the participants professional practice contexts. Therefore it is fundamental to create processes that assess the different types of learning that occur in each setting/context – processes of collaborative assessment. The proposed research questions are part of a bigger research project where these different types of collaborative assessment practices in online Courses are being investigated, based on the analysis of the online interactions on the discussion forums, the works produced by the students (both individually and in small groups), and answers to questionnaires applied in the different stages of the investigation. In this particular case we are interested in exploring questions related to group work in online ODL and its assessment. Two main research questions were asked: How do students perceive this way of assessing group work? How much do the students believe that the process of group work (assessed by them) should weigh when compared with the group’s final product (assessed by the instructors).

The Context and the Participants This work was developed in the context of a Curricular Unit called Using ICT for Learning and Assessment that is part of the degree plan for the Master in Elearning Pedagogies and the Master in Educational and Multimedia Communication at the Universidade Aberta. The Curricular Unit (class) where this study was conducted has been developed based on a Pedagogical Model for online learning developed by a group of researchers from the Universidade Aberta, Department of Education (Pereira et al., 2003). Based on this model, the adopted methodology is structured around two completely asynchronous and complementary work strategies. Firstly, it is based on the participants’ independent study and reflections of the presented documents. It requires the participants to read critically, to identify the main thesis defended by the authors, and elaborate their own opinion. Secondly, it requires that the participants work collaboratively with their peers, participating in online forums where they debate and (re)construct collectively their learning. The participants are expected to work in small groups and to find the best solutions for the problems and cases that they are confronted with. The goals of this Curricular Unit included the development of the participants’ metacognitive, argumentative and evaluative competencies. More specifically they were: • To be able to communicate conclusions, knowledge and reasoning clearly while being able to argument and substantiate their own point of view. • To critically reflect about the usage of technologies in education. • To creatively use technological artifacts with pedagogical goals. • To substantiate their decisions about the use of multimedia and/or educational communication tools in a variety of settings. The participants where assessed based on three different approaches: a) their participations on the online discussion forums; the group project; and a final paper. The group project was developed during the fourth, and last, stage of this Curricular Unit. The assessment of the group project was divided in two: 1) the process of elaboration of the project (1/3); and 2) the final project product (2/3). Since the Instructors responsible for this Curricular Unit recognized from their previous experience that in this type of activity the participants often develop a large majority of their work outside of the Virtual Classroom Software (VCS) and use other tools (such as MSN® or Skype®), it was decided to attribute the responsibility of assessing this component to the participants. The Instructors, however, developed a set of criteria that the participants should use to guide their self-assessment and that of their peers. These criteria included: • Commitment to the group project. • Relevance/pertinence of the research made. • Presentation of innovative elements to the project. • Contribution to the group dynamic. Once the group project had ended each group was responsible for sending the instructors a small document with the classification attributed to each participant accompanied by a written justification. The participants were 36 master students from the referred programs. From this sample, only 23 answered the questionnaire, 12 were male and 11 female. All of the participants were employed professionals, 15 were 5-12th grade teachers, 5 were professional development trainers, 3 were University instructors and 1 was a psychologist. The ages ranged from 30 to 55. 10 of the participants had previous experience in CSCL at different levels. Data collection and analysis

Data analysis involved iterative analysis and revision of the coding scheme (Miles & Hubberman, 1994). Two of the researchers derived the initial coding key from the ideas embedded in the data gathered from a questionnaire that 23 of the participating students completed, and from the online discussion forums used during the group project phase of the Curricular Unit. During the analysis the initial coding key was revised to account for emergent sub-codes and all of the data was recoded using a final coding scheme. Some of the codes were quantified in order to foster a more meaningful comparison of the data by allowing patterns to be identified and further explored (Chi, 1997). We then identified emerging themes in the coded data, and generated preliminary assertions for each research question based on the data. The third researcher tested the viability of the generated assertions by seeking both confirming and disconfirming evidence from the available data sources. The data was processed using the NVIVO7® qualitative data treatment software. Results From the produced analysis emerged 4 main themes that we now present from the most represented to the least: it’s perfect on paper; lack of thrust in their peers; reflective practices; and responsibility. It’s perfect on paper This theme illustrates the participants’ agreement with the suggested assessment process but difficulties in operationalizing it. They agree with the proposal that the elaboration of the project is assessed by the participants, based on criteria previously defined, while the instructor becomes responsible only for the final group product, also with previously defined criteria. What is more, the participants agreed with the suggested methodology were they had to participate individually in discussion forums, complete a group project and present a final paper. When asked if they endorse the proposed assessment process, almost all the participants (91%, all except 2) agreed with it “I agree with the proposed separation of final product assessment from the process of participation in the development of the project… It’s perfect on paper.” (Amalia), leaving the final product assessment as the instructor’s responsibility, while the assessment of the group work project elaboration stayed in the hands of the participants “The proposed process seems appropriate to this situation.” (Fatima). In spite of the overall agreement with the proposed assessment process, the participants point out some barriers to the implementation of this type of assessment. Firstly, they consider it dangerous to the social interaction between the group participants, in the words Rute (Professional Development (PD) trainer) “It’s always a very delicate question as it may raise relationship problems between the group participants, compromising the group work.” Also Lucas, teacher and trainer says “(the assessment) may be inflated or distorted by personal or relational reasons consequence of an insufficient knowledge about their peers.” Secondly, they point out their colleagues’ lack of courage and lack of justice to successfully perform their peers’ assessment “…as long as the participants had the courage and the sense of justice that we often lack”. (Amalia). Finally, even though the participants agree with the proposed assessment process their previous experiences, in other settings, makes them feel more comfortable transferring to the instructor the responsibility for identifying and penalizing the participants that do not fully contribute to the group work process “The instructor … should question the group and assess the participants’ individual contributions to the group work.” (Carla, teacher), “The instructor should be informed whenever one of the participants is not contributing” (Rute, PD trainer)

Lack of thrust in their peers

In this theme are included some of the reasons that make the participants to refrain from trusting their peers ability to perform an impartial assessment. Thus, the second most emerging theme (65% of the participants) was the lack of confidence that the participants demonstrated on their peers ability to perform the self and peer-assessments. Several reasons emerged for this situation including: the participants’ lack of experience in this type of assessment “Without experience in this type of assessment, it is a very difficult thing to do.” (Ricardo, University Instructor). Besides the lack of experience in this type of assessment, and consequentially difficult to implement, other also refer the awkwardness of having to assess their peers: “It’s not something that the students feel comfortable doing, therefore, unless there is a situation where someone is clearly not participating the tendency is to return to the instructors the same classification for all group members…” (Joaquim, teacher) While others reinforce the idea that this is not a competitive setting “as we are not here to compete, and usually empathize with our colleagues the most common result is to only say good things about them, except in very flagrant situations.” (Victor, teacher). Others show a more emotional response showing a complete distrust for their peers reasons for the classifications that they have attributed “I do not believe in my colleagues peerassessment results” (Matilde, teacher), “Some colleagues do not fully participate in the group work process and are not able to be objective on their assessment, shifting the group participants’ classifications.” (Carolina, teacher). All these reasons are usually accompanied by a preference for shifting this responsibility to the instructor “The solution may be to warrant that all of the group work is done in such a way that is visible and can be monitored by the instructor. This way the assessment would be more fair.” (Marta, teacher), “It would be better if each group always worked in a visible space where they were forced to present all the work process, so that the instructors could accompany the group progress.” (Catarina, teacher). Reflective practices Here we present the participants reflection on the benefits that they have identified for their practice, improving their reflective practices and critical thinking. Almost half of the participants (43%) have recognized the benefit of this assessment process to their reflective practices. The results seem to indicate that the participants’ reflective practices about the work that they have performed and their own process of learning constitute a fundamental piece of online learning and assessment settings. The fact that they have to write about a certain theme and answer questions posed by their peers makes them ponder and reassess their initial positions before posting their own answers. This is illustrated when Maria José (teacher) says: “It allows us to ponder on what we did, on what we didn’t do, and on what we could have done.” They distinguish this assessment process as responsible for provoking them to reflect about their own participation in the group work as well as that of their peers “I consider this type of assessment very important as it prompts us into reflecting about our own performance and that of our colleagues.” (Catarina, teacher) or when Carolina (teacher) says: “it makes be more conscientious about our learning process.” Related to the emphasis given by the participants to their reflective practices and selfassessment procedures emerges a related dimension associated with the development of a critical thinking attitude about their own work and participation, as well as that of their peers, while allowing them to rethink their way of engaging in group work. “It allows us to develop a critical thinking perspective about our work” (Pedro, teacher), or when Rafael, a teacher that also has executive functions in his schools reflects:

“I consider it very important as it forces us to be reflective about our own work, stimulates a critical attitude that surpasses the natural complicity between colleagues.” However, there is another aspects valued by the participants comments in this dimension, the contribution to social interaction and relationships. They recognize in this practice an advantage for their social interaction with their colleagues as Pedro (teacher) says, “it favors our inter-personal relationships.” Establishing good working social interactions/relationships may benefit collaborative learning, or in other words, when they exist, participants may feel more willing and confident to contribute and support each other. Finally, it is something that they consider important to translate into their professional setting, “If I don’t accept to be assessed how can I assess my own students?” (Marta, teacher). In spite of all the identified positive aspects, the results also indicate that this is a task that the participants consider difficult to implement, but to which they recognize merit, as Marta (teacher) puts it “it is a difficult thing to do, however, the contribution of my peers can help me introduce corrections in my way of working.”, or in the words of Carolina (teacher) “When we are asked to complete our self and peer-assessments in a group work, it is not an easy task to achieve, but it carries positive effects”. These comments show how much metacognitive and reflective practices can be important for learning and assessment in online VCS settings. Moreover they alert us to the difficulties associated with their implementation, in particular in the cases of peer-assessment. The aptitude to communicate and share knowledge (emotional, theoretical or procedural) requires a considerate effort and reflective ability. Responsibility In this theme we explore the participants’ awareness that competence and responsibility are cornerstone to this assessment practices. The theme responsibility emerged on 35% of the participants that seem to accept self and peer-assessment so long as the participants assume responsibility for their learning “The participants aren’t children, they’re adults and responsible persons, and as such, they are self conscious of the work that they have done and should be able to self assess themselves and their colleagues.” (Carla, teacher) However, they recognize that it is not a common practice in their culture to make public this type of assessment. It is a judgment that they are used to make informally, but not publicly, as Susana, PD trainer says: “What was asked isn’t new. We always perform that “judgment” after a group work, we just aren’t used to having to verbalize it and take on responsibility for it. In this type of learning process it is important that we start taking on that responsibility maturely.” The participants also consider this as an important process to make accountable for their lack of participation those colleagues that did not contribute to the group work, and the instructor is not aware of it, as Sandra (teacher) puts it, “It’s important, as sometimes the final product is hurt as a consequence of the lack of cooperation of some colleagues, and consequently if just the product is assessed that assessment is often unfair.” The final group work product is seen by most of the participants as their main task. Their contribution is usually enthusiastic and dynamic, implying the assumption of responsibility to complete their part in the project. They also contribute to the analysis and assessment of the required data and resources necessary for the project development. Likewise, they are receptive to their peers’ comments about their work, and also to participate in the assessment of the work done by them. All this implies that they must be ready to take risks

and assume responsibility for their work. However, the results show us that some of the participants in this study still need to develop their practices in this area. Discussion In this research we drove to study processes of collaborative assessment, used in the context of a Curricular Unit from an online Master Program, and how they were perceived by the participants. An overall analysis of the presented results gives us an indication that even though the participants have incorporated the concept of “collaborative learning”, in the production of instructional designs that incorporate collaborative work, they still reveal some distrust towards the assessment processes associated with group work, in particular in the case of peer assessment. The relationship between the main themes identified in this research is illustrated in Figure 1. Clearly the most prevalent conclusion must be that the participants support this form of assessment as is clear on theme “It’s Perfect on Paper”. Furthermore, they identify as benefits its promotion of reflective practices and on the participants’ responsibility. Nevertheless, they recognize difficulty in the operationalization of this process in the theme “Lack of trust in their peers”. Figure 1: It’s Perfect on Paper Lack of thrust in their peers

It’s It’s Perfect Perfect on on Paper Paper

Reflective practices

promotes

Responsibility

Remembering our initial research questions, we wanted to know: How do students perceive this way of assessing group work? How much do the students believe that the process of group work (assessed by them) should weigh when compared with the group’s final product (assessed by the instructors). Let us start by addressing the second research question. In general we have observed that the large majority of the participants agreed with the suggested process for assessing the group project. The instructors are responsible for the assessment of the final product that accounts for 2/3 of the final project grade, while the participants are responsible for the assessment of the group work process that accounts for 1/3 of the project grade. This is clearly visible in the description of the finding entitled “It’s Perfect on Paper” supported by 91% of the participants. The participants are able to identify the advantages of this practice for their own assessment and for their professional life, in areas that go from the development of social and relational competencies to the improvement of their argumentative skills.

However, it is precisely on this identified benefit to their social competences that they start raising some issues concerning the operationalization of this process. The participants seem to consider that because they have to play the role of evaluators and assess their group peers contribution to the project, this will hinder their social interaction. Moreover, they do not recognize on their colleagues the necessary sense of justice and courage to make a fair and unbiased judgment of themselves and their peers. To obviate this problem they suggest that the instructor should assume a regulator role and supervise this process. This is probably a reflection of these participants past unidirectional educational and professional experience, where there is one clear responsible for the evaluation and classification process. Only recently have we started to notice the appearance of assessment processes where the participant’s contribution is expected and valued. We make the argument here for the importance of collaboration in online ODL and assessment processes to go with it, as “what is assessed in a Course or a program is what is valued” (Swan et al., 2006, p.45). Even the definition of what it is to work collaboratively (Dillenbourg, 1999) is recent and still being developed and discussed. Therefore, it is natural that the assessment practices associated to this type of learning are also recent and are still going through the process of being appropriated and understood. In the described context, it is understandable why the finding about the lack of trust on their peers is so prevalent. The participants have little or no experience in this type of assessment, they do not feel comfortable in the role of evaluator, and consequentially, distrust their peers to attain that role too. As one might expect in such a case, they prefer that the instructor supervises the whole process and becomes the main responsible for the assessment. It is still very hard for these participants to accept an instructor that positions himself in a more horizontal role with respect to them, and they demand from him to position himself more vertically as an arbiter of knowledge (McConnel, 2006), and to clearly assume the function of regulator of the participants’ contributions and assessment. To answer our other research question: How do students perceive this way of assessing group work? We will now comment on the remaining two themes that emerged form our research: reflective practices and responsibility. The results from this research show that both self and peer-assessment require the participants to exercise responsibility for their own learning. This agrees with the previous work of Vonderwell et al. (2007). In fact, these authors emphasize that “student self-determination and responsibility influenced the degree that the students took advantage of the asynchronous nature of online discussions” (p. 320). However, this is not something that the participants are used to have to be accounted for, even though they recognize that is something that they usually do informally, but with no consequences. Also in their professional experiences they did not have the chance to experience formal situations where they are required to assess their peers. To facilitate what is clearly a new process for the participants, it is very important to clearly explicit the assessment criteria to be used, to make the process as fair, transparent and less biased as possible. The chance to participate and collaborate in the group projects gives the participants the opportunity to confront their own points of view and competencies with those of their peers. One of the characteristics of collaborative learning according to Dillenbourg (1999) is exactly this possibility that the different types of possible interactions have to influence the participants’ cognitive processes, giving place to new misunderstandings that will in turn model and enable the “dynamics of collaborative learning” (p.10). The results indicate us that the participants recognize the importance of group work but are concerned about the damaged provoked by those who do not cooperate and contribute to the final product, hurting its success. For this reason, this type of assessment process allows them to re-establishes some measure of justice distinguishing the contributions of each of the peers for the final product. Moreover, the results also indicate us that the participants recognize in this way of assessing group work the advantage of re-establishing justice on the

process for all those situations where the instructor is not able to supervise the contributions of the participants. In this research all the participants have pointed out the importance of this assessment process to promote their own reflective practices about their work and that of their peers. The awareness of what they have learned, and how they learned it, as well as the monitorization and self-assessment are also present in others research (Vonderwell et al., 2007).Furthermore, some of the participants go one step further and recognize the relevance of this process to their social interaction and its influence on the development of a critical thinking attitude towards their participations and the work that they have done. Some participants will only post their responses after carefully analyzing those of their peers, in order to compare and contrast their points of view and their learning, identifying strengths and weaknesses and assessing their colleagues responses as well as the instructor’s. The relationship between these assessment practices and their professional practice is also highlighted by several participants as significant, for allowing them to develop assessment competencies that they recognize as important and valuable for their own practice McConnel (2006) emphasizes that self and peer-assessment processes allow participants to assess “their own work in ways that are applicable to their future professions …. (and) help students develop enterprising competences” (p.91). However, given the great diversity of backgrounds amongst the participants and their very different professional responsibilities, it is clear that only a small minority was prepared to implement and operationalize this assessment practices. Even so, the experience that they have gone through with these new types of assessment has lead several of the participants to be now in, what we can call, a transition state where they are more capable of engaging in such practices. Recommendations for practice and research In the context of online collaborative learning, both self-assessment and peer-assessment constitute forms of assessment that should be implemented, and therefore requiring appropriate types of activities to do so. The significance that the participants attribute to these activities will be reflected on the way that they participate, conceptualize and discuss the questions that will be raised. This process contributes to the participants’ inner dialogue and self-reflection, and also to the dialogue with the other participants in the community, in every learning environment. To design online learning environments, where formative and summative assessment practices are present, requires the clear definition and discussion of the assessment criteria, to promote the participants appropriation of the concepts and ideas that we want them to learn. In this context, metacognition processes are essential as “student awareness of these metacognitive processes can enhance assessment practices” (Vonderwell & Alderman, 2007, p. 324). In such cases, we suggest the creation of small reflection activities and/or summaries in the sequence of the online discussions. If we want to promote the participants motivation and engagement, it is desirable to make use of a range of methods and assessment strategies. Conversely, the defined assessment criteria should reflect the desired forms of participation, helping the participants to identify and understand what is valued. In this research, the participants have initiated a process of appropriation of ideas about learning and instruction, engaged in discussions centered on selected themes, and completed individual and group activities, related to their professional practices, framed by a set of assessment criteria for each particular activity. This research leads us to believe that the participants would benefit if the instructor shared with them a set of “exemplary practices”

about working in small groups, with the goal of facilitating their acquisition of metacognitive and self-regulatory competencies, to go in hand with a more regulatory stance form the instructor. The participants demand from the instructor a more strongly established “regulator” role. In fact, we have observed that the processes necessary to the development of collaborative assessment competencies, in particular self-assessment and peerassessment, requires a supportive learning environment (McConnell, 2006). It is necessary to develop a sense of thrust between the instructor and the participants, and between the participants themselves, otherwise there will always be a subconscious climate of distrust with inevitable consequences in the assessment process. For the participants to engage and trust in the collaborative learning process and on the proposed assessment procedures by their peers, it is fundamental for them to believe that they are working with a common goal and that the assessment is taken seriously by all. Given the participants very diverse backgrounds it is quite important to promote the discussion and clarification of the proposed assessment criteria, and even prolong the time frame for this discussion if necessary. Throughout this process, besides being necessary to develop and maintain a shared authorship relationship between instructor and participants, it is necessary to guaranty the creation of settings and activities that will promote in turn the development of the participants own self-assessment criteria. It will be necessary to conduct a more specific research project in order to understand how some of the groups are able to achieve in their discussions, group project, and peer assessments, high levels of quality; while others act with distrust and, consequentially, demand a more instructor centered environment while calling for more feedback. What are the characteristics that identify such groups? This study suggests several areas of possible further research. Many questions remain unanswered and new ones have emerged. What types of constraints prevent some participants from fully engagement on the group work? How can we change the negative perception that some of the participants show towards peer-assessment? How can we make the participants more autonomous and responsible (in terms of assessment)? Which methods/strategies/processes are best suited to assess online discussion collaborative learning? References Barlow, M. (1992). L´evaluation scolaire, décoder son language. Paris: Choronique Social. Bates, T. & Poole, G (2003) Effective Teaching with Technology in Higher Education: Foundations for Success San Francisco: Jossey-Bass. Chi, M. (1997). Quantifying qualitative analysis of verbal data: A practical guide. The Journal of the Learning Sciences, 6 (3), 271-315. Comeaux, P. (2005). Assessment and learning. In P. Comeaux (Ed.), Assessing Online Learning, (pp. xix-xxvii). Bolton, MA: Anker Publishing Compan, Inc. Dillenbourg, P. (1999). What do you mean by “collaborative learning”? In P. Dillenbourg (Ed.), Collaborative learning: Cognitive and Computational approaches (pp. 1-19). Oxford: Elsevier. Elwood, J., & Klenowski, V. (2002). Creating communities of shared practice: The challenges of assessment use in learning and teaching. Assessment & Evaluation in Higher Education, 27(3), 243-256. Garrison, R & Anderson, T. (2003). E-Learning in the 21st Century: A framework for research and practice. Routledge. Jonassen, D.H. (2005). Modeling with technology: mindtools for conceptual change. Columbus, OH: Merrill/Prentice-Hall. Lave, J., & Wenger, E. (1991). Situated learning: Legitimate peripheral participation. Cambridge: Cambridge University Press. MEDEIA (2007). http://www.medeia.org/ McConnell, D. (2006). E-Learning Groups and Communities. Berkshire: Open University Press. Miles, M. & Hubberman, A.M. (1994). Qualitative data analysis. Thousand Oaks, CA: Sage.

Pereira, A.; Mendes, A. Q.; Mota, J. C.; Morgado, L. & Aires, L.L. (2003). Discursos, Série Perspectivas em Educação, nº1, pp. 39-53. Pereira, A., Mendes, A. Q., Morgado, L. Amante, L. & Bidarra, A. (In Press). Modelo Pedagógico Virtual da Universidade Aberta. Lisboa: Universidade Aberta. Perret-Clermont, A.-N., Perret, J.-F. & Bell N. (1991). The social construction of meaning and cognitive activity in elementary school children. In L. Resnick, J. M. Levine, & S. D. Teasley (Eds.), Perspectives on socially shared cognition (pp. 41-62). Washington, DC: American Psychological Association. Pinto, J. & Santos, L. (2006). Modelos de avaliação pedagógica. Lisboa: Universidade Aberta. Rovai, A. P. 82004). A constructivist approach to online college learning. Internet and Higher Education, 7, 79-93 Strijbos, J.-W. & Fischer, F. (2007). Learning and Instruction, 14 (4), 389-393. Swan, K., Shen, J., & Hiltz, S. R. (2006). Assessment and Collaboration in Online Learning. Journal Asynchronous Learnin Networks,10(1), 45-62. Swan, K., Shen, J., Fredericksen, E., Pickett, A., Pelz, W. & Maher, G. (2000). Building knowledge building communities: Consistency, contact and communication in the virtual classroom. Journal of Educational Computing Research, 23(4), 389-413. Vonderwell, S., Liang, X.,& Alderman, K. (2007). Asynchronous Discussions and Assessment in Online Learning. Journal of Research on Technology in Education, 39(3), 309-328. Vygotsky, L. S. (1978). Mind in society: The development of Higher Psychology Processes. Cambridge, MA: Harvard University Press. [original publicado em russo em 1932] Wertsch, J. (1991). A sociocultural approach to socially shared cognition. In L. Resnick, J. M. Levine, & S. D. Teasley (Eds.), Perspectives on socially shared cognition (pp. 85-100). Washington, DC: American Psychological Association.