Designing Peer Formative Assessment with Open Badges in Online Learning Environments
Stylianos Hatzipanagos, Kings College London & Steven Warburton, University of Surrey
ContextA project that explored the effectiveness of peer feedback and assessment; Investigated whether open badges can support learning in online environments through enhanced engagement; Revisited and evaluated badge typologies with the aim of identifying the most appropriate typology that supports peer feedback and assessment; And created a clear correspondence between the typology of badges and peer feedback and assessment practice in online classrooms.
RationaleUse of digital badges to support peer feedback and assessment in higher education
By the use of open badges in encouraging and rewarding competences, knowledge and skills acquired in digital learning spaces, we aimed to reinforce the value of peer assessment and by extension peer feedback and review activities in supporting student learning.
Peer Assessment DefinitionPeer assessment is to be understood as an educational arrangement in which students assess the quality of their fellow students work and provide one another with feedback.Dochy et al., 1999
Formative e-assessment can:
engage learners with tasks that enhance learning outcomes and test the development of higher order capabilities;enrich learning approaches by making feedback central to all assessment activities;promote a dialogue in relation to feedback, peer and self assessment activities.Hatzipanagos & Warburton 2009
Peer assessment/peer ReviewPeer assessment is frequently underused in online learning environments. Its informal learning value is not exploited by peers as it is not explicitly linked to summative assessment activities. There is a reluctance to participate in such peer review activities and an ambivalence of tutors in relation to their value. Informal aspect is not clearly or meaningful embedded in the learning process.
Peer assessment attributes (Dochy et al)What the research saysBadgesValidityPeers are prone to produce ratings based on uniformity, race and friendship if there is no extensive training in peer rating.Majority of studies showed an acceptably high validity and reliability in a varietyof fields.Clear criteria embedded in badges definition. Validity can be high if badges are well defined. Fairness.Student perceptions of fairness of peer assessmentStudents were very positive about the effects of alternative assessment on their learning.Openness and clarity in microcertification fundamental requirements of a fair and valid assessmentsystem. Accuracy.Agreement between student and tutors. No consistent results.There can be objective approaches of testing accuracy in microcertificationEffects.Peer assessment fostered an appreciation for the individuals' performance within the group and interpersonal relationships in the classroomBadges can capture interpersonal relationships (soft skills) and performance (hard skills)Implications for practice.valuable as a formative assessment method butfriendship marking; collusive marking; decibelmarking; and parasite marking..Appropriateness of emergent technologies such as badges for formative assessment Assessing mishaps can be prevented by learning analytics and digital trail in new environments
MethodologyThe Open Badges used in Kings College London and the University of Surrey in the UK in two distinct programmes. Kings programme (Postgraduate Certificate in Academic Practice) targets novice lecturers, PhD students with a teaching role, librarians and information specialists who have chosen to study a Supporting technology enhanced learning module. Peer Assessment is an integral part of these modules and learners are strongly encouraged to participate in formative assessment activities. There was an opportunity to link performance and peer review in these activities with open badges.
MetricsAssigning badges was a semi-automated process. Use of the learning environment triggered automated assignment of badges for metrics associated with use of the learning environment e.g. in the context of computer mediated communication, learning behaviours were monitored and examined when students used discussion fora and the social media communication. Metrics such as number of posts to a discussion forum, number of log-ins, response to other posts, sharing content and reflections were used.Other badges were deployed by application and evidence of achievement, moderated either by the module tutors, the project team or by peers on the course. A protocol for awarding a badge was developed by the project team in collaboration with the module tutors and learners e.g. we specified the group of users who could approve or reject badge awards. Evaluation of the project activities was carried out by collecting data from the online learning environment, using focus group interviews and by end of module and programme evaluation questionnaires.
ProjectionsBy rewarding positive behaviours the participants would feel more motivated to complete learning activities. The aim of this approach was to manage any motivational drops that can commonly be experienced in online/blended courses.Participants were encouraged to push their practice beyond the central learning design journey and engage in the more challenging aspects of the activities provided. For example, in terms of their engagement with a wider community, and development of their identity and responsibilities within, across and beyond that community.The project would provide a mechanism for self-reflection by the academics engaged in the study. They would be able to evaluate the benefit of using a badging system within their own course designs.
Concrete EXAMPLESFor example, the ability to award a badge to your peer evaluator if they provide good constructive advice on your work.
Digital Badges evaluation
Q2: How do badges compare with other ways of demonstrating your achievements?
Q5: Do you think that badges motivate learners in an online environment?
Q14: Do you see badges becoming part of your digital academic identity?
Perceptions of valueRandom orchestrations between learners that are not part of the same community. Due to the limited time I spent with my fellow students I didn't get to know them well. Therefore I felt that the use of badges was not particularly usefulSoft indicator of achievement, used for formative rather than summative purposes. in lieu of marking or registers they could be used as a 'soft' indicator of achievement (e.g. a record of optional on-line courses taken)Overall caution against the use of badges don't think the possibility of winning a digital badge really changes adult behaviourunless digital badges are established to international- or the very least national standards as a well recognized metric of achievement I wouldn't think they will affect online learning.Student centred approach should worksomething that the students themselves defined and awarded, rather than it coming from me (also a good way to gauge what they think of the whole thing!)
What do you think would help the adoption of badges?
I think a clear rationale for the benefits to both the student and staffDefining their purpose more clearly, and applying them consistently to something that is meaningful to students' learning and achievements. Instinctively, it's quite hard to take them seriously. Is it the name? or the varied application? Not sure.If someone badges feed into the assessment process -i.e. maybe 5 extra marks could be added to the summative assessment to recognize participation and contribution.If they were linked to module creditAs the criteria for digital badges are still very loose I don't think it is likely that they are used in evaluating academic contribution. Stricter and well defined metrics are in place for academic professional development (at least in Life Sciences).I might be wrong about this - and it's a bit of a chicken and egg situation - but I dont see them as an important part of a professional identity, and I gather from friend on corporate e industry (e.g. cloud computing) that they are not taken seriously there either. If I felt that they were something that employment panels cared about and that validated my use of e-learning tools, then I'd swallow my scepticism, adopt them enthusiastically and show off about the ones I have earned. Of course, if none of us decide to adopt that approach then the panels won't care, and so on around the circle... it's just really difficult for me not to see them as something rather childish and believe that my students would feel the same.I think there's potential use for badges as part of the summative assessment (which would then require further guidelines and transparency).Academia is already a game measured by the 'badges' of qualifications, publications, citations and research grants -- further gamification seems unnecessary
What we would like to see in the future Integration of badges with university systems; value of badges for summative assessment; promotion; recruitmentBetter integration of badges with external certification systems (Mozilla open badges, Open Badge Factory) and these with social mediaClear distinction between hard and soft transversal skills; but both essential in peer assessment activitiesEmbed micro-portfolios of badges in PDP (personal development planning) and portfolio building for staff and students.
RefereNcesDochy , F. , Segers, M. & Sluijsmans, D. (1999) The use of self-, peer and co-assessment in higher education: A review, Studies in Higher Education, 24:3, 331-350, DOI:10.1080/03075079912331379935.Hatzipanagos, S. & Warburton, S. (Eds) (2009). Social Software and Developing Community Ontologies. 598 pp. London: Information Science Reference, an imprint of IGI Global ISBN 978-1-60566-208-4.
If you have questions or FeedbackEmail email@example.com