Peer Assessment

Use Case for Peer Assessment developed by Steve Jeyes and Robert Sherratt at Hull. The component use cases of Peer Assessment are as follows:

Author Assessment Players: Peer Assessment Author, Students, Group, Peer Assessor, Assessment Analyst Descriptions: 1. Tutor authors assessment 2. Tutor defines key criteria for assessment 3. Tutor defines structure/weighting for group and tutor marking 4. Tutor decides how group and peer marks will produce individual marks 5. Tutor schedules key events for tracking 6. Tutor notifies students of the above

Evidence:

Hull, 1. Assessment-XML structured document of requirements inc. boxes you would want to include etc. inc. additional student feedback via a textbox: form authoring service 2. Allocate forms to courses and groups within courses.

Loughborough, 1. Form of Criteria authored and relative mark / scale given for each

SPARK ( UTS University of Technology Sydney), 1. Academic identifies weighting for group assessment task and key assessment criteria for marking the group submission 2. Academic explains how group marks are adjusted into individual marks using SPARK and method for formulating groups 3. Academic develops assessment* criteria and rating scale** to be used calculating the adjustment factor, in collaboration with students and after considering past student feedback

Group Players: Peer Assessment Author, Students, Group, Peer Assessor, Assessment Analyst Descriptions: 1. Obtain student cohort from SIS 2. Check for existing group data 3. Decide on group allocation method : random, by ability etc 4. Select group membership 5. Store group membership

Evidence:

Hull, 1. Group selection 2. Import users - Ws interface TO SRS/SIS in the future 3. Group creation - link to SRS/SIS future

Loughborough, Linked to Student record System - updated every 24 hours - isolated into a separate PHP class - downloads modules and students - internal copy then used for WebPA system (ENTERPRISE) Lecturers like to control selection and creation of groups. Groups are though held externally so could be reused for other work / modules / systems etc (external group service ENTERPRISE)

SPARK ( UTS University of Technology Sydney), 1. Academic explains how group marks are adjusted into individual marks using SPARK and method for formulating groups 2. Academic communicates group project mark to each group

Mark Assessment Players: Marker Descriptions: 1. Marker compares candidate response with the marking scheme 2. Marker assigns marks to candidate based on the marking scheme

Moderate Assessment Players: Moderator Descriptions: 1. Moderator views candidate results: 2. Moderator views analysis of group marks -check distribution for skew, high or low median 3. Moderator views analysis of test behaviour -check standard deviation for skew, high or low value -check standard deviation for skew, high or low value 4. Moderator validates results -check if any question has abnormal marks (doesn't follow standard distribution) 5. Moderator adjudicate cases for special consideration ( decide mark adjustment 6. Moderator approves assessment marks

Notify During Assessment Players: Peer Assessment Author, Students, Group, Peer Assessor, Assessment Analyst Descriptions: 1. Tutor reminds students who have not done peer marking to complete it 2. Tutor declares peer marking closed

Evidence:

Hull, Original driver was to save time in marking, first iteration of the model works, but not very well, still taking a long time to use - Students not submitting peer marks needs chasing 1. Notification - individual and or group email 2. Email notification inc. link to the form inc. ability to send out again - notification service 3.Marks published via Blackboard - integration.

Loughborough, One of key reasons for using online would be good monitor/ notification/ evidence system i.e. have they peer marked, if not notify, if not re-notify if not tell them closing and keep records, selectively email those not completing - currently blanket emails all even those that have completed Currently 4 states Pending - either being created or scheduled to run in the future Open - currently being peer assessed Closed - closed for further peer assessment Marked - marked and moderated

Notify Post Assessment Players: Peer Assessment Author, Students, Group, Peer Assessor, Assessment Analyst Descriptions: Tutor sends Group mark to each member of group Tutor provides group feedback Tutor provides individual feedback Individual marks published

Evidence:

Hull, Original driver was to save time in marking, first iteration of the model works, but not very well, still taking a long time to use - Students not submitting peer marks needs chasing 1. Notification - individual and or group email 2. Email notification inc. link to the form inc. ability to send out again - notification service 3.Marks published via Blackboard - integration.

Loughborough, One of key reasons for using online would be good monitor/ notification/ evidence system i.e. have they peer marked, if not notify, if not re-notify if not tell them closing and keep records, selectively email those not completing - currently blanket emails all even those that have completed Currently 4 states Pending - either being created or scheduled to run in the future Open - currently being peer assessed Closed - closed for further peer assessment Marked - marked and moderated

Notify Pre Assessment Players: Peer Assessment Author, Students, Group, Peer Assessor, Assessment Analyst Descriptions: Tutor explains how group marks are adjusted into individual marks Tutor describes methods for formulating groups Tutor declares peer marking started

Evidence:

Hull, Original driver was to save time in marking, first iteration of the model works, but not very well, still taking a long time to use - Students not submitting peer marks needs chasing 1. Notification - individual and or group email 2. Email notification inc. link to the form inc. ability to send out again - notification service 3.Marks published via Blackboard - integration. Loughborough, One of key reasons for using online would be good monitor/ notification/ evidence system i.e. have they peer marked, if not notify, if not re-notify if not tell them closing and keep records, selectively email those not completing - currently blanket emails all even those that have completed Currently 4 states Pending - either being created or scheduled to run in the future Open - currently being peer assessed Closed - closed for further peer assessment Marked - marked and moderated

Notify Players: Timetabler, Tutor Descriptions: 1. Timetabler notifies assigned candidates that an assessment has been scheduled; time, date, location, instructions for attendance, prerequisites 2. Tutor notifies assigned candidates of the grade that has been transferred to the candidates gradebook

Reflect Players: Tutor, Student Descriptions: 1. Tutor reflects on if group learning and assessment appropriate 2. Tutor reflects on previous performance 3. Tutor designs support to best ensure successful learning and assessment 4. Tutor collects student feedback 5. Tutor analyzes student feedback 6. Tutor researches behaviours and possible improvements 7. Tutor evaluates effectiveness 8. Tutor reflects on this year's performance

Run Assessment Players: Peer Assessment Author, Students, Group, Peer Assessor, Assessment Analyst Descriptions: Provide opportunity for group to assess the performance of their peers within a specified a schedule. Tutor decides to allow or disallow anonymous access Tutor monitors performance of groups via tracking

Evidence:

Hull, 1.One shot opportunity to fill in the form could also include self-assessment - form rendering service - data storage service 2. Allow/ disallow anonymous views of data –Paul disables this, experience shoved they knew who had given bad marks to. Loughborough, 1. Students assess each other against given criteria i.e. Open - currently being peer assessed

SPARK ( UTS University of Technology Sydney), 1. Students, conscious of assessment criteria used in project/task and groupwork, work on group task culminating in submission of group project/task 2. Academic monitors groups during completion of group project/task 3. Students reflect on group process in completing task and rate all members of own team including self against agreed criteria within a rating period Schedule Players: Timetabler Descriptions: 1. Timetabler defines events / states 2. Timetabler allocates dates / timescales to events / states 3. Timetabler defines tracking required

Track Players: SIS Descriptions: 1. SIS logs that assessment has been assigned to candidate 2. SIS logs marks achieved by candidate 3. SIS logs assessment has been moderated 4. SIS logs assessment grades have been awarded

This usecase is composed of the following usecases:

Related items
This usecase is composed of the following usecases:
 * is composed of::Author Assessment (Peer Assessment)
 * is composed of::Group (Author Assessment)
 * is composed of::Mark Assessment (Peer Assessment)
 * is composed of::Moderate Assessment (Peer Assessment)
 * is composed of::Notify (Author Assessment)
 * is composed of::Notify During Assessment
 * is composed of::Notify Post Assessment
 * is composed of::Notify Pre Assessment
 * is composed of::Reflect (Peer Assessment)
 * is composed of::Run Assessment
 * is composed of::Schedule (Author Assessment)
 * is composed of::Track (Run Assessment)