Managing Software Development (COMP3120)

Peer Assessment

A common complaint about assessed group work is that each student receives the same mark, regardless of individual contribution.

To help overcome this problem, we use peer assessment as a method of enabling each team member to recongise individual contributions to the team's overall output and who better to know who has contributed what than members of the team. Thus each student receives an adjusted mark.

However, the we also use peer assessment to help us achieve one of the stated learning outcomes of COMP3120, that is that students will "further develop their collaborative and team work skills".

To ensure that this outcome is met we need to assess group process -- how successfully the group worked together and how well they managed their work -- as well as the group product -- the quality of the formal assessment items. As part of the requirement to develop skills in reflection and assessment, students also assess their own as well as the contributions of their team members.

We use both self and peer assessmnt. Self assessment is about learning to make judgements about the standard of one's own work and peer assessment is about judging the standard of other students' work.

Research has shown that there are many benefits to students when self and peer assessment are used as part of an academic course. These benefits include the development of generic skills such as improving teamwork and developing communication, interpersonal and organisation skills.

Peer Assessment Algorithm

Although essentially the algorithm used by WebPA-x is that devised and used in WebPA there are a few core differences between them.

The algorithm used to calculate peer assessed ratings works on the concept of each student being given a pie which they divide up among their peers according to their assessment of the value of their peers' contributions. Although we collect self assessments, the ratings that a student gives themself is not used when calculating the WebPA-x peer rating. To overcome the issues surrounding one student deciding to award a maximum rating of 5 and another student awarding a maximum rating of 3, ratings are moderated prior to calculation of the WebPA-x peer rating. Should a team member fail to submit a peer assessment then the algorithm treats it as if they rated everyone as performing equally.

Worked Example

Assessment

Each student shall submit Peer Assessment using the CECS implementation of WebPA-x within 5 days of each item of assessment being due, i.e. four times during the course.

A Student User Guide is available.

Peer assessment MUST be done indivdually and there must be NO collusion between students.

Peer assessments are mandatory, but they do not attract any marks. Late or incorrect submission will be penalised by 2% of total marks. Students who do not submit the peer assessment will be awareded 50% of their assessed mark.

WebPA-x

WebPA-X works with Firefox, Chrome or IE9. However, while it is likely that WebPA-X works with other browsers we cannot guarantee that it does, and we are aware of errors when using it with IE7 (the version of IE on the PCs at uni - however all PCs at uni also have Firefox).

Web-PaX has been set up on a College server, is subject to usual College security and is maintained and administered by College IT staff.

WebPA-X Assessment Reports

Students are encouraged to discuss their individual WebPA-X assessment reports with either their tutor or the course co-ordinator.

Important Note

Peer assessment is confidential, that is, individual student ratings are available only to the course co-ordinator. Tutors will have access to Peer Assessment Reports which they may discuss with individual students in an effort to help those students improve their performance.