COMP3100/COMP3500/COMP4500 - SE_Projects

Peer Assessment

A common complaint about assessed group work is that each student receives the same mark, regardless of individual contribution.

To help overcome this problem, we use peer assessment as a method of enabling each team member to recongise individual contributions to the team's overall output and who better to know who has contributed what than members of the team. Thus each student receives an adjusted mark.

However, the we also use peer assessment to help us achieve one of the stated learning outcomes of the SE_projects, that is that students will learn to

• "work as an effective member of a team to implement a software based solution that delivers measurable value to an industry or university client" (COMP3100/COMP3500)
• "work as an effective team manager of a small team of peers to implement a software based solution that delivers measurable value to an industry or university client" (COMP4500).

To ensure that this outcome is met we need to assess group process -- how successfully the group worked together and how well they managed their work -- as well as the group product -- the value delivered to the client. As part of the requirement to develop skills in reflection and assessment, we also get students to assess both their own and the contributions of their team members.

We use both self and peer assessmnt. Self assessment is about learning to make judgements about the standard of one's own work and peer assessment is about judging the standard of other students' work.

Research has shown that there are many benefits to students when self and peer assessment are used as part of an academic course. These benefits include the development of generic skills such as improving teamwork and developing communication, interpersonal and organisation skills.

Peer Assessment Algorithm

Although essentially the algorithm used by WebPA-x is that devised and used in WebPA there are a few core differences between them.

The algorithm used to calculate peer assessed ratings works on the concept of each student being given a pie which they divide up among their peers according to their assessment of the value of their peers' contributions. Although we collect self assessments, the ratings that a student gives themself is not used when calculating the WebPA-x peer rating. To overcome the issues surrounding one student deciding to award a maximum rating of 5 and another student awarding a maximum rating of 3, ratings are moderated prior to calculation of the WebPA-x peer rating. Should a team member fail to submit a peer assessment then the algorithm treats it as if they rated everyone as performing equally.

Worked Example

Assessment

At the end of each term, each student shall submit Peer Assessment using the CECS implementation of WebPA-x.

A Student User Guide is provided. Additionally, wattle has a short video explaining the use of WebPA-x.

Peer assessment MUST be done indivdually and there be NO collusion between students.

Peer assessments are mandatory, but they do not attract any marks. Late or incorrect submission will be penalised by 2%. Students who do not submit the peer assessment will be awareded 50% of their assessed mark.

NOTE: When awarding students their individual marks the course examiners and team mentors will consider whether the mark calculated by WebPA-x is fair when considering performance during the review (if appropriate) and observations made by mentors during mentor meetings, as well as attendance at and participation in lectures.

WebPA-x

WebPA-X works with Firefox, Chrome or IE9. However, while it is likely that WebPA-X works with other browsers we cannot guarantee that it does, and we are aware of errors when using it with IE7 (the version of IE on the PCs at uni - however all PCs at uni also have Firefox).

Web-PaX has been set up on a College server, is subject to usual College security and is maintained and administered by College IT staff.

WebPA-X Assessment Reports

Students are encouraged to discuss their individual WebPA-X assessment reports with either their team mentor or the course co-ordinator.

Important Note

Peer assessment is confidential, that is, individual student ratings are available only to the course co-ordinator. Mentors will have access to Peer Assessment Reports which they may discuss with individual students in an effort to help those students improve their performance.